Tuesday, March 28, 2017

ItemWebApi: maxJsonLength Property Exceeded

Sitecore's Item Web API  is extremely useful is so many ways - however as your database grows, you may encounter a limit on the size of the response Sitecore is able to return.

After installing the Sitecore Item Web API  feature to an existing Sitecore v6.6 site, I encountered an error when querying a huge part of the content tree using the following request:
/-/item/v1/?sc_database=master&scope=s&query=/sitecore/content//*

Due to the sheer amount of data the request attempted to retrieve, I received the following response:

"Error during serialization or deserialization using the JSON JavaScriptSerializer.
The length of the string exceeds the value set on the maxJsonLength property."


I initially figured the solution would be as simple as setting the MaxJsonLength property in the web.config, but soon realized that this setting had no real positive effect.

DISCOVERY

At this point, I took it in upon myself to decompile/reflect the Sitecore.ItemWebApi.dll in hopes of overriding the serialization method in order to set the MaxJsonLength property when initializing the JavaScriptSerializer.  

After some digging, I discovered two relevant methods I needed to override:

JsonSerializer

SerializeResponse

The SerializeResponse method serializes the API's response using JsonSerializer.Serialize to return the response in JSON format.

The Sitecore.ItemWebApi.config patch file defines the SerializeResponse pipeline:

SOLUTION

I started a new Website Project in Visual Studio, then
  1. Added the /App_Config/Include/zzz/ folders
  2. Copied over relevant binary files into a /References folder
    • Sitecore.ItemWebApi.dll
    • Sitecore.Kernel.dll (for Sitecore 6.6 - use your site's specific version*)
    • Newtonsoft.Json.dll (v3.5 for Sitecore 6.6  - use your site's specific version*)
  3. And added an Override folder to house the overriding code:



To override the existing Sitecore.ItemWebApi.Pipelines.Request.SerializeResponse definition, I created a patch file called zWebItemApiSerializedResponse.config and added it to the /App_Config/Include/zzz/ folder:

 <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">  
      <sitecore>  
          <pipelines>  
              <itemWebApiRequest>  
                <processor patch:instead="*[@type='Sitecore.ItemWebApi.Pipelines.Request.SerializeResponse, Sitecore.ItemWebApi']" type="Sitecore.SharedSource.ItemWebApiCustom.Override.SerializeResponseOverride, Sitecore.SharedSource.ItemWebApiCustom" />   
              </itemWebApiRequest>  
           </pipelines>  
      </sitecore>  
 </configuration>  

In the Override folder, I created two class files based on the decompiled Sitecore.ItemWebApi.dll  code:

SerializerOverride - inherits ISerializer
 using System;  
 using System.Web.Script.Serialization;  
 using Newtonsoft.Json;  
 using Sitecore.Diagnostics;  
 using Sitecore.ItemWebApi.Serialization;  
 using Formatting = System.Xml.Formatting;  
 namespace Sitecore.SharedSource.ItemWebApiCustom.Override  
 {  
   public class SerializerOverride : ISerializer  
   {  
     public string Serialize(object value)  
     {  
       Assert.ArgumentNotNull(value, "value");  
       var serializer = new JavaScriptSerializer {MaxJsonLength = Int32.MaxValue};  
       dynamic parsedJson = JsonConvert.DeserializeObject(serializer.Serialize(value));  
       return JsonConvert.SerializeObject(parsedJson, (Newtonsoft.Json.Formatting) Formatting.Indented);  
     }  
     public string SerializedDataMediaType => "application/json";  
   }  
 }  


SerializeResponseOverride - inherits SerializeResponse
 using Sitecore.Diagnostics;  
 using Sitecore.ItemWebApi.Pipelines.Request;  
 namespace Sitecore.SharedSource.ItemWebApiCustom.Override  
 {  
   public class SerializeResponseOverride : SerializeResponse  
   {  
     public override void Process(RequestArgs arguments)  
     {  
       Assert.ArgumentNotNull(arguments, "arguments");  
       SerializerOverride serializer = new SerializerOverride();  
       arguments.ResponseText = serializer.Serialize(arguments.Result);  
     }  
   }  
 }  


Once this project successfully built, I copied over the new App_Config file, and the projects Sitecore.SharedSource.ItemWebApiCustom.dll file to my solution's /bin folder (add the .pdb file to the /bin folder if you want to debug!)

RESULT

You can confirm your patch file is properly overriding the original SerializeResponse pipeline using /sitecore/admin/showconfig.aspx and searching for SerializeResponse on the page.

If everything went well, you can refresh the API call to see that the JSON response size restriction has been lifted!

Let me know in the comments if you have any suggestions or issues with this approach.
Also feel free to drop a line if this post has helped you out!



Thursday, March 9, 2017

Sitecore Performance & Troubleshooting Tools

As a consultant within a Managed Services practice, I am often tasked to dig into ancient developer code in search of bottlenecks. Discovering performance issues on Sitecore builds can be tricky because each site you may work on follow different development patterns.

I found that the most common issues only surface after accumulating data, where some components may not have been tested using more than a few items of data.  The fix often ends up being some modification to specific Sitecore queries, or perhaps changing the way data is retrieved from Sitecore (switching to pull from a Lucene index).  Whatever it is, finding the bottleneck is just the first step.

Luckily for us, there are plenty of tools available that make troubleshooting and pin-pointing these issues a thousand times easier.  Regardless of Sitecore version you're working with, here are four performance / troubleshooting tools I often fire up from my arsenal to get to the bottom of slow load times.

SITECORE LOG ANALYZER

This is a given!  If something odd is going on in your Sitecore website, one of the first places to look for clues is the Sitecore logs.  This handy tool developed by Sitecore loads the entire Sitecore log folder and allows you to filter by date, string, message type (ERROR, INFO, DEBUG, etc).    This is often much easier than sifting through flat .txt files.



https://marketplace.sitecore.net/en/Modules/Sitecore_Log_Analyzer.aspx

SITECORE DEBUG TOOL

Built right into Sitecore - I only started utilizing this tool a few years into my career working with Sitecore, but it's been a life saver in a few instances.  My guess is that devs often forget that this exists.  Some features include full trace information for each rendering and profiling - which comes in handy when attempting to identify bottlenecks.  If you're trying to identify which rendering on a page are being pulled from the Sitecore cache, this is one of the best way to find out.

To enable the debugger, the following prerequisites must be satisfied:

  1. Ensure the "allowDebug" and "enableDebugger" attributes for the site are set to true in the site node of the Sites.config.
  2. Ensure you're logged into Sitecore with a user member utilizing the 'sitecore\Sitecore Client Developing' role.



Additional information on how to use the Sitecore debugger can still be found here:
The Sitecore Debugger

APACHE JMETER

Apache JMeter is an open-source Java application which (among other handy features) is great for running load/stress tests and providing the calculated results for analysis.  When configured properly can send multiple HTTP requests to a designated URL location and provides an average load time throughout the increased traffic.  For me, this has been a great way to obtain visualizations of page load time behaviors before and after making caching or other performance enhancement changes.



For more: https://www.digitalocean.com/community/tutorials/how-to-use-apache-jmeter-to-perform-load-testing-on-a-web-server 

 http://jmeter.apache.org/

TELERIK FIDDLER

I'm a big fan of Telerik's suite of developer tools.    Fiddler is one of those tools I always make sure to have installed on my dev box.  If you don't already know, Fiddler logs all HTTP/S traffic between your computer and the web.  This comes in handy when gathering statistics of internal or third-party calls, viewing client headers and/or cookies, viewing results of an API call and more.



http://www.telerik.com/fiddler

TELERIK JUSTTRACE

While JustTrace isn't free, it's definitely one the of easiest .NET profiling tools I've worked with.   Its intuitive tracing mechanism which allows us to identify which .NET components take the most CPU and memory resources which ultimately affect page load times.  In most cases the snapshots provide exact methods to look into and tweak until a more reasonable load time is obtained.




If you haven't tried these tools, I'd recommend playing around with them at some point.  They may become an integral part of your own troubleshooting whether during development or post-production support.  

Have you used any of these in the past?  What are your favorite performance troubleshooting tools?

Let me know in the comments!