Lucene search optimisation
A client, using a highly customised version of the Supermodel CMS had upgraded to the latest version of the CMS. The updated CMS changes the behaviour of Lucene search so resulting objects are lazily loaded via a CursoredIterable. Only objects that will be used are loaded, avoiding expensive database calls for data that would be discarded. After upgrading, Gin indicated that a single search collector method was responsible for > 60% of all runnable time, and deeper investigation showed the time was split mostly between reading the search index and populating the objects that were found:
Through the above Gin chart, the client was able to determine that the same Lucene document was being deserialised multiple times and that in each search, more objects than necessary were being loaded. The combinbation of these factors were leading to drastically increased response times. Using Gin to identify these problems allowed the client to make a small code change in the CMS resulting in a subtantial performance enhancement:
The above graph shows the max response time immediately following the rollout of the updated CMS, and the effect of various mitigating actions on September 13, and the deeper code change rollout on Sepember 25.