How Fast Is your Search Engine?

The search engine is at the heart of most information management systems as it is often the principal conduit by which users navigate to the information they are looking for.

As a leading provider of information management systems, Lexum has spent a lot of time developing a search engine platform that it applies to all its products. Our search engine provides professional features that are not typically found in the run of the mill web search engine. For example, Lexum’s search engine has the ability to limit results to those that contain search terms that occur within the same sentence or paragraphs.We also do fairly advanced faceted searches, return documents that cite other documents and a host of other advanced functions that allow professionals to easily find what they are seeking within a large document set.

SpeedmeterExperience shows, however, that one of the most important factor in user satisfaction is speed. This is easy to understand intuitively. We all have visited that slow website where it seemed that every interaction came with that unsatisfying “pause”. Research has shown, however, that even tenths of a second have very important impacts on user engagement. Tests by Greg Linden at Amazon revealed that a 100 milliseconds increase in web page load time decreased sales by 1%. And the effect is not linear. Tests at Google in 2007 showed that if Google displayed results within 900 milliseconds instead of 400 milliseconds, traffic and ad revenues dropped by 20%.

This is true not just of search results but of every web page. However, while regular web pages can be served quickly by observing engineering best practices, search is much harder to speed up because of the complexity involved in search algorithms, especially with large data sets. As a result, search is often the slowest component of most web sites. That should be apparent to any person who has searched for a hotel or a flight or has performed a Lexis-Nexis search.

Lexum, however, takes search performance very seriously. A lot of effort is spent on optimizing our search algorithms for speed and the results speak for themselves.

Consider, for example, the canlii.org web site which uses the Lexum search engine. The site currently hosts close to 20 million pages of searchable legislative text containing around 5 billion words and processes around 130,000 search requests per day. Yet, despite these staggering numbers, the average search request is received, processed and the results are sent to the user in only 161 milliseconds, faster than many sites serve mere Web pages.

Because search is, as we’ve said earlier, the principal conduit by which your users navigate your content, Lexum works very hard to ensure that their experience will always be fast and free of delay.

Photo credits.