Monday, February 23, 2009
A single search on Google can involve 1000 machines.
Ever wonder how Google is able to bring you search results so quick. Well for years now Google has kept many of its secrets under wraps locked up in their Google vault. Recently though Jeff Dean gave a keynote talk at ESDM 2009.
According to Dean, a typical Google search consumes the processing power of 1000 machines. He also stated that in that 10 years the number of Google searches has gone up by x1000. Dean also claimed that Google has brought their search query time down to 200ms, 10 years ago they were running around 1000ms. The largest growth though can be found in the update latency of Google’s website Crawlers. They have dropped by a staggering x10000, which has reduced the crawler update speed from several months to now just a couple of minutes.
Dean went on to say that the improvements are a result of switching the holding indexes to now being completely in memory. The result is that it takes thousands of machines to process a search query, but it has made near instantaneous searches possible.