Google has released an "uncut" 8-minute video of a search quality meeting to show the process of how Google improves its search engine. However, the video is hardly revealing – I wish they'd give us search marketers a little bit more meat.
The footage was captured last December 1, 2011 at Google's weekly “Quality Launch Review” meeting. These same meetings take place every Thursday to discuss possible algorithmic improvements that are being made and future improvements. Meetings are held in Mountain View and developers join via video conference from all around the world.
The focus of this meeting, led by Amit Singhal, was on misspellings and how to improve spelling corrections for long queries.
A few things Google wants you to notice (via the Inside Search Blog):
- Even relatively subtle changes get intense scrutiny by our search evaluation and ranking teams. The specific change discussed in this video improves spelling suggestions for searches with more than 10 words and it impacts only .1% of our traffic. Still, you can see the scrutiny and thoughtfulness that goes into approving this change.
- Every change has a dedicated search quality analyst assigned to study the impact. This analyst is not part of the engineering team building the change, but instead offers a separate opinion on whether the change is good for users.
- The search team relies heavily on the results of experimental data to make decisions. During the meeting, we rely on detailed analyst reports including the results of click evaluations and side-by-side experiments. These reports can sometimes be more than 25 pages long.
- Launch reports include specific examples to illustrate broader trends in the data. Rather than manually change one example, our engineers look for algorithmic ways to improve millions of queries.
- Search algorithm improvements often rely on and impact many different systems, so engineers with expertise in different areas all need to come together to make the best decision for the user, balancing all the tradeoffs involved (relevance, spam, latency, cost, language impact, etc.)
What do you think of this video? Did you like it? Does it help you to know more about Google search, or what other topics would you like to see discussed by Google in these types of videos?