Google Secret Lab, Prelude from Henk van Ess's new Search Bistro blog looks at how Google uses human reviewers to improve search quality. Ess has a screenshot and a Flash movie of how the system works, for Google's temporarily hired Q&A checkers. Very nice details so far that I've never seen posted anywhere before. However, this type of system isn't new.
WebCrawler used to have a feedback system like this back in the late 90s, and other major search engines have had human reviewers, as well. Gather enough data of what's deemed good and bad, and the hope is that you can create a better search algorithm to improve relevancy.
What almost certainly is not happening is that the human reviewers are actually moving sites up and down with their ratings. Ess is promising more details in future posts, so watch his blog. Want to discuss? Visit our forum thread, Secrets of http://eval.google.com revealed?
Upcoming Webinar: PPC Pause and Reflections for 2013
Thursday, December 12 - 2013 was a major turning point in search advertising. With Google's Enhanced Campaigns and Bing's innovative Smart Search capabilities in Windows 8.1, now is a great time to pause, reflect, and plan for the new year. Webinar attendance is free. Sign up today!