Clearly, I should have written about this when it came to my attention in late November -- but to be honest, I really didn't want to offend anyone. Now that the story has appeared in several other places, well -- what the heck.
It turns out that for the past three months, if you did a search on Google for a particularly insulting phrase -- think "dumb Oedipus" -- the top result was to the official George W. Bush campaign store. How on earth did this happen?
In late November -- yes, AFTER the election, I heard from HugeDisk, which is apparently a men's satirical magazine. They had written an article that, among other things, linked to the George W. Bush campaign store. The article wasn't about President Bush in particular -- Bush wasn't even mentioned, but the link was included as a response to a fake advice column.
Being a new site, HugeDisk was thrilled to discover Google had spidered their content. In looking to see how much of their site was listed, they tried various obscure terms that appeared on their pages -- including the now-famous insult -- which brought the campaign store to the top of Google's results, where it remained until last week.
"I'm thinking that the combination of our just-spidered link to George's site coupled with his site's obviously huge PageRank score was, by a fluke, enough to rank his page number one for that term. I want to know if it's likely that we're responsible for this fluke," HugeDisk's Michael McCole asked.
It turns out that McCole was right on target. By linking to the campaign store using that insult, the store was apparently made relevant for the words, given the way Google analyzes link text to understand what pages are about. In addition, the various other non-insulting links to the campaign store gave it some importance in Google's ranking mechanisms. So, while a lot of pages may be relevant for that particular slur, none of them had as much importance as the campaign store itself. That appears to have allowed the campaign store to rise the top of the list, no doubt a victory it rather not have had.
When I asked Google about this in December, they wouldn't say that the HugeDisk link was the only thing that made this happen but did admit that it had an impact.
"Google does use the text of links to a page in determining whether the page is a good result for a query, and so the link that you referenced would contribute to the ranking. The system is complicated enough that we can't easily tell if that's the only factor or if there are other contributing factors," said Craig Silverstein, Google's director of technology.
In Google's defense, Silverstein added:
"Generally, the process of analyzing links to a page to gather information about the page does help us return better results. For example, if you search on Google for 'bush for president merchandise,' you get the official campaign store as the number one result," Silverstein said (this is no longer the case).
"Of course, some link text is misleading, just as the text on some pages is misleading. We think we do a pretty good job of weighting all of the data so that we give mostly good results and few bad results, but nothing is foolproof. Examples like this are valuable to us because we can use them to tune our algorithms. This example is something we'll keep in mind as we continue to improve our ranking algorithms."
Google's not the only search engine to have experienced this type of oddity when using link analysis. After Infoseek released a new version of its ranking algorithm dubbed ESP in May 1998, some users were surprised to discover the Disney web site coming to the top of Infoseek's results in response to some porn terms. What happened?
Infoseek had started making more use of link analysis, and as it turns out, many porn sites ask users to click on a "Yes" link to verify they are over 18 before admitting them further into the site. As a joke, those who click on the "No" link might be sent to a site like Disney. As a result, the proximity of Disney's hyperlink on pages also containing porn terms helped make the Disney site relevant for those terms.
Unlike the case with Google, Infoseek quickly made a change to eliminate the problem -- perhaps because only months later, Disney made an investment in the company :) Unfortunately for Bush, the change didn't hit Google until last week. Google says this wasn't intentionally done.
"We never make changes to specific search results. As you know, this is totally against our policy of always serving objective, tamper-free search results," said Google spokesperson David Krane.
One possibility for the change -- other than free Bush merchandise being sent secretly to Google as a bribe -- is that the original page at HugeDisk no longer includes the link to the Bush campaign store, so a normal refresh of Google's database might have taken that into account. That goes along with Google's explanation:
"In this specific case, the search results changed because we posted major updates to our index shortly after the query was discovered by the mainstream press," said Krane.
Ironically, a search for the insult now brings up a page about the entire matter from the HugeDisk site itself. My guess is that the HugeDisk home page has some minor degree of PageRank importance, which it in turn has transmitted to the explanation page through its link.
Of course, Google makes use of many factors other than links to determine relevancy, and the HugeDisk page certainly has become very relevant for this particular term, given the publicity generated.
George Dubya a dumb mofo: official
The Register, Jan. 24, 2001
And how about that search at some other search engines?
Google Link is Bush League
Wired, Jan. 25, 2001
Don't like profanity, don't read the story. For the record, I did qualify my quote to say that even many Gore supporters probably would think this description of Bush was too harsh.
Ask Hester HugeDisk -- Installment One
The page that is probably responsible for Bush's dubious victory at Google. The link to the Bush site is removed. But, if you search for this at Google -- cache:http://www.hugedisk.com/story.asp?ar_id=9 -- you can still see the original page.
Lookin' For Liv In All The Wrong Places
The Search Engine Report, Nov. 3, 2000
This Google blip wouldn't make Liv Tyler very happy, though it's still hard to see which page for this particular search she would have liked best.
More Evil Than Dr. Evil?
The Search Engine Report, Nov. 1, 1999
The classic Google blip, that put Bill Gates on par with Satan. By the way, the answer to the riddle mentioned in the story is "nothing." See http://www.exodus-coach.co.uk/A_Riddle.htm.