File this under interesting but perhaps nothing, big on the perhaps nothing part. I was checking on the infamous miserable failure search today that has long brought up the official George W. Bush bio in top position. I was surprised to see on Google that while the page is still listed there, it's a "link-only" or what Google calls a "partially-indexed" listing that doesn't show a title or description.
What's that? That's when a page hasn't actually been spidered by Google but ranks solely based on the links pointing at it. Google has long spidered this page, so why did it suddenly change to link-only since I last looked about two weeks ago?
The first thought is that the White House web site has banned spidering of this page, especially given its past history of trying to keep some content out of search engines (see my The US White House & Blocking Search Engines article for more on this). But checking the robots.txt file at the site, I don't see any such banning. Nor do I see any on-page use of the meta robots tag to keep the page out. In addition, you can see that Yahoo, MSN and Ask Jeeves are still listing it, evidenced by their cached copies (Yahoo, MSN, Ask).
Chances are, it's just a glitch with Google. For whatever reason, it may have dropped the page, as can happen from time-to-time. There's a slight chance that the White House could be specifically spotting the Google spider and specifically giving it on-page banning instructions, however. I haven't had a chance to investigate this more and can't stress enough that in all likelihood, this is some Google spidering glitch. Regardless, the fact the page isn't indexed has had no impact on the ranking.
Introducing... ClickZ Live!
SES Conference & Expo has merged with ClickZ to bring you ClickZ Live! The new global conference series takes on the identity of the industry's premier digital marketing publication, ClickZ.com, and kicks off March 31-April 3 in New York City. Join the industry's leading tech-advertisers in the advertising capital of the world! Find out more ››
*Super Saver Rates expire Jan 24.