Aaron Wall over at SEOBook.com has an excellent write up on the recent indexing phenomenon at Google. Google has been indexing fewer and fewer pages and webmasters are trying to figure out how to get more of their pages indexed and found by searchers. Aaron posted a blog entry he named The Google Crawling Sandbox.
The title of the post comes from the concept of Google slowing down its crawl process, like it would when ranking new sites (the whole Google Sandbox concept). Aaron explores the rumors of these theories but when push comes to shove, he nails it down to unique content. The more “legitimate useful content” the more likely people will want to read it, the more likely Google will crawl it, the more likely Google will index and then rank it.
Postscript From Danny: One of those commenting on Aaron’s blog highlight a key quote from him, one that well deserves that attention:
The less your site needs to rely on Google the more Google will be willing to rely on your site.
This goes right back to the “don’t do things only for search engines” or don’t be totally dependent on search engines messages that many have said over the past years and is worth repeating. No, you don’t want to ignore them — but if you’ve done everything to revolve around them, that’s a good sign you aren’t running a “natural” site of the site search engines are trying to reward.