Changes Afoot at Yahoo & MSN

Search engine marketers have long been awaiting changes at Yahoo and MSN. Finally, signs of transition are beginning to appear.

Yahoo has begun crawling the web with a new indexing robot — foreshadowing some interesting potential developments as it gets ready to drop Google as its primary search partner.

The new robot is called Yahoo Slurp, maintaining consistency with the Inktomi Slurp crawler, purchased by Yahoo more than a year ago. Although the crawler has many similarities to the Inktomi crawler, there are some interesting new features described on the help page for the robot.

One is that the crawler is collecting documents from the web to create a “searchable index for search services using the Yahoo search engine. For months, speculation on various search engine forums has focused on Yahoo’s presumed switch from the Google index to the Inktomi index.

But other features described on the help page suggest something more is afoot.

For example, although the crawler is functional (and if you have a web site, you can see its footprints in your logs), Yahoo says that crawled pages may not show up in search results. Why? According to the Yahoo Search FAQ, “The documents will be indexed and included into the search database in the near future.”

Yahoo is also now creating a cache of web pages similar to Google’s.

What else is changing at Yahoo? The company won’t comment yet.

Meanwhile, over at MSN, LookSmart listings have been dropped, and Inktomi results have been promoted. MSN has also rolled out a new search beta. More details are available to Search Engine Watch members at the link below. We’re tracking the new developments at Yahoo and MSN closely and will have full details in an upcoming SearchDay as soon as they’re available.