The Search Engine Update, May 5, 1997, Number 4

Search Engine Watch

May 5, 1997 - Number 4

================ About The Update ================

The Search Engine Update is a twice-monthly update of search engine news. It is available only to those people who have subscribed to "A Webmaster's Guide to Search Engines."

============ Site Changes ============

There's been some minor housekeeping throughout the site. The Search Engine Resources page has been broken up into individual topics, such as "Search Engine Technology" and "Search Engine Revenue." A new topic is "URL Checkers," which summarizes URL checking services such as PositionAgent. I've also added a page with Power Searching tips for the major search engines. Links to all of these can be found via the Resources page:

The Strategic Alliances page has also been updated to reflect the new Netscape line-up, as well as WebTV and a recent NetGuide review.

In the subscribers-only section, I've added another working document on implementing the robots.txt file to keep search engines out of your site. There's also a very rough, incomplete search engine timeline. I'm adding new events as they occur and working backwards as time allows.

Also, the detailed information about each search engine is now located only in the subscribers-only sections. These were the pages formerly found from the "How Search Engines Work" page. These pages are included in the downloadable version of the web site.

The subscribers-only section is located at

================== Search Engine News ==================

Netscape Net Search Changes

The Netscape Net Search page is most important place a search engine or web guide can be listed. This is the page that appears whenever someone pushes Netscape's Net Search button, and many do. In fact, I've often heard people saying things like "Do a Netscape search."

Of course, there's no such thing as a Netscape search. Searches done via this page are actually sent to one of several search engines. But this underscores the point that when people are looking for something, many go to this page. Any search engine listed on the page is going to get traffic. That's why they pay so much to be listed here.

On May 6, the Netscape Net Search page is significantly changing for the first time in over a year. There have been some smaller changes, but this is a major revamp of how things work. These changes are important to the web marketer, because the changes will influence which engines people use -- and thus which engines you need to be most concerned about.

Continuing as top-listed search services are Excite, Infoseek, Lycos and Yahoo. The are called "premier" providers. Specifically how much they are paying for this privilege hasn't been released. Last time, it was $5 million per year. Rumors are that its roughly the same amount for the next coming year, though payment is supposed to be tied to the actual amount of traffic each engine receives.

That traffic could significantly change. In the past, there were five top choices. Each time the page loaded, a different one of the top five was highlighted. Chances are, many people used whatever appeared, rather than clicking on one of the non-highlighted choices.

Now, there are only four top choices and a fifth "Customize" option. Those selecting this option cause a box to appear, where they can replace the Customize tab with a fifth search engine choice, either AOL NetFind, HotBot, LookSmart, or WebCrawler. These are called "marquee" providers. Further, a second option allows them to choose whether to rotate the top choices or set one to be the default.

What will people do? Who knows, yet. I tend to think that people change defaults as little as possible, which would mean12 that the page will continue to rotate between four top choices. But I can also see some people clicking on the Customize tab causing one of the marquee providers to become the default setting for every visit through confusion.

How confusing can two options be? Well, I love my mother, but you should see her operate a computer. Thinking like her, the first option asks her to pick from one of five marquee providers. So she picks one. Then the second option gives her a list of all the premier AND marquee providers, plus an option to rotate selections. It's all explained, but Mom - bless her heart - doesn't tend to read these things. No, she gets confused and sets the second option to be whatever she chose in the first option. Suddenly, one of the "second-tier" marquee providers becomes her default choice by accident.

So who wins and loses with these changes? It's status quo for the premier providers. They stay where they were. HotBot and LookSmart gain from the changes, since they rocket up to the top of the page as marquee providers, rather than being listed below the fold with only text links, as they were.

Search.Com and AOL NetFind move up, event though they aren't really search engines themselves. Search.Com taps into Alta Vista listings, by default, while AOL NetFind is a branded version of Excite.

WebCrawler gets a demotion. While it's listed as a marquee providers, previously it had been one of the top five picks. Finally, Open Text goes the way of Magellan, now no longer listed at all on the page.

While you're by the Netscape Net Search page, also be sure to drop in and see the new Netscape Guide page, which is being produced by Yahoo.

Netscape Net Search page

New Netscape Net Search Preview Page

Netscape Guide By Yahoo


Excite Launches Channels

Excite launched its new "channels" look on April 21, where information is organized around topics such as "sports" and "games." Lycos, you may recall, unveiled a similar reorganization last month.

To be honest, the whole channels metaphor had left me sort of yawning. After all, these sites already had reviews organized by subjects. But my jaw literally dropped when I looked at what Excite had done. I knew they had tours, bulletins boards, chat and more -- but seeing it all brought together under a particular topic was an eye opener. I couldn't help but think Excite is beyond being just a search service -- they're close to being an online service.

Of course, that's just what Excite and the other search services going this route would like. Instead of being just transit points -- Internet airports, if you will -- they can be destinations. It's easier to target demographics for your advertisers with a channel metaphor. Futhermore, by keeping visitors within the site with internal content such as chat, you keep them seeing your ads.

The New York Times article below touches on some of these issues, although it makes a stretch in trying to call Alta Vista's LiveTopics a unique twist on channels. LiveTopics may be neat, but they have nothing to do with the type of changes Excite and Lycos have recently made.

How does this affect the marketer? If more and more people turn to browsing, rather than plain searching, then being featured in these channels could become more crucial.

If you go by Excite, be sure to visit the My Excite channel. My favorite section is the TV listings at the bottom of the page, which tells me all the shows I'm missing, now that I live in Britain. It's nicely done and customizable -- assuming it doesn't crash your browser, as it did mine.


Systems for Dealing With Information Overload NY Times

Search Engines turn on to channels C|Net News.Com,4,9873,00.html


Push Is Coming - And So Are The Push Search Engines

Sick of hearing about push yet? Well, here's something else to think about. Should the integration of push into the Netscape and Microsoft browsers cause people to prefer pushed content, then they may be turn to services that index this content. For web developers, that opens a whole new can of worms. Exactly how will push content be indexed different than ordinary web pages? Are there traps to be avoided?

I'll be looking into this over the coming months. In the meantime, Excite is already poised to be a leader, having cut a deal with PointCast to index its new "Connections" channels.

Excite Inks Deal To Index PointCast Channels Web Week


Harder Life for Agents?

Word is that it suddenly got harder this month for offline agents to gather results from Excite, Lycos and Magellan. It could be due to general upgrades, but perhaps the services are trying to keep people from tapping into their results without actually visiting the web sites.


Search Engine Stocks

Last issue, I mentioned that Yahoo is the only major search service to show a profit. The Washington Post has an excellent look at why that Yahoo profit isn't quite what it seems to be.

You Can Find Lots on the Web, Except Maybe an Operating Profit Washington Post

=================== Search Engine Notes ===================

+ Pages have occasionally dropped out of Alta Vista in April, only to reappear again. If your site seems to have suddenly disappeared, check back in a day before panicking and see if it has reappeared + A Lynx user reports that Lycos is no-longer Lynx friendly, and apparently not concerned about it, to her dismay. I have no idea if other search engines are more friendly to the text-only browser. + Point is apparently committed to going through all its submissions but is so backlogged that it has only reached those from September 1996.

You may also wish to check out some articles that I worked on for NetGuide Magazine that are now online:

Supercharge Your Web Searches NetGuide, 5/97

Reviews and details of each of the major search engines

What Makes Search Engines Tick NetGuide, 5/97

An overview of the challenge of indexing the web and what goes on under the hood of the various search engines.

=============== Stolen Material ===============

I recently discovered that a company has appropriated material from my web site, which is being used as its own. The company is Net-Promote, an online promotions firm. All requests to remove the plagiarized material have been refused. If you support the guide, I would ask that you not use this company's services and pass the word to others to do the same.

========= End Notes =========

To unsubscribe from this list, send a message to, with the following in the subject field:

unsubscribe searchupdate

If you have problems, just send a message to

This newsletter is Copyright (c) Danny Sullivan, Calafia Consulting. It should not be distributed. If you are not a subscriber and somehow are receiving a copy of the newsletter, learn how to subscribe at: