The Search Engine Report, 01/31/97, Number 3

Search Engine Watch

1/31/97 - Number 3

The Search Engine Report is the email companion to "A Webmaster's Guide to Search Engines," It keeps you informed changes to the site and general search engine news useful to web developers. Feel free to pass on this newsletter to others.

A word from this month's sponsor:


Are Your Search Engine And Directory Listings Paying Off?

Chances are, the majority of your Website traffic comes from the popular directories and search engines. Try NetGambit's PositionAgent now and find out if your Website (using selected keywords) is prominently listed in the leading search engines and directories. Use our FREE TRIAL version at

Subscribe to the full version and PositionAgent will automatically monitor the top 20 pages of results for up to 5 selected URL's and keywords and report to you weekly. Then use our search engine listing resources, and you'll be getting more hits in no time! Launching February 3, 1997


There have been a number of new pages added to A Webmaster's Guide to Search Engines, include tutorials on using meta tags, helping search engines cope with frames and ways to check on your URL. A full run-down is listed after the news.

Search Engine News


WebCrawler bumps Magellan

First Excite bought Magellan, then it acquired WebCrawler. Now like a parent playing favorites, it has chosen between them and given WebCrawler the nod as favorite child.

On January 22, WebCrawler took over Magellan's top spot on the Netscape Search page. This is the most important place a search engine can be positioned, due all those people pushing Netscape Navigator's Net Search button. When last reported, companies were paying $5 million to have their homes on this Boardwalk of the Internet. The yearly contract should expire in April, so what for more changes then.

Meanwhile, Excite also maintains its position on the page, giving it two of the five top slots. What all this means for Magellan remains to be seen. The CEO stated "Magellan's editorial strengths and 'Green Light' capability continue to be leveraged across our flagship brands." Sounds like Magellan may be slowly absorbed by the two services over time.

Alta Vista combats spam

Alta Vista started the year off by becoming the latest engine to publicly announce penalties against those who attempt to spam. Lycos began the trend back in May 1996.

Alta Vista's move is making waves. In particular, the engine is vulnerable to spammers because of its near real-time indexing. A page submitted will be in the index in a day or two. This makes it easy for spammers to repeatedly test variations of a page in an attempt to move up the rankings. It also makes it easy for slightly different variations of the same page to be submitted in an attempt block others from the top ten.

Thus, Alta Vista has begun blocking numerous submissions of a particular URL or URLs from a similar domain that are done over a short period of time, such as in a day. Alta Vista is also watching for pages with numerous keywords or keywords unrelated to the pages. In a worst case, a webmaster will find all their pages removed. To spot these pages, many submissions are being reviewed by humans. You'll know if you are in trouble if Alta Vista's Add URL form refuses to process a submission. It will tell you that too many URLs have been submitted from that site. Accidents occur, of course. Pages can be mistaken as spam, and blocked root URLs might contain fine, ethical sites within them. Let Alta Vista know using its feedback form if you feel there's been a mistake.

PositionAgent checks search engine position

Tired of checking with each search engine to see how you rank under different keywords? NetGambit's PositionAgent sends your keyword queries out to major search engines, then scans up to 20 pages of results for your URL and reports your position. The site and service launches on February 3.

LookSmart: Reader's Digest summarizes the web

Backed by Reader Digest, LookSmart is a directory launched last October to catalog the web. Like many directories, it features editor reviews of web sites. Unlike many directories, Java is an integral part of the search interface. It takes a little getting used to and is probably either a "love it or hate it" situation. If you hate it, you can always choose the "simple" non-Java interface. The directory accepts submissions, for those of you looking for another place to publicize.

LookSmart also gets a bump toward the majors by now appearing on the Netscape search page, in position 11.

Yahoo changes listing display

A new year, a new look-at least for how Yahoo lists web sites. Alphabetical is out; relevancy is in. Categories that best match keywords entered are listed first, then sites that best match the keywords in the title or description come next. If you loved the old way, you can chose it as an option.

Maybe next year Yahoo will come up with a way to randomize the listings under a particular category. Those who browse aren't likely to spot the poor souls down in lower portions of the alphabet in some categories. Of course, I'm sensitive. My last name begins with "S," so I was never getting picked first for anything in school :)

Instant Indexing

Both Alta Vista and Infoseek have been offering impressive, near-real time indexing of a web page. Just keep in mind that only the pages submitted will receive this fast service. It can still take 2 to 4 week for the rest of your site to be added to the engines.

Meanwhile, Excite is considering increasing the refresh of its catalog from every 2 weeks to twice-daily, according to Newsbytes. The 1/22 story is available at the Newsbytes site to subscribers:

Infoseek adds free news services

Infoseek has launched Daily News, the first of several news services it plans to offer visitors. Daily News lets users search articles from six major news publishers, including the Los Angeles Times. Infoseek hopes this will this will distinguish their service from competitors such as Yahoo and Excite. Oddly, the Los Angeles Times has a stake in Excite, yet its news feeds are being used by Excite-competitor Infoseek. No one said the web makes sense.

Site Changes

The entire site has been updated expanded. Links to some of the new additions listed below are on the site map page,

* Site Map: Listing all the key pages, in order of a suggested itinerary for first time visitors.
* Strategic Alliances: New information has been posted.
* Search Engine Features: The chart has been expanded with new categories.
* How Search Engines Work: Each engine now has its own page for information not fully covered by the Search Engines Features page.
* Tips for Success: Now has links to expanded information about using meta tags and coping with frames.
* Search Engine Resources: New links have been added.
* Site Awards and Press: What people have been saying about the guide.
* Honor Roll: Who's supporting the guide through subscriptions.
* Meta Tag Tips: How to use them, and what they will and won't do for your page.
* Frames Tips: How to implement them and still get indexed.
* Checking Your URL: How to find out if you've been indexed.
* Measuring Your Popularity: Using search engines to see who has linked to your pages.

As you can see, a lot of new information has been added. That's left me with little time to do much proof reading. In other words, expect a few typos here and there.

Also, some of the additions are still works in progress. I will continue to update and expand as time allows.

Site Survey

I have a very brief survey on the site to help me plan future development. Please consider spending a minute or less and provide me with feedback.

Coming Soon: The Book?

I'm considering doing the guide in book form. If you are a publisher, or know a good one, I'm appreciate hearing from you.

Wanted: Your Site I'm working on an article for NetGuide magazine on search engines and am testing the engines in various ways to determine freshness. I'm checking for recently added sites. Please let me know if: you've opened a site in the past month under a new domain. I'd also like to hear from people who have updated their sites within the past month.

Before I get 2,000 URLs sent back at me, please only let me know if the site is under a root domain (no urls), if your pages are NOT dynamically generated, if you do NOT use frames and if your site has 50 or more pages. Also, I need to hear from people by Sunday evening, February 2. Don't worry about sending me URLs after that.

I'll let you know if I decide to use your site as part of the research.

Gains and Losses

PC Meter released its latest survey in mid-Janunary. Netscape also revised its NetSearch page. Finally, Yahoo changed the way it lists search engines at the bottom of its page. These have all caused various ups-and-down. As usual, full details can be found on the page.

Key: (+) overall gain, (˜) status quo, (-) overall loss.

HotBot (+): Moves up due to the Yahoo change and goes from 10 to 9 in the PC Meter rankings.
LookSmart (+): Makes first appearance on the Netscape NetSearch page.
WebCrawler (+): The addition to the Netscape Search page brings new hope that the service will do well despite the takeover by Excite.
Yahoo(˜): No major changes, although it did go from the third to second most-popular site in PC Meter's overall ratings. Infoseek (+): Moves up with both Yahoo and PC Meter, from 4 to 3.
Alta Vista (-): Drops from position 11 on the Netscape page to position 19.
Excite (˜): Moves up due to the Yahoo change but also drops in the latest PC Meter rankings, from 3 to 4.
Lycos (˜): No major changes.
Magellan (-): Dropped from Netscape NetSearch page, a crushing blow.

Thanks To

A number of people sent tips on checking your URL through Alta Vista and other engines. The new URL page was a result. I also had interesting feedback and tips from a number of people, including: * Warren Dodd * Neal Das * Tony Whittaker

Thanks also to Steve McKay, for his usual programming assistance. Steve offers a range of web production services, and you can visit his site at

Copyright Danny Sullivan, Calafia Consulting "A Webmaster's Guide to Search Engines"