THE SEARCH ENGINE UPDATE
6/4/97 - Number 6
About The Update
The Search Engine Update is a twice-monthly update of search engine news. It is available only to those people who have subscribed to "Search Engine Watch."
Please note that long URLs may break into two lines in some mail readers. Please cut and paste, should this occur.
"A Webmaster's Guide To Search Engines" becomes "Search Engine Watch"
I'm happy to announce the launch of "Search Engine Watch." This new site contains all of what you've previously found in "A Webmaster's Guide To Search Engines," along with some new sections and changes to make the site more accessible to search engine users, as well as webmasters.
I hated to tamper with what many people tell me has been a successful web site, but I felt it was time the guide got its own domain and a more recognizable name. I wish I could say typing the new address is shorter than the old http://calafia.com/webmasters/, but it's about the same length. However, you'll probably be able to remember the new address of http://searchenginewatch.com/
Within the site, you'll find these main sections:
A Webmaster's Guide To Search Engines
I've kept all the main pages related to how search engines index your web pages within this section, which keeps the name of the old site. Much of the material has been updated, and there are some new additions. These include pages on how search engines rank web pages and search engine software for your web site.
Search Engine Facts And Fun
This section has been built material within the old site that was related to search engines but not necessarily to how they index web pages. Some new pages have also been added. In particular, search engine users will find power searching tips, links to tutorials, as well as the rundown on major search engines and services. There is even a game to test your search engine knowledge.
Search Engine Status Report
This section is completely new. Here you will find the Search Engine EKG, which graphically shows search engine visits to two sites. You can use it as a guide for what to expect for your site. The Search Engine Response Times page shows the current time it takes to get questions answered from overworked support staffs. The Strategic Alliance chart also resides here. More performance-type information will be added, over time.
Search Engine Resources
This remains as it was with the old site, a collection of links to offsite information about search engines, in a variety of areas.
About The Site
Here you can find about the site's history, current ad rates, site awards and what's new.
That's a rundown on the changes to the public site. I'll be adding some material and making changes to the Subscriber-Only area next week. The area is located at:
It will remain at that address for the time being. I plan to post a rough piece on analyzing the keyword people use to reach your web site, plus I've got material I'll be updating to the various How Search Engines Work pages.
Please note that I'm not updating the downloadable version of the site until early next week. There are still some changes that need to be made to the site, and consequently, to the downloadable version. I will post it as soon as possible.
I want to express my thanks to all of you who have subscribed. Your support helped me decide to move forward with expanding the guide into Search Engine Watch. I also intend to keep adding to subscriber benefits, over time.
Pardon The Dust
There's still some construction at the site. I plan to promote its new presence beginning June 9, but I wanted Search Engine Report subscribers to be able and access the new information now. The Search Engine Status Report section will still be coming together over this week. Additional updating will also be made. Please feel free to tell me of any typos or glitches you may spot. They're out there -- and in this edition of the Search Engine Report, I'm afraid. Shifting from Word to FrontPage for HTML production hasn't made proofing any easier. But that's another story....
If any of you are responsible for links to the old site, please feel free to update them with the new address:
Within the next week, I will also begin automatic referrals of requests to the old site to the new address.
Search Engine News
On May 20, SEARCH.COM was relaunched, with several significant changes. No longer is it the all-in-one style search page of the past. Instead, it has been transformed into a branded version of the Infoseek search engine, though with some important twists.
Everything has been streamlined and made fast loading, and there is only one search box -- direct to Infoseek. Previously, there had been two search boxes, with the first (and mostly likely to be used) defaulting to Alta Vista. The second defaulted to Yahoo, but also allowed a selection of other search engines.
"The idea was to make it really fast," said spokesperson Karen Wood. "We decided to strip down SEARCH.COM and make it 100% searching." The new, fast loading look is also one that you can expect to see migrate toward other CNET sites, Wood said.
SEARCH.COM dumped Alta Vista in favor of Infoseek as default search engine primarily because Infoseek has an associated directory. SEARCH.COM wanted to provide a search-by-browsing alternative.
SEARCH.COM is not just Infoseek with a SEARCH.COM look, however. Infoseek technology is being used to do custom crawls of select sites on a daily basis. SEARCH.COM is the only search engine to be providing this type of hyperactive freshness, at the moment.
The closest competitors would be the news searching services, such as NewsTracker from Excite, and NewBot from HotWired (see the item below). They crawl important news sites on a daily basis (and are extremely useful, so try them).
SEARCH.COM, however, is crawling a list of editor-selected sites twice per day, for subjects ranging from Computers to Health, not just news. That means that searches in these categories will bring up some of the freshest results on the web. About 500 to 1,000 sites in each category are crawled.
SEARCH.COM is continuing to provide links to general and specialty search services all over the web. And direct searches can still be sent to Alta Vista, Excite, Lycos and Yahoo via links at the bottom of each results page.
HotWired Launches NewBot News Agent
(Repeat from last issue, but with new information)
From the people who brought you HotBot is NewBot, an offline news agent and web search utility. NewBot taps into over 100 news sites on the web, scanning them on a daily basis. News sites are crawled separately from HotBot, and information is placed into an index just for NewBot's use.
NewBot can also search web sites and newsgroups. For these searches, it taps into the HotBot database. However, the ranking criteria are different. The same search on NewBot vs. HotBot does not list the same sites, in the same order, although the differences are usually minor. Also, by default, NewBot list sites most recently added to HotBot first. They can be reordered in terms of relevancy.
At the moment, only an ActiveX version of NewBot is available, due to an exclusive deal with Microsoft. The ActiveX version only takes a few minutes to download. Netscape users can rejoice in July, when they get their own version. Also, in late June, you'll be able to do form-based searches as with other search engines from the NewBot site.
HotWired Sends in NewBot To Deliver Personalized News
NY Times, 5/97
Microsoft Find It Fast Page Changes
Early in May, it was the crucial Netscape Net Search page that saw search engines change position. Now Microsoft's Find It Fast page has done its own housecleaning. Gone are Alta Vista, HotBot, NetGuide Live and Yahoo. Remaining are AOL NetFind, Excite, Infoseek and Lycos.
The change is somewhat confusing, because the search page that loads into IE4's new search window -- which is sent from the Microsoft site -- still lists several of the dropped search engines and doesn't list AOL NetFind.
Meanwhile, Yahoo remains the default choice for autosearches performed by typing "go:" followed by a keyword in the address field of the browser. But perhaps the tightening of ties between Yahoo and Netscape last month caused Yahoo's drop from the Find It Fast page.
Microsoft Find It Fast page
Changes Coming For Excite's Add URL Feature
(Repeat from last issue, but with new information)
Excite is upgrading its Add URL feature. When completed, sometime by the end of June, the new service will add each page submitted to the index within seven days, as opposed to the usual three-week wait.
In addition, the new feature will notify webmasters when the page is actually added to the index. Should the spider fail to get the page after three attempts, the webmaster will also be notified and asked to resubmit.
FYI, I recently saw a site lose most of its pages out the Excite index for no apparent reason, and I got a report from another person who encountered the same thing. This has sometimes happened in the past with Excite. The search engine usually gathers back any pages that may have been dropped. So hang in there, if it happened to you.
Ticketmaster, Microsoft and Search Engines
By now, many of you have heard of the Ticketmaster - Microsoft dispute over inside linking. To recap briefly, Ticketmaster is suing Microsoft for linking to pages within its web site. It says that all links should come through its front door, the home page of the site. Otherwise, people miss out on seeing some of Ticketmaster's ads, and the company loses money.
Naturally, there's a lot of hype and concern that if Ticketmaster should somehow win, the web will come to a screeching halt, as linking becomes off-limits.
Funny, though. Ticketmaster doesn't seem to be suing any of the major search engines. When I checked in mid-May, Alta Vista has 240 links to the Ticketmaster site. HotBot has 2,438 links. Infoseek has a whopping 22,570 links. All of these are to inside pages, along with the home page, of course.
Ticketmaster could stop this inside linking by installing a robots.txt file to ban search engines from indexing its site. Does it? Nope. Feel free to check for yourself. Enter:
If Ticketmaster wanted to stop inside linking, this file would exist, and it would have explicit instructions stopping robots and agents from linking to the site this way.
Clearly, Ticketmaster is selective about its claim to only allow home page linking -- a claim that probably can't be enforced in the first place and which is weakened by it allowing search engines into the site. You can't have it both ways.
HotBot Offers Librarians Free Phone Tutorials
HotBot is providing library professionals with free, one-to-one, phone tutorials to introduce them to the search engine's capabilities. The tutorials last between 15 to 20 minutes. A free 6-month subscription to Wired also comes with the tutorial. Send email to coordinator Judy Chen, [email protected], or call +1 (415) 276-8464 between 10 am to 6 pm, PST. Please don't contact her unless you are a qualified library professional.
Excite, IDG feud over ad
IDG is planning a search service of some type for launch this summer, and it parodied an old Excite ad on a California billboard to promote the service. Excite was not pleased.
Discussions Made Searchable
You can search for mailing list, newsgroups and of course web pages. But what about all those web based forums and discussion groups out there? Forum One taps into over 65,000 web message systems.
Product Comparison Search Service
One of the few electronics items we shipped from California to London was our stereo system, so naturally the CD player died about a month after we hooked all the wires back up. I heard about Compare.Net recently, gave it a try, and came away impressed. It quickly gave me a list of CD players in my price range, with a side-by-side comparison. In fact, the only thing missing was the ability to order what I wanted.
NET MONEY | Excite Buys Market Share While Price Is Still Cheap
Web Week, 6/2/97
It costs a lot of money to buy your competitors, and the acquisition of WebCrawler by Excite is one of the reasons the search service is planning a second stock offering. Whit Andrews of Web Week provides an excellent rundown where Excite has been spending its money and how it has increased page views, as a result.
Lycos Closing In On Profits
C|Net has a short piece on Lycos reducing its operating losses in the last quarter, making analysts believe that profits are not far off.
Lycos numbers approach profitability
Web mall blasts Yahoo over listing
Yahoo listed an online Chicago mall, but only in the Business category, not in the city listing he wanted. He complained. News.Com listed. No one mentioned that his description was probably more important than his category choice in terms of coming up. I'm sure Yahoo can be heavy-handed at times, but this is hardly the best example.
Search Engine Articles
Wieviele Seiten hat das Web?
(How many pages are out there?)
Internet Intern, 5/97
An interesting article on search engine sizes. The summary below is German and English, but the article is German-only.
Angaben der Suchmaschinen-Betreiber |ber die Anzahl der von ihnen durchsuchten und gespeicherten HTML-Seiten schwanken. Manche Mirror-Sites geben sogar mehr an, als die Originale. Andererseits stagnieren die Angaben einiger Sites trotz des enormen Web-Wachstums schon seit Jahren. Wieviele Seiten gibt es wirklich und wie kommen die Zahlen der Suchmaschinen zustande?
Information provided by search engines concerning the number of web-pages that are spidered and stored by them look very different. Some mirrors even proclaim to have more of them then the original data-bases. On the other hand, some sites show the same numbers for years. How many pages are really out there and how can the different numbers provided by search engines be explained?
Making search time on the Internet pay off
Philadelphia Inquirer, 5/29/97
A librarian's perspective on searching the web. It's short, to the point, provides some good search strategy tips and has one of the best pieces of advice when it comes to searching. Is what you want already in a library? You remember books, don't you?
Getting to the Source: Is It Real or Spam, Ma'am?
A look at search engine spamming and optimizing pages for search engines. Be warned that it's overly simplified and incorrect in some places. For example, tables can reduce a web page's relevancy, but that doesn't mean a search engine only reads a little of the table and ignores the rest of the page, as stated.
Web Techniques Search Issue
Web Techniques has a variety of articles this month relating to searching, but off the usual beaten track. Here's a rundown:
Crawling towards Eternity
Web Techniques, 5/97
Internet Archive is making snapshots of the entire web to preserve it, text, graphics and all. This article takes a look at what's involved.
Searching With Isearch
Web Techniques, 5/97
A look at the free Isearch search engine, for use of indexing a web site.
Searching, Browsing, and Metasearching with Sensemaker
Web Techniques, 5/97
Up close with the technicalities behind SenseMaker, a metasearch service at Stanford University.
The Infoseek Redesign: A Study in Give-and-Take
Web Review, 5/97
In just three weeks, Infoseek radically changed its look. Web Review tells you how it happened.
Find it Faster
Internet World, 6/97
Not online at the moment
A variety of tips on searching the web. I haven't seen the article, just the reference to it.
The latest PC Meter results just came out late yesterday, though they are no longer online. I'm waiting to gather complete numbers. In the meantime, Excite reports WebCrawler and Excite are ranked number 2 and number 5 in the navigation category. Watch the the site for more details.
Open Text has returned to having a presence, albeit a small one, on the Netscape Net Search page. They were dropped during the redesign last month.
Search Engine Notes
Lycos has cleared up the trouble reported last issue with adding pages from outside the US.
Open Text reopened its submission service in April, but I've yet to see pages appear that were submitted back then. The current posted processing time is two weeks.
To unsubscribe from this list, send a message to [email protected], with the following in the subject field:
If you have problems, just send a message to [email protected]
This newsletter is Copyright (c) Danny Sullivan, Calafia Consulting. It should not be distributed. If you are not a subscriber and somehow are receiving a copy of the newsletter, learn how to subscribe at:
Twitter Canada MD Kirstine Stewart to Keynote Toronto
ClickZ Live Toronto (May 14-16) is a new event addressing the rapidly changing landscape that digital marketers face. The agenda focuses on customer engagement and attaining maximum ROI through online marketing efforts across paid, owned & earned media. Register now and save!