THE SEARCH ENGINE UPDATE
December 3, 1998 - Number 42
About The Update
The Search Engine Update is a twice-monthly update of search engine news. It is available only to those people who have subscribed to Search Engine Watch, http://searchenginewatch.com/.
Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.
In This Issue
+ General Notes
+ AOL Buys Netscape
+ Searchopolis Offers Kid-Friendly Search
+ Infoseek Releases New Intranet Search Software
+ Shopping Search Round-Up
+ Submission Changes At MSN Search
+ Latest Ratings Show Changing Audience Shares
+ Opinions On Submitting Sought
+ Search Engine Notes
+ Search Engine Articles
+ Subscribing/Unsubscribing Info
I've been busy doing a lot of updating throughout the site. Let me cover changes in the public area first:
I've adding some new resources to the various search engine listings in the Facts area of the web site, and the Search Engine Alliances page has also been updated.
Sadly, I'm retiring the Search Engine Profits page. I'll leave the page up for the next few days, but with a notice that it will soon be removed. It's actually one of my favorite pages, but there hasn't been enough time to maintain it properly. I'll place it in the Subscribers Only area for future reference.
The search engine ratings pages have all been updated, and there were so many significant changes that I've decided to cover them in a separate article, below.
The MetaCrawler Top Search Terms page has been updated with October 1998 information, and November 1998 information will be posted shortly.
You can find links to all the pages above on the site's What's New page, listed below.
In the Subscribers Only Area, I've updated all pages that deal with how the individual search engines work, and I've added new pages that cover Ask Jeeves, GoTo, LookSmart, RealNames, Snap, MSN Search and Inktomi. Have fun reading! If you have been keeping up with recent newsletters, much of the material will already be familiar to you. If you are pressed for time, the Excite, Infoseek, WebCrawler and Yahoo pages have had only light revisions.
I will be posting a new version of the offline edition by next Monday. I'm sorry I couldn’t have it ready in time to announce here, but all the updating hasn't left enough time to make a new copy.
Subscribers Only Area
Search Engine News
AOL bought Netscape -- is there anyone who still hasn't heard? I'm stepping back from this one, because there is plenty to read in other places. When the dust settles, I'll jump back in and report on the actual changes that take place as they relate to search and navigation.
I will make a comment regarding the consolidation issue that's everywhere. At the end of 1996, top executives at the major search engines -- and many analysts -- all agreed that by the end of 1997, we'd see a consolidation in the industry. The mentality was very much that you can only have Coke or Pepsi -- not a variety of choices.
As 1998 closes, there are more search and navigation sites than ever before. Yes, newcomers such as GoTo and Snap lack the traffic of the major players like Yahoo, Excite, Infoseek and Lycos. Those companies benefited from having been first movers on the web, and for having a core product that everyone needed: search.
I think search is going to remain a product in demand, and that's why these new sites will survive and even more may appear. Search services are like shoes. What fits one person perfectly may not be good for someone else. People also own different pairs. There's room for many pairs on the shelf -- or many search services on the web.
Of course, I don't expect these new sites to suddenly have the traffic the major search players command, traffic that has turned them into "portals" that route millions of people to destinations on the web. But I also think it's foolish to think we may have a few major sites that control what we visit, in the way we have tended to have major television networks that control what we see.
The web is not television. It is a completely new medium, and despite its fast and incredible growth, things are far from set in stone. Big money in the form of AOL, Microsoft and others will have an impact in making big sites that stand above others and which influence where we go. But it's cheap for anyone to publish on the web, a key difference from other media. And good content attracts: just ask the people at NASA.
I think at the end of 1999, people will still be watching for the consolidation in portal sites in the same way they have been with search sites. I doubt it will come. I could be wrong, but by then the Millenium Bug will have plunged the world into such turmoil that no one will remember.
Below are news roundups on the AOL-Netscape story and two particularly interesting articles:
Wired News: AOL Eats Netscape
News.com: The making of the Netscape buyout
ZDNet News Special Report: AOL To Merge With Netscape
The birth of an Internet network?
Salon, Dec. 1, 1998
Scott Rosenberg writes convincingly that the merger is not the death of the Internet, of independent web sites, or the creation of a mega site that will rule all. I couldn't agree more.
Netcenter partners react to buyout
News.com, Nov. 24, 1998
A good round-up of what major search sites think of the deal. In short: we're not scared.
Last month I wrote briefly that Searchopolis, a new kid safe search engine, had opened in beta. This month, I have more details about how the filtered search service works.
Searchopolis is backed by N2H2, a Seattle-based firm notable for the Internet filtering service that it provides to schools and businesses. That service, called Bess, works with proxy servers to prevent access to sites that have been identified as objectionable or off-limits.
Human editors identify sites for the block list, and these are classified into 30 different categories of objectionable content. These includes sites about pornography and hate talk, or sites that may be banned by employers because of productivity issues, such as sports and job hunting sites. Categorization gives filtering customers a wide degree of choice over what to block, and even what time of day to block it.
N2H2's clients are mostly schools, and despite filtering, they were finding that a kid safe search engine was needed. That's because in response to innocent searches, objectionable sites were appearing in the results at the major search services -- complete with graphic descriptions.
N2H2's filtering package kept children from actually reaching these sites, but it did nothing to prevent objectionable material from appearing in search descriptions. That lead the company to consider providing its own search solution.
The company teamed up with Inktomi, which provides the power behind Searchopolis. The brains behind what to filter comes from N2H2's block list, which covers about 8 million web pages. FYI, the company estimates that about 8 percent of the web is pornographic in nature or would be considered objectionable by its standards.
N2H2 provides Inktomi with an updated block list twice a week. It also continually tests Searchopolis, to see if any material may have slipped through.
As a further method of protection, the public version of Searchopolis uses a slight date restriction, so that pages only a day or two old won't immediately appear. Editors use a version without this restriction, which helps them spot problems before the public would even see them. I doubt the restriction will impact the quality of most searches, and it is a fair trade-off to help keep the results clean.
Searchopolis isn't the only filtered, crawler-based search service. AltaVista and Lycos both offer filtering solutions. These put more emphasis on automatically tagging pages as objectionable when they are spidered. In contrast, N2H2 believes its human approach will provide more accurate screening.
"Typically, keyword filters are prone to block out sites that contain that word," said Jim O'Halloran, N2H2's marketing director. "The upshot with the human review is that we overcome that restriction, because our reviewers immediately see what is pornography."
In other words, keyword filters can inadvertently screen out valuable material, such as pages that mention "breast" because they discuss breast cancer treatments. However, AltaVista has said its filtering software is smart enough to let "good" pages through. And both AltaVista and Lycos also make some use of human-compiled block lists.
The Searchopolis results certainly looked very clean, when I ran some test queries. Those looking for kid safe search will welcome it as a new option to choose from.
Searchopolis also has a partnership with LookSmart, which provides a branded-version of its directory for the service. There is nothing expressly kid-oriented about this, in comparison to kid-directories such as Yahooligans and Disney's DIG. But it does provide users the ability to browse sites without encountering vulgar or sexually explicit descriptions.
Kids Search Engines
You'll find services meant to be safe and friendly for children here, including Yahooligans and DIG. Information about AltaVista and Lycos filtering options is also available.
Infoseek has released a new version of its intranet and web site search software, Ultraseek Server 3. The company also announced a new add-on for the package called CCE, for Content Classification Engine, which organizes information automatically into Yahoo-like categories of information.
With CCE, users can browse topics and see documents available in a list format. Also, when users perform keyword searches, they are shown related topics in the same way the Infoseek web-wide search engine currently presents matches from its directory within its search results.
CCE can create a basic directory structure quickly with two techniques. It can import and analyze information from a site map and classify documents accordingly. Also, by examining the directory structure of a server, it can make educated guesses about how to categorize documents.
For example, all documents relating to employee benefits might be kept in a benefits directory on a server, while employee contact details might be in another directory. If this type of structure exists, then CCE will take advantage of it to build a category tree.
Administrators can also add new topics and change categories, as desired.
As for the core Ultraseek Server software, it offers new support for XML, SSL encryption, date range searching and extended lexical support for 10 different languages. The package is currently used at over 500 sites. Clients include companies such as Ford, Boeing, Hewlett-Packard and CNN.
Pricing for Ultraseek Server begins at $995 for 1,000 document sites, with price breaks as document numbers increase. CCE pricing starts $4,995 for sites with up to 50,000 documents. The software can be downloaded online via the URL below.
'Tis the season but there just wasn't time to do a proper review of shopping search sites in time for the holidays. Instead, I offer up some important related announcements that you may find interesting.
Both Yahoo and Excite rolled out one stop shopping services, just in time for the season. The services aim to make shopping easier by letting users register once, then purchase from multiple merchants without having to reenter payment details.
Keep an eye on these services. There have been many attempts at creating virtual wallets for use on the web, but none of them have become widely accepted. But if Yahoo or Excite end up with a large base of registered shoppers, they could offer their databases to any merchants to use as a standard. That means you could go to an online shopping site, indicate that you had a Yahoo or Excite profile, and then have the site recover this information for you to speed your checkout.
Yahoo Shopping features products from over 2,700 merchants. You can search or browse offerings, add products to a virtual cart, then enter payment and shipping details once regardless of the number of merchants involved.
The service then remembers this the information, for use if you shop again. Only your credit card number is not stored, to ensure that no one can hack into your account and go on a spending spree. There are also options to specific different shipping addresses per order, in case you are sending gifts.
The Yahoo Shopping site provides another reason for online merchants to consider opening a site through Yahoo Store. All Yahoo Store merchants are automatically included in the Yahoo Shopping site. They don't get preference in search results, but just being included without any extra effort is a great benefit and wonderful exposure.
The inclusion of all these stores is also a big reason why Yahoo was able to roll out a shopping service covering a huge number of merchants, in comparison to the 20 or so listed within Excite's competing Holiday Shoppe.
As with Yahoo, users need only register once at the Excite Holiday Shoppe to purchase products from any of the participating merchants. Information is then stored in Excite's Express Order system, for later use. This system only works with the Holiday Shoppe at the moment, but it will be extended to all Excite merchant partners, in the future.
Establishing an Express Order profile is straightforward. You simply select an item in the Holiday Shoppe, and then a single profile page appears. You provide billing, shipping and payment details, which are then passed on to the merchant and saved for use later.
A nice touch is the ability to store everything but your credit card number. That will reassure the suspicious that no one can hack into their account and go on a spending spree. Also, if you have an existing Excite personal account, your Express Order information will be linked to this. It also makes establishing an Express Order account just a little faster.
Also at Excite, Consumer Guide reviews are now available within the Shopping Channel. You browse major product types to find reviews of different models from the consumer review publication.
Meanwhile Buyer's Index, a search engine of shopping sites, has added information from two online consumer protection sites to give shoppers more information about online merchants.
Buyer's Index lists information from 10,000 companies, and shoppers can look for products there through keyword searches. Now the service also displays BizRate and Public Eye symbols within its merchant listings. Buyers simply click on the symbol to see that merchant's reliability report at the BizRate or The Public Eye web sites.
Finally, Catalog City is a relatively new site that allows you to search for mail order catalogs or online merchants. It also offers one stop registration and a shopping cart system, like those described at Yahoo and Excite, above.
Happy shopping, everyone -- and happy holidays!
Excite Holiday Shoppe
Excite Consumer Guide Information
BizRate polls customers about satisfaction, after they have purchased from its 60 gold certified merchants. The company also performs reviews of its 400 silver certified merchants.
The Public Eye
The Public Eye collects customer satisfaction reports from shoppers that purchase from over 3,500 Public Eye Platinum certified merchants. Public Eye also uses undercover shoppers to test the reliability of some 500 Gold certified merchants.
Learn about Yahoo's electronic storefronts for online merchants here. The second URL explains how Yahoo Store relates to Yahoo Shopping.
MSN Search has begun accepting direct submissions to its index. Those with access to the paid Submit It submission service can access this feature now, and by the middle of the month, anyone should be able to submit using either the free version of Submit It or a form at the MSN Search page.
Until now, HotBot has been the most significant Inktomi-powered search site with a submission page. That means it has been serving as the primary way web developers could notify Inktomi's spidering system of new sites. This also means that HotBot has been serving a de facto submission service for all Inktomi-powered sites.
The launch of an MSN Search submission page changes this. It provides MSN Search with a degree of control over exactly what appears in the index that Inktomi builds for it.
"We wanted to have some predictable results," said Bill Bliss, MSN Search's general manager. "You could go over to HotBot and submit there, but we were not guaranteeing that was going to work."
An MSN Search submission page also means that MSN Search will have its own submission policies. These are still being developed, but so far, Bliss says that the service is inclined to limit sites to submitting only one URL -- whatever the site's home page is. After that, the service will leave it to Inktomi to look for additional pages.
In fact, this is exactly what the current "preview" submission feature at Submit It is doing. It submits one URL per site to Inktomi on MSN's behalf, rather than doing a "deep" submission of multiple pages. It will be up to the Inktomi spider to find other pages.
Inktomi has been a pretty good crawler, so you can expect it will gather many pages from your web site. But exactly how deep it will go, and how quickly pages will appear, are still issues that are being decided. For the moment, MSN Search simply says that it wants the submission process to be "robust" and better than at other search engines, in terms of turnaround time.
Obviously, much remains to be worked out. Expect there to be changes as MSN Search deals with submission issues for the first time.
Meanwhile, there has been another change related to Microsoft's recent purchase of LinkExchange. Sites listed in the LinkExchange Surf Point directory are being promoted within the MSN portal and MSN Search results page.
Go to MSN Search and look for something. On the left-hand side of the results page, you'll see a prompt to search at LinkExchange. Clicking on the link brings up matching sites from the services Surf Point Directory. You'll also find Surf Point promoted throughout the MSN portal site.
To be in Surf Point, you have to become a Link Exchange member. This is free to do. Once you become a member, you can submit your site to an appropriate place within Surf Point. I highly recommend that you enroll right now, because I expect you'll see Surf Point become an even more important resource within MSN Search. I'm planning a closer look at this directory in the future.
Besides, LinkExchange is a wonderful resource for site owners. I've been very impressed with the collection of free tools and support the service offers members under one roof. These include an easy way to set up a mailing list, a traffic counter, a site code checker and lots of good marketing resources such as the LinkExchange Digest.
There's also the banner exchange component, of course -- I've never personally found this to bring much traffic, but it remains another free option to take advantage of. Plus, banners may become more valueable now that they will be displayed within the MSN network of web sites.
See the second URL for information specifically about submitting to MSN Search via Submit It. I suspect using the preview won't make that much of a speed difference in getting listed versus waiting for the free submit page, which should appear soon. But, if you've been thinking of using the service already, this is certainly a good incentive.
I've posted the latest search engine rating results from Media Metrix, RelevantKnowledge and NetRatings. Although Media Metrix and RelevantKnowledge have merged, separate data is still provided from both, at the moment.
There are some notable changes, which I'll summarize briefly below:
Yahoo has seen a drop each month since May in its share estimate from Media Metrix. The same is true at RelevantKnowledge, except for one month. NetRatings also shows an overall decline. It's not a plunge -- Yahoo remains comfortably above its rivals -- but it is noticeable and has gone on long enough to feel like a trend.
Excite is also showing a decline according to two of the services. The drop is most noticeable at Media Metrix.
Lycos has overtaken Infoseek's number three spot, or number four if you count Netscape as a search site, according to all the ratings services. RelevantKnowledge and NetRatings still show it as a neck-and-neck race, while Media Metrix shows Lycos comfortably ahead.
Both HotBot and Snap had dramatic rises at Media Metrix in October, about double their share from June 1998, and putting them both in the 8 percent range. The services have been running television ads recently, which seem to have paid off. But at NetRatings, the increases are not so dramatic. Perhaps those surveyed by Media Metrix watch more TV.
Finally, at RelevantKnowledge, GoTo posted a significant jump, up to a nearly eight percent overall share.
Search Engine Status Reports
You'll find links to pages that provide more information about the latest ratings at the top of this page, in the "Popularity" section.
Haven't seen the HotBot commercials? Tune in online.
Submitting to search engines can be confusing for many people. There are questions about meta tags: how to format them, how many times can terms be repeated, and should you use them at all. People don't know how often they should submit, or if they should submit all their pages. Rules can be conflicting about what's considered spam and what's not.
I'm planning a poll to see what exactly is most problematic for people, which perhaps may lead to some common standards and solutions. Before doing so, I'd like to hear from site owners about what they consider to be the biggest concerns. In this way, I can better understand what to ask.
So if you are confused, concerned, perplexed or troubled -- come vent. Or even better, post some thoughtful, specific suggestions or wish list ideas. Go to the address below, scroll down to the Search Engine Watch section and choose the Search Engine Watch Feedback link. You'll see a folder called Submission Guidelines for your comments.
Search Engine Watch Forum
Search Engine Notes
Search Engine Conference Announced
Last April, I had the pleasure of attending the Infonortics search engine conference in Boston. The two-day event had a full schedule of interesting speakers, who covered everything from web search engines to video indexing.
Information for the 1999 conference is now available. It will again be in Boston, from April 19 to 20. There will be speakers from Direct Hit, IBM's Clever Project, Ask Jeeves and Northern Light, to name only a few. I'll also be doing a session on web searching trends.
Search Engines and Beyond
HotBot Beta Site Available
HotBot's beta site is open to the curious. The main difference you'll discover is that top web sites as ranked by Direct Hit are presented directly within the search results page, rather than being an option that users see only if they click on the Direct Hit link.
HotBot Beta Site
MetaCrawler Owner Launches Portal
Go2Net, which owns the popular MetaCrawler metasearch service, has launched a new portal site that incorporates information from its various web properties. The site is in beta and available at the URL below. You'll find MetaCrawler forms a central anchor point for the portal, but the metasearch engine also continues to operate as a separate service.
SearchUK Adds Keyword Prompter
SearchUK, which indexes pages from United Kingdom web sites, has added a keyword prompter to its service. By selecting "Show Related Words" just above the search results, the service will present a list of alternative terms that can be used to refine a query.
Photo Search Fight Continues
Last month, I wrote that photographer Leslie A. Kelly complained to AltaVista regarding its new photo search service, alleging that it infringed upon his copyright and trademarks. Since then, AltaVista has removed his images from its index and told Kelly that it considers the matter closed, he reports. Kelly says he will press on until the service compensates him for use and damages. He has also sent a letter to Lycos, alleging similar infringement of his material in its photo search service.
Search Engine Articles
Find it on the Web
PC Magazine, December 1998
The magazine released its annual survey of search sites. Editors' Choice awards went to Yahoo for simple search, Northern Light for advanced search, Ask Jeeves For Kids for kids' search, and MetaCrawler for metasearch. Honorable mentions went to Google for simple search and HotBot for advanced search. The article is a great round-up of large, small and niche services. I only wish the reviews had been longer, and that the services not selected as top picks had still been rated, somehow.
PC Computing 1998 MVP Awards: Search Engine
PC Computing, December 1998
HotBot takes the top honors in the search engine category, while Excite and Yahoo get finalist mentions. Infoseek's Express metasearch software took the MVP for web utility (see second URL).
Netscapesucks.com ordered to cease and desist
News.com, Dec. 1, 1998
Netscape is threatening the person so upset with SmartBrowsing that he created the netscapesucks.com site. The company claims trademark infringement.
Netscape, Infoseek reshape deal
Bloomberg, Nov. 29, 1998
Infoseek's rotation within Netscape's Netcenter is to be reduced next year.
Tag, You're It! XML Supercharges the Net
Industry Standard, Nov. 13, 1998
How XML is being used for some specialty search needs.
Below are sponsor messages that ran in this month's issue of the Search Engine Report, which may
be of interest to Search Engine Update readers.
When a prospect does a search for a keyword related to your products or services, do you appear in the top 10 or does your competition? Submitting alone does nothing to insure good visibility. WebPosition is the first software product to monitor & to help improve your search positions in the top search engines. Rated 5 out of 5 stars by ZD Net.
Try out WebPosition yourself FREE:
eGroups.com makes it easy for anyone to start an email group.
It's flexible and FREE, and a great way to keep in touch with:
*co-workers *family *colleagues *alumni *teams
*people interested in almost any topic *customers
eGroups.com: Join more than TWO million members in discovering
the power of free email groups at http://www.eGroups.com
Looking for a FREE way to jazz up your site and attract visitors?
Link it to your Deja Community, the newest service from Deja News
+ Make your site interactive by adding discussions on any topic
+ Bring your users back more often and keep them around longer
Creating your own free Deja Community takes less than 5 minutes!
Get started right away or find out more:
KEYWORD BUYS FOR PENNIES PER CLICK -- GoTo.com.
GoTo.com is the most cost-effective way to drive targeted traffic to your site. Get top placement and qualified leads for pennies per click. You only pay when someone clicks your listing, and you control your site’s ranking in the search results! Where else can you get keyword advertising for pennies per click? Click here:
How do I unsubscribe?
+ Send a message to firstname.lastname@example.org with the following as the first line of the body:
How do I subscribe?
+ The Search Engine Update is only available to paid subscribers of the Search Engine Watch web site. If you are not a subscriber and somehow are receiving a copy of the newsletter, learn how to subscribe at: http://searchenginewatch.com/about/subscribe.html
How do I see past issues?
+ Follow the links at:
Is there an HTML version?
+ Yes, but not via email. View it online at:
How do I change my address?
+ Send a message to email@example.com
I need human help!
+ Send a message to firstname.lastname@example.org. DO NOT send messages regarding list management or site subscription issues to Danny Sullivan. He does not deal with these directly.
I have feedback about an article!
+ I'd love to hear it. Use the form at http://searchenginewatch.com/about/contact.html.
This newsletter is Copyright (c) Mecklermedia, 1998