THE SEARCH ENGINE UPDATE
March 31, 1998 - Number 26
About The Update
The Search Engine Update is a twice-monthly update of search engine news. It is available only to those people who have subscribed to Search Engine Watch, http://searchenginewatch.com/.
Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.
In This Issue
+ Open Text Repositions Its Search Index
+ Life After Yahoo Discussion
+ Infoseek Launches Advanced Search
+ WebCrawler Debuts Lighter Look
+ AltaVista's Imperfect Mirrors
+ Northern Light Offers New Billing Service
+ HotBot Add URL Service Reawakens
+ Search Engine For Internet Professionals
+ Search Engine Notes
+ Search Engine Articles
+ Subscribing/Unsubscribing Info
Go easy on me. Typos may lurk in this issue, as I'm currently traveling and am without a printer. Proofing on-screen is never fun, especially on a laptop screen.
The Subscribers-Only Area has now moved to a new location. The new address is:
Please bear with the occasional bad link or broken image, as a result of the move. I plan to hunt these down over the coming month.
Within the area, in the Fact File section, there is a new Search Talk page, which lists various places people discuss search engines, ranging from promotion tips to technology. This was previously the Search Engine Mailing Lists page, but I've revised and added new information to it.
Search Engine News
The Open Text Index used to be one of the major search services, always compared alongside players such as Infoseek and Lycos. But from mid-1996 onward, the service fell into decline. Open Text had made a decision to concentrate on its software products, and the quality and prominence of its web-wide index suffered.
For example, AltaVista replaced Open Text as Yahoo's search engine partner in mid-1996. The Open Text Index URL submission page closed for several months later that year, and the size of the index remained static, despite the web's growth. By mid-1997, there was no compelling reason for marketers to list their sites with Open Text or for searchers to use the service.
Open Text put the final nail in the Open Text Index's coffin on March 16, completely closing the service. However, in its place, it has erected a new, business-oriented service called Livelink Pinstripe.
After allowing the Open Text Index to slowly wither away, Livelink Pinstripe is a surprising and impressive effort from the same company. The service is colorful, organized and smartly leverages partnerships with various providers to make information available.
The web index is now based on crawl of business-oriented web sites, with the database updated every two weeks. Sites include businesses on the Fortune 1000, Global 500 and Forbes 500 lists, as well as sites with content about business issues. The idea is that this focused database will present results more suited for business users.
The idea is also that this business-oriented service will also attract a business audience interested in purchasing Open Text's software packages, the company said. While advertising is accepted, the site will also pay for itself by being a more effective and better targeted showcase than the old index.
Search can be done across all sites in the database or just among sites that belong in particular categories that Open Text calls "slices." These categories include topics such as "Construction" or "Retail Trade."
Pinstripe also offers specialty services in cooperation with partners. WhoWhere provides people searching. Quote.com makes stock searching available. All specialty services are available via navigation bar at the top of the screen.
A custom crawl of news sites is also planned within a week or so. However, only 10 sites will be indexed, making other news search services such as Excite's NewsTracker or Wired's NewsBot better choices.
I've not tested the quality of results from Pinstripe vs. business searching at the major search engines. However, results tend to be subjective, anyway. If you are a business user and unhappy with your current service, experimenting with Pinstripe certainly seems worth a try.
As for web marketers, Open Text should have been marked off your "must submit to" list long ago. Now it most definitely is, as submissions are no longer accepted or really suitable for the new service. If submitting to the major search engines isn't enough, consider up-and-coming spider-based services such as Northern Light, PlanetSearch and GoTo.
Open Text's Livelink Pinstripe
News Search Engines
Looking for news? Here's a list of news search engines.
Life After Yahoo Discussion
The Online Advertising Discussion List had an interesting thread called "Can we live without Yahoo?" Moderator Richard Hoy raised the issue, discussing the difficulty of explaining to clients that they may not get listed in the directory.
"This is a bit of a headache for an agency like mine. How do you explain to a client that they might not get listed when Yahoo's own (submission) form doesn't clearly state this," Hoy asked.
Responses ranged from "I managed to get in with no problem" to "I'm still trying." These reflect many of the same sentiments expressed during Search Engine Watch's Yahoo Special Report. Conducted last year, the findings in the report still remain very true.
A portion of the discussion revolved around the idea that getting listed may now be more a PR effort. Sending out press releases and building up the overall awareness of a site may convince Yahoo editors to list it, some felt.
One person voiced an obligation theme that often comes up when discussing Yahoo's reluctance to list everything submitted:
"Internet users have demonstrated great support for Yahoo, indeed, built much of the content of Yahoo one URL at a time and to a large and significant extent made it possible for Yahoo to be what it is today," wrote Bob Schmidt. "It would not be asking too much of Yahoo to use some of its rich resources to organize itself so as to make it possible for every web site to be listed in the appropriate category."
There were also a few complaints about outdated links, along with concern about inaccurate descriptions and the inability to get these changed.
"I have not had trouble getting my clients listed. I have had problems getting listings edited or categorized correctly. One description had a typo in it (not original to my submission, apparently inserted by a Yahoo editor). They didn't care. Another client, Swiss Army Brands, received an irrelevant description, nothing like my original submission," wrote Kim Brooks.
Finally, some expressed surprise to that Yahoo considers itself a "media company," not a search service. This shouldn't be that much a surprise. Over the past year or so, most of the search services have made substantial moves to transform themselves into more than just listing services.
Search Engine Watch's Yahoo Special Report
Summarizes the experience of over 150 people who have submitted to Yahoo, as well as provides a long interview with Yahoo's Director of Surfing about listing policies.
Mining Company Illustrates Yahoo Limits
The Search Engine Report, Jan. 9, 1998
Illustrates problems many others may face in terms of being properly categorized or described.
Online Advertising Discussion List
This URL takes you to a subject-sorted list of discussions for March 1998. Scroll down until you see the sections "ONLINE-ADS>> Can we live without Yahoo" and "ONLINE-ADS>> We CAN live without Yahoo" These are the threads to read.
Infoseek added a new advanced search page on March 24, allowing novice users to make better use of its power searching features.
Drop-down boxes let users choose whether to search within an entire web page, the title of a page, the URL of a page or within the text of hyperlinks for matching terms. These aren't new features for Infoseek, but this page allows users to employ them without having to know search commands.
There are three rows of these boxes, and they can be used together to build powerful queries. For example, one could use the first row to say "document must contain the phrase oscar winners" and the second row to say "document should not contain the word 1998." The result should filter out any pages about this year's Oscar awards.
The page is also the place to go if you want to see more than 10 results at a time. Now you can specify 10, 20, 25 or 50 results per page.
The advanced page also allows users to narrow searches to within Infoseek's "collections." These are custom crawls it does of sites it feels are important for categories such as "Business" and "Computers." Searches are only done among pages from these sites. Infoseek also provides this functionality to Search.com.
Narrowing can also be done geographically, restricting results to domains that match particular countries or regions of the world. There can be problems with this type of restriction, such as when a non-US site uses a .com domain. That's where AltaVista's Search By Language feature can be useful, instead. However, the narrowing will be helpful to many people. There's also the ability to filter by the five most used US domains, .com, .edu, .gov, .net, and .org.
Infoseek also added dates to its listings about a month ago. These reflect when a page reports it was created or changed, and that can be a big help in guestimating whether a search engine's index is fresh or old.
Also, Infoseek is continuing to develop the page, so expect further changes and refinements over the coming weeks, as well as more features.
Infoseek Advanced Search
Regional Editions Overview
Explains some of the problems involved with doing country-specific searches based on domain filtering.
AltaVista Search By Language Feature
Explains how AltaVista's Search By Language feature can be used to find regionally specific web pages.
On March 9, WebCrawler debuted a new look meant to position the service as a fast, simple guide to the web.
The changes are mostly subtle and by no means have turned WebCrawler into a watered-down version of Excite. You'll find just as much information presented on the WebCrawler home page as on the Excite home page, and the same is true as you drill down into each service's various channels.
"You don't want to make WebCrawler less functional. There's a constant tension between simplicity and power," explained Joe Kraus, senior vice-president and co-founder of Excite.
Making WebCrawler too simplified might cause users to feel they were missing something, Kraus said. Instead, efforts were concentrated on cosmetics. "What we struggled with was to try to make the feel of the page simpler."
One way of doing this was to use soft colors and small fonts on the service. Such changes are meant to give WebCrawler a different and softer "voice" than its sibling-service Excite.
Along with colors and fonts, Kraus said descriptions were written to be friendlier to new users and page load times reduced, in order to make the service appeal to modem users.
Aside from cosmetic changes, WebCrawler also gained to new features appearing on its home page.
"Today on WebCrawler" features information within the service, focused on a different topic each day. Sunday starts with Health & Fitness, then follows Work & Careers; Kids, Family & Relationships; Personal Finance; Computing; Entertainment; and Travel.
"Daily Toolbox" leads to a list of web gadgets relating to the daily topic. For example, the "Health & Fitness Toolbox" has links out of WebCrawler to an online allergy quiz, a calorie counter and a hospital locator, among others. Smart web marketers might want to consider adding some useful gadgets to their sites, in hopes of getting listed and drawing traffic.
Keep an eye on what's happening at WebCrawler. There's constant talk about Excite transforming its properties from search services to online services. But Excite hopes to program these services in the way a television network programs its content.
It's already doing topical programming via channels. Now WebCrawler is the experiment with daily programming. It's another step toward getting users thinking, "What's on WebCrawler," as opposed to, "What's on the web? I'll search for something with WebCrawler, then leave to go elsewhere."
AltaVista maintains a network of mirror sites outside the United States. These are meant to be identical copies of the main AltaVista index, but in fact, all but one of the mirrors list fewer pages than the main service.
AltaVista confirmed the problem and said that it is working on it, but that it could not release more details on the situation, at the moment.
It is likely that the mirror sites don't yet have the hardware structure of the main site, which was upgraded late last year when the index was increased from 30 million to 100 million web pages.
Some spot-checking of URL counts seems to indicate that the mirrors have about one-quarter to one-third the pages of the main index.
This is important to search engine users outside the US. Mirrors may provide a slightly faster response, due to their closer physical proximity, but they are not as comprehensive as the main service. Related to this, they will display substantially different results than the main index.
AltaVista's Canadian mirror is the exception. Recently launched, the mirror includes 12 million more Canadian web pages than the main service contains. Canadians, and those looking for information about Canada, should certainly compare AltaVista Canada to the main service, to see if the extra pages produce better results.
As a reminder, there is no need to submit web sites to each of the mirrors. A submission to any of them simply causes the California-based spider to visit a site and add pages to the main index there. This index is then copied and sent to the mirrors.
AltaVista Search Network
Lists all the mirror sites.
Northern Light introduced its new Enterprise Account Service on March 30. The service allows businesses and organizations to be billed on a monthly basis for accessing pay-per-view documents from within Northern Light's Special Collections area.
Northern Light said the billing service is expected to especially appeal to groups wishing to do online research but which require monthly invoicing, as opposed to credit card billing.
"We have had librarians and professional researchers who want to use our service but they can't use a credit card. That was a stumbling block," said Hilary MacPhail, Northern Light's Director of Enterprise Marketing.
Despite that problem, many professional researchers have already taken advantage of Northern Light's Special Collections, MacPhail said. The new system simply makes it easier for others to use the unique service, which has begun calling itself a "research engine."
It's a good term, because Northern Light blends together two worlds that are important to researchers: research databases and the World Wide Web.
While the average web surfer is used to searching the web for free, material from many publications is not published online, so it cannot be found using an ordinary search engine. Instead, this material is available from research services such as Lexis-Nexis and Dow Jones.
Professional researchers have long used research services, but they've not been as accessible to a broader audience. One main reason is that they tend to be expensive. Even searching for matching documents can cost money, with some.
Northern Light has continued to grow its own research database and now has more than 3,400 sources contributing to it, such as the Atlantic Monthly, The Economist and Rolling Stone.
But unlike the traditional research service model, Northern Light charges by the document, not by connect time. That lets anyone search for free for matches, and users then pay between US $1 to $4 to read the full-text of the story.
Of course, Northern Light also offers a free web-wide search service, like a normal search engine. But unlike a normal search engine, search results present the best matches from both free documents on the web and those in the Special Collections area. This can be very helpful for professional researchers, because it allows them to check both important areas at the same time.
Northern Light Enterprise Account Service
Normally, any page submitted to HotBot via its Add URL page should appear within 48 hours. However, normal operation of the Add URL service has been disrupted over the past few weeks, due to development work. Thus, many pages have yet to be listed.
HotBot says the service is supposed to return to normal this week. Pages previously submitted via the page should automatically be added without requiring resubmission, but it would be prudent to check on this in a few days to ensure they appear. If not, consider resubmitting.
A few people have also noticed that pages have gone missing from the index. This happens from time to time with all the search services, but it seems to have been more noticeable with HotBot recently.
Generally, HotBot says that any pages that go missing will most likely be reindexed automatically, as part of the normal crawling process. However, should you find a particular page still remains lost, do a direct submission using the Add URL page. That should solve the problem.
HotBot Add URL Page
NetSearcher is a new search engine aimed at Internet professionals. The service uses Lycos to crawl a select set of sites considered important to the Internet industry. The search engine is backed by Mecklermedia.
As explained last month regarding devSearch, a search engine for web developers, the advantage to using a niche search engine is that the search results can tend to be more relevant. Off target pages are less likely to "accidentally" make it to the top, because they are never indexed.
For example, imagine a search for "frames" on general search engine. It's possible that pages about window frames may be top ranked, when a searcher really wants to know about browser frames. A niche search engine avoids this situation by only indexing sites specific to a particular topic.
Search Engine Notes
BullsEye Offers Super Searching From The Desktop
I haven't had a chance to play with a new search tool called BullsEye, but a trustworthy soul tells me it is impressive. It apparently offers metasearching, site discovery services and other tools of great benefit to searchers. I plan to take a closer look over the coming month, but those interested now can participate in a limited beta test.
BullsEye Download Page
Yahoo Adds Instant Messaging
Yahoo launched its instant messaging system, Yahoo Pager, on March 9. Similar to ICQ or Excite's PAL, it allows instant communication between users online.
Search Engine Articles
Compaq will put search on keyboard
Inter@active Week, March 16, 1998
Compaq says that later this year, it will put one button access on its computer keyboards to connect users with search services.
How Digital Built a Huge Net Index
Internet World, March 23, 1998
The nuts-and-bolts about AltaVista's hardware.
Netscape Forms Web Division
Wired, March 28, 1998
Netscape has created a new division to help develop its site and profit more from the visitors that come there. This could put it in direct competition with players such as Yahoo and Excite.
Complicating matters, though not mentioned in the story, is the fact that negotiations are underway regarding placement for search services within the site. Will competitors now be barred, or might they feel they have the upperhand and refuse to fork over the cash they've paid Netscape in the past? Also, Yahoo and Excite both currently have deals to produce much of Netscape's online content. Will this change, now that Netscape plans to do more in-house?
Plotting an Engine's Course In a Hypercompetitive Space
Internet World, March 30, 1998
Q&A with Excite's CEO George Bell, focusing mostly on retailing partnerships.
Crossroads For AltaVista As Compaq Deal Nears
Internet World, March 30, 1998
Explores various decision points AltaVista currently faces, from renegotiating its deal with Yahoo, to trying to turn a profit and perhaps make new owners Compaq happy.
Below are sponsor messages that ran in this month's issue of the Search Engine Report, which may be of interest to Search Engine Update readers.
When a prospect does a search for a keyword related to your products or services, do you appear in the top 10 or does your competition? Submitting alone does nothing to insure good visibility. WebPosition is the first software product to monitor and analyze your search positions in the top search engines. WebPosition has been compared to similar "services" on the web and has been overwhelmingly voted the best and most accurate tool for search position management.
With WebPosition you'll know your exact positions for an unlimited number of keywords. You'll know if you drop in rank. You'll know when a search engine FINALLY indexes you. You'll know when you've been dropped from an engine. WebPosition is rated 5 out of 5 stars by ZD Net, and includes a 110 page guide to improving your search positions.
Try WebPosition yourself for FREE at:
Ask Jeeves, http://AJ.com, is the fastest and easiest way to find answers online. Really! Ask a question in plain English and be taken to one -- and only one -- site that answers your query. No more "8564 matches to your query"! A safe kids version is at http://AJKids.com.
Customized versions of Ask Jeeves make ideal automated customer service agents, technical support agents, human resources experts, or sales assistants. For more information see http://AJ.com/FactSheet/.
Webreference.com - The Webmaster's Reference Library
FREE NEWSLETTER and more info at <http://www.webreference.com/>
Your one-stop Web dev resource - 7-time Winner PC Mag's Top 100
To unsubscribe from this list, send a message to [email protected], with the following in the subject field:
If you have problems, just send a message to [email protected]
This newsletter is Copyright (c) Mecklermedia. It should not be distributed. If you are not a subscriber and somehow are receiving a copy of the newsletter, learn how to subscribe at:
This Year's Premier Digital Marketing Event is #CZLSF
ClickZ Live San Francisco (Aug 11-14) will bring together the industry's leading online marketing practitioners to deliver 4 days of educational sessions and training workshops. From Data-Driven Marketing to Social, Mobile, Display, Search and Email, the comprehensive agenda will help you maximize your marketing efforts and ROI. Register today!