WebCrawler Gets "Deportaled"

WebCrawler has a new look and feel, returning to its roots and living up to its longtime motto "It's that Simple." The revamped search engine has ditched nearly all of its "portal" features and is now focusing exclusively on providing high-quality web search results.

WebCrawler is owned by Excite, though for years it used its own, much smaller index of web pages to provide search results. Now WebCrawler and Excite share the same index, and search results are identical for each service.

The key difference between the two is WebCrawler's new emphasis on a clean, uncluttered interface. The search form is prominent immediately beneath the familiar WebCrawler logo, with options to search the web, news, photos, or all three.

Two buttons allow you to choose a standard search, or use Excite's "Zoom In" feature that opens a new window to suggest other terms that might help you refine your search. Beneath the search form are links to Excite's directory.

It's that simple!

Gone are features like yellow pages, maps, horoscopes, chat, and the other trappings of most portal sites. If you like these features, don't worry -- they're are still available via Excite's home page.

Search results are also clean and uncluttered. Directory results are presented first, in a bulleted list. Web site results are next, with a list of the most relevant sites for you query. Clearly identified sponsored (paid) links are at the bottom of the result page. Unlike Excite, which provides 10 results at a time, WebCrawler returns 25 per query.

WebCrawler is now essentially a "pure" search interface to Excite's index and directory. It's targeted at the searcher who wants results and isn't interested in loitering at Excite's more "sticky" portal site, according to Jill Robinson, Product Manager, Search at Excite.

AltaVista experimented with a similar concept with its Raging Search site, launched in May 2000. Raging Search never caught on, however, and AltaVista pulled the plug on the service little more than a year later.

It's unlikely WebCrawler will meet a similar fate. Indeed, search has consistently been the most popular activity on WebCrawler, according to Lynne Mariani Pogue, Director of Product Management for Search at Excite. The goal of the redesign is to better meet the needs of WebCrawler's current users, and to gain new users who are attracted by the clean interface.

Besides, WebCrawler is the Internet's equivalent of a grizzled veteran, holding claim to being the first full-text search engine to appear on the web. It was created by University of Washington graduate student Brian Pinkerton, and went live on April 20, 1994. At the time, WebCrawler's database contained documents from just over 6,000 different servers on the Web.

WebCrawler quickly became an Internet favorite, receiving an average of 15,000 queries per day by October 1994 (these days, by contrast, most major search engines handle millions of queries per day). On November 14th, 1994, WebCrawler served its 1 millionth query, "nuclear weapons design and research."

Pinkerton sold WebCrawler to AOL in March of 1995. But the search engine didn't mesh well with AOL's proprietary service, and in November of 1996, WebCrawler was acquired from America Online by Excite. For a couple of years thereafter, WebCrawler was considered one of the major search engines. Recently, however, WebCrawler has seemed more like an orphan than a full member of the Excite family.

WebCrawler's "de-portalization" is a welcome change to the venerable search engine, as is Excite's apparent commitment to paying attention to WebCrawler's users and tuning the search engine to better fit their needs. Excite has also made improvements to its back end -- for more information, see Danny Sullivan's coverage in the Search Engine Report (link below).


What's Changed?
Details about the changes made to WebCrawler.

A Brief History of WebCrawler
A short but fascinating account of the history of the web's first full-text search engine, from WebCrawler creator Brian Pinkerton.

Excite Makes Changes
Excite has finished making changes to its search interface, mostly minor alterations that the service hopes will improve usability. Here's what you will find that's different.

How WebCrawler Works
Site subscribers have access to extensive information about how WebCrawler and Excite work, including submission tips, how the search engine crawls, spam penalties, how summaries are created and more details. Use this link to learn more about becoming a site subscriber:

Search Headlines

NOTE: Article links often change. In case of a bad link, use the publication's search facility, which most have, and search for the headline.