The Search Engine Update, February 6, 2001, Number 94


About The Update

The Search Engine Update is a twice-monthly update of search engine news. It is available only to Search Engine Watch "site subscribers." Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.


In This Issue

+ Site & Conference News
+ The End For Search Engines?
-- (full story online, summary and link provided)
+ Going, Going,
-- (full story online, summary and link provided)
+ AltaVista Tightens Belt, Stays Hopeful On Search
+ LookSmart Cuts Staff But Gains Big Amazon Deal
+ Excite, NBCi Also Reduce Workforce
+ Yahoo Gets Paid Listings
+ Inktomi Gets European Submission Partner
+ Google Does PDF & Other Changes
+ Patent Wars II: CMGI Strikes Back
+ Bush's Dubious Victory At Google
-- (full story online, summary and link provided)
+ WebTop Search Rage Study
-- (full story online, summary and link provided)
+ Internet Top Information Resource, Study Finds
-- (full story online, summary and link provided)
+ Interesting Search Engine Articles
+ List Info (Subscribing/Unsubscribing)


Site News

Hello Everyone--

This month, the newsletter is heavy on the business side of search engine developments. I try not to concentrate on this aspect too much, because I know it's boring to many people. However, with the closure of announced and several other search engines having had recent layoffs, I think there is concern even among non-business types about whether their favorite search engine is going to be around much longer.

I think the news sounds much worse than it is, and I've been busy on a series of articles I hope will provide you with some perspective on the changes, whether you are worried from a searcher's point of view or wondering from a site owner's or web marketer's perspective.

The shorter articles are within the emailed version of this newsletter, but one was simply too long to include and is available online only -- "The End For Search Engines?" I do encourage you to read this, where I compare the current doom and gloom now with the same apocalyptic statements that were issued in 1997. Search engines changed their business models then and survived, through we ended up with portals that certainly didn't please some web searchers. We're seeing a change again, one where pure search may be rewarded, but the trade-off will be whether search engines can balance paid listings, paid inclusion, paid submission and other revenue-generating programs with a quality search product that doesn't alienate users. A link to this article is in the newsletter, below.

In happier news, I'm pleased to announce the winners of the 2000 Search Engine Watch awards. Google won as both Outstanding Search Service and Most Webmaster Friendly Search Engine. Ixquick and Dogpile won for Best Meta Search Engine, while news search service Moreover won for Best Specialty Search. Two services were also inducted into the Hall Of Fame: Yahoo and FindLaw. More details can be found via the awards page below, and thanks to all of you who participated! I'm looking forward to expanding the awards next year, based on your many comments and suggestions.

2000 Search Engine Watch Awards

Within the web site, I've completely reorganized the former "Pay For Placement?" page. You'll now find a brand new page called "Buying Your Way In," which itemizes the major paid programs offered by various search engines in a nice little chart. You know I'm not happy until everything's in a chart. The old Pay For Placement page continues, but this now contains only links to past articles I've written on the topic of paid programs, along with articles written by others. Both can be found via the What's New page, below:

Search Engine Watch What's New

In the subscribers area, I'm still doing an update to the How GoTo Works page, although all the key information you need is there. The main thing I'm working on is transforming the page to reflect the fact that GoTo is really more a search engine ad network now, rather than a destination search engine. As such, it's important to understand where and how your links will be distributed to other search engines, especially given that GoTo doesn't allow you to pick and choose -- a feature it really needs to consider implementing. I have added one key part of my update to the existing GoTo page -- a chart showing where your paid links will appear, depending on your position. One new place is over at Direct Hit, which is now carrying three GoTo links at the top of its results.

How GoTo Works

Password Finder


Conference News

The next Search Engine Strategies conference is just 10 days away! Coming to London on February 15, I'll be presenting and moderating sessions at the conference that features experts on search engine marketing issues and panelists from the various major search engines themselves.

Services participating include AltaVista, Google, GoTo, Inktomi, LookSmart, Lycos Europe, MSN Search, Netscape/The Open Directory and Yahoo. There will be a special presentation on regional and language issues, of interest to those based in Europe or who need to be listed with European search engines. We will also be having roundtable sessions where European search engine marketing experts will answer questions about advanced issues.

Search Engine Strategies is also coming back to the US, in Boston on March 20 and 21. This will be our first two-day event, with the second day devoted to "searcher" issues. Come to the day that suits your interest or get a discount if attending both days makes sense. An agenda is available now.

Information for both conferences can be found via the new web site for all Search Engine Strategies conferences, past, current and future:

Search Engine Strategies


The End For Search Engines?

The impending closure of Go only underscores the dramatic changes that have been taking place among the major search engines over the past few months. Money is tight; new revenue is being sought anywhere, and no one seems guaranteed a future. Will your favorite search engine be around tomorrow? For searchers, such losses could mean less diversity in search results. For web marketers, a consolidation could mean less likelihood of being found. It's scary sounding stuff, and no one knows the answers. However, a look back can provide us some perspective on how the future may unfold. We've had doom and gloom before, in 1997, and survived. See the full story, below:

The End For Search Engines?
The Search Engine Report, Feb. 6, 2001


Going, Going,

The site is to close by the end of February, according to a surprise announcement today from owner Walt Disney Company, marking the end of one of the web's oldest major search resources. More details on the closure, including refunds to those who paid for yearly spidering, below:

Going, Going,, Jan. 29
, 2001


AltaVista Tightens Belt, Stays Hopeful On Search

AltaVista had another round of layoffs in January, losing 200 people, which cuts the company workforce to about 600 total. AltaVista is retaining its search engineering staff and says it fully intends to continue developing its own web wide search product, rather than outsourcing this to a third party such as Inktomi, FAST or Google.

"We're absolutely not reducing our equipment or search engineers. As a matter of fact, it's the opposite," said Ralph DiMuccio, AltaVista's industry relations manager. "Another group that was left virtually untouched was the enterprise group."

The enterprise group is AltaVista's division that produces software that companies can use for intranet and site specific search. AltaVista sees this as a growth area, nor is it alone. For instance, last year Inktomi acquired Ultraseek, the enterprise product that grew out of Infoseek. This was so Inktomi could diversify its search offerings and compete for intranet and site specific clients.

AltaVista's search engineering team supports both its web search and enterprise search operations, and the company feels the synergy between web and enterprise is essential to success in both areas.

"If we are going to be a serious leader in the enterprise software market, then we need a group behind that to take what works on the web and productize it," DiMuccio said.

I wouldn't necessarily agree with this. Web search and enterprise search have different requirements, and I don't think you have to be a web search company in order to successfully serve the enterprise market. There certainly are some synergies that can come out of doing both, of course. More importantly, a company like AltaVista gets considerable play when pitching its enterprise product out of the fact that some of the core technology is used for web search. If AltaVista outsourced its web search, then it would lose this advantage, irregardless of the fact that it might not reflect any real problem with the enterprise package.

Of course, by offering web search, AltaVista can also go after a third market: portals and vortals who want to license web search. The company has about 50 to 60 such clients currently, it says, and plans to go after more.

"We are very aggressively pushing into the search licensing market, and maybe we'll get back some of the bigger clients we used to have," DiMuccio said.

The biggest of those clients was Yahoo, lost in 1998 to Inktomi, which itself lost to Google in 2000. AltaVista also gained, then lost, MSN Search to Inktomi in 1999. With both Yahoo and MSN Search, one reason suggested for the loss was AltaVista's attempt to be a strong portal competitor to those companies. Those portal pretensions were dropped in September of last year, and the change to concentrate on search, and nothing but search, showed up last week with AltaVista's redesigned home page.

Missing is the more channelized design meant to bring users into sticky areas of the site. Indeed, things like photo albums and free home pages are gone or going away. Instead, the face of AltaVista has been simplified, so that the focus on search really is front and center.

"What you are also going to see is the ending of most of our content relationships. AltaVista Live will be gone," said DiMuccio. "There's no portal play. It's search and directory, going forward."

Conclusion? Wait and see. Lycos once suggested that it had no plans to drop its own homegrown web search product yet went ahead and did exactly that last year in favor of FAST and Inktomi. However, Lycos did not also have a software division nor did it do much with web licensing. So, perhaps AltaVista will make it in maintaining its own search index, and the web would probably benefit from having an additional resource like this. As for the viability of the AltaVista site itself, that will clearly be tied to whether the company can monetize the service.

"You are seeing a struggle to find the right business model," said DiMuccio. "We have to be able to stand on our own as a company and keep growing revenue, even though the advertising pie may be shrinking."


AltaVista Cuts 200 Jobs, Jan. 18, 2001,2198,3531_564251,00.html

A few more details specific to the recent cuts.

Terra Lycos lassos AltaVista's Raging Bull, Jan. 30, 2001

As part of its jettisoning of non-search features, AltaVista sold its Raging Bull stock site to Terra Lycos.

No IPO for AltaVista...Again, Jan. 10, 2001,2198,3531_557401,00.html

Forget the IPO plans. With dotcom stocks in the dumps, AltaVista has dropped plans to go public for the near future.


LookSmart Cuts Staff But Gains Big Amazon Deal

LookSmart cut its staff by nearly one-third last month, but this was due primarily to the continuing refocus at LookSmart on selling listings that are distributed to partners, rather than trying to sell advertising on its own site.

In fact, just after the layoffs, LookSmart announced its biggest "Subsite Listings" sale to date, to Amazon. The deal allows content from Amazon to be deeply categorized in the LookSmart directory. That means Amazon will likely appear in response to a wider range of queries, so its traffic from LookSmart partners such as MSN Search should increase.

In return, Amazon -- like other Subsite advertisers at LookSmart -- will pay LookSmart a per click fee. LookSmart did not release how much the Amazon deal is worth, only that it is the largest such deal to date. In its last three quarters, listing services have generated $10 million for LookSmart.

To see an Amazon link in action, try a search for "cable guy" at MSN Search, which uses LookSmart results. That should bring up a page within Amazon about The Cable Guy on sale in DVD format. This type of deep listing to content within a site typically would not be found within directory results, because editors would not have the time to index a site so thoroughly and, importantly, maintain those links in the long term. But, since LookSmart is paid by the click for its Subsite listings, such deep categorization is possible.

The recent layoffs did bring one search related loss -- an end to the LookSmart Live question answering service. However, one of LookSmart's acquisitions is starting to work for itself. Over 100,000 URLs from, the volunteer web directory that LookSmart bought last year, have now been included in LookSmart's listings, the company says.

Overall, it would be foolish to see the recent layoffs as a sign of LookSmart being on the ropes. Such thinking simply doesn't take into account the completely new paid inclusion business model that LookSmart is following, which should be successful, given that its search partners are looking for cash.

My favorite quote on LookSmart's prospects has come out of an earnings update by Merrill Lynch's Henry Blodget: "As expected, whacks guidance and 30 percent of employees. Still a real company, though (believe it or not), with plenty of cash. New bar set so low management ought to be able to get out of bed in morning and fall over it."

By the way, the new approach of a directory doing deep listings may produce something we've never needed before -- clustering of directory results.

Clustering, when applied to crawler-based results, means that only the very best page from a particular web site is displayed, in response to a search. Before the days of clustering, you might do a search only to find the top results were dominated with listings from one particular web site. Clustering, now used by virtually all major crawler-based search engines, eliminates this problem and produces more variety in the results.

Clustering has never been needed before for human-compiled listings, because it was unlikely that these results would have more than one or two listings for a particular web site. Now that LookSmart can afford to spend more time categorizing sites -- because it is paid by the click to do so -- these deeper listings can result in one site crowding out others.

For instance, a reader reported that a query for "nutritional therapy" at MSN Search, which uses LookSmart data, produced so many matches from the same site that she didn't feel her site could ever get found for that phrase.

As it turned out, 12 of the top 15 matches I looked at recently all led to different pages within the same web site. There were pages dealing with topics such as bladder infection and heart disease, but because they all included the words "nutritional therapy" in their descriptions, they came up tops for this search.

Obviously, that's not good for a site owner trying to be found for that phrase. More importantly, it's not good for a searcher looking for diversity in search results. To be fair, I haven't seen much of this happening at MSN Search or other LookSmart partners. It's just something that could happen more frequently, as deep categorization continues, unless clustering is introduced. For its part, MSN Search says its considering a fix to the problem.

"We are definitely taking a serious look at this for our next release, by doing things like limiting Subsite listings to three per results page," said Bill Bliss, general manager for MSN Search.

Also, when I first reported on LookSmart's Subsite listings back in November and listed the search engines involved, Excite responded that it wasn't part of the program and shouldn't be listed as a participating service. Given this, I pulled the reference to Excite from the article.

Fast-forward to last month, when LookSmart posted new information on its web site that does a much better job of explaining both its normal paid submission programs and the per click Subsites program that are available. Checking this out, I noticed that the new marketing materials for the Subsites program listed Excite as a partner -- odd, because as I have explained, Excite previously had told me they were not.

This was a mistake on LookSmart's part, and Excite has since been removed -- it still does not participate in the program. Despite this, Excite has been receiving Subsite listings in its feeds from LookSmart. These don't include tracking codes that let LookSmart do billing and share revenues with its partners, so Excite has been making no money from them. Nevertheless, as LookSmart still receives an up front fee for Subsite listings, LookSmart still benefits to have them carried by partners even without tracking codes.

It is not yet clear how this will be resolved. Excite seems to want a feed free of any Subsite listings, unless it becomes a partner. LookSmart seems to indicate that as long as their are no tracking codes, carrying the Subsite listings shouldn't be a problem. I'll keep you informed on the resolution.

Remember, this applies only to the special Subsite listing. There is no problem with Excite carrying LookSmart's more regular listing. It is a full partner there.


LookSmart inks directory deal with Amazon, Jan. 17, 2001

More details on the Amazon deal.

LookSmart Looking Up, Jan. 17, 2001,,12_562131,00.html

Details on the Amazon deal and revenues at LookSmart in general.

LookSmart Pains On 4Q Revision, Jan. 11, 2001,2198,3531_558671,00.html

LookSmart is cutting its staff by a third, due to dropping ad revenues, and it is consolidating operations to focus more tightly around a different type of "ad" product -- selling listings within its directory.

LookSmart Submit Page

Updated materials that outline LookSmart's various listing programs.

Pay For Placement

Past articles about LookSmart's Subsite program can be found in the paid inclusion section of this page.

Search Assistance Features

More about clustering can be found here.


Excite, NBCi Also Reduce Workforce

Along with other search-oriented services, Excite and NBCi also cut staff in January.

Excite@Home, which runs the Excite web site in addition to offering broadband Internet access and other services, laid off 250 people and retains about 2,750 on payroll. In terms of search, the layoffs had the most impact on Excite's directory listings, rather than its web wide crawling.

Though Excite has taken LookSmart's directory feed since August 1999, the company has always tried to massage the listings into category organizations it preferred. Those days are done, because the people responsible for this were let go in the recent layoffs, Excite says.

In addition, those layoffs caused some delays in Excite getting the latest version of the LookSmart information online at Excite. The company expects that its data should be refreshed in the near future. Ordinarily, the LookSmart data should be refreshed at least every two weeks, Excite says.

As for NBCi, it cut 150 jobs throughout the company, in January.



Excite@Home lays off 8 percent of staff, Jan. 23, 2001

More details on the Excite cuts.

NBCi Cuts Workforce, Jan. 18, 2001,,3_563581,00.html

More details on the NBCi cuts.


Yahoo Gets Paid Listings

If the appearance of paid placement listings on search engines has been a revolution over the past few months, then the war is over, because they've now come to that most conservative of search engines, Yahoo. A new "Sponsored Sites" program launched last Thursday allows web sites to be positioned at the top of Yahoo's commercial category listings.

"I hope this is yet another way that the directory can serve as a powerful vehicle for merchants to reach their intended audience, just as that has been a side effect until now," said Srinija Srinivasan, Yahoo's editor in chief, about the new program.

Srinivasan also saw value for Yahoo's users, likening the way paid listings are called out to the way advertisers in Yellow Pages may call attention to themselves by running larger ads or using bold text.

"That information is extremely salient, the size of the ad, bolding, both of these things add elements to that page," Srinivasan said. "From the users' standpoint, this information literally didn't exist before [at Yahoo”."

Before October of last year, virtually no major search engine offered paid placement listings. Now, with the new Yahoo program, Excite remains the only major service not to have them. The growth underscores the popularity of paid listings as a new revenue stream for search engines.

Nearly all of the major services have opted to use GoTo's paid links, which means any site that can prove some type of basic relevancy for a particular term is able to buy its way to the top of the results, for that term. In contrast, Yahoo's system is more restrictive. Only sites already approved by Yahoo editors and listed with its directory can purchase paid listings.

Yahoo's paid listings do not appear on the search results page that appears in response to a keyword search, as is the case with most other services. Instead, they only show up within a well-defined "Sponsored" box at the top of Yahoo's category pages. Paid listings also only appear within commercial category, which are easily identified because of the new yellow reverse bar used to highlight the category name, rather than the normal blue one. For example, compare these two categories:

Shopping & Services: Beanie Babies

Recreation: Beanie Babies

The first is a commercial category about Beanie Babies, so it features yellow bars and has a spot for any sponsored listings. The second is a non-commercial category about the stuffed animals, so it has blue bars and no sponsored listings area.

The advantage to the site owner is that Yahoo's sponsored listings allow them to leap to the top of their category page, where users might more easily find them. Ordinarily, sites are listed alphabetically, though Yahoo uses an automated system to identify sites it considers "Most Popular," in order to place them above the Alphabetical listings.

Previously restricted to only non-commercial categories, this Most Popular section has been expanded throughout the directory, Yahoo says. This means that on commercial category pages, you'll see Sponsored links first, the Most Popular section second and Alphabetical listings last. In non-commercial areas, it will be the Most Popular section first, then Alphabetical listings, as has been the case since last April. Most Popular results are only supposed to appear in categories with 20 or more listings.

Paid listings are sold by the month, with the fee ranging from $25 to $300, depending on the popularity of the category. Some categories may be priced higher than $300, and these must be purchased through Yahoo's ad department, rather than the online self-serve program.

To enroll, begin the process from the Yahoo Sponsored Sites home page (see link below). After entering your URL, you'll be shown what categories your site is already listed in at Yahoo, and you can then choose to promote it into the Sponsored links box at the top of those categories. You'll also be told exactly how much it costs to promote your site, for each particular category.

Remember, your site MUST already be listed in a Yahoo category to be eligible to use the Sponsored Sites program. If you are not already listed, submit using the Yahoo Business Express program. If accepted, you can then go back and pay to promote your site higher on the page.

Unfortunately, you can only appear on the same page where your site is already listed. Many people complain that they are listed within only one of several possible and appropriate places at Yahoo, and this program does not solve that problem. In short, it will make you more visible in the categories where you are already listed, but it will not increase your visibility elsewhere in Yahoo.

"A truly useful commercial directory needs to be commercial, but there are certainly editorial aspects, also," Srinivasan said, explaining the limitation. "The real value that our editors bring is the organization of that information. To allow sites to be listed wherever they want would be a compromise," she said.

Similarly, this also means that you cannot customize the description that appears for your site within the Sponsored box. Instead, it will match whatever Yahoo editors have approved for your regular listing.

The connection between where your site is listed and where sponsored links can be purchased makes it even more important to lobby for multiple category listing of your web site. If you are submitting for the first time, be sure to provide any truly appropriate alternative categories and succinctly explain why your site should be included. Don't be afraid to [politely” argue your case.

"We'd absolutely be receptive to the feedback when people say they are relevant to other areas," Srinivasan said. "Now that we have Business Express, each submitter has that one on one handholding through the process."

If you've already submitted and feel you should be added elsewhere, make use of the Yahoo Change Form to request an alternative category. Additionally, it may make sense to consider breaking up your site into separate web sites. Both of these topics are covered on the "How Yahoo Works" page listed below.

Do keep in mind that you are not going to have the possibility of deep categorization, such as offered (on a per click basis) by LookSmart. Instead, you'll probably be most successful if you focus on one or two additional areas where you have a strong case for inclusion.

Of course, another common complaint about Yahoo is that requests sent via the change form may go nowhere, prompting some to ask for a paid service like Business Express but just for changes. Yahoo still has no plans to do something like this. The main challenge is that offering a "yes" or "no" answer to listing a site, as with Business Express, is simple in comparison to dealing with the range of different change requests that come in to Yahoo.

"The question of changes come down to such shades of editorial judgment that it is hard to articulate and justify to the submitters' satisfaction why our perspectives differ, if we don't agree," Srinivasan said. "If there were extremely hard and fast criteria we could articulate, then we would be able have such a program."

Approval of your paid listing should come within five days. Up to five paid links can appear in the Sponsored box. If there are more than this, Yahoo then rotates all advertisers equally. Unfortunately, Yahoo provides no statistics to measure clickthrough. That means you will have no easy way to tell if people are clicking on your sponsored link or your regular link further down the page.

Sponsored links will also have the word "Sponsored" appear next to them in the regular editorial listings, which I think is somewhat unfortunate. It may give the impression that sites are only being listed in these editorial areas because they have paid, when this is not the case.

Interestingly, Yahoo felt adding "Sponsored" to listings in the editorial areas would actually be a benefit to both users and site owners.

"I hope that being a Sponsored Site says something about your site and being a business that wants to be around that differentiates you in a good way," said Srinivasan.

Upon reflection, she said Yahoo might reconsider this part of the program, if users indeed took the sponsored flags the wrong way.

"If that's the perception, then I'd certainly want to rethink that, Srinivasan said.

Yahoo Sponsored Sites Home Page

Sign up for the program here.

Yahoo Sponsored Sites Help

More information and assistance about the program can be found here.

How Yahoo Works

In depth coverage of submitting your site to Yahoo.

Password Finder


Inktomi's Free Add URL Penalty

Submitting your page to Inktomi via the free Add URL pages of its partners such as HotBot causes them to incur a ranking penalty. What?!!! Relax, it's not as bad as it sounds, though disclosure of this change, made in the middle of last year, would have saved confusion among those in the search engine optimization world. See the full story, below:

Inktomi's Free Add URL Penalty
The Search Engine Report, Feb. 6, 2001


Inktomi Gets European Submission Partner

Inktomi named WebGravity in January as its new European partner for its paid inclusion program, and orders can be placed there or with Position Technologies, Inktomi's original US-based partner.

There's no need to submit to both companies. Either one will get your pages listed with Inktomi's paid inclusion system. So, which should you use?

WebGravity has pricing in UK pounds, while Position Technologies has pricing in US dollars. Since both charge your credit cards, it's more an issue of getting a favorable exchange rate rather than actually having the appropriate currency.

Even with the high exchange rate of the dollar vs. the pound, Position Technologies per page pricing is currently cheaper. That may not be so for other currencies. Make your own calculations, then go with the partner that makes sense.

Inktomi Paid Inclusion Program via WebGravity

Inktomi Paid Inclusion Program via Position Tech


Google Does PDF & Other Changes

Google now includes listings of Adobe PDF files from across the web, a first for any major search engine and a feature long overdue for them to offer. PDF, for Portable Document Format, is a popular means for researchers, among many others, to publish information. By including PDF content in its listings, Google makes its service even more useful for those trying to get into the nooks and crannies of the web.

Not all of Google's computers have been updated with the PDF information, which means that is pretty random as to whether you'll encounter the PDF-enhanced listings.

"If you did a query, depending on the luck of the draw and how busy the data centers are, you might not get PDF search," said Craig Silverstein, Google's directory of technology. All the computers should be updated by the end of this week, he said.

When that happens, users will have access to the full text of 13 million PDF files. They will appear mixed among the normal listings of HTML documents, when relevant for a particular query. However, PDF files will be prefaced by a [pdf” label next to their title. This is to help warn users before they click on these documents, as they can sometimes be quite large to download. They also require having the Adobe Acrobat reader to view them. If you don't have Acrobat, don't worry -- clicking on the "text only" below the listing will let you view a text only version of the files.

Want to restrict your search to just PDF files at Google? Make use of the inurl: command AFTER your search words. For instance, a search for "colleges inurl:pdf" tells Google to find documents containing the word "colleges" and which have pdf in their URL. So far, this seems to pretty much ensure that you only get back PDF files. Multiword queries also seem to work -- "amazon ebay inurl:pdf" brings back PDF files mentioning both of those companies, for example.

That inurl: command look interesting? Here are some other power commands you can add to your Google arsenal:

allinurl: is supposed to tell Google to find ALL the words you specify after it, within the URL of a web page. In contrast, inurl: is supposed to find ANY of the words you specify after it in the URL, rather than all of them, Google says.

You can also try allintitle: to find all the words you specify within the title of a document, while intitle: is supposed to find ANY of the words.

There's also a site: command, which is useful for finding pages just from a particular site -- " google" would bring up all the pages from Search Engine Watch that mention Google, for instance.

Unfortunately, using site: with your domain alone won't work to bring up all of your site's pages. Instead, you'll need to enter at least one other word -- try for something you know appears on every page but which isn't a stop word, like "the." You may find that repeating the words in your main URL will do the trick, such as " searchenginewatch." It looks odd, but it usually works.

Of course, rather than trying to remember these commands, you can also make use of Google's advanced search page (listed below).

Google now also supports the Boolean OR command -- sort of....

"To say we support the OR command would be a great overstatement. People have discovered if you type in a capital OR, Google does something with it," Silverstein said.

What's happened is that Google has added some behind the scene logic to catch variations of words written with diacritical marks, such as accented words like pjches (French for peaches), even when they are written without the accents, Silverstein said. This logic means that if you do a search for two words, with a capital OR in the middle, you may get something similar to a Boolean OR working for you.

In addition to crawling PDF files, Google has also been indexing some dynamically generated content since the end of last year.

"We've been expanding the kinds of dynamic content that we crawl, developing mechanism so we can tell if we are getting trapped. We're starting to crawl more, and we're very excited about it," Silverstein said.

Google has also grown its index of WML and HDML pages, designed for WAP browsers, to 2.5 million pages. An option to find this is supposed to be available when visiting the Google home page using a WAP browser. Be sure NOT to place a slash after the .com in Google -- otherwise, Google will fail to detect you are a mobile user.

Google is also offering a new "University Search" program to universities, where it will allow them to make their sites searchable for free. See below for a link to more information.

In other developments, some WebPosition users have found in past months that they are blocked from running queries at Google. This is by no means a universal problem and seems mostly to effect those running a large number of queries from a fixed IP address. Google has no plans to remove this blocking and considers any robotic requests, no matter how small, to be a violation of its terms of service.

"Our philosophy is that we are happy to have people look at our results, in particular if we have the ability to serve ads up to them. Part of our business model is advertising," Silverstein said. "An automated system, which strips our results....we do not look to kindly on that."

Chances are, ordinary individuals will not find themselves blocked. It's the larger drains on Google that light up the radar screen. "When people start using resources that are noticeable to us, then certainly that's an issue," Silverstein said.

Google has also expanded its AdWords program so that up to eight paid listings per page may appear along the right side of the main results. Previously, this was limited to three paid listings.

Finally, some reports of pages disappearing from Google around mid-November have come in. Google attributes these to temporary glitches during crawling that can happen and says they should be fixed by the most recently released version of its index.


Google Advanced Search

Google Toolbar

Want to search a specific site from Google? That's built into this nifty toolbar, as well as the ability to search the entire web with Google.

Google Ventures into the Invisible Web
About Web Search Guide, Jan. 31, 2001

Chris Sherman has an excellent in depth look of PDF search at Google.

Adobe Acrobat Home Page

Learn more about PDFs and download the Acrobat software that lets you read PDF files from here.

Adobe PDF Search Engine

Adobe's own PDF-specific search engine can be found here. Until Google, this was the best we had, for locating PDF files. Now, its 1 million documents is small compared to Google's 13 million PDF files.

Google Adds Title Syntax (Finally) And URL Syntax
ResearchBuzz, Dec. 6, 2000

Tara Calishain provides a close up look at playing with new Google power commands.

Boolean Searching on Google
Search Engine Showdown, Nov. 3, 2000

Guide from Greg Notess about trying to make use of OR at Google

Google University Search Sign Up

Does your university need a site specific search capability? You can sign up for Google's free offering from here.

Up Close With Google AdWords
The Search Engine Update, Oct. 16, 2000

Earlier article covering the basics of Google's AdWords paid links program.

The Google Interview
Search Engine Matrix Live Chat, Jan. 31, 2001

Google was recently featured in a live chat session for site owners that you will find archived here -- use the drop down box to find the session.


Patent Wars II: CMGI Strikes Back

"Tempest in a teapot."

That what I said in 1997, when Lycos announced it had patents on search engine crawling, and I see nothing different in light of AltaVista's recent claims.

CMGI's chairman and CEO David Wetherell, whose company has a majority stake in AltaVista, suggested in an Internet World article that some patents recently awarded to AltaVista might give it the ability to seek licensing fees.

"Virtually everyone out there who indexes the web is in violation of at least several of those key patents," Wetherall was quoted as saying, adding further that the company would take action against potential infringers soon.

The first reason I'm not worried about this is that Wetherall clearly doesn't know what he's talking about. In the article, he repeats a claim that AltaVista has tried to make before -- that they were the first to spider the web. They were not. Major search engines such as WebCrawler and Lycos were operating a year before the idea of a search engine was even discussed as Digital, which gave birth to AltaVista.

Who was really first? It depends on your definition. The World Wide Web Wanderer is widely recognized as the first web crawler, and it was launched in 1993. However, it didn't let you search anything. Instead, it was designed to measure the growth of the web. In contrast, WebCrawler seems to be the first full-text crawler-based search engine similar to those we use today.

Another example of Wetherall's lack of knowledge is when he discusses a specific set of patents about indexing distributed databases. That sounds similar to a patent Infoseek claimed -- back in 1997.

Infoseek rolled that out almost a week after Lycos let loose a scare about patents it expected to be granted related to web searching. Despite a scary sounding statement from the Lycos chief scientist at the time, four years later, nothing has ever come of this.

Most important is the fact that our current group of search engines all use their own different types of technologies to generate results, and many have patents on the exact techniques they use. That hasn't prevented other search engines from coming up with their own techniques. For example, Direct Hit has patents relating to the use of clickthrough measurements to improve results. That hasn't stopped Inktomi, Yahoo and others from tracking clicks.

Concern over AltaVista's threats probably would have disappeared, but flames were fanned when Alan Emtage, who created the Archie FTP search service in 1990, said he stood by to help anyone fight attempts by AltaVista to pursue its patents. That produced another round of articles, which in turn made AltaVista's threats seem even more real.

I asked AltaVista for the record whether they really do plan on taking action such as outlined by Wetherall -- no response yet. I'll let you know if I get one. More likely, the threat will be allowed to quietly disappear.

The Internet World Interview: David Wetherell
Internet World, Jan. 15, 2001

The interview that sparked the concern over AltaVista's patents.

CMGI Claims Patently Wrong
Wired, Jan. 31, 2001,1282,41508,00.html

More on Emtage's suggestion that his prior art on searching would invalidate AltaVista's claims.

Search Engine Creator: AltaVista Patents Bogus, Jan. 31, 2001,1928,2001_576031,00.html

Longer interview with Emtage, on the AltaVista claims.

Patent may let Lycos license spiders, Sept. 2, 1997,4,13881,00.html

Sound familiar? "Anyone on the Internet using smart spidering technology is potentially an infringer," according to Lycos's chief scientist, Dr. Michael Mauldin. "We're going to be looking into what our competitors' spiders do. Obviously, requesting royalties is one of the options open to us."

Search Engines and Legal Issues

See the "Patents" section for other articles about search engines claiming patents.

A History of Search Engines

By author Wes Sonnenreich, this covers the early days of search engines on the Internet.


Bush's Dubious Victory At Google

Clearly, I should have written about this when it came to my attention in late November -- but to be honest, I really didn't want to offend anyone. Now that the story has appeared in several other places, well -- what the heck. It turns out that for the past three months, if you did a search on Google for a particularly insulting phrase -- think "dumb Oedipus" -- the top result was to the official George W. Bush campaign store. How on earth did this happen? See the full story, below, to learn more:

Bush's Dubious Victory At Google
The Search Engine Report, Feb. 6, 2001


WebTop Search Rage Study

In January, I reported on how long it takes until people experience search rage, according to a study conducted on behalf of search engine I've since been provided a full release of the study. It provides wonderful insight into on how often Internet users in the United States search, what they look for and the satisfaction or frustration they encounter, when searching. In short, Americans search practically every other day on average, spend about 1.5 hours per week searching, are mostly seeking news and entertainment information and generally do locate what they are looking for. More details can be found at the article below:

WebTop Search Rage Study
The Search Engine Report, Feb. 5, 2001


Internet Top Information Resource, Study Finds

A study conducted on behalf of answer service that monitored 74 people over four days during the Fall of 2000 shows that Americans need answers to four questions per day, spend nearly 9 hours per week looking for those answers, online or off, and that the Internet has become their top information resource -- a remarkable change, considering it has only been widely available for about six years. More details can be found at the article below.

Internet Top Information Resource, Study Finds
The Search Engine Report, Feb. 6, 2001

Search Engine Articles

How Moreover Got 10.21% Click Through on Ads
ContentBiz, Feb. 2, 2001

The headline pretty much says it all. Case study of a paid link campaign at Google.


Test Results for Paid Search Engine Listings
B2BMarketingBiz, Jan. 30, 2001

Case study comparing paid link programs at different search engines. Google is put across as the clear winner, but it's not so clear after a closer reading of the story. Though CPMs couldn't be compared between Google and GoTo, the cost per click (CPC) is easy to calculate and more important in this case than CPM. However, the key factor for giving Google the winner's nod is that sales from Google were higher than those from GoTo. This underscores the fact that all traffic is not equal, and that some search engines may provide a better audience than others.


Mining the 'Deep Web' With Specialized Drills
New York Times, Jan. 25, 2001

Long article examining why you might turn to a specialized search tool to find information that ordinary search engines might miss, be it "Invisible Web" content, news information or on other topics. By the way, the search I did here was actually for "linda chavez" rather than "chavez," but the results reported were as dismal.


The Fundamentals of Quality Search
ClickZ, Jan. 18, 2001

Tips from Gerry McGovern of acclaimed web agency NUA, on making basic site search better.

More Fundamentals for a Quality Search
ClickZ, Jan. 25, 2001

As above, focusing on advanced site search and search results pages.


Repositioning the Doorway: Part 1
ClickZ, Jan. 17, 2001

Within the search engine optimization industry, the definition of terms such as "doorway pages" vary depending on who you ask. Given this, how can outsiders trust SEO specialists, this article asks.

Repositioning the Doorway: Part 2
ClickZ, Jan. 24, 2001

An argument that the search engine optimization industry needs standards to survive and thrive. One difficulty here, especially in terms of pricing and approach, is that different companies have different needs. If you have a great web site with lots of wonderful copy, a search engine optimization consultant that works directly with your page titles and meta tags may be able to significantly boost traffic over time, for a one time fee. If you are an adult web site with all of your content hidden behind a password system, those pitching doorway page services are far more likely to be of help.

Where your site falls between these ends of the spectrum is impossible to predict, and you might even find that employing different companies, with different approaches, makes sense. In short, there's no one size fits all solution. When you hear different companies give different pitches, this isn't so much because one is "right" while the other is "wrong." Instead, they've each evolved different models based on the clients they serve and the skills they possess. Ask for references, talk to their clients and go with the ones that serve companies similar to yours.


Where Did All the Bots Go?
ClickZ, Jan. 17, 2001

Every six months or so, someone comes out with a desktop search utility that's supposed to be a search engine killer. There are rave reviews, then the product disappears. Personally, I don't think users like add-ons that they must (1) download, which sometimes takes a long time and (2) may live apart from their browser. The Google toolbar (, for example, is useful because it adds itself within seconds right to your browser -- though users still need to download it (it only takes seconds to do). Bill McCloskey offers another pessimistic voice in this article. By the way, if despite all this you are sold on the idea of search add-ons, you might look at Zapper ( I haven't reviewed it yet, but some favorable comments have come in.


The Old-Fangled Search Engine
Washington Post, Jan. 13, 2001

Search engines have not made libraries and librarians obsolete.


Report: Portals Gaining On Stand-Alone E-tailers
E-Commerce Times, Jan. 15, 2001

When it comes to online shopping, portals are a strong starting place for consumers.


Suspicious-Traffic Allegations Plague Pay-Per-Click Search Company, Jan. 8, 2001

Allegations of clickthrough fraud are being raised by a advertiser. The company says its a misunderstanding, not fraud.


Internet Marketing Firm Wins New Clients by "Tweaking" Google's AdWords Program, Dec. 18, 2000

A marketer finds success with Google paid links, but don't overlook this key point: "So much success has to do with your landing page." In short, be sure when they arrive at your site, everything's set for them to convert toward your goals.

My Reading List

Thanks this month to items spotted in....


Search Engine Guide Newsletter

Search Engine Showdown Newsletter

List Info

How do I unsubscribe?
+ Follow the instructions at the very end of this email.

How do I subscribe?
The Search Engine Update is only available to paid subscribers of the Search Engine Watch web site. If you are not a subscriber and somehow are receiving a copy of the newsletter, learn how to subscribe at:

How do I see past issues?
Follow the links at:

Is there an HTML version?
Yes, but not via email. View it online at:

How do I change my address?
+ Send a message to

I need human help with my subscription!
+ Send a message to DO NOT send messages regarding list management or site subscription issues to Danny Sullivan. He does not deal with these directly.

I have feedback about an article!
+ I'd love to hear it. Use the form at