The Search Engine Update, December 3, 2001, Number 114


About The Update

The Search Engine Update is a twice-monthly update of search engine news. It is available only to Search Engine Watch members. Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.


In This Issue

+ Site News: Help With The Search Engine Watch Awards!
+ Search Engine Strategies Comes To Denmark This Month!
+ Search Engine Marketing: You Like It, You Really Like It!
-- (full story online, link provided)
+ Going Beyond HTML Raises Security Concerns With Google
+ InfoSpace Approved To Buy Excite
+ Lycos To Charge For Search? Well, Not Really...
+ LookSmart Adds Paid Listings Option For Small Businesses
+ Kanoodle Feeds Lycos & Other Paid Listings News
+ SearchDay Articles
+ Search Engine Resources
+ List Info (Subscribing/Unsubscribing)


Hello Everyone--

The time for the next Search Engine Watch awards is coming, and you are invited to participate. In about a week, I'll be posting a form allowing you to suggest categories for the awards. We'll likely be repeating all the same categories from last year, but there may be new categories you think should be added. If so, let us know.

In January, an actual voting form will go online. It will allow you to send me and Search Engine Watch associate editor Chris Sherman your thoughts about which search engines deserve awards, for their work during 2001.

To know when the category suggestion form is live, keep an eye on the What's New page, below. To learn more about the awards, see the information from last year's awards, at the second URL listed.

Search Engine Watch What's New

Search Engine Watch Awards


Search Engine Strategies Comes To Denmark This Month!

Search Engine Strategies returns to Europe this year, coming to Copenhagen on December 12. The day-long event features sessions designed for both beginners in search engine marketing and those who are experienced.

I'll be speaking at the event, along with other experts and search engine representatives from FAST, Google, Inktomi,, LookSmart and Overture. The program is being organized by I-Search moderator Detlev Johnson and European-based search engine marketing expert Mikkel deMib Svendsen.

Those interested in sponsoring or exhibiting at the event should contact Frank Fazio Jr,, for more information. Those interested in attending can find the conference agenda and more information via the URL below.

Search Engine Strategies Denmark


Search Engine Marketing: You Like It, You Really Like It!

Last month, I wrote about how I was going to begin using the term "search engine marketing" rather than "search engine optimization" to cover the wide range of activities that involve obtaining traffic from search engines. Apparently, readers like the idea. Over 30 people sent emails to me on the subject, with the vast majority of them in favor of the change. A sampling of responses, from those for and against, can be found below:

Search Engine Marketing: You Like It, You Really Like It!
The Search Engine Report, Dec. 3, 2001


Going Beyond HTML Raises Security Concerns With Google

Now that Google is indexing a wide range of document types beyond HTML and plain text formats, potential security concerns are cropping up, both for searchers and webmasters.

From the searcher point of view, the concern is that you might unwittingly open yourself up to viruses that are embedded in non-HTML files, such as Word macro viruses.

Until recently, search engines only delivered you to "safe" HTML or text files. It was possible that even these type of files might try to harm you, such as via JavaScript exploits. However, anyone who browses the web was already exposed to such potential threats routinely, and they generally don't have problems.

In contrast, people do not routinely open data documents such as Word or Excel files from those who they do not know. Google has changed this, because its search results now contain direct links to such files from across the web. These direct links mean that users might unwittingly open infected files.

For example, try a search for "clearcutting and fish populations in idaho." The second result is an oddly named document called "Clearcutting in." If you were to click on this link, instead of the document loading in your browser, your computer would instead launch Microsoft Word (assuming you have it installed).

This is because the link leads to a DOC file, a data file used by Microsoft Word. Such files can contain viruses, and if you open one without protection, you'd be exposed to any virus inside.

The safe alternative is to always view such results using the "View as HTML" link that Google provides. You'll see this link any time Google lists a non-HTML or text format file. By following it, you will be shown a safe, HTML version of the listing in your browser.

Ideally, Google would switch things around. By default, I think the main link should bring up the safe HTML version while the "View as HTML" link would instead say something like "View Original File Type." That would greatly reduce the odds of searchers getting accidentally infected by a virus. Google says it's something they'll consider.

"We're going to continue to take a close look at this, because as you know, our users and their experience with Google is our number one priority," said spokesperson David Krane.

Krane also said that Google is noticing that when non-HTML content is offered, many users are opting to use the "View as HTML" choice. Aside from avoiding viruses, another good reason to do this is because the HTML versions are typically smaller than the actual data files, which means they load faster.

Another important point to note is that while the potential for viruses to hit searchers exists, the reality is that this hasn't seemed to have actually happened.

"We've yet to see email from any of our users complaining about computer viruses that they obtained via our search results," Krane said.

Meanwhile, some webmasters are reportedly shocked to discover that Word documents, Excel files and other material they make available through public web sites can now be found by searching at Google. There's even the further concern that some of these documents might contain sensitive information, such as credit card numbers or password information.

The reality is that Google hasn't "created" a security problem with these documents. It has simply exposed them. ANY document that is made available on an Internet server (be it web, FTP, Usenet, etc.) can be found by anyone. People can (and do) even create their own spiders to seek documents of particular types, such as email harvesters that roam the Internet in search of email addresses.

If a document is sensitive, don't place it on the Internet, period. What if you must expose it to the Internet, so that selected individuals outside your company or organization can access it? Then establish a password protection or "authentication" system for your web server, and make these documents only available to those who have a username and password.

Authentication systems will stop crawler-based search engines in their tracks. It's an even better solution that using a robots.txt file, because listing sensitive data that you don't want indexed by a spider in your robots.txt file is essentially a menu for any human who reads the file to find that information. An authentication system reveals nothing, and it has the added plus of keeping humans out, as well.

Keep this in mind. None of the major search engine spiders will try to access authenticated information. However, a custom spider or a nefarious human may still try to hack their way in. Authentication is a barrier to them, but not absolute protection.


Google Unveils More of the Invisible Web
SearchDay, Oct. 31, 2001

In-depth review of new coverage of non-HTML files provided by Google. Search Engine Watch members -- use the link on this page to reach the members-only edition written for you, which covers issues about making sure the titles of your non-HTML documents make sense and how to prevent non-HTML documents from being indexed.

New internet search could turn up viruses
New Scientist, Nov. 28, 2001

Touches on issues in the story above, with more quotes from Google.

The Google attack engine
The Register, Nov. 28, 2001

Hackers might be able to use Google to attack servers, switches and routers, this article says.

Google, others dig deep--maybe too deep, Nov. 26, 2001

A long, in-depth look at the security concerns, with quotes from various analysts.


InfoSpace Approved To Buy Excite

InfoSpace's bid to acquire the Excite web site was approved last week, and results there are expected to be powered by InfoSpace-owned Dogpile, in the near future.

For searchers, the impending change offers no compelling reason to visit Excite. Dogpile's results are so heavily dominated by paid listings that if you seek a meta search service, there are much better options to consider.

For web site owners, such a change will mean those of you with paid listings on Overture, LookSmart, Sprinks and are likely to see a traffic increase. This is because paid listings from these services are used to form the first page of Dogpile's results.

Expect this traffic spike to eventually drop off, in the coming months. That's because Excite is essentially only a shell of what it once was, and the new ownership by InfoSpace seems mainly a move to capitalize on what remaining traffic there is, until that dwindles away.

Those of you who had editorial listings sending you substantial traffic from Excite will see a drop, since the Excite will no longer be providing these. The Excite crawler, which was the oldest crawler remaining of the major search engines, now yields that title to AltaVista.


Dogpile Spared from Extinction
SearchDay, Nov. 29, 2001

More details from Chris Sherman about the acquisition and what to expect at Excite.

Meta Search Or Meta Ads
SearchEngineWatch, May 23, 2001

Explains how once valuable meta search results such as those provided by InfoSpace have lost their value due to the heavy inclusion of paid listings. Also suggests some alternative meta search engines to try.


Lycos To Charge For Search? Well, Not Really...

Last month, the head of Terra Lycos in the United States told Reuters that the company was looking to diversify its revenue stream by charging for subscription services, including search.

Is it really true? Will Terra Lycos be the first major company to try to charge for search since Infoseek's failed experiment with the model, back in 1995?

Nope, it's not true, at least from a searcher's perspective. What Terra Lycos means by search "subscriptions" are really services aimed at the web site owner market.

"The initial services are going to be targeted at the web site owner. They will not be paying for search results but more services that a web site owner would want to utilize," said Bryan Burdick, vice president of portal services for Terra Lycos.

Burdick wouldn't say specifically what these services will be, but did say they'd be in line with what some other search engines have offered site owners.

"Our first set of services are not going to be terribly unique, but we will have some twists to it that will be value added," Burdick said. He added that the new services should be available after the New Year.

My prediction? Expect and other Lycos-owned search engines to offer paid inclusion on their Add URL pages. Of course, Terra Lycos doesn't crawl the web itself but instead contracts this out to FAST. So, any "paid inclusion" program with Terra Lycos will simply be sending your URLs on to FAST, with the two companies sharing revenue.

Given this, you needn't wait until after the first of the year, if you want to get started. FAST already has a paid inclusion program in beta testing, so you can enroll in it now. It's expected that this program will come out of beta testing later this month, with finalized pricing.

But what about consumer search subscriptions? Might that also happen? Not likely, says Terra Lycos:

"We have no immediate plans to have consumer-oriented subscriptions," Burdick said. "It would be pretty difficult for anyone out there to start charging for their existing search results set," he explained. "There's really not much you have invested as user."

The idea here, as I've written about before, is that there's no strong barrier to switching search providers, as there is with email providers. For example, if Yahoo began charging for its email accounts, some users would pay just so as not to have the hassle of telling everyone their new email address with another provider.

In contrast, if Google were to suddenly start charging for search, despite the high-quality of results, many people would probably still find that other search engines gave them what they wanted, for free. The barrier against switching just isn't as strong.

What about the idea of being able to search and not see banners, paid listings and other advertisements?

"Consumers aren't even willing to pay to have banners removed," Burdick said. He likened the idea to the old concept of having cable television in order to escape ads. "Today, most cable channels do have ads."

The exception here is with premium channels, like movie offerings. That might also be the case with search. Subscriptions might work for those wishing to perform "premium" searches in particular topic areas. But for general searching, expect it to remain free from all major providers.

By the way, last month I praised Lycos for finally clearly labeling the paid listings it carries as "Sponsored Search Listings." Sadly, not three weeks later, the company was back to its old habits, calling them "Featured Listings" and now "Products & Services."

Terra Lycos Says Exploring Paid Services
Reuters, Nov. 8, 2001

Original article where subscription search and other services from Terra Lycos are mentioned.

How Yahoo Will Make You Pay, Nov. 23, 2001

A look at how Terra Lycos-competitor Yahoo may try to get users to pay for services through its access partnership with SBC Communications.

FAST Partner Site

More about FAST's paid inclusion program can be found here.

Lycos Redeems Itself With Relaunch
The Search Engine Update, Nov. 5, 2001

More about how Lycos utilizes search results from FAST.

Time For The Search Dividend?
The Search Engine Report, April 2, 2001

Some previous comments from me on the idea of search subscriptions.


LookSmart Adds Paid Listings Option For Small Businesses

A new program from LookSmart has made it possible for small businesses and others with low publicity budgets to get listed within the paid placement area of its search results. All of LookSmart's small business products have also been consolidated into a new "LookListings Small Business" area, which provides submission management tools. More about the changes can be found via the URL below:

LookSmart Adds Paid Listings Option For Small Businesses
The Search Engine Update, Dec. 3, 2001


Kanoodle Feeds Lycos & Other Paid Listings News

Kanoodle has a new route to place you on inexpensively. The company has purchased over 1,000 "Start Here" links from Lycos, for terms such as "affiliate marketing" and "zip drive." Anyone who is the number one bidder for these terms at Kanoodle will show up as the Start Here link at Lycos.

In addition, if you purchase one of the selected terms, your link is also supposed to be distributed as a "Start Here" link at the top of results at, as well, though this wasn't working when I checked on it today.

Kanoodle was offering a list showing all the terms it had purchased from Lycos, but the company says it had to remove this, for contractual reasons. Instead, Kanoodle is working on a new way to show existing clients which terms it can provide Start Here placement for.

Start Here links at Lycos appear at the very bottom of the "Products & Services" area of the search results page (or the "Featured Listings" or the "Sponsored Search Listings" area. All are names used within the last month). The main listings in this area continue to come from Overture, but the Start Here links are an alternative paid listing option you can explore.

Start Here links are sold by Lycos itself. They sell for about $35 to $60 CPM (that's 1,000 impressions), depending on volume purchased and the keywords involved.

An exception here is for any of the keywords that Kanoodle now sells. To appear for any of the terms Kanoodle has purchased from Lycos, you need to be the top bidder for that term at Kanoodle, to appear as the Lycos Start Here link.

Over in the Europe, UK-based Espotting picked up both its 4,000 advertiser -- Ford -- as well as a new distribution outlet, AltaVista France. The company distributes its paid listing to a wide range of other partners in Europe, including Lycos, Ask Jeeves, UK Plus, and Netscape Search.

And in some older developments from my files, FindWhat expanded its placements in portions of the CNET network in October, while's paid listings were added to in mid-September.



Lycos Start Here Links

More about Lycos Start Here links, which you can purchase directly from the Lycos advertising department, if they are not one of the terms prepurchased by Kanoodle.

Espotting Expands CNET Deal
@NY, Oct. 1, 2001,1471,8471_894391,00.html

SearchDay Articles

Here are some recent articles that may be of interest, from Search Engine Watch's daily SearchDay newsletter:


Search Engine Anti-Optimization
SearchDay, Nov. 28, 2001

A novel proposal would allow webmasters to *reduce* search engine rankings for specific keywords and phrases, deliberately making pages all but impossible to be found by search engines.


Build Your Own Portable, Personal Search Engine
SearchDay, Nov. 27, 2001

Take a page from Google's playbook and create a searchable cache of important web pages with your own portable, personal search engine.


Fyuse: An Eclectic "My" Portal
SearchDay, Nov. 26, 2001

Fyuse takes the concepts of web-clipping, push-technology, affiliation and distributed content and encapsulates them into a slick, easy to use personalized portal customized to your tastes.


Twelve Cool Sites and Tools for Searchers
SearchDay, Nov. 21, 2001

Create your own web image database, search for streaming multimedia, automatically track changes to your favorite web pages -- check out the dozen sites and tools covered in this roundup.


Search Engine Glossaries
SearchDay, Nov. 20, 2001

Baffled by Boolean? Confused about cloaking? Stumped by stemming? What you need is a good search engine glossary.


An "X-Ray" Toolbar for Web Surfers
SearchDay, Nov. 19, 2001

The Alexa Toolbar aggregates all kinds of useful information about the site you're currently viewing, from contact info to related sites and categories, available with a simple click of a link.


Search Engines with Autopilot
SearchDay, Nov. 14, 2001

If you're often repeating the same query, you can both improve your results and take the drudgework out of searching by taking advantage of search engines that 'fly' on autopilot.


On the archive page below, you'll find more articles like those above, plus have the ability to sign-up for the free newsletter.

SearchDay Archives

Search Engine Resources

ResearchBuzz & ResearchBuzz Extra

I've mentioned ResearchBuzz in this newsletter before, but it is worth repeating both for regular and new readers. Tara Calishain's regular newsletters provide a wealth of information for searchers, especially regarding specialty search engines. Her regular newsletter is free, and those who support her work with a subscription receive ResearchBuzz Extra, which provides even more great information from Tara, such as last month's news search engine "bakeoff." Consider a subscription, both for the information and to help the author behind it.

iLOR HydraLinks

Earlier this year, I wrote about how I liked some of the features that Google-powered iLOR made available, such as the ability to easily build a custom list of sites you want to visit or to find your way "back" via an "anchor" to your original page of search results. Now you can have such features at Google, another search engine or anywhere you browse the web, via this new software from iLor. It downloads in less than two minutes and is available as a free trial, for 30 days. After that, you have to pay the somewhat steep $30 registration price. The second URL has my earlier review of iLOR.

List Info

How do I unsubscribe?
+ Follow the instructions at the very end of this email.

How do I subscribe?
The Search Engine Update is only available to paid members of the Search Engine Watch web site. If you are not a member and somehow are receiving a copy of the newsletter, learn how to become a member at:

How do I see past issues?
Follow the links at:

Is there an HTML version?
Yes, but not via email. View it online at:

How do I change my address?
+ Send a message to

I need human help with my membership!
+ Send a message to DO NOT send messages regarding list management or membership issues to Danny Sullivan. He does not deal with these directly.

I have feedback about an article!
+ I'd love to hear it. Use the form at