The Search Engine Update, Aug. 21, 2002, Number 131


About The Update

The Search Engine Update is a twice-monthly update of search engine news. It is available only to Search Engine Watch members. Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.


In This Issue

+ Search Engine Strategies Comes To Munich
+ Terra Lycos To Launch Paid Placement Network
+ Search Engine Resources
+ SearchDay Articles
+ Search Engine Articles Review
+ List Info (Subscribing/Unsubscribing)


Hello Everyone--

Every year since 1997, I do a sort of "West Coast tour" of the various search engines based in Silicon Valley, to catch up in person with things they are working on and get a general update. I've just returned from the latest one, doing all my visits before the San Jose Search Engine Strategies conference.

I'm now loaded down with tons of information, so expect a massive brain dump throughout the coming issues of the newsletter. First, however, I have to finish unpacking and find all my notes! They're buried under about 10 pounds of See's Candies ( If you are a Californian like me, you know well how great See's chocolates are. Unfortunately, over the years my wife's British family has also learned to love them, forcing me to bring back more and more boxes to our home in the UK. And then they eat all my favorites!

One big change is that the tour is easier these days. I used to do two search engines per day, and it took a solid week to get through all the major companies. This year, without players such as Excite, Go (Infoseek) and NBCi (Snap), it made for a shorter tour. One nice thing was to see that several good people from these former companies have now surfaced and are working with other search players.

By the way, there is an "East Coast tour" that I do as well, but that happens in the spring. Chris Sherman and I had a good update earlier this year from East Coast-based Terra Lycos, and the company has pushed into a new arena this month, announcing plans for its own cost-per-click paid listings system. There's a long article about the plans that I've now posted, so just follow the link below.

I wanted to mention two quick hits of news where I haven't yet done articles. Last newsletter, I wrote that Google's ads were to appear on Ask Jeeves in September. It looks like things have moved faster than the companies were predicting, because the Google ads are now up at (Teoma remains using Overture, for the time being). Rather than do a separate article on this, I've simply updated the page that discusses how Ask Jeeves works. If you need a refresher, you'll find it below:

How Ask Jeeves Works

The other item is that Inktomi recently moved its index up to the 2 billion page range, putting us firmly into the middle of "Size Wars III," where all the crawlers start coming up to parity in their index sizes.

For instance, Google has quietly creeped its figures to 2.5 billion, past the 2.1 billion mark FAST announced in June as trumping Google. Now Inktomi is also in the 2 billion+ range, and you'll probably see some jockeying for position continue for a bit. Meanwhile, there are grumblings on how everyone is counting pages -- are duplicates really removed from the figures, is spam included, and so on.

I'll probably revisit this in more depth in the future, but the main point to take away is this. Size isn't the only reason to pick a search engine, and despite issues on how exactly to count pages, all three of the players I've named now are probably close enough in coverage that it's not the main issue to choose one over the others.

Instead, it will come back down to perceived relevancy -- did the search engine you chose find what you were looking for? Feature sets will also be important, such as whether you like particular options that are made available.

Want some more background on size issues? See the page below, a long-standing feature of Search Engine Watch. The latest figures aren't yet posted, so ignore the bar chart for the current situation (though I will get that fixed soon). Instead, take a gander and how we've had past size wars on the trend chart as well as the many articles listed that talk about the complex issue of measuring size.

Search Engine Sizes

Changing topics, for years -- literally years -- various people have asked if Search Engine Watch has an affiliate program. I'm happy to say that the answer is finally, "Yes!" The program allows you to link to Search Engine Watch and earn a 10 percent referral fee if visitors from your site become Search Engine Watch members. More details about the program can be found below.

Please note that there is a specific caveat about creating "doorway pages" solely designed to pitch Search Engine Watch memberships. One of my pet peeves about affiliate programs is the amount of search engine spam they generate. I certainly don't want to contribute to this madness. However, this caveat shouldn't pose a problem to anyone with original and substantial content, who hopes to support that content through affiliate earnings.

Search Engine Watch Affiliate Program

Finally, I've updated the How AOL Search Works page. If you read the article about Google and AOL in the last newsletter, there won't be any surprises for you. However, if you'd like or need a refresher, the page stands ready for use.

How AOL Search Works


Search Engine Strategies Comes To Munich

Search Engine Strategies continues on its 2002 tour, this time coming to Munich on Oct. 17 & 18. As with all SES shows, there will be a variety of sessions about improving editorial listings in search engines and how to advertise effectively on them. There will also be an emphasis on German search engine marketing. A full agenda can be found below:

Search Engine Strategies Munich

For those in North America who missed our recent San Jose show, you have a last chance this year: Dallas, on Dec. 11 & 12. You can sign-up via the URL below to be informed when the conference agenda is ready.

Search Engine Strategies Dallas


Terra Lycos To Launch Paid Placement Network

Terra Lycos is set to enter the paid placement game alongside players such as Overture and Google, when the company opens its "InSite AdBuyer" program to advertisers in the coming weeks. The article below explains how the new program will populate "sidebar" ads rather than replace existing paid listing that are provided to Terra Lycos on Overture. Tom Wilde, general manager of search services for Terra Lycos, also explains why his company is making this strategic move.

Terra Lycos To Launch Paid Placement Network
The Search Engine Update, Aug. 21, 2002

Search Engine Resources


Not connected with Search Engine Watch, this new site aims to provide "a look at how Google's monopoly, algorithms, and privacy policies are undermining the web," owner Daniel Brandt tells me. Currently it features two articles. One is an interesting letter to Google criticizing its use of cookies that don't expire for decades. It also notes that Google (like most search engines I know of) tracks user IP addresses and search terms, and that such search data is apparently linked with a user's cookie. A response from Google is also posted, but it says nothing to address Brandt's concerns. Other pages at the site also analyze what's in the Google cookie.

The second article suggests that Google's use of "PageRank" to help rank web pages needs to be regulated, claiming that it is discriminatory against new sites and produces the same inequities that some see in paid placement. The article has some serious flaws. First and foremost is the common mistake that the PageRank is the most important factor to how a page scores at Google.

Google rates the popularity of every page on the web based on the links leading to those pages. This produces a standalone link popularity score called "PageRank," named after Google cofounder Larry Page.

There are formulas followed to weight how various links are calculated, but it should be noted that the exact methods in use are not the same as shown on a much cited research paper ( that Larry Page and Sergey Brin published back in 1998 when they were developing Google at Stanford. Years have passed; the web has changed and Google has had to grow much more sophisticated about how it calculates scores, especially given that its popularity makes it a magnet for those trying to manipulate its rankings.

Of course, PageRank still exists -- but pages are also selected for ranking based on the context of the links pointing at them, as well as the content of the actual page itself. When Brandt writes, "There are several search engines that have made interesting advances in content analysis and even visualization, but Google is not one of them," he unfairly glosses over the fact that Google does substantial content analysis of pages.

There are many other broad and unsupported comments like this that make PageRank sound super-powerful. "It's much more common for a low PageRank to completely bury a page that has perfect on-page relevance by every conceivable measure," Brandt writes. I don't know that this is true. I know it is commonly believed, but running through Google's results can often show that there are indeed pages with "high" PageRank coming behind "low" PageRank pages.

More importantly, anyone experienced in search engine marketing knows that for some, "perfect on-page relevance" simply means having come up with the right gibberish to fool a search engine. The use of link analysis by Google and other search engines has largely saved and revived crawlers as useful web search tools. Back in 1999, it seemed they were simply going to drown under the weight of spam.

Here's another worrying comment: "Those who launch new websites in 2002 have a much more difficult time getting traffic to their sites than they did before Google became dominant. The first step for a new site is to get listed in the Open Directory Project. This is used by Google to seed the crawl every month. But even after a year of trying to coax links to your new site from other established sites, the new webmaster can expect fewer than 30 visitors per day."

I don't think that's true at all. Webmasters will experience all types of traffic levels, and a good new site will likely get plenty of traffic from Google. Certainly I get plenty of reports from people who feel Google is serving their new sites well.

Google is also the search engine that Search Engine Watch readers have voted twice as most "webmaster friendly" ( The many comments associated with those votes weren't from those happy that they could spam Google. Instead, they were from site owners large and small, new and old, that felt the search engine had given them access to an audience based on their merits.

Brandt is absolutely correct in one of his closing statements: "Overall, linking patterns have changed significantly because of Google," he writes, suggesting these changes aren't for the best. He's right. Cross linking for purely promotional purposes has gone haywire, and as the obsession grows -- and the industrial attempts to build link popularity rise -- Google and all the crawlers will be under increasing pressure to add something new to their mix to keep search results useful.

As for regulating PageRank, that's not worth the time. Instead, one thing that came out clearly from many webmasters at the recent Search Engine Strategies conference is the concern that Google and other search engines might fail to crawl sites, ban web sites or drop web sites as a means of pushing these sites into paid placement and paid inclusion programs. If anything will deserve monitoring, it is this issue -- and it's a topic I'll be returning to in a subsequent newsletter.

SearchDay Articles

Here are some recent articles that may be of interest, from Search Engine Watch's daily SearchDay newsletter:

Battling Information Overload
SearchDay, August 21, 2002

Deluged with data? Coping with copiousness? Try these techniques for conquering info glut from internationally renowned super searcher Mary Ellen Bates.


News Sites Beat Search Engines in Customer Satisfaction
SearchDay, August 20, 2002

A respected customer satisfaction index reports that users are happier with news and information sites than search engines or portals -- and predicts the demise of several laggards.


Stumped? Ask the Library of Congress
SearchDay, Aug. 15, 2002

Some of the world's best 'search engines' work at the U.S. Library of Congress, and they're available online to answer your questions by chat or email.


Happy Birthday, Lycos!
SearchDay, Aug. 14, 2002

Eight years ago today Lycos opened its virtual doors to the public, offering a slew of novel features that forever transformed the web search landscape.


A Clutch of Calculators
SearchDay, Aug. 13, 2002

Searching for a calculator for a specific need? The Calculators Online Center has descriptions and links to thousands of specialized calculators on the web.


The Technology Behind Google
SearchDay, Aug. 12, 2002

Want a behind the scenes glimpse at the technology that powers Google? Take a guided tour with Jim Reese, Google's Chief Operations Engineer.


A Guide to Search Engine Marketing
SearchDay, Aug. 8, 2002

A new book tackles the ambitious challenge of explaining how search engines work, and the optimizing and marketing techniques that can improve search engine rankings.


The Seven Deadly Nyms
SearchDay, Aug. 7, 2002

Virtuous searching takes more than hard work and clean thinking -- you must keep constant vigilance against the seven deadly nyms that can play the devil with your search results.


Deja Vu: The Web Remembered
SearchDay, Aug. 6, 2002

Deja Vu allows you to experience the web's early years of stark text landscapes navigated with 'line mode' browsers -- a fascinating snapshot of the environment endured by the web's pioneers.


The Web Intelligence Consortium
SearchDay, Aug. 5, 2002

A group of researchers has launched a new consortium focused on artificial intelligence and advanced information technology on the next generation of Web-empowered products, systems, services, and activities.


On the archive page below, you'll find more articles like those above, plus have the ability to sign-up for the free newsletter.

SearchDay Archives

Search Engine Articles

Search Engine Tips That Get Clicks
WebTalkGuys, Aug. 2002

I was interviewed about search engine marketing recently for the WebTalkGuys radio show. Those interested can read a transcript of excerpts or listen to an archived version of the show.


Search works to clean up its act, Aug. 19, 2002

Review of disclosure changes at search engines in the wake of FTC guidelines about paid content.


Inktomi's troubles hit headquarters
San Francisco Chronicle, Aug. 16, 2002

Inktomi may have to pay $114 million by the end of August to buy the headquarters it is currently leasing. It's got the money, but spending it would leave cash reserves low.


Google Fixes Security Flaws in Search Toolbar
IDG News, Aug. 8, 2002,aid,103706,00.asp

Security holes were found in the popular Google Toolbar, but the company says patches have now automatically been transmitted to anyone using it.


TopDog is back online
Pandia, Aug. 6, 2002

According to this article, the DC Micro version of the TopDog rank checker wins in a court case over the version provided by Michael Lange. The decision means that the DC Micro version can once again be found at While the facts of the article are in dispute by Michael Lange (see next article below), there's no dispute to the closing sentences of the article. I don't know who is to blame, but Pandia is correct in writing, "Readers report that they no longer know who to trust. They wonder which company will survive and if there will be anyone left to give them the service and the update they have paid for. TopDog's competitors are the only parties that benefit from this conflict."

Fake TopDog Controversy -- Rebuttal and the Facts!
Michael Lange, Aug. 2002

Michael Lange tells his side of the TopDog dispute via this article on his web site, where he also continues to offer his version of the software via


All The News That's Fit to Google
@NewYork, Aug. 6, 2002

Google's web search results and ad links are to be integrated into article search results at the New York Times and

List Info

How do I unsubscribe?
+ Follow the instructions at the very end of this email.

How do I subscribe?
The Search Engine Update is only available to paid members of the Search Engine Watch web site. If you are not a member and somehow are receiving a copy of the newsletter, learn how to become a member at:

How do I see past issues?
Follow the links at:

Is there an HTML version?
Yes, but not via email. View it online at:

How do I change my address?
+ Send a message to

I need human help with my membership!
+ Send a message to DO NOT send messages regarding list management or membership issues to Danny Sullivan. He does not deal with these directly.

I have feedback about an article!
+ I'd love to hear it. Use the form at