About The Update
The Search Engine Update is a twice-monthly update of search engine news. It is available only to Search Engine Watch members. Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.
In This Issue
+ Site News
+ Conference News
+ NBCi Down, Probably Out
-- (full story online, link provided)
+ AltaVista Drops RealNames, Promotes GoTo Link
+ The Position Checking Wars
+ Excite Paid Listings Appear
+ Search Engine Finance Round-Up
-- (full story online, link provided)
+ iLOR Makes Google Even Better
-- (full story online, link provided)
+ New Search Engine Resources
+ Interesting Search Engine Articles
+ List Info (Subscribing/Unsubscribing)
Within the Members-Only area, I've now posted full updates of the How Inktomi Works and How AltaVista Works pages. If you've kept up with the newsletters, you'll already be familiar with what's covered on these pages. However, everyone may still find it useful to read them as a refresher. The Search Engine Submission Chart page has also been updated. It provides an at-a-glance overview of submitting to the major crawler-based search engines. I've also added a new section to the Submitting To Search Engines & Encouraging Crawlers page, which covers advantages to having a "shallow" versus a "deep" web site structure to encourage deep spidering. These pages can all be found via the Members-Only What's New page, below:
Members-Only Area What's New
The next two-day Search Engine Strategies conference comes to San Francisco on August 16th and 17th. It will again feature the first day focusing on search engines and promotion issues, with the second day featuring panels designed to help Internet searchers better understand how to use the search tools available to them. A preliminary agenda should go up toward the end of this month. Those interested in sponsoring or exhibiting should contact Frank Fazio Jr, [email protected], for more information. Attendees can signup to be notified of when more session information is available via the URL below:
Search Engine Strategies: San Francisco 2001
NBCi Down, Probably Out
The future of the NBCi site as a search resource is now very much in doubt, given the decision by US television network NBC -- which is NBCi's largest shareholder -- to close its spinoff company. The full story can be found below:
NBCi Down, Probably Out
SearchEngineWatch.com, April 19, 2001
How NBCi Works
It is still worthwhile to submit your site to NBCi. The site remains running, visitors continue to use it, so it's another resource where you may get found. Moreover, the LiveDirectory system makes it easy and fast to be included. The page above explains more about this. However, don't expect your site to be promoted into the main directory. Also, the paid promotion option has been "temporarily" suspended, but you can expect this is probably a permanent move. Should it come back, I certainly wouldn't recommend using it, given the uncertainty of NBCi's future.
AltaVista Drops RealNames, Promotes GoTo Link
AltaVista was the first major search engine to add RealNames links to its listings, and but that partnership came to an end last week, and the links are no longer.
"RealNames has been dropped, and we have no plans at this time to bring them back," said Vaughn Rhodes, senior director of product marketing for AltaVista.
RealNames links have been on AltaVista since May 1998, but the two companies failed to agree on a new deal to keep them at AltaVista, according to RealNames.
"We could not reach business terms, so we did not renew our contract," said RealNames spokesperson Katie Greene.
RealNames also inadvertently lost another major search engine relationship with the closure of Go.com, when that service switched over to using GoTo's paid listings rather than its own in-house results, last month.
RealNames links continue to be offered at Google, iWon and MSN Search. In addition, RealNames remains the default database that Internet Explorer checks, if someone enters a search phrase into the browser's address bar. These browser resolutions make up half of RealNames resolutions.
"More people are using their browser rather than search engines to directly navigate to the sites they are seeking online. Last week, 52 percent of our total resolutions were from the browser line, or 12.4 million," Greene said. In contrast, AltaVista generated about 8 percent of RealNames resolutions.
For the web marketer, the AltaVista-RealNames partnership was initially a good deal. For a low fee, you could register a RealNames keyword containing the search terms you wished to target. Some AltaVista users who then searched for those search terms would select the RealNames link and in turn be shown several RealNames keywords containing that term. Many site owners found this an effective way to indirectly leap to the top of AltaVista's results and tap into traffic.
Over the past year or so, the fees to register RealNames keywords have risen. In addition, the ability to purchase guaranteed placement on nearly all the major search engines, including AltaVista, has emerged. This combination has made RealNames a less-important means of getting traffic for generic terms via search engines. However, it still remains a system worth examining for those who want to receive traffic for their brand names.
In other developments, AltaVista has integrated a fourth GoTo link into its pages. Last issue, I wrote of new "Featured Site" links that were appearing above AltaVista's regular crawler-based listings. Originally, there was only one link, which was sold via AltaVista's advertising department. Now, a second link has been added, this one coming from GoTo. It will be the top paid listing from GoTo for whatever you search for.
In addition, "Sponsored Listings" from GoTo continue to appear at the bottom of AltaVista's results page. These are now paid listings in position 2, 3 and 4 from GoTo.
While the change is a plus for those advertising via GoTo, it also sees AltaVista backsliding from a policy of openly labeling paid links on its service. The "Featured Site" heading doesn't communicate to users clearly that the links are paid placements. Of course, AltaVista is not alone in being unclear. iWon, HotBot, Lycos and Netscape Search also use language such as "Featured" or "Partner" results, rather than calling the listings what they are, "Sponsored."
The change at AltaVista also shows one of the glaring weaknesses of the GoTo distribution model. Having your links distributed to major search engines is now easy (if you have the money), but knowing exactly how and where they will appear continues to be confusing. One day, the top link from GoTo is at the bottom of AltaVista's results -- the next, without any warning, it shows up at the top of the page. It certainly threw off one Search Engine Watch reader, who wrote:
"It would be helpful if you could explain in one of the upcoming newsletters what exactly AltaVista is doing with regards to their 'Featured Listings'. According to your last newsletter, AltaVista was supposed to be selling these directly, but some of my clients' GoTo listings are appearing under the Featured Listings section at the top of the results."
The reader also asked, "Why should anyone buy a Featured Listing if the site will already be featured via GoTo bidding?"
One answer is control. If you want to be on AltaVista and only AltaVista, their internal program allows this. That can be especially helpful if you feel that AltaVista brings you visitors more likely to convert into customers. Of course, if you are also purchasing GoTo links, also buying AltaVista's own links for the same words probably makes no sense.
Meanwhile, the unexpected change at AltaVista also could produce problems for a GoTo advertiser at number four for a particular word. Suddenly, their listing makes it onto the AltaVista results for the first time -- and perhaps gets a traffic increase (and thus, a cost increase) they were not prepared for.
How AltaVista Works
More about Featured Site links and GoTo links at AltaVista.
Using RealNames Links
Illustrated guide to how RealNames links appear at its major search engine partners. I haven't yet removed the references to AltaVista and Go, but that will happen shortly.
The Position Checking Wars
One of the most memorable moments during last month's Search Engine Strategies conference was during the "Meet The Search Engines" panel, when Google was asked about its blocking of WebPosition Gold software. Should it be avoided? Yes, came the reply from Google -- it and other automated rank checkers should be shunned.
There was an audible collective gasp from the audience, with one person in the front then exclaiming something to the effect of, "What are we to do?" From here, Google backpedaled. Rank checking tools cost it resources, because they burden its servers without providing it any compensation. However, the company also recognizes that people do have a desire and need to perform rank checking. Thus, those using rank checking software in moderation are not likely to be blocked, while the company will also be considering ways it might be able to provide rank checking directly.
Several people have asked me what I think the solution is, so I'll share my thoughts. First and foremost, I recommend concentrating on log analysis rather than rank checking. Log analysis shows you exactly how people reached your site, rather than what happened for the terms you are only guessing they are interested in.
Log analysis also gives you a better long term perspective. A program such as WebTrends, for instance, can show you the sum total of traffic you receive from any particular search engine. Focusing on changes to that sum total is better that fixating on changes to individual rankings, because you might discover that even though you've lost some key rankings, the impact on overall traffic is negligible.
Of course, rank checking isn't going to go away. There are still large corporations which run reports in order to satisfy high-level executives that their sites can be found in the top results at search engines for brand names. There's certainly the larger market of web marketers who rely on ranking reports to measure the effectiveness of their work. Search engine optimization companies working on behalf of others may also find it difficult or impossible to get access to log files, so rank checking becomes essential.
Ideally, the search engines themselves would provide these reports, probably for a fee. The advantage to everyone is that terms of interest could be checked only once, which would reduce the skewing automated programs can cause in the popularity of terms. For example, is "sex" really such a popular search term, or are there instead 100,000 different companies trying to rank well for sex that run reports on the term several times per day, thus inflating its popularity?
In an perfect world, the search engines would know all the terms people want to check. They'd check each of these terms once per day, then the results would be sent en masse to anyone who wanted to know where they stood for those terms.
The only problem here is that it is inefficient for site owners to have to work with each search engine individually. That's where makers of rank checking services have an opportunity. If they cut deals with each of the different search engines, they could provide unified reports to their customers, while the search engines earn a portion of the revenues.
As a result, everyone should be happy -- except maybe for the web marketers. That's because the days of "buy it once, rank check forever, for free" would likely end. Instead, web marketers would probably have to pay an annual fee or a fee based on volume of queries.
Of course, the alternative is to see rank checking services possibly become extinct, if search engines decide to act aggressively against them. Given this alternative, most web marketers would probably decide paying a reasonable fee to be worthwhile.
I'm planning to revisit this issue formally in the near future, so you can get a fuller account of the issues involved directly from both the search engines and the makers of rank checking services. For now, the main point is, as I've written before, it's only those who run a large number of rank checking reports every day who are likely to find their software blocked from hitting Google. If you limit your reports, you'll probably be OK.
There's also a related concern. It's one thing to have your ranking checking software stopped. However, if you run a position report, are you then likely to find your web site itself removed from a search engine's index? After all, several people have written to me noticing that the WebPosition.com site has been excluded from both Google and AltaVista. I find that particularly ironic, because this allows the many mirror sites run by WebPosition affiliates to rise to the top, rather than the official WebPosition site itself.
Most people needn't worry that their sites will be banned or demoted because they rank check. I've certainly never had a report from anyone who found running a rank check caused their site to be banned. Moreover, it would be especially hard for search engines to implement such penalties fairly. That's because anyone could run a report for their competitor's site, so innocent parties could be harmed.
To be absolutely safe, I would never include a company name or URL among the terms I was checking. Remember, it is only the terms themselves that are sent to search engines.
For instance, when you send WebPosition on a "mission," it asks you to provide the domain names you want checked. These domain names are only known to WebPosition itself. It does not send them to the various search engines it checks. Instead, it only sends the keyword information that you provide. When the answers for those keywords come back, then WebPosition scans the matches internally, without the search engines' involvement, to create a position report. Given this, as long as the keywords you specify contain no company names, you've passed along no identifying information to link the report to your site.
The only exception is if you access the Internet from a static IP address that can be linked to your company. In that case, the search engine might guess at your identity. However, it is only likely to bother doing so for anyone who runs a large number of reports. If you run reports sporadically, you aren't making yourself a target. Also, you definitely don't want to make use of any URL verification features. These are specifically designed to run queries to find your pages on search engines, thus identifying them to the search engines themselves.
So far, I've stressed that using the position checking features of programs such as WebPosition aren't likely to cause problems. It's a completely different story if you use the doorway page creation features that some position checkers may also offer. If a search engine can identify certain structures that mark a page as being created by a doorway page template, it is possible that they may choose to exclude those pages. This happened in the past with WebPosition at AltaVista, but I haven't had any reports of similar problems for some time. Basically, the main advice is this. The more distinctive and rich in content your pages are, the less likely they are going to seem like spam.
And what exactly is spam? It's a question that always comes up at the Search Engine Strategies conference, and AltaVista gave an answer at the last one that all the search engines would agree with and which site owners should take to heart: "Spam is defined more by intent than any specific technique used."
In other words, rather than being overly worried about the rules governing the use of invisible text, text cloaked by using cascading style sheets, text hidden in form areas, doorway pages, etc., search engines are far more worried about why someone does something on a web page, rather than exactly what they do.
For instance, if you had an all-graphical web page and decided to give it some textual content by adding a description that wasn't visible because the text color matched the page background color, some search engines might technically see this as spamming. However, they'd be far more likely to allow it assuming you were fairly describing the page's content. Your intent was honest, so the page would probably be seen as honest.
In contrast, feed a search engine hundreds of near-duplicate pages all aimed at getting a top ranking for a particular term, and the search engine is likely to view your intent as hostile and take action against you as a spammer.
More on the issue of the position checking wars and WebPosition's past problems with doorway page identification can be found in the resources below. If you are interested in these issues, all are well worth reading.
Thoughts from the Search Engine Strategies Conference in Boston in March 2001
Online Web Training, April 2001
Robin Nobles provides an excellent summary of the position checking wars and an "open letter" format to all parties involved on solutions. She also provides further search engine optimization tips from the last Search Engine Strategies conference.
I-Search: WebPosition Gold Special Issue
I-Search, March 30, 2001
This issue about WebPosition focuses primarily on complaints -- the program's lack of coverage of non-US search engines, that it doesn't make doorway page creation as easy as it sounds, that its doorway page templates got someone banned. However, defenders of the program offer it support.
I-Search #315: WebPosition Responds
I-Search, April 2, 2001
Brent Winters, president of First Place Software, which makes Web Position Gold, constructively responds to complaints, criticisms and concerns about his product. He also touches specifically on the past problem with AltaVista identifying doorway pages created by WebPosition, which he says is no longer an issue.
Tapping Into Natural Traffic
In the special WebPosition Gold issue of I-Search above, you'll see many comments from people who are frustrated over their attempts to gain good rankings with doorway pages. For over five years now, I've repeatedly said, concentrate on building good content, and you'll be rewarded with traffic from search engines and elsewhere. If you've been feeling lost among the doorway page morass, this article revisits the simple, basic things you should be doing to existing pages in your web site to make them search engine friendly.
Keywords Used To Find Your Web Site
How to analyze your log files to determine what words people are using to find your site.
Paid links from FindWhat are now appearing at Excite. You'll find the top three listings from FindWhat displayed at the bottom of the Excite results page, under the heading, "Sponsored Links Provided by FindWhat.com."
Search Engine Finance Round-Up
Inktomi dropped its earning expectations earlier this month, blaming a downturn in demand for its caching product. That's caused the company to layoff 25 percent of its staff, or 250 people. What's interesting here is that for the longest time, Inktomi almost downplayed its search products, because some analysts saw these as a weakness for the company. For instance, when Inktomi lost the Yahoo account last year to Google, a loud response from Inktomi was, "Hey, we do more than search!"
Now, search products might be the company's saving grace. Search revenues were up 65 percent last quarter when compared to the previous year. That was about US $21 million -- and not too far off the $36 million Inktomi expects caching to earn for its second quarter. Inktomi announces its quarterly earning today, so expect more news stories and analysis to come.
Meanwhile, Yahoo gets struck by dotcom malaise, laying off 12 percent of its staff, or about 420 people. It also hires someone -- new CEO Terry Semel, formerly of Warner Bros.
Excite@Home says it plans to concentrate on its broadband business, suggesting that its commitment to maintaining the Excite web site may be waning. Also, Excite's European operation is being approached by possible buyers.
Image search company Ereo, whose star seemed bright after signing a deal with Excite, has cut 75 percent of its staff due to lack of funding.
In other financial news, is Google's growing popularity going to cost it search partnerships estimated to provide half its revenue? Will CMGI be selling AltaVista for much needed cash? And FindWhat's loss widens, but the company remains optimistic.
Articles about all the stories mentioned above can be found on the roundup page below.
Search Engine Finance Round-Up
The Search Engine Update, April 19, 2001
iLOR Makes Google Even Better
iLOR is a new search service that takes the power and relevancy of Google's results and adds on some nifty features than many searchers may find useful. You can find a full review via the URL below:
iLOR Makes Google Even Better
SearchEngineWatch.com, April 19, 2001
Search Engine Resources
InfoGrid Internet Explorer Bar
Allows Internet Explorer users to change the search button from its default behavior of using MSN Search to instead meta searching at several major search engines via the InfoGrid site. Also provides access to news search services and other resources.
A wonderful newsletter always filled with helpful advice on researching information online. Free Pint now is available via a .com address, in addition to its original freepint.co.uk domain.
Looking for US population data? This new research portal provides data about children, fertility, marriage and much more.
Search Engine Articles
Searching the Web gets easier with engines that try to read your mind
US News & World Report, April 16, 2001
Now Google saves lives! Someone wondering if they were having a heart attack did a search on Google, found a page explaining the symptoms, then got to a hospital for help. Without Google, "I'd be dead today," he's quoted as saying. A look at various search engines, search technology and tips.
Yahoo Still a Haven to Sex Searchers
SiliconValley.internet.com, April 13, 2001
Yahoo has never been porn-free, but a recent announcement to expand porn offerings within its shopping area caused the service to come under fire. In response, Yahoo is ousting any stores, classified ads, auctions and banner ads related to pornography. However, the service will continue to list porn sites within its directory listings.
No Bots Allowed!
Interactive Week, April 12, 2001
I've written before that the robots.txt file might be crucial in legal cases involving search engines interacting with web sites. This article takes another look at some of the issues involved, with some comments from the original author of the robots.txt standard.
Search engines got you frazzled? This freebie searches smarter
ZDNet, April 12, 2001
Review of new search companion for your browser called Clicksearch.
Seeding the Engines: Part 1
ClickZ, April 11, 2001
Step-by-step to optimizing web pages for search engines, beginning with keyword search and content tuning.
Napster Dances To A New Gigabeat
SiliconValley.internet.com, April 10, 2001
Filtering out illegally-copied song files hasn't been easy for Napster, so the company has acquired Gigabeat.com, which provides music search technology.
Search site stumbles in a key area -- simplicity
Detroit Free Press, April 10, 2001
The new version of Copernic's meta search software didn't impress reviewer Mike Wendland.
There's No Such Thing as a Free Search
Afternic, April 10, 2001
A long Q&A interview with me on the growth of paid participation programs, tips on getting by without paying.
MSN adds music and 'sounds like' search
InfoWorld, April 4, 2001
MSN apparently has a new music search service that lets you locate songs that sound like other ones.
Your Ego Just Took a Blow
Wired, April 4, 2001
EoExchange has closed its EgoSurf.com, Daily Diffs and EoMonitor sites, all of which offered different types of monitoring services.
The Keys to a Good Search Engine Positioning Strategy
ClickZ, April 4, 2001
Tips on performing search term research.
Misys website tactics may break the law
Sunday Times, April 1, 2001
A large software maker in the UK was found to be including the names of its competitors in meta tags on its site and is now investigating how this happened. Also cites stats from a Computing magazine survey that estimated that Internet retailers in the UK may have lost as much as US $750 million apparently through other sites making use of their trademarks in meta tags. I haven't seen the actual survey, but I find it far fetched that anyone could make anything approaching a realistic estimate of this.
Moreover Tackles Indexing News on the Web
Information Today, April 2001
Long Q&A with Moreover CEO Nick Denton, about the news search service and plans to extend the company's technology in new directions.
InFind Meta Search Closes Temporarily, Perhaps Permanently
SearchEngineWatch.com March 8, 2001
Forgot to list this article last time. InFind was a meta search site with a small but loyal following. Now it's gone. Here's why and a suggestion to try a new, similar service.
How do I unsubscribe?
+ Follow the instructions at the very end of this email.
How do I subscribe?
+ The Search Engine Update is only available to paid members of the Search Engine Watch web site. If you are not a member and somehow are receiving a copy of the newsletter, learn how to become a member at: http://searchenginewatch.com/about/subscribe.html
How do I see past issues?
+ Follow the links at:
Is there an HTML version?
+ Yes, but not via email. View it online at:
How do I change my address?
+ Send a message to [email protected]
I need human help with my subscription!
+ Send a message to [email protected]. DO NOT send messages regarding list management or membership issues to Danny Sullivan. He does not deal with these directly.
I have feedback about an article!
+ I'd love to hear it. Use the form at
Meet Your Favorite Search Engine Watch Contributors
Many of SEW's leading expert contributors will be at ClickZ Live, the new online and digital marketing event kicking off in New York (March 31-April 3). Hear from the likes of: Thom Craver, Josh Braaten, Lisa Barone, Simon Heseltine, Josh McCoy, Lisa Raehsler, Greg Jarboe, Dan Cristo, Joseph Kerschbaum, John Gagnon, Eric Enge and more!