THE SEARCH ENGINE REPORT
August 2, 2001- Number 57
By Danny Sullivan
Editor, Search Engine Watch
Copyright (c) 2001 INT Media Group, Inc.
About The Report
The Search Engine Report is a monthly newsletter that covers developments with search engines and changes to the Search Engine Watch web site, http://searchenginewatch.com/.
The report has over 170,000 subscribers. You may pass this newsletter on to others, as long either part is sent in its entirety.
Did you know that there's a longer, more in-depth version of this
newsletter? The twice-monthly "Search Engine Update" newsletter is
just one of the many benefits available to Search Engine Watch members
Learn more about the advantages to becoming a member at this page:
Link for AOL users:
Membership Sign-Up Info
Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.
In This Issue
+ Site News
+ Search Engine Strategies Conference Only Two Weeks Away!
+ Search Engine Marketing Finally Getting Respect
-- (full story online, link provided)
+ Consumer Group Asks FTC To Investigate Search Ads
-- (full story online, link provided)
+ LookSmart Overhauls Submission Process, Increases Prices
-- (full story online, link provided)
+ New AllTheWeb.com Goes Live
+ A Webby, More Images, Date Range Search & Search Voyeurism At Google
+ Search Engine Security Concerns
+ NBCi Switches To GoTo
+ Scout Report's Susan Calcari Passes Away
+ Search Engine Resources
+ SearchDay Articles
+ Search Engine Articles
+ List Info (Subscribing/Unsubscribing)
The Jupiter Media Metrix Search Engine Ratings page has been updated. The latest figures show that Go and NBCi, which have dropped out of the portal game, have also lost audience share. Meanwhile, Excite and AltaVista show further weakness, while Google and GoTo continue to climb.
Jupiter Media Metrix Search Engine Ratings
Chris Sherman has been very busy with the daily SearchDay newsletter. If you don't already receive it, please be sure to see the SearchDay section in Part 2 of this newsletter for links to interesting articles about a new way to use your favorite search engine from the address bar of Internet Explorer and an article about the new volunteer directory, JoeAnt. Those of you who are interested in site and enterprise search issues will also find a special SearchDay with an update of developments in this area, provided by guest writer Avi Rappoport, of SearchTools.com.
Finally, I've been plugging away to update my pages about searching tips, power commands and special features. This will also include updates on how to locate your URL in search engines and do backward link lookups. It's been a massive job, and there are lots of new commands and tips to share. I expect to have all that ready by next week, fingers crossed. Keep an eye on the What's New page, and you'll know when it is ready.
Search Engine Strategies Conference Only Two Weeks Away!
It's two weeks and counting down until the next Search Engine Strategies conference. To be held in San Francisco on August 16th and 17th, the conference features an entire day devoted to marketing your site on search engines. Sessions are designed to bring beginners up to speed on promotion issues, while there are also in-depth sessions designed for more advanced marketers.
The conference is also unique in having several panels that feature representatives from the actual search engines themselves. Get first hand advice, direct from the source! The second day of the conference also allows marketers more opportunity to hear sessions on promotion issues.
Web searchers -- you're not left out! Day 2 features two "searcher-oriented" tracks designed for you. These panels will give you tips on searching better, as well as a glimpse into behind the scenes issues that influence how well search engines work for searchers.
Developers and systems administrators also have a track for them on Day 2, designed to help you understand how to add a search engine to your own web site, intranet or to create a vertical search feature. Learn how improving site and ecommerce search can lead to more sales or how having better intranet search can mean productivity savings.
I'll be speaking at the conference, along with other search engine marketing and research experts. There will also be speakers from the search engines themselves, including About.com, AltaVista, Ask Jeeves, Excite, FAST Search, Google, GoTo, Inktomi, LookSmart, MSN Search, Netscape/The Open Directory and Yahoo.
The conference is sponsored by AltaVista and exhibiting companies include ah-ha.com, AltaVista, Build Your Own Directory, Business.com, Did-it.com, EasyAsk, Fast Search & Transfer, FireSpout, Inceptor, INFERNOsearch, Inktomi, LexiQuest, LingoMotors, Position Technologies, Quigo Technologies, Search Boss, SiteLab, Sprinks, Web Ignite, WebSeed and WebGenius.
Those interested in sponsoring or exhibiting should contact Frank Fazio Jr, [email protected], for more information. Attendees can find the agenda or sign-up for the conference via the URL below:
Search Engine Strategies: San Francisco 2001
Search Engine Marketing Finally Getting Respect
To some degree, search engine marketing has been like the Rodney Dangerfield of online advertising -- it's gotten no respect. Or, at least it has gotten no respect in relation to the time and effort analyst firms have put into understanding it compared to banner advertising. The good news is, that's all about to change. The positive financial performance of GoTo is making people take notice, though all the signs have been there for ages. The full story can be found below and includes links to several interesting surveys, as well as a recap of some financial reports.
Search Engine Marketing Finally Getting Respect
The Search Engine Report, Aug. 2, 2001
Consumer Group Asks FTC To Investigate Search Ads
Is it deceptive advertising to include paid listings in your search results and not clearly label them as ads? A group backed by consumer advocate Ralph Nader believes so, and it's asking the US Federal Trade Commission to take action against seven major search companies. Commercial Alert filed its complaint with the FTC on Monday, claiming that AltaVista, AOL Time Warner, Direct Hit, iWon, LookSmart, Microsoft and Terra Lycos are violating US law by inserting paid listings within their search engine results "without clear and conspicuous disclosure that the ads are ads." A full story can be found below:
Consumer Group Asks FTC To Investigate Search Ads
SearchEngineWatch.com, July 17, 2001
LookSmart Overhauls Submission Process, Increases Prices
The good news is that over the past month, LookSmart has given site owners greater flexibility than ever before to get listed with the service. The changes are especially beneficial for those with non-commercial content. The bad news is that getting listed is now actually more complicated and will cost many commercial sites 50 percent more. The article below explains how the new integration of Zeal.com and LookSmart's listings should help non-commercial sites, as well as the new "Express Modify" and "LookListings" options available to those with marketing budgets.
LookSmart Overhauls Submission Process, Increases Prices
The Search Engine Report, August 2, 2001
New AllTheWeb.com Goes Live
FAST Search's AllTheWeb.com got both a new look and new functionality in July, improvements that have turned the site into a top-notch resource for searchers.
AllTheWeb has always been an important site, because it has consistently offered one of the largest collections of web pages available. Consequently, it was a great resource when you needed to search for information on unusual or obscure topics.
The site's weakness was that it lacked many features that other search engines offered, such as automatically clustering listings, so that one web site didn't dominate the results. It also had generally poor spam filtering and fairly average relevancy for more popular queries. As a result, it wasn't a resource I'd typically recommend as a first stop for average web searchers.
All that's changed, now. There's every reason to consider AllTheWeb as one of your top choices when searching. Its new features and relevancy improvements have made the site far more appealing.
One of the most notable things about the new AllTheWeb is the what it calls "universal search," where the search engine automatically brings back information from different collections that it maintains. For instance in addition to a web page catalog, AllTheWeb also has database of pictures, video clips, MP3 and FTP files from across the web. When you do a search, results from several of these different sources may be presented, in response.
For example, take a search for "britney spears." By default, you are shown matching pages from across the web, leading off with Britney's own official web site. However, on the right hand side of the screen is a picture of Britney and links to bring up more pictures or video clips that seem to be about her.
Generally, the suggestion of pictures or video clips in what AllTheWeb calls its "side bar results" is most likely what you'll see. However, FTP or MP3 suggestions might also appear. Of course, you can also specify exactly which database you'd like to search against by using the links that appear under the search box, on the AllTheWeb home page.
AllTheWeb has also added new search tips that may appear to the right of search results, in a "Search tips" box. Look back at the "britney spears" search, and you'll see that the tip suggests using quotation marks to perform a phrase search. In addition to educating users, you can also select the link in the box to perform the suggested tip.
Clustering, which AllTheWeb calls "site collapsing," is another welcome new feature. It wasn't uncommon to do a search at AllTheWeb and find that all the top results came from the same web site. This problem has now been greatly reduced, and you shouldn't see more than two pages in the top results from any particular site.
If you do want to see more pages from a particular site listed in the results, simply select the "more hits from" link that appears below a listing. You can also override site collapsing for all your searches by using the options on AllTheWeb's customize page.
That new customization page also offers a variety of other features. You can control the porn filter, disable search tips or side bar results, stop term highlighting and more. In addition, there are a variety of new search commands that let you search within URL text or link text. These are summarized on the "Basic Help" page, and all the help files are an easy read and provide a well-done summary of how to use the service.
AllTheWeb also used to have noticeable problems with spam, but you should now find that this has been greatly reduced, due to new filtering that the service is using.
The service has also announced that it will refresh its index every nine to 12 days. Most search engines update on a roughly monthly basis. The last search engine to make such an explicit freshness promise was AltaVista, back in June 1999. It almost immediately failed to keep that promise. It will be interesting to see if AllTheWeb does better.
Index size is 625 million pages, where it has been since around March. That's just below the 700 million or so pages that Google has in its full-text index. FAST also has 70 million listings in its multimedia picture and video catalogs, 2 million MP3 listings and 150 million FTP listings.
All these changes have been done to make AllTheWeb a more attractive destination for users. This is completely opposite of what happened when the site launched back in April 1999. It was meant primarily to demonstrate Fast's technology to prospective portal partners, so special features for searchers weren't added.
Does the change now mean that Fast wants to compete against the portals that it also wants to power? It's more an attempt to coexist, in the way that Google operates alongside its partners such as Yahoo and Netscape, as well as turn AllTheWeb's successes into benefits for FAST's customers.
"The focus of our site is not to abandon all the portal customers and become the number one search destination ourselves. Our focus is toward monetizing AllTheWeb and using that to build monetization technology for potential customers," said Stephen Baker, Fast's director of business development and marketing.
The key is continuing to develop AllTheWeb's ability to dynamically and intelligently pull information from different databases. Fast believes this can help e-commerce sites and others better monetize their services.
Of course, Fast will still continue providing web search services to those who want it. It just expanded its agreement with Terra Lycos in July, so that Fast information can be used by all Terra Lycos sites worldwide. New addition at Lycos using Fast data include Lycos properties for Argentina, Brazil, Mexico and Venezuela. Lycos now also has the ability to sell advertising to appear on the AllTheWeb site.
AllTheWeb Customize Page
It's Fresher at FAST
SearchDay, July 25, 2001
Guest writer and search expert Gary Price puts Fast's claim to be the freshest search engine to the test. It easily beat AltaVista but fared less well against Google.
A Webby, More Images, Date Range Search & Search Voyeurism At Google
Google has enlarged its image search service, added a new date range searching option and unveiled a way to discover popular search topics at the service. The service also received its third Webby award in July. More details are below:
Google Image Search
Google released a new, 66 percent larger index for Google Image Search in July. Google Image Search now enables users to search and browse 250 million digital images, 100 million more than the first index, which was released in June.
Google Date Range Search
Google has introduced the capability to restrict your search to pages that have been updated with a certain time frame. Google date range search uses a drop-down menu to restrict searches for web pages that have been updated in the past three months, six months, or year. Once a date range has been specified for a search, another drop-down menu will appear at the top of the results page, enabling users to manipulate the date range without back-clicking to the advanced search page. Google's date range search feature is available at the advanced search form for both Google and Google Groups.
Brand new, this provides a look at what people are searching for at Google. Shown are top gaining queries and declining queries, and these are being archived each week. Other treats currently shown are most misspelled queries, as well as languages and operating systems used when interacting with Google.
Google Wins 2001 Webby Award for Best Practices
Google Press Release, July 19, 2001
Google wins the new "Best Practices Award" at the recent Webby Awards, which means it was judged best among competing sites in content, structure and navigation, visual design, functionality, interactivity, and overall experience. Last year, Google won for Best Technical Achievement Award and the People's Voice Award, also in the Technical Achievement category.
Search Engine Security Concerns
Security issues with Lycos and Google came up in July. They aren't likely to impact many, if any, users, so don't get panicked. Here's a rundown on what happened.
At Lycos, the search engine was found to render some site descriptions as actual HTML code. Specifically, when it saw the characters < and > in a description, rather than render them as text, it was turning them into HTML commands, if they surrounded appropriate tags. For example, if this was in a description:
The security report filed about this makes it sound like it would be fairly easy for someone to get high ranking pages and then take advantage of users by manipulating descriptions in this way. The reality is that such pages are far more likely only to appear for obscure searches, making the value of this hack minimal.
Nevertheless, it should be something that is corrected. Despite having been reported to Lycos in mid-July, the problem was still happening yesterday.
"We are aware of this, and I was assured earlier this morning that our engineers are working on the fix immediately," said Terra Lycos spokesperson Kathy O'Reilly.
Meanwhile, Google's advancements to index dynamic content means that it was possible for crackers to get into DCShop shopping cart systems and perhaps find credit card information. Google has since removed links to dcshop.cgi URLs, once the potential problem was reported.
Search Engines HTML Parsing Vulnerability (Lycos)
SecuriTeam.com, July 27, 2001
Security warning about the Lycos problem. Second URL has additional information.
Lycos Example Query
If the bug is still there, you'll see how an input form appears for this entry (don't worry. There's no security problem with viewing this example).
Google removes links to credit card loophole
Fairfax IT, July 26, 2001
More details about the problem at Google.
NBCi Switches To GoTo
In April, I wrote about how NBCi was getting out of the portal business. This month, the company finally shut down the directory it had produced itself. Instead, search results at NBCi now come from that favorite of the failing portals, GoTo. The results are not listed as paid at NBCi, unlike at Go.com, which about a month ago finally added a big "Sponsored Search Results Provided by GoTo" disclaimer to the GoTo results that it now uses.
NBCi Down, Probably Out
SearchEngineWatch.com, April 19, 2001
Covers the initial announcement that NBCi was pulling out of the portal game and links to articles about the closure of Go.
Scout Report's Susan Calcari Passes Away
Produced by librarians and educators, the Scout Report is a weekly highlight of top resources that are valuable to any researcher. It's also one of the oldest and most respected "best picks" newsletters on the web. Sadly, the founder and executive director of the Scout Report, Susan Calcari, passed away on July 8, after a battle with breast cancer. More about Susan can be found below, as well as how to contribute to a scholarship in her name.
Susan Calcari 1956-2001
The Internet Scout Project
Sign-up to receive the Scout Report here.
Search Engine Resources
Designed to let you search for messages posted to message boards across the web. Only the automotive and computer topics are currently live. Unfortunately, you can't do a keyword search against all boards. Instead, you must choose a topic and subtopic in order to search.
Like BoardReader (above), this lets you search message boards across the web. Unlike BoardReader, you can do a keyword search against all boards or browse categories to narrow in on a particular area.
Want a search engines that searches through your programming source code? This application might be your solution.
Canadian search engine, with results gathered primarily from user submissions. Thumbnails of web sites shown next to each listing.
Another site emerging from the ashes of Go Guides is Info World, which catalogs sites primarily related to travel and tourism. It was built initially by spidering all the web sites that were cataloged as regional or travel-related from Go Guides, and adds to this collection come from user submissions and continued spidering. It doesn't look pretty, but some test queries I ran related to travel and tourism weren't bad at all.
Here are some recent articles that may be of interest from Search Engine Watch's daily SearchDay newsletter:
SearchDay, July 24, 2001
While Napster remains shuttered, another search and play service offers a compelling alternative for getting your online music fix. MSN Music is about as opposite from Napster as you can get -- and for both music fans and musicians, that's a good thing.
The Extreme Searcher's Guide to Web Search Engines
SearchDay, July 19, 2001
The Extreme Searcher's Guide to Web Search Engines is one of the most comprehensive, authoritative and just downright useful guides to what goes on under the hood of the major search services.
Search Engine Cloaking: The Controversy Continues
SearchDay, July 18, 2001
SearchDay readers sound off on the ethics, practicality and effectiveness of search engine cloaking. Also links to original article with feedback defending cloaking.
New Search Patents
SearchDay, July 17, 2001
Newly issued patents offer fascinating glimpses of emerging search technologies -- including those that may pose serious threats to your favorite search engine.
RealSearch with Internet Explorer
SearchDay, July 15, 2001
If you're running Internet Explorer, you can forget visiting the home page of the major search engines and directories from now on. Just enter your query into IE's address window, prefaced by the name of the search service you'd like results from, and RealSearch will take care of the rest. Also covers the new date range search feature offered by Google.
Special Search Tools & Products Issue
SearchDay, July 12, 2001
Guest writer and respected industry expert Avi Rappoport provides an update on developments in the world of search tools and products that make sites and intranets searchable.
Hang Ten with SurfWax Metasearch
SearchDay, July 10 & 11, 2001
SurfWax is a metasearch engine with some powerful advanced features, including the ability to build your own customized gateways into the invisible web. In two parts, with URLs to both parts shown above.
JoeAnt Scours the Web
SearchDay, July 5, 2001
From the ashes of Go Guides has emerged JoeAnt, a new volunteer-driven directory of the web.
You can also sign-up for SearchDay on that page to get more articles like these during the workweek, along with search engine headlines from across the web.
Search Engine Articles
eBay Listings Now On AltaVista
SiliconValley.internet.com, August 1, 2001
eBay listings are now being integrated into AltaVista's shopping results.
AltaVista: In search of a turning point
News.com, July 31, 2001
A look at the rise and decline of AltaVista, with some estimates on how much the company may be earning and losing, along with future directions.
GoTo.com Spins Off Auction Business
InternetNews.com, July 30, 2001
GoTo is staying focused on paid listing by selling off its auction service.
Search technology gains recognition
InfoWorld, July 30, 2001
Short news on new search and categorization technologies for corporations from Verity and SmartLogik.
The author of the new book Search Engine Positioning answers questions and discusses the topic. Part 1 is the first URL, Part 2 the second.
Increase your Click-Throughs with Killer Title Tags
SearchEngineGuide.com, July 24, 2001
Using the terms you want a page to be found for in the HTML title is essential. Unfortunately, some people don't take the extra step of ensuring the title is also written to attract clickthrough. Good tips on making your titles standout.
Optimizing for International Search Engines: Part 2
ClickZ, July 18, 2001
Many suggestions from readers of Paul Bruemmer's column on important search engines for various countries and regions of the world. Also links to his original article on the topic.
Tasty New Search Engines
About.com Web Search Guide, July 16, 2001
Review of two new crawler-based search engines, Teoma, which I wrote about last month, and Wisenut, which sounds similar to Teoma and well-worth a look.
Taking The Plunge in the Pay-Per-Click Link Pool
Link Mensch, July 16, 2001
Short, concise overview of getting started with paid placement search engines.
Lookle bows to Google might
The Australian, July 16, 2001
Apparently, Lookle looked a lot like Google, so Lookle changed its look to avoid problems. Google, by the way, told me that they did indeed send a letter to Lookle. Google's concern was that they felt the site imitated Google's look and feel too closely, plus it was operating as a meta search engine, using Google's results without permission. Lookle still appears to be meta searching other search engines without any apparent attribution, judging from a portion of the URL that appears after you do a search: "engines=FindWhat | About | AOLSearch | Excite| Fast | Lycos | MSN | Netscape| Northern_Light |WebCrawler."
Yahoo rises as Semel touts gains
News.com, July 12, 2001
Yahoo reports a net income of $8.7 million for last quarter, beating analyst expectations of no net income. The company had a net loss of $48.5 million, when restructuring and acquisition-related charges are included.
Hey Everybody, Yahoo's On!
The Standard, July 12, 2001
By lowering its earning projections, Yahoo has made a $45 million quarterly loss seem like a success.
Google's Successful Search
BusinessWeek, July 10, 2001
Review of Google's business side, how the company is surviving and thriving in a market where other Internet companies -- including some of its competitors -- are struggling. And hey -- a new IPO prediction, this time for 2002 (though no one from Google is cited as saying this). Also, Google doesn't handle all of Yahoo's search queries, only those that aren't matched by Yahoo's own human-compiled listings. Finally, just as Tara Calishain of ResearchBuzz.com noted in her review of the article, in no way should Google be seen as going down the portal path by offering address lookups or stock quotes. True portal features in the past were designed to make people "stick" within the site. In contrast, Google's features mesh perfectly with its search mission of passing people through its site.
Search engines still searching for profitability
Upside, July 9, 2001
Detailed recap of how search engines have shifted to integrating paid listings and other means of making money, including enterprise search. Lots of quotes and comments from major search players, and check out the part about Google leaving its first $100,000 in investment money in a desk drawer for a month because the yet-to-be-born company had no checking account. Sequoia Capital's Michael Moritz is incorrect when quoted as saying Yahoo has always outsourced its search technology. Yahoo has always had its own search technology -- human editors and internal software to sort through those human-compiled listings. Yahoo has only outsourced for the crawler-based results that kick in if there are no matches from its own listings.
Moreover.com chief steps down
The Guardian, July 9, 2001
Moreover cofounder and chief executive Nick Denton is stepping down, to let the company grow under a former DoubleClick and Oracle executive, Susan Atherton. "The truth is I'm a media guy who has adapted pretty well to running a business but Susan's got 15 years experience," said Denton.
A Master of Headline Grabbing
Time, July 9, 2001
Profile of news search service Moreover.
Illuminating the Web
Time, July 9, 2001
A look at the challenge of indexing the invisible web and enterprises.
Power Up With New Metasearch Tools
About.com Web Search Guide, July 9, 2001
Review that covers the advantages of meta searching and some relatively new meta search tools, SurfWax, Vivisimo and qbSearch.
Dot-com demise reaches researcher Jupiter
News.com, July 5, 2001
Review of problems faced by research and web site rating firm Jupiter Media Metrix, including accusations that its measurements aren't as accurate as rival Nielsen NetRatings. I like the part about companies that are switching to NetRatings because they feel Media Metrix underrepresents them. Want to bet that the minute NetRatings should fail to deliver, these companies will go looking for another ratings firm?
Find Canada on the Web
About Web Search Guide, July 2, 2001
In depth review of how to find Canadian information on the web, eh.
Porn sneaks past search filters
News.com, July 2, 2001
Keeping porn images out of image search results is quite the challenge. While image search engines still can't read images in the way they can text, some can at least identify colors, helping keep files dominant with flesh tones out of the results. It only it weren't for all those darn baby photos messing everything up -- "Babies tend to be showing a lot of skin," said image search company LookThatUp.
Yes, Virginia, Viral Marketing Really is a Key Search Engine Optimization (SEO) Component
Viral Marketing Monthly, July 2001
By running a viral marketing campaign, you are really running a link building campaign -- and thus getting the benefits that can bring from search engines.
Upstream: Video Searching
Technology Review, July/August 2001
Review into research about making video searching easier.
New IT company aims to increase speed of Internet, database searches
Pittsburgh Post-Gazette, June 28, 2001
Vivisimo is a great meta search engine, but the company actually aims to use its technology to help businesses manage information. Also how the company got its name (which I think looks and sounds great, but which I also always misspell).
How do I unsubscribe?
+ Use the form at http://e-newsletters.internet.com/searchengine.html or follow the instructions at the very end of this email.
How do I see past issues?
+ Follow the links at http://searchenginewatch.com/sereport/
Is there an HTML version?
+ Yes, but not via email. View it online at
I didn't get Part 1 or 2. Can you resend it?
+ No, but you can view the entire issue online, via the link above.
How do I change my address?
+ Unsubscribe your old one, then subscribe the new one, using the links above.
I need human help with a list issue!
+ Write to [email protected]. DO NOT send messages regarding list management issues to Danny Sullivan. He does not deal with these.
I have feedback about an article!
+ I'd love to hear it. Use the form at http://searchenginewatch.com/about/contact.html.
Meet Your Favorite Search Engine Watch Contributors
Many of SEW's leading expert contributors will be at ClickZ Live, the new online and digital marketing event kicking off in New York (March 31-April 3). Hear from the likes of: Thom Craver, Josh Braaten, Lisa Barone, Simon Heseltine, Josh McCoy, Lisa Raehsler, Greg Jarboe, Dan Cristo, Joseph Kerschbaum, John Gagnon, Eric Enge and more!