About The Update
The Search Engine Update is a twice-monthly update of search engine news. It is available only to Search Engine Watch members. Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.
In This Issue
+ Site News
+ Search Engine Strategies Coming To Dallas
+ Additional Search Resources About Terrorist Attacks
+ Yahoo Changes Results, Increases Submission Price
+ Free Search Engine Submission Is Still Alive!
-- (full story online, link provided)
+ Google May Get Personal
-- (full story online, link provided)
+ Google's New Look & New File Types
+ Excite@Home Goes Bankrupt; AltaVista Cuts Staff
+ Surplus Of Search Engine Marketing Reports
-- (full story online, link provided)
+ SearchDay Articles
+ Search Engine Articles
+ List Info (Subscribing/Unsubscribing)
Within the site, these pages have been updated:
Search Engine Submission Tips
A new five-part "Essentials Of Search Engine Submission" section has been added that describes the key steps you need to take to get listed for free or by fee with the web's major search engines.
Search Engine Alliances
Shows how search engines get their editorial listings or give these to others.
Buying Your Way In
Shows how search engines get their paid listings.
Search Engine Strategies Conference Coming To Dallas
Did you miss the Search Engine Strategies conferences held earlier this year in Boston and San Francisco? Don't worry -- you've got one more chance in 2001. On November 14 & 15, Search Engine Strategies will be coming to Dallas, Texas.
Once again, I've organized two days' worth of sessions packed with information about search engine marketing. If you are a beginner and know little, the conference will bring you up to speed. Advanced? There are plenty of more in-depth sessions to choose from. Been to Search Engine Strategies before? There are a number of new and improved panels.
In addition to creating the event program, I'll be speaking at the conference, along with other search engine marketing experts. There will also be speakers from the search engines themselves, including About.com, AltaVista, Ask Jeeves, FAST Search and Inktomi.
Those interested in sponsoring or exhibiting should contact Frank Fazio Jr, email@example.com, for more information. Those interested in attending can find a conference agenda and more information via the URL below. Be sure to see the "Conference at a Glance" page, if you've come before, for a rundown on what's new.
Search Engine Strategies
Additional Search Resources About Terrorist Attacks
Here are some new search resources related to the terrorist attacks on New York and Washington DC that have come to my attention, since the last newsletter.
The Terrorism Directory
SearchDay, Sept. 24, 2001
Who are the people we call terrorists? Why do they commit such horrific acts? The Terrorist Directory helps you conduct your own intelligence operation via the web
The Trade Center Disaster: Industry Response
Internet.com, Sept. 2001
Focuses on how the ecommerce and advertising industries have coped with the attacks.
Lexus Nexus: Attack On America
If you are a Lexus Nexus subscriber, a special area has been compiled about the attacks.
Our changing response to the attacks can be seen in our search queries. In the first week, queries for "world trade center" and news sites were common, as people sought the latest information about the disaster. Then we saw queries about giving donations (red cross), or learning more about those suspected to be behind the attacks (osama bin laden) or ways to fight back symbolically (american flag). Now things are more "back to normal," in that the top search terms are not almost entirely to do with the attacks, as was the case two weeks ago at Lycos. However, many of the searches remain related to the disaster.
Some more "normal terms" are also beginning to return at Yahoo, though the attacks are still a dominate theme.
You can see how attack-related terms are still present at Google but changing in their nature -- "anthrax" is a new worry, rising from out of nowhere, for example.
News knocks out sex in Net searches
Reuters, Sept. 19, 2001
Search logs reflect how deeply the attacks in New York and Washington DC affected people. Almost without precedent, the "traditional" terms that are always high in popularity were replaced by those related to the tragedy.
Yahoo Changes Results, Increases Submission Price
Last week, Yahoo's search results pages underwent probably the most significant change since the service began. Yahoo also increased the price of its Yahoo Express submission service to $299, last month.
For years, Yahoo has followed what I call the "CSP" model of search results: presenting category listings first (the C), then web site listings (the S) and finally matching web pages (the P).
As Yahoo has grown, this model made it likely that users would get overwhelmed by matching category listings, when searching for popular topics. For example, a search for "travel" would find 80 categories, meaning that you needed to hit the "Next" button 3 times before seeing any web site matches.
Of course, many people would immediately find a relevant category in the first page of results and click through to it, thus giving them access to the web site listings compiled by Yahoo editors for that topic. Nevertheless, some people want both category and web site matches all at once, and the Yahoo change has been designed to satisfy them.
"A number of users in our usability tests said they want to see more of those [web site matches” on the first page, said Scott Gatz, general manager of search and directory at Yahoo. "With the changes, we can we present more of those, as well as highlight the concept of our categories."
Matching category links still remain. When found in response to a search, they come at the top of the results, under the heading "Category Matches." You may see up to five category listings. If there are more than five matches, you can access the entire list by using the "Next" link to the far right of the Category Matches heading.
Yahoo has also made its category names more readable. For instance, the "Air Travel" category in the past would have been listed in the search results as "Recreation > Travel > Air Travel." Now all categories have been given short, user-friendly names to appear in the search results.
"For the new folks coming on to the net, they need a little easier way to read what the categories about." Gatz said. "One of the biggest things is making the names shorter and easier for them to understand."
The friendlier names weren't created through an automated process but rather through human review.
"Our surfers actually reviewed each and every last one of our categories and put thought into what the best short name for that category should be," Gatz said.
For example, a search for "california" brings up the "California > Southern California" category link, rather than using just the last two words of the category "path," which would have made it "Counties and Regions > Southern California"
The categories that are listed first also feel more relevant or important than in the past. For instance, a search on "travel" used to bring up some regional listings that probably weren't relevant to many people. Now, categories such as "Air Travel" and "Travel Tips" are right at the top.
Yahoo, cagey as always, won't explain what has been done to produce this improvement. My assumption is that as part of the short-name creation process, editors also flagged important categories in some way, so they'd rank better. I also suspect that the clickthrough measurements that Yahoo conducts are also being implemented to help improve the results.
After the category matches, you may now get up to 20 web site matches. They come under the "Web Site Matches" heading. These are web sites that have been reviewed by Yahoo editors and added to one of its categories. In contrast to category links, clicking on a web site link takes you directly to the web site, rather than to a list of web sites in Yahoo about a particular topic.
Yahoo has also introduced numbering of the web site matches. This aids those who click on a site, then want to return to the results and pick up where they left off to review new sites.
"In our usability tests, the numbers definitely gave people some context as to where they were," Gatz said.
As with categories, the Web Sites listed feel much more relevant than in the past. Again, Yahoo won't say exactly what's being done. My assumption is that, as with categories, they are making more use of clickthrough data.
As a basis for this assumption, try a search for "american flag." Why are the sites shown coming up first?
The first site is about flag football, which isn't at all relevant. However, this underscores that having the words in your title and site description are still very important to Yahoo.
When entering the site's category, we also see that it is at the top of the list. That means it is probably attracting decent clickthrough, especially in that it is also one of the best sites available in this category. The high clickthrough rating, in combination to having the relevant search terms, probably helps land it high in the search results page.
The second site is "Betsy Ross and the American Flag." Reviewing its category shows that another site is listed as "cool." So, how come that site isn't doing better? Again, the words are important here. That cool site lacks the words "American flag" in its title.
The third site, "American Flag Store," has the search terms in its title and shows up in the Most Popular area of its category. That site also is making use of the Sponsored Sites option, within its category. It could be -- and this is really speculative -- that the clickthrough from even the Sponsored Sites area is taken into account, when calculating Most Popular scores. If so, that would be one good reason to consider making use of the program.
As before, it remains important that your site make use of the key terms you want to be found for at Yahoo, in the description and in the title, if possible. It's also important that the title inspire people to clickthrough (as much as possible), assuming that you believe, as I do, that clickthrough is being used as part of Yahoo's ranking scheme. In addition, you want to ensure that your site has the best content you can offer, so people will stay there, as time on site is sometimes used in clickthrough relevancy.
Should you immediately go clicking on your site? I wouldn't recommend this in a massive way. Such actions are likely to be detected and compensated for. However, it probably wouldn't hurt to click on your own site from time to time, either in the search results or when listed in the category results.
Below each web site match is a "More sites about" link. If you click on that link, you'll be taken to category where the web site "lives" within Yahoo. That category will also list similar sites. For instance, searching for "train travel" brings up a web site called "How to Travel Europe by Train." Selecting the "More sites about" link then gives you a list of sites compiled by Yahoo editors about European train travel.
Finally, by using the "Next" button at the bottom of the results page, you'll eventually move past Web Site Matches to the "Web Page Matches" area, as noted by that heading. These are results that come from Google, which is why the Google logo appears at the top of the results page. Selecting links from here takes you directly to matching pages that Google has found from crawling the entire web.
As a reminder, Google results are provided for those times when Yahoo's editors themselves have not categorized anything that seem to match your search topic. For instance, search for "feeding hedgehogs" at Yahoo, and Google results come up immediately. That's because Yahoo's editors themselves haven't reviewed any sites on this topic -- or more appropriately, haven't written a description about any sites that use those words. By "falling through" to Google, Yahoo users can still receive some relevant results.
Of course, if you compare the same search at Google itself, you'll notice that at Google, more matches are found. The results are also slightly different. This is because Yahoo does not dig as deeply into Google's results, which saves Yahoo money. Because of this, if you are trying to do serious research on obscure topics from across the entire web, you're better off going directly to Google itself.
Under each Web Page Match, you'll see a "More Results From" link. Unlike the "More sites about" link described above, this link brings back other pages from the same web site as the page that is listed. In other words, you won't get a list of web sites on a particular topic. Instead, you'll simply be shown a list of web pages from a particular web site, which match your search terms.
Another significant change at Yahoo was the increase in its "Yahoo Express" submission price from $199 to $299.
The program is required for anyone submitting to one of Yahoo's commercial categories. It is the first price change for the service, since it launched back in February 1999, with the exception of the June 2000 increase to $599 for adult site submissions.
The move follows LookSmart's lead in August, when LookSmart raised its "Express Submit" price from $199 to $299. Yahoo declined to say whether LookSmart's increase had made it more comfortable taking its own price up. Instead, the increase was due to high demand and the need to keep quality up, Yahoo said.
"There's really been a strong demand for that product," Gatz said. "People still see the value in such a product. They see it as a very good one-time investment."
It's hard to see why a 50 percent price increase is necessary to keep the service level up, but it's much easier to understand that Yahoo no doubt saw the listings it provides as undervalued, especially coming after the LookSmart move. And, even at the higher price, getting listed in Yahoo remains one of the bargains in terms of gaining web traffic.
Getting non-commercial content listed within non-commercial categories remains free at Yahoo. To help site owners (along with users) understand what's commercial, Yahoo's commercial categories now clearly say "Yahoo Commercial Directory" in the yellow reverse bar at the top of the page.
For example, compare the commercial "Movie Memorabilia" category to the non-commercial movie humor category:
If you're submitting, be aware that Yahoo no longer requires that you first find a category. Instead, you can leave it to the editors to pick one for you. However, it is highly recommended that you still make your own category selection.
Finally, Yahoo is continuing to use its "Most Popular" listings, in some categories. One reader wrote last month asking why these disappeared. Gatz said that if you've seen them disappear from some categories, this is probably a case where Yahoo's editors have decided that the category didn't require a Most Popular list.
How Yahoo Works
Very detailed guide to the process of submitting successfully to Yahoo, available to Search Engine Watch members. Explains what Most Popular sites are, reasons to carefully choose a category and tips on how to get non-commercial content listed.
Free Search Engine Submission Is Still Alive!
Both Yahoo and LookSmart have recently increased their submission prices, while AltaVista not to long ago followed Inktomi's lead in offering a paid inclusion program. FAST is expected to bring out its own program shortly. Meanwhile, paid listings have gained ground at every major search engine. Is it any wonder that people are feeling that free search engine submission is dead?
It's not. Despite the multitude of paid participation programs being offered, it is indeed possible for web sites to get listed for free. However, it's also necessary for web site owners to recognize that by spending money, the listing process will be easier for them. The article below looks at the issue in more depth, with comments from businesses feeling pressed for cash.
Free Search Engine Submission Is Still Alive!
The Search Engine Report, Oct. 2, 2001
Google May Get Personal
With last month's acquisition of Outride, Google may be poising itself to go forward into an area of search refinement that no major player has gone successfully before: personalized search results. With personalized results, a person would get back a list of results that takes into account some of their demographics. For example, if you had registered yourself as a man, you might see a different set of results when searching for flowers than a woman might see. A look at the concept, and Outride may or may not bring it to Google, can be found below.
Google May Get Personal
The Search Engine Update, Oct. 2, 2001
Google's New Look & New File Types
Google's home page, known for its elegant simplicity, has gotten a little more complex. New "tabs" were introduced yesterday, to give users easier access to the search engine's features.
Don't worry. Purists are unlikely to be upset by the changes. The tabs are simple, highlighted links sit above the search box.
The tabs allow users to more easily reach Google's other search databases: images, newsgroups and results from the Open Directory's human-compiled listings, in addition to Google's own crawler-built web page index.
"We created this tabbed interface look because we thought it was an easy way to give users access to the information we offer," said Google spokesperson Eileen Rodriguez.
The tabs also appear on the Google results page, letting users easily switch between search indexes, as desired. For example, if you did a regular search for "madonna," on the results page you could select the "Images" tab to switch from matching web pages about the singer to matching images.
Ironically, AltaVista -- which Google is oft-seen as usurping for those interested in "pure" search -- also went in for tabs in 1998, when the service grew more complex than just pure web search. After numerous redesigns, AltaVista finally dropped the tabs concept last May.
Of course, AltaVista did a lot more than add tabs, as the service grew. Its foray into being a portal, complete with a massive amount of new links and content added to the home page, is a major reason why even AltaVista admits it lost pure search users.
In contrast, Google proudly says that the latest changes actually reduced what few words were already on the home page by 30 percent. The company also says that the tabs have been found effective, in user testing.
"Using the tabs has increased usage in those areas and made it easier to access the information," Rodriguez said. In particular, she said there were significant increases in the take up of image and directory search.
Google said it evaluated several design options, including integrating matches from all its different databases into one page. However, the tabbed option was seen as most effective by users.
Google is also planning to go beyond indexing only web pages and PDF files, in the near future. Google won't say exactly which new document types are to be added, but it's fair to say that Microsoft Office formats, such as .doc and .xls, will probably be among them.
It is because so many people save important information in these formats that Google is likely to support them. That is the reason Google cited when it added support for Adobe Acrobat PDF files, back in February.
A relatively new power command also lets you narrow your search to find documents in particular formats. The command is filetype:, and you follow it with the extension you want to search for. For instance:
california power crisis filetype:pdf
brings back PDF files that contain the words "california power crisis." In contrast:
california power crisis filetype:asp
brings back Microsoft Active Server Pages (ASP) files, while
california power crisis filetype:html
brings back ordinary HTML files that end in .html, that contain the words. It will not bring back HTML files the end in .htm, however. Technically, Google considers those to be a different file type, simply because the ending is different.
By the way, you can also start your query with the filetype: command, should you prefer.
Keeping Tabs On AltaVista
The Search Engine Report, Feb. 3, 1998
When AltaVista expanded, it decided -- as Google is deciding now -- that tabs above the search box seemed best.
Fourth Time Lucky For AltaVista?
The Search Engine Report, June 4, 2001
And then the tabs were gone, as AltaVista struggles back to define itself as more a streamlined search service.
Google Does PDF & Other Changes
The Search Engine Report, Feb. 6, 2001
Discusses Google's implementation of PDF support.
Logitech and Google: The One-Click Search Engine
SiliconValley.internet.com, Aug. 21, 2001
Logitech iTouch keyboard users are to get a key dedicated to Google.
Why They're Agog over Google
BusinessWeek, Sept. 24, 2001
A look at Google's growth in traffic and business prospects. FYI, the latest from Google is that 3/4 of its income comes from advertising, not half.
Excite@Home Goes Bankrupt; AltaVista Cuts Staff
Two of the web's oldest search engines had more bad financial news, with Excite@Home filing for bankruptcy last Friday and AltaVista also announcing a new round of layoffs last month.
Excite@Home operates the Excite.com portal, which uses its own in-house search technology to create listings. Launched in late 1995, Excite is currently the longest-running crawler-based search engine.
The company is looking to sell the Excite portal to a buyer, but prospects of both that happening and the portal surviving in any current form are unlikely. Both Go.com and NBCi sought to offload their portals earlier this year, without success. Neither did anyone step forward to take over the in-house search technology both players had developed.
Today, from a search perspective, both sites now simply serve up paid listings -- Go uses GoTo, while NBCi uses paid listings-heavy Dogpile.
Paid listings also seems the likely future for Excite. The company currently has a paid listings partnership with FindWhat. That makes it reasonable to expect Excite to begin using FindWhat's paid listings in the near term, if it decides to shutter its in-house search operations.
Such a move would be a short term gain for FindWhat. It would immediately have access to even more of Excite's traffic, but that traffic is likely to drop, as the portal's development is abandoned.
I didn't get a comment back from Excite@Home in time for this article on any plans, but I'll keep you updated, as I hear more.
Over at AltaVista, the company cut 30 percent of its staff and named a new chief executive officer, James Barnett, formerly of president of MyFamily.com. Most of the cuts apparently came from the shopping search side of the company. After Excite, AltaVista is the web's second-longest running major crawler. It launched in December 1995.
Excite@Home files for bankruptcy
News.com, Oct. 1, 2001
More details about the filing. The broadband side of things has been sold to AT&T, which has no interest in the portal side.
Excite@Home Pulls The Plug
SiliconValley.internet.com, Sept. 29, 2001
A closer look at the broadband deal.
Excite May Be Out; FAST Has Further Layoffs
The Search Engine Report, Sept. 4, 2001
Short, past article from me on problems at Excite, plus a recap of how other, older major search engines have fallen.
AltaVista names new CEO, cuts staff
News.com, Sept. 17, 2001
So, AltaVista's rankings have dropped because it "recently" cut its affiliate program, as the new CEO suggests in this article. Wrong excuse. The Yahoo deal did apparently end last month, but that wouldn't have impacted the July ratings mentioned in this article. A more wide ranging affiliate was discontinued back in February,. It paid a few cents per search, for people who sent AltaVista traffic. Of course, Google ended its similar program at the same time. Google's traffic has grown, not plunged, since then. So, why has AltaVista seen such a drop?
In addition, AltaVista's traffic has been dropping since June 2000, when it had almost a 20 percent share of the web audience, according to Jupiter Media Metrix. In contrast, Google had a 5 percent audience share. In July 2001, the situation was almost swapped. Google enjoyed a 16 percent audience share, while AltaVista struggled with a 7.6 percent share.
Don't blame the affiliates. The likely answer here is that there's a sizable "pure" search audience on the web that will give its dedication to one major search engine. AltaVista had 'em and lost 'em to Google.
Jupiter Media Metrix Ratings
See how AltaVista and Google ratings compare against each other for the past several months, via this page. Later in October, I'll also be updating it with June, July and August 2001 information. Search Engine Watch members can also use the Past Media Metrix Search Engine Ratings link at the bottom of the page, to go back even further.
Ad Market Woes Push About.com into E-commerce Waters
AtNewYork, Sept. 26, 2001
Facing its own financial woes, About.com closes about half its 750 web sites.
Terra Lycos reiterates lowered outlook
Reuters, Sept. 26, 2001
Don't expect a profit from Terra Lycos for at least a year, due to the downturn in advertising.
Surplus Of Search Engine Marketing Reports
It's been a busy few weeks for reports about search engine marketing issues, with three different publications having been released recently. They cover the issues of selecting a search engine marketing firm, which search engine marketing strategies seem to work and a review of how the web sites of Fortune 100 companies rate in terms of search engine friendliness. The first URL below is a long review covering each of the reports. Direct links to the reports themselves are also shown below.
Surplus Of Search Engine Marketing Reports
The Search Engine Update, Oct. 2, 2001
Buyers' Guide to Search Engine Optimization & Positioning Firms
MarketingSherpa, Sept. 2001
This is the report that covers selecting an SEO firm.
Search Engine Optimization Strategies: A Marketer's Perspective
CyberAtlas, August 2001
This is the report written by me and Search Engine Watch associate editor Chris Sherman on search engine marketing strategies.
How visible is the Fortune 100 to Web Searchers?
This is the report about Fortune 100 companies.
Here are some recent articles that may be of interest, from Search Engine Watch's daily SearchDay newsletter:
Search Engine Strategies: Buying Your Way In
SearchDay, Sept. 27, 2001
If you think you can't buy your way into search engine results, think again -- the number of new fee based programs is at an all time high.
Writing for Search Engines
SearchDay, Sept. 26, 2001
There's a fine art to developing content that maintains a good balance between ranking well in the search engines and still appeals to the intended audience.
Leveraging Your Content
SearchDay, Sept. 25, 2001
Though they're 'cool,' advanced design techniques such as frames, Flash and dynamically generated Web sites pose challenges for search engines.
Special Search Tools & Products Issue
SearchDay, Sept. 19, 2001
In this issue of SearchDay, guest writer Avi Rappoport provides an update on developments in the world of search products.
Search Engine Optimization Strategies
SearchDay, Sept. 18, 2001
Experts describe web page design solutions that can help boost visibility in search engine results.
On the archive page below, you'll find more articles like those above, plus have the ability to sign-up for the free newsletter.
Search Engine Resources
The Mediwarp health search engine provides easy access to more than 500.000 health and medical related web pages, complemented by a huge directory of preselected web sites.
Simple and fast, Daypop aims to let you search against over 4,000 newspaper and magazine web sites, as well as web logs. Like Google, pages are also cached, so you can view them in case the original page no longer exists or can't be reached. You can limit your search to a variety of date ranges. Well worth a visit.
Search Engine Articles
To Google or to GoTo?
BusinessWeek, Sept. 28, 2001
Writer Heather Green pits Goto against Google and finds that both are good in different situations.
The Road to Link Popularity
ClickZ, Sept. 26, 2001
Tips on building relevant links to your web site.
Jeeves Solutions Unveils JeevesOne
SiliconValley.internet.com, Sept. 24, 2001
Want Ask Jeeves-style answers for your web site without having to hire the company's editorial staff? Now the software toolset for managing your own knowledgebase is available, for $100,000 per CPU installation.
Search Engines Often Ignore Sites Using Flash
MarketingSherpa, Sept. 20, 2001
Search engines don't read Flash. That means if you use Flash, they may not find navigation links in the Flash content that leads them other pages. Flash-heavy pages also probably lack enough HTML content that would allow them to get ranked well for particular terms. This article has more tips on what not to do.
An Interesting Overture & A Different Kind of Positioning
Iconocast, Sept. 20, 2001
This issue of Iconocast focuses on search engine optimization, from the GoTo name change and the company's financial prospects, to trends and a case study on boosting Iconocast in search engines.
Register.com And Applied Semantics Make "DomainSense"
LA.internet.com, Sept. 20, 2001
The folks at Applied Semantics, formerly known as Oingo, are applying their technology to help Register.com's customers come up with alternative domain name options.
Daypop Searches News and Weblogs Up to the Minute
About.com Web Search Guide, Sept. 17, 2001
Longer review of the Daypop news search service, listed above in the Search Engine Resources section.
My Reading List
Thanks this month to items spotted in....
Search Engine Guide Newsletter
How do I unsubscribe?
+ Follow the instructions at the very end of this email.
How do I subscribe?
+ The Search Engine Update is only available to paid members of the Search Engine Watch web site. If you are not a member and somehow are receiving a copy of the newsletter, learn how to become a member at: http://searchenginewatch.com/about/subscribe.html
How do I see past issues?
+ Follow the links at:
Is there an HTML version?
+ Yes, but not via email. View it online at:
How do I change my address?
+ Send a message to firstname.lastname@example.org
I need human help with my membership!
+ Send a message to email@example.com. DO NOT send messages regarding list management or membership issues to Danny Sullivan. He does not deal with these directly.
I have feedback about an article!
+ I'd love to hear it. Use the form at