THE SEARCH ENGINE REPORT
5/5/97 - Number 6
About The Report
The Search Engine Report is the email companion to "A Webmaster's Guide to Search Engines," http://calafia.com/webmasters/. It keeps you informed of changes to the site and general search engine news useful to web developers. Feel free to pass on this newsletter to others.
Resellers Wanted for "VisitorDirect" -
Directs Targeted-Visitors To Client Web Sites.
NEW! WebSpecialist's VisitorDirect is one of the most cost-effective ways
to drive Quality, Targeted-Visitors to a Web Site.
WebSpecialist, LLC is currently offering wholesale reseller alliances with
Web Promotion Companies, Internet Service Providers, Webmasters and Web
Designers, Advertising Agencies, and Internet Marketing Firms.(Retail
customers will be routed to resellers)
VisitorDirect is a unique Web promotional tool that directs targeted visitors
to client Web sites by negotiating highly relevant search engine results
for client websites using keywords specified by the client.
WebSpecialist's proprietary coding technique is the only one of its kind
and is not to be confused with conventional submission techniques which
employ the use of meta tags, keyword titles, or unacceptable spamming
techniques such as keyword loading, or multiple search engine submissions.
Would you like more information about becoming a VisitorDirect dealer?
There's been some minor housekeeping throughout the site. The Search Engine Resources page has been broken up into individual topics, such as "Search Engine Technology" and "Search Engine Revenue." A new topic is "URL Checkers," which summarizes URL checking services such as PositionAgent. I've also added a page with Power Searching tips for the major search engines. Links to all of these can be found via the Resources page:
The Strategic Alliances page has also been updated to reflect the new Netscape line-up, as well as WebTV and a recent NetGuide review.
Haven't subscribed to the web site yet? Consider it, because in the subscriber area, you can now find a document on how to detect when a search engine comes to visit your site. There is also a Search Engine EKG, which shows graphically the activity of different search engines over time. See which ones have a normal rhythm vs. those that are nearly "flat line." Thanks to Marketwave for providing a copy of HitList Pro 3, which made analyzing the search engine activity a breeze. They ca n be found at http://www.marketwave.com/.
Subscribers also receive a twice-monthly newsletter, an offline edition to the guide, other goodies and that warm, fuzzy feeling for helping to ensure "A Webmaster's Guide to Search Engines" continues as a quality resource. Subscription information is at:
Search Engine News
Netscape Net Search Changes
The Netscape Net Search page is most important place a search engine or web guide can be listed. This is the page that appears whenever someone pushes Netscape's Net Search button, and many do. In fact, I've often heard people saying things like "Do a Netscape search."
Of course, there's no such thing as a Netscape search. Searches done via this page are actually sent to one of several search engines. But this underscores the point that when people are looking for something, many go to this page. Any search engine listed on the page is going to get traffic. That's why they pay so much to be listed here.
On May 6, the Netscape Net Search page is significantly changing for the first time in over a year. There have been some smaller changes, but this is a major revamp of how things work. These changes are important to the web marketer, because the changes will influence which engines people use -- and thus which engines you need to be most concerned about.
Continuing as top-listed search services are Excite, Infoseek, Lycos and Yahoo. The are called "premier" providers. Specifically how much they are paying for this privilege hasn't been released. Last time, it was $5 million per year. Rumors are that its roughly the same amount for the next coming year, though payment is supposed to be tied to the actual amount of traffic each engine receives.
That traffic could significantly change. In the past, there were five top choices. Each time the page loaded, a different one of the top five was highlighted. Chances are, many people used whatever appeared, rather than clicking on one of the non-highlighted choices.
Now, there are only four top choices and a fifth "Customize" option. Those selecting this option cause a box to appear, where they can replace the Customize tab with a fifth search engine choice, either AOL NetFind, HotBot, LookSmart, Search.com or WebCrawler. These are called "marquee" providers. Further, a second option allows them to choose whether to rotate the top choices or set one to be the default.
What will people do? Who knows, yet. I tend to think that people change defaults as little as possible, which would mean12 that the page will continue to rotate between four top choices. But I can also see some people clicking on the Customize tab causing one of the marquee providers to become the default setting for every visit through confusion.
How confusing can two options be? Well, I love my mother, but you should see her operate a computer. Thinking like her, the first option asks her to pick from one of five marquee providers. So she picks one. Then the second option gives her a list of all the premier AND marquee providers, plus an option to rotate selections. It's all explained, but Mom - bless her heart - doesn't tend to read these things. No, she gets confused and sets the second option to be whatever she chose in the first option. Suddenly, one of the "second-tier" marquee providers becomes her default choice by accident.
So who wins and loses with these changes? It's status quo for the premier providers. They stay where they were. HotBot and LookSmart gain from the changes, since they rocket up to the top of the page as marquee providers, rather than being listed below the fold with only text links, as they were.
Search.Com and AOL NetFind move up, event though they aren't really search engines themselves. Search.Com taps into Alta Vista listings, by default, while AOL NetFind is a branded version of Excite.
WebCrawler gets a demotion. While it's listed as a marquee providers, previously it had been one of the top five picks. Finally, Open Text goes the way of Magellan, now no longer listed at all on the page.
While you're by the Netscape Net Search page, also be sure to drop in and see the new Netscape Guide page, which is being produced by Yahoo.
Netscape Net Search page
New Netscape Net Search Preview Page
Netscape Guide By Yahoo
Infoseek Gets Specific On Spamming
Infoseek's instant indexing feature is apparently proving popular with search engine spammers, causing Infoseek to add some additional warnings. Now it specifies that by submitting pages via the Add URL page, the submitter agrees to Infoseek's "Guidelines for Adding a Web Page." This basically means that you agree not to spam the search engine.
Infoseek has also created a "What is Spamming" page, with specific warnings against:
+ Overuse or repetition of keywords.
+ Use of meta refresh faster than the human eye can see
+ Use of colored text on same-color background
+ Use of keywords that do not relate to the content of the site
+ Duplication of pages with different URLs
Infoseek notes that it is possible some non-spammed pages may accidentally get rejected. I saw this first hand, recently.
The submitted page had a white background, with a blue table and white text within the table. Infoseek seems to have read the body background color and looked for any matching font color, regardless of the fact that the blue table made the font visible. An example of the actual code is in the subscription-only section of the web site.
Moral of the story? Check on the pages you submit, to ensure that there aren't problems. By checking on this page, Infoseek warned me that it had been rejected because of the concern over color. The code was changed, and the page was accepted when resubmitted the next day.
Add URL page
Guidelines for Adding a Web Page
What is Spamming?
Check URL Status page
Excite Launches Channels
Excite launched its new "channels" look on April 21, where information is organized around topics such as "sports" and "games." Lycos, you may recall, unveiled a similar reorganization last month.
To be honest, the whole channels metaphor had left me sort of yawning. After all, these sites already had reviews organized by subjects. But my jaw literally dropped when I looked at what Excite had done. I knew they had tours, bulletins boards, chat and more -- but seeing it all brought together under a particular topic was an eye opener. I couldn't help but think Excite is beyond being just a search service -- they're close to being an online service.
Of course, that's just what Excite and the other search services going this route would like. Instead of being just transit points -- Internet airports, if you will -- they can be destinations. It's easier to target demographics for your advertisers with a channel metaphor. Futhermore, by keeping visitors within the site with internal content such as chat, you keep them seeing your ads.
The New York Times article below touches on some of these issues, although it makes a stretch in trying to call Alta Vista's LiveTopics a unique twist on channels. LiveTopics may be neat, but they have nothing to do with the type of changes Excite and Lycos have recently made.
How does this affect the marketer? If more and more people turn to browsing, rather than plain searching, then being featured in these channels could become more crucial.
If you go by Excite, be sure to visit the My Excite channel. My favorite section is the TV listings at the bottom of the page, which tells me all the shows I'm missing, now that I live in Britain. It's nicely done and customizable -- assuming it doesn't crash your browser, as it did mine.
Systems for Dealing With Information Overload
Search Engines turn on to channels
Push Is Coming - And So Are The Push Search Engines
Sick of hearing about push yet? Well, here's something else to think about. Should the integration of push into the Netscape and Microsoft browsers cause people to prefer pushed content, then they may be turn to services that index this content. For web developers, that opens a whole new can of worms. Exactly how will push content be indexed different than ordinary web pages? Are there traps to be avoided?
I'll be looking into this over the coming months. In the meantime, Excite is already poised to be a leader, having cut a deal with PointCast to index its new "Connections" channels.
Excite Inks Deal To Index PointCast Channels
Harder Life for Agents?
Word is that it suddenly got harder this month for offline agents to gather results from Excite, Lycos and Magellan. It could be due to general upgrades, but perhaps the services are trying to keep people from tapping into their results without actually visiting the web sites.
Forget Ads, Sell Keywords
None of the major search engines sells keywords linked to listings. For example, you can't buy "sex" and ensure your site comes first in the listings. However, you can buy that word so that your ad appears whenever it is entered.
Web Week has an interesting article where it is revealed that more than half of Infoseek's ad revenue is now from keyword-linked ads. Similarly, Alta Vista now charges more for a keyword-linked ad than for placement on the home page.
Scoreboards and Search Pages Are Hot Spots for Advertisers, Web Week http://www.webweek.com/current/markcomm/spots.html
Concern Over Robot-Inflated Page Views
There are lots of robots that visit web sites, and apparently concern is growing that they can inflate page views. I would tend to think that at most, robots might take viewings up an extra 10% -- which is probably easily offset by the number of viewings not registered due to caching. Inter@active Week has a nice piece on the problems robots can cause. Subscribers who want to know how to detect these visitors should see the Projects In Progress section of the subscriber-only area, mentioned above.
Agents: Artificial Inflation Of The Web Kind, Inter@ctive Week
Search Engines Partner With TV
Now it may be television ratings that determine which search engine is most important to the web marketer. On April 8, Infoseek and NBC announced a partnership making Infoseek the preferred search engine for the new web sites that NBC will be offering to local affiliates
This follows on the partnership announced last month between AOL NetFind, a branded-version of the Excite search engine, and ABC News. NetFind is supposed to be the preferred search engine for the ABC News web site, which launched on May 1.
Will CBS and Fox follow suit and find a search engine partner? Don't touch that dial!
NBC/Infoseek Partnership Press Release
Search Engine Stocks
To date, only Yahoo has shown a profit among the major search services. The San Francisco Chronicle has a nice story on search service stock prices. MSNBC also provides a good summary of changes underway at the different services to make them more attractive. And finally, the Washington Post has an excellent look at why that Yahoo profit isn't quite what it seems to be.
The Engines That Couldn't, SF Chronicle
Searching for success - or survival, MSNBC
You Can Find Lots on the Web, Except Maybe an Operating Profit
CEOs From Major Search Service Meet, Agree to Agree
What happens when the CEOs from Yahoo, Excite and Infoseek meet? No fisticuffs, just smiles all around. Future strategies and advertising campaigns were discussed, and the most interesting prediction was from Excite's CEO George Bell, who said he expects only two major search services to remain by 1999. He didn't predict which ones. A short summary of the discussion:
Search "Battle" a Peaceful One, Netday News
Search Engines Hit The Road
Online travel reservation systems and search engines are coming together. Read about the various partnerships in this article from C|NET News.Com:
Have search engine, will travel
Search Engine Notes
+ WebCrawler is crawling again, based on my log reviews. Submissions done in mid-March seem to be producing visits 3 to 4 weeks later.
+ HotBot is also crawling, pretty much on target with a visit 3 weeks after submission, as it says it will do.
+ Open Text has reopened its submissions page, and I'm waiting to gauge the crawler's response time.
+ Lycos is now detecting which country visitors are coming from and automatically redirecting them to a regional version.
+ Pages have occasionally dropped out of Alta Vista in April, only to reappear again. If your site seems to have suddenly disappeared, check back in a day before panicking and see if it has reappeared
+ A Lynx user reports that Lycos is no-longer Lynx friendly, and apparently not concerned about it, to her dismay. I have no idea if other search engines are more friendly to the text-only browser.
+ Point is apparently committed to going through all its submissions but is so backlogged that it has only reached those from September 1996.
You may also wish to check out some articles that I worked on for NetGuide Magazine that are now online:
Supercharge Your Web Searches
Reviews and details of each of the major search engines
What Makes Search Engines Tick
An overview of the challenge of indexing the web and what goes on under the hood of the various search engines.
I recently discovered that a company has appropriated material from my web site, which is being used as its own. The company is Net-Promote, an online promotions firm. All requests to remove the plagiarized material have been refused. If you support the guide, I would ask that you not use this company's services and pass the word to others to do the same.
To subscribe to this mailing list, send email to [email protected] with the following as your message:
Alternatively, you may use the form at http://calafia.com/webmasters/list.htm
To unsubscribe, send email to [email protected] with the following as your message:
The contents of this page can be found at
This newsletter is Copyright (c) Danny Sullivan, Calafia Consulting.
This Year's Premier Digital Marketing Event is #CZLSF
ClickZ Live San Francisco (Aug 11-14) will bring together the industry's leading online marketing practitioners to deliver 4 days of educational sessions and training workshops. From Data-Driven Marketing to Social, Mobile, Display, Search and Email, the comprehensive agenda will help you maximize your marketing efforts and ROI. Register today!