Mining Company Illustrates Yahoo Limits

Mining Company Illustrates Yahoo Limits

From The Search Engine Report
Jan. 9, 1998

As a general rule, Yahoo lists web sites under only one or two categories. By limiting listings, Yahoo hopes to ensure that its users are not overwhelmed by seeing the same sites everywhere within the service.

Like all rules, this one gets broken. Sometimes Yahoo editors feel a particular section of a web site deserves its own listing. Sometimes they think that a site deserves to be in several categories.

The Mining Company has struggled unsuccessfully to get Yahoo to break its rule for the many sites that it operates. The situation illustrates problems that Yahoo, webmasters and searchers all face in the effort to classify the web.

The Mining Company launched with great fanfare in April 1997. It was widely heralded as a new navigation service, a competitor to Yahoo and other search services. Its own press releases touted the fact it would help people find what they were looking for better than search engines.

In reality, this is a bad description of the service. With a search engine, you'd typically search for a topic, then "exit" to one of the destinations listed. In contrast, the Mining Company is a collection of destinations. There are over 500 Mining Company sites, ranging from Mark Twain to Southern Cooking, each run by a different "guide." The sites are not just a collection of links. Most of them are full of information on their topics.

The guides behind each of the sites are attracted to publishing via the Mining Company for a variety of reasons. The service provides hosting, easy to use templates, chat areas and other functions, as well as handling ad sales. That's a relief for many people. They could run a site on their own, and some of them did so before the Mining Company existed. But by joining up with the Mining Company, they can concentrate on their content.

As you might expect, guides began submitting their Mining Company sites to Yahoo shortly after launch. They were told that Yahoo would not list them because their sites were really "subpage submissions." The main listing for the Mining Company would have to suffice for all the sites, Yahoo told them.

"For large, substantive, comprehensive domains like yours, we continually struggle with when to break out the individual parts and separately list them. We'll continue to think about that in your case, but for the foreseeable future we really believe that by providing a prominent, well-described listing for your main hub page, we strike the best balance between serving our users and managing our limited resources," the Mining Company was informed, by Srinija Srinivasan, who oversees the listing process.

The Mining Company is not the first site to encounter this problem. Many web publishers have sites that simply cannot be properly classified with two categories and 25 words. But the Mining Company is a useful for examining the impact of this problem on search engine users, because of the overall excellence of their sites.

It's fair to say that many of the Mining Company's sites are the best on the Internet for their particular topics, as witnessed by the many awards individual sites have won. In some cases, they may be the only comprehensive sites on particular topics. But the Mining Company's "hub" listing does not, and cannot, encompass them at all.

To see this in action, consider what the guide producing the Mining Company's Christianity site was told:

"The URL you submitted is one part of a larger site that already exists in Yahoo After reviewing both the submitted URL and the existing URL that encompasses it, we've determined that the existing listing is adequate for users to find your site."

The Mining Company is listed under two categories, and Yahoo has not provided it with a description, feeling the context of the categories summarizes the site well enough. Let's examine that hub listing:

Business and Economy: Companies: Internet Services: Search and Navigation
Mining Company

Computers and Internet: Internet: World Wide Web: Searching the Web: Web Directories
Mining Company

Now imagine someone comes to Yahoo and searches for "christianity." Since that term is not associated with the listing, the Mining Company site will not appear. Nor will even the categories be listed, as the word does not appear in them. That means that Yahoo visitors will have no chance of discovering the Mining Company's Christianity site.

Another example is the Mining Company's Personal Web Pages site, one that I know well. It provides advice on getting the most out of the free web space offered by various vendors. There aren't many sites like it. A search for "free web pages" in Yahoo won't find it. You discover places that offer free space, but no sites are listed with advice on the many things to be aware of when using free web space.

So aren't users missing out because of this policy, with the Mining Company specifically, and perhaps with other sites generally? Yahoo's Srinivasan responds:

"While I agree that for the user searching in Yahoo for "christianity," it's perhaps less than ideal that they won't see the Mining Company's Christianity site in our Christianity category, [but” that same user also won't see Excite's Christianity Channel or Infoseek's Christianity category.

We simply can't re-index these sites within our own index; it would be practically infeasible and the user experience would suffer. I don't think the user wants to go to Yahoo's basketball category, see links to Excite's basketball area, Infoseek's basketball area, the Mining Company's basketball area, etc., then go to Yahoo's philosophy area and see those same analogous links again...just as we wouldn't expect Excite or the Mining Company to point to every analogous Yahoo category from every one of their categories."

In part, Yahoo is considering the fact that there are literally thousands of sites out there that may wish to have sublistings. There's fear of a "slippery slope" if they become too liberal. "If we start, where do we stop," Srinivasan said.

To this, the Mining Company notes that there are plenty of sites that are listed beyond the main hub.

"There are many examples of other sites with multiple Yahoo category listings, including: ZDNet, The Rough Guides, Argus Clearinghouse, Citysearch, ESPN Sportzone, TV Guide, GeoCities, all the TV networks’ TV show sites, Excite’s various services, ISP-produced features and home pages and many, many others," wrote Jacob Levich, the Mining Company's Director of Guide Mentoring, in a letter asking Yahoo to reconsider its decision.

Srinivasan offers various explanations, not the least being that Yahoo constantly makes judgement calls. With the Mining Company, the consistent branding throughout the site gave it the nudge toward a central hub-only listing, while with GeoCities pages are radically different from each other, without GeoCities branding throughout (though this was before GeoCities recently implemented pop-up ads that appear when personal pages are accessed).

"The Mining Company is not an invisible ISP. They are a very visible support network," Srinivasan said. This is consistent with what she in response to Levich's letter: "The design and interface of these sites clearly indicates that they're each a part of a larger whole, with the consistent navigation frames and look and feel running throughout. We think this is analogous to many other cases where we choose not to break up the sites."

Srinivasan pointed out to me another prominent site where sub-sites are not listed, for exactly the same reason. She didn't want the company publicly named, but the example is similar with the Mining Company's situation.

Of course, many of the ZDNet sub-sites listed in Yahoo have common branding but still get in. MSN is similar. Its Car Talk site, Encarta site, CarPoint site and numerous others are listed. They do look different, though all feature a prominent drop-down box allowing navigation between MSN sites.

Then there's Yahoo itself. It has numerous sublinks to its own content, from Yahoo Classifieds, to Sports, to Lottery Results, each regional guide and more. Shouldn't there simply be a main hub listing for the service?

"It's unabashedly our guide," Srinivasan said, explaining that Yahoo would be dumb not to promote itself. But it's not just self-interest, she adds. "If we're going to bother to make a whole sports area, we had better think that it's the best sports area out there."

In the end, the situation points out the delicate balancing act that Yahoo's editors face when they consider how to list sites. They have some general rules, but unlike a library, there is no classification system set in stone. Instead, the consensus of the various Yahoo surfers acts as a living, ever-changing system.

"There are no perfect solutions to this (or at least if there are, we haven't figured them out yet!), there are just partial solutions that are practically feasible, competitively sound, and maximally beneficial to the user. All of these factors get weighed in, and we try to decide how to make the call," Srinivasan said.

For search engine users, that means that Yahoo should be one of several search tools used. By using several services, you may find some important sites that might otherwise be missed.

For web publishers, it's another illustration that depending on Yahoo for traffic is a dangerous strategy. You may not get in. If you do, you may not be classified as you think you should. Descriptions and categories are sometimes revised, causing sites to lose traffic. Smart sites learn there's "life after Yahoo" and embark on link building and other Internet publicity efforts.

For the Mining Company, and in particular for its guides, it's unfortunate that the decision has gone against them, at the moment. They aren't being judged on their content. Some sites were listed independently before joining the Mining Company. Now, as part of the organization, they don't qualify. With so many exceptions being made elsewhere, it's somewhat surprising Yahoo feels the Mining Company's sites should be excluded -- especially as they readily agree to their quality.

It's important to remember, however, that the guides have gained the efforts of an organization that can advertise itself and its sites in other ways. The Mining Company can, and has, gone well beyond Yahoo to build up its traffic.


Mining Company