Search's Long Tail

If you've somehow missed discussion of "The Long Tail," it's a reference to a landmark article of the same name by Wired editor Chris Anderson that ran last October. It covered how the media and entertainment industries will succeed not by pushing only mass market hits that are popular among many but by also mining the "long tail" of interest among a few in less-popular books, songs, movies and more.

Sure, maybe thousands want to buy a hit song. But add up all those who want to buy lesser-known titles, and they might generate as much or more revenue than the hits themselves. As a merchant, you want to tap into both the "head" of interest and the "long tail" that follows behind.

The tail makes much more sense when you see charts that illustrate it, such as the one below:

Those are the top 100 queries related to shoes on the Overture network, from an article I did last September about Overture shifting to a broad match system (Overture Shifting To Default Broad Match).

The point I was explaining in that article is that there are a large number of queries that happen far less often than the "leading" terms like "shoes" or "running shoes" at the head of the list. Most queries form the long tail that's illustrated behind the head. Tap into the tail, and you've got sizable traffic, as well as traffic that often is reported to convert better than less general terms.

In other words, search has a long tail too. Indeed, I've long heard references for years to the "search tail" or "query tail" I've illustrated above. I'm not sure where that phrase it originated and with whom, but tails aren't a new concept to the search world.

While it might not be new to search, it's certainly great to have the tail becoming more popularized in general. That's because it will further help those search marketers who mistakenly fixate on only the most popular terms to realize they need to consider the tail as well.

Over the years in my regular SES session "Intro To SEM," I've also referred to the search tail as getting the "onesies and twosies," the queries that might only happen once or twice in a month. Maybe they don't seem important because of the low volume individually, but tap into lots of onesies and twosies, and you can be doing well.

So if you're a search marketer hearing about The Long Tail for the first time, rejoice! Many of you have intuitively been mining it for ages. If you haven't, start! That means considering broad matching on both Google and Yahoo. For organic SEO, it means having lots of good content that will naturally tap into the tail of queries.

Meanwhile, some examples of how the search tail is becoming popularized and integrated as part of current Long Tail awareness for further reading:

  • The long tail of software. Millions of Markets of Dozens: Out this week from the blog of Joe Kraus, one of Excite's cofounders, this has a chart of the Excite search tail from his day. Kraus says 97 percent of the service's traffic came from the tail and that the "real reason" his company went out of business was because it didn't know how to make money from that tail.

    I'd beg to differ on that point. Excite gained search-targeted listings through a partnership with FindWhat in April 2001. Over six months later, Yahoo finally got the same through its partnership with Overture. Both partnerships let these companies earn off the search tail. Excite was ahead of the game yet died. Yahoo survived. Blame lack of tail targeting? No, something else was at work.

    I'd say Excite went bankrupt because among other things, it couldn't afford to continue competing in the expensive portal competition it was trying to fight. Maybe if it had gone after the tail even earlier, it would have survived (my The End For Search Engines? article explains why this was hard).

    But as noted, the fact that Yahoo pulled through doesn't mean that failing to target the tail can take such heavy blame. Also be aware that while Excite the company died, Excite the web site is still going. The traffic is much reduced, but it's earning revenue today for the company that runs search there, Infospace.
     
  • Google's Long Tail: From Chris Anderson's Long Tail blog, you can see an example of how Google in February was trying to tap into the new understanding and popularity of the Long Tail idea to illustrate to investors how it is a long-tail type of company. Google's always been this way, but the better understanding of what it does (and what Yahoo's Overture pioneered back in its GoTo days) is growing.
     
  • On the Trail of the Long Tail: Also from February, this post on the Yahoo Search Blog shows Yahoo directly saying the same thing -- we're a long-tail company. Sadly, they didn't provide any charts to help illustrate this more!
     
  • The Paid Inclusion Dinosaur article I wrote from June last year looks at why Yahoo has stuck by its guns to run the program -- doing so lets it tap into the giant tail of queries that marketers even using broad matching might never consider, especially giant when you understand that paid inclusion ties into the editorial results that are far more likely to be clicked on than ad listings. From the longer version of the article for SEW members:

    Right now, Overture and Google struggle with how to fill out the "tail" of search queries with ads. These are queries that happen only a few times per month but in aggregate represent a huge amount of unsold inventory.

    For example, advertisers focus on high frequency terms such as "shoes," which Overture reports had over 1,000,000 requests in April 2004. It sells for a top bid of around $0.55 and has 90 advertisers competing for it.

    In contrast, "winged track shoes" happened only 27 times last month. No one is bidding on the term. That's most likely because the low frequency doesn't make it seem worth the time. Yet someone searching for something so specific might convert better than the more generic "shoes" searcher.

    Paid inclusion is a perfect solution for unsold ad inventory like this. Simply tell the advertiser that you'll spider their site and let pages appear in the paid placement area for a flat, low-cost rate when there is space available. Advertisers can still target the important terms, but they gain easy visibility for other important ones. Yahoo reduces its unsold inventory. Searchers are spared confusion.

  • Search Engine Marketing Goes Mainstream: A report out of our SES Boston conference in March 2003, it covers how back then the "tail" of search was being discussed in both economic terms as well as a search marketing tactic.
     
  • Just Right: Targeting the Tail: Cached copy of a LookSmart presentation at SES Australia in March 2003 that says all the things you're hearing about today in terms of the Long Tail. It mentions the "keyword tail" and the "search tail" and concludes:
     
    • Top 50% of queries = 80% of volume
    • 80/20 rule does not regularly apply to search
    • Focusing on just the first 20% of keywords will invariably miss most of the value