Search Engines As Leeches, The Difference Between Paid & Free Listings & Keyword Price Rises

Jakob Nielsen’s just posted a
Search Engines as
Leeches on the Web
article that makes a good point, don’t be too search
engine dependent. However, he muddles his point by confusing the issue of paid
search advertising and free “organic” listings. A closer look at that, plus how
“super conversion trackers” and “brand idiots” are likely to keep pressure on
keyword prices.

As a reminder, the major search engines give you two main types of listings
when you do a search. There are the “organic” or “free” or “natural” listings
that they gather from crawling the web. They don’t charge for these listings
(though Yahoo’s paid inclusion program kind of
the water over there). These listings are like the editorial content you get at

Search engines also carry paid ads. Pay, and you can get listed for terms you
want without hoping that it just happens naturally.

Jakob says:

I worry that search engines are sucking out too much of the Web’s value,
acting as leeches on companies that create the very source materials the
search engines index.

That will resonate with many who have long voiced similar concerns that
search engines are making tons of money by gathering “content” from sites from
across the web to make their listings.

If suddenly every site on the web were to block Google from indexing them,
Google would have a crisis in short order. Its main “content” would have gone
away, and the ads alone aren’t going to keep attracting searchers.

Web site owners have not done that, however. That’s because by and large,
they’ve found that search engines drive more traffic to them than they cost in
terms of bandwidth of being indexed.

WebmasterWorld has become a classic case study of this. Google and other
search engines were
banned in
November along with “rogue” spiders, because somewhat similar to Jakob’s “leech”
metaphor, they were seen to have been sucking down more bandwidth than it was
worth supporting.

WebmasterWorld founder Brett Tabke was often
quoted saying he had
the best sleep in months after blocking the spiders. His sleep may have
improved, but what to do about the major spiders didn’t go away. By the end of
December, Brett had done a 180 degree turn and
let the major
spiders back in

Until now, WebmasterWorld’s been about the only major site I can think of
that has tried to block spiders. Craigslist was rumored to have done so, but
that wasn’t true.

I do believe concerns over spidering are growing, especially as we have more
spidering from both the major search engines and from rogues that are out there.
Back in October,

The lie of distribution–search engines return very little value to news/blog
sites yet hog bandwidth and increase server loads
from Tom Foremski was an
example of this.

As I commented on his blog, it’s fair to say that despite grumbles, that the
vast majority of site owners do not consider search engines leeches. If they
did, they would deleech themselves by blocking spiders. It’s not hard to do. A
simple change to the robots.txt file will block all the major search spiders.
But no on does this, because they want the traffic. Even Jakob’s
own file isn’t blocking Google and

But back to Jakob’s point, it turns out he really isn’t talking about the
“source material” being leeched but instead about the high cost of advertising.
Again to his opening statement, with the key part in bold.

I worry that search engines are sucking out too much of the Web’s value,
acting as leeches on companies that create the very source materials the
search engines index

And the evidence of this?

Paid search confiscates too much of a website’s value.

What? Paid search “confiscates” a site’s value? Since when did search engines
suddenly show up at a web site and demand the owner sign-up for advertising?
We’ve long had rumors that a site that doesn’t advertise might find themselves
banned with various major search engines. We’ve even had
of “monetization targeting” where site owners have found that doing an ad or
paid inclusion buy might clear up a spam banning problem. But by and large,
there are plenty of web sites that spend nil with search engines on advertising
and get plenty of traffic.

In fact, the exhausting, annoying, tiring, boring, you name it regular
updates to Google generate plenty of forum fodder that show people aren’t
spending and getting traffic from search engines. If they weren’t, they wouldn’t
be freaking out any time Google undergoes a major algorithm change that sends
rankings dancing and for some, traffic plunging. They wouldn’t worry, because
they’d have had both a balance of paid and natural search listings that helped
them ride out the rough times if there was an issue on the natural side.

Instead, the
October 2005 “Jagger” update
showed plenty of site owners are still
dependent on getting traffic from search engines for free. The
Nov. 2003
Google Florida update
should have taught many not to be free listing
dependent, but clearly they remain this way. And the lessons not to be dependent
were in place even before this.

So overall, the issue doesn’t involve free listings. Jakob’s really concerned
about the rising cost in search advertising. Over time, as he’s worked with
client sites, they’ve been able to pay more by pushing up conversion rates. But
at some point, others catch up and the margins of what his clients can pay is
reached. He says:

If your search bid stays the same, your ad will sink off the page as more
and more competing sites improve their design enough to afford higher bids.
Our site therefore has no choice but to increase its own bid to $7.99 per
click if it wants to stay in business.

This simply isn’t true unless Jakob’s clients are making the mistake of
depending solely upon paid search ads to gain customers. If that’s the case,
yep, you should be looking to diversify. More on this in a moment.

Jakob also says (and the bolding is his):

This is great news for search engines: they can double their income by
doing nothing
. Just sit and wait for all other websites to improve — then
skim off the increased earnings.

In other words, the search engines get to make more by doing nothing because
the advertisers are learning they can afford to pay more. And they sound pretty
evil. But rewrite it this way:

This is great news for the Super Bowl: the cost of buying a commercial
keeps going up even though they do nothing. Just wait for advertisers to be
willing to pay more — then skim off the increased earnings.

Honestly, perspective like the above is very much in order. Consider:

  • Search advertising has long been undervalued. People still pay less than a
    dollar for some paid search ads and they obtain clients that will be with them
    for life. But few advertisers are calculating the lifetime value of search.
    Jakob doesn’t appear to be consider this. The examples he gives are to the
    purchases made directly from a click on an ad. Do his B2B customers come back
    to the site directly the next time and buy? If so, the only reason they did so
    was because they found the site through search in the first place. If you
    don’t factor that lifetime value, then you fail to understand how much you
    really can afford to pay.
  • Advertisers are getting smarter. Those who better measure conversion know
    they can pay more and they are. The search engines are to blame for this?
    They’re simply seeing that their undervalued advertising medium is finally
    growing to what it really is worth. The real person to be upset with is that
    other advertiser. And the solution is to get smarter.
  • It’s an open marketplace. Is Apple is a leech on consumers for
    overcharging for MP3 players that you can buy from others that work as well if
    not better but lack the Apple logo? The search engines aren’t forcing people
    to buy ads. If advertisers can’t afford to continue paying high prices, they
    won’t. And when they don’t, the prices will fall. The exceptions are if the
    search engine conspire in some way to force purchases (say they really do link
    buying ads to getting other types of listings) or when they set artificially
    high minimum bids (Google tried doing this but had to drop many because people
    weren’t willing to pay).

So rather than “despite search engines, websites can make money,” as Jakob
says, the reality remains that because of search engines, plenty of web sites
are making money without spending a dime, pence, euro, yen or whatever on search

I completely agree that anyone running a web site should heed Jakob’s “search
engine liberation” advice of alternative ways to promote a web site, such as
considering RSS, email newsletters and so on. But this isn’t suddenly new
advice. Any long-time internet marketer would tell you not to depend just on
search engines. Thinking “beyond search engines” has been the core of my
basic tips
since I put them up

in 1997:

Search engines are a primary way people look for web sites, but they are
not the only way. People also find sites through word-of-mouth, traditional
advertising, the traditional media, newsgroup postings, web directories and
links from other sites. Many times, these alternative forms are far more
effective draws than are search engines. The audience you want may be visiting
a site that you can partner with or reading a magazine that you’ve never
informed of your site. Do the simple things to best make your site relevant to
search engines, then concentrate on the other areas.

It’s all a matter of balance. Don’t obsess over search engine listings, but
don’t ignore them, either. Do a variety of online marketing activities — and do
a variety of offline ones, as well. Search — both paid and free — is a
component of any campaign. But it isn’t something you should depend on, any more
than you should depend on all television advertising, all print ads, all RSS ads
or a strategy of no advertising at all. If you are not diversified, you’ll have
a weakness that might hit when you least expect it.

To conclude, no one should put all their eggs into any basket, search or
otherwise. It’s absolutely true that search engines are not the end all be all
and that sites can thrive and survive without them. But many sites also can
thrive and survive better by incorporating them into a diversified publicity

Search engines definitely can do more to help those with support on the
organic side of things, which is especially needed since webmasters do indeed
provide the content that the search engines depend on. The good news is that
last year,
we saw
more changes and developments to give webmasters new tools than ever

Finally, ad prices will likely continue to rise, and different advertisers
will react in various ways. John Battelle recently
pointed at a blog

that FTD might be nearing all it can afford to spend. But just
today, we
on a survey showing four out of five advertisers saying they can
still afford to pay more, though the question of whether a plateau is being
reached is raised. Then the latest Fathom
report on
keyword prices saw a continuing “downward spiral,” as MediaPost put it. I
haven’t looked closely at the latest numbers, but the sample is so small (500
terms) that I’m generally wary on depending on it as a foolproof predictor.

From my part, I see two main issues with keyword prices going forward: Super
Conversion Trackers & Brand Idiots.

Super Conversion Trackers are those who will indeed track a lifetime
value of someone who comes to them from search. They’ll understand that the
initial purchase may lead to more and more purchases over time and feel
comfortable paying multiples above competitors to gain a lead. That will push
some out of the bidding. See
Most Conversions
Happen Offline; You Need To Measure These!
for some further thoughts on

Brand Idiots are what some marketers think derisively of others who
jump into bidding without linking it to a direct ROI target. They can screw up
bidding on what seems to be “logical” or “fair” amount that most in the
marketplace may assume. But brand idiots are part of that marketplace, and you
can expect to see more of them.

Automakers Buy Up 80% Of
Ad Space On Car Sites For 2006
from AdAge is a good example of this. It
explains how automakers are going more and more online to extend their brands.
Edmunds, a car research site, expects to take those brand dollars and buy more
search as well as display ads to fuel that desire. That’s big brand money that’s
going to be fueling those buys and putting pressure on others trying to compete.

Big Guys Crowd
Out Little Guys in SEM Arena; Some Branding Focused Advertisers Willing to Spend
“Whatever” It Takes
C’mon In Brand
Owners, The Search Water’s Fine
has more on these type of moves.

Stuck in a bidding war?
How To Get
Out Of Bid Wars A Winner?
over at our Search Engine Watch Forums may have
some helpful advice if you’re already tracking and improving conversions as much
as possible and getting some brand idiot money is not an option.

Want to comment or discuss? Visit our SEW Forums thread,

Search Engine Leeches, Dependency & Losing Perspective

Related reading

The middle ground for single keyword ad groups (SKAGs)
Five ways blockchain will impact search marketing
New visual search innovations tap human emotions and biological buying triggers