Rich Skrenta, the CEO of Topix, recently offered up some thoughts about Strategies for Beating Google. One thing he states clearly is that going head to head with Google is not the way to go. Too much reputation, lead time, and money.
In this light, it's interesting to think about Yahoo, and it's strength in providing destination sites. In this context, even though being #1 would obviously better, being a solid #2 may well be an OK place to be (my thoughts, not Rich's).
Rich also offers up the thought that going vertical may be the answer. It's a decades old strategy for dealing with markets that have a single dominant player. No one can be all things to all people. You need to find those soft spots in the behemoth and take advantage of those.
Here is a quote from Rich:
"You need to position your product to sub-segment the market and carve out a new niche. Or better, define an entirely new category. See Ries on how to launch a new brand into a market owned by a competitor. If it can be done in Ketchup or Shampoo, it can be done in search."
Lots of people are trying, and the forecasts for revenues from vertical search are very impressive, with a forecast of One Billion Dollars for 2009, we are talking about real money.
The next question you need to ask yourself then is, will this be divided among a hundred small companies? Or will a new large player emerge from all this potential revenue. One of the things that is trick about this opportunity is that many of the vertical search engine players that are emerging are highly vertical.
They perform custom crawls of custom data sets, including data not open to the web, apply custom algorithms to the data from their crawls, and then offer up a custom presentation of the data to their users. They are truly vertical. However, too vertical means "not scalable". This fits these start ups because they only need to get to a certain size to provide the management team and the investors the exit they are looking for.
That will make it tough to scale a collection of such search engines into a Google killer. On the other hand, for the same reasons, it is difficult for Google to pursue these types of opportunities. This type of business may not meet their margin expecations and scalability requirements. Of course Google does have it's Custom Search Engine initiative.
But this is based on adding a human editorial layer on top of the current Google index, which is a very cool thing, but it can't go as far as custom crawls, custom indexing, and custom presentation of the results. All in all, I agree with Rich, there is plenty of gold in them thar hills, we will just have to see how it all comes together.