June 23, 2008

Google Trends for Websites Adds to Comparison Sites Confusion

If you want to see how your site is doing compared to the competition, then the new Google Trends for Websites is yet another option to conduct your research. Following the lead of sites like Compete.com and Alexa, Google Trends for Websites allows you to see a graph of traffic to the sites you designate.

But Google Trends for Websites is only adding to the confusion caused by Compete and Alexa. I used all three to compare SearchEngineWatch.com, SearchEngineLand.com, and SEOmoz.org, three of the leading search blogs. I got three very different graphs. Check out these screenshots.

With all three, there are definite seasonal dips. But these graphs may speak more to the popularity of Google, Alexa and Compete than they do of the websites you may search. Alexa makes the sites look like they've seen traffic decline, and Compete makes the sites look like the traffic has increased, beginning with a big jump last June. Incidentally, almost any site related to internet marketing seems to have a big jump last June on Compete, which was discussed at search marketing conferences and made known to a wider audience.

Furthermore, Google Trends for Websites does not offer numerical values to give you a ballpark figure of how a site is performing. Alexa and Compete do. Still, Google Trends will likely become the most authoritative source for comparison data since Google has access to far more data than Compete or Alexa.

Posted by Nathania Johnson at 9:14 AM | Permalink | Comments (7)

December 18, 2006

Keyword Research Best Practices

Barry pointed out a great response to thread at Search Engine Watch forums by Paid Search Guru Ian McAnerin. A member had asked Forum visitors which industries they "would not touch with a 10 foot SEO pole?"

Ian answers led to some additional excellent discussion at SEW and a couple of gems in the SER comments.

Keyword research is a topic that is considered to be very basic by many in the SEO and Paid Search fields. This is likely due to the fact that it has been one of the few constants since the early days of SEO, when tools began to appear that were geared towards finding the right keywords. Since, many writing about the subject have indicated the same core needs: relevancy and popularity, including Danny from way back when, Kevin Lee, Shari Thurow, and most recently Christine Churchill.

Ian's post at Search Engine Watch makes three main points: First that some industries may be too difficult to venture into without specialized experience; secondly, you may not want to venture into some industries due to business concerns (he cites Realtors as being especially “difficult” when it comes to payment or buying in to the value); and lastly that your personal belief set may be in conflict with the particular industry, such as Hate or Porn sites, for example.

Ian comment raised some good follow up questions, and he defends his opinion that one should “cut their teeth” by targeting more localized terms. The whole topic leads well to a discussion of the core competency of keyword research. When venturing into a new space, it is likely that many SEO's are at a slight disadvantage due to being unfamiliar with terms. When deciding on whether to accept a project, it actually takes a fair amount of diligence on the part of any SEO; otherwise they may be simply saying “sure we'll get you ranked.” This could be an alarm signal.

Using geo-modified keywords as the target can also prove to be difficult if not properly done. In some cases, there may be a majority of searchers using the city or town before the more general term (i.e.: Timbuktu hotel) while in others, people may use it more often after the term. The fact is that without excellent and trusted keyword research, only trial and error will lead to the required log files that report the actual activity. This trial and error period can be greatly eased by having an unlimited paid search budget to run all keywords on broad match across all engines for at least 2 or three months. Unfortunately not everyone has the budget to do that. However, running these types of campaigns on a local basis may be somewhat helpful.

One comment that was very insightful at the SER blog was that “generally the most competitive websites have the highest cost in PPC advertising. Find the biggest spenders and you have the stiffest competition.” Although this is a generalization, it holds fairly true. When making a decision as to whether to venture into an industry for SEO, a quick check of the results pages for Paid Search listings can save a good amount of time for small SEO/SEM shops.

Posted by Chris Boggs at 9:09 AM | Permalink

November 8, 2006

In The Election Results Race, Yahoo's The Winner

Like many Americans, I wanted to know what was up with the vote in the US midterm elections this morning. As a search analyst, I then wanted to know how the search engines performed in helping me find out. The results are in! Yahoo's the winner by far, but I'd still take the New York Times over it. Come along for an illustrated tour.

Google told us last month that Google Earth was all geared up to be an election guide. That's great if you've downloaded Google Earth and wanted to learn more before the election. But how about a quick, fast summary of what happened yesterday? What's Google got for us?

The Google home page is as minimal as always, no help there -- not even a special logo as in the 2004 race.

How about a search for "election results," which I think is a fair query to try. After all, using Google Trends, I can see a huge spike for that term after the last elections in the US:

I also checked the volume for just "results," and that was even higher whereas "elections" was much lower (see them all compared here). So my two queries for this test were "election results" and "results." On Google, both disappointed.

Here are election results on Google:

CNN's top with 2004 results! I know -- web search is always behind the times. That's why Google inserts that big news results OneBox unit above the regular results. Let's click on the main news link there, which takes us to news results:

Pretty bad. News about the dollar, stock prices -- but who won?!!! I've got to really work to figure this out, especially compared to the New York Times, as I'll show at the end of this story.

Maybe I head to the actual Google News home page:

Nope. I get some headlines telling me about the Democratic house victory, but it could be much better.

FYI, checking on a search for just results, I don't even get the news headlines inserted. Overall, I found Google to be a pretty poor resource.

How about Yahoo? The home page there immediately gives you some news:

If I actually gravitate to the picture and resist the pull of the search box, the "Full elections coverage" link takes me to a Full Coverage page with lots of info, including an interactive results page (my link takes that out of the normal pop-up box, but it still works great):

This is very, very nice. I can see at a glance who is ahead in the race for control of the US House Of Representatives, plus with a click I can check out the Senate or governor races. Selecting any state also gives me the information about races with that state.

I love this. It very similar to what impressed me at the New York Times. I hope Yahoo searchers found it. However, I suspect many bypassed it. To understand why, let's do that search for election results:

Similar to Google, Yahoo inserts a big "News Results" shortcut unit above the regular results, to help detour searchers into the freshest results. Of course, searcers might bypass that. If so, unlike Google, Yahoo has managed to get the CNN 2006 results page up rather than the CNN 2004 page. Nice. After that, there's Fox News 2006. But c'mon -- Yahoo's own special election results are third. This is one case where I'd totally applaud a little hand manipulation to get that to the top, especially to highlight that interactive results summary page.

Still, the web search results for this particular day at Yahoo far outshine Google. That's almost certainly due to some human editing, which is fine. Along with the sites I've mentioned, you get the New York Times politics page, USA Today's politics page, C-SPAN's 2006 results page, the ABC News politics page, CBS News's 2006 page, politics from the LA Times, then the Washington Post's 2006 results page. All of these are excellent choices. If Yahoo did human intervention to make this happen, kudos to them. You can check out a snapshot of the entire page here.

Over at Google, nothing is either timely or general enough. The Virginia state election board, California election info, assorted things dating from 2004 -- then oddly Virginia and California get another bump for their 2006 pages. Ugh. See the entire list in the snapshot here.

What happens if we detour into the news area that Yahoo promotes at the top of the page? Disappointment:

Yes, relevant news stories. And the image results to the side are kind of fun. But some hand help could have made a difference. How about a promo for that awesome election map of Yahoo's?

Let's go over to Ask, where I had high hopes. Ask has made a big deal of its special Smart Answers for the election, and they are cool. But will I see them? Yes, if I search for election:

I'd also get to this box if I went to the Ask home page and clicked on the Election Day link there:

But for election results (what I believe to be the more popular query), all I get is a small news unit:

The news unit will take me over to some news results, but like Yahoo's, these aren't thrilling. It's pick and choose through what you want, rather than any type of easy overview. As for a search on just results, that doesn't even bring back the news unit at all.

The overall web search results, similar to Google, are underwhelming. Nothing really helpful for the 2006 results pops up (see the full results in the snapshot here).

Even the special Smart Answers box, had it shown up, isn't that helpful for what I want now -- RESULTS! None of the featured links with it takes me to results.

Microsoft, what have you got for me at Windows Live Search? On the home page, nothing. For search on election results, it's disappointing old or non-targeted results (screenshot here). Unlike the others, there are no news results inserted above these. A search for just results is no better. If I specifically try a news search for election results, as with the others, there's no attempt to get me a comprehensive overview. It's up to me to review each story and hope for a good match.

Ironically, at the largely overshadowed MSN site, similar to Yahoo, I get a big election photo on the home page along with links, including one called "state-by-state results" that leads to MSNBC here. And over there is a pretty neat "Democracy Dashboard" giving me that type of overview I wanted:

It's a pity Windows Live didn't reach out to either MSN or MSNBC and do something special to point to this or somehow integrate it into the results.

What about AOL? From the home page, it's pretty easy to spot a link to a AOL election page with results for the House, Senate and more:

Searching for election results brings back disappointing Google listings in the main results. However, the new FullView column does a good job of dividing news into elections overall, US Senate coverage, US House coverage and more. And clicking on any of the "View all" links brings up the special AOL election page (see the full page here).

Now to the New York Times. I headed over there pretty much by chance. There are any number of newspapers I might of thought of off the top of my head, and usually its my original home town paper of the Los Angeles Times. But I hit the NY Times today, and boy was I glad.

Right on the home page, above the "fold" is an easy-to-spot election map. In seconds, it organized the most important information I was looking for into a way for me to know what was going on:

Drilling into the full map was even better. There, I could click on any state -- in particular the undecided ones -- and see the current situation:

Just when I was thinking "what if," I saw the "Create Outcomes" tab where I could click on a state and flip it to the Democrats or the Republicans to see how it might go with the Senate. Outstanding!

Other newspapers or web sites might have done as well with similar displays. If so, my apologies that this wasn't a review of the best election results sites. Instead, it was really meant to see how well the search engines held up as information resources for this particular news event.

Overall, I've written many times before that there's a role humans can play in search results. Today -- this was a perfect example of that. Yahoo almost certainly put some human effort into crafting results, and it was the clear victor in terms of quality of what was coming up in web search listings. AOL comes in second, again where human effort has helped its FullView listings help make up for the poor crawler-based results from Google.

In third, I put Google and Ask. Google's results were poor, but at least it floated some news results that may have helped. Ask, I was rooting for. But that Smart Answers box simply wasn't showing up for the queries I thought people were doing. Even if people were getting it for "election," it wasn't helpful to get election results. I really appreciate the effort, and if this had been for something other than actual results, Ask would have been great. In last place -- Windows Live.

This campaign of sorts is also one of those classic "what if" races. With just a little more effort, Yahoo would have had a landslide victory by getting people to its great overview page. The same is true for AOL. Ask, with just a bit more thought, could have had that box coming up for "election results" rather than just "elections" and added some links to get people to actual results. Windows Live, if it had remembered its MSN origins, might not be in last. And Google? A company that's all about organizing information might not have put in such a poor performance if it used some human power in the way the New York Times did.

Postscript: See also Case Study: Digg Versus Google News Traffic from me on my personal blog that covers how this article ultimately brought in lots of traffic from Google News from those unable to find election results there, along with lots of other data and a comparison to traffic from a top story at Digg on the same day.

Posted by Danny Sullivan at 4:17 AM | Permalink

October 13, 2006

Consumer Satisfaction Doesn't Equal Market Share

Earlier this week J.D. Power and Associates released the findings of its "2006 Residential Online Service Customer Satisfaction Study." The study, based on a survey of more than 10,000 U.S. residential Internet users, found that Yahoo Messenger ranked the highest among IM clients and that – and this is something of a surprise – InfoSpace's Dogpile had the highest rankings among search engines.

Here are the rankings of IM clients in terms of customer satisfaction:

1. Yahoo! Messenger 2. MSN Messenger 3. Windows Messenger 4. Instant Message Average 5. Google Talk 6. Trillian 7. AIM/AOL Instant Messenger

Compare that to U.S. IM market share (per Nielsen//Netratings):

1. AOL 2. MSN 3. Yahoo 4. Google

The J.D. Power report also said that U.S. residential IM usage was flat vs. 2005, at 36%. Yet the survey found that "among customers who report using IM on a regular or occasional basis, nearly 70 percent report that to some degree, instant messaging has replaced the use of traditional telephones." There are implications here for traditional telephony that are striking and worth further exploration.

Probably more interesting to readers of this blog are the search-engine findings. The survey reported that 75% of residential Internet subscribers used multiple search engines.

Here's the market-share breakdown that J.D. Power found:

1. Google: 51% (up 8 points from 2005) 2. Yahoo: 17% (down 4 points) 3. AOL: 9% (down 1 point)

The release doesn't report on the respective shares of Ask or MSN/Windows Live. Presumably they constitute the remaining 23% of usage or something approaching that.

Compare comScore August search market share data:

1. Google Sites: 44% 2. Yahoo: 28.7% 3. MSN: 12.5% 4. AOL: 5.6% 5. Ask: 5.5%

Here are the J.D. Power survey's customer satisfaction findings. Little used Dogpile was ranked number one:

1. Dogpile 2. Ask.com 3. Google 4. Yahoo! Search 5. AOL Search 6. MSN Search 7. Internet Explorer (treated as a search engine in the survey)

I don't have any insight into the survey methodology so we have to take the results at face value. But 10,000+ respondents is a very large sample. A disconnect is the difference between search engine market share and the satisfaction ratings. Based on these findings one would think that if Ask and (especially) Dogpile could gain broader awareness and visibility they might be able to gain some share.

Posted by Greg Sterling at 11:19 AM | Permalink

June 12, 2006

'Search: Thy Name Is Google'

Last week, MediaPost reported on an Outsell report, "How Information Providers Can Keep Pace With User Demands For Time-Saving Solutions," which showed Google increasing its market share dominance but also reportedly increasing dissatisfaction with the quality of search results. To get a better look at these provocative findings, we obtained a copy of the report.

The analysis compared surveys conducted in early 2005 and 2006 among U.S. "knowledge workers" in four sectors, Corporate, Government, Health Care and Academia. It found increasing amounts of time spent on "information tasks" (gathering and analyzing information). The two primary culprits were the increasing proliferation of information and increasing inefficiency in information retrieval ? in particular of web search.

The report cites two statistics to draw the inference that there's increasing dissatisfaction with web search. First it cites an increase in the use of corporate intranets as a starting point for information (up from 13% to 19%). It also cites "search failure rates" (defined by users as unsuccessful searches for desired information) as a serious problem. Failure rates across the four categories were 31% on average. This is indeed striking.

Unfortunately, the report neglects to further drill down and discuss whether and how these failure rates are specifically affecting knowledge workers' behavior now and attitudes toward usage of search engines in the future. (It does reflect usage of a range of information resources, which are basically flat across the two surveys, except for the rise in intranet usage).

Also striking was the market share data reflected in the report. The sample size was 5,740 respondents in the early 2006 survey. And while these "knowledge workers" are not perfectly representative of the general U.S. population the sample size is large enough to merit some generalizations to that larger population.

Danny last week compared the market share numbers for April from comScore, Nielsen and Hitwise. Of the three, Hitwise had Google with the largest market share at 59%. Outsell's report shows and even larger margin (again, not necessarily representative of the population as a whole). But the Outsell distribution is as follows:

Google: 74% Yahoo: 15% MSN: 7% Other: 4%

During the survey interval Google gained share at the expense of all its competitors.

Another finding of the report is that wireless data usage has now reached "critical mass," with almost 70% of users accessing some sort of content over wireless devices.

In the MediaPost article Outsell analyst Roger Strouse muses on the reasons for failed searches:

"[A] variety of factors are to blame for search failure, including deliberate manipulation of Google's semantic system: 'From early on there's been a problem with what people used to call spamdexing, where people would apply tags to their Web sites that actually had no relation to the content at all. And that problem is still there, and is definitely going to have to be resolved some time soon.'"

Right now there are few effective alternatives to search engines for consumers as a starting point for research when they don't know where to find information. One quick and facile argument is that vertical search threatens Google et al. However I would argue that the relationship between vertical and general search is complex and the emergence of vertical search doesn't negate general search usage in those categories automatically.

The report's findings are interesting and certainly point out that everyone can do better in more efficiently providing relevant results to users. However, the report confirms that until something better and more relevant comes along, for now "Search, They Name Is Google."

Posted by Greg Sterling at 12:42 PM | Permalink

May 2, 2006

MSN Bests Google & Yahoo in Search Shootout

Interlink, a Cincinnati web marketing research, search engine optimization and consulting firm, recently ran some tests to assess performance of major (and minor) search engines. Surprisingly, MSN took top honors in relevancy.

Like the search engine relevance tests we've done from time to time here at SEW, this isn't a controlled experiment, but rather a test of how well search engines return "reasonable" results. As such, this study should be regarded as interesting but not definitive.

Over at the 360, Susan Kuchinskas reports:

MSN bested the top two search providers, along with Ask, AOL, Gigablast and Wisenut, when all five factors were considered: relevancy, freshness of content, failure rate, difficult search results, and non-organic or extra features. Difficult searches were queries such as ?car dealer fargo north dakota? and ?appliance repair des moines.?

More information and full results of the study are available here.

Posted by Chris Sherman at 4:43 PM | Permalink

April 26, 2006

Is Google Search More Relevant Or Is There A Brand Factor?

The battle of which search engine is the most relevant has been going on for years. Yesterday at SES Toronto's Searcher Behavior Research Update panel we had two presenters talk about studies they have conducted to determine which search engine is the most relevant. Google has won the relevancy battle in those studies, but was it due to Google's brand or due to Google's search results?

An InternetNews.com article also has a nice summary of the session. But let me explain them in a nutshell for you.

Gord Hotchkiss showed part two of his study, where he conducted an eye tracking test on users on both MSN, Yahoo and Google search results page. Google's results showed that people were more likely not to look down the page at results below the 2nd or 3rd listings, but MSN and Yahoo results showed that the user was much more likely to look down the results to results as low as 6 or 7. Does that mean Google's results were more relevant, sooner? Meaning people did not have to scroll to see more results sooner? He said, maybe or maybe not. He noted that the layout of Google's results at that time had bolded the keywords queried by the searcher, on the page - which may have prompted searchers to click on those results sooner with the bolded words than without the bolded words.

Lance Jones was the next speaker he conducted a study that took all the branding off the Google results and tested searchers to see if they like the results with and without knowing the results were from Google. The users scored the results from scores from 0 to 1,000. The users scored the results that were identical on both groups, 800 for Google branded results and 737 for unbranded Google results. This shows that knowing the results are from Google, adds a bias to how relevant the results are or are not.

So is Google more relevant? Well, my RustySearch live results show Google is now the leader in relevancy. When I first published the results, Yahoo was in the lead. At some point in the last 3-months or so, Google took over the lead.

The bottom-line is that relevancy is an incredibly hard factor to measure.

Posted by Barry Schwartz at 12:26 PM | Permalink

Search Stats Don't Have to be Boring

There's no shortage of data related to search marketing, with everything from market share numbers to frequency of search terms analyzed by market research groups. Some find all this data deathly boring—but it doesn't have to be. In fact, with the appropriate presentation style, search data can be downright entertaining, as it was at a recent Search Engine Strategies panel. Christine Churchill has the rundown on the panel in today's SearchDay article, Searchonomics: Search Statistics Made Fun.

Posted by Chris Sherman at 10:27 AM | Permalink

April 18, 2006

comScore: Google Leads On March 2006 US Search Share

You may have seen some rumblings a couple of weeks ago about Google being up in search traffic compared to competitors, based on comScore figures. I'm going to do a big drill down on that later this week, bringing in some NetRatings and Hitwise figures as well. But more comScore figures have just arrived for March, so here they are:

Share of Online Searches US Home, Work & University Users

Search Network March 2005 March 2006 Change Google 36.4% 42.7% +6.3% Yahoo 30.6% 28.0% -2.6% MSN/Microsoft 16.5% 13.2% -3.3% AOL/Time Warner 8.9% 7.6% -1.4% Ask 5.5% 5.9% +0.4%

Other notes from the comScore release:

• Americans conducted 6.4 billion searches online in March, up 10 percent from last month and 15 percent from last year. The increase in search queries from the previous month marked the largest gain over the past twelve months.  
• Google Sites led the pack with 2.7 billion search queries performed, followed by Yahoo Sites (1.8 billion), MSN-Microsoft (849 million), Time-Warner Network (486 million), and Ask Jeeves/Ask Network (376 million).  
• The toolbar search market continues to be dominated by Google and Yahoo!, which combined for more than 95 percent of toolbar searches in March. Google led the way with 48.9 percent, while Yahoo! captured 46.5 percent.

Posted by Danny Sullivan at 10:31 AM | Permalink

April 4, 2006

The Thrill & Danger Of Measuring Relevancy Through Ego Searches

Last week, you may recall that Ask got a rave review from the Wall Street Journal. Robert Scoble saw that, then did an ego search for himself and decided Ask doesn't measure up based on that. Fair criticism? Sure, to some degree. But then again, it's easy to take a single search for anything and show that any of the "leaders" in search have problems, as well.

In Search Of The Relevancy Figure from me back in 2002 goes through the various ways that search engines are measured up. Ego Search is a long-standing one, as I described:

"Ego Search" is another style that can be bad, and one that I still see journalists and others often perform. In an ego search, you look for your name. If you fail to come up tops for it, you conclude the search engine's relevancy is poor.

In some cases, perhaps this is true. If I search for "bill gates," it's reasonable to expect to find the official web site for Microsoft Chairman Bill Gates. But what if you aren't as well known as Bill Gates or have a popular name? What if you've built a web site in some free hosting service that is shared by spammers? These might be issues that push you down, and for good reason. Moreover, are you're going to condemn an entire search engine as bad, based on one search? Well, I've seen it happen.

So is Ask bad because it didn't find Robert Scoble's current blog. Yes, for a variety of reasons:

• The new blog's been out there enough that I would expect it to be listed in the top results  
• Others can get to it via his old blog, which is listed in the top results, but it would be better not to make that detour  
• It matters to Robert

The last point is probably the most important. While ego searches can be bad, they remain one of the main ways anyone will test their favorite search engine. You've got to get them as right as possible, and that's not happening with Robert.

For the record, in my searching today, Google and Yahoo list both the old and new blog (in that order), while MSN gets the new one first, then the old one further down (though it's the opposite if you search for robert scoble rather than scoble)

Also for the record, Robert himself makes the job tough for any search engine. He does no redirection from his old to new site, which pretty much makes it impossible to do the very best thing, list only the new blog.

Think about it. If you search for Robert, why send you to his old blog, just to make you detour? I'm not saying pages from his own blog shouldn't come up for relevant specific queries. But in a search for him by name, you'd really only want to point people to his current blog.

On the flip side, as I wrote on Dave Naylor's blog recently, keeping both blogs going with no redirection is also smart marketing that ensures you dominate the results for your name :)

Back to the search, I also tried using his first and last name. Ask deserves some kudos on that, as robert scoble puts a picture of Robert at the top of the page, based on pulling material from Wikipedia. Nice! Sure, his new blog still isn't listed, but that doesn't mean some other features aren't helpful.

Earlier I said Ask is bad for failing the ego search test for Robert. But bad as in bad on that specific query, rather than bad overall. Condemning an entire search engine based on it going overboard, as Robert does when he says:

But, Walt, this doesn't portend good things for Ask.com. To be included in the top three you've gotta be as good as Google. Ask isn't even in the same neighborhood yet.

Wow. So all I've got to do is find a query where Ask succeeds and the others fail, and then Ask's in the neighborhood. Well, John Battelle helps out here. The Last Name Test post he just put up shows how he can easily find himself on Ask for his own name, just as he's found on Google, Yahoo and MSN.

Then again, that search also points out an Ask Jeeves failure. I remarked about this on our Daily SearchCast show when talking about Ask's rave review by the Wall Street Journal. Ask was having issues with not consolidating web sites correctly.

In particular, the day of our show, I'd done a search at Ask to find one of my articles. I queried invisible tabs, and I got this:

Searching With Invisible Tabs Searching With Invisible Tabs. By Danny Sullivan, Editor December 2, 2003... searchenginewatch.com/searchday/article.php/3115131   Searching With Invisible Tabs Searching With Invisible Tabs. By Danny Sullivan, Editor-In-Chief December 2, 2003... www.searchenginewatch.com/searchday/article.php/3115131

That's the same page, listed twice. Ask is having a canonical issue, not realizing that the www URL leads to the same page without the www prefix (and this despite us doing redirection).

You can see the same issue with John's search for battelle:

John Battelle's Searchblog John Battelle's Searchblog Thoughts on the intersection of search, media, technology, and more. Search and Newsletter battellemedia.com/

John Battelle's Searchblog About John Battelle. Federated Media. UC Berkeley. Columns. That Book I Hear He Wrote. Consulting. Contact... www.battellemedia.com/

All those listings are from the first page of Ask's results. The first and third listing are the same page, just one is shown with the www prefix.

So now we can say that Ask is clearly not in the same neighborhood as Google, Yahoo and MSN! Hold on there.

Let's go to MSN and look for invisible tabs. I'm not saying that my article should be the only thing listed, but look at the result at the bottom from MSN:

invisible girl ABchao Brad Bubs Connie Daynah Dodo Dodo's Tabs Heather Jen Jessie Kristin Leah Leigh Sanne Sarah Shiny Shiny Shopgirls! TechLog Meta </a> .org - asm > Valid XHTML XFN WordPress

In this case, I'm getting a page that has nothing to do with invisible tabs of any type other than they simply having both of those words on the page somewhere. But over at Ask, every listing in the first results is related in some way, either to my article, or to how they apply to a programming issue, or to the idea of guitar tabs you use to play the song Invisible. By this measure, MSN isn't in the neighborhood of Ask.

As for domain issues, MSN's fixed the embarrassing issue it had with every cars.com subdomain dominating (shall I say domainating) a search for cars, but then when I search for movies today, I get all these from the go.com domain:

• movies.go.com
• casanova.movies.go.com
• video.movies.go.com/sincity

None of these are "indented" results, either, so it doesn't feel like the subdomain issue is completely solved. I'm guessing MSN fixed it to check back three levels but not four.

Let's swing back to Yahoo. MSN's domain issues could be excused in part because as a search engine technology, they're still an infant. But here's scoble over at Yahoo showing these domains (numbers show the listing order):

2) scobleizer.wordpress.com

3) www.kunal.org/scoble

5) www.kunal.org/scoble/archives/2004_09.html

6) radio.weblogs.com/0001011/2005/02/19.htm

8) radio.weblogs.com/0001011/2004/01/27.html

9) scobleizer.wordpress.com/2006/02/13/the-brrreeeport-report

OK, it's not unusual for search engines to show up to two pages from the same domain in the first page of results. Google's popularized this concept through its indented results. I'm not even saying that Yahoo should be indenting results. But if you're going to show more than one page per web site, at least get coordinated enough to put then next to each other. If I just saw a scobleizer.wordpress.com page at the top of my result, I don't want to see another page from that domain at the bottom.

How about the widely acknowledged leader in search, Google? Let's do scoble over there. Down at number six is this:

scobleizer.manilasites.com/

That's a link-only URL, a page Google only knows about through links pointing at it, rather than having visited the page. What happens if I try to visit it? 404 Page Not Found. Heck, I don't even get that. I get a complete non-resolution to the site, a time-out. It doesn't look to exist any more. So why's this making it at number six for that search? Why isn't Google smart enough to drop this entirely useless listing?

To sum up, ego searches will remain a powerful way anyone measures the relevancy of a search engine personally. However, they also remain a dangerous way to assess the overall health and quality of a service. Ultimately, to measure the quality of a service, you remaining wanting an entire battery of different tests run, measuring relevancy and usability.

Posted by Danny Sullivan at 9:57 AM | Permalink

March 30, 2006

Ask.com Gets Good Mossberg Review in WSJ

Gary Price points to a write up by tech guru Walter Mossberg in the Wall Street Journal, where he says "Ask holds its own with Google, and even beats the champ (Google) on some searches." Mossberg also adds "Ask.com is well worth a try if you want to benefit from some features that go beyond Google." Read the full article here.

Posted by Barry Schwartz at 11:18 AM | Permalink

China Search Market Usage To Expect Decline In Growth

Shak informed me about this article that shows the growth of China's search engine market is expected to slow down over the next 18-months. Edward Yu, CEO of Analysys International, blames the expected decline in growth of search usage to "poor user experience, unstable advertising effects, and some irregular channel operations."

The China search market is considered to be in the "initiation stage," and the search results do not meet searchers' expectations.

According to Analysys International's research of 1500 samples, only 20% search engine advertisers obtained results that exceeded their expectations, while about 29% didn't get the results they had expected and were considering reducing or suspending their search engine marketing expenses.

Posted by Barry Schwartz at 9:15 AM | Permalink

March 7, 2006

New French Relevancy Study Puts Yahoo & Google Tied For First But Search Overall As Disappointing

Jean Véronis has published the results of a relevancy study he conducted with his students at the Université de Provence showing that, at least for French searches, Google and Yahoo are tied on the relevancy front but that search engines overall disappoint. However, he also takes a different spin on the stats that give the search engines higher marks and puts Yahoo slightly ahead of Google.

Jean summarizes the study here, with a more detailed PDF rundown here. Google, MSN and Yahoo along with Exalead, Voila and Dir.com were evaluated. In summary, here's what was done:

• Students were assigned topics and allowed to craft queries in French however they thought best to search for information on those topics.  
• The first page of results from the search engines involved for each query were retrieved. Only French-language results were pulled back, and the porn filter at each service was used (and US Department Of Justice, take note --- the study found these worked very well).  
• Students then reviewed the URLs retrieved to see if they were generally on the topic of the search. Each URL was scored to the relevancy of the topic, 0 being worst, 5 being perfect.

Scores were:

• Google/Yahoo: 2.3
• MSN: 2.0
• Exalead: 1.8
• Dir: 1.4
• Voila: 1.2

As you can see, no service gained above the 2.5 middle ground on the 0 to 5 scale. But at the suggestion of Jakob Nielsen, Jean did a new comparison of scores, to take into account that users may be happy if they get at least one good result per search, rather than 10 of them. So -- if I understand right -- he calculated the average highest score for URLs on search results pages.

In other words (and again, if I understand right), he looked at all the search results pages from Yahoo, then looked at the highest scoring URL on each page, then figured the average. Relevancy shoots up with this much tighter examination, and Yahoo pushes slightly ahead of Google, while MSN and Exalead also close the gap:

• Yahoo: 4.5
• Google 4.3
• MSN: 4.2
• Exalead: 4.1
• Voila: 3.6
• Dir: 3.4

Posted by Danny Sullivan at 10:04 AM | Permalink

January 19, 2006

Keynote Says Google, Yahoo Preferred in North America

Keynote Systems has released the results of its annual study of North American searchers and their satisfaction with search engines. The study ranked user satisfaction by tracking the search behavior of 2,000 users on AOL Search, Ask Jeeves, Google, MSN Search and Yahoo, and found Google and Yahoo were clear #1 and #2 favorites, respectively. Today's SearchDay article, Survey: Google, Yahoo Still Favorites in North America, has the details. Also see yesterday's SearchDay article, Study: Google #1 in China, for details of a similar study Keynote performed in China.

Posted by Chris Sherman at 10:47 AM | Permalink

January 10, 2006

Googling For Google Keynote Video

Gary wrote about the AOL acquisition of Truveo, which talks about how it will supposedly do a better job than other video search engines in finding relevant, timely content. I'm going to rope Gary into doing some testing, which should be fun. But I thought I'd try a quick one myself. What do I get in a search for Google on these services? In my mind, if they are timely, I should get something about the Google keynote at CES last year. Let's go!

• Truveo: Cool, there's 9 minute clip provided by News.com, right at the top. And the other clips listed are in order of upload, so they all seem relevant to the most recent Google news.  
• Google: Nada, nothing. But I was intrigued by this Google Local and the Kansas City Royals commercial, put online by Google itself. My guess is this was played during Kansas City Royals games on the big screens during the game. But what if I sort by date? Will that get me the keynote? Nope. How about if I look for google ces? No results at all. What happened? Google Video doesn't crawl the web, so it's not going to find the great News.com clip.  
• Yahoo: Nothing. google ces? Nothing. Sort by date maybe? Not an option. Unlike Google, Yahoo does crawl the web, so the absence of News.com content is notable. Heading over to Yahoo News and choosing the AV option? No luck.  
• AOL Singingfish: Nothing, nor anything with google ces, even though Singingfish crawls.  
• YouTube: Same as AOL & Yahoo, no luck. But like Google, YouTube has the excuse of not crawling the web.  
• Blinkx: At the bottom of the list, and if I use the little slider to sort by date, it moves up a bit.

Overall, Truveo delivered well. But this is also involving a current events query. If I really wanted that old Brady Bunch episode, it won't come through for me on that.

In the end, you're likely to see video search get just as refined as web search did -- video search for news, video search for entertainment, videocasts and so on.

Postscript from Gary Although they DO NOT offer access to the News.com report of the keynote, AOL Video Search has a number of current video news stories about Google. Also, Yahoo News (limited to audio/video content) has a recent Reuters report (video) and an NPR story about the Google/AOL deal.

Posted by Danny Sullivan at 10:40 AM | Permalink

December 2, 2005

MSN Using Human Quality Raters to Review Results

Via Threadwatch (and its new man in charge, Aaron Wall), word that MSN Search is also using quality raters to revew and rate results.

We've posted in the past about quality raters at Google and Yahoo in these and other posts: + Google Looking for Spanish Language Quality Raters + Google Rater Hub - Details Of Human Review System Unveiled + Interested in Evaluating Search Quality and Advertising for Google? + Want to Be a Relevance Analyst? This post talks about similar types of reviewer positions at Yahoo.

Posted by Gary Price at 5:11 PM | Permalink

November 25, 2005

Bill Gates: MSN Search Will Soon "More than Match" Google in Terms of Relevance

In an interview with Information Week, Bill Gates talks about Microsoft Research and speaks briefly about search and of course mentions a competitor that begins with the letter "G". Gates says that coming soon MSN will have "more than matched" Google in terms relevance. Here's the full passage:

Gates says: Search is an amazing example where we relied somewhat on an outside company, Inktomi, which Yahoo bought, then decided to build our own search effort essentially from scratch. Now, in a very short period of time, we will actually have more than matched the kind of relevance that Google can deliver. The role of Microsoft Research in that has been phenomenal.

Posted by Gary Price at 9:34 AM | Permalink

November 10, 2005

PC Magazine Reviews Six Comparison Shopping Sites, PriceGrabber Gets Top Honors

In a PC Magazine review of six comparison shopping sites titled: Price-Comparison Sites Strive to Save You Time and Money, Rick Broida takes a look at:

+ Become.com + mySimon + PriceGrabber.com + Shopping.com + Smarter.com + Shopzilla Broida's article also includes a look at Buy.com, Amazon.com, and eBay. Interestingly (maybe another article?) no mentions of Froogle, Yahoo Shopping, and/or MSN Shopping. Bottom Line? PriceGrabber grabs the editors' pick. Btw, PriceGrabber also powers "product" search on Ask Jeeves. Finally, if you follow the shopping search engine scene, make sure to visit Brian Smith's, ComparisonEngines.com blog.

Posted by Gary Price at 2:49 PM | Permalink

November 8, 2005

Why We Use Various Search Engines

InternetRetailer has done some nice charts off of a Majestic Research/comScore report looking at why we use particular search engines (for Google, it's the results; for others, it's because you're doing other things). The stats also look at awareness of paid links and tolerance of demographic and behavior targeting. Here's a summary:

For the question of why people use particular search engines, top reasons for each major service were:

• Google: 68 percent say it's because it has the best results.  
• AOL: 65 percent say it's because they are doing other things at AOL, like checking mail and other non-search activities.  
• MSN: 63 percent say it's just like AOL, because they are doing other things there.  
• Yahoo: 52 percent say it's just like AOL, because they are doing other things there. Yahoo got the second highest marks for having the best results after Google, with 33 percent choosing that reason.

The report found that AOL and Google users were the most likely to notice sponsored links (82 and 81 percent, respectively) while MSN users were the least likely to notice them (69 percent).

As for privacy, 58 percent said they weren't worried about being demographically or behaviorally targeted as long as it was disclosed and they could opt out. And 27 percent said they'd keep using a search engine even if they couldn't opt out.

Haven't tracked down the actual report yet; will postscript, if I can find it.

Posted by Danny Sullivan at 1:11 PM | Permalink

September 22, 2005

Microsoft Quashes Search Relevancy Report

New Search Engine From Microsoft Gets Cool Welcome from the Wall Street Journal (paid reg. required) delves into the rumors we heard in August and confirms them -- that the latest Keynote findings on search relevancy have been suppressed by Microsoft. The last we had were back in January. Why? Well, MSN Search is found to have slipped in those metrics. For its part, Microsoft says it asked for the study not to be released because it questions the methodology.

See? Further evidence it really is 1999 again. I used to get relevancy figures from NPD back then in a scatter-shot approach. I'd know who was ranked first, maybe third, but any search engine that didn't think they were doing well would quash releases of their own figures.

I covered this problem in my In Search Of The Relevancy Figure article in 2002:

Public release of this data is also important. Some companies that contracted with eTesting Labs in the past refused to let the tests be made public, if they did poorly. Similarly, the NPD Group used to do consumer surveys, where the search engines' own users rated their relevancy. Those search engines that did well often released their figures, while those that did poorly kept quiet.

While it may be tempting to sit on bad news, if the search engines want us to take seriously their claims of relevancy, then they have to agree to release both the good and the bad. If a search engine does poorly, then that poor performance should be an excuse to work harder.

I recently dared Google and Yahoo to report on relevancy and get past the size morass. So far, no word. I can tell you that both companies have spent a serious amount of time looking at the best way to spin the issue, far more than I suspect is being spent on coming together to benchmark performance. Guess Microsoft won't be dared at all.

Honestly, if Microsoft and Yahoo execs can collaborate with Google for its stealth Zeitgeist strategic partner meeting, is it really too much to think they can't come together to benchmark relevancy? I know intimately the difficulties in measuring such a subjective thing. But this is your CORE product, gang. This is what you do -- serve search results, and you have no way of proving to the world how good you are. Really, screw knowing who is biggest -- we don't even know which of you is most relevant!

It goes to the business model so crucially. More and more I hear about people talking about Google's search not being as good. Is it? What's the proof of this beyond ego searches? Traffic numbers staying up are one indication it might be doing OK. But the more Google moves away from core search into areas like, hmm, chat and wireless internet access, the more it becomes vulnerable to the "you aren't staying focused" accusations. In turn, those help fuel believe that relevancy might be worse, even if it's not.

In short, even the relevancy winner according to anecdotes and word-of-mouth, Google, has a vested interest showing a relevancy fitness report to the public that can be believed. That's because anecdotes and word-of-mouth can easily turn against it. Google, like all the major players, need to get us the benchmarks.

Meanwhile, I'll say this. Microsoft may be one of Keynote's biggest revenue streams as the WSJ reports and thus has clout to help Keynote make a "business decision" not to release the findings. But here's another business decision to consider. You've just wiped out any faith I have in your figures at all.

The next time Keynote trots them out -- assuming there's a better business climate allowing that to happen -- I'm going to be sitting here dubious if I should even trust them.

Honestly, I can't say enough how tired I am -- and how tired everyone should be -- of these types of games. Google's growing its index -- fact -- but won't budge the home page number until it obviously feels it will be able to wallop any counterclaims by Yahoo. So that number on the home page is meaningless other than as a PR hamburger count stunt. Yahoo played the game as well, of course, deciding it was prudent to finally publish its figures only when knew they'd have a count far larger than Google could trot out. Microsoft's playing the "don't release relevancy figures" game.

We did this once already in Web 1.0 or Search 1.0 or whatever you want to call it. If we're in Web 2.0 or Search 2.0, let's see things advance. No more games. Come together and go fund someone to properly study and benchmark how you are doing, and publicly pledge you will live and die by those results. Don't roll back into the rut of last decade.

Aside from the relevancy quashing, the story also looks at how Microsoft doesn't seem to be gaining in the search wars despite the technological investment in build its own search engine and backing it with marketing money. It has been noticeable as when I did the various traffic numbers last month how MSN hasn't really gained share. One interesting tidbit also is Microsoft thinking of sharing ad revenue with computer makers to help lock in potential search traffic.

Posted by Danny Sullivan at 10:41 AM | Permalink

September 21, 2005

Want to Be a Relevance Analyst?

If you're in search-related job, here are two that just might be relevant. (-: The other day I learned that Yahoo Search Marketing has posted two openings for Relevance Analysts (part of the Relevance Assessment Team) at the Yahoo Search Marketing hq in Pasadena.

What would you do? The primary responsibility of the Relevance Analyst will be to test new and existing matching technologies by scoring listings from a relevance perspective and providing feedback on the results.

• Relevance Analyst: Yahoo Inc
• Relevance Analyst, International: Yahoo Inc

Posted by Gary Price at 3:22 PM | Permalink

September 12, 2005

Looking for Katrina Images On Google & Yahoo

Over the weekend I went searching for Hurricane Katrina imagery in the Google Images and Yahoo Images databases. It's been two weeks since the storm hit the now devastated gulf coast and I wanted to determine what a typical user might find when entering the simple but descriptive query [Hurricane Katrina] in the image search box at Google and Yahoo.

At Google Images only 36 images (I'm not kidding, are seen on the results pages (screen cap here) and the relevancy of a few of these images are questionable (they have nothing to do with the storm and the devastatlon it caused).

Btw, I ran the search several times over the weekend to make sure a technical error was causing so few images to appear.

Many of the images that Google does offer are charts and maps of the storm before it slammed into the gulf coast. If you're looking for imagery that illustrates the destruction Katrina caused, you'll find very few. I was surprised that with the tens of thousands of images found on the open web, I would have found more. Perhaps this exercise also gives us some idea about how often Google Images is updated?

How did Yahoo Images do with the Hurricane Katrina query?

MUCH better than Google.

The total estimated number of images for the [Hurricane Katrina] query at Yahoo Images is over 6,100 but that number means little since I was only able to view 531. That said, most -- but not all -- of these images have a direct relationship to the storm and its aftermath. You'll find images from various sources culled from the open web along with imagery available from Yahoo News.

One thing I was unable to find via in either Google Images or Yahoo Images was material from the FEMA Photo Library, U.S. Dept of Defense Image Collection, and other sources inclding Orbimage and DigitalGlobe. Of course, this once again illustrates why knowledge of specialized/fcoused databases is so important.

Posted by Gary Price at 10:59 AM | Permalink

Tara Gives Thumbs Up To Yahoo Images; Yotophoto For Copyright Free Images

Contemplating Yahoo Images and Google Images is a short item from Tara over at ResearchBuzz finding that in a search for [rss buttons], less is more. Yahoo had fewer results than Google but was more on target with what she wanted. Meanwhile, her Search Engine of Free, GNU FDL, Creative Commons Images looks at the new Yotophoto image search engine designed to give you images you can use without copyright worries.

Posted by Danny Sullivan at 9:45 AM | Permalink

August 25, 2005

Yahoo (Initially) Beats Google For More Relevant Listings For Google Talk

In a post titled: Searching for Google Talk on currybet.net, Martin Belam reports on a quick test he ran yesterday when he went looking for links about the new Google Talk service in the web databases at Google and Yahoo. According to Belam's report, at 5pm yesterday "no links about the new messaging application in the top ten" at Google. However, Yahoo, "had already indexed and ranked the wikipedia article about the new application. In addition Yahoo! pulled in three news stories about the Google Talk launch at the top of the results set." Martin's post includes a couple of screen caps of what he saw when he ran his test.

Posted by Gary Price at 10:27 AM | Permalink

August 17, 2005

Yahoo Virtually Tied With Google On Customer Satisfaction

I wrote earlier of wanting relevancy figures. In lieu of that, how about some customer satisfaction figures? Yahoo! Gains On Google In Customer Satisfaction from Forbes covers the latest data from the University of Michigan's American Customer Satisfaction Index that puts Yahoo a hair's breath behind Google for satisfaction.

Google earned 82 out of a 100 possible satisfaction points, with Yahoo following at 80. However, Google is classified in the "search engines" category while Yahoo is classified in the "portals" category. Different questions may be involved to assess satisfaction and if so, then the scores might not be comparable.

I've looked through the ACSI site to see if the different classifications are an issue, but I've had no luck. You can check out the FAQ page and this fact sheet, as well as this commentary on the latest scores. None explain exactly what questions were asked in each category.

You can find the actual data from the ACSI here. Below is a chart that looks at scores for the various players over the past four years. Overall, Google has stayed on top but flat while competitors have generally risen, though the remain behind.

Postscript: Ask Jeeves sends a note that the testing was done before it had reduced ads on its pages, something it obviously feels will lead to an increase next time.

Postscript 2: ACSI tells me "figures are comparable across categories and that the same questions are used regardless of category," so it's fair to compare satisfaction of those in the portal category to the search category. Postscript 3: ASCI also tells me that while it can't show the actual questions asked, as these are considered proprietary, this page explains more on how things are measured and this PDF file is a sample survey for the automotive industry that gives you a general sense of what's asked.

Posted by Danny Sullivan at 9:26 AM | Permalink

June 27, 2005

Ballmer: We'll Catch Google In Relevancy In 6 Months

We've had Microsoft CEO Steve Ballmer make jabs at Google before. However, ZDNet Australia article Google in sight as Ballmer vows .NET push reports a new spin, that "in the next six months, we'll catch Google in terms of relevancy," Ballmer said.

It will be interesting to see how this will be proved in six month's time. As Danny points out here, trying to determine search relevancy is a difficult thing to do.

That said, if you look at the results from Barry's unscientific but very interesting survey, the difference in relevancy between Google, Yahoo, Ask Jeeves, and MSN Search is already very small.

Of course, relevancy is very important but to "catch" Google, MSN needs to build mindshare. This might be their biggest challenge. No matter what Google does, it seems to get press attention and buzz everywhere (not only in the tech press). Plus, I have the gut feeling that many users have the belief that no matter what MS does, it could never be as good, cool or useful at compared to what Google provides.

As I've said many times before, it's not only about building a quality product but it's also now about changing user behavior (come see what we're doing) and beliefs (only Google is and will ever be cool).

Other comments Ballmer made while visiting Australia relating to Google:

• "Innovation in search is not done."  
• "We'll use search to peer into a range of business applications which would allow multiple applications to be searched simultaneously."

Ah, metasearch. One interface to use and to learn. This is something I've been talking about for a long time!

Posted by Gary Price at 4:14 PM | Permalink

June 24, 2005

Rating Search Engines By Playing Trivial Pursuit

We've written many times about how search engines are moving to presenting direct answers in response to some questions. Trivia Stump Smart Search Engines from the Associated Press has writer Michael Liedtke deciding to test how well these direct answers seem to work by involving the search engines in a game of Trivial Pursuit.

Invited to play were Google, Yahoo, MSN, Ask Jeeves and Answers.com. Ask and Answers were a "small step" ahead of the others, but even they rarely met the goal of providing a single clear answer, he writes.

The scorecard, with points awarded for getting him to the right answer via the first link on the page, with twenty queries in all:

• Ask & Answers.com: Tied with 10 out of 20 times, 50 percent success rate
• Google: 8 out of 20, 40 percent success rate
• Yahoo: 5 out of 20, 25 percent success rate
• MSN: 2 out of 20, 10 percent success rate

Of course, a search engine might more fairly be deemed successful if it listed a page leading to the right answer anywhere on the search results page, not just in the first link. Liedtke took that into consideration. He also examined how often they completely failed to lead him to a page with the answer after he scanned a generous 3 pages of search results. The scorecard:

• Ask, Answers.com & Google: Failed 3 out of 20 times, 15 percent failure rate (or 85 percent success rate)
• MSN & Yahoo: Failed 6 out of 20 times, 30 percent failure rate (or 70 percent success rate)

Posted by Danny Sullivan at 8:42 AM | Permalink

June 9, 2005

CNET Compares Nine Web Engines

Ben Patterson from CNET has written reviews of nine web search engines along with putting together a chart that lists various features.

Google is the favorite, while Yahoo is a close second. Other engines include:

• A9
• AltaVista
• AOL Search
• Ask Jeeves
• Looksmart
• Lycos
• MSN Search

One point that should have been included in each review is where the database(s) for each service came from. For example, no mention that Google provides the web and image search databases at AOL and A9. The same is true with the Yahoo web, image, and news databases at AltaVista. Actually, instead of spending time on AltaVista why not include Exalead and Gigablast. Each of these services provide their own crawl of the web.

Posted by Gary Price at 2:50 PM | Permalink

June 2, 2005

Google Rater Hub - Details Of Human Review System Unveiled

Google Secret Lab, Prelude from Henk van Ess's new Search Bistro blog looks at how Google uses human reviewers to improve search quality. Ess has a screenshot and a Flash movie of how the system works, for Google's temporarily hired Q&A checkers. Very nice details so far that I've never seen posted anywhere before. However, this type of system isn't new.

WebCrawler used to have a feedback system like this back in the late 90s, and other major search engines have had human reviewers, as well. Gather enough data of what's deemed good and bad, and the hope is that you can create a better search algorithm to improve relevancy.

What almost certainly is not happening is that the human reviewers are actually moving sites up and down with their ratings. Ess is promising more details in future posts, so watch his blog. Want to discuss? Visit our forum thread, Secrets of http://eval.google.com revealed?

Posted by Danny Sullivan at 3:17 AM | Permalink

May 31, 2005

Shmoogle: Randomizing Google Results To Show Good Stuff Beyond Top 10

Thanks to Philipp over at Google Blogoscoped for pointing out a new site called Shmoogle.

This new service takes Google results and shows them in a random order. Why? The sites creator, Tasila Hassine, is trying to make a point. She writes:

This tool touches upon several crucial issues on the web such as Search Engine Optimization. Shmoogle instantly neutralizes Page rank and the whole SEO industry induced by it. Yet it addresses other fundamental issues such as retrievability vs. visibility. While all pages on the net are equally retrievable, they are certainly not equally visible.

Hassine has a good point and one that I make quite a bit in my presentations and classes to both librarians and the general public.

Just because it's "on the web" and has been crawled by a web engine doesn't mean that it's easily retrievable/visible. As I've said before, the Invisible or Deep Web in 2005 is every result beyond number 6 or 7. (-:

Why is this an issue? Here are just a few reasons that come to mind:

++ Keywords Selected You use the term "pop" but the perfect result uses the word "soda."

++ Number of Keywords Used by the Searcher

++ Effort and Time Searcher takes what they find during the first search and does nothing else to possibly improve their results. They also want it "all" in just a few seconds.

++ Lack of Searching Skills Like I've said many times, people don't use most of the tools engines offer to create more precise results. I'm not just talking about advanced search resources but also the fact that many of the large engines offer specialty tools like image, news, and discussion search. Most of the engines will tell you that the click-through rates on these services are very small as compared to the primary web engine. Udi Manber said a few months ago that search engines are not mind readers. He's right and a little bit of education about search could go a long way.

++ The Searcher Doesn't Look Past the First Results Page Many more results are available but so what?

++ Search Engine Overlap Different results at different engines but does the searcher look in more than one place?

++ Of course, another issue is that the data itself is just not on the open web. It might be available "via the web" but the searcher doesn't know where else to turn to find it. Again, a specialized database might have just what they are looking for. Yes, sometimes these databases cost money but many times people don't have any idea that they have free access to these fee-based services from home or office.

These reasons and many others are why I think we've seen lots of interest in verticals in the past few months. Many times, assuming the searcher knows about the resource (here comes marketing again since people can't use what they don't know about), a searcher can get a good if not better answer in a shorter amount of time by searching a smaller, focused database.

A study published by Outsell last month pointed out that searchers in the workplace are "shifting away from their Internet research methods from just four years ago" and relying more on other sources including librarians and their intranet.

I often wonder if making large web engines larger with more content will make everything easier versus keeping things in small, focused databases and using meta/federated search technologies to (if needed) search disparate databases simultaneously using a single interface. Databases to help with database selection would also be possible.

Dynamic clustering, what Clusty.com provides can help the searcher quickly surface results that they might not see on the first page of results. In a white paper from Clusty's owner, Vivisimo, they argue that their technology can provide "a selective ignorance." Personalization based on a users preferences and their past search behavior can also be of assistance in helping material become more visible.

Posted by Gary Price at 5:40 PM | Permalink

May 17, 2005

Google Ranking Alternative Domain Tops For Searches On Google

The Google.com does not rank #1 for Google thread in our Search Engine Watch Forums looks at how Google has recently been returning not the URL of its own home page first in response to a search on Google but instead a redirected URL. In other words, rather than getting:

www.google.com

You sometimes get this URL ranked first in a search for google.

https://desktop.google.com/

That looks like Google Desktop is being top ranked -- but note the https prefix. Rather than this being the Google Desktop site at http://desktop.google.com, it's instead a secure server that redirects to Google.

This has been an off-and-on situation from at least the beginning of this month, when the thread began. Google Blogoscoped noted the return of it happening again yesterday. Today, we seem back to "normal." Meanwhile, see the second comment at Google Blogoscoped in response to its story. It's a nice explanation of how this confusion is apparently caused by how SSL requests are treated if a certificate is declined.

Posted by Danny Sullivan at 10:26 AM | Permalink

RustySearch Search Relevancy Challenge: Everyone's Pretty Average, So Far

Barry Schwartz's Search Engine Relevancy Challenge is still going strong at RustySearch. How's it going so far? He's had 5,000 rated search so far and gives an update here: RustySearch Hits 5,000 Rated Searches.

I like the cool little "search engine relevancy dials" he's added to show how each service is doing. Here's the non-cool bullet point rundown:

• Yahoo, 3.4
• Google, 3.4
• Ask, 3.3
• MSN, 3.1

What do those numbers mean? A score of 1 is poor, while a score of 5 is excellent. So overall, everyone's pretty much average. More details are in his post, and don't forget to participate by trying RustySearch.

Posted by Danny Sullivan at 9:47 AM | Permalink

May 12, 2005

Dogpile Gets New Look, Overlap Comparison Tool & Study

Dogpile has released a significant upgrade to its meta search engine, allowing easy comparison of search results across the major search engines. Dogpile has also introduced a new comparison tool that visually illustrates search engine overlap (or lack thereof) in the top results for Ask Jeeves, Google and Yahoo.

In today's SearchDay article, Dogpile Enhances Meta Search, Offers Comparison Tools, I take an in-depth look at these new services, and also comment on some new research that quantifies search engine overlap and why it's important for both searchers and search marketers alike.

Posted by Chris Sherman at 1:39 PM | Permalink

May 6, 2005

Early Results from Search Engine Relevancy Challenge

Early results from Barry Schwartz's Search Engine Relevancy Challenge have just been posted. About 3,200 results have been rated so far and it's just about a dead heat between Ask Jeeves, Yahoo, Google, and MSN. More about the SERC in this post.

Posted by Gary Price at 1:28 PM | Permalink

May 3, 2005

RustySearch: A Search Engine Relevancy Challenge

We all know that trying to define and measure search relevancy is quite a challenge. There are so many variables both human and mechanical) that can come into play. Nevertheless, it's important to try. Today, news of a new site from Barry Schwartz (you might know him as RustyBrick) that he calls RustySearch. The site is a search engine relevancy challenge to measure relevancy from one of four randomly chosen engines. Details on how it all works here.

You might also want to take a look at this SEW Forums thread: Coke vs. Pepsi Challenge for Search Engines from about three weeks ago where Barry and others (including Danny) discuss the idea of such a test. This thread has even more links to material about relevancy.

For many non-search geek types (the people who are not likely to take this test) their happy with whatever they find quickly, often not realizing that something else could better serve their needs. We can call it the principle of least effort. However, as I've said many times before, a little bit of user education (I'm talking minutes not hours) on how to create a query and use some of the features many engines offer can help the searcher create precise queries that might produce more relevant results. Perhaps Udi Manber said it best at PC Forum a few weeks ago when he told the audience that search engines are not mind readers. I would also argue that user education also extends to vertical engines and specialized tools. Sure, verticals can produce very relevant results but unless the searcher knows that the engine exists, they can't get these results. In this sense, user education is also how a vertical let's the public know that they are open for business. It's also important to consider the authority (where the info is coming from) and currency of the information being retrieved when looking at relevance. Critical info skills might be more important today than every before.

Posted by Gary Price at 11:34 AM | Permalink

February 7, 2005

All Ads At MSN Search? How First Impressions Count

I've read a number of popular press accounts over the past week trying to assess how well the new MSN Search measures up against the other major search engines. The difficulty with these or any accounts is that, as I've written before, unless you conduct a battery of tests, coming up with a meaningful relevancy assessment isn't each. Is MSN a certain percentage better or worse than Google? No one knows.

The Chicago Tribune review, Search engine surge a boon, but it's ads nauseam from MSN, provides an example of this. It takes looks at MSN Search, conducts a query for "pearl harbor" and comes away unimpressed, finding things too commercial:

Much more than Yahoo and Google, MSN's search results are centered on commercial sites and people with things to sell rather than useful and relevant information.

Is it true, even for this single query? I thought it would be fun to do a breakdown and find out. Below are all the unpaid listings on Google, titles only, in the order shown. I'm including the three product search results that also show up at the top of the Google home page. I'm also showing the full information displayed to the searcher, though the exact layout is obviously different.

Google:

• Remember Pearl Harbor - $18.74 - dvdplanet
• Pearl Harbor (History Channel) - $22.99 - pcmall.com
• Pearl Harbor (History Channel) - $22.99 - MacMall
• National Geographic: Remembering Pearl Harbor--history, maps Experience the Pearl Harbor attack: virtual war maps, survivors' personal stories, rare photographs, time lines, and more from National Geographic. ... www.nationalgeographic.com/pearlharbor/ - 24k - Cached - Similar pages
• Pearl Harbor Remembered main menu Pearl Harbor Memorial information. Map of Pearl Harbor. A brief overview of the attack. ... Additional source of information about Pearl Harbor. ... my.execpc.com/~dschaaf/mainmenu.html - 4k - Cached - Similar pages
• Pearl Harbor Raid, 7 December 1941 This page features a historical overview and special image selection on the Pearl Harbor raid by the Japanese, 7 December 1941. ... www.history.navy.mil/photos/ events/wwii-pac/pearlhbr/pearlhbr.htm - 21k - Cached - Similar pages
• Pearl Harbor Attack: Index of Action Reports Index to Action Reports and other special reports from the Japanese attack on Pearl Harbor, 7 December 1941. ... The Pearl Harbor Attack, 7 December 1941. ... www.history.navy.mil/faqs/faq66-1.htm - 13k - Cached - Similar pages [ More results from www.history.navy.mil ]
• Touchstone Pictures' "Pearl Harbor" - Official Web Site From producer Jerry Bruckheimer and director Michael Bay comes Touchstone Pictures'/Jerry Bruckheimer Films' Pearl Harbor, an epic saga of love and war. ... video.go.com/pearlharbor/ - 12k - Cached - Similar pages
• Pearl Harbor (2001) Pearl Harbor - Cast, Crew, Reviews, Plot Summary, Comments, Discussion, Taglines, Trailers, Posters, Photos, Showtimes, Link to Official Site, Fan Sites. ... www.imdb.com/title/tt0213149/ - 51k - Feb 5, 2005 - Cached - Similar pages
• USS Arizona Memorial (National Park Service) Oil droplets bubble to the surface of Pearl Harbor above the USS Arizona, creating a vivid link to the past. On a quiet Sunday morning ... www.nps.gov/usar/ - 21k - Feb 5, 2005 - Cached - Similar pages
• Attack At Pearl Harbor, 1941 ... Attack At Pearl Harbor, 1941. Printer Friendly Version >>>. The surprise was complete. ... Homer N. Pearl Harbor: Why, How, Fleet Salvage and Final Appraisal (1968). ... www.eyewitnesstohistory.com/pearl.htm - 27k - Feb 5, 2005 - Cached - Similar pages
• EyeWitness to History EyeWitness. has a new location: eyewitnesstohistory.com. You will transferred to our new site in a few seconds. Please bookmark the ... www.ibiscom.com/pearl.htm - 1k - Feb 6, 2005 - Cached - Similar pages
• Pearl Harbor - Mother of All Conspiracies The world's best webapge on the Pearl Harbor conspiracy and how Roosevelt committed treason to get America into World War II by encouraging a Japanese attack. ... www.geocities.com/Pentagon/6315/pearl.html - 65k - Cached - Similar pages

Now here's what was on MSN Search for unpaid listings:

• Attack At Pearl Harbor, 1941 ... Pearl Harbor: The Day of Infamy - An Illustrated History by Dan van der Vat Day of Infamy by Walter Lord Attack At Pearl Harbor, 1941 Printer Friendly Version >>> T he surprise was complete. The ... o www.ibiscom.com/pearl.htm o Cached page o 2/4/2005 *
• Pearl Harbor.com Guide to Pearl Harbor Hawaii ... Guide to USS Arizona Memorial, USS Bowfin Memorial and USS Missouri Memorial. A Pearl Harbor.com Exclusive... by George E. Elliott Jr. . Edited by David J. Castello Japanese warplanes rev up ... o www.pearl-harbor.com o Cached page o 2/5/2005
• National Geographic: Remembering Pearl Harbor--history, maps ... and review recommended resources. Register for Nationalgeographic.com E-Mail Newsletters Pearl Harbor, a Touchstone Pictures film starring Ben Affleck, opens May 25, 2001. Visit the official movie ... o plasma.nationalgeographic.com/pearlharbor o Cached page
• Pearl Harbor (2001) ... from New York City to Italy were enlisted to produce period shoes in sizes 10 and 11. (more) PEARL HARBOR Memorabilia Books All Products User Comments: 10 out of 13 people found the following comment ... o us.imdb.com/Title?0213149 o Cached page o 2/7/2005
• Battleship Missouri Memorial, Pearl Harbor, Hawaii Official home of the Battleship Missouri Memorial in Pearl Harbor, Hawaii. Battleship Missouri (BB-63) Memorial in Pearl Harbor, Hawaii - Official Site. Tickets and Reservations How to Get Here Read ... o www.ussmissouri.com o Cached page o 2/6/2005 * DVDs about Pearl Harbor from Pearl Harbor.org
• DVDs about Pearl Harbor from Pearl Harbor.org Home History Speeches Vintage Videos Photos Ships Forum ... o www.pearlharbor.org/center-dvd-pearl-harbor.asp o Cached page o 2/7/2005
• Pearl Harbor Raid, 7 December 1941 ... to World peace. This page features a historical overview and special image selection on the Pearl Harbor raid, chosen from the more comprehensive coverage featured in the following pages, and those ... o www.history.navy.mil/photos/events/wwii-pac/pearlhbr/pearlhbr.htm o Cached page o 2/6/2005
• Hawaii Activities - Oahu Activities - Oahu Surf - Oahu Tours - Hawaii ... Bring your kids and see the real Polynesia. AlohaTopTen.com - Enjoy the history and beauty of Pearl Harbor, fabulous luaus, evening shows, island tours or a sunset dinner cruise as the lights of ... o www.hawaii.com/visit/oahu/activities/index.php o Cached page o 2/5/2005
• Amazon.com: DVD: Pearl Harbor (2001) ... Dolby Digital 5.1), English, French (Dolby Digital 5.1) Journey to the Screen: The Making of Pearl Harbor History Channel show "Unsung Heroes" Faith Hill's music video "There You'll Be" DVD-ROM ... o www.amazon.com/exec/obidos/tg/detail/-/B00003CXTG?v=glance o Cached page o 2/6/2005

OK, what have we got that's the same? In this comparison, I'll also annotate a summary from my perspective on the pages shown.

• National Geographic: Remembering Pearl Harbor--history, maps: From National Geographic, what looks to be a good collection of material you'd like to see in the top results. Both Google and MSN list the exact same page, though MSN picks up the alternative address of plasma.nationalgeographic.com/pearlharbor.

• Pearl Harbor (2001): Pearl Harbor was also, of course, a Hollywood blockbuster. This is a page from the Internet Movie Database on the file. Again, MSN has a slightly different URL than what Google lists, but they both lead to the same exact page.  
• Attack At Pearl Harbor, 1941: What looks to be a good page with some eyewitness accounts of the bombing. Google actually lists this same page twice, both the current address and a page that redirects to the new location. MSN lists the page once, the redirection address that leads to the current page.

• Pearl Harbor Raid, 7 December 1941: Leads to a section at the US Naval Historical Center, which would appear to be a good choice. Google also lists a second page from this same web site -- different content, but still from the same web site.

What's unique at Google?

• USS Arizona Memorial (National Park Service): Official web site for the USS Arizona memorial.  
• Touchstone Pictures' "Pearl Harbor" - Official Web Site: Official movie web site of the film Pearl Harbor.  
• Pearl Harbor Remembered main menu: What looks to be a hobbyist web site about the raid. The info may be good; it's just the look and feel is not that of a professionally produced site.  
• Pearl Harbor - Mother of All Conspiracies: What looks to be a hobbyist site presenting an alternative view of the Pearl Harbor attack.

What's unique at MSN?

• Battleship Missouri Memorial, Pearl Harbor, Hawaii: Official web site for the USS Missouri memorial in Pearl Harbor.  
• Amazon.com: DVD: Pearl Harbor (2001): By the movie, from Amazon.  
• DVDs about Pearl Harbor from Pearl Harbor.org: What looks to be a page from a hobbyist web site about Pearl Harbor. This page is a list of related DVDs for sale, and the site feels a bit like it's oriented toward attracting people to sell products.  
• Hawaii Activities - Oahu Activities - Oahu Surf - Oahu Tours - Hawaii: Page from a tourist site on visiting Hawaii, and not a very good one about visiting Pearl Harbor, at that.  
• Pearl Harbor.com Guide to Pearl Harbor Hawaii: What looks to be a hobbyist guide to visiting memorials to the battle. POSTSCRIPT: Phil Ringnalda correctly points out than rather being a hobbyist site, this instead appears to be a site that may have lifted text from a National Parks Service site and is designed to drive traffic for hotel reservations across the US.  

So looking back on the unique items, I'd like all Google's choices. At MSN, the first one is good, but I like Google's choices better. The last four are more iffy. So verdict on the Trib's single query view? Again, never base overall conclusions on a single query, but...

MSN does feel more commercially oriented looking at this query and others I've personally seen. But that's more to me a sign it's a newer search engine that's still needs to evolve picking out the good stuff from things that might simply be designed to please a search engine's ranking algorithm.

The fact the technology seems to need more resistance to old-style on-the-page search ranking tactics is something that's come up on our forum and others I've read. SEO for MSN on our forums looks at this more.

Layout can have an impact. The Chicago Tribune reported seeing ads on Google and Yahoo for the query, just like those on MSN. But height and display of the first ads on MSN were larger than at the other two -- and thus fed to the impression that it's like all ads, all the time, at MSN.

I wrote about this before in my Microsoft Blogger Scoble Sticks With Google -- Though MSN Deserves A Break -- even getting out my ruler to measure ad height. Even if the ads are fewer, the look-and-feel will have a big impact on those viewing the site.

Even in the editorial results, you can see this reflected. Remember those Google results? Google also had these three additional results that came out of Froogle, which were integrated into the home page through a OneBox display:

• Remember Pearl Harbor - $18.74 - dvdplanet
• Pearl Harbor (History Channel) - $22.99 - pcmall.com
• Pearl Harbor (History Channel) - $22.99 - MacMall

Add those in, and Google results might feel a little more commercial than the Tribune original came away with. But those results looked a bit different from the "regular" results and so quite likely were completely ignored by the Tribune reviewer. That's good news for those searchers who really didn't want to get product information in this query. It's bad news for those who might.

That also points out the importance of search subjectivity. This reviewer probably wasn't thinking of Pearl Harbor in terms of a commercial perspective. But if you had wanted to purchase the movie or documentaries about the attack, your view of the results might similarly change.

Posted by Danny Sullivan at 3:05 PM | Permalink

January 13, 2005

Survey: Google Still Leads, But Competitors Closing Gap

In today's SearchDay, Yahoo & MSN Closing the Google Gap, Chris Sherman looks at a new study that finds while Google is maintaining its leadership position with searchers, Yahoo, MSN Search and Ask Jeeves have all made significant improvements over the past year and are narrowing Google's mindshare advantage.

Posted by Danny Sullivan at 8:39 AM | Permalink

January 5, 2005

Defining Search Relevancy

How do you measure the relevancy of search engines? That's a topic of discussion right now in one of our forum threads, What Is Relevancy. I've dropped a post with a lot of background material on the subject into the thread. Have thoughts on this tricky subject? Please come by and contribute.

Posted by Danny Sullivan at 9:17 AM | Permalink

See More Posts From:

This Week | This Month

 

• Send us feedback
|
• Technical questions or bug reports?
|
• Newsletter problems?
|
• Receive news via SEW RSS feeds

• Incisive Interactive Marketing LLC. 2009 All rights reserved.
| Legal Notices, Licensing, Reprints & Permissions |
• Privacy Policy
|
• Affiliates

var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www."); document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E")); var pageTracker = _gat._getTracker("UA-564586-7"); pageTracker._setDomainName(".searchenginewatch.com"); pageTracker._trackPageview(); window.collarity_appid = "incmedia"; //> //>

• Send us feedback
|
• Technical questions or bug reports?
|
• Newsletter problems?
|
• Receive news via SEW RSS feeds

• Incisive Interactive Marketing LLC. 2009 All rights reserved.
| Legal Notices, Licensing, Reprints & Permissions |
• Privacy Policy |
• Affiliates