Google In Controversy Over Top-Ranking For Anti-Jewish Site

NOTE: An earlier edition of this story filed on April 23 noted that the Jew Watch site had been removed from listings at Google.com. Google has since said this was due to the site having been removed from the web itself and not because of any overt actions on the part of Google. More on this below. The story has also been further updated to examine how the Jew Watch home page, rather than the entire site, currently appears to have been removed. More on this here.

Google has added a disclaimer to the search results that come up in a response for a search on jew, to counter complaints about an anti-Jewish site that until recently ranked number one.

The disclaimer appears at the top of the search results page, occupying either the banner-like space above Google's unpaid results that has long been used for paid advertisements or comes atop the paid ad area on the right-hand side of the page. It reads:

Offensive Search Results
We're disturbed about these results as well. Please read our note here.

The pseudo-ad (it even carries a Sponsored Link disclaimer) leads to an explanation from Google saying that they don't endorse the site.

The explanation also implies that because the word "jew" is often used in an anti-Semitic context, this had caused Google's automated ranking system to rank Jew Watch -- apparently an anti-Semitic web site -- number one for the query.

The Jewish Journal (and perhaps other sources) wrote about the query back in March, but the story picked up wider attention in April.

Risky Intervention

The result has left Google in a tricky situation. The company doesn't want to manually alter or rank the sites that appear for any query. Such "hand manipulation" might make it seem as if Google doesn't trust its own automated search algorithm. A likely bigger fear is that changing one query will leave Google open to pressure to change others.

As a result, Google was put in the strange situation of defending the Jew Watch site as coming up tops because Google's ranking algorithm supposedly "reflects the popular opinion of the web," as it told News.com.

As I wrote previously about the "miserable failure" search at Google, popular "opinion" suggests that the web en masse has voted to help make a particular site come up. Yet in this case, Google reports that only 55 sites actually link to Jew Watch.

That's popular opinion at work? Of course, there may be more links being counted, given the flaw (and more explanation of this here for Search Engine Watch members) of Google not actually reporting all the links it knows about.

It's probably well past time for Google to stop talking about its results as reflecting the web's opinion. Instead, it would be safer for the company to say that it tries to use a variety of factors to automatically determine what it hopes will be the best sites possible in response to a query. This system isn't perfect and might not necessarily reflect general opinion on the web or elsewhere for any particular query. However, it does work effectively for many people, for a large variety of queries.

Yeah, that's not a PR-friendly mouthful -- but it's more accurate.

Intervention In Removing, Not Altering Ranks

Of course, Google does indeed intervene manually in search results. It removes material because it may be deemed illegal, as was the case in the infamous chester guide search. The company also removes material in response to DMCA complaints and also because for spamming reasons, as this article explains further.

Such interventions make some marketers confused (or even livid) when they read Google's oft-repeated claims of no hand manipulation of search results. To them, such removals as I've described above are hand manipulation. You can get a flavor of such confusion in this recent WebmasterWorld forum thread.

These interventions are not specifically rank related. When they happen, Google doesn't try to reorder the ranking of how a page appears. Instead, it simply pulls the page from the index entirely. And if you aren't in the index, you naturally no longer rank number one. But to save confusion, it might be better for Google to be clearer in saying that they don't chose by hand which sites rank well.

By the way, I asked Google previously about the reference in a Wired article about wanting to "attach" better sites to queries to ensure it had good information available. I remember being disturbed by this, just as some in the aforementioned thread were, as it indeed suggested that Google was doing hand-ranking in some cases.

I was told by Google that this was a misinterpretation on the part of Wired. The Google engineer apparently meant that the Google search algorithm would be tweaked to produce better results, not that the results would be reordered by hand.

Apology, Potential Improvement, But Complications

In new developments, Google is apparently considering how it might somehow flag what it determines to be hate sites, perhaps as a means of warning its searchers.

This came out as part of an apology by Google to the Anti-Defamation League. But one now wonders if Google will find itself having to apologize to US president George W. Bush and his supporters because Bush still comes up first for miserable failure or to US presidential candidate John Kerry because of the latest prank that makes him number one for waffles.

The above is not said to downplay the serious offense many Jews and others felt when they learned of the result for the query "jews" on Google. It's intended to highlight just how complicated the situation is becoming for Google -- and potentially other search engines -- as the public takes a much closer scrutiny at search results.

It may be that Google will have to start indeed considering manual intervention into actual rankings, especially given that the latter two examples of Bush and Kerry have come about through overt attempts to make those results happen. But the danger Google could get into is also illustrated by what happened when AOL did this.

As I wrote earlier, AOL altered the results its gets from Google to manually remove the Bush listing from the top of the miserable failure search. That didn't rectify the problem for liberals like Michael Moore and US Senator Hillary Rodham Clinton, who'd been targeted by conservative linkers in the way Bush had been targeted by liberal linkers.

Where'd It Go?

When I started working on this story on Friday, April 23, the Jew Watch home page was the number one ranked site on Google for the query jew. The site had been bumped to the number two spot some days before but apparently rose back up. However, something completely new happened when I went to file around 5PM Eastern time that day. Jew Watch suddenly no longer ranked in the top results at all.

In fact, the entire site appeared to have been dropped from the index at Google.com. To verify this, I ran a special search to see all pages that Google.com knew about from the jewwatch.com site. No pages were listed.

Confusing the issue was the fact that the same search run via the Google UK web site brought back a reported 175 pages. For the site to have shown up in a Google UK search across the entire web but not in similar Google.com search suggested one of two things:

  1. Google had perhaps removed the site from its web index, and the changes had yet to hit the computers that serve queries to those based in Europe (where I'm based).
  2. Google had perhaps removed the site for those searching at Google.com, perhaps for some legal reason in the US, but decided the site could remain for those using Google's non-US web sites.

I was unable to get a clear answer from Google before filing that story. I've since been told by Google that they did not remove the site for any legal reason nor to quell the controversy.

Instead, Google says that the site was dropped from Google simply because the site was no longer online when Google's crawler last visited it. This was due to a campaign by activists to get the site booted from the web in general.

"Google did not blacklist or make any other manual change to intentionally remove the jewwatch.com website from our index. It does not currently appear in Google's search results because the website was offline for a number of days last week. In our most recent crawl of the web, we were unable to reach the jewwatch.com website, therefore it was not included in our index. Now that the site is back up again, it's likely that at some point soon, jewwatch.com will re-appear in Google. You'll also find related information about the jewwatch.com website being taken offline here: http://www.removejewwatch.com," spokesperson David Krane emailed.

When updating this story with Google's quote around 5AM Eastern time on April 24 (though based in the UK, I'm using Eastern time as a reference point), I went back to recheck the situation. I found that Google.com now brought back a reported 175 pages listed -- the same as when checking the Google UK site.

NOTE: In an earlier version of this story, I'd mentioned that my check at Google.com at 5AM on April 24 first brought back only the Jew Watch home page in the Google index as a partially-indexed listing, then when I immediately rechecked, I found a reported 175 pages. The reason for this discrepancy turned out to be mistake on my part. The first time, I searched for pages from jewatch.com, a misspelling, not jewwatch.com.

Just The Home Page Gone

Since I wrote my original article on April 23, censorship writer Seth Finkelstein concluded that Google had blacklisted only the Jew Watch home page.

To be clear, my early searches found that ALL pages from the Jew Watch site had been removed at Google.com. There was no reason to assume that just the home page might have been targeted when no pages at all were showing at all.

I quickly reviewed the searches that later brought back a reported 175 pages from Jew Watch from Google.com on April 24, to see if the home page was within them. I thought it was. Here's what I saw:

As it turns out, that highlighted URL is not the home page. To illustrate, look at these three (of at least six different ways) the home page might have its URL possibly be shown by Google:

  • http://jewwatch.com
  • http://www.jewwatch.com
  • http://www.jewwatch.com/

The strange looking URL for what I thought was the home page -- http://www.jewwatch.com/" -- is the URL with ASCII code for a quotation mark after it. In other words, it's this:

  • http://www.jewwatch.com/"

It's very similar to what the home page listing might look like, but it's actually a completely different page. So, Finkelstein is correct that the home page doesn't seem to appear within the Google index, at the moment.

Google said earlier it has done no banning, that this was a natural consequence of the site being down.

So why might the home page be gone while other pages from the site remain? I've sent a question to Google about this. Meanwhile, one possible explanation that comes to mind.

It's not unusual for Google to revisit some pages at web sites, especially home pages, more frequently than others. It's possible that Google was revisiting the Jew Watch home page more often than other pages within the Jew Watch site.

When the entire site went down, Google may have tried to revisit the home page and couldn't find it. This could have led to the home page being dropped. While the page was technically gone, Google could still see active links to it. Meanwhile, since the internal pages hadn't been revisited, Google never noticed they were offline. They remained in the index.

That's the possibility. But for the same reason, you'd have expected the Jew Watch home page to be back by now. If it was so often revisited and updated within the index that it was dropped before other pages in the site, it should have seen a similar fast return to being indexed.

Blogger Arthur Guray, who has been writing about the issue, tells me that the Jew Watch web site was down from April 16 through April 22. During that period, it apparently retained its listings in Google, as well as either the top or a first page ranking for "jew."

This means that it took Google about seven days before it apparently dropped the Jew Watch home page on late April 23. Search engines generally don't drop pages right away, specifically to help avoid problems if a site goes down temporarily.

It could be just coincidental that the site came back up right when it came to the end of the grace period Google may have allowed it. Again, I have a follow up question out about this.

Not Google Bombing & Better Search Commands Needed

Interestingly, there's no suggestion that Jew Watch was involved in a Google bombing campaign to achieve its position. Finkelstein points this out, and I agree. The site appears to have gained its prominence naturally, through a combination of on-the-page factors and link analysis.

How much a role links play is impossible to accurately measure. Searchers are simply not provided with the tools we need to do this by the search engines, something I hope will change.

As said earlier, Google only reports 55 links to the Jew Watch home page, but more links may be involved. After all, Yahoo reports 839 links to the page. Yahoo's index is on par in size with Google's, so it's fair to say Google knows about hundreds of URLs linking to Jew Watch that it doesn't report.

What you really want from Google (and all search engines) is both the means to accurately count the number of links to a particular page and the ability to restrict that count to links that contain particular words. Call it a "link context" command. This combination is something that only AltaVista has ever offered. In the past, you could do a search like this at AltaVista:

url:www.jewwatch.com anchor:jew

With that search, AltaVista (supposedly) would show you all the pages linking to Jew Watch and using the word "jew" in the links. I say supposedly, because this didn't always seem to work well at AltaVista.

Visiting AltaVista when writing this, I noticed that the anchor command is no longer offered. Gary Price tells me it was withdrawn as AltaVista was shifted over to using results from Yahoo, which owns the site.

Google Bombs Blow Up Elsewhere

While Jew Watch didn't get its top ranking thanks to Google bombing, an active campaign to push it out has moved a Wikipedia entry into the top spot at Google and Yahoo-powered AltaVista, AllTheWeb and MSN Search (at Yahoo itself, Jew Watch is listed on the top of the SECOND page of results, so effectively invisible to searchers).

At Yahoo, the Wikipedia entry comes at the bottom of the first page of results. As for Bush, he's still number one at Yahoo for miserable failure, and Kerry is number one for waffles.

These examples demonstrate as I've written before that Google bombs have an impact on other search engines, not just Google -- though I have to note that flavor of link analysis used by Ask Jeeves (powered by Ask-owned Teoma) seems more resistant. Searches for waffles and jew don't show the bomb impacts seen in Google and Yahoo-powered results, though a search for miserable failure does.

I think it would be more accurate to call Google bombing instead link bombing or search bombing. And with such bombs going off, why not just stop using link data entirely? Because it's still incredibly helpful.

Instead, the solution to link bombs, as I wrote recently, is likely to come in the form of personalization and specialized search databases.

Perhaps we might also see Google take action against Google bombers themselves, something the company hasn't yet done. Notice in Google's explanation where it says:

The only sites we omit are those we are legally compelled to remove or those maliciously attempting to manipulate our results.

The malicious part is a reference to search engine spam, those trying get to the top for some personal gain. But those involved in Google bombing campaigns -- let's make Bush tops for miserable failure by linking; let's make Kerry number one for waffles -- could be construed as maliciously attempting to manipulate Google's results, as well.

Solution? Those taking part in such campaigns perhaps might find their pages containing such links yanked. More likely, you'll see Google and the other search engines look for new ways to discount or minimize the impact such campaigns have.

Search Headlines

NOTE: Article links often change. In case of a bad link, use the publication's search facility, which most have, and search for the headline.

aQuantive Developing Rich Media Tool...
ClickZ Today Apr 26 2004 12:08PM GMT
Yahoo gives Messenger an 'all new' look...
CNET Apr 26 2004 11:46AM GMT
TOP 10: 'Portal' sites...
Netimperative Apr 26 2004 11:09AM GMT
Google to defend ad sales practice in French court...
Forbes Apr 26 2004 11:05AM GMT
Google Vs. Yahoo...
Forbes Apr 26 2004 10:23AM GMT
Yahoo's Compelling Cash Flow Growth...
Forbes Apr 26 2004 10:23AM GMT
Google Chooses IPO Underwriters...
Forbes Apr 26 2004 10:23AM GMT
Google hit by lawsuits...
New York Daily News Apr 26 2004 9:53AM GMT
Google's Goal: 'Understand Everything'...
Business Week Apr 26 2004 9:18AM GMT
What Eric Schmidt Found at Google...
Business Week Apr 26 2004 9:18AM GMT
Google's Gmail Is Great -- but Not for Privacy...
Business Week Apr 26 2004 9:18AM GMT
Gotta go, Google?...
USA Today Apr 26 2004 9:13AM GMT
Interest Mounts as Google Disclosure Nears...
New York Times Apr 24 2004 3:08AM GMT
Google revisits policy on hate sites...
San Francisco Chronicle Apr 23 2004 12:45PM GMT
AOL Opens More Doors to Its Search, E-Mail...
PC Magazine Apr 23 2004 10:58AM GMT
powered by Moreover.com