Declan McCullagh posts an update to his great earlier story looking at how Google is censoring results at China. I talked with him a bit about this today and actually found myself even more upset over what Google's doing. That's because rather than just censoring what China is telling them to block, Google's actively coming up with its own list.
My assumption had been Google and other companies were given a list of sites by China to block. I'm actually still waiting to hear back from Google to talk more about what exactly they are doing. But Declan writes in his original story:
China's government has an extensive Internet filtering process in place that controls which overseas Web sites its citizens can access. (A 2005 study by the Open Net Initiative called it "quite thorough.") With that filtering as a guide, foreign companies are expected to build their own lists of Web sites to delete from Chinese search listings.
Got it? China will give the search engines some advice on censoring and guidelines, but it remains up to the search engines to do the actual dirty work.
If Google was going to cave in China and create evil Google, at the very least they could have pushed back to say they'd block specific sites given to them. Instead, they seem to be blocking a combination of known sites plus other sites that might have objectionable material, all based on what they decide themselves.
In particular, as Declan notes, they may have been relying on a modification of their SafeSearch filter used to keep children from stumbling into porn. Feed that filter some things that are considered "bad" and then it will keep the bad stuff away. But bad needn't necessarily be porn. Bad could be potentially "subversive" material.
If this is what's happening -- if censorship in China from Google means a combination of blacklisted sites but automated filtering -- then it explains some of the disclosure failures that Declan found earlier.
Google's disclosure system in China is almost certainly based on the same system they use for DMCA disclosures in the US (for more about this, see my Google Now Censoring In China story). That system seems to work by looking for URLs that have been knowingly removed from the results and inserting a message that they were pulled at the bottom of the search results page.
By the way, Google does NOT do this type of disclosure in France and Germany to my knowledge, despite what I've seen the company say, such as cofounder Sergey Brin yesterday:
In France and Germany there are Nazi material laws. One thing we do, and which we are implementing in China as well, is that if there's any kind of material blocked by local regulations we put a message to that effect at the bottom of the search engine. "Local regulations prevent us from showing all the results." And we're doing that in China also, and that makes us transparent.
I've looked at the removals in France and Germany for well over a year now. I've never see such messages coming up to notify anyone that material was pulled.
While there's a mechanism likely in place to handle specific URLs and sites that are pulled, there's nothing likely in place to indicate material removed by an algorithmic based filter. For example, do a SafeSearch search at Google, and it doesn't tell you that anything in particular has been removed. It simply says "Safesearch on" next to the search results count and a message such as:
The word "porn" has been filtered from the search because Google SafeSearch is active.
So maybe this keyword-triggered notification is working in some cases but failing to go active if a site is removed. In other words, the message may be showing up if you search on a forbidden word. But search for something "innocent" but where the filter still pulls out a site automatically, one that wasn't on a specific blacklist, and that might not trigger a notification.
The easy solution is to scrap the filtering and publish a list to everyone of all the sites that are on the blacklist. That would be transparent. A better solution would be not to do the filtering at all.
No filtering? But then Google can't be in China, where it will still help all those people who want access to at least some information. So what. Why does Google need to be in China? It's not like people in China can't do these types of searches on other search engines such as Baidu? It's just that by not being in China, Google itself won't make money. It might miss out on the big Chinese market that so many investors want to tap into.
Google being in China helps itself more than China and simply does not fit into the "Don't Be Evil" mantra we've been spoon fed for several years now. As a reminder, here's what Google told investors about that in its IPO filing:
DON?T BE EVIL
Don?t be evil. We believe strongly that in the long term, we will be better served?as shareholders and in all other ways?by a company that does good things for the world even if we forgo some short term gains. This is an important aspect of our culture and is broadly shared within the company.
Google users trust our systems to help them with important decisions: medical, financial and many others. Our search results are the best we know how to produce. They are unbiased and objective, and we do not accept payment for them or for inclusion or more frequent updating. We also display advertising, which we work hard to make relevant, and we label it clearly. This is similar to a newspaper, where the advertisements are clear and the articles are not influenced by the advertisers' payments. We believe it is important for everyone to have access to the best information and research, not only to the information people pay for you to see.
It's hard to trust a system that's widely censored. It's difficult to see how they are unbiased when widespread censorship makes them inherently biased. Why is paid inclusion so evil that it must be named as a bad thing in the IPO. Because it involves payment? If payment is so bad, then caving into widespread government censorship is a form of paid exclusion that should similarly be avoided.
Well heck. It's not that much material involved in the paid exclusion. After all, when Google chose to censor Google News in China, we were told that it was less than two percent of the sources out there. That's tiny!
Yep. And Yahoo has said that only 1 percent of its index is made up of paid inclusion material. But that tiny amount is enough for Google to view Yahoo as tainted. Evil, as it were, isn't a percentage game. It's about the principles involved.
Postscript: Looking for a search at Google Germany today on "nazi," I see this notice now at the bottom of the page:
Aus Rechtsgründen hat Google 1 Ergebnis(se) von dieser Seite entfernt. Weitere Informationen über diese Rechtsgründe finden Sie unter ChillingEffects.org.
This looks new. I don't recall seeing it earlier this week. A similar search on Google France brings up no message like this.
Over at Google Canada, a search for "jew" does not bring up the Jew Watch site that was removed there in 2004 for legal reasons, Google's told me.
FYI, I don't like the sites that have been removed -- I'm simply pointing out they are gone without disclosure.
Postscript 2: Google's just added a very good post explaining their side of the argument for being in China on the Google Blog.Postscript 3 from Gary For a look at what's often referred to as the Great Firewall of China, a BusinessWeek article that we blogged two weeks ago (before this week's news about Google.cn) is worthy of your attention.
Agree, disagree, just like to talk and discuss things? Feel free to comment on this issue in the Google Agrees To Chinese Censorship at our Search Engine Watch Forums.