Many advertising agencies, design firms, and even web hosting companies offer search engine marketing services for their customers. However, some of the methods they use to obtain top positions are considered to be spam by the major search engines. How do you tell the difference?
A special report from the Search Engine Strategies 2003 Conference, August 18-21, San Jose, CA.
Many advertising agencies, design firms, and even web hosting companies offer search engine marketing services for their customers. However, some of the methods they use to obtain top positions are considered to be spam by the major search engines. How do you tell the difference?
Search engine marketing (SEM) can be difficult, even when planned for from the beginning. Making the problem worse is that sites are often inherited and are a mess as far as SEM is concerned. Is a site not ranking due to a technical problem? Has it been banned for bad behavior? Where do you start? The panelists at the Cleaning Up the Mess session examined this issue and provided some guidance for those dealing with this problem.
“Learn what looks funny; use simple forensics right there in your browser to spot dead giveaways,” advised Anne Kennedy, Managing Partner at Beyond Ink LLC. “If it looks or sounds funky, it probably is.”
Kennedy said to watch out for companies that create doorway pages, even if they do not specifically call them doorway pages. “For example, avoid companies that put red-flag clauses like this one: Creation of 300 ‘Directory Information Pages’ (D.I.P.). Each D.I.P. will consist of a unique URL and domain. Each D.I.P. will link directly to Client’s web (to) specifically target the Client’s ‘Key Words’ and ‘Key Word Phrases.'”
Other red-flag phrases include attraction pages, envelope pages, gateway pages, channel pages, and hallway pages. “Except in pay-per-click (PPC) search engine advertising,” Kennedy said, “all optimization work should be done to your site, not the sites your vendor owns.”
Before assuming a site has been banned from a search engine, all panelists recommended looking at possible technical reasons for a site disappearing from a search engine index, such as a robots.txt file, a new URL structure, session I.D.s in query strings, .htaccess files, and secure pages.
“One of our clients actually secured their entire site,” said Matt Bailey of the Karcher Group. “Webmasters can also block access to the entire site if a robots.txt file is used improperly.”
Some people confuse being “in” a search engine with being “ranked” in a search engine. Web pages can be “in” a search engine index without ranking well. However, web pages cannot rank well without being included in the search engine index.
Therefore, one of the first steps to seeing how well a site is performing in the search engines is to check how many pages are actually in the index. Both Shari Thurow, Marketing Director at Grantastic Designs, and Bailey presented ways to determine the index count at the different search engines (substituting your domain name for company.com).
Google:
site:company.com company.com
or
inurl:company.com or inurl:www.company.com
Inktomi:
domain:company.com
AlltheWeb:
url.all:www.company.com
or
site:company.com
AltaVista:
url:company.com
Teoma:
site:company.com company.com
If pages from your site are included in the search engine index over a period of time, then it probably has not been banned.
Shari Thurow described a “mess” she had to clean up for a large medical portal site, MedicineNet.com. MedicineNet is an online healthcare media publishing company with over 22,000 pages of content. They are the publishers of Webster’s New World Medical Dictionary from Wiley. The site receives over 1.5 million unique visitors per month.
“Actually, when MedicineNet first came to me, I had a difficult time believing that their site had been banned in Google; the content was too good,” she said. “Most of the time, a site will disappear from the search engines due to a technical reason, such as a spider requesting pages from a server during the time a server is being rebooted or upgraded.”
What she found was that MedicineNet had actually been banned at Google due to the optimization strategies from previous marketing firms. “These other firms purchased over 90 different domain names and tried to get all of these domains listed in the directories, even though these new domains had identical content to the main MedicineNet site,” she said. “Well, Google discovered this due to the cross-linking structure, and sure enough, the site was banned.”
Kennedy offered some additional advice on this topic. “Check the backward links to your web site,” she said. “The companies that link to your site can be an indication of spam, based on the URL structure.”
“That is how we truly determined that the site had been banned at Google. All of the links were eliminated from the index, along with the main site,” said Thurow. “To top it all off, when I filled out Google’s spam report form, one of the representatives replied and said the site had been banned.”
To check for links to your web site, panelists suggested performing the following queries (again, substituting your domain or URL for company.com):
Google:
link:company.com
Inktomi:
linkdomain:company.com
AlltheWeb:
link.all:www.company.com
AltaVista:
link:company.com
To get MedicineNet back in Google, Thurow said, she had MedicineNet place 301 redirects on all of the 90+ domains. A 301 redirect communicates to the search engines that a page has been moved permanently. After that, she communicated to Google that MedicineNet had followed all of the guidelines for inclusion into their index, and asked that the site be respidered.
“Google and all of the search engines will definitely check to see if the banned site follows all of the guidelines,” she said. “Since MedicineNet contains over 22,000 pages, it took a few months for Google to re-index the entire site, after they were given the green light. It took almost six months.”
To contact the search engines directly regarding a possible spam penalty or to dispute a possible spam penalty, Thurow recommended using the following email addresses:
Google:
Inktomi:
AlltheWeb:
Teoma:
But the mess is far from complete. “Unfortunately, a lot of sites [including directories” still link to the former spam domains,” Thurow explained. “Even with the 301 redirects, we still have to find all of the sites that link to former spam domains and request a link modification. Though this presents an opportunity for MedicineNet to get better descriptions and links back to their site, it will take a long time to get everything in place. ”
Anne Kennedy summed up the session by stating, “Avoid being a spam penalty waiting to happen; the rules are there in plain view at Google and Inktomi.”
Google’s Submission and Spam Policies
http://www.google.com/webmasters/
http://www.google.com/contact/spamreport.html
AlltheWeb’s Submission and Spam Policies
http://www.alltheweb.com/help/webmaster/faq.html
http://www.alltheweb.com/info/about/spam_policy.html
Inktomi’s Submission and Spam Policies
http://www.inktomi.com/products/web_search/spampolicyfaq.html
http://www.inktomi.com/products/web_search/guidelines.html
AltaVista’s Submission and Spam Policies
http://www.altavista.com/help/search/faq_web
http://addurl.altavista.com/addurl/new#rls
http://www.altavista.com/help/contact/search
Craig Fifield is Product Manager for Microsoft bCentral’s Small Business Web site optimization and submission service, Submit It! and their web site statistics package FastCounter Pro.