Search engine penalties are present and pervasive, and are a primary method used by search engines to control webmasters. Unless webmasters understand what they are and what to do about them, their websites could easily trigger a penalty, losing traffic and revenue.
Coping with Search Engine Penalties Search engine penalties are present and pervasive, and are a primary method used by search engines to control webmasters. Unless webmasters understand what they are and what to do about them, their websites could easily trip a penalty, losing traffic and revenue.
Do you remember the old days of brick and mortar stores? Imagine the amount of work required to set up a store—locating a space, fixing it up and arranging the location, purchasing inventory, promoting the store, waiting for customers to walk in.
For e-tailers, this seems an antiquated method and one that has been surpassed by the ability to engage in global businesses through the internet. However, the search engines are trying to rein-in e-tailers and align our way of doing business closer with the brick and mortar approach.
To do this, search engines wait a few months before giving new websites visibility and good ranking in search results, which translates into profits for a web site. They want online merchants to only have a few stores that we can dedicate time and attention to. If we have too many stores and don’t pay enough attention to each, all of them could flop.
In this two-part article, I’ll discuss penalties I’ve observed and discussed with other search engine optimizers. I’ll also show how to tell the difference between perceived and actual penalties, and most importantly, how to keep your site out of the penalty box in the first place.
Many people assume that their site has been penalized if the site experiences a loss in search engine rankings. However, many of these perceived penalties are in fact a reflection of other changes or issues that may be taking place with the search engines or even within a webmaster’s own site.
For example, many people recently posted messages in the Search Engine Watch Forums stating that their sites had been penalized, as they were no longer ranking well. After further research, the search marketer discovered that the problem was that his host had banned Google from crawling all of his clients’ sites. Since the pages weren’t accessible to the search engines, the search engines dropped the pages from the database. This is an example of what many thought was a penalty, but which was in fact the fault of the web host.
Other perceived penalties include:
Before assuming your site has been penalized, do some thorough research to understand what is really going on. Research your logs to determine if there is any unusual activity. Call your host to determine if they are aware of any problems. Check your robots.txt file to make sure it’s not disallowing search engine crawlers.
Also, spend some time analyzing your search engine result pages (SERPs). How are your competitors doing? Do they appear to have recently been actively promoting their sites? Have they acquired more links? Compare their site with yours and make sure your site still follows the guidelines necessary to rank at the top for your preferred search terms. If not, basic search engine optimization should your first step, rather than imagining your site has been penalized.
While some penalties are imaginary, search engines do appear to give actual penalties. Some seem fair, such as when a site “pushes the limits,” of what’s considered acceptable. Other penalties may not seem fair at all—especially if your site is the one that’s been penalized. Your concern is to determine whether or not your site has a penalty, and if it does, what to do about it.
The difficulty with search engine penalties is that it’s impossible to determine with certainty whether or not your site does have a penalty, and what the penalty is. Why? The search engines don’t publicly acknowledge the existence of penalties, and won’t generally respond directly if asked whether a site has been penalized. Therefore it’s impossible to offer a hard-and-fast rule that can help you look for penalties. Likewise, there are no all-encompassing methods to remove the penalties from your site.
Also, remember that search engines use many criteria to determine ranking, and these factors change constantly. For example, in a recent Google patent, other ranking factors mentioned include user and history data. In general, longer-established sites are favored over newer sites. Popularity, as measured by the number of people have bookmarked your site or how many people click on your link when it appears in the SERP’s are also important. If any of the history or user-generated data is not favorable to your site, you may not have a penalty but just may not rank well.
The best I can do is give you some general guidelines or ideas to help you get started in identifying what’s wrong with your site and how you can fix it. Also, as every search engine has a unique algorithm, they have different systems for applying and removing penalties, so we have to review each search engine separately.
Google has the most advanced automatic penalty system of all the major search engines. They have multiple filters/penalties in place that may get automatically applied and removed.
Most of Google’s automatic penalties are applied on the basis of percentages. For example, some sites may be penalized for buying site wide links, while other sites can get away with it. Why? The penalty is triggered depending on the number and quality of incoming links to each site.
Let’s assume Site A has inbound 1000 links from 800 different domains. Site B also has 1000 inbound links, but all of the links are coming from one domain. Site A can get away with buying a couple of small site wide links, while Site B will get penalized for doing so. This type of filter ensures sites that have legitimate reasons for applying this technique are protected, while sites that don’t will get penalized.
Whenever you’re studying any potential penalties, it’s important to keep this idea of percentages in mind, while also understanding that the percentage threshold varies according to the SERP and keyword category. While having 60% of site wide links for one site may be okay, the same percentage may take another site down.
There are no set numbers as each SERP, based on the number of competing pages, has different filters in place. However, it’s always a good idea when dealing with percentages not to surpass 60% density (of links from one domain, anchor text, etc).
This also applies when looking at links from bad neighborhood or scraper sites. A legitimate site will have a lot of valuable, natural inbound links. As a result, scraper sites will pick it up and also link to the site. However, as the site would already have a lot of valuable, natural links, this wouldn’t damage the site.
If a site only has inbound links from non-valuable, scraper sites, then the site won’t rank well, as it wouldn’t meet the algorithmic requirements to have search engine prominence.
Again, it’s all a matter of percentages. As a webmaster or SEO, it’s important to research your SERPs and gain a clear idea of what percentages are applicable for your target keywords.
Here are some penalties that I’ve seen applied by Google in my own search engine optimization efforts. Again, Google has neither confirmed nor denied that they apply these penalties—rather, I’m describing behavior I’ve observed.
Site Wide Links
If you buy a link on every page of one website, and 80% or more of your inbound links are coming from just one or two domains, you could easily get penalized. These links can be valuable if—but only if—you have enough inbound links from other domains to counteract the effect.
Anchor Text Density
If too many sites link to your site using the same keyword, you may get an anchor text penalty for that particular keyword. For example, if your site sells computer parts, and you buy a few links and engage in a reciprocal link campaign always using the same words in the link text, the anchor text density may be too high. Consequently, your site won’t rank well for that keyword, as the penalty is associated with the keyword, not the site. Your site may rank well for other keywords that have a more realistic density, just not for the keyword with the high density.
Shared IP Address
If your site shares an IP address or a Class C address with a site or group of sites that Google has identified as spammers, Google may have applied an IP or Class C penalty so every site in that IP or Class C domain suffers. Check other sites in your IP or Class C address to determine if they are “healthy” and ranking in the engines with adequate promotion.
The Sandbox
The notorious sandbox is a filter based on the date a site was registered and when it started acquiring links. This filter varies, depending on the keyword sector, quality of the site, and quality of its inbound links. Some sites get out of the sandbox more quickly than others. This is based on how many competing pages there are for the site’s primary topic, how relevant the content is, etc. The sandbox is less of a penalty than it is a filter, designed to prevent webmasters from creating hundreds of sites and polluting Google’s data centers with irrelevant content.
Over-Optimization
We started seeing this penalty after Google’s “Florida” update in 2003. Google started looking for obvious on-page optimization, where the title, keywords, description, URL and anchor text used identical or closely matching keywords. If detected, a site would be penalized.
Network Interlinkage
For people who own multiple sites, this penalty can be devastating. Basically, this prevents people from interlinking all their sites together. If they do, this creates a closed web map, a loop of sites closely connected without other websites linking in to this network. This creates a closed loop and Google assumes it’s a network used to boost rankings, and applies penalties accordingly.
Links from Bad Neighborhoods
If your site has many links from sites that have very low quality, your site could incur penalties for being associated with bad neighborhoods. This prevents people from acquiring links carelessly, and makes the link gathering job more difficult and time consuming. Also, it prevents people from doing blind link exchanges and listing their site with free-for-all pages.
As webmasters cannot control who links to them, these penalties mainly occur if there is some reciprocity between good and bad sites, or if a large percentage of the links come from sites that don’t have value.
Multiple Redirects
If your site is set up solely to redirect visitors to another website, this will generally incur a penalty. Many websites are simple pages with the sole purpose of attracting visitors to send to another site. This technique is popular for affiliates, who create optimized pages with their affiliate codes, and then automatically redirect visitors to the target affiliate site. The search engines don’t see this as adding unique content to their database and may penalize or even ban these sites.
Non-relevant Content
Many people optimize for non-related keywords to try to increase their traffic. For example, if your website is about weddings and you optimize it for Britney Spears in hopes of bringing more people looking for the pop diva to your website, you’ll likely draw a penalty. The search engines want the users to find whatever they’re looking for, and will penalize sites that prevent them from achieving this goal.
Gibberish Content
To create more pages, people will create gibberish pages optimized for their target keywords. These pages are computer generated and the content will often make no sense to users. Google is implementing many methods using semantic analysis and is increasingly capable of determining whether or not sentences have meaning. This enables them to penalize sites, or pages, that use this technique.
Blog and Guestbook Spam
Another automated way of inflating link popularity is to employ blog and guestbook messages that are little more than spam. Google can detect this by examining commonalities in all the links to the site. However, there is evidence that a few sites are still getting away with this technique due to the percentages issue mentioned earlier.
Renting Space
Lately, Google has been frowning on the technique of “renting space” on other people’s sites. A few high profile sites have been caught selling sub domains and sub pages, which has resulted in search engine penalties to those sites. I have also seen evidence of sites being dropped entirely from Google for allowing other commercial websites to rent space on their sites.
Ways of engaging in this technique effectively include buying space in links that have similar topic to that of your site’s. Also, try to make sure that the content matches the topic of the site on some semantic level. Otherwise, if there is no semantic connectivity, it shows to Google that the content is external and may cause a penalty for the page, or for the site.
Integrating the content into the body of the page is also a great way to rent space, as it would fit in with existing content and would only dilute the semantic theme in a minor way that wouldn’t cause penalties.
I believe this is mainly a manual penalty and applied by human editors who review the page and determine whether or not the page is paid or is a natural part of the site.
Buying High Profile Links
To increase PageRank and search engine visibility, many sites buy links from other high profile sites. To discourage this manipulation, Google implements a method so that sites buying links receive a slight penalty from these links, causing a temporary reduction in rankings. The sites selling the links don’t incur penalties, but do incur a modifier on the site which prevents PageRank from being passed.
Buying high profile links is still one of the best techniques available to SEOs. However, it is essential to select links carefully. Are they relevant to your site’s theme? Are the high profile links considered authorities? Do they appear in the backlinks of sites that they link to? Do they appear in the backlinks of sites ranking well in your industry? Evaluate the sites carefully and make sure it appears as though it were a standard media buy and not a deliberate page rank manipulation.
Duplicate Content
This penalty is pervasive and applies to many sites. This penalty is applied when a large variety of the text on one page is replicated on another page. The page with the lowest PageRank will get a penalty.
There are many other ways to incur this penalty. Based on how you set up your site, you may incur this penalty if your site isn’t configured for the www.site.com and site.com to redirect to the main site. If you have multiple domains: site.com, site.org, site.net, and these aren’t set up correctly, you may acquire duplicate content penalties. If other people copy your content and they have higher page rank, your site may be the one to receive the penalties. To check how many sites have the same content, search for snippets of content that are unique to your site, with ” ” around the words, to see how many other sites come up for the query.
Competitors Reporting your Site as Spam
If you are engaging in questionable tactics, your competitors may report your site to Google, which could then trigger either a manual review or eventually, an automatic penalty.
Manual Penalties
Sometimes, if a site not only bends but outwardly breaks the rules, Google will ban the site and remove it from its database.
My theory is that there are many filters that “trigger” a manual review. A human looks at content that triggered an alert, and determines the degree of the infraction. If it’s severe enough, a page or in extreme cases an entire site will get deleted from Google’s database.
Your site may have been penalized by Google if:
Google offers specific recommendations in its guidelines for webmasters:
These quality guidelines cover the most common forms of deceptive or manipulative behavior, but Google may respond negatively to other misleading practices not listed here (e.g. tricking users by registering misspellings of well-known websites). It’s not safe to assume that just because a specific deceptive technique isn’t included on this page, Google approves of it. Webmasters who spend their energies upholding the spirit of the basic principles listed above will provide a much better user experience and subsequently enjoy better ranking than those who spend their time looking for loopholes they can exploit.
Most of Google’s penalties are automatic, and once you remove the offending factors, the penalties will be automatically lifted. You may have to wait at least a month, but at least the penalties will be removed. To speed up the process, you can also search for high quality, trusted sites to link to your site. This will give your site TrustRank points and help your site recover quicker and more effectively.
Once you’ve identified and fixed the problem, you should write to Google and ask them to help you fix the problem. Use the web form set up for this purpose at http://www.google.com/support/bin/request.py.
Tomorrow I’ll take a look at the search engine penalties applied by Yahoo and MSN.
Marcela De Vivo is a search engine optimization consultant and the owner of SEOmind.com.