Hosting Issues and SEO

By Mark Jackson , September 16, 2008

Every time a user or search engine spider visits a Web page, the server that page resides on returns a code. These codes show whether the page loaded correctly, was moved, or if there's a problem. Understanding these Web server responses to user and search engine requests is a critical aspect of SEO and Web site usability. Relay the wrong response, and you could create a world of hurt for your Web site in the search engines, or have some dissatisfied users on your hands.

Basic Server Response Codes

There are many different response codes, many that you will most likely never have to worry about. But there are a few that will keep coming up, which have the potential to make your life easier or more difficult. Here are the definitions of the basic server response codes.

200 -- OK. The request has succeeded. The vast majority of all requests should generally end with this response. In layman's terms, this means the request was understood, accepted, and the page you requested will appear.

301 -- Moved Permanently. The requested resource has been assigned a new permanent URL and any future references to this resource should use one of the returned URLs. Clients with link editing capabilities ought to change the existing URL to the new URL if it happens to rank for a given keyword. If the URL doesn't rank, and if the URL structure has many dynamic parameters, it may be wise to rewrite the URLs (using either ISAPI rewrite or Mod_Rewrite) to include keywords within the URL string.

When undertaking any rewrites of URLs, you should use 301 redirects to ensure that the SEO value of the existing URL is passed along to the new URL. In summation, this means the document has permanently moved to a new location, and search engines should reference the new location.

302 -- Found. The requested resource resides temporarily under a different URL. Since the redirection might be altered on occasion, the client should continue to use the old URL for future requests. This way, the search engines cache the old URL and not the new one. This keeps any rankings the page may have until you decide where you might want to permanently redirect this page. This is commonly referred to as "temporarily redirected," which means the document has moved to this location for a while. Search engines don't forget about the old location.

404 -- Not Found. The server has not found anything matching the request URL. No indication is given indicating whether the condition is temporary or permanent. The 410 (gone) status code should be used if the server knows that an old resource is permanently unavailable and has no forwarding address. This status code is commonly used when the server doesn't wish to reveal exactly why the request has been refused, or when no other response is applicable. See the common pitfalls below on how the misuse of these redirects can impact SEO efforts.

For more information and definitions on server response, check out the status code definitions on the W3.orgsite.

Common Pitfalls

Below are some of the common pitfalls in using server response codes, as this relates to search engine optimization:

Using 302 temporary redirects when the change is permanent. When you've changed the URL of a document and have no plans to change it back, you should use a 301. Using a 301 redirect will help you pass along any value/rankings that the legacy URL had. That being said, some search engines do better than others in following the 301 redirect. If your experience is anything like ours, you will find that Google does a fine job of associating the two, though Yahoo will take a while to fully value change.
Custom 404 pages resolving to a 200 OK server response. When a page can't be found on some content management systems, it should provide a custom 404 page letting the user know that the page no longer exists and pointing out main sections of the site. While this is great for the user experience, serving these pages incorrectly can cause a bunch of these "page not found" pages to be indexed in the search engines. By ensuring that these pages are properly referenced as 404s, you can properly redirect the visitor to a relevant page and ensure that there are not any issues of "duplicate content" (pages for which the only text on the site says "page not found").
Sometimes, the search engines aren't as good as advertised in following redirects. MSN Live has indexed 301 redirects as blank pages, and examples of 302 hijacking seem to make a comeback every now and then. We have witnessed very quick responses to 301s from Google and are still hoping to fully recover from some 301 redirects with Yahoo after several months. MSN is hit or miss, depending on what they're doing in any given month. Bottom line: do 301 redirects when you have to and consult with an SEO professional if you believe there could be any chance that you could lose rankings as a result of any URL changes.
Canonical issues. Your first response to this may be, what the heck is a canonical issue? Here's the thing. You should reference your site -- in particular your home page -- in one way. You'll notice that if you try www.searchenginewatch.com, it redirects to http://searchenginewatch.com/. You'll also notice that our site resolves to www.vizioninteractive.com even if you try http://vizioninteractive.com. Many sites link to their home page internally as, www.example.com/index.html. Each instance of a site resolving to a given URL is a different address, as far as search engines are concerned. To get the inside scoop on canonical issues, you should read Matt Cutt's blog.

Knowing the basics of server response codes and when to use them correctly is critical for any competitive Webmaster or search marketer. You can check your own server responses with many online resources including this status code checker. If you have any questions regarding redirects and content management system setup, contact a search marketing professional.

Mark Jackson is off this week. Today's column ran earlier on Search Engine Watch.

Back to Article