The Art and Science of SEO Site Audits [Best of SEW 2010 #10”

Editor's note: As 2010 winds down, we're celebrating the Best of 2010, our top 10 most popular columns of the year on Search Engine Watch, as determined by our readers. Every day over the next two weeks, we'll repost the most popular columns of the year, starting at No. 10 and counting down to No. 1 on Dec. 31. Our countdown begins today with our No. 10 column, which originally was published on November 4. Enjoy!

SEO site audits are of critical importance. It is an investigation, analysis, diagnosis, and strategic dive into a site from the SEO, usability, and information architecture perspectives (most audits leave off the last two) that produces (through hard work and experience) a set of well-vetted recommendations.

Site audits allow us to benchmark and baseline a site's strengths, weaknesses, and areas of opportunity; to prioritize the work that is necessary for a site to excel in organic search rankings; and to provide actionable, documented recommendations that guide a client's implementations.

In short, SEO audits are essential.

No matter how long you've been doing them, there are fresh challenges to find, new areas to uncover and explore, and new opportunities to be found. This is especially true in team settings.

Because of its subjective nature, SEO thrives in a collaborative environment. SEO teams working together build off one another's input in remarkable ways.

SEO Project Management Tool
Project management tools make collaborating and documenting SEO audits efficient.

This article will explore the process of undertaking technical SEO audits that we've developed over the years. It will not be exhaustive, some things should stay confidential, but I will give you plenty of direction to start your own SEO auditing work. If you're already a SEO familiar with audits, hopefully you'll find one or two gems within this article.

SEO is Art & Science

"The first step in diagnosis is to find the root cause." - Vanessa Fox

SEO truly is both an art and a science, and audits reflect this dual nature. The issues documented in a SEO audit are reinforced with data and analysis.

But in most cases, these problems were arrived at by "following one's nose" and using experienced hunches to probe and poke until a problem shows itself.

This isn't always the case, and there is plenty of low-hanging fruit to pick out there for SEO: from title tags and meta data, to internal linking and 404 errors. But to find the really good stuff, that is where the artistic and even creative side of SEO is applied.

Yet, there is a scientific side to SEO, as well. Obviously, data and analytics are essential pieces of the work, along with disciplined testing and analyses. While many times it's the hunches a SEO has that allow her to discover things, the process cannot and will not stand up to a disorganized approach.

You must have a formal structure and procedure with which to work through things, because SEO audits (especially for large enterprise-level sites) can be severely complicated affairs.

There's no way to approach an audit formulaically, but using guides such as the following will allow you to cover the major areas needing consideration:

Partial List of SEO Factors Investigated

A partial list of factors investigated during SEO audit work. Checklists make the work more organized and structured.

This is only a partial list, but should give a good idea of the areas covered by a technical SEO site audit.

A Framework for SEO Audits

In addition to checklists such as the above, it helps to build a framework for approaching audits. Even small sites are easier to investigate in this way; for large, complicated sites, an organized approach is essential.

Consider breaking down your SEO audits into the following framework, which offers a conceptual model for breaking down (usually large) web sites into pieces. On a practical level as well, this makes things much easier.

The framework is as follows. Web sites consist primarily of the following patterns, which will need to be taken into account:

On-page

  • Domains: The top-level domain (TLD), which can have within it multiple sub-domains.

  • Sections: These represent the organizational hubs where categories (and sometimes other sections) are located.

  • Categories: These represent organizational reference points for pages and media (and sometimes other categories).

  • Pages: Web documents in the form of whatever language -- xHTML, PHP, ASP, etc. and either static or dynamic (or a combination).

  • Media: Images, videos, documents (such as PDFs), sound files, etc.

And, of course, URLs are the associated signature, or uniquely identifying piece, of each example of the above components.

For a full treatment of this topic, see my article, "The SEO Guide to Information Architecture."

In practice, each of these aspects are really part of the whole, like spokes on a wheel, so the SEO doesn't need to pick and choose which components to investigate separately. The work is more cohesive.

An SEO audit takes into account more than on-page factors, of course, so off-page criteria also must be accounted for. The following partial list covers the main factors:

Off-page

  • Backlinks: Quantity, quality, diversity, recency, freshness, stability.

  • Social Media Signals: Frequency, Sources, Diversity, Authority/Influence.

  • Cache Dates, Crawl Frequency, Crawl Thresholds: Content frequency and crawl efficiency, along with domain authority, are primary factors.

  • Indexed Pages: Unfortunately this metric can be a little flakey.

  • Related Sites: Your site's neighborhood is still meaningful mozRank.

  • Toolbar PageRank.

You can go really deep into the link profile alone. Also, while toolbar PageRank is still useful for some types of analysis, mozRank is a better metric to use in general.

The Primary Factors

Every site is different; every site has a unique signature and carries with it strengths and weaknesses. However, the following short list is normally a good place to start:

  1. Content: Is there content, and does it have links? Does the content update frequently, and is it timely? Is the content more than just articles and blog posts, is it also interactive tools, games or resources? Does it represent a significant contribution to the web? Is the content optimized and structured well?

  2. URL Structure, Consistency and Canonicalization: Are the URLs structured well, and devoid of session IDs and tracking parameters? Are they consistent? Are the URLs short and friendly to read, share, and list in SERPs? Is the canonical version consistently used?

  3. Site Architecture & Navigation: What is the architecture of the site, and how do the various navigations interlink their sections and categories, pages and media? What is the average click depth of the various portions of the site?

  4. Internal PageRank Flow: Are internal links consistent? How many links are given on pages, and what is the average number of links a given URL will secure on the site? How is nofollow and robots exclusion being addressed? What is the overall strength of various portions of the site? Where are the strongest URLs on the domain?

  5. Site Performance: Is the site fast? Is it easy to use, and even fun to use?

How to Consolidate and Deliver the Reports

One aspect of SEO audits not usually given enough attention is the final deliverable. It is essential that this is clear, concise, and not overly complicated.

The document needs to be shared among diverse business units, from marketing and SEO teams, to development, content, IT and UX teams, so there cannot be an exclusive amount of "inside baseball" type of language.

The most important component is the prioritized summary of action items, with associated (estimated) level of effort. Each recommendation should have the problem documented, its impact explained, and the solution offered.

If it's possible to offer one or two alternative solutions, all the better. However, it's helpful to keep the deliverable concise and to the point: most of the discussion will take place on the review calls. Leave the audit deliverable as simple and precise as possible.

The Best SEO Audit Tools

The best audit tools are the ones you're experienced using. You won't be able to get all of these, some are proprietary, but there are worthy alternatives for everything listed here.

For large-scale SEO analysis, a robust crawling tool is essential. Tools such as SearchClu from Define offer excellent insight in a user-friendly interface.

SearchClu Site Analysis

The IIS SEO Toolkit is a splendid tool, and it's quite fast. Use it in place of Xenu for most auditing needs. This image is from SEO Gadget's excellent tutorial.

SEO Gadget Site Analysis Report

Xenu is a good choice for quick work, such as importing a small list of URLs (keep the total URLs Xenu crawls to under about 10,000 for best results) to view response codes.

Google Webmaster Tools is an SEO's best friend. Using the internal linking report, for example, can reveal weaknesses in a site's internal link graph:

Google Webmaster Tools Linking Report
Using GWT data to show that over 75 percent of this (very large) website's pages has 10 internal links or less; almost 50 percent has only 1.

Sometimes the best tools available are advanced search operators at Google. Use them to your advantage.

For a quick look at a website the way a crawler will experience it, use Lynx or SEO-browser.com.

Charles is the go-to for diagnosing more advanced issues, such as conditional redirects and that sort of thing, and when you need to deeply investigate GET and POST sessions.

Web Page Speed and ySlow are the best tools for site performance checks.

For toolbars, the combination of SEO for Firefox, SEOBook toolbar, SEOmoz (now on Chrome!), and SEOQuake can't be beat. Always use the Web Developer Toolbar, of course, and the WAVE toolbar can sometimes come in handy, too.

If you've got the chops, sometimes log file analysis leads to the best insights of all in SEO audits. Make use of log file parsing tools such as Splunk. Unix geeks like us can use AudetteMedia's own logfilt for SEO log file analysis, too.

Splunk Log File Analysis
Using Splunk for SEO log file analysis.

A Site's Unique Signature

There is an important consequence of delving deeply into a site over a period of many hours: One begins to see the forest for the trees, begins to understand the sites overall internal PageRank flow, its special weaknesses, and areas of opportunity.

Mentally processing all of the little things encountered into a whole picture of a site, its special "stamp" or signature, provides special insight into what will be required to improve the site's traffic.

This unique signature influences how a site is crawled and ranked, and is the sum of all its various components, from scoring factors applied to a URL, to the crawl experience of a search engine spider.