Over the years I've read countless reasons for rewriting URLs -- basically cleaning them up at the Web server level so search engines can index them easier. There's as much confusion and controversy surrounding URL rewriting as Darwin's theory of evolution.
My high school science teacher pounded into my head that "survival of the fittest" means species most suited for their environment will adapt and survive. URLs live in search engines. They need to adapt, too.
Many people understand the value of rewriting URLs, but not exactly why they need to do it. Plus, there's more than one opinion on the main reason for a URL rewrite. We won't go into the differences between rewriting URLs for Apache versus Microsoft's IIS server hosts. Instead, we'll focus on the "why."
Avoiding Duplicate Content
I found expert SEO advice online from Yahoo engineers, including Priyank Garg, director, product management, Yahoo Web search, in a Yahoo Search blog post. Yahoo offers a rewrite-helping tool available through Yahoo Site Explorer.
Blog post comments (some from this week) indicate Yahoo's tool is beneficial. It supports the reasoning behind the mod rewrite for dynamic sites -- to avoid duplicate content.
What if you don't have a dynamic Web site? Do you still need a rewrite?
Getting More Pages Indexed
Pages created in a CMS (Content Management System) or as part of eCommerce platforms, such as IBM Websphere, will include parameters or underscores and other delimiters or special characters that make it difficult for the search engines to understand them. That may cause less pages to be indexed.
I didn't need to go far to affirm this reasoning. We've worked with several Websphere sites at Brulant. One of the main elements we recommend is "URL flattening" to improve crawling, with the added benefit of including keywords in the URL.
Our engineers have created a continuum that list nine stages a Websphere URL must go through to become fully optimized.
Here's a before and after example:
- Before: http://www.host.com/webapp/wcs/stores/servlet/StoreView?storeId=10001&catalogId=10002&langId=-1
- After: http://www.host.com/Fall/Widgets
For more detail on the process, you can contact engineer John Sisler through his blog SEO Hardcore.
This is one strategy I'd love to get feedback and opinions on in the SEW Forums.
Let's go to the most commonly cited reason nowadays. One version of a URL rewrite enables Webmasters to insert targeted keywords they'd like to rank for within the URL itself. This is one factor many SEOs (define) believe is important to help a page rank for a particular keyword.
In this case, a coded rewrite isn't necessary; instead, the actual directory and file naming structure of the site is changed to make the URL more friendly.
SEOmoz's "Ranking Factors" lists the keyword-in-the-URL factor as being of "moderate importance." SEO expert and pundit Aaron Wall says it's "not weighted anywhere near as much as an exact match domain name, but helps improve CTR (define) (and thus relevancy if CTR factors into relevancy scores) and some people will link to pages using the URL as anchor text."
Google and other search crawlers sometimes avoid difficult URL structures to prevent getting caught in a loop and crashing a site during a "confused" crawl. IBM alludes to this reasoning in WebSphere Application Server V6 Scalability and Performance Handbook.
Every one of these most-cited reasons for a URL rewrite is potentially a valid reason to consider the labor-intensive task. I plan to write a follow-up later this year with some experiments we've conducted with Websphere and eCommerce clients.
Frank Watson Fires Back
Frank Watson: I've always loved proper names for URLs and domains. I used the old hyphenated three word domains a bunch of years ago. I definitely take note of the well-sourced article. What I'm hoping is that most people are using this methodology already. If not, you gave them the argument. Now who listens?
Chris, you're well-versed on URL rewriting. I'll have to storm the beach next week.
Until then, click here to share your opinions on this subject at the SEW Forums.
Meet Your Favorite Search Engine Watch Contributors
Many of SEW's leading expert contributors will be at ClickZ Live, the new online and digital marketing event kicking off in New York (March 31-April 3). Hear from the likes of: Thom Craver, Josh Braaten, Lisa Barone, Simon Heseltine, Josh McCoy, Lisa Raehsler, Greg Jarboe, Dan Cristo, Joseph Kerschbaum, John Gagnon, Eric Enge and more!