Revisiting Meta Tags

Revisiting Meta Tags

By Danny Sullivan, Editor
The Search Engine Update,
Dec. 5, 2002

In October, I wrote an obituary for the meta keywords tag. Given that Inktomi was the last major crawler to still support the meta keywords tag, I didn't think it was worth the time or bother for many webmasters to use. It's now time for a follow-up on that article, because there was plenty of reaction and feedback to it.

First and foremost, a few people came away with the mistaken impression that I was saying that all meta tags were dead. Not so. I always specifically referred to the meta keywords tag, in that article. In contrast, there are many other types of meta tags that exist and continue to be used by web-wide search engines, browsers and for other purposes.

In particular, when it comes to crawler-based, web-wide search engines, the meta description tag continues to be very useful. In fact, the Search Engine Watch "members" version of my article about the decline of the meta keywords tag included a long look at how the meta description tag continues to be supported by crawlers (What's a Search Engine Watch member? See the membership page!).

Latest Happenings With Descriptions

Indeed, Google appears to have recently added more support for the meta description tag, from what I'm seeing. Previously, Google formed its descriptions -- or "snippets" -- as it calls them, by extracting relevant text from the body of a web page. In some rare cases, as with a "text-light" page, the meta description tag might be used. Now the meta description tag is even being used even for some "text-normal" pages. Google also continue to use snippets, in other cases.

For example, a search for "cars" brings up in the top spot, with a description that seems to come out of the meta description tag. A search on "hunting" brings up, again with the meta description being used. A search on "running shoes" sees and with meta descriptions used. For "reading advice," is listed first and a great example of a text-rich page still getting its meta description tag used. The same is true for the Epinions page listed in that search (and am I the only one notice that Epinions seems to rank well for some many different things these days, at Google?).

I asked Google for the official word on what's happening, but I haven't heard back, as of yet. When I hear more, I'll pass it along.

Last month, Teoma also seemed to begin providing more support for the meta description tag. In October, I noted that Teoma made descriptions by combining text from a meta description tag along with a Google-like snippet formed from text extracted from the web page. Today, Teoma now seems to be using the meta description tag alone, if there's a good deal of confidence in the tag, such as if the search term appears within it. Otherwise, a snippet is used.

Unfortunately, Teoma won't confirm exactly what's happening. "To be honest, we are constantly changing our methods, and I do not feel comfortable verifying any particular method because we often change these processes with no notice," said Paul Gardi, vice president of search at Ask Jeeves, which owns Teoma.

So be aware -- what's currently happening now at Teoma is subject to change.

Inktomi also shifted how it forms descriptions last month. Meta tags will be used for paid inclusion customers, guaranteed, Inktomi says. For non-paid inclusion pages, descriptions will generally come from one of three ways: from how the page is listed within the LookSmart directory, via a snippet created from extracting text from the page or by using the meta description tag, if one is present.

It's difficult to predict exactly which description will be used, but it is more likely that LookSmart or meta tag descriptions will be used for home pages, those at the top level of a site, while "inside" pages are more likely to have snippets.

Inktomi says that the logic here is that home pages are more general, so an overall summary of the page from LookSmart or the meta description tag might be more useful than extracting text. In contrast, an inside page may be more likely to match a specific topic, so having a very specific description made by dynamically abstracting a snippet makes sense.

As a reminder, you'll find a summary of how descriptions are formed at the major crawlers on the Search Engine Display Chart, along with a link to my last article about how descriptions are formed by the major crawlers that I've not covered above.

How do you deploy the meta description tag? Take a look at the How HTML Meta Tags Work page in Search Engine Watch. It covers that tag, the meta keywords tag, offers tips about the importance of title tags and provides reference material about other meta tags.

More Feedback & Clarification On Meta Tags

I also had a couple of people note to me that various meta tags are often used for intranet or enterprise search engines, and they've got no disagreement on this from me. In fact, I noted this in the supporting material listed at the end of my original article. However, it bears repeating. Other meta tags, such as Dublin Core meta tags, exist for specialized search engines. If you know your content is being indexed by such search engines that support these type of tags, by all means, consider using them. It's just the web-wide search engines that don't support them.

My favorite reaction was from a few people who noted that I still had meta keyword tags on the site. "So, Danny, if the meta keywords tag is of so little use, then why do YOU still use them," came the queries.

I confess. It was indeed a conspiracy. I figured if I could get everyone else to stop using meta keywords tags, it would allow my own pages to...(please picture me here with my little finger in my mouth and laughing like Dr. Evil from Austin Powers)...DOMINATE THE WEB!!!

The reality is far more boring. The tags you see were written by me back when the meta keywords tag enjoyed more support. I'm no longer writing new ones, since I don't think they are worth the time. Similarly, it's not worth the time to remove the existing ones. It doesn't hurt to leave them, so why bother removing them.

Indeed, this is the key point I was making in my original article. Over the years, I've seen people fret and worry far too much about the meta keywords tag. I've always said that if you are short on time, to skip the meta keywords tag. Now that support is so low, I'm saying it even more strongly. Feel like you've got better things to do? Then pass on using the tag. However, as always with anything search engine related, if you personally feel the meta keywords tag is helping you for whatever reason, by all means keep using it!

While my original article said only Inktomi supported the meta keywords tag, Jill Whalen of the excellent High Rankings Advisor newsletter and a few other readers said they noticed that Teoma was picking up some of their pages for words that only appeared in their meta keywords tag. (Jill, by the way, generally agrees in her recent "No Meta Keywords Tag" article that the meta keywords tag probably isn't worth the bother unless you have some "funky-techie" synonyms not in your regular body copy.)

After more investigating, I found this to be true. Teoma does indeed appear to be indexing text in the meta keywords tag. Teoma had always said in the past that it did not support the tag. When queried about the latest finding, Teoma said it couldn't say yes or no about support. As noted earlier, things are currently changing so much with Teoma that the search engine suggests that if its currently support the tag, that may not be the case from in the future.

A Closer Look At Meta Keywords

I didn't just stop at testing just Teoma. I also ran some queries on all the major crawlers that were designed to bring up pages that had particular words in the meta keywords tag. If the search engine didn't read the meta keywords tag, then the pages wouldn't appear., AltaVista, Google and WiseNut all say they do NOT support the meta keyword tag, and the testing supported this. As already noted, Teoma does seem to support the tag, at least for the moment. Inktomi also continues to show support, exactly as the company has said it does.

The testing, however, highlighted an important change in the way Inktomi processes the meta keywords tag. Yes, commas in the tag DO now make a difference.

Before we go on, let me say once again that the meta keywords tag is not that important. Even Inktomi stresses that the tag is not that important. However, if you are a believer in it, then the information below will be of interest.

If you've already decided the tag is not worth the hassle, then read on if you are curious, but remember at the end that you can save yourself all the worry and hassle by simply not using the tag.

Oh, The Difference A Comma Makes

Whether you should you put commas between words and phrases in your meta keywords tag is an age old question in the SEO world. I've long said it makes no difference. This was based on talks with the search engines and on testing, and the archived More About The Meta Keywords Tag page provides past advice on why leaving out the commas might have been helpful. That page has a meta keywords tag example like this, where phrases are separated by commas:

It then shows another example where by dropping the commas and combing terms, one can retain all the key phrases without having to repeat the word "fish" so much.

Now for this second example, let's assume that none of these words or phrases actually appear in the body of the page itself. Moreover, just pretend these are all really unique terms, so that only a few pages across the web make use of them. That will help us understand how Teoma and Inktomi deal with the content just in this tag.

At Teoma (and other search engines that supported the meta keywords tag in the past), we could find this page easily by searching for any of these phrases:

"selling fish"
"fish food"
"feeding fish"
"fish farmers"

This is because all those words appear in those exact order in the meta keywords tag. The word "selling" is right next to the word "fish," so if the meta keywords tag's content is read in the same way a search engine might read ordinary body copy, the word proximity is preserved. In fact, a page with this tag come up in a search even for:

"selling fish food raising fish farmers feeding fish tanks"

In contrast, Inktomi no longer operates in this way. It did in the past, and when the change happened is uncertain. Inktomi itself couldn't confirm it. What is certain is that when it comes to the meta keywords tag, Inktomi is only storing the proximity of words between commas. So let's go back to that first example:

In this case, Inktomi will store these phrases:

selling fish
fish food
raising fish
fish tanks
feeding fish
fish farmers

Notice in the example this section:

food, raising

A search for "food, raising" or "food raising" would find this page at Teoma, because despite the comma separation, Teoma would see that the words are next to each other. In contrast, it will not work with Inktomi. As far as Inktomi's concerned, when it sees a comma between words and groups of words in the meta keywords tag, those words are kept in isolation.

What To Do?

As said repeatedly, Inktomi is the only major search engine publicly saying that it supports the meta keywords tag. This means that if you are going to still use the tag, it does make sense to consider changing things to please Inktomi. This means that adding commas could help Inktomi realize you have important phrases in your meta keywords tag, and this in turn could MAYBE help you rank a tiny bit better for those phrases.

Assuming you are going to continue with the meta keywords tag for Inktomi, here's some additional information. Traditionally, about 1,000 characters of the meta keywords tag's content (everything between the quote marks after the content= portion) has been indexed by the major search engines that have supported the tag. So what's the situation now? Forget characters entirely and think words, Inktomi says.

"Our engineers are reluctant to give out an exact number since it's not set in stone, and since there are a number of exceptions to the rule, but I think we can safely recommend to content publishers that they keep to less than 25 words."said Ken Norton, director of product marketing at Inktomi.

A bit more from Inktomi on writing the meta keyword tag for its use can be found on the company's Content Policy FAQ.