How Search Engines Look at Links

A special report from the Search Engine Strategies 2002 Conference, March 4-5, Boston, MA.

Representatives of Google, Fast, and Ask Jeeves/Teoma shared inside information about page ranking and linking analysis at the Search Engine Strategies Conference in Boston, March 4th.

The panel, featuring Ask Jeeves Vice President of Search Paul Gardi, FAST Software Engineering Manager Rolf Michelsen, and Google's Director of Technology Craig Silverstein, offered candid views on how their search engines evaluate web sites to determine relevancy and ranking.

Craig Silverstein said Google from the outset wanted to use a more human process to determine if a site had value related to particular keyword phrases. Looking at meta data, he said "We began developing a mathematical way to determine if others think your site has value."

Google evaluates the number of incoming and outgoing links to a site, the circular associations of linking around the Web and data related to communities of Web sites, trying to identify local areas with unique themes in common.

According to Silverstein, it's relatively easy to get good ranking in Google. "If you make it easier for us we like you," he said. He stressed that you should be associated with reputable sites, making sure those sites have keywords related to yours. It's also crucial to use targeted keywords throughout your site, and have relevant explanatory text around your keywords and links.

Paul Gardi from Teoma/AskJeeves said, "We begin with text analysis." Next, Teoma considers "general popularity," comparing sites, for example, with one link coming in vs. a site with three links coming in.

Gardi stressed that link popularity alone wasn't enough. "Spamming techniques can create problems so we also look at status," he said. What is status? Higher status sites have a number of quality sites linking to them. You want these high-status sites linking to you.

"Using Direct Hit ['s now defunct search engine”, we add click popularity to help us form Local Subject Communities," said Gardi. This means that Teoma evaluates a whole community of sites to see if there are common themes that unite them and if there are "authority sites" in that mix. Teoma is constantly "...looking at members of the community to see who they have chosen as authorities," said Gardi.

For good ranking, he added that you need to make sure there are some strong "authority" sites in your community of links, sites within a common thematic mix, and sites with a significant number of links in to them.

Rolf Michelson from FAST (which powers Lycos Search in addition to its own search engine), said FAST first looks at the network of linking within a site and for a consistent theme. He described the process of link evaluation as looking for "content ranking" or "content classification." FAST is "...forming a bubble, a statistical structure..." within which they can evaluate the kind of company you are keeping.

We have always known that links from good sites brings us traffic. And now we have begun to pay additional attention to our linking strategy.

Remember, a quality product for a search engine is a relevant group of quality results for a particular keyword search. Think about the search engine's goals first, and your own second!

So what analogies did the search engine representatives use in recommending how to optimize our sites related to their linking analysis algorithms? In the summary below, the words in quotes are the direct phrases of the presenters.

First, strive to be in "good company." Don't spend a lot of time in the bad part of town if you are going to run for political office! All three companies are able to judge the "reputation" of the pages and sites that link to you.

Ask yourselves the following questions:

Are the sites that link to you in your same field? Pursue "theme analysis" not only for your site but also for your linking partners. Just as your own site should have a consistent theme, it's also vital that the sites who link to you are in the general "community" related to the general subject of your site.

Are the sites that link to yours linked to in turn by others in the same field? The more "popular" your friends are, the more "popular" you are. Remember being in your teens when it was so important that you be in the "right crowd?" To a large degree, the same is true of how your pages are evaluated by search engines.

Do you have some "high quality" sites linking to you that are "authorities" or experts in your field? Remember, the search engines know everything about the sites linking to yours. If they are authoritative sources in your field and they link to your site, then the engines infer that you are likely providing a valuable source of information and high relevancy to the keyword phrases that characterize your site.

Finally, are all your pages optimized for relevant keyword phrases and linked back and forth within your site? This will flesh out the theme of your site and when your "friends" are doing the same, you all become a strong "community of authorities" encouraging the search engine to present your pages to searchers.

Ask/Teoma's Gardi summed up the session for all participants, suggesting that for best results, you should optimize all pages, don't mix with bad company, and make sure link text says something important.




Terry Plank is co-owner of the Academy of Web Specialists and Director of Marketing. A search engine marketing professional since 1996, he is also co-owner of a local ISP in the San Francisco Bay Area.

About the author

Chris Sherman is a frequent contributor to several information industry journals. He's written several books, including The McGraw-Hill CD ROM Handbook and The Invisible Web: Uncovering Information Sources Search Engines Can't See, co-authored with Gary Price. Chris has written about search and search engines since 1994, when he developed online searching tutorials for several clients. From 1998 to 2001, he was's Web Search Guide.