Meet the News Search Engines

News aggregator services gather information from the Web and provide news to other Web sites, offering some great opportunities for search engine marketers.

A special report from the Search Engine Strategies conference, August 8-11, 2005, San Jose, CA.

On the “Meet the News Search Engines” panel, representatives from major news search engines talked about how to prepare and submit news information.

“What we’re seeing today is that aggregating news content is really a driving force with how people perceive news,” said Jim Pitkow, CEO of Moreover, one of the most popular news aggregators. “People want to get a variety of news topics across a wide variety of sources.”

Moreover gathers its news information from more than 12,000 handpicked, editorially-ranked online news sources from over 126 countries in 36 languages. What makes Moreover unique is how it differentiates different types of news.

“We treat blogs separately from news,” Pitkow explained. “News is more about journalistic integrity, with more control and standards. Blogs tend to be more about opinions. It is our job to separate the facts from opinions—the news from those who have opinions about the news.”

Users can host their news feeds with Moreover, using its ping server, or they can submit their news feed for inclusion. “It will be reviewed by editors who will determine if its actually news or if it’s a blog. There is no charge for inclusion,” said Pitkow. “We do not accept any sponsorships. It is important that we appear unbiased to our customers by the end of the day.”

Chris Tolles vice president sales and marketing at, finds news and news search interesting because what he calls the “reference web” is being supplanted with the “incremental web. “The reference web is goal directed whereas the most important thing about the incremental web is that you are getting fresh information—news, commentary, etc.”

“News search is about news coming to you that is local and topical,” he said, “and, hence, more relevant.” differentiates its news information via categorization and zip-code level localization. News is available via its web site, partner feeds, RSS feeds, email alerts, and JavaScript iframe.

“For our business model, we take about 10,000 news sources, combine this information with ad models such as Google AdWords and others,” he said. “We categorize and localize it, and then send it to 300,000 news channels and to our own site. We present news to the user much like a newspaper.”

Yahoo News has been in the business of news search longer than any major search engine. August 9, 2005 was the ten-year anniversary of Yahoo news.

“The first thing I want to do is possibly challenge a common misconception that Yahoo News is built on search technology,” said Neil Budde, general manager of Yahoo News. “The truth is that Yahoo News is built on a combination of content that we host on Yahoo.”

“We get that content through relationships with leading news providers, both wire services and about 90 other publications that we work with to bring a portion of their content onto Yahoo News,” Budde continued. “We then marry that with our world-class technology that pulls in various different strengths of the Yahoo network, including search, and then we oversee that with a team of human editors. So, it’s a little different approach to news.”

Google recently announced its RSS and Atom output for Google News. “You can now get any Google News page, any section, any search or customized composite page available over RSS and Atom,” said Nathan Stole, product manager of Google News. “It is available in English only, for now.”

“Google News has its own crawlers that goes out and is very, very rapidly scanning all the new sites that we have in our corpus, basically trying to identify new articles as they appear,” said Stoll. “What we do once we find that new article is extract out the title, body, and photos.” Google then clusters related articles and prioritizes the news stories.

News optimization pitfalls

Google’s Stoll provided a number of tips and pitfalls on optimizing news feeds and articles for Google. These tips are also applicable to the other search engines.

  • Review your robots.txt file or the robots exclusion meta-tags to ensure that you are not preventing search engine spiders from crawling your news articles.
  • Do not re-use URLs. It makes it difficult for news search engines to track news articles over time.
  • Try to make the URL structure of the news article as spider-friendly as possible.
  • Publish one news article per page. When a news search engine clusters information about a topic, web pages which contain multiple articles do not appear to be focused on that topic.
  • Do not write article titles or headlines that are unclear or vague. Keyword-focused, factual, and concise headlines help a news article get classified more accurately.
  • Registration barriers often lead to less page views over time. People do not like to click on a news search engine link only to be told that they must fill out a registration page to read the full article.

“Users really do react negatively to things like slow loading pages, slow ads, and registration barriers on new sites,” said Stoll.

Multimedia files in news search engines

Many news search engines offer news in multimedia formats as well as text formats. For example, in the Yahoo News section, the file-tab navigation offers photos, audio files, slide shows and video files. Google News includes photos during its crawling process.

“Expect to see a lot more broadcast news (audio and video, advanced speech to text recognition, full transcripts) from us this year,” said Pitkow.

Shari Thurow is the Marketing Director at Grantastic Designs, Inc. and the author of the book Search Engine Visibility.

Search Headlines

NOTE: Article links often change. In case of a bad link, use the publication’s search facility, which most have, and search for the headline.

Who Comes Up With This ROI Stuff, Anyway? Oct 17 2005 4:04PM GMT
A Day in the Life of a Search Engine Marketing Agency Oct 17 2005 4:00PM GMT
Yahoo’s Sr. Director of Product Management Interviewed on podCast411
Kansas City Infozine Oct 17 2005 3:59PM GMT
Quintara Search
Pandia Oct 17 2005 3:57PM GMT
The SEO and the Blacksmith
ClickZ Today Oct 17 2005 3:56PM GMT
AOL to take Blogs’ Pulse via Intelliseek
ClickZ Today Oct 17 2005 3:54PM GMT
Microsoft Camp Studies Blog Search
eWeek Oct 17 2005 3:51PM GMT
Digitas Replaces With SearchIgnite
Media Post Oct 17 2005 3:50PM GMT
Inform Technologies Bows News Aggregator
Media Post Oct 17 2005 3:38PM GMT
Q and A: Why can’t we get good results in Google?
Dear Kalena Search Engine Advice Column Oct 17 2005 12:14PM GMT
I Speak Search
Search Engine Guide Oct 16 2005 9:42PM GMT
Free SEO Tools
Search Engine Guide Oct 16 2005 9:41PM GMT
Seeking keyword ROI
SEOMoz Oct 16 2005 9:40PM GMT
Bugtraq: Re: Google Talk cleartext proxy credentials vulnerability
SecurityFocus Oct 16 2005 8:34PM GMT
Online advertising ‘clicks’
Hindu Business Line Oct 16 2005 8:12PM GMT

Related reading

Simple Share Buttons