Getting Listed In Google's Froogle Shopping Search Engine

Getting Listed In Google's
"Froogle" Shopping Search Engine

By Danny Sullivan, Editor
The Search Engine Update,
Dec. 17, 2002

Last week, Google unveiled its long anticipated shopping search engine. Called "Froogle," the service is available to searchers in experimental "beta" form. Chris Sherman provides an overview of how the shopping search engine operates in his recent SearchDay report, Online Shopping with Google's Froogle. In this article, I'm taking a closer look at how merchants get listed within Froogle.

Before starting, it's important to stress that if you are worried about being listed in Froogle, then you'd better have already sorted out the issues of getting listed with other shopping search engines, such as DealTime. Why? Because Froogle currently has virtually nil exposure to the web searching audience, while other shopping search engines are far better positioned.

There's no doubt that the profile of Froogle will be raised, over time. Certainly Google is likely to add a "Shopping" tab at some point, as well as possibly suggest shopping search results within the regular web search results page, in the way it currently suggests appropriate news or directory content. So, get your house in order for Froogle now, and you'll be ready when traffic grows.

However, if your content is suitable for Froogle, then also ensure that it is being distributed to other shopping search engines. To help you, Chris Sherman recently ran an entire week of SearchDay articles about shopping search. All the stories are listed in the Nov. 22, 2002 "Week In Review" recap at SearchDay. Be sure to follow the links to members-only editions of articles, where indicated.

So how is Froogle gathering its listings? The service is leveraging the existing web crawling that is done for the main Google service, operating a special Froogle crawler to gather content from some large ecommerce web sites and taking a giant step into the unknown for Google by accepting data feeds from content providers. And, in all cases, no paid inclusion fees are involved.

Into Froogle Via Crawling

Let's start off with how content is added via Google's regular crawler. As most are aware, Google indexes pages from across the web, storing the content of those pages in an index, which searchers can query against. With the addition of Froogle, some pages in the index are now also flagged as "product" pages, which should also be queried as part of a Froogle search. In other words, if you are listed in Google, then there's a chance that your product-oriented pages may also already be listed in Froogle.

What makes your page worthy of inclusion in Froogle? The company shies away from providing specifics, but generally, it confirms that your pages ought to look like typical "product" pages. For example, is there a photo of the product, with a price and description shown? Do you provide an item number and perhaps a button to add the product to a shopping cart? All are clues that you are selling something on the web.

"You can imagine that we are looking for things like prices, certain formats that recur on ecommerce sites," said Craig Nevill-Manning, a senior research scientist at Google and the engineering manager for Froogle (and a special thanks to Craig, who despite being ill at home yesterday, still made it to a phone to provide information about Froogle for today's newsletter).

Nevill-Manning gave one great example of how Google may decide whether a page is selling something rather than providing just information. Imagine a review about a digital camera. The cost of the camera in the review may be rounded to the nearest dollar, such as $600. However, a merchant will often not round up, instead showing pricing at $599.99, so as to stay under the $600 mark. That typical pricing technique is one of many tips that Google may be looking for to identify a page that sells a product, rather than merely discusses it.

In conclusion, if your pages have enough clues, then you may already be included in Froogle. If not, then look at some of the pages already listed in Froogle and consider mimicking how they are designed. That may help. Alternatively, you can take the much easier approach of providing a data feed.

Feeding Froogle

Making a data feed is easy. It's simply a tab-delimited file that you will FTP on a regular basis to Google. Think speadsheet. Your "feed" is simply a list of all your product page URLs in one column, then the corresponding product names in the next, then the product descriptions, prices, URLs to product images and the "categories" each product belongs to. You can also provide some optional information that Froogle may use in the future, such as color or size.

"If they have anything they can export into Excel [Microsoft's spreadsheet application”, then it is fairly trivial to export it to us," said Nevill-Manning.

Information on how to apply to send a data feed, along with more details on what's allowed, can be found on the Information for Merchants page at Froogle. Be forewarned. While Froogle aims to accept all qualified data feeds, the demand to send these in is far greater than was expected. This means that it may take several weeks before Froogle processes your application, Google says.

"We've been deluged, and we are gearing up," Nevill-Manning said. "We want to have the resources to deal with this on an ongoing basis, so it will take us a little while to change all that."

For some large merchants, Google also gathers content using a special Froogle crawler. However, Google sees the ideal solution as all sites providing product data feeds.

"It sort of benefits everyone if instead of downloading a million web pages, you give us a feed," Nevill-Manning said.

Froogle Searches Fields, Not Full Text

Aside from saving time and bandwidth, data feeds also ensure that Froogle is more accurate about what it lists. When content is obtained through crawling, Froogle essentially lifts information from the page to create its own "feed" for a particular site.

In other words, Froogle doesn't search against the entire text of pages that have been found via crawling. Instead, it only searches against key elements from those pages that correspond to the fields it would get in a data feed, such as price, product name and description.

This bears repeating. Froogle is not a full-text search engine, in the way that Google is. Instead, Froogle only searches against key elements or "fields" of information, and the fields for each page are either defined explicitly through a data feed or implicitly through Google making its best guess, when it crawls a "product" style page. Moreover, since any type of guessing may be wrong, the best way to ensure the accuracy of how you are represented is to do a data feed.

Freshness And Ranking Issues

You also have more control over freshness, with a data feed. Most of those not in a feed will have their pages revisited by the regular Google crawler on a roughly monthly basis. The Google "fresh" crawler may hit some pages more often, if they seem to change frequently. However, with a data feed, you can update your content according to the schedule you know is best, such as daily or weekly, rather than leaving Google to guess at what's best.

How about ranking? What factors influence whether a product comes up tops in Froogle? The answer is that this is still being worked out. The techniques used for the regular Google web search results are not the same as used by Froogle.

"It's a very, very different problem from web search. For a start, we have much more structured data," said Nevill-Manning. "I think the real answer is that the ranking function is going to be changing a lot. Even if I told you what it was today, it will be changing."

I think the advice to take away from this is not to worry so much about how you are listed in Froogle, at the moment. For general queries, there may be no rhyme or reason as to why a particular page seems to come up well, for the immediate future. You could start trying to analyze things to determine a pattern and act upon those findings, but given that things are in flux, this is probably not a good use of time.

In contrast, for queries on specific or unusual products, you'll probably stand a better chance of ranking well simply because there are fewer people listed with those products. The good news here is that this is an easy situation to optimize for. Simply get a data feed going as soon as Froogle will allow. Ensure that your products are fully described in the product name and description fields you provide, and you should be able to come up for the more specific queries naturally.

Affiliates Not Wanted

Who can't be in Froogle? Anyone who doesn't sell things directly, which especially means affiliate sites. For example, if you've created pages for different books, which people ultimately purchase by clicking through to Amazon on your affiliate link, Google doesn't want to include your pages.

"Our concern is that the user experience is optimal, so if we present results on Froogle, we want people with a single click to get to where they can buy something," Nevill-Manning said.

Building Categories

Now for a few closing notes, starting with the category structure you'll see on the Froogle home page. How is Froogle coming up with this structure? Editors created the initial framework, then hand-selected some product pages to fall within particular categories. In some cases, this is made easier in that product sites themselves may have their own category structure, in which they list their products.

Once the initial categories were "seeded," the product pages then served as "training documents" to help Froogle categorize new pages. Also, now that the data feed structure is being established, site owners can indicate the Froogle category they want pages to be associated with. However, this isn't a required step. Instead, you can use the category names from your own site, and Froogle will work with these.

"All we are asking is to give us whatever they display on their site [as a category name” and then we'll take it and map it to the Froogle hierarchy," Nevill-Manning said.

In Froogle Does Not Mean In Google

Earlier I noted that if you are listed in Google, you may also be listed in Froogle. However, it doesn't work the same in the opposite direction. There will be pages in Froogle that are not listed in Google.

In particular, if you provide a data feed, that information remains in Froogle only. If some of the same URLs in your data feed also appear in Google, this is because the Google crawler has found them on its own, not because the limited data from your feed is flowing into the Google web page index.

Also as said earlier, Google is not charging for people to be included in the index. However, could Google be signing up as an affiliate to various sites selling products, then using affiliate links within Froogle as a way to earn money? It certainly is possible, but that's not happening, the company says.

Price Sorting And Other Changes

How about the big missing feature, the ability to sort by price?

"I'm sure well get there. Price is a little tricky, because for a very general search, it doesn't really make sense," Nevill-Manning said. "For a specific search, it makes more sense. I agree, that should be something that comes."

So keep your eyes open, for that and other features that Nevill-Manning says will appear every few weeks, going forward. Among these will be the inclusion of non-US web sites, once Google feels things are stable enough to expand more.

Online Shopping with Google's Froogle
SearchDay, Dec. 12, 2002
http://searchenginewatch.com/searchday/02/sd1212-froogle.html

Google has launched a beta version of a new shopping search tool called "Froogle" that the company claims is the most comprehensive product search engine available on the web.

Google Encroaches on eBay's Turf by Bringing Buyers and Sellers Together
AuctionBytes, Dec. 13, 2002
http://www.auctionbytes.com/cab/abn/y02/m12/i13/s01

Touches on how eBay content may be included in Froogle listings.

New Test Site From Google Focuses on Products for Sale
New York Times, Dec. 13, 2002
http://www.nytimes.com/2002/12/13/technology/13ONLI.html

DealTime, an established shopping search engine, made $30 million this year through merchant partnerships. Google is shunning the merchant inclusion or transaction fees, instead expecting to make Froogle pay for itself through paid listing advertising that runs alongside Froogle's editorial results.

Froogle Discussion
WebmasterWorld.com, Dec. 12, 2002
http://www.webmasterworld.com/forum3/7646.htm

Lots of discussion of Froogle among search engine marketers, with ample comments interspersed from Google representative GoogleGuy.

So, You Could Ask Jeeves About the Price
InternetNews.com, Dec. 10, 2002
http://www.internetnews.com/ec-news/article.php/1554621

Ask Jeeves added shopping search to its site this month, as well. Rather than produce the service itself, PriceGrabber.com powers shopping search at Ask Jeeves.

First FEO Tip!
WebmasterWorld.com, Dec. 13, 2002
http://www.webmasterworld.com/forum3/7682.htm

FEO -- that's Froogle Engine Optimization. A few tips on doing better, or just getting listed, are beginning to be shared.