Lycos & FAST Launch Paid Inclusion

After months of beta testing, FAST has finally opened its paid inclusion program to the general public. However, FAST is not selling inclusion into its web index directly. Instead, FAST is allowing its search partners to resell the service to web site owners and site promoters.

"We're not going to sell this direct," said Stephen Baker, FAST's director of business development and marketing. "We view it as a service for the portals and our partners to sell and make additional revenue."

Terra Lycos is the first of these partners to offer FAST inclusion through a non-beta program, via the "Lycos InSite" programs that opened to the general public last week on the web site. The programs lets site owners purchase either inclusion into the FAST web page index or a bundled inclusion and site search product.

"Lycos InSite Select" is the pure inclusion product, guaranteeing that a page will be added to the FAST web page index within 2 days, then updated every 2 days within any changes, for a year. That FAST index provides the main editorial results on the Lycos search results page, and the index is also used by all other FAST partners, as well as FAST itself, on the AllTheWeb site. The price is $12 per URL per year, in addition to an $18 annual membership fee to open an account.

As is the same with paid inclusion programs offered by other major crawlers, entering a URL through the program does not guarantee that it will rank well for any particular term nor give it a ranking boost.

"Lycos InSite does not affect the ranking of a site within the results," said Tom Wilde, general manager of search services for Lycos. "We do not alter the relevancy of the search engine based on paying versus non-paying inclusion customers. The lower cost of the paid inclusion service relative to paid placement services reflects this."

If you do want a guaranteed listing for a particular term at Lycos, the way to do that is through Overture. Lycos carries the top three listings from Overture for any particular keyword in the "Sponsored Search Listings" area of its results. In contrast, paid inclusion means you'll be in the database used to find matches for the "Web Results" section of the page. You aren't guaranteed to show up for any particular term, but by having greater representation in the database, you may find that you gain greater traffic.

When you enter the program, you'll need to "Label Your Site" to begin. Why? Because in the future, you'll be able to have multiple groups of URLs all managed under the same account. Watch for that to come in the spring, Lycos says.

Once you've selected a label, you'll then need to choose whether to do "Standard Submission" or "Advanced Submission."Make your choice carefully, because it influences what gets picked up.

Using "Standard Submission," FAST will begin crawling at the URL you specify and work its way "down" into your site from there. It will be difficult to predict exactly what it will pick up, because it will try to determine unique paths into your content, then follow each path in turn, up to the numbers of URLs you've paid for.

In contrast, using "Advanced Submission," you can list each and every URL you want included. This is a more accurate way to ensure only what you want to pay for gets picked up, but it is more time consuming, in that you have to assemble a URL list.

Assuming you go the Advance Submission route, FAST allows you to mix URLs from different domains under the same account. In other words, you could submit these five URLs:

Mixing URLs can be good, because it means you do not have to keep paying an $18 annual membership fee for each separate domain.

Once you've made your URL choices, you'll next have to classify your web site as being "Content," "Ecommerce" or "Adult." More in-depth explanations of each classification are listed on the submission form.

What happens to this classification information? It's used for Lycos market research, to help determine future webmaster-oriented offerings. Your choices to not help rank your site or classify it in any way. So don't panic, especially if portions of your site seem to fit one category, while others fit another.

After entering your information, you'll need to provide payment details. Eventually, once everything has processed, usually within two days or less, you can manage your account through the "Search Services" area. One key feature lets you "Edit existing page URLs," which means you can swap URLs without charge, a real plus.

Why would you do this? Let's say that you submitted a URL initially. It gets listed, but you don't see it sending you any traffic. You could optimize the page, making changes that you think might improve its ranking with FAST. Another solution would be to swap in a new page to be included, in place of the old one. That new page might start ranking well on its own, saving you time and not costing you more to make the swap.

In short, you pay to have a number of URLs included, but the specific URLs can be changed at any time. There is currently no limitation on swapping. Be aware, however, that it will take two days for your new URLs to show up, after a swap.

What about the URLs that were previously included or when the annual fee runs out, if you don't renew. Do they get dropped?

"When a user cancels their subscription, pages that existed in the FAST index previously will be added back into the index, with no service level guarantees. All other pages are added to the free submission service with no guarantee of inclusion," FAST's Baker said.

In other words, if the page was in FAST before, you stay in, though you'll be revisited on a more "regular" basis. If you weren't in FAST before, you get dropped -- though you could get picked up in the future for free.

The Search Services area also provides other reports, such as whether there were problems spidering your selected URLs and a log of spider visits to your URLs. A report showing all the sites linking to your URLs is promised, though you can get this information manually by doing a search at FAST itself (see the Checking Your URL page below).

Other reports are promised to come later this year, including those showing clickthrough on your included URLs and rank checking, along with more.

"I don't want to be too specific for competitive reasons, but suffice it to say it is our strategy is to make the reporting as valuable to the customer as inclusion is. Lycos InSite will transition into a search engine marketing tool as much as a paid inclusion/hosted site search service," Wilde said.

A search engine marketing newsletter with tips is also promised to appear in April.

There are some restrictions on what you can submit through the program. Content that is illegal or infringes on copyright or other rights of third parties is specifically ruled out. Mirror sites or pages that just redirect to other pages are also banned. It's also against the rules to "spam" via the program, and FAST provides some general guidelines about what it considers spam on the FAST Spam Policy page, listed below.

FAST further said that doorways pages and cloaked pages are not allowed through the self-serve programs. However, cloaking via XML feeds is OK, for those making use of the custom quote option for bulk submitting, described further below.

"We do this because it is much easier to monitor the XML feeds and cloaking/redirecting practices of such large customers," said FAST's Baker.

Though some help files at Lycos and FAST say that pages with &, %, =, $ or ? symbols in them cannot be processed, this is incorrect. If you are using paid inclusion, they will be added. Pages like these may even be crawled and included for free. FAST says this wording is being removed.

Help files also advise that pages with the same titles as other pages in a site won't be indexed. This is also incorrect, FAST said when I followed up on it. I'd expect to see that removed shortly, too.

The "Lycos InSite Pro" program bundles inclusion into the FAST web page index and provides you with the ability to make your own site searchable. By installing simple templates, you can have a search box on your site that lets visitors search against the URLs you've paid to have included with FAST. This is in addition to getting those URLs listed with the FAST web-wide index. Pricing is done on a monthly basis, $189 per month for up to 250 pages, then $279 for up to 500 pages.

The hosted search offering leaves me less than thrilled, because despite paying a monthly fee, Lycos still imposes its branding on your web site. The search box and search results pages in hosted search will carry the Lycos logo. I think the branding should only happen in a free hosted search option, such as those offered by people such as Atomz, MondoSearch and Google. Generally, when you pay for hosted search, the branding goes away. That should be true with Lycos, as well.

In addition, the current templates force you to show visitors an option to search the entire web using Lycos, when you might prefer not to offer this. Lycos says that it sees the option as a feature webmasters may want. However, if it turns out that this isn't so, changes will come.

"Clearly, if theres strong objection about that, wed hear about it and respond to it," said Wilde. He also said that Lycos will "aggressively roll out" enhanced customization features for hosted search service this spring.

To me, the real plus for going with Lycos Insite Pro is if you want to include 188 or more URLs with FAST. Using hosted search saves you money if you have that much volume or more. For 250 URLs, using the monthly hosted search option saves $750. For 500 URLs, hosted search saves you $2,670 per year.

Remember, you can use the hosted search option even if you never implement it. Given this, it makes no sense to use the non-hosted search Lycos Select option, if you have more 188 URLs to include.

Also be aware that you can mix and match URLs, even if you are using the Lycos Insite Pro option. So, again this makes sense as a price saver, for those of you who have multiple clients you want to include. However, also be aware that if you do mix and match URLs, you'll have a problem if you do decide to implement hosted search. The URLs from different sites will be included in the results.

Both Lycos InSite services are self-serve programs is designed for those with up to 500 pages that they'd like to have indexed. Over that amount, pricing is done on a custom quote basis. XML is the preferred way to take in bulk feed of pages. Pricing is done on a flat fee per URL basis, though lower than with the self-serve program, and usually has a cost-per-click component, as well.

"We are flexible, however, and can provide flat rates for customers with budget limitations. Pricing is determined by the type of content that is provided and the expected ROI. We plan to work with these customers on a small amount of data for a couple of week trial period to understand the ROI that they see from being included in the index. The final pricing is determined after this is understood," said FAST's Baker.

What about free Add URL? Lycos still offers it, but submitting there is no different that submitting via the AllTheWeb Add URL page. The advantage to using the AllTheWeb Add URL page is that you don't need to open a Lycos Network membership. That's free, but you may not want to have one, just to submit.

Regardless of which method you use, there's no guarantee that your page will show up, just because you submit. FAST says URLs, if they are going to appear, will do so within 6 weeks. It is allowed to submit multiple URLs in a "deep submit," but that doesn't really help ensure they'll get included. Instead, the best method to get your pages indexed is to build links from quality sites to your site. That will help FAST's link crawler naturally find you.

You might also notice slight differences between the results at Lycos and AllTheWeb, both in ranking and in counts reported. Why is this happening? Lycos says that at its site, they boost pages written in English, since the portal is targeted toward English speakers in the US. In contrast, AllTheWeb has a more international mix of traffic, so boosting pages in English isn't done there.

In addition, beyond ranking, what's actually retrieved may be filtered out -- both at and AllTheWeb. That's why you'll see some different counts, where more matches may be found at one or the other of the services. I'll see about getting more clarification about this, in the future.

And in some other news, you may recall in an earlier newsletter I wrote about the "Start Here" links that Lycos has long sold. Kanoodle had prepurchased some of these links for certain terms, allowing the top bidder at Kanoodle to show up as the Start Here link at Lycos.

That program with Kanoodle is now being phased out, Lycos says. In addition, the company is planning to bring out new features for the Start Here links. The How Lycos Works page listed below gives more details about them.

How about Europe? Lycos Europe, a separate company 30 percent owned by owner Terra Lycos, is beta testing paid inclusion for its Lycos Germany web site. The program cost 38 per year for the first URL, then 19 for each additional URL, including German VAT. That's about the same amount charged through the US program. There are are also bulk prices, 2,450 for 250 URLs and 3,950 for 500 URLs. Those prices are exclusive of VAT and must be ordered by telephone.

Inclusion via Lycos Germany gets you into the FAST web index just as inclusion via Lycos US will. In other words, you only want to do inclusion via one program, because either will get you into FAST. Using both programs is a waste of time and possibly a waste of money. That's because the terms at Lycos Europe is not checking to see if someone has submitted via the program and won't issue a refund, if you use both program. So, pick ONE program and don't duplicate URLs.

There's also no need to use the Lycos Germany program to be seen as "German" as opposed to the program to be seen as "US." Both programs feed into the same FAST index and don't attach any country-specificity to your URLs.

In fact, all Lycos Europe sites -- Austria, Belgium, Denmark, France, Germany, Italy, Netherlands, Norway, Sweden, Switzerland, Spain and Russia -- use the same FAST index as does Thus, if you are included in FAST, you could possibly come up, in a search using the "International" option. Using the local option may bring your pages back, if these sites also depend on FAST for local web searching and your content is seen as specific to that particular country, based on language, domain or links to it.

So why choose one over the other? For those paying in euros, the Lycos Germany option might save you a bit depending on the exchange rate with the dollar. You'll need to check on this when you are about to order. However, when it comes to the bulk options, the hosted site solution offered through Lycos US works out much cheaper than getting the same amount of URLs with Lycos Germany.

FAST inclusion via Lycos Europe is only offered through a bundled directory submission option. Lycos Europe operates its own, "homegrown" directory of web sites to help power its listings, unlike Lycos US, which makes use of the Open Directory to power the "Categories" portion of its results.

Given this, if you want to be found via a particular Lycos Europe web site, you'll want to do a submission to the Lycos Europe directory. For those with budgets, this is easier through a paid submission program offered in Lycos Germany and coming in May to Lycos UK and France.

The program charges 147 (including VAT) for a yes or no answer to being accepted, within five business days and will list you for one year. The new "Surftipp" program provides even more prominent placement, for 928 (including VAT) per month. Pages with erotic content can use similar programs, but they pay much more to be listed.

After you've made your directory submission, you can then "add on" paid inclusion to the FAST crawler. By May, a crawler-only inclusion option will be made available.

Just as Lycos Europe is run by a company separate from Terra Lycos, which runs, so there are other "Lycos" web sites that are not wholly-owned by Terra Lycos. Examples of these are Lycos Korea, Sympatico Lycos and Lycos Japan. They are also important because they do not use FAST for their search results. FAST only added Asian language support last fall, which the company says is one reason why the Asian sites may have gone with other providers.

Lycos Korea makes use of Google results through a new partnership established in October 2001. If Google has found your web site, you are available there -- and if you have pages written in Korean, using a Korean domain or linked to from Korean web sites, you'll be more likely to come up for Korean-specific searches, Google says.

Sympatico Lycos is a Canadian portal that also uses Google, and it carries both Google editorial results and Google AdWords, allowing you to purchase placement at Sympatico Lycos via Google, if you choose to target the Canadian market through the Google AdWords program. This deal began at the end of January. Sympatico Lycos is 71 percent owned by Bell Globalmedia and 29 percent owned by Terra Lycos.

As for Lycos Japan, results there are powered by Wisenut, in a deal that was announced last December.

Finally, what about the old FAST PartnerSite service that the company beta tested for several months at the end of last year? It's still out there, but it's not something consumers can purchase. Instead, that's what FAST partners resell, using different names such as Lycos InSite.

Lycos InSite

Learn more about the paid inclusion programs from FAST & Lycos or signup here.

Lycos InSite FAQ

Answers to many questions about the paid inclusion program are summarized on this page from Lycos.

Checking Your URL

Learn how to find who's linking to you on this page.

FAST Spam Policy

Lycos Insite Custom Quote Form

Use this page to get a quote for submitting more than 500 pages through paid inclusion.

Lycos Launches Paid Inclusion
SearchDay, Feb. 25, 2002

Chris Sherman touches more on how the Lycos hosted search solution compares to those offered by Atomz and Mondosoft.

Need a site search solution. Avi Rappoport's site gives you the full rundown on remote-hosted as well as software-based solutions.

How Lycos Works

Basic summary on how Lycos assembles its search results page, for Search Engine Watch members.

How FAST (AllTheWeb) Works

Explains how the FAST crawler gathers its listings, for Search Engine Watch members.

Fast/AllTheWeb Add URL Page

Free Add URL page for FAST.

Lycos Europe Express Directory Submission

Information about paid inclusion and directory submission programs from Lycos Europe.


Powers some results at Lycos Korea and Sympatico Lycos.

Lycos Korea

Sympatico Lycos


Lycos Japan

Contains about 80 million pages crawled by WiseNut.

More About Countries and Languages

Tips on making your web site country-relevant to country-specific search engines, for Search Engine Watch members.

FAST Records Record Earnings for Q4 and 2001
FAST Press Release, Feb. 13, 2002

FAST announced a $2.1 million profit in its last quarter, at least on a EBITDA basis and excluding share-based compensation, equity in losses of investee and non-recurring items. That's up from a $1.1 million loss in the previous quarter. The company also recently raised $28 million in funding.

Cost cuts help Terra Lycos narrow loss
Reuters, Feb. 26, 2002

Terra Lycos lost $36 million last quarter, an improvement over the previous quarter.