A longer version of this article for Search Engine Watch members looks at how clickthrough refinement has been used in the past, why it failed before and examines how Eurekster creates personalized results.
Finally, personalized search results seem to have arrived for the masses. Personalized search has long been promised as an important next step for increasing relevancy. Now it comes not from Google or Yahoo but instead from tiny Eurekster, which opens to the general public today. Prior to this, the site had been a beta test involving only a few hundred people over the past couple of months.
Personalized search? The concept has been that by knowing some things about you, a search engine might refine your results to make them more relevant. A teenager searching for music might get different matches than a senior citizen. A man looking for flowers might see different listings than a woman.
Eurekster's twist on this concept is to provide personalized results based not on who you are but who you know. Friends, colleagues and anyone in your Eurekster social network will influence the type of results you see.
"Word of mouth is the most common way we filter information in real life, and Eurekster amplifies this everyday process to deliver search results that matter most to users and their friends and contacts," said Grant Ryan, Eurekster's chief executive officer.
Search Filtered By Friends
For example, take a search for "thunderbirds," the popular Gerry Anderson television show that I loved as a kid and that my children are now enjoying on DVD. Without refinement, Eurekster brings back a variety of results, leading off with the US Air Force Thunderbirds flight team web site.
There's nothing wrong with the US Air Force site coming up first. After all, it's certainly relevant for this search. But I'm thinking of the sci-fi series. Scrolling down, I find the official site for the show listed in position three. By clicking on the site, my interest in it is registered by Eurekster.
Now picture that one of my friends does the same "thunderbirds" search. They'll see different results, based on my actions. The Thundersbirds TV show site moves up to position one, with a little Eurekster "e" icon next to it. That's to alert them that someone in their network likes this site. Since most of my friends are into sci-fi, this search will probably feel more relevant to them.
The potential of using your friends or colleagues is enormous. Imagine Eurekster being used by all the employees of a medical research firm, where many might do similar medical-related queries. With Eurekster, all the employees can be linked together and benefit from the searches and selections made by their colleagues.
Libraries are another institution that might latch on to the Eurekster concept. Librarians are constantly asked by patrons for assistance. Eurekster would allow librarians to collaborate invisibly with each other and share what they've found to be the best for various queries.
There are downsides. Not all of my friends have the same interests as me. In addition, as my social network grows -- because my friends invite their friends and so on -- commonalities that are useful get diluted.
Some type of categorization will almost certainly be necessary. I can imagine dividing my network into subgroups, for example. All my colleagues who cover search would be in one group, my fellow youth group volunteers in another, my friends who live in California in their own and my friends who live in the UK in a fourth. Depending on the search I do, I might want to share and refine my search in some way with a particular group, to benefit from a particular commonality.
Such functionality is coming over the next few months, Eurekster says, as well as the ability to group contacts by interests, such as sports or business. In that way, you can also get a sense of what's hot by those who are into particular subjects.
"You can see what the search results are like with different hats on and do that different categorization," Ryan said.
Sharing Searches, Sites
Eurekster does more than refine your results. It can also show you the things that other people in your network are looking for. So when I searched for "thunderbirds," anyone one else in my network sees that query get listed in the "Your Friends Recent Searches" area in the right-hand column of the Eurekster search results page.
This can be pretty cool. When I saw "Firewire backups on sale" show up in that area, I knew one of my friends was looking for a Firewire backup solution. As it turns out, I'd just bought a similar solution of my own. So, I clicked on the small envelope icon that appeared next to the query. This let me email the person, to say I had info they might be interested in.
My network is incredibly small -- all of three people right now. I had a pretty good idea who had done this search, but I wanted to just test out the functionality. The idea behind it is more of the opposite. If you see a search that someone has done, you can get in touch with that person perhaps to discover what they found.
The email feature only appears for those in your direct network or extended network (where other networks are linked to yours by a common contact). Don't want to get emailed? Then you can opt-out using your account preferences.
I actually thought it might be useful to have an option where you can choose to let anyone email you, even if they aren't part of your network. To me, it could be a great way to build up new people, by spotting queries that you have in common. It could come, but Eurekster is keeping things limited, for the moment.
"As always, when you start a service, you try to set the default to what you think people would like," Ryan said.
Of course, what your friends look for is supposed to help you more directly, by influencing the actual listings you see when you search for the same thing. So, you needn't email for help. Just search for the same thing that you've seen someone else look for. Any site they liked, as determined by Eurekster, will be flagged with that "e" icon.
Eurekster will also display "Your Friends Recent Sites," which are sites those in your network have recently been to. It also keeps track of the top queries and sites visited by those in your network, as well.
While I love my friends, I don't want them seeing everything I search for. And in a small network like mine, it's pretty easy for others to guess who might have looked for something.
To protect privacy, Eurekster makes it possible to do a "private search" by checking a box below the search box. Do this, and your search will not be shared with others. Should you forget, you can click on a small X symbol that shows up next to any queries you've done, to remove them.
Both opt-out features are welcomed, but I'd still prefer that the default action be that all searches are private. The downside to this is that if people don't remember to share their searches when appropriate, the network doesn't function as it should. However, I think that by making all searches public by default, people will inevitably share material they don't want out there.
It's important to note that your actual name is never displayed alongside a query or a site that was visited. In addition, the larger your network gets, the more anonymous you become. Say you have 30 or more people in your network. Guessing at who did a particular search or visited a particular site becomes harder. Also, a porn filter does prevent porn queries from being displayed, a further bit of protection against some possibly embarrassing queries.
Aside from privacy from your friends, what about privacy in general? Yes, you will get a cookie with a unique user ID, which is the main issue that some have had with other search engines such as Google. Unlike Google and other major search engines, the cookie doesn't last for years. It expires after a year, if you don't revisit Eurekster.
Of course, being cookied with a unique ID or cookied for a long time isn't necessarily a problem, and my Search Privacy At Google & Other Search Engines article from last year covers this in depth. Instead, Eurekster brings in the entirely different issue also addressed in that article, that you are a known, registered individual (rather than an anonymous cookie ID) and linked to particular searches. That makes it much easier for your data to potentially be abused.
The How Eurekster Works page provides some assurances about privacy. The company says it won't reveal who did a particular search or visited a particular site. However, I suspect it would do this if legally compelled to. The policy also doesn't address how long your searches are retained. Some like Google Watch and the EFF believe such data should be destroyed on a regular basis.
Ultimately, anyone with serious concerns about privacy may shy away from the service, something Eurekster recognizes. In order to work properly, it does need to monitor what you search for and what you visit. The company hopes the promise of anonymity and extra tools to explicitly keep some things private will satisfy most concerns.
"To do personalization, we have to have some information, and we give you the ability to delete that information to make it private. In the future, we will give users even more tools by which to further control their data," Ryan said.
Google Replacement Or Enhancement?
Eurekster is not a replacement for Google, Inktomi or other search engines that crawl the web. Indeed, Eurekster has no listings of its own. Instead, it works to refine the results that are provided by others.
When you search on Eurekster, the listings are coming from Yahoo-owned AllTheWeb, which is an excellent search engine. Eurekster then alters those results depending on the choices made by your social network.
Eurekster can also do this for any other search engine, as well. AllTheWeb is being used because the company has an agreement with to use its listings (and also carries Yahoo-owned Overture paid listings, which appear under the Sponsored Search Results heading). Eurekster could strike an agreement with other companies and do the same on their own sites.
"This doesn't replace the algorithmic [crawler-based” results," Ryan said. "We want to partner with anyone and everyone. We see it as an additional layer that adds value."
The company says some search companies have expressed interest, though it wouldn't name any. Eurekster's sister company SLI has a relationship with some smaller search companies, such as Comet Systems, Excite Networks and CNET's Search.com to provide related searches technology. So, perhaps we'll see some movement with these partners first.
Eurekster also sees its system as something that may be of interest to social network web sites. These are places such as LinkedIn, Friendster, Tribe and Rhyz. They let you meet people based on who you already know, and they've been gaining recent attention from the press and surfing public.
"There's lots of people who want to grow their own networks, so we're kind of network agnostic," Ryan said. "We're happy to work with anyone's private network."
Incidentally, it was rumored that Google tried to buy Friendster last year. While that doesn't seemed to have panned out, Google perhaps is still interested in starting a social network of its own. Evidence? Try a search for social networks on Google and notice that an ad for jobs at Google comes up. Is it recruiting? If so, I didn't find any jobs explicitly mentioning this.
If not social networking, we certainly know that Google has a number of personalization technologies at its disposal. Google bought Kaltix last year, a start-up company of three people that attracted attention by promising undisclosed advances in personalized search. Back in 2001, Google acquired Outride, another firm experimenting with personalized search results.
Choose Your Engine?
One feature I'd love to see is the ability to use Eurekster with your favorite search engine. I think it would help in getting people to try it. Those comfortable with Google, for example, might want to run Eurekster "on top" of Google's results.
Such a thing will require Eurekster to strike new agreements, and it doesn't have one with Google, for the moment. But the company did express interest in my idea that perhaps it could let users make use of their own Google API keys, in order to legitimately call Google's results into Eurekster. If this happens, I'll pass along more details.
Will Personalization Fly?
I started out by calling Eurekster tiny, but the company has a wealth of search refinement experience. Go back in time to GlobalBrain, which was a promising search technology that I wrote about in 1998. GlobalBrain was founded by brothers Grant and Shaun Ryan. Snap -- then later called NBCi -- bought GlobalBrain but failed to take the technology forward as the company pulled back from search in the wake of the dotcom crash.
The Ryans bought back the GlobalBrain technology and founded SLI Systems. As mentioned, SLI systems has been using it so far to provide related search functionality to a number of minor search sites, handling 300 million queries per month, the company says. Now SLI is one of the cofounding companies behind Eurekster (the other being social networking company RealContacts, also founded by the Ryans).
Will the second time out be lucky for the Ryans? It remains to be seen. But one thing's for certain. Unlike back in 1999, the majors are showing a real interest in personalized search.
Google (see News.com), Yahoo (see Wall Street Journal) and AOL (see Search Engine Watch) have all made public statements that they view personalization as a vital step forward in improving search -- and as mentioned earlier, Google even owns at least two companies, Kaltix and Outride, that were involved with it.
Eurekster is first out of the block for 2004, but we'll likely see the others follow, in some way. Past issues about privacy and user resistance will remain, (see Google May Get Personal & Searching for the personal touch), but there appears the will on the part of the majors to solve these issues.
NOTE: Article links often change. In case of a bad link, use the publication's search facility, which most have, and search for the headline.