GlobalBrain is technology that uses clickthrough data combined with user data to deliver highly targeted search results. It has been developed by Dr. Grant Ryan and a team of programmers in New Zealand, and is being marketed to search firms by a Bay Area business development consulting company called Double Impact.
"Based on the incredible interest and feedback we've received from test users, Portal CTOs and CEOs and industry analysts, we expect to solidify a major exclusive relationship and release the technology to the public within a few short months," said Double Impact principal and GlobalBrain board member Steven Marder.
At the core of GlobalBrain is a system of measuring what users select from search results. That sounds like Direct Hit, which currently has a partnership with HotBot. However, GlobalBrain is not a copycat system attempting to cash in on Direct Hit's success. I heard about both technologies at nearly the same time, about six months ago, before either launched publicly. It seems very much the case of a good idea -- counting clicks -- being developed independently.
Both systems watch to see which pages users actually select from among the search results. These pages are considered better than others, especially if a person spends much time viewing them. In this way, searchers invisibly vote for the pages they like.
Direct Hit uses this clickthrough data to rerank HotBot's "normal" listings, and it often produces more relevant results. HotBot users tap into Direct Hit by performing a search, then selecting the "Top 10 Most Visited Sites" link that appears above the HotBot results.
GlobalBrain steps beyond Direct Hit with its concept of profiles. GlobalBrain asks searchers to voluntarily register themselves by country, age, occupation, favorite sport and gender. This user data is then linked to clickthrough data. This allows GlobalBrain to deliver results targeted to particular profiles, an extremely powerful feature.
For example, a person living in the United States will have a completely different concept of what is relevant for "football" than someone who lives in the United Kingdom. With profiling, GlobalBrain can intelligently deliver results about American football to the US surfer, while the UK surfer gets information about what an American would call soccer.
Likewise, profiling can be done by interest area. At Excite, a search for "giants" brings up special information about the New York Giants football team. That's a great enhancement, unless you are a fan of the San Francisco Giants baseball team. With GlobalBrain, someone who indicates that their favorite sport is baseball, or that they live in San Francisco, would be more likely to get information about the Giants baseball team, rather than the football team.
GlobalBrain also offers an innovative keyword suggester that leverages what people click on to build a thesaurus of related terms.
For example, imagine someone searches for "bicycling" and then selects a page from the results that are displayed. They may then search for "cycling." Because the two keywords were entered one after the other, the system assumes they may be related. The next time someone searches for bicycling, it will then suggest "cycling" as an alternative.
The system also measures which of the suggested keywords users click on. This helps it learn which keywords are best related to each other, in the same way it learns which web pages people find useful.
Currently, Excite is most advanced in this area of suggesting alternative words. It watches for links between words that appear on pages, rather than within searches. For example, if it sees many pages where Apple appears with computer, it may then suggest computer as a word that should be included in the search.
The GlobalBrain solution is not necessarily better than what Excite does, but rather another approach toward achieving the same goal.
GlobalBrain also provides automatic bookmarking, which is probably more understandable as saved searches. The system will show you past searches, along with the link you visited and found most relevant for each keyword. This bookmarking is also used to help refine search results to an individual level.
The data gathered through bookmarking means that two people with identical profiles (country, sport interest, occupation, etc.) might still get two different sets of results in response to a search, because the system is tailoring matches to what they've selected in the past.
One possible concern with this automatic mechanism is the fear that searches could be tied to an individual. I've long been expecting for some journalist to track down what a politician is searching for while at work, in the way that some have sought video rental records.
It's theoretically possible to do this now, but only with a great deal of difficulty. For example, personalization services at Yahoo and Excite, among others, allow for search topics to be saved. This makes it easy for a user to run the search in the future -- you just click on the saved link.
Someone who could access this personal information might be able to discover if some embarrassing searches are being stored. However, the first problem is that the companies aren't likely to release it willingly. In fact, several of them have just pledged ad space to promote privacy issues.
"We would never communicate data on an individual level," said Excite search product manager Kris Carpenter.
Second, it would be difficult to conclusively prove that any individual actually created an account, Carpenter pointed out. After all, no positive identification is required to open a personalized account.
For example, someone could go to a coworker's computer, quickly create a fake account, then store some embarrassing searches. The computer could be linked to the account via a cookie, but there's no way to prove that the computer's owner actually created the account.
Beyond these problems, the main difficulty in gathering this data would be that most people probably have not bookmarked their searches. Someone who actually managed to force their way into an account by some means would likely discover nothing incriminating there.
This is why GlobalBrain's system might cause a concern with some people. It would store searches automatically. All the other hurdles in getting this information would remain, of course, but people can still be hypersensitive about privacy. To address this, GlobalBrain's solution would be offline storage of searches.
"Privacy has been key concern when designing the GlobalBrain system, hence there will be an option to download software that stores the personal bookmark information on a users computer, said Ryan, "The only person that will ever have access to this data is the person that created it"
In addition to the above features, GlobalBrain also provides an ability to sort pages into different groups, such as "popular" or "high flyers." Popular displays pages just as Direct Hit does, where pages are ranked in order of popularity based on clickthrough and time viewed. High flyers list pages in order of those with increasing popularity. A "New" option shows the newest pages that have been found relating to particular terms. "Past Preferences" shows results based on a user's individual preferences.
By default, GlobalBrain results show a mixture of the four groups. In this way, relevancy is kept high, yet an element of discovery is also mixed in.
GlobalBrain even makes some provision for the use of clickthrough relating to the submission process. The basic idea is pages would be submitted with associated keywords and a targeted profile. In this way, a site owner could submit a page intended only for a UK audience, or for a baseball fan. If the page ranked well, it would keep its ranking only if people were visiting it.
While the system is meant to discourage spam, I think it would actually attract more, at least in terms of submissions. However, the concept of linking pages, or perhaps even sites, to profiles is an interesting one that warrants investigation. One easy twist might also be to require registration for those submitting pages. Such a system could even learn if pages from some submitters are ranked well by users, perhaps thus giving future submissions priority.
GlobalBrain is taking a different strategy with its technology than Direct Hit. That company is offering its technology to anyone, through branded partnerships. In contrast, GlobalBrain has been talking with various search engine companies to find a single partner to purchase its technology.
GlobalBrain says that two companies are currently testing its system, to see how it works in the real world, and a third is about to evaluate it. Ultimately, it hopes the system will launch with one of the companies within a few months.
"The system has worked so well in our extensive trials that we are very excited about launching it and providing users with a more satisfying and relevant search experience," said Ryan.
As for Direct Hit, it sees any deal that GlobalBrain may cut as strengthening demand for its technology.
"I think GlobalBrain's entry into this field validates the Direct Hit technology of user-based popularity ranking as the best method of organizing Internet search results, as compared to link-based methods," said Gary Culliss, chairman and founder of Direct Hit.
Additionally, Direct Hit has been developing its own profile-based searching system. As with GlobalBrain, it would deliver results tailored to particular types of people, based on their profiles.
Privacy ad campaign to launch
News.com, Oct. 7, 1998