The change being implemented on July 1 will not coincide with the launch of MSN's own search technology. A crawler-based MSN search engine has been in development since last year, but there remains no announced launch date for this.
Instead, the redesigned MSN Search site being unveiled in July will continue to be powered by Yahoo, as it currently is today.
Fixing Featured Sites
MSN Search has long had a "Featured Sites" area at the top of its search results page that contained a mixture of editorial picks, promotion of MSN's own content and paid ads from MSN advertisers. It is these ads that potentially put MSN in variance with FTC recommendations.
Issued in 2002, the FTC recommendations called for any listings where placement is guaranteed to be separated from other listings and be clearly labeled using language that indicates the listings are sold. In particular, the FTC disliked ambiguous headings such as "Featured" in favor of more descriptive ones such as "Sponsored."
MSN Search also has long had a "Sponsored Sites" area. Paid listings have appeared here through its partnership with Overture. After the July 1 change, the Featured Sites area will be removed and any paid listings -- whether sold directly by MSN or purchased through Overture -- will appear in the Sponsored Sites area.
In particular, there will be two Sponsored Sites areas. One is a box at the top of the search page, giving ads "inline" placement. The other ads will run along the right-hand side of the screen Google-style, in what Search Engine Watch calls "sidebar" placement and what MSN refers to as the "right rail." In both cases, a "Sponsored Sites" heading is clearly displayed.
MSN has long sold its own advertisements directly to advertisers, and this Featured Sites program will continue to be offered. Up to three advertisers in this program can appear in the more prominent inline sponsored search box. If all spaces aren't sold by MSN, then Overture listings will fill this box. Overture listings also appear in the sidebar boxes.
The Featured Sites area isn't quite dead. In some cases, a new "Editorial Featured Site" link will appear between MSN's sponsored results and its main results. The wording is problematic, since MSN says that this link may lead to an MSN service or partner. Should any payment be involved, this would still fall afoul of the FTC recommendations. MSN says it is still working through the details of how this link will appear.
Changes Based On Testing
MSN itself isn't pitching the change as something that was done to please the FTC. Instead, the service cites testing it has done recently and feedback received from both searchers and advertisers as reasons to clean up its search page.
"The results showed consumers were clicking more often on links, spending more often on MSN Search, and our query base was increasing," said MSN product manager Karen Redetzki, about searcher reaction.
As for advertisers, Redetzki said they were happy that the changes might help MSN broaden its reach and attract more clicks to their ads. Advertisers in MSN's own advertising program were said to be pleased that the change made them appear more prominent.
Key advertisers and other important MSN partners will hear first hand about the change today, as MSN's Strategic Account Summit begins for two days. The event gathers 400 of MSN's top advertisers and industry leaders. MSN briefed the press on its planned new look this week, so that when the details came out during the summit, there wouldn't be a big surprise.
Those who've visited MSN's search beta site recently wouldn't be shocked by the change. MSN says it has tested various different looks there over the past few weeks. But what I've seen, and what is still being shown now, looks identical to what MSN is proposing for July. That July look is illustrated below:
Main Results Continue From Yahoo
The main listings on MSN Search's results page -- those under the "Web Pages" heading -- currently come from Yahoo. These are mixture of pages found by Yahoo's crawling of the web and content obtained through Yahoo's content acquisition program, some of which involves paid inclusion.
After the July change, the main listings at MSN will continue to come from Yahoo. They simply will no longer have a "Web Pages" heading coming above them.
At some point in the future, MSN expects to replace this data with that found by its own crawler. As previously said, this won't happen in July. Instead, it's more likely to happen toward the end of the year.
"We've been telling people, later this year, look for something from MSN Search," said Redetzki.
Differences Between Yahoo & MSN
If the results at MSN are from Yahoo now, then why don't they match exactly what you see at Yahoo?
"You may see a variation of results based on the different relationships we have with each partner," said Yahoo spokesperson Diana Lee.
That's the standard answer given in the past by Yahoo-owned Inktomi, when various partners might use its data but not show the same results. Each partner might decide to perform particular tweaks, such as filtering by language or perhaps adjusting the ranking algorithm factors slightly.
It's interesting to note that MSN and HotBot -- which by default uses Yahoo's results -- seem very similar or identical while Yahoo sometimes shows great differences. It suggests that Yahoo is using its own unique ranking algorithm on its own site and providing partners with a feed using a different system. I'm following up further with Yahoo on this.
Crawler Development & Hiring
All eyes are watching for when MSN will release its crawler-based results to the public. How's the progress going?
"We're on track to the key milestones that we've set for ourselves," Redetzki said.
Some of those milestones would obviously include building MSN's search technology team. But as I wrote last month, the company recently lost one of its big personnel wins, former Overture chief technology officer Paul Ryan, for undisclosed reasons. This follows an earlier departure of a former AltaVista employee who was helping to build the Microsoft crawler, as reported by the New York Times.
MSN is also still advertising for key positions -- and a recent Newsweek story featured a Google engineer who chose Google over Microsoft in part because Microsoft's talent was only in "first grade" according to her, when it comes to search.
As the engineer now works for Google, the anecdote is obviously suspect. But what about all those job openings? How can milestones be met when key vacancies remain open?
"We don't have all the key people in place," acknowledges Redetzki. "It is such a huge effort. We're definitely not struggling for the headcount, but we are trying to get it up."
Debating Paid Inclusion Internally
If MSN is cleaning things up in terms of paid placement, then why isn't it doing more in terms of paid inclusion disclosure?
Part of the answer is that MSN is already in compliance with FTC recommendations. As long as paid inclusion doesn't provide a ranking boost -- which MSN supplier Yahoo says is the case -- paid inclusion need only be disclosed via a search engine's help pages.
MSN currently does this through an About link that appears next to the Web Pages heading on the results page. This leads to an explanation that says:
Within Web Page results, there may be links where the Web site owners have paid for either expedited review of their site or paid for clicks to their site. These sites are ranked using the normal algorithm applied to all links within each section, with no change in rank due to payment.
While it may be in compliance, paid inclusion is currently being actively debated at MSN. Redetzki said the service is pondering dropping paid inclusion listings entirely, possibly segregating paid inclusion results, perhaps labeling them if they remain mixed among unpaid results as well as maintaining the status quo.
"We are debating it right now," Redetzki said. "We're looking at test we just ran and looking at the results and trying to determine the relevancy story."
When will a decision be made? Potentially, it could coincide with the July 1 rollout of MSN's redesign, Redetzki said. However, that's not a guarantee that a decision will be made by that date -- it could go longer or happen sooner.
What About Operating System Integration?
Many assume that Microsoft will eventually "win" the current search wars by rolling out tight integration of search within its operating system. I've written before that these assumptions overlook the already tight integration that exists.
But how about it? Is MSN working closely with the Microsoft operating system division and browser team to knock out Google and Yahoo?
"We're so focused on this algorithmic [crawler-based” search engine that this is our main priority, for that to be released at some point in time, hopefully by the end of this year. Longhorn [Microsoft's next generation operating system”, that's two years plus," Redetzki said. "That's Longhorn and not MSN Search. We're not waiting for other product groups to do releases."
Certainly MSN will work with other parts of Microsoft as it makes sense, of course. For the MSN Toolbar release in January, it was MSN Research that helped develop the Highlight Viewer functionality.
"They continue to come up with all these new and exciting features for MSN search that we're looking to incorporate sooner rather than later," Redetzki said. "They also continue to look at how can we improve the overall experience for consumers and provide answers for people's questions."
Interestingly, the MSN Toolbar is no great technological wonder. It's more an attempt to keep up primarily with the Google Toolbar and actually lacks functionality like word jump buttons or a page information feature. As for search integration, MSN Search is already integrated into the browser through the address bar and the search button -- and a healthy chunk of MSN's audience comes to it this way. How much exactly MSN won't say, but it admits it's sizable.
Instead, what the toolbar really does is raise MSN's visibility as a search brand. The company says it had over 1 million US consumers download the toolbar during one week in March alone. Unlike many MSN Search users -- who got to MSN Search because of its browser and portal advantages -- these are people who are consciously choosing that they want to search with Microsoft.
"It's really our first step in finding out if people are choosing us beyond in typing in the address bar or pushing the [search” button," Redetzki said. "That's one of the key reasons that we were so excited is after these 1 million downloads."
That type of loyalty is in the long run potentially more valuable than any "in-built" advantage. After all, Microsoft has had that in-built search advantage for years and still runs behind Google and Yahoo.
While the toolbar does build loyalty, it's not yet the real marketing push that will come in the future, Redetzki said.
"We aren't after loyalty right now. We've got to build a darn good product before we are going to start marketing it," she said. "We don't feel like we are there yet."
What About LookSmart?
Last month, I wrote about how LookSmart results continued to appear at MSN Search, despite MSN having ended its relationship with LookSmart. This month, news came that MSN had signed an agreement to use LookSmart's data "for periodic use at its discretion."
Is this the same data that MSN had publicly dismissed as having poor relevancy? What's going on?
"In the US, the LookSmart band has been removed and isn't coming back," Redetzki said, referring to the layer of results that used to come before crawler-based results. She also reaffirmed that the layer was pulled to improve quality.
"We removed the LookSmart band was because of the poor relevancy," Redetzki said.
Outside the US, LookSmart data might be used in various ways depending on the needs of MSN's nearly 40 country-specific and regional search sites, Redetzki said.
LookSmart data does remain on the MSN Search home page in the US, for those who care to browse listings rather than get them through keyword searching. That's something MSN definitely wants to retain, for the time being.
"We found some consumers still using the web directory. We asked LookSmart, 'Hey, can we keep using it'," Redetzki said.
So browse from the MSN home page, and you'll still hit LookSmart data. Perform a keyword search, and you shouldn't -- at least not through any LookSmart-MSN partnership, MSN says.
There was a time when Yahoo-owned Inktomi used to carry LookSmart's individual listings as part of a paid inclusion deal. LookSmart says that deal ended in February. Because of this, there's should be no reason you'll see a URL containing a LookSmart tracking code -- with one key exception.
LookSmart says that in some cases, MSN may have "hard-coded" certain listings to come up in response to popular terms. It may be that some hard-coded links using LookSmart tracking codes may have accidentally remained. If so, these would contain "CCD" as part of the URL, which is unique to the feed LookSmart has given MSN.
For its part, LookSmart says it gets little information on how MSN uses its data and has no control over it.
"We have no guidance from MSN as to when they will or won't use or data. They're free to use or not use any way they need. They have used a layered approach with our data in recent weeks/months, but it sounds like they have discontinued that practice and are strictly using LookSmart data for category browse and some Featured Sites," said Dakota Sullivan, vice president of marketing at LookSmart.
Sullivan said that the data had been noticed by LookSmart to be used more in the recent past on Mondays and Tuesdays, which are high-volume search days.
Sullivan also defended LookSmart's relevancy, saying that the testing MSN did was only based on its UK data.
"The only testing that Microsoft ever did was in the UK, where our UK directory is approximately 1/10th the size of the US directory," he said. "Also, the algorithm they used in the UK is different than in the US and a generation older," he said.
As a result, LookSmart wasn't surprised that MSN may have been underwhelmed.
"They never did the test in the US, where we told them they'd get the relevancy that we'd promised," he said.
What's the advertiser to do and think? LookSmart obviously still has some presence on the MSN home page. While this is hardly as good as being in the keyword-driven search results, it still makes LookSmart holding an important partnership and a top second-tier paid player to consider.
As for LookSmart, it certainly sees any use by MSN of its data as a short-term win only.
"We fully expect this transitional phase with MSN will end at some point," Sullivan said. "If at some time they want to make other arrangements, well be happy to accommodate them."
Meanwhile, the company is looking at how to build up new partnerships that, while they may have nowhere near the volume of MSN, can still drive quality traffic to its advertisers. It also hopes that it can deliver such traffic at a lower price that Overture and Google, helping ease the "squeeze" that smaller advertisers may feel as bid rates increase.
"Is there going to be a place left in search where the little guy can go," Sullivan asked? "We're also finding there is quality traffic on other sites. Finding it is a needle in a haystack....but it is out there."