Searching & Navigating Via Internet Explorer
Everyone knows that the Internet Explorer browser lets you surf the web. But did you realize that Microsoft's popular browser also has some search capabilities built into it? In this article, we'll explore how IE lets you search and navigate on the web.
Notice that I said both "search" and "navigate." Searching the web and navigating the web are not synonymous. They represent different behaviors, though they are often confused. It's worth exploring the two concepts more, to better understand how Internet Explorer, as primarily a navigational tool, may still be helpful for your search needs.
Search Versus Navigation
When someone searches the web, they often have no predetermined destination in mind. For example, someone might go to Google and enter "books" in order to find an online bookseller. They get a list of possible destinations, which may likely include Amazon. From this list, they may choose to visit Amazon or one of the other choices.
When someone navigates the web, they usually do have a particular location in mind. For instance, they may wish to reach Amazon. If they know Amazon's web address, they can enter that into their browser and go directly to the Amazon web site. If they don't recall Amazon's address, they may use a search engine to locate it. While they performed a search, the intent of their request was navigational, to reach the Amazon site.
So search and navigation are radically different types of behaviors, yet people may use search engines to do both. This means that a good search engine has to be able to handle both search and navigational requests. Indeed, Google's popularity as a search engine is in large part due to the fact that it is also a good "navigation engine." It has an excellent ability to find the right site, when you can't recall how to reach that site directly.
Just as Google does double-duty, so must Internet Explorer. The browser is primarily designed to fulfill navigational requests. People who want to reach a web site should enter a web address into the browser's address bar, which is then "resolved" to help them find the destination web site. However, people also use IE's address bar to search the web.
To understand how Microsoft allows IE to perform both navigation and search functions, let's explore what happens when someone uses Internet Explorer's address bar in various situations. I'll be covering the operation of the latest version, Internet Explorer 6, though much of what's discussed applies to IE5, as well.
The Microsoft Domain Name System
In old browsers, anything that you put into the address bar was passed on to the internet's "domain name system" or DNS, to help you reach a web site. For example, if you entered amazon.com, DNS would look up to see where the "amazon.com" web site was located on the internet, then feed the location to your web browser.
Internet Explorer uses DNS, but it also performs other checks to understand how to best handle what you enter into the address bar. Indeed, IE performs so many checks that in some ways, Microsoft is operating its own domain name system, which incorporates "normal" DNS and more.
For example, let's say you enter a valid domain name such as "amazon.com" into the address bar. Doing this causes IE to check DNS to see if there's a site by that name. If so, then you are sent to it. But what happens if you enter a domain name that has no site, such as "amazonbooks.com?" Amazon owns that domain name, but they've not linked it to any site, so there's no way for DNS to "resolve" it.
Because there's no web site, DNS sends IE a message that the site couldn't be found. In turn, IE tells this to the user, but it also adds on information that it thinks will help the user locate the web site. It may suggest "similar" web sites, those that have domain names akin to what you originally entered.
IE finds these sites to suggest by spellchecking the domain you entered against a list of thousands of popular web sites it has provided to it from a company called comScore. Any domains with spellings similar to what you entered originally may be listed.
IE also provides links to web sites that it thinks may be related to the original site you wanted. These "related" links are simply the same top answers that you would get if you had searched for the domain name on Microsoft's MSN Search service. You'll be shown any "Featured" web sites, which may include advertising listings, as well as editorial results.
There was some stink raised when Microsoft moved to this new system of enhanced "web site not found" pages last fall from the old method of simply saying that the site couldn't be located.
This seemed rather unfair. It's far more useful to have the browser make some suggestions than to leave the user wondering what they've done wrong. Nor, as sometimes suggested, did the change replace the "page not found" or "404 error pages" you receive when you can't find a particular page within a valid, operating web site.
Dealing With Words
Sometimes people enter actual words into the address bar of a browser, rather than properly formatted domain names. For example, they might enter "amazon" or "hotmail," failing to add the ".com" portion that would turn those words into valid domain names.
As we'll see, depending on the words entered, IE will help users either navigate to a particular web site with the aid of the RealNames system or provide answers from the Microsoft search engine, MSN Search.
Do people know that entering words into the address bar can help them search the web? Not at first, perhaps -- but Microsoft thinks those that try it quickly learn about the search functionality built into the address bar.
"I think it is valid to assume that users discover that you can do searches by accident and or because someone tells them, and the reason I assume that is because we've never gone out of our way to make [the functionality” obvious," said MSN Search general manager Bill Bliss.
Once it is discovered, Microsoft said that users may stick with it. The company has analyzed the traffic from users who search via the address bar and discovered both that the searches appear intentional in nature, rather than accidental, and that those making use of address bar searching do so frequently.
"The people who use it use it a lot," said Bliss.
How many searches are conducted per month via the address bar? MSN wouldn't release those figures, but it did say that about 10 to 15 percent of MSN Search's overall traffic comes from address bar queries. Also, be aware that this figure does not include any aforementioned DNS errors.
The Real Names Factor & MSN Search
When ordinary words are entered into the address bar, IE's first line of defense is to check the RealNames system. RealNames is a long-standing alternative web site address system that uses real words, what it calls Internet Keywords, to link to web pages.
When you enter ordinary words into IE, it checks the RealNames database to see if there's an answer. For example, let's say you entered "amazon books." IE would see that these words have been registered by Amazon as keywords in the RealNames system and direct you to the Amazon web site.
The same is true if you entered "wells fargo bank," "american airlines" or "barnes and noble." In all these cases, the names of the companies resolve to the company web sites, thanks to deals with RealNames.
If you are entering popular generic terms, rather than big brand names, then it is generally unlikely you'll be delivered directly to a particular company's web site. This is because RealNames will not sell extremely popular generic terms to companies, nor does Microsoft want them to.
"You want to do the right thing for the user, and taking them to the right place is always a good thing," said John Krass, MSN Search's director of business planning. "If someone typed in the word 'books' and you took them to Bill's Books Store, they'd say 'What happened?'"
So for generic terms, MSN treats your request as a search action rather than a navigational action. As a result, your query will be sent to MSN Search rather than RealNames, and results from it will appear on your screen.
Given this, the IE address bar is like a built in connection to the MSN Search service. Just as those who use the Google Toolbar can query Google without going to the site, so too can the IE address bar query MSN Search -- and it's a good search engine, as well.
Nevertheless, as explained further below, you can change things so that IE's address bar will hit the search engine of your choice -- even if that's Google.
Page Not Found Error Pages
Previously, I've mentioned what happens if you try to reach a web site that doesn't exist, one that cannot be resolved via DNS. But what happens if you can reach a web site but a particular page at the site that you want no longer exists or perhaps never existed?
In these cases, the web server will send IE a "404" error code, which means that the page can't be found. Along with the code, most web servers will send the content of a 404 error page -- which is typically useless, saying nothing helpful other than that the page can't be located.
Here's another situation where IE tries to be helpful. When it gets a standard 404 page, it replaces that with what it calls a "friendly" page. This friendly page will suggest checking the spelling of the address you entered or to visit the web site's home page (though the home page links IE was showing me never seemed to work). These friendly pages will also appear in Netscape Navigator, if you have it installed on the same system along with IE.
Custom 404 Page
Some web site owners have created what are known as "custom 404 pages," which can provide very helpful information when a page can't be found. Here are some examples where you can see these:
Here are some examples of sites without custom 404 pages:
Fortunately, IE is smart enough not to override custom 404 pages, if they exist. IE looks to see if, in a page not found situation, whether the 404 page being sent from the web server is larger than 512 bytes.
If this is the case -- as it would be with the vast majority of custom 404 pages -- then IE won't replace the custom page with its own friendly error message.
FYI, you absolutely should create custom 404 pages for your web site. One reason is that if you should kill a particular page, a search engine may still try to send people to it for some time. By using a custom 404 page, you can help people more easily locate your home page, search your site or perhaps find alternative pages. Indeed, you can have whatever you want come up on those pages.
Custom 404 pages are fairly easy to make, but the exact procedure can vary depending on the web server that you use. Ask your web site administrator for help, and there are also some articles below that provider further guidance.
The IE Search Pane
While you can search from IE's address bar, IE gives you greater search functionality if you use its new "Search Companion," which is available to those running IE6 on Windows XP. Not running this configuration? Don't worry -- I'll discuss the traditional "Search Pane" option further below.
To get the Search Companion, you push IE's search button, which looks like a magnifying glass in the IE6 toolbar. Doing this splits your browser window into two, with the left-hand side being the narrow "Search Companion" portion that asks you, "What are you looking for?"
Entering your query causes results from MSN Search to appear in the right-hand window. Meanwhile, the left-hand Search Companion portion of the browser displays specialty search "tasks," as I'll call them, as well as sponsored listings.
Let's take a search for "cars" to see what happens, in the Search Companion. The lower half of the companion window is taken up by "Sponsored Links," which are paid listings from Overture for that word. The upper half of the companion window asks "What would you like to do?" and lists tasks such as "Find information about cars" or "Find used car information."
Selecting these task links tends to lead you to specialty search resources or information vendors. For instance, the used car option directs you to get information from Microsoft's CarPoint site, as well as Epinions and the Kelley Blue Book.
The tasks I've seen suggested have been very helpful, so much so that you want them to migrate to MSN Search itself. That could happen, given that Microsoft has often tested new ideas for search within the IE browser before transferring them to MSN Search site.
"You can think of Search Companion as a way for us to understand tasks better," said Krass. "We'll learn from that and take the best things to other places."
If you want to see examples of different tasks that have been programmed, open the Search Companion and click on the sample question below the search box. After following a particular question, you can use the "Start a new search" option to get back to the Search Companion's opening text, where a new sample question will be shown.
You might assume that all the task resources listed in Search Companion simply lead to Microsoft's own content or that of its partners. However, this is not the case at all.
"It wasn't a constraint of who's paying us but who has good answers," Krass said, about the task resources that are listed.
For example, in a search for "buying books online," the top option is to "Find an online book." This leads you to find answers not from Amazon, as you might cynically expect, but instead from a book site from the University of Pennsylvania.
Similarly, a search for "roseola" brings back a task to "Find information about the disease roseola." In turn, you get answers from MedLine, as well as several other choices, including the ability to check Google's Diseases & Conditions directory (though this didn't work correctly, when I tried it).
Other Search Engines & Jump Highlighting
At the very bottom of the Search Companion, an option that you should always see is to "Automatically send your search to other search engines." Selecting this brings up choices such as AltaVista and Google, where clicking on their links bring back results from those services.
This didn't always work properly, however. For example, a search for "camping sites in yosemite" got translated into "camping yosemite" in MSN Search, even if it was surrounded with quotation marks. This also meant that asking for the search to be forwarded to other search engines meant they got the wrong terms.
The other option you'll always see at the bottom of the pane is "Highlight words on the results page." This is a really nice feature. Even though it says "results page," it will still work on any page you've selected from the results. When using highlighting, you'll see the individual words you searched for shown as links in the companion window. Each time you click on a word link, you'll jump to the next occurrence of that word on the page.
Here's an example that explains things better. Let's say you search for "converting dollars to euros" from within the companion. In the results window, you get answers from MSN Search. You click on one of those results, and the page you selected now fills the results window. Going back to the search pane, you select the "Highlight words" option. The pane will change, and you'll see options like this:
dollars (1 of 6)
converting (1 of 3)
euros (1 of 19)
If you click on dollars, you'll jump to the first place where the word "dollars" appears in the page you brought up in the results window. Click again, and you'll jump to the second occurrence, and so on. You can also use the "Highlight other text" to enter additional words or phrases that you'd like to jump to.
This is a great way to jump within large documents, but sadly, getting to it via the companion is a difficult process. It would be nice if Microsoft made it a standalone button.
If you can't wait for that to happen, then try the Google Toolbar. Any terms you place in the toolbar's search box can be highlighted on a page, and the words themselves become buttons. Push a word button, and you'll jump to that word in the document.
Changing The Search Companion
So far, everything I've discussed is the default behavior for IE6 on Windows XP. What if you'd prefer something different? Well, let's explore the options, because if you roll-up your sleeves, you might be happier with IE's performance.
First let's revisit the Search Companion. When you first open it, there's a choice to "Change Preferences." Following this gives you an option to have an animated character appear within the pane, and you can choose from a selection of characters. They don't really do much, but if you like them, you can have them.
There are also a couple of options that deal with searching your own computer's files. We'll bypass those and go straight to the "Change Internet search behavior" option. From here, the first choice is to remove the "Search Companion." If you do this in favor of "Classic Internet Search," then when you close and reopen your browser, the Search Companion will act differently.
In fact, what will happen is that the Search Companion will be replaced with the Search Pane, the same thing that those using IE5 or IE6 on a non-Windows XP system will see.
When using "Classic Internet Search" or the Search Pane, the top option you'll see is to "Find a Web page." Using this will bring up results from MSN Search right within the left-hand Search Pane window itself. You can then select a link and see it show up in the main right-hand window. This is a nice way to always have your results list present, as you explore some of the answers given on it.
The other option from "Change Internet search behavior" is to choose your default search engine. Don't like MSN Search? No worries -- you can make Google or several other choices your default, if you so desire. However, this will only change the default option from when searching within the Search Companion, not for when you use the address bar.
Changing your address bar search engine can be done, but it's complicated in IE6 on Windows XP. Here's what to do:
- Open the Search Companion
- Change to Classic Internet Search
- Close and reopen the browser
- Open the Search Companion/Search Pane again
- Select the Customize option
- On the new window that appears, select Autosearch settings
- On the new window that appears, select the search provider you want and how you want IE to act when you search in the address bar
- Back on the main Customize Search Settings window, if you want to go back to the Search Companion functionality, choose the "Use Search Companion" option. Otherwise, you'll stay with Classic Internet Search/The Search Pane
FYI, Microsoft admits that this procedure is a pain but said it might be made easier, in the future.
Changing Other Options
There are more behavior choices you can set, if you go to Tools from the IE menu bar, then Internet Options, then the Advanced tab. Here, you can disable the "Show friendly HTTP error messages," if you don't want IE to provide extra help if you are trying to reach a site that doesn't exist or a page that doesn't exist and which has no custom 404 page.
The "Search from the Address Bar" section has four different choices. By default, it will be the last option that happens when you search, where if you enter a search term, IE will take you to the most likely site, if it has a RealNames match.
Change this to "Just display the results in the main window," and you'll disable the RealNames lookup. Instead, your search terms will be sent to MSN Search, which will send back results.
The "Display results, and go to the most likely site" choice means that a RealNames lookup will happen, and you'd be taken to any matching site. However, the search pane would also open up, to show matches for the term from MSN Search.
Finally, the "Do not search" option will completely disable IE from trying to check RealNames or from performing a MSN Search, from the address bar.
Recap For Webmasters
As a site owner, what does IE's search and navigational capabilities mean for you? Let's recap some things that may be helpful.
+ Pay attention to MSN Search. Huge numbers of people either directly go to the site or are routed there in various ways by IE. This means that getting listings with LookSmart and Inktomi, which power MSN, can pay off in traffic. See the pages for these services listed below, for more advice.
+ Make a custom 404 page. They'll be displayed by IE and help users better locate content at your web site. Resources about doing this are below.
+ Consider purchasing RealNames, if you are a large brand holder. It may help ensure people are sent directly to you. I'm also planning an updated review of the RealNames system for a coming newsletter. However, some key tips are listed further below.
+ Try some searches using the Search Companion for terms related to your site and try some of tasks that come up. Then follow these to see what information resources get listed. These are places you may wish to be in.
Internet Explorer Home Page
Information from Microsoft about IE.
How Search Engines Work
You'll find pages explaining how MSN Search, LookSmart and Inktomi work here.
Microsoft gives error pages new direction
News.com, Sept. 5, 2001
Describes the change to IE's "web site not found" messages. These are not the same as a "page not found" or "404" message. IE's behavior in 404 situations did not change. Indeed, there was no "standard" error page for IE to be intercepting, when an entire web site is not found. Since the web site does not exist, it's impossible for it to communicate with the browser.
As I said in the article, I expect to fully revisit the RealNames system in a coming newsletter. I ran short of time to complete my write-up for this issue. In the meantime, here are some key details to keep in mind, at least for how RealNames operates in the United States.
RealNames has dropped all the confusing pricing options and packages that it has had in the past. Instead, you now have two choices, to purchase "Basic Keywords" or "Keywords Plus."
Basic Keywords can be any term -- even generic terms -- that are deemed to be below a certain traffic threshold. For example, you could register the term "replacement notebook batteries," if you wanted. RealNames allows this despite it being a generic term, because the company doesn't believe many people will enter that term into the address bar of the IE browser.
Basic Keywords cost $49 per year, for the US. Anyone entering your registered term within IE will be taken straight to your web site, so this might be a good value for some key generic terms. However, if the terms prove popular enough, RealNames could at any time declare that they need to be upgraded to Keywords Plus status. Also, though Basic Keywords will bring people directly to your web site, RealNames will frame your site so that its own window appears above your pages.
Keywords Plus are for terms considered to be extremely popular, either generic or company-specific. You'll pay a premium for these -- $299 to review your choice, plus $199 per year to maintain it. However, there are several pluses. People can reach your site directly via IE by entering your terms, such as your company name, into the IE browser. The RealNames frame will also not be displayed. In addition, anyone searching on MSN Search will see your site listed among the very top choices.
Helping Lost Site Visitors: The Error 404 Handler
WebReference, Oct. 4, 2001
Examines making custom 404 pages for Apache and Microsoft Internet Information Server.
404 Research Lab
Here's a nice find from the WebMama newsletter. The site has instructions on making custom 404 pages for various web servers and examples of layouts.
Tips like the 404 page above and advice on search engines can be found from the WebMama Journal.
Explorer Web searches lost without Java
Chicago Tribune, Nov. 19, 2001
Discusses a problem with IE crashing when doing searches at places such as Yahoo and suggests getting a Java plug-in as a solution. Doesn't sound right to me at all -- a search box at a major search engine shouldn't be dependent on Java. Nevertheless, if you've had this problem, here's a solution to try.
Speed Searching with Lycos Fast Forward
SearchDay, Feb. 14, 2002
Lycos' new Side Search feature adds a new link to search results that lets you easily preview pages without having to click back and forth to the result page.
You might discover that someone -- and not Microsoft -- has changed Internet Explorer to operate the way they want it to work, rather than how you want it. This page has tips on getting back to normal.
Non MSN Search
Scumware Forums, March 11, 2002
More advice for dealing with situations where default choices in IE have been changed against your will.
The Original Search Marketing Event is Back!
SES Denver (Oct 16) offers an intense day of learning all the critical aspects of search engine optimization (SEO) and paid search advertising (PPC). The mission of SES remains the same as it did from the start - to help you master being found on search engines. Register today!