Articles on Robots Exclusion Protocol

RSS

Copyrights, Trademarks and Search Engines

Web site owners must be on constant guard with copyrighted and trademarked materials, both to make sure competitors don't 'borrow' content and to avoid provoking search engines in legal gray areas. Web site owners must be on constant guard with copyrighted and trademarked materials, both to make sure competitors don't 'borrow' content and to avoid provoking search engines in legal gray areas. 0 Comments

Avoiding Search Engine Woes with Multiple Domains and Websites

Operating multiple sites using numerous domains is a great strategy for businesses appealing to a wide range of customers. But this approach may trigger searchengine spam penalties, if you're not careful with implementation. Operating multiple sites using numerous domains is a great strategy for businesses appealing to a wide range of customers. But this approach may trigger searchengine spam penalties, if you're not careful with implementation. 0 Comments

Avoiding Search Engine Woes with Multiple Domains and Websites

Operating multiple sites using numerous domains is a great strategy for businesses appealing to a wide range of customers. But this approach may trigger search engine spam penalties, if you're not careful with implementation. Operating multiple sites using numerous domains is a great strategy for businesses appealing to a wide range of customers. But this approach may trigger search engine spam penalties, if you're not careful with implementation. 0 Comments

Search Engines and Web Server Issues

What is the best way to move a site from one server to another without affecting search engine visibility? After a site redesign, how can you communicate to the search engines that old URLs should be redirected to new URLs without affecting positioning? A panel of experts addressed these and other server-specific issues. What is the best way to move a site from one server to another without affecting search engine visibility? After a site redesign, how can you communicate to the search engines that old URLs should be redirected to new URLs without affecting positioning? A panel of experts addressed these and other server-specific issues. 0 Comments

A Conversation With Gigablast's Matt Wells

Most major search engines rely on an army of hundreds of people to create and maintain their services. Not Gigablast -- it's a high-quality search engine built and operated by sole proprietor Matt Wells. Most major search engines rely on an army of hundreds of people to create and maintain their services. Not Gigablast -- it's a high-quality search engine built and operated by sole proprietor Matt Wells. 0 Comments

Blocking Crawlers With Robots.txt

Explains how the robots.txt standard lets you tell search engines which web pages NOT to index via a text file placed in your server's root HTML directory. Explains how the robots.txt standard lets you tell search engines which web pages NOT to index via a text file placed in your server's root HTML directory. 0 Comments

Happy Birthday, Aliweb!

Aliweb, a pioneering web search engine that used advanced technologies way ahead of its time, made its debut nine years ago, on November 30, 1993. Aliweb, a pioneering web search engine that used advanced technologies way ahead of its time, made its debut nine years ago, on November 30, 1993. 0 Comments

The Big List of Web Robots

Who sent that web robot, and what is it doing crawling around on your server? Identify and track robots with this list of hundreds of active crawlers, link checkers and other autonomous cybercritters. Who sent that web robot, and what is it doing crawling around on your server? Identify and track robots with this list of hundreds of active crawlers, link checkers and other autonomous cybercritters. 0 Comments

Leveraging Your Content

Though they're 'cool,' advanced design techniques such as frames, Flash and dynamically generated Web sites pose challenges for search engines. Though they're 'cool,' advanced design techniques such as frames, Flash and dynamically generated Web sites pose challenges for search engines. 0 Comments