SEO News

Robots Txt Files

RSS
  1. 5 Googley SEO Terms – Do They Mean What You Think They Mean?

    Robots.txt won't prevent the URL from being indexed, just the page content. A robots.txt is used to stop a page or part of a site from being crawled and indexed, but not the URL(s) itself. To prevent a page or set of pages from having their URL...

  2. SEO & Website Redesign: Relaunching Without Losing Sleep

    Is there a folder in the robots.txt file that is inaccurately excluding pages that should be visible? Have meta robots tags been placed on pages that shouldn’t have been tags? A site redesign gives you the opportunity to re-code, condense...

  3. Proper SEO and the Robots.txt File

    Don't list all your files in the robots.txt file. What is a Robots.txt File? Another meta element, if implemented incorrectly, that can also trip up websites is robots.txt. I also remember reviewing a company's website and noticing that several...