SEO News

Web Spider

RSS
  1. 5 Googley SEO Terms – Do They Mean What You Think They Mean?

    If you have the specific page in the robots.txt, the spider won't read the page and this means your noindex tag remains unseen by Google and the URL is very likely to get indexed. As long as Googlebot fetches the page, it will see the noindex meta...

  2. Website Launch Checklist: 25 Things to Test Before Your Site Goes Live

    On a site with fewer than 500 URLs, you can use Screaming Frog SEO Spider Tool for free to find bad URLs. W3C-valid code is the one thing you can do prior to launch to have some confidence around a search engine spider being able to crawl your site.

  3. How To Use HTML Meta Tags

    Told the spider when to come back meta name="revisit-after" content="30 days"> The text in these tags is not displayed, but parsable and tells the browsers (or other web services) specific information about the page.