SEO News

Spider

RSS
  1. Website Launch Checklist: 25 Things to Test Before Your Site Goes Live

    On a site with fewer than 500 URLs, you can use Screaming Frog SEO Spider Tool for free to find bad URLs. W3C-valid code is the one thing you can do prior to launch to have some confidence around a search engine spider being able to crawl your site.

  2. How to Quickly Identify Duplicate Content With a Site Crawl

    There are many tools out there, but Screaming Frog is definitely one of the most popular/powerful scrapers and it is the spider of choice for this tutorial. This article is a simple breakdown of how to go about using an SEO site crawler to quickly...

  3. Facebook Ads 80% Bot Claim, Examined! Why You Shouldn’t Lose Faith Just Yet

    By comparing the two numbers and filtering out any user-agent that declares itself a bot (spider or crawler) we were able to determine the total percentage of visits without JavaScript enabled. Looking at aimClear’s total organic traffic from every...