SEO News

Web Robots

RSS
  1. Google Dorking: Feds Warn Against Malicious Cyber Actors

    Use robots.txt to prevent search engines from indexing your site. Frequently test your website using a Web vulnerability scanner. The Department of Homeland Security, the FBI, and the National Counterterrorism Center have issued a warning against...

  2. Panda Has a Smartphone – Here Are 7 Things You Can Do to Test It Now

    As Google notes, you should not block JavaScript and CSS via robots.txt. The default report will be filtered by Web (for Web search). There are a boatload of user agents you can download from the Web that you can import into User Agent Switcher.

  3. SEO Strategies for JavaScript-Heavy Single Page Applications or AJAX Sites

    And while Google has been continuously evolving the JavaScript capabilities of its robots, Bing/Yahoo has lagged. which translates hashtag URLs into a query string for indexing by robots. Client-side JavaScript is great for making innovative and...

  4. Google Webmaster Tools Now Finds Smartphone Crawl Errors

    Blocked URLs: A blocked URL is when the site's robots.txt explicitly disallows crawling by Googlebot for smartphones. Typically, such smartphone-specific robots.txt disallow directives are erroneous. They’ve made some changes to their crawl errors...