SEO News

Site Crawler Grub

RSS
  1. Search Wikia Asks Users to Share the Load

    Wikia will immediately release Grub to the open source community, and make both the crawler and source code available at Grub.org. Grub's modular design allows developers to extend and add functionality to the crawler, improving the quality and...

  2. How LookSmart Works

    Finally, LookSmart also owns a company called Grub, which has technology used to crawl the web. More about Grub can be found in the Help LookSmart Crawl the Web article from Search Engine Watch. There is no way to submit directly to WiseNut crawler...