Majestic SEO has announced a major upgrade in their update cycle, increasing updating frequency from daily to hourly. Now, their Fresh Index stats show near 200 billion unique URLs.
Dixon Jones explains, “...the improvements in index size are more than skin deep. Now we have better insight into which URLs are important, we have been able to improve the crawler controller.”
This means improved URL discovery and Flow Metrics data, he wrote. Jones expects the crawler controller improvement will result in an increase of 20-25 percent in the referring domains count.
Of the Fresh Links announcement, Jones wrote, “We not only want to have the largest Link Map on the planet… we also want the fastest and the freshest. We always felt we had, but in terms of absolutely getting the links to you more quickly than anyone else, there was just one milestone we felt the need to overcome.”
The historic index will still update monthly, while the Fresh Index will continue updating approximately hourly, where new data is available. The upgrade has begun in Site Explorer and will become available in other services as they can integrate it.
Majestic SEO first launched their Fresh Index last March to much industry fanfare.