You can’t truly have a successful search engine optimization (SEO) program without evaluating your performance over time and knowing how potential technical issues can affect your website from a crawling, indexing and user experience perspective.
Don't forget to review your robots.txt file periodically - many sites block crawling which hurts indexing and ranking. We discussed the overlap between social and search, and why social drives important signals that help determine rankings – with...
If not, broken links can lead to a poor user experience while also inhibiting the search engine bots from effectively crawling your site. I use my programming skills on a regular basis in SEO, and especially when performing SEO audits.
Big data startups are crawling all over the social space right now, and for good reason. Social is a hugely exciting space right now. For businesses it gives us, for the first time, the ability to connect with customers individually, and in an...
You must also keep a regular check on the core SEO tasks of monitoring the crawling and indexing. You share quality content with the groups, circles, or people you're connected with on various social media platforms.
I believe that Facebook would block this level of crawling. Imagine how much crawling of Facebook that Google would have to do to get the Facebook data in any reasonable type of timeframe. In "SEO Revelations for 2013", I wrote this about rel...
Screaming Frog is a good application for crawling competitive domains for to review overall website structure. According to data from the 2013 B2B Content Marketing Report, a coordinated effort between MarketingProfs and the Content Marketing...
Meanwhile, if Google’s already crawling your content with Googlebot-Mobile, it’s already aware of your configuration. See if Google is crawling your mobile content by checking log files for Googlebot-Mobile.
Google is a massive "connections engine," crawling the web discovering associations, affiliations, citations, mentions, links, and other connections that give topical context, trust, and authority to brands.
Like users, search engines crawling the website must understand it. While this article has only brushed the surface of how you should cater to a visitor or crawling bot experience, these are some of my biggest pet peeves that are often times easy...
Crawling is also a high priority for us and as many have seen over the past year, we’ve continually ramped up our crawl pace and depth. And while we're talking expectations, it’s smart to keep in mind that there is no guarantee for crawling...
Think of crawling a user’s social and open graph connections across hundreds of thousands or potentially millions of nodes for every search. The potential for Facebook’s new Graph Search feature is huge.
Important note: You should never, ever use automated translation – but if you must due to the nature of your industry or the size of your website, then make sure you use your robots.txt file to block search engines from crawling any auto...
Note, for bulk crawling, you should use a tool like Screaming Frog or Xenu, but for specific pages, the web developer plugin comes in very handy. Technical SEO audits are one of the most powerful deliverables in all of SEO.
This is helpful for diagnosing if you’re having a crawling issue in Bing beyond just using Bing Webmaster Tools. By that, I mean the art of doing SEO stripped down, without tons of tools. In a world that is saturated with SEO software solutions...
Crawling Basics on December 11 This is Bing’s first crack at a set of official guidelines around most major topics affecting webmasters looking to optimize their sites for Bing. The broad categories covered in their guidelines are:
Too often, waiting on the client or development team to implement changes to a site that will help reduce excess crawling and indexation can hinder making the bottom line. Duplication is negatively affecting the ability of your ranking pages to...
Through the redesign, we enacted several SEO elements, which have helped to allow and some instance entice crawling bots to frequent the site more often…and more pages at that. I passed this on to the client for which I got the quick reply of, “Why...
And given the pace that end users now generate content, and the limitations that the protocol of the world wide web places on crawling, it’s not likely that it ever will. “Picture the communications future: every home linked via cable TV, push...
In most crawling tools URLs are shown as they are requested. This article is a simple breakdown of how to go about using an SEO site crawler to quickly identify duplicate content. There are many tools out there, but Screaming Frog is definitely one...
