A paper presented at the 10th International Conference on Extending Database Technology conference in Munich near the end of March, Indexing Shared Content in Information Retrieval Systems (pdf), jointly authored by employees of Yahoo, Google, and IBM, discusses how to limit index sizes of search engines by reducing the amount of duplicate content contained in their indexes.
After reading it, I started considering and listing some of the problems that sites may have that could cause search engines to not index the pages of those sites, or display them in search results. My list is in a post at SEO by the Sea, titled Duplicate Content Issues and Search Engines.
Optimising Digital Marketing Campaigns with Search, Social and Analytics
At SES London (9-11 Feb) you'll get an overview of the latest tools, tips, and tactics in Paid, Owned, Earned, Integrated Media and Business Intelligence to streamline your marketing campaigns in 2015. Register by 31 October to take advantage of Early Bird Rates.