Subscribers to a news alert about “big data” will receive no less than 10 new articles per day. Search marketers may scratch their heads and wonder what’s new as search is the granddaddy of big data. And yet there are innovations in big data that materially help search marketers achieve more impact.
Big Data is a New Category, Distinct From Other Technologies
Big data describes the continuous processing of very large, public, and changing data sets using distributed data processing and data storage systems. There is no question that mainframes and super-computers process very large and changing data sets – but they process centrally and use proprietary data sets.
Google, Facebook, Twitter and Yahoo all innovated at leveraging distributed, lower cost computing and storage to provide consumers with data rich services. Big data pertains to the use of these same types of technologies for enterprise data rich services. That’s not to say that big iron is obsolete – it means that many more enterprises and companies can leverage data rich services than before. Big data can be the big equalizer.
Consider the evolution of paid search marketing technologies. The largest paid search marketers have agencies and developers create proprietary bidding management and performance analysis applications in order to maximize their return on investment.
In the last five years, Efficient Frontier, Kenshoo, and Marin Software have brought automated bid management and optimization into the reach of many more search marketers – and their algorithms are trained on big data – interpreting the trends and feedback from search performance into optimized bidding for their subscribers. It’s a long way from manually entering data into a spreadsheet.
Cloud does not equal big data although big data technologies are often only available through cloud delivery. There are plenty of cloud technologies that are not big data. There are plenty of SaaS applications that are not big data because of their single tenancy – there is, by design, no sharing of data across subscriptions.
Big Data Includes Infrastructure, Analytics & Applications
What are the big data solutions? To simplify segmentation, there are three broad segments – infrastructure, analytics, and applications.
The foundation of big data is the technologies that support distributed data processing and storage. These technologies include Hadoop and Cassandra. If you are an application developer creating new applications that require massive amounts of changing data to be processed efficiently, these and similar technologies are likely part of your application architecture. But for search marketers and other end users, big data infrastructure is the plumbing that makes the magic.
Much of the hoopla around big data is focus on big data analytics. These solutions interpret public data to answer questions for business people – providing reports that were previously manual, painful to generate and often not profitable to maintain.
For example, there is now an array of search marketing analytics providers ranging from SEOmoz to Rio SEO to BrightEdge and Conductor. Instead of proprietary reporting systems or manually collected data, these products crunch big data to show their clients how they are performing in search and sometimes social media.
The distinction between these and web analytics vendors such as Google Analytics and Adobe SiteCatalyst is that the big data analytics are focused on the activity off your website while these provide insight about performance on your website.
The big win in big data are big data applications. They are distinct from big data analytics because they actively adapt websites based on big data insights. This nascent category has the applications that automatically act on the insights from specific big data analytics. These are not general-purpose applications – they ask specific questions of the data and act on those specific questions.
In our case, the question is why isn’t a given web page capturing more demand for its content. This requires that we interpret the page, the demand it is capturing and web-wide demand for that sort of content.
It is the same question that SEO professionals ask of the pages they improve – with the exception that it is focused on the long tail where it is never efficient to have people, with their extraordinary judgment and creativity, spend their time.
With larger sites, there is no way to answer that question at scale for every page, every day without a big data application. And the big data application must be continuously learning from public data and customer experience to improve its actions.
Big Data Applications Outperform their SaaS Predecessors
Big data applications combine scale with insight – insight derived from very broad data sources that are inefficient to collect and integrate on an individual basis. Insights powered by algorithms built by specialized machine learning engineers. Often they are priced on performance – not seats or pages – but rather on the results from the actions they take.
The ROI calculation is simpler – if you're getting more of the result you wanted, you pay for that lift. Traditional SaaS still doesn’t connect the cost to the benefit. And innovative SaaS leaders like Salesforce.com are adding big data applications, such as Data.com to their product suite as they recognize it’s potential.
Search Marketing Professionals Embracing Big Data Applications Outperform
Big data applications don’t write and they don’t merchandise. They don’t “spray and pray” by randomly generating actions.
It is critical that analysts, companies and industry watchers understand the underlying work driving an application to determine if it is a big data application and, more importantly, one that they can trust to make decisions in the best interest of their end users and their websites.
Big data applications systematically address perplexing questions at massive scale – continuously learning, adapting and improving. Their early adopters win big and propel their employers and themselves ahead of their competitors.