Is big data really a new concept?
After all, industry analysts have been noting the torrid growth of data for a number of years. For example, in the past, research firm IDC has said that enterprise data storage would grow tenfold from 2005-2011, with transactional data growing at a 32% compounded annual growth rate and unstructured data growing 63% annually.
So what’s really new?
Forbes contributor David Williams notes that if the term “big data” just means “more data,” it might be called “lots of data.”
What’s really different in the era of big data is that substantial changes in technology mean that companies can use data analytics to more effectively process the vast volumes of data they are producing. And they can use that analysis to inform better business decisions.
“Consider Google, whose ‘data’ is effectively the internet – they basically download and index the internet as a business,” he adds. “This data change forced new technology advancements and caused a paradigm shift in data management.”
Williams’ take on what’s different about big data compared to lots of data is as follows:
- Instead of relying on a sample of data to predict outcomes, companies can now use data analytics to parse through all the data.
- In the pre-big data era, companies used predictive models based on batches of data that were not analyzed in real-time. Now, analytics allows the real-time analysis of data.
- Big data technologies allow the analysis of unstructured data – data like emails, photos, comments on social networks, videos, sensor data – as opposed to the previous limitations of only being able to analyze structured data that live in a database.
Using analytics to mine all the data as opposed to a sample allows the type of “mass personalization” to the individual consumer level that Amazon has perfected to recommend products based on past purchasing behavior, Williams adds.
Grocery giant Kroger has been using big data analytics to personalize offers to consumers for the past two years, and its sales and profits have grown 77% during that time. Kroger CEO calls this method his “secret weapon” in the battle to fend off competitors. The company also is generating millions by selling that data to manufacturers like PepsiCo and Procter & Gamble.
But to garner the type of competitive advantage Kroger has amassed, C-level executives must accept that big data is more than just bigger volumes of data, according to Harvard Business Review.
In the article, HBR contributor Michael Schrage, a research fellow at MIT Sloan School’s Center for Digital Business, notes that he asked 100 senior executives at a recent conference how much more profitable their businesses would be if they had free access to 100 times more data about their customers.
“But not a single executive in this IT-savvy crowd would hazard a guess,” Schrage notes. “One of the CEOs actually declared that the surge of new data might even lead to losses because his firm’s management and business processes couldn’t cost-effectively manage it.”
Schrage notes that many executives shouldn’t be committing themselves to “bigger data,” but instead to a desired business outcome. He recommends that firms understand that big data initiatives should focus on enhancing or transforming user experience rather than automating managerial decisions.
“Virtually every organization that’s moving some of its data, operations or processes into the cloud can start asking itself if the time is ripe to revisit their value-creation fundamentals,” he adds. “In a new era of Watson, Windows and Web 2.0 technologies, any organization that treats access to 100X more customer data as more a burden than a breakthrough has something wrong with it. Big data should be an embarrassment of riches, not an embarrassment.”
