Promises and Pitfalls: The Difficulty of Big Data

Promises and Pitfalls: The Difficulty of Big Data
Reading Time: 2 minutes

Big Data comes with big hype. Companies wax poetic about the benefits of real-time insight—airlines can predict with near-perfect accuracy when planes will land and supermarkets can order exactly the amount of food needed to satisfy hungry shoppers, drastically reducing waste. These aren’t pipe dreams; there’s potential in Big Data to fulfill promises and deliver actionable insight.

But there are also pitfalls. As a recent Customer Experience Optimization report discovered, almost two-thirds of companies asked said they were “overwhelmed” by the sheer volume of incoming data. Can businesses sidestep the issues and tap the dream of Big Data?

Running on Empty

The study also found that 85 percent of organizations weren’t able to “extract the full value” from accessible data sources, in effect giving them just a taste of what’s possible, but leaving them frustrated by what’s left behind. As noted by an Econsultancy article, part of the problem is volume: the sheer number of data sources available and the speed at which information is generated makes it hard for any company to keep up. What’s more, data from multiple channels—email, social media, mobile devices, eCommerce sites—is often gathered and stored separately, creating a “silo” effect when companies try to uncover insights.

The true value of Big Data stems from its size, and more importantly the ability to connect seemingly random pieces of data from anywhere in the information ecosystem to produce actionable results—what TIBCO defines as Fast Data. For the moment, however, businesses are struggling to stay afloat under the deluge of data, let alone make headway.

Getting a Grip

According to First Post, part of the problem comes from expecting too much. Often, companies want Big Data to ensure insights are tested for value against business outcomes, close the loop between actions, reactions and learning, and automatically deliver point-of-decision insights. In fact, that’s beyond Big Data’s purview. That’s where Fast Data comes in.

For many companies, however, that’s where the data discussion stops: give IT the data stream, budget, and tools they need and the rest is smooth sailing, right? But organization and categorization alone aren’t enough to drive insight; they merely set the stage for success. It’s easy to see this in the market at large: With two-thirds of companies already running a Big Data strategy, the idea is no longer cutting-edge or new—yet mature solutions and data-savvy businesses are still struggling.

Getting a grip on Big Data, then, starts with the recognition that it’s not all promise: getting sucked into the pitfall of too much information and not enough insight is entirely possible. But there’s a way out. Event-driven, insight-led solutions that go a step beyond data to deliver contextually relevant results across information silos is the next step forward for analytics, and critical in tapping the raw potential of data volumes, velocity, and variety.

Jump the pit, fulfill the promise; it’s time to go beyond the data.