Are You Suffering From Big Data Chaos?

Reading Time: 2 minutes

Companies that use data to gain insight have been an area of major interest for venture capital firms. Whether it’s a social media company that uses peoples’ profiles to generate marketing insights, or a transportation company looking for better routes to reduce fuel consumption, business data has proven itself as a valuable asset. However, for as many problems as this data can solve, it’s also creating new ones. With so many sources of data, many of these same companies find they are slipping in their ability to use it.  

Deliver Better Insight

In response to this trend, several companies have started to offer solutions to the big data problem. Check out the following slide from Luma (updated March 2013) to see just how crowded this space has become. In the marketing segment alone, it’s all too clear that there is a level of data chaos like we have never seen before. Everyone claims their solution will work with your existing process, handle your big data, and deliver better insight, but promises often fall short. Some companies are spending time, money, and patience only to discover more problems.

luma

The real issue is that we don’t have just one system—we have multiple systems. Big data issues are not about the volume of data, but rather the number of data sources. As a result, we are forced to visit numerous sources to glean any insight into the performance of the business. This provides a very limited perspective of data. Furthermore, it’s a serious drain on time and resources. And that’s just marketing systems.

We need to think about how to integrate data sources with other lines of business to create an intelligent enterprise. We’re seeing a level of data chaos like never before, but not for the reasons you may think. In reality, the volume of data isn’t usually the culprit, instead, it’s issues like:

  • Numerous data and reporting solutions
  • Various data storage and retrieval locations
  • Technology moving faster than a company’s capital resources
  • Constant change in technology upgrades

Business intelligence and analytical software companies would lead us to believe that purchasing their self-serve BI tools will fix the issue, but it often ends up making it worse in the long run because analytics quickly become inaccurate, delayed, and untrusted.

From what I’ve seen, companies are beginning to take a step back. They’ve slowed down their purchasing decisions and have started to think about what they actually need from their data, making sure it aligns with their strategic objectives. It is time to go back to the basics and think about the kind of data we need to run our businesses. We need to develop smarter data warehouses that can assist in the collection of these numerous data sources. We need to rethink our delivery methods to mobile devices, and not just provide another dashboard squished down to fit the device.