The science (or should that be the art?) of data analytics has, traditionally, been divided up into a sequence of very distinct and separate processes: the collection of data, whether from a single source or, more commonly these days, a combination of multiple sources—followed by its analysis, typically offline, and some time after, the assembly of the data itself.
That linear model, however, is about as relevant to modern IT and how we use it as batch processing, procedural programming languages and the Cathode Ray Tube (CRT).
Ok, the analogy itself may not bear particularly close scrutiny as, unlike the CRT for example, the traditional approach to data analytics does still have its uses. But it is starting to show its age and, paradoxically, doing so just as analytics and its practitioners move center stage as the volume of data from embedded systems, social media, and wearable technology skyrockets, and vast pools of structured and unstructured data become available both inside and outside the enterprise.
As discussed in some of my previous posts, those types of data have an implicit “use by” date, so need to be analyzed, processed, and used in a very short window of time.
Developments such as these in the wider IT world are rapidly overtaking the ability of conventional data analytics to deliver the goods, with changes needed to make the technology both more relevant and responsive to the way we live now.
If nothing else, we need to stop thinking in terms of sequential processes—collection followed by analysis, then action. Not least because we no longer need to explicitly seek out data—it’s everywhere. We’re virtually drowning in the stuff.
Moreover, we can no longer wait for that data to be assembled and analyzed later. We’ve got the data already. What we need is the analysis and, of course, we want it now. And that requires for analytics to be moved much closer to the data sources involved—to become deeply and invisibly embedded just about everywhere. For every app, if you like, to become an analytic app.
This doesn’t do away with the need for the kind of Big Data analytics we’ve become used to in the past 10-15 years. Big Data is still relevant and an important enabler of moves towards putting analytics intelligence out closer to where the data is made. No matter how relevant, however, the focus needs to shift away from processing large data sets to thinking about how best to filter the huge amounts of data coming from social media networks, the Internet of Things and the like, and at how to use those data streams to deliver exactly the right information to the right person, at the right time.
Bottom line?
We need to ask big questions rather than worry about Big Data or its collection, and ask those big questions in real time where the data is collected instead of in another place at another time. More than that, we need to understand that the value to be had is, not in the data itself, but in the answers to those big questions and the ease and speed with which the insights of data analytics can be gained and made available to those best able to act on them.
