Forrester has just announced a Forrester Wave on a new category of technology they are referring to as “big data streaming analytics.” We are proud to announce that we have been named as a leader in that report. The Forrester report is important because it details a fundamental shift in computing physics and the next generation of software that we call Fast Data.
Fast Data is a style of software architecture designed to derive instant insight from new and increasingly valuable sources of business information, like streaming sensor data from consumers and the Internet of Things. It also looks at structured and unstructured streaming social media data from sources such as Twitter, or user behavior data transmitted by mobile devices. In the end, it’s all integrated with streaming data from inside the traditional enterprise, including inventory, staffing, and operations.
Maximizing In-Store Data
Big data streaming analytics is essential for disruptive applications. Examples of such applications include:
-In-store experience management aided by proximity detection devices such as iBeacons
-Mobile payment fraud detection
-Predictive failure detection for undersea oil production or industrial equipment
-Social, sentiment-driven real-time marketing programs
In short, any business application that makes decisions in real time on large amounts of system-generated data will have big data streaming analytics at its core.
Why a new software category? Unlike Hadoop, data warehouses, or data lakes that focus on storing, managing, and batch analyzing large volumes of data at rest, Fast Data flips traditional computing on its head with computational elements designed for moving data, such as:
-Stream filtering
-Stream aggregation
-Stream correlation
-Event-driven rules engines
-Time windowing
-Temporal pattern matching
-Geo-location
-Continuous query processing
-Push-based alerting
-Controls to manage automated action
Add More Power to Your Tools
Streaming analytics allows IT departments to process terabytes of data in-memory while the data is still in motion, and follow through with business notifications and automated action. Although a series of projects in the space—like Hadoop Flume, Twitter Storm, and most recently Apache Spark—have started to add stream-processing capabilities, they should be used in conjunction with streaming analytics. For example, TIBCO StreamBase and the TIBCO StreamBase LiveView Data Mart already have a publicly available adapter for Flume and Apache Spark that works as a scalable stream collection stage. Combining these tools with streaming analytics makes the data more accessible (via the live data mart). It also enables more operational awareness and control through dashboards, operator consoles, and interactive control of event-driven business rules. Lastly, it powers automated reaction through transaction processing, messaging, and application integration.
In recent customer meetings I found it very useful to draw the Fast Data architecture on a whiteboard. As you can see in the video, big data streaming analytics is at the core of Fast Data, but a complete Fast Data solution requires a holistic architecture, including:
1. Stream data collection from any source (machine data including log data, application data, transaction data, and internet feeds)
2. Real-time streaming analytics of temporal data
3. Analysis of historical data
4. Visualization technology for consuming large volumes of changing data
5. Bus-based architectures for systems integration, along with fast and scalable messaging technology
6. Open and extensible connector technology to incorporate new streams of data
7. Elastic and scalable computing infrastructure to meet storage and processing needs of the future
It’s great to be in the leader position for big data streaming analytics. In our opinion, we are there because of several unique elements of our platform:
a) The industry’s first and only live data mart: Unlike competitors that only provide developer tools to IT, StreamBase LiveView brings streaming analytics to business end-users for operational business intelligence.
b) Radically faster time-to-market: With visual programming, event-based rules management, and the industry’s only “app store” component exchange for open-source streaming analytics examples and components, it’s easier to complete projects.
c) Most complete Fast Data stack: We are the only company in the industry with an entire organization dedicated to the vision of Fast Data. This enables us to integrate our messaging, in-memory data grid, log data management, and analytics products into one cohesive stack, which enables our customers to build solutions quickly, with a lower overall TCO.
d) Integrated streaming and non-streaming analytics for predictive analytics: TIBCO Spotfire is a leader in the non-streaming analytics world and we’ve been hard at work integrating StreamBase with Spotfire into a product we call “event analytics.” No other vendor has this vision or the ability to execute on this opportunity.
We are the only ones with a complete Fast Data stack using commercial and open-source software.
You’ll hear a lot more about big data streaming analytics in the coming years. It’s a central part of what we consider the future of software and analytics: Fast Data.
