Leading capital markets analyst AITE Group recently published a research paper that examines the current state of play of streaming analytics in the capital markets. Ten years ago, automation was all about equities’ algorithmic trading on Wall Street. Now, algorithmic is everywhere: real-time risk management, real-time regulatory compliance, and intelligent client services. And it’s not just equities anymore; according to AITE Group, even the slowest-adoption asset class—fixed income—is over 50% automated.
Streaming analytics is the technology associated with applying algorithms and rules to data in motion, while “traditional” analytics deals with analyzing data after it’s stopped moving. Streaming analytics help IT express rules that are temporal, pattern-based, and help correlate multiple streams of data, and are evaluated with each market data tick, with each electronic execution, with each customer request for a P&L update.
Streaming analytics lets IT easily ask questions in seconds like, “Tell me which orders, among the tens of thousands executing right now, are in trouble at this very moment.” That question is placed directly into the stream of data flying around the firm, proactively finding needles in the haystack as they happen. Years ago, these questions would be answered once a day, based on yesterday’s data. Now, they’re answered in the moment, with continuous, predictive, query processing on fast-moving data.
Streaming analytics lies at the core of this transformation. But streaming analytics is just one part of an overall trend toward the use of Fast Data—data in motion. Fast Data technologies are more than just analytics, and include new memory-based streaming data warehousing, event-driven rules engines, high speed messaging, predictive analytics, event-driven BPM, social analytics, and real-time master data management.Check out TIBCO’s whiteboard explanation of this Fast Data reference architecture.
Here’s a summary of AITE’s findings, and the 11 implications of the capital markets’ architecture for Fast Data and streaming analytics:
- Algorithmic Trading: Algorithmic trading was the first app in capital markets to broadly apply Fast Data. The two main components of algorithmic trading are liquidity sourcing and liquidity routing, a natural process for algorithms that use greater intelligence to react to multiple data streams and take action to send orders to various market venues. And the innovation here is constant. Recently, use of TIBCO’s FTL messaging technology has grown dramatically as the need for speed on Wall Street never stops.
- Trading Strategy Development: This is the development of trading strategies that rely on the greater infusion of technical analysis and fundamental parameters. Strategy development requires looking for patterns in market data that exhibit certain properties and building subsequent patterns out of lower-level events. In recent years, this capability has been extended to capture and analyze unstructured data such as news feeds and sentiment analysis. The ability to rapidly prototype new strategies is one of the most natural fits for streaming analytics. TIBCO’s efforts to optimize the R engine for Fast Data have been lauded by industry analysts.
- Continuous Streaming Market-Making: In today’s highly electronic institutional trading FX environment, market-making has become extremely automated. Understanding the nature of client interaction with shown prices and liquidity has been yet another ideal place for Fast Data to shine. This provides visibility to determine leading liquidity providers, which in turn has immediate impact on rebates and intraday liquidity management analysis. Profitable market making also relies on being able to resist predatory forces, so it is a necessity to immediately sense, analyze, and respond. Connectivity is king here, which serves to grab as much real-time pricing information a firm can get to create a consolidated, streaming view of the market. TIBCO’s platform has over 150 connectivity points alone, including over 70 in the capital markets for the FX, equities, fixed income, and commodities markets.
- Real-Time Pricing: One key to success on Wall Street has been the ability to leverage streaming analytics to revamp pricing engines. Streaming analytics and the Live Datamart help firms easily configure and closely monitor their data, adjust access, and provide tiered pricing levels. The balance of power has shifted as technology has enabled almost every firm to intelligently process and participate in making prices.
- Fast Data-Based Transaction Cost Analysis: Slippage is expensive. The development of real-time trade analytics is another natural fit for Fast Data platforms. Firms are leveraging streaming analytics and high-speed messaging to process massive amounts of data from various execution venues to construct real-time execution schedules, thereby improving overall execution performance on the fly. Firms are also able to create feedback loops as execution occurs and combine static data types with real-time information to add intelligence when incorporating routing decisions into transaction cost analysis.
- Continuous, Real-Time Execution Consulting: With sell-side electronic execution tools becoming more commoditized, hands-on execution consulting services have become one of the few competitive differentiators for global broker-dealers. Capturing streaming execution data and measuring overall performance has become even more important in recent years as return on investment remains stagnant on the buy-side. From a technology perspective, the Live Datamart for streaming data provides business intelligence and built-in visualization for trading operations. Master Data Management can no longer be an off-line activity; trading firms must “spell check” their customer data in real time to gain a single live view of their customers, and automate based on that live view during the day.
- Live Routing Metrics: The decision of where to route various types of order flow is not trivial, particularly in the fragmented electronic market. From an alpha and best execution point of view, the performance of each routing choice is crucial. If a trading desk knows that a specific exchange or venue is slower or faster than a competitor, and they know that at the temporal detail of milliseconds, it presents a huge trading opportunity. Live routing systems help firms gain that advantage, particularly as volatility increases and performance metrics at each venue diverge.
- Streaming Order Book Aggregation: The flexibility to create various means of looking at liquidity across multiple exchanges and alternative trading systems is a powerful tool across asset classes. Any market that sees fragmentation of liquidity across various liquidity pools and has the ability to adjust across instruments and venues will be increasingly important. Built-in market data connectivity and native real-time aggregation is the key technology enabler of streaming order book aggregation.
- Continuous Risk Management: It is no longer an option for firms to monitor and act on trading risk in real time. By combining low-level risk analytics across a division as well as an enterprise, real-time risk management requires the processing of many events in each level of a firm. Streaming analytics enable firms to create an environment where increasingly detailed real-time events can be used to minimize and understand risk. The ability to assess risk with greater granularity is more important as the traditional asset class correlations no longer apply.
- Continuous Compliance, Surveillance and Case Management: With regulatory scrutiny deepening over trading behaviors, the need for continuous compliance, real-time trade surveillance, and case management has grown over the last few years. Compliance must consider new Fast Data sources including instant messaging, email, and social media feeds. Connectivity, the Live Datamart, streaming analytics, real-time R for predictive analytics, business process management (BPM)-based Case Management, and high-speed messaging, all have played an important role in the area, aiding the interaction of different data types to identify and correct suspicious trading activities and compliance issues as they happen.
- Streaming Best Execution: The ability to prove best execution has been a core tenet of various regulatory changes in the pipeline, and it is not solely focused on price. Streaming analytics helps firms correlate pricing data with dozens of other sources of real-time data simultaneously to meet and document their best execution obligations. In memory data grid technology helps a Fast Data platform correlate streaming data with historical data as well to get a holistic real-time picture of execution effectiveness.
Fast Data technologies are all over Wall Street, and innovation continues to drive new opportunities and new disruptions in the market. The leading firms are not only adopting Fast Data for these well-known trading and risk use cases, they are also developing a familiarity with the technology to enable them to use it in novel ways that will keep their business at the forefront of customer experience, marketing, employee productivity, and profitability. Where is your firm applying Fast Data technology?
