One of the challenges for organizations investigating event processing technologies is “how do we know what is important”? Obviously everyone will have different views on this. Nonetheless, as TIBCO has the most experience in CEP across multiple domains and the largest customer base for a CEP technology tool, it seems to make sense to present a sample evaluation guide that shows the typical considerations and features needed.
First, a few comments. TIBCO deals with enterprises who can have sophisticated requirements around their complex event processing needs. TIBCO customers generally use CEP for operational intelligence applications covering anything from business activity and event monitoring to automated intelligent business processes. Hence TIBCO solutions handle high throughput (not just latency) and payload size (business messages can get very large), whilst providing the fault tolerance and resilience capabilities that 24×7 organizations require. But every class of CEP application has different requirements: for example an algorithmic trading solution might prioritise ultra-low-latency over other features like failover-support and logic and payload complexity. Ergo, for every use case the “example scoring factor” provided below will need to be adjusted to match the feature’s importance to the use case.
| Feature | Detail | Reason | Example Scoring Factor |
|---|---|---|---|
| 1. Event Channels | A. Specific channel types required | Example: JMS, MQ-Series, HTTP… | 10 |
| B. Custom channels | Example: custom event type for SCADA | 3 | |
| 2. Modelling | A. Event models | Event metadata, hierarchy, inheritance | 10 |
| B. Event object / concept models | Object history, hierarchy, inheritance | 10 | |
| C. State and status of complex events | State model and flow | 8 | |
| 3. Processing Elements and Expressions | A. EventConditionAction rules | Basic event filters, aggregations | 10 |
| B. Inference rules | Intelligent event processing | 6 | |
| C. Continuous queries | Event streaming aggregations, statistics | 6 | |
| D. Temporal regular expressions / logic | Concise pattern recognition | 6 | |
| E. Specialist analytics and statistical functions | Capabilities for pattern discovery, possibly in conjunction with developer interface (see 7) | 6 | |
| F. Other context support | How to handle context across time, such as GVs etc | 6 | |
| G. Extensibility | How to add custom functions / algorithms / analytics? | 6 | |
| 4. Event augmentation | A. Database access | JDBC read/update etc | 4 |
| B. SOA access | Invocation of existing services | 6 | |
| C. BPM / workflow access | Invocation of manual steps | 3 | |
| 5. Stateful processing and management | A. Event storage in-process | In-memory support | 8 |
| B. Event storage out-of-process | Low latency cache and/or high latency backing store | 8 | |
| 6. Platform capabilities | A. Scalability via distributed processing agents | For large processing loads (logic and/or event size and/or rate) | 8 |
| B. Scalability via distributed event cache | For supporting event patterns over processes, resilience | 8 | |
| C. Failover / resilience features | Hot standby and event handling if agent fails | 8 | |
| D. Hot deploy / event-based updates | Hot deploy of new rules, queries or patterns, or dynamic update through events | 8 | |
| E. Operations and Management | UI and/or reports to monitor performance and/or adapt capabilities | 8 | |
| F. Performance supporting features | Components to minimise latency like preprocessor, algorithms etc | 10 | |
| G. Deployment environment support | Desired or preferred hardware/OS/cloud platforms | 8 | |
| 7. Development Environment | A. Standardised GUI tool | Example: Eclipse-based | 8 |
| B. Verification tools | Logic verification such as diagram generation and reports | 6 | |
| C. Validation tools | Unit and bulk test capabilities | 6 | |
| D. Debugging | Local / remote debugging capabilities | 6 | |
| E. Team features | SCCS / project repository / team development | 4 | |
| 8. Business interface | A. Business Event / Rule / Decision Management | Management (editors, impact analysis) for business-controlled event processing aspects | 6 |
| B. User Workflow / control | Control and security for business updates to event processing | 6 | |
| C. Event monitoring dashboard | Display of events, and complex events such as KPIs | 6 | |
| 9. Project considerations | A. Track record | Numbers of Customers / users in same domain | 6 |
| B. Availability of SIs and development partners | Available assistance if needed | 2 |
One final checklist or scorecard item cannot be added to the list above, but is at least as important as the technology factors. This is the client-vendor relationship, and I was reminded of this when another CEP vendor tweeted recently about how they had been gaining business partly due to another vendor’s “aggressiveness” in business dealings. So, just like everywhere, caveat emptor…. and maybe add a higher score to 9.A. above! 🙂
As usual, comments, feedback and suggestions welcome.
