One of the interesting requirements for XTP/XEP (as mentioned in the previous blog posting) is that such performance (and the need for fault tolerance / failover) requires co-operation across multiple systems, ideally generalised in a multi-agent architecture [*1]. The problem here is usually in re-factoring to change performance characteristics as the project develops, to optimize the architecture to the performance needs, available compute nodes, and network (as well as changes to event throughputs).
One solution is to take the agent-based approach, where agents can be re-configured (through model parameters) very simply. Agents can:
- execute declarative production rules “as required”, removing most of the need to specify / re-engineer (/ debug etc) “entry” and “exit” points;
- use queries to partition data (/event objects) and process loads to separate agents;
- rely on shared distributed cache to provide a “co-operative model” of events and data for processing.
The re-factoring / re-architecting process is, to some extent, as simple as specifying which rulesets are assigned to which agents, and defining more or fewer agents as required:
a. Revise named agents (aka agent types and roles)
b. Re-allocate rulesets (or queries) to appropriate agents
c. Deploy (or re-deploy, with appropriate care) in numbers required.
So the agent modeling aspect is to map the event processing elements first to agent base types (e.g. inference or query [*2]) and subtype (e.g. event preprocessor, event router, event/data processor, batched event/data processor, etc) and verify the event flow / data flow is complete and without contention. In this way you can quickly re-architect a TIBCO BusinessEvents application from single-node to multi-node, high performance to extreme performance…
Notes:
[1] Agent representations and modelling is currently the subject of an OMG standard, AMP, currently under development.
[2] CEP agents in BusinessEvents come in 2 forms: inference agents and query agents, roughly equating to CEP agents and Event Stream Processing agents…
