BAM for the EDW “process”

Reading Time: 2 minutes

A Solution Architect posed an interesting question over on the Complex Event Processing LinkedIn discussion board run by Tim Bass, which I reproduce here as it references TIBCO CEP:

In my case I have to think as to monitoring/governing a EDW (enterprise data warehouse) where we have a classic approach (host systems (in different legal entities of the Group) -> staging area -> edw -> dm) that can be thought as a state machine but we have also a lot of Human tasks thinking about data quality processes.
So, my reasoning have to be .. If I have human tasks then I have to build a BPM and using a BAM/CEP in order to control our environment?
In any case could BAM+CEP (re -thinking about Progress or Tibco solutions) without BPM solve my problem?
(just a little detail, my BPM is Websphere Process Server) …

Of course there are multiple terms and technologies in play here:

  1. data cleansing and validation – typically rule based (technology: BRE; product: TIBCO BusinessEvents), and often-times requires manual intervention (technology: BPM workflow; product: TIBCO iProcess); note that this could also be considered ETL
  2. data lifecycles – this could be considered a “state” issue (technology: state machine; product: TIBCO BusinessEvents), but is probably orthogonal to “data management” (technology: MDM; product: TIBCO CIM)
  3. operational awarenessBAM over IT systems, monitoring their state usually via their state transitions (events) (technology: CEP; product: TIBCO BusinessEvents).

Interestingly, one of TIBCO’s technology tenets is “no rip n’ replace”, meaning customers (like this one) use TIBCO CEP with non-TIBCO BPM. And the need to use a BPM tool will depend on the need to manage the manual interventions (i.e. workflow) and the complexity of the tasks / process models – again a popular TIBCO technology mix is using “TIBCO BPM+” a.k.a. CEP driving BPM.

Of course, an interesting question is also whether the “new operational data” actually represents useful events that should be precessed as such en route to the EDW, for faster decisions…