Data Transformation Done Right

Reading Time: 3 minutes

In the always-on world we live in today, it seems there’s a constant push to achieve new levels of productivity in everything we do. We’re all asked to do more with less, get to results quicker, innovate, and in general, do it all better and faster than the competition. That’s a pretty tall order.

In the world of application integration, a key driver of developer productivity revolves around the degree of automation (automatic code generation) provided and the extent to which artifacts generated are capable of being reused. The more tasks that are automated, the more productive the developer will be.

The truth is, at first glance, many integration solutions on the market look similar in terms of how they work. Most integrate their tooling within an Eclipse IDE and many use a visual drag-and-drop metaphor to create services and integration points. Pre-packaged connectors are generally available that support popular commercial applications, although the quality, quantity, and level of support for the connectors can vary significantly.

However, when you dig below the surface a little bit, you will find some important differences in how this functionality is implemented that have a direct impact on developer productivity. This is particularly true if you are evaluating open source solutions. Developer productivity is a hard-to-quantify, but very real component that goes into calculating your total cost of ownership (TCO).

For example, one of the most common functions in creating any integration flow is the formatting of data as it’s passed from one application to another. Data needs to be transformed from the data structure in the source application to a different structure required by the target application. Depending on the nature of the applications involved, some of these data transformations can be quite complex.

With TIBCO Business Studio, a visual tool is provided to perform this data mapping, executing simple drag-and-drop functionality. Once the mapping is complete, all of the code required to do the XSLT transformation is automatically generated by the tooling. No hand coding is required by the developer. Not only is this highly productive, the XSLT artifact can now be reused in other integration flows with no modification—or easily modified to support a different set of mapping needs.

When this approach is compared to how data mapping is done in MuleSoft, for example, the developer experience is quite different. MuleSoft is an example of an open source product that does not provide visual data-mapping capabilities in their entry-level product. Developers need to create their own custom XSLT files, or scripts, for every data transformation they need to make.

Without solid visual-mapping capabilities, the creation of integration points is significantly more complex, requiring greater skills and more time (i.e. less productivity). The end product is also not as agile, since small changes to the data structures require rewriting of the scripts, testing, etc. The concept of normalized data formats is out the window, reducing the likelihood of reuse of services, further driving up TCO.

Not only does data mapping in this environment require hand coding, developers also need to learn the MuleSoft proprietary expression language (Mule Expression Language or MEL) to filter, route, or otherwise act upon the different parts of the Mule message object. This will have a negative impact on both developer productivity and artifact portability.

There are a lot of different reasons why people buy integration platforms and one size does not fit all. But, if developer productivity, artifact portability, and ease of use are important factors in deciding which solution is best for you, dig a little deeper than the marketing literature and whitepapers. Download trial versions of the products you’re interested in give them a workout. Talk to your friends about their experiences. And when evaluating open source software, make sure you look at the TCO, not just the upfront purchase price.

After all, isn’t one of the key reasons we all invest in technology to stay on the productivity curve and win the hand-to-hand combat with competitors? Set yourself up to win.

Download a trial version of ActiveMatrix BusinessWorks today.