Any craftsman worth their salt has a deep toolbox, knowing that the right specialty tool at hand makes all the difference when completing a high-quality job in a timely manner.
The same can be said for IT, and especially data engineers, responsible for providing data to business consumers. To perform their work, quickly and well, they need to have all the right tools in their data integration toolbox.
Right Tool, Right Job
A broad toolbox with the right combination of data integration capabilities can be extremely valuable when responding rapidly to data requests. It can mean the difference between frustrating delays or punctual satisfaction and better business results.
But there are a variety of data integration tools available today. And it is important to select the right tool if you want to get the job done successfully. To help data engineering leaders ensure their team’s toolbox has everything they need, there are 7 recognized styles for data delivery:
● Bulk/batch (ETL, ELT)
● Replication
● Messaging
● Data services orchestration
● Synchronization
● Data virtualization
● Streaming data integration
Contrary to what some may believe, it’s not about selecting just one of the above tools and assuming it will solve your data integration needs. Many companies will need a combination of data delivery styles to find success—a data integration toolbox that meets their various business goals. According to Gartner, “By 2021, more than 80 percent of organizations will use more than one data delivery style to execute their data integration use cases.”(1)
Why Include Data Virtualization in Your Toolbox
Of all those different data delivery styles, data virtualization is one of the most crucial ones and should be included in your toolbox. Data virtualization is a very useful tool that augments the existing data integration tools that you may already have. It provides a pragmatic integration approach that organizations are increasingly adopting to use along with their favorite ETL/ELT and replication tools.
In fact, according to Gartner’s 2018 Market Guide for Data Virtualization, “Through 2022, 60 percent of all organizations will implement data virtualization as one key delivery style in their data integration architecture…In 2011, only 11% of surveyed organizations reported that they were utilizing data virtualization in a focused set of use cases.” (2)
Cloud Data Sharing is Driving Data Virtualization Demand
But let’s face it, most people try to get by with as few tools as possible. So even when they know about power saws, they won’t buy one until they are faced with a “big job” that requires hundreds of cuts. Cloud data sharing is proving to be just that kind of “big job.”
Why is that? Well, while data and workloads are rapidly moving to the cloud, many legacy systems remain on-premises. This creates a number of new cloud data integration patterns including:
● Hybrid: Integrating on-premises and cloud data
● Multi-instance: Integrating multiple data sources within a single cloud provider
● Multi-cloud: Integrating data across multiple cloud providers
● All of the above: Integrating on-premises, multi-instance, and multi-cloud
With this “big job” of cloud data sharing, comes a need for the right tool to get the job done. Data engineering teams have found that data virtualization is the right integration tool across all these cloud integration patterns. Regardless of the on-premises and cloud topology, data virtualization provides complete, consistent, business-friendly data wherever it happens to live, without moving or replicating it. And even better, its virtualized, metadata-driven data views adapt quickly as your cloud topology changes.
To learn more about how data virtualization can help complete your data integration toolbox and improve your cloud data sharing, check out these educational data virtualization resources we’ve curated for you.
And stay tuned for my next blog on who can most benefit from data virtualization and how to get started!
(1) Gartner, Magic Quadrant for Data Integration Tools, Ehtisham Zaidi, Eric Thoo, Nick Heudecker, 1 August 2019
(2) Gartner Market Guide for Data Virtualization, Sharat Menon, Mark Beyer, Ehtisham Zaidi, Ankush Jain, November 16, 2018
