Patterns surround us in our everyday lives. Subconsciously, we navigate our way around the world by recognizing patterns, and taking appropriate actions based on our experiences of what to expect when we encounter that same (or a similar) set of patterns. Just as in life, the data that flows through our organizations is chock-a-block with patterns, and just as in life, if we can recognize those or similar patterns, our organization can take the appropriate actions.
Patterns in Data
There are two levels of patterns in and about data that we need to be aware of. The first is the patterns hidden within the data being processed and managed by our traditional systems. Let’s call them type 1 patterns. As humans looking at the data, we might, for instance, conclude that multiple customer records from different systems are actually about the same customer. Of course, it would be easy if each record had exactly the same content; in real life, of course they often don’t. There are numerous differences, inconsistencies and errors we need to deal with. Some of these are illustrated below:
Humans take these types of differences and errors in stride and, with enough similarity in enough of the attributes, can easily reach an accurate conclusion about what the patterns in the data are telling them; in this case, that these records are actually about people in the same household.
Pattern recognition of this type by humans includes a great deal of fuzziness. This enables us to not ignore, but take into account differences and errors; for instance, in names and addresses, misfielding and sparse data. The great part is that we do this without really thinking about it. Our systems, on the other hand, have a terrible time when faced with this sort of data. There is no fuzziness in the way current systems try and determine similarity. To a large degree, the content of attributes are the same – or are not the same; this, for instance, leads to the situation where systems can’t tell that the sample data is about people in the same household.
It seems obvious that we need to provide our systems with the same type of fuzziness humans employ when faced with the types of differences and errors that are rife in our data. This would enable these systems to become much smarter about how they deal with this type of problem. And, of course, this in turn enables higher degrees of automation in business processes where today only humans are able to deal with the data. For an illustration of capabilities against several different types of data in several languages.
To see powerful capabilities at work, check out real, live demos against a wide variety of types of data in a variety of languages and check here for more detailed information, including customer success stories, recorded webinars, whitepapers etc.
Stay tuned for more about the power of patterns!
