TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Data Quality Predictions for 2019

If you're serious about data quality, pay attention to these four key trends in 2019 and beyond.

• By Geoff Grow
• December 21, 2018

It was an exciting year to be in the data quality business. In terms of both new capabilities, and the growing strategic business role of data quality professionals, we have seen a lot of growth -- and, more important, legitimacy. Compared with even a few years ago, we are now an increasingly mission-critical part of organizational efforts ranging from fraud prevention and compliance to revenue generation. The emergence of roles such as the chief data officer (CDO), as well as the continued growth of data governance efforts, point to where we are headed as a profession.

For Further Reading: The Essential Role Data Quality Plays in Compliance Accessible Data Preparation: 6 Data Quality Tips Reducing the Impact of Bad Data on Your Business

Against that backdrop, here are four trends I feel are going to occupy more of our attention in 2019.

Consumer Privacy Compliance Will be Front and Center

If there was a single seismic event that changed the way businesses looked at data quality this year, it was the rollout of strict consumer privacy legislation such as Europe's GDPR regulation. And guess what? It won't stop there.

Consumer privacy regulations will continue to proliferate and force companies to change the way they do business. In 2019 and beyond, get ready for more legislation and privacy initiatives, ranging from Canada's just-introduced PIPEDA laws to the new California Consumer Privacy Act, signed into law this summer and taking effect in 2020. Compliance efforts will continue to be a growing and increasingly complex business function reaching into the C-levels of most organizations.

Once upon a time, bad contact data and sloppy direct marketing were accepted by many as a cost of doing business. Today, threats of steep fines and regulatory enforcement are driving everyone to clean up their act as well as their customer-centric data -- which, to me, is a positive development. From here, data quality tools such as contact data validation will remain at the top of the agenda as organizations are forced to have genuine and accurate information on their customers and prospects to avoid risking non-compliance penalties.

Enterprises Will Demand Greater Security and Uptime From Vendors

As businesses become increasingly accountable for data security standards -- and increasingly vulnerable to being liable for issues ranging from data breaches to cybercrime -- more of them are now demanding that vendors have strict data security/cybersecurity measures in place at both a procurement and a contract level. Moreover, these firms are often tying their business interests to their promises of uptime and availability.

Smart vendors will need to be increasingly proactive about data security and uptime standards to be competitive in the marketplace. For example, we offer our customers service-level agreements and publicize our own data security measures such as encryption, firewalls, and access controls -- not just for the sake of good marketing but because we know that without documented bank-grade security and guaranteed service levels, we aren't in the running for many major clients.

Reliance on APIs Will Grow

We will probably always have a stratified market for data quality applications, ranging from batch one-and-done database cleaning all the way to integrated real-time processing capabilities built directly into marketing, CRM, and other business automation platforms. From where I sit, the latter will clearly continue to grow and dominate the market. People like -- and are increasingly demanding -- the integration and ease of use that come from API-connected services.

Increasing reliance by businesses on interconnected digital solutions will continue to drive this growth of APIs. I am far from alone in this belief: in its 2019 predictions for CIOs, IDC states that by 2021, line-of-business needs will drive 70 percent of CIOs to provide "agile connectivity" via APIs and interconnected digital solutions.

Data Quality Will Become More Strategic

If there is one trend we see in our own business and elsewhere, it is that data quality applications are shifting from back-office functions such as database cleanup to strategic, revenue-generating ones.

Organizations are discovering that the same databases and infrastructure they use for data hygiene can also support valuable business intelligence functions such as demographics and targeted marketing. Moreover, they can boost the yield and revenue potential of contact data assets by completing and appending data, filtering out useless noise and fraudulent information, and transforming silos of individual touch points into integrated multi-channel contact data.

For data quality professionals, the good news is that exciting new capabilities are going to continue coming through the pipeline. This also brings with it added responsibilities. Over time, I predict that data quality efforts will increasingly be expected to have a revenue-generating component.

A Bonus Prediction

Here is a final prediction that I can guarantee with complete certainty. Given the rapid pace at which data quality issues are still growing and evolving, 2019 will never be boring for any of us who call this profession home. I am looking forward to a very exciting new year.