Poor data quality is one of the biggest threats to data-driven initiatives, no matter an organization’s size, country or industry. In fact, according to Experian’s 2019 global data management research, on average, organizations believe that nearly one-third of their data is inaccurate and of low quality.
The costs of bad data have had a significant impact across the business world. As reported by Gartner’s Data Quality Market Survey, poor data quality is responsible for an average of $15 million in losses annually. These staggering numbers come as no surprise when you consider the massive volumes of data being created, ingested and transformed daily. The continuous struggle to maintain data integrity and maximize data value is the biggest challenge many organizations face in data management.
The risks of bad data are incredibly high. Bad data breeds distrust among data users, preventing them from leveraging data to gain critical business insights. Inaccurate data can quickly pollute downstream processes and proliferate throughout an organization’s data supply chain. And the idiom “garbage in, garbage out” aptly applies to data analytics, where poor quality data can lead to flawed business intelligence. Businesses today need more than a basic data quality tool to ensure data asset value. They need a comprehensive data quality strategy that uses advanced controls, quality scoring and monitoring, rooted in a framework of data governance. Integrating data governance and data quality is a critical step to a successful data management strategy.
As data is created internally or absorbed from a third party, it is critical to verify and maintain its integrity. As data travels through the data supply chain, it is exposed to new processes, procedures, transformations and uses, which all impact data quality. This data must be tracked and reconciled to monitor the attrition of data across the process. It is vital to proactively solve data integrity issues before poor data quality creates significant problems for companies and negatively impacts customer experience, brand reputation, operational efficiency and business decisions.
Enterprise data governance provides a foundation for protecting data integrity across the data supply chain. It establishes the people, technologies and processes to empower business users to easily understand, access and apply enterprise data for business purposes. It educates data consumers regarding data usage, meaning and quality levels, to build trust and encourage data utilization. But that trust can’t be built without robust data quality controls to ensure reliable and accurate data.
To ensure data integrity enterprise-wide, companies should embed a variety of data quality checks directly in their data governance framework. Traditional checks for data quality ensure the completeness, consistency and conformity of data. These validations are critical for measuring the quality of data being used for BI and Analytics. The data must be trustworthy, to generate reliable analytics results that enable the business to make good decisions. Controls for balancing and reconciliation assure that data arrives accurately at its expected location. These validations are used to monitor critical business processes. Missing or inaccurate data can have negative impacts on a business, ranging from lost revenue to reputational damage.
Timeliness checks are also necessary to monitor when files arrive and flag any late or missing files. Timeliness validations are valuable when dealing with third party data. SLAs need to be monitored to ensure third parties are not negatively impacting your business. Statistical controls are essential to validate data sets based on statistical values, like expected standard deviation or industry-defined methods. Finally, reasonability checks are vital to affirm data values meet defined and expected thresholds. Statistical and reasonability checks allow businesses to find data anomalies that otherwise may go undetected by traditional checks.
When businesses enforce data quality checks within a data governance framework, organizations can easily score and monitor data integrity, preventing data issues and building trust among business users. Still, data governance is about far more than ensuring data quality.
Data governance is also about building data understanding across the business. There are a few approaches a business can take to understand their data. The first is a top-down approach. You need to identify what the critical processes in the business are. Once those are defined, you need to see what applications and data are critical in those processes. This is a good starting point to understand where you need to have data quality rules. The second approach is a middle-out approach. You need to identify any regulatory policies. Once identified, you can start looking at the processes and data associated with those policies and implement data quality checks based on them. The third approach is a bottom-up approach. In this approach, you need to discover and profile your data. This will give users an understanding of what’s available and where validations are needed to improve the data.
Data understanding ensures that data is used appropriately to maximize value and minimize risk. However, it also encourages business users to increasingly leverage data assets for analytical insights that can lead to a competitive advantage.
Organizations that take advantage of the synergies between data governance and data quality have a stronger data management strategy that generates better analytics results. Data integrity checks ensure the quality of data before use, while data governance provides business users easily understood, well-prepared data. By combining data quality with data governance efforts, business users can quickly develop meaningful insights from their data to spur organizational growth.
Are you looking for additional information about integrating data governance and data quality to get more from your data? Check out the case study below.
For a deeper dive into this topic, visit our resource center. Here you will find a broad selection of content that represents the compiled wisdom, experience, and advice of our seasoned data experts and thought leaders.