Business Data Lineage – The Key to Collaboration

A  Disconnect Exists between IT and Business. Learn How Business Data Lineage Can Aid in Collaboration

David LindblomOctober 17, 2017

Download White Paper

When you run to the store to pick up milk, you never stop to think about the many steps it took to get there. When you make a mobile deposit, you never stop to consider the technology needed to get a check sitting on your dresser into your account without stepping foot inside of a bank. Of course all of this information matters, it’s just not regularly thought about.

Understanding Data Lineage

Data lineage is very similar. The idea is that data goes from point A to point B, or even C or D.  It may even stop along the way and mask its identity, but somehow it makes it to its destination in the format it needs to be. It’s understood that data moves through a variety of stores, extraction and transformation points throughout an enterprise. But lineage isn’t something that’s thought about—it just happens. But maybe it’s time to think about data lineage? If you don’t understand how data has been altered and where it has moved through your data supply chain, how can you ensure its ongoing integrity?

Because data traverses throughout various systems and platforms, it’s difficult for organizations to track, monitor, and assert mature data governance standards, while also providing an audit trail for data’s movement.  Data is changed and transformed in many ways, and then used for analytics or reporting by a variety of users, making data lineage the key to implementing a mature data governance strategy.

Variations in Data Lineage

Data lineage is often technical in nature and thus the mappings might not be easily understandable for a business executive. Truth is, various roles within organizations will have very different perspectives when it comes to data lineage. Each outlook represents the “truth” of data lineage according to their viewpoint, but may not represent the views of others whose perspectives represent the business use and governance of the data. This is why collaboration is critical between IT and business folks. Technical lineage is valuable for those folks in IT or perhaps for more of the application owners, but if business cannot provide their input for lineage, the data governance approaches will not be sustainable.

Often, compliance mandates drive organizations’ need for data lineage and a larger data governance program.  Whether it is for Basel, GDPR, CECL, Dodd Frank, Solvency II, CCAR, or other compliance areas, the penalties for noncompliance can be hefty, and the manual effort to manage all of this data is daunting and costly.

Or, perhaps, there are needs like operational processes such as payments not being executed, bills not being sent or sent incorrectly, misstatements in financial reporting, or running inaccurate analytics off of poor data. Maintaining proper data lineage can help prevent costly data quality issues and inaccurate or insufficient data governance processes.

Thus if the origin or ownership of data is unclear, or data goes missing or cannot be tracked, the costs can be hefty and the reputational damage even worse.  Transformations that occur, various environments, and various formats can often cause confusion on who is overseeing the data. Additionally, various types of artifacts need to be tracked in those places, which can differ, but are important to view.  Whether the data is housed in a database, data lake, legacy system, CRM system, or other source, it needs to be monitored, mapped, standardized from a data governance perspective, and accounted for.  The data may be in motion, but all data needs to be validated and given strong data governance oversight.

Business users are often the ones who really understand what the data means for the business. They understand what artifacts are of vital importance, they understand the day-to-day changes, and likely understand that regulatory compliance mandates result in increased oversight for these items.

Finding a solution that is user-intuitive and can be operated by this business-centric person is critical.  Also a solution that is web based and can be installed quickly and easily via the cloud is important.

Key Considerations for Business Data Lineage

Below I’ve identified 7 key differentiators to consider for business data lineage:

  • Platform that allows for collaboration among lines of business, IT, audit, risk, and compliance teams
  • Rules-based approach to allow for faster changes to data governance processes
  • Workflow to provide a visualization of the lineage that is easily understandable
  • Ability for ownership of the data to be more clear cut
  • Need for analytics, including advanced analytics like machine learning, to predict what types of trends are leading to typical problems with data quality and data governance in certain commonly identified areas
  • Ability to connect automatically with core systems and other tools so that data quality metrics can be viewed and results of data governance practices can be seen from various disparate sources and environments
  • Ability to pull data metrics automatically from your data quality stack of tools, to ensure progress with KPI’s and KRI’s

In closing, once a data lineage process is in place, it needs to be operated by the business in order for it to be successful.  Collaboration with IT and other organizational units is integral to ensure that a future data governance structure is actually operational on a day-to-day basis.

Key benefits that the business team will experience through business-centric data lineage:

  • Fewer manual processes to put together data lineage
  • Less data quality challenges
  • More accurate reporting of data lineage
  • Better holistic understanding and visualization of how data traversed and who is accountable throughout their organization
  • Quicker resolution of data governance and data quality challenges because communication is streamlined via a web-based data lineage/data governance platform equipped with a workflow
  • Nimble changes to data governance artifacts can be executed more seamlessly

Data lineage helps with the maturity of a data governance program because it provides the needed information, at the right level of granularity and context for business to understand and direct the program.

To learn more about implementing a data governance strategy, check out the white paper below.

Get Insights

For a deeper dive into this topic, visit our resource center. Here you will find a broad selection of content that represents the compiled wisdom, experience, and advice of our seasoned data experts and thought leaders.

Download White Paper