The Building Blocks for a Metadata Management Foundation
Part 1: Laying the Groundwork for a Metadata Management Foundation
Deep in the information age, trends develop and mature at lightning speed. As big data has become the norm and organizations grapple with volumes and velocity of data never before seen, the latest buzzword among data management professionals is “metadata management,” and for good reason. Organizations can use metadata to classify, manage and organize the massive amounts of diverse data collected across their enterprise. This information is crucial to both understanding and effectively deploying resources to support enterprise departments. And metadata provides crucial information to enable true predictive analytic insights. If you’re still unclear about metadata, check out my blog metadata 101.
However, managing metadata requires far more than a data governance tool. It requires a proper foundation, clear processes and the right people to execute the work. The building blocks of metadata management, then, consist of both the tools and technology that comprise a strategy, as well as the combination of people and process to create a culture of support and accountability. As with any major business initiative, there will be minds to change and challenges to overcome right from the start, which is why you can’t afford to wait.
In this two-part blog series, we’ll first tackle how to lay the metadata management foundation to execute a successful, sustainable, and scalable metadata management solution. In our next post, we’ll take a closer look at how organizations can build a metadata culture for long-term success.
Meeting Challenges from the Beginning
A typical challenge confronting businesses today as they seek to implement data initiatives is getting buy-in from upper management. Another common, and related issue, is the amount of time it can take to realize or demonstrate ROI. When investing in metadata management, there will be immediate impacts, though the full ROI will be realized over a longer span. Properly managing metadata requires persistence that few organizations will sustain unless the processes are built into the fabric of the work. But when fully implemented, the pay-off is substantial, providing organizations with increased profits and improved operational efficiencies, optimized data utilization, and maximum value from data assets.
Organizations rarely have sufficient resources to staff multiple projects concurrently, and those that require specialized expertise such as a data scientist are in high demand, causing bottlenecks. Not only are people competing for resources or are spread thin across various projects, but projects that require broad participation and collaboration among divergent lines of business further undermines progress. Operations is consumed with the day-to-day business, marketing is busy with creating campaigns and content, IT is juggling myriad requests and priorities from data to environments to end user support, and finance is buried in balancing the books. Getting everyone in the same room at the same time to discuss, organize and define data is a near impossibility, not to mention a low priority for most stakeholders.
It’s High Time to Get Your Team Involved
So how do you get each department not only involved with, but invested in, metadata management? Each department needs to realize that proper metadata management will help them find the “gold nuggets” buried in organizational data that can lead to better business decisions and greater profitability. If finding the right patterns, trends, insights and actionable information isn’t enough incentive to get employees involved with metadata management, then upper management will have to leverage every resource they can to instigate change. This can include creating visibility for metadata work, carving out career paths, and being creative to build short term and long term incentives that are consistent with the organization’s culture and policies.
Once an organization has both budget and buy-in from upper management, and has laid the groundwork for what metadata management can achieve with employees and departments across the enterprise, they can begin building a metadata management foundation.
Laying a Metadata Management Foundation
When starting a metadata management program, there are three fundamental steps that must be handled by people within or outside the organization. They are:
- Design of the Model and Implementation of the Tool: Every business is different, and each one needs to ensure that their model is customized to fit their specific needs. The architect of the metadata model must guarantee that organizations are collecting the right inventory of metadata to solve their individual business problems. In addition, the tool must also be configured to meet ongoing business needs. This step should be handled by an internal specialist or an outside consultant with experience working with all types of metadata. Ideally, the architect reports directly to the Chief Data Officer (CDO).
- Oversight and Management of the Metadata: As with any project, there needs to be an assigned project manager to ensure everything is going as planned. In this case, organizations need a manager who understands the metadata model to guarantee that once the tool is designed, that the right information is being collected and properly maintained, and that the work is done correctly and on schedule. The manager role is not highly technical, but can see to it that that the metadata model is networked together to promote usability in design through implementation and beyond.
- Acquisition of Metadata: There are three types of metadata that need to be collected. The first two types are physical and logical metadata. Physical metadata deals with the location of the data and logical deals with the flow of data through an enterprise. The data lineage of logical metadata provides critical information on where data is coming from and going to, and both of these types are technical in nature. For this reason, collection of these types of metadata requires an analyst with technical skills. Much of the physical and logical metadata can be refreshed automatically once it has been initially collected.
Conceptual metadata, on the other hand, deals with the meaning and purpose of data as understood from a business perspective. It is the “data” in people’s heads, and must be collected from actual people within each line of business. The metadata architect can oversee this effort, but it needs to be a strategic and collaborative effort. Whoever is tasked with collecting conceptual metadata will have to prioritize and plan for what metadata to target first, to target quick acquisition of high value information. It involves a process of one-on-one, small group, or larger workshops with people from across different lines of business, and the collection and documentation of their business definitions to make sure that discrepancies are known and documented so that bad assumptions about data do not lead to bad decisions.
Once an organization has an established metadata management foundation, business and technical users will be able to quickly find data repositories and details on its lineage and reliability – in other words, where the data came from, how it got there, which transformations it has undergone, its level of quality, and its relationship to other data and reports.
For more information about data governance and metadata management, download this datasheet.Download the Data Sheet