Big data is no longer a new phenomenon; it is now just the norm, with millions of organizations across the world having started a big data implementation. However, not all of them are successful. Today, many organizations struggle to generate value from the huge amount of data they ingest, making it difficult to access, use and analyze the data that might best meet their needs. In fact, a recent survey by NewVantage Partners found that “more than 85% of respondents report that their firms have started programs to create data-driven cultures, but only 37% report success thus far.”
Organizations are now realizing that they must rely on metadata to classify, manage and organize the massive amounts of diverse enterprise data found across the organization. Metadata shows business and technical users where to find information in data repositories while providing details on where the data came from, how it got there, which transformations it has undergone, its level of quality, and its relationship to other data or reports.
Best practices with metadata starts at the point where the data enters the organization. With so much differing information on the internet, we’ve highlighted the most common types and functions of metadata. By understanding what these mean, organizations can begin to build systems that allow advanced data management with a demonstrable ROI.
Metadata allows organizations to describe and document the data, but most organizations struggle to implement metadata solutions. In order for metadata to help an organization to better understand, track, and retrieve data, they must first learn how metadata itself can be effectively structured. This begins with understanding the three different types of metadata.
Physical Metadata: This is the type of metadata that deals most directly with the physical location and storage of data. It consists of technical metadata that can be automatically populated by the database or by any technical application that moves, changes or stores data.
Logical Metadata: This type of metadata concerns the design of data flows through the system. This could include schema or other design documents that map out how data is supposed to go from intake to end consumption by the data analyst or other data consumer. This metadata can be captured from data modeling software or from systems architects.
Conceptual Metadata: This type of metadata deals with the meaning and purpose of data as understood from the business perspective. It can include the typical uses of data, and how particular datasets are used in business processes. Most of this information must be captured from the minds of business users.
These three types of metadata represent three vantage points from which to view and understand data. It is necessary to acquire and keep refreshed all three types of metadata in order to take full advantage of your data assets. Now that we understand how metadata needs to be managed, we need to look at how metadata is used.
Metadata can be used to summarize basic information to advanced functionality. Some of the most common examples of how metadata is used includes search, browse, syndication, access permissions and more. More advanced usage includes:
Metadata management is no easy task with plenty to consider. Organizations need to ensure metadata is stored where it can be accessed and indexed so it can easily be found. The quality of the metadata needs to be consistent so all users can trust it and the same data needs to be kept up-to-date over time. So how do you create, maintain, update, store, publish and handle metadata? That’s where a strong data governance strategy can help.
To leverage metadata, organizations will need to operationalize data governance by employing a comprehensive data governance platform that delivers a complete view of an organization’s data landscape. The platform should include interactive data visualization and lineage capabilities and deliver transparency into all aspects of an organizations data assets.
In addition, the platform should have automatic discovery capabilities, enabling the capture and monitoring of changes to metadata. Once changes are discovered, the technical metadata relationships may be investigated to deliver meaningful insights on data. With the proper data governance platform, enterprises can empower their data community to use metadata to help jump start their big data initiatives.
The real value comes to light when you can take the technical lineage and translate it into a business lineage – this is one of many ways to capitalize on metadata value by pairing it with data governance.
To learn more about managing metadata and creating a glossary, download the data sheet below.
For a deeper dive into this topic, visit our resource center. Here you will find a broad selection of content that represents the compiled wisdom, experience, and advice of our seasoned data experts and thought leaders.