Knowledge graphs, metadata management, and data lineage are examples of new metadata capabilities that many companies have actively discussed and developed. There are some challenges associated with the implementation of these capabilities:
· Definitions of these capabilities are ambiguous and depend on the context.
· These capabilities intersect each other to a great extent and have many dependencies.
· Their implementation is time- and resource-consuming.
Metadata management is a company’s ability to discover, gather, and integrate metadata of required quality to enable data lifecycle. Metadata can be of various types. Various single and complex objects constitute metadata. Data lineage is an example of a complex metadata construct.
Data lineage is a description of data movements and transformations at various abstraction levels along data chains and relationships between data at these levels.
Enterprise knowledge graph is a description of linked data and metadata produced within a business.
Metadata management (MM), knowledge graphs (KG), and data lineage (DL) are data management capabilities that have a lot of similarities and some differences. In any case, they intersect each other to a great extent.
Metadata management forms the basis for knowledge graphs and data lineage.
MM, KG, and DL capabilities have similarities and differences.
Let’s first consider similarities:
1. All concepts deal with metadata.
2. The following business drivers are relevant for all three capabilities: data integration, business change, and IT cost reduction.
3. MM, KG, and DL implementation require automated and manual methods.
4. MM, KG, and DL can be implemented in various IT environments (on-premises, cloud, hybrid) and using different data architecture types (centralized, decentralized, and hybrid).
5. MM forms the foundation for both KG and DL initiatives.
6. All these capabilities are used in data integration-related business cases.
7. KG is one of the technologies that can be used to document data lineage.
In the end, let’s look at some differences:
1. If MM and DL focus only on metadata, KG also links data and metadata.
2. If business changes lead to MM and KG implementation, regulatory requirements are the leading driver for a DL initiative.
3. MM and DL repositories can use both relational and graph databases. KG can be realized using graph database technologies.
The post Knowledge Graphs, Data Lineage, and Metadata Management appeared first on Solutions Review Thought Leaders.