Different business stakeholders have quite different expectations and requirements for data lineage. The common understanding of data lineage remains the same and describes the movement and transformation of data from its source to its destination. At the same time, a method to document data lineage varies depending on the stakeholders’ expectations and needs.
1. The subject of documentation
Metadata and data value lineage are two entirely different types of data lineage.
Metadata lineage
Various stakeholders have quite different expectations and understanding regarding data lineage. Data management and IT professionals usually understand data lineage as the documentation of the data processing and transformation made by means of metadata. Often, some professionals use the term “metadata lineage” only in the context of the automated data lineage on a physical level. Such an approach is not entirely correct. The description of data lineage at any level of abstraction is also metadata lineage. Simply, at different levels of abstraction, you use different metadata.
Data value lineage
The business stakeholders’ understanding of data lineage differs from that of their data management and IT colleagues. Business stakeholders want to see data transformation at a data instance level.
2. The layer of documentation
These are business, conceptual, logical, and physical. We discussed these layers in the section devoted to the metamodel of data lineage. Different companies use various numbers of layers and constituent components to describe data lineage. They also use different terminology to describe these layers.
3. Direction of documentation
Depending on the direction, we recognize vertical and horizontal data lineage.
The conventional definition of data lineage specifies the horizontal type of data lineage. It demonstrates the path that data flows from the origination point to the point of usage. Horizontal data lineage can be documented at each of the four layers.
Vertical data lineage links data lineage components between different layers.
4. A method of documentation
Depending on the method, we talk about descriptive or automated data lineage.
Descriptive data lineage is a method to record metadata data lineage manually in a repository.
Automated data lineage is the method to record metadata data lineage by implementing automated processes to scan and ingest metadata into a repository.
The post Data Lineage Types appeared first on Solutions Review Thought Leaders.