The analysis of different sources has delivered this list of concepts comparable to data lineage:
· Data value chain
· Data chain
· Data flow
· Integration architecture
· Information value chain
The analysis of these concepts has led to the following conclusions:
1. Different reference industry guidelines have diverse viewpoints on data lineage.
2. There is no aligned, unambiguous definition of data lineage. Definitions have changed through time.
3. Several other concepts have definitions similar to data lineage. All these concepts describe data movement and transformation at different levels of abstraction.
4. The names of these concepts are often used interchangeably. A data chain is considered to be the same concept as data lineage. Data flow is defined as a type of data lineage. Data lineage, data flow, and information architecture are different names for the same concept. Figure 1 demonstrates these dependencies.
Figure 1: An overview of relationships between different concepts
5. Data movement is described in the limits of the point of data origin/source/beginning to the point of usage/current location/end/target. The words used to describe these limits demonstrate one of the important features of data lineage: the relativity of its scope or length. It means that data lineage documentation can be limited by the relative “begin” and “end” of a data flow.
6. Data lineage maps the movement of data between systems, applications, and data sets and can be documented at different levels of abstraction.
7. Data lineage maps data movement to business components such as organizations, business processes, and roles.
This analysis has delivered a set of components that constitute data lineage-related concepts:
· Business processes
· Business functions and roles involved in business processes
· IT assets such as systems, applications, databases, and networks
· Data models at the conceptual, logical, and physical levels
· Business rules and their technical implementation in the form of ETL (Extract, Transform, Load) processes
Different data lineage-related concepts document data pathways at different levels of abstraction and use components corresponding to these levels. The analysis has resulted in the identification of the four levels of abstraction with corresponding components:
· Business level (business processes, business functions and roles, and IT assets_
· Conceptual (objects of conceptual or semantic data models)
· Logical level (objects of logical data models)
· Physical (objects of physical data models)
The post Data Lineage and Other Comparable Concepts appeared first on Solutions Review Thought Leaders.