Technical Data Lineage
A technical data lineage is the technical and detailed representation of the transformations that apply to the data. This representation is intended for technical data managers and data engineers.
From HOPEX you can access the technical data lineages available in . is an automated data lineage platform designed to extract, from a selection of source systems (data sources, file systems), the technical information of data flows, transformations and dependencies.
MANTA Import Overview
MANTA and HOPEX integration involves importing into HOPEX the dictionary of metadata scanned by MANTA, together with their descriptions and references (URLs), so that their technical lineage can be consulted in MANTA from HOPEX.
Imported objects are metadata from data sources such as deployed data stores, schemas, meta dataset and meta fields.
|
Object mapping
|
||
|---|---|---|
|
MANTA
|
HOPEX
|
Properties
|
|
Server
|
Deployed server
|
MANTA identifier, name
|
|
Database
|
Deployed data store
|
MANTA identifier, name
|
|
Schema
|
Meta dataset schema
|
MANTA identifier, name, MANTA link
|
|
Table
|
Meta dataset
|
MANTA identifier, name, description, MANTA link
|
|
Column
|
Simple meta field
|
MANTA identifier, description, data type, length, decimal, MANTA link, identifier field, referencing field
|
|
View
|
Metadata view
|
MANTA identifier, name, MANTA link
|
|
DBMS
|
Software technology
|
Name
|
Processes (stored procedures, functions, triggers, programs, scripts, reports, technical lineage) are not imported into HOPEX. This information is stored in MANTA and referenced by HOPEX through metadata.
Prerequisite Conditions
Authorizing url access to MANTA objects
Technical data lineage functionalities are available with the HTDL product option, which is protected by a license.
To view MANTA technical data lineages from HOPEX, access must be authorized between the MANTA server and the HOPEX server, which can be deployed on site or in the Cloud.
Also, make sure that the URLs of the objects extracted from MANTA and imported into HOPEX are public and accessible to the HOPEX Data Governance solution. These URLs are generated in the "Dictionary.csv" file, in the "manta_hyperlink" column.
Contact your MANTA administrator to specify a URL for access to MANTA from HOPEX.
Setting CSP (Content Security Policies) options
To view URLs imported from MANTA, you must also add the frame-ancestors 'self' directive to the CSPs to allow integration of external resources:
1. Open the HAS console.
2. Click Modules > Module Settings.
3. In the Content Security Policy field, add frame-ancestors 'self'.
Setting a unique server name when extracting metadata
MANTA typically extracts metadata from a set of connections for a given technology (e.g. all Oracle databases).
In the case where MANTA extracts different server names at each connection, it is important to specify a unique name for the server. Indeed, this name is used to identify the namespace of metadata exported from MANTA and imported into HOPEX.
To configure the server name:
1. In Manta Admin UI, open the data source connection settings.
2. In the User-set Server name field, enter the server name.
Revision management in MANTA
To scan a source system, MANTA connects to that system and analyzes the metadata before feeding it into its repository. During a scan, the new metadata is loaded directly; updated metadata is compared to existing assets to detect changes in data lineage.
Each scan of a source system results in a revision in MANTA. A revision is a time frame that shows how the analyzed system looked at the time of the scan.
When you run a scan in MANTA, you can create a new revision or enrich an existing revision with minor revisions.
It is important to note that metadata already imported into HOPEX must still exist in the last major revision or its minor revisions so that the MANTA URL provided in HOPEX gives access to their data lineages. Indeed, the URL displayed in HOPEX corresponds to the latest major revision in MANTA.
Importing MANTA Data
Data from MANTA is exported in a .csv file, named Dictionary.csv. This file can then be imported into HOPEX as a .zip file, for use in a data catalog.
To import the Dictionary.csv file in HOPEX Data Governance:
1. Click the Tools > Manta Import navigation menu.
2. The data lineage import wizard opens.
3. Specify:
• the data catalog, which will reference the imported metadata
• the data center that will hold the metadata
• the .zip file containing the data lineage from MANTA.
For import into HOPEX to work, make sure that the .csv files are at the root of the .zip file and not in folders.4. Click OK.
The scheduler window appears. It indicates that the file will be imported in batch mode.
5. Click OK.
At the end of the import, a notification and an e-mail are sent to the user who launched the import and to the owner of the data catalog.
Once the metadata has been imported into HOPEX, the technical data lineage can be viewed by calling up the MANTA Viewer tool.
Viewing the Technical Data Lineage
Accessing the technical data lineage
You can access from HOPEX Data Governance the MANTA data lineage of a meta dataset. To do this:
1. Open the properties of the meta dataset.
It must be a metadata that has been imported from MANTA.2. Click the Characteristics page.
3. On the right side of the page, click the Visualization of data lineage in MANTA.
4. Log in to your MANTA account.