Video : ER/Studio Enterprise Team Edition

Data Lineage

Data lineage is the ability to track the origin, movement, and transformation of data from its source to its final destination. It is a critical aspect of data management and governance, and it provides a historical record of the data. That includes information about where it came from, how one transformed it, and where one used it. Data lineage is important for ensuring data quality, compliance, and understanding the impact of data changes. With data lineage, organizations can identify any issues or potential errors in the data, trace the impact of data changes, and verify the accuracy of the data. It is also useful for regulatory compliance, as it provides transparency and accountability regarding data usage.

Implementing a robust data lineage process provides the following benefits:

Data lineage makes it easier to discover, track, and correct data process anomalies. That can be crucial for ensuring the accuracy and integrity of the data of an organization.
Data lineage allows organizations to migrate systems. That lowers the cost of new information technology (IT) development and application maintenance.
Data lineage helps organizations to combine new and existing datasets with an agile data infrastructure.
Tracking data lineage can help determine when new or changed data requires retraining models and reducing model drift.
Data lineage provides necessary visibility and context into the data of an organization. That enables them to comply with regulations.
Data lineage frees up time for IT to focus on strategic initiatives instead of mapping data.

Watch this video to discover how ER/Studio can help you set up a robust data lineage process.

Transcript

Expand

00:02

What is Data Lineage?

Hello and welcome to this presentation on IDERA ER/Studios Data lineage feature. My name is Anil Mahadev. I’m a senior solutions architect here at IDERA. So what is the agenda for today? We’ll be talking about what exactly is data lineage? Why data lineage versus visual data lineage and we’ll be wrapping this up with a demonstration. So what is data lineage? Data lineage helps you to identify the data origins from where it came from and how it moves over time. Data lineage provides greater visibility and flexibility in order to trace those errors back to the root cause in a data analytics process. Now let’s say, for example, if you’re trying to reverse engineer from a particular database and you want to know where all the relationships are being present, ER/Studio through Data Lineage can help you to leverage those areas. For example, if you wanted to actually go ahead and know how exactly an SSI package would be, you could use ER/Studio’s data lineage feature to do so.

01:42

It is also helping you to organize, structure and model your data from multiple data sources both relational and unstructured. So why data lineage? According to the BI network, a visual representation of data lineage helps you to track data from its origins to its destination. It also explains the different processes involved in the data flow and their dependencies. Also, the metadata management is the key input to capturing Enterprise data flow and presenting data lineage as a whole.

ETL

Now let’s talk about how you can identify and see the difference between the data lineage capability versus the visual data lineage feature. In Data Architect standard, the visual data lineage indicates that in your studio we have the Data lineage tab which is primarily used to document ETL processes from scratch and this is often a manual process. Data lineage, on the other hand, helps you to import ETL processes through its sophisticated bridges that automate the process for you.

03:20

ER/Studio Data Architect

Some of the examples would be Microsoft SSIS, Informatica, IBM DataStage and many more. With that, let’s jump right into the demo. Here I have, ER/Studio Data Architect. Enterprise. Team edition running. With that, we supply you with a host of meta wizard bridges as well as the key data lineage feature. If you were to go to file import file from external metadata in this demonstration, we’ll be taking a look at how to reverse engineer an SSIS package.

SSI

Here I have a Microsoft SSI package that I have. You can choose any one that you have. Here you can see that we have a variety of other data sources that you can bring from. Here you have Oracle Data Integrator and you can have any of the other data sources that you need. You have Informatica, you have MicroStrategy and plenty more. Right now for this example, I’ll be choosing myself SSIS choose next.

04:41

Data Lineage

Here I’ll just give it a friendly name go ahead and click next, then let ER/Studio do the magic at this point, what it’s doing, it’s actually processing that SSIS package that will give you both all the information from its control flow, its data flow, et cetera. You can actually go ahead and click Finish. Now let’s turn our attention to the Data Lineage tab. I’m going to go ahead and click on that and I’m going to click on my orthogonal layout to see what it’s brought to me. Here you can see you have a variety of packages and information about that package being displayed. What I would like to do is I’ll click on the zoom in button and I’ll zoom into this particular area. Here you can see I have my transformation being linked to the currency rate and the fact currency rate here.

05:48

Documentation Metadata

Now this is part of the Data Lineage capability where it has automatically gone ahead and picked the relevant columns. That’s one of the coolest features in the Data Lineage area. Now here you can also go ahead and let your ETL developers know what exactly you would like to have a business definition for. Here you can go ahead and bring all of that metadata as well. Here if you wanted to go ahead and generate another set of documentation metadata, you can go ahead and fill in the business information as well. That was how easy it was to work with the Data Lineage feature by bringing in an SSIS package. The ideal advantage is that our database lifecycle management solutions allow database and It professionals to design, build, monitor and improve data systems with complete confidence. We are at the forefront of getting this latest and greatest technologies into the hands of our customers.

07:15

I would like to thank you for attending this quick demonstration on how to look at the Data Lineage feature. If you have any questions, you can follow me on Twitter at SQL Server Cowboy or send an email to my email address for any further questions or requests for any further demonstrations. With that, I thank you for your time and have a good day. Bye.

Topics : Data Modeling,Enterprise Architecture,Metadata,

Products : ER/Studio Data Architect,ER/Studio Enterprise Team Edition,ER/Studio Team Server Core,

IDERA ER/Studio Enterprise Team Edition is a powerful business-driven data architecture solution that combines multi-platform data modeling, business processes, and conceptual modeling with cross-organizational team collaboration for enterprises of all sizes. Data and business professionals can easily capture and share models, metadata, data sources, and glossaries across the organization for improved alignment between business and IT. The data lineage capability enables users to see the flow of data through its life-cycle and trace the transformations from all data sources.

Request a demonstration of ER/Studio Enterprise Team Edition

Contact IDERA:

Contact Sales