From the course: Complete Guide to Data Lakes and Lakehouses
Join today to access over 25,600 courses taught by industry experts.
Data cataloging
From the course: Complete Guide to Data Lakes and Lakehouses
Data cataloging
- [Instructor] In the previous video, I talk about the importance of metadata. Now I'd like to expand on the topic and explore how metadata is made accessible through data catalogs. A data catalog is a centralized repository, which contains rich metadata that provides tools for data discovery, user collaboration, and governance. The primary purpose of a data catalog is to make the data in lakes searchable and understandable for users, essentially democratizing data access. Let's discuss some of the core functions of data catalogs. As we have seen before, data catalogs are mainly used for metadata management. They integrate with the metadata management system to provide detailed information about data, such as it source, structure, and usage. Catalogs are also used for search and discovery, since they can offer advanced edge capabilities to help users quickly find relevant data based on attributes and text. Another important aspect that catalogs provide is data lineage and provenance…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.