From the course: Data Quality: Transactions, Ingestions, and Storage
Join today to access over 24,900 courses taught by industry experts.
Ingesting dirty data
From the course: Data Quality: Transactions, Ingestions, and Storage
Ingesting dirty data
- [Instructor] For our next step, we're going to intentionally ingest bad data to see how the report changes and identify what improvements we can make to the data platform. Now, in the previous step, we used a Python script that was already preset. We're now gonna use the PostgresIngestionCSV class and run it manually. And that's essentially what's happening in the background. So we're gonna run the cell. Again, highly encourage you to read through the logs and see what was created through that script. But our goal is to see what changed in the system, so let's run this same report again with our SQL query. And below we can quickly see, as expected, our dirty data is in here, as highlighted with code negative 99 and test. Just to quickly go see what that looks like in our actual data. We can go dirty data right here. And we can go click on parking violation. And we can see the summons number, the code, and the…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.