From the course: Complete Guide to Google BigQuery for Data and ML Engineers
Join today to access over 25,300 courses taught by industry experts.
Data engineering and machine learning in BigQuery - BigQuery Tutorial
From the course: Complete Guide to Google BigQuery for Data and ML Engineers
Data engineering and machine learning in BigQuery
- Okay, next what we're going to do is just take a real high-level tour of different ways of doing data engineering and machine learning in BigQuery. So what I want to discuss are some different methods for analyzing data in BigQuery, how we can work with BigQuery Studio, which is really sort of the main UI tool for working with a variety of very specific tools like notebooks. And then also, I want to take a quick look at just combining the advantages of BigQuery and Spark, or Apache Spark. So one of the things, just as a real basic level that we do in data engineering is we do basic data quality checks. Well, if you're in BigQuery and you want to do that, you might want to do things like row count verification, you might want to check your data for range checks. Basically these are like sanity checks. So for example, things like cost of a product or the selling price should never be a negative number. So we want to be able to quickly implement checks on that. We also want to be able…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.