From the course: Build a No-Code ETL Pipeline with Google BigQuery
Unlock this course with a free trial
Join today to access over 24,500 courses taught by industry experts.
What is Google Cloud Storage? - BigQuery Tutorial
From the course: Build a No-Code ETL Pipeline with Google BigQuery
What is Google Cloud Storage?
- [Instructor] Before we start building our pipeline, let's briefly talk about Google Cloud Storage or GCS. GCS is a fully managed object storage service that allows you to store and access your data in the cloud. Think of it as a file system on the internet that can hold everything from simple text files to large data sets. GCS is highly scalable and can handle everything from a single file to billions of files, making it ideal for both small-scale and enterprise-level projects. You can access your data from anywhere, integrate it with other Google Cloud services and manage it through a simple and intuitive interface. So why are we using GCS in our project? While BigQuery allows direct CSV file uploads, storing our data in GCS first will provide several benefits. The first is archiviation and backfill. Each version of our data will be saved in a storage bucket, allowing us to keep track of changes over time and easily restore previous versions if needed. Generality, this setup…
Contents
-
-
-
-
(Locked)
How data load will work1m 40s
-
(Locked)
Introduction to data4m 44s
-
(Locked)
What is Google Cloud Storage?2m 42s
-
(Locked)
Put data in Google Cloud Storage3m 35s
-
(Locked)
Create table in BigQuery4m 35s
-
(Locked)
Introduction to BigQuery Data Transfer Service1m 43s
-
How we will manage data6m 4s
-
(Locked)
Use Transfer Service to ingest data6m 40s
-
(Locked)
Schedule transfers with Transfer Service3m 19s
-
(Locked)
Identify data transfer issues6m 18s
-
(Locked)
Common issues with data transfer5m 21s
-
(Locked)
-
-
-