From the course: Data Engineering Project: Build Streaming Ingestion Pipelines for Snowflake with AWS
Unlock the full course today
Join today to access over 24,800 courses taught by industry experts.
Installing Snowpipe Kafka connector
From the course: Data Engineering Project: Build Streaming Ingestion Pipelines for Snowflake with AWS
Installing Snowpipe Kafka connector
- [Instructor] In our previous video, we learned more about Snowpipe. In this video, we'll be installing our Snowpipe Kafka connector. So continuing our active session from earlier, let's install Kafka Connector for Snowpipe by running a series of commands. I provided these commands in a file in the Related To This Course section so you can copy paste to save yourself time. But it is important we walk through these commands and you understand what they're doing. So I'm going to start by throwing in this make directory command and we'll create a directory called Snowpipe. This is where we're going to install everything. Then we're going to jump over into that directory, so we'll do cd. This next command will clean any cached packages in order to ensure we don't run into issues on installation. Sudo is a bash command for super user do and allows you to run commands as a root user. In other words, the most powerful user.…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Contents
-
-
-
-
(Locked)
Setting up your MSK cluster and EC2 instance3m 52s
-
(Locked)
Setting up your keys5m 8s
-
(Locked)
What is Snowpipe?1m 42s
-
(Locked)
Installing Snowpipe Kafka connector3m 26s
-
(Locked)
Set up config for provider-MSK connection and create topic6m 35s
-
(Locked)
Preparing your Snowflake environment6m 23s
-
(Locked)
Setting up your Snowpipe Kafka connect config4m 29s
-
(Locked)
Sending data to Snowflake3m 25s
-
(Locked)
Final considerations2m 8s
-
(Locked)
-