From the course: Complete Guide to Databricks for Data Engineering
Join today to access over 25,300 courses taught by industry experts.
Understand workflows in Databricks - Databricks Tutorial
From the course: Complete Guide to Databricks for Data Engineering
Understand workflows in Databricks
- [Instructor] Data picks also provide a new feature that is the workflows. Let's just see what is these workflows are. Workflow is an orchestrated sequence of the task. So sometimes you have a crystal clear workflow in the mind, "That my task one would be this. Task two this and task three will be this." And you want to execute all those in this specific order. In that case, the workflow is something you can us. In the workflow you can create these sequence of the task. Sometimes you might be using it for data pipeline, for machine learning, or for any other types of jobs. These workflow which you create, you can schedule it so that you want to execute this workflow at a very specific time interval you can run it. This workflow eventually is like chaining of the task, and you can monitor these tasks just by monitoring a specific workflow. Let's just see some of the components of this workflow. The task. The task is basically a small unit of work. It could be running a notebook. It…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.