From the course: Scala Essential Training for Data Science (2017)

Unlock this course with a free trial

Join today to access over 25,300 courses taught by industry experts.

Working with JSON files

Working with JSON files - Scala Tutorial

From the course: Scala Essential Training for Data Science (2017)

Start my 1-month free trial Buy for my team

Working with JSON files

“

- [Narrator] In this session I'd like to show how easy it is to work with JSON files when we're working with Spark. Now I've started my Spark RPL. Now if you've viewed previous lessons, you're probably familiar with the idea of creating a Spark session, so I'm simply going to import org.apache.spark.sql.SparkSession and I'm going to create a context variable called Spark and I'm going to make a Spark session. I'm going to call the builder and I'll give this an app name. And we're going to call this "DataFrameExercise" and I'm going to simply ask Spark to get or create this Spark session. Great, so now I have a Spark session that I can work with to load my JSON file. So I'm going to clear the screen and I'm going to load up a JSON file. And let me just flip over to a text editor to show you what that looks like. This is the same data that we've worked with in the department division table. Instead of using a comma-separated file, I formatted this as a JSON file. So let's load that into…

Contents