Apache Spark™
Developer Certification Workshop 5 Days

@jaceklaskowski / StackOverflow / GitHub
Books: Mastering Apache Spark / Mastering Spark SQL / Spark Structured Streaming

Jacek is best known by the online books:

## Day 2 — Spark SQL

1. [Standard and User-Defined
              Functions](spark-sql-standard-functions-udfs.html)
                * [Exercise: Using upper Standard Function](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Using-upper-Standard-Function.html)
                * [Exercise: Using explode Standard Function](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Using-explode-Standard-Function.html)
                * [Exercise: Difference in Days Between Dates As Strings](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Difference-in-Days-Between-Dates-As-Strings.html)
                * [Exercise: Using UDFs](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Using-UDFs.html)
              1. [Basic Aggregation](spark-sql-basic-aggregation.html)
                * [Exercise: Calculating aggregations](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Finding-maximum-value-agg.html)
                * [Exercise: Finding maximum values per group (groupBy)](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Finding-maximum-values-per-group-groupBy.html)
                * [Exercise: Collect values per group](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Collect-values-per-group.html)
                * [Exercise: Multiple Aggregations](https://jaceklaskowski.github.io/spark-workshop/exercises/sql/Multiple-Aggregations.html)
              1. [Joins](spark-sql-joins.html)
                * [Exercise: Finding Most Populated Cities Per Country](https://jaceklaskowski.github.io/spark-workshop/exercises/sql/Finding-Most-Populated-Cities-Per-Country.html)

## Day 3 — Advanced Spark SQL

1. Exercises
                * [Exercise: Using pivot to generate a single-row matrix](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Using-pivot-to-generate-a-single-row-matrix.html)
                * [Exercise: Reverse-engineering Dataset.show Output](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Reverse-engineering-Dataset-show-Output.html)
                * [Exercise: Flattening Array Columns (From Datasets of Arrays to Datasets of Array Elements)](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Flattening-Array-Columns-From-Datasets-of-Arrays-to-Datasets-of-Array-Elements.html)
              1. [Windowed Aggregation](spark-sql-windowed-aggregation.html)
                * [Exercise: Finding 1st and 2nd Bestsellers Per Genre](https://jaceklaskowski.github.io/spark-workshop/exercises/sql/Finding-1st-and-2nd-Bestsellers-Per-Genre.html)
                * [Exercise: Calculating Gap Between Current And Highest Salaries Per Department](https://jaceklaskowski.github.io/spark-workshop/exercises/sql/Calculating-Gap-Between-Current-And-Highest-Salaries-Per-Department.html)
                * [Exercise: Calculating Running Total / Cumulative Sum](https://jaceklaskowski.github.io/spark-workshop/exercises/spark-sql-exercise-Calculating-Running-Total-Cumulative-Sum.html)
                * [Exercise: Calculating Difference Between Consecutive Rows Per Window](https://jaceklaskowski.github.io/spark-workshop/exercises/sql/Calculating-Difference-Between-Consecutive-Rows-Per-Window.html)

Prerequisities

Be prepared to get the most out of the workshop

Prerequisities / Experience

Hands-on programming experience using Scala
Experience developing Spark applications

Prerequisities / To Be Installed

Java Platform, Standard Edition (Java SE) 8
Apache Spark
IntelliJ IDEA Community Edition with Scala plugin
sbt
Linux or macOS preferred
- Consult Running Spark Applications on Windows when on MS Windows

In-Class Preparations

Make Instructor's Life Slightly Easier. Thanks!

Introduce Yourself

First name
What's your experience with Spark?
Any production experience with Spark?
What certification do you think of?
Have you already signed up?

Addendum

Put your name on paper in front of you
- Stick to your laptop if possible
Lunch at 12:45pm

Apache Spark™ Developer Certification Workshop 5 Days

@jaceklaskowski / StackOverflow / GitHub Books: Mastering Apache Spark / Mastering Spark SQL / Spark Structured Streaming