• Jacek Laskowski is an independent consultant
  • Specializing in Spark, Kafka, Mesos, DC/OS, Scala
• Among contributors to Spark 2.1, Spark 2.0 (since 1.6.0)
• Contact me at jacek@japila.pl or follow me on twitter at @JacekLaskowski <-- and learn more Spark!
• Delivering Development Services | Consulting | Training
• Leader of Warsaw Scala Enthusiasts and Warsaw Apache Spark
• Java Champion

https://github.com/jaceklaskowski

https://bit.ly/mastering-apache-spark

Among contributors to Apache Spark 1.6

Among contributors to Apache Spark 2

Among contributors to Apache Spark 2.1

Ranked #96 in Spark contributors

http://stackoverflow.com/users/1305344/jacek-laskowski

https://twitter.com/jaceklaskowski

Agenda (1 of 2)

• Understanding Spark SQL
• Just enough Scala (to learn Spark SQL in notebooks)
• Data access (csv, json, jdbc, parquet)
• Manipulating DataFrames
• Creating temp tables
• From declarative SQL to using Spark SQL programmatically

Agenda (2 of 2)

• Standard Functions and UDFs
• Aggregations (group by, sum, count)
• Complex Aggregations (windows, UDAFs)
• JOINs
• Optimizing Queries
• Accessing Spark Docs for Scala