Apache Spark 2 Workshop 4 days
@jaceklaskowski / StackOverflow / GitHub
Gitbooks: Mastering Apache Spark / Spark Structured Streaming
- Jacek Laskowski is an independent consultant
- Specializing in Spark, Kafka, Mesos, DC/OS, Scala
- Among contributors to Spark 2.1, Spark 2.0 (since 1.6.0)
- Contact me at jacek@japila.pl or follow me on twitter at @JacekLaskowski <-- and learn more Spark!
- Delivering Development Services | Consulting | Training
- Leader of Warsaw Scala Enthusiasts and Warsaw Apache Spark
- Java Champion
https://github.com/jaceklaskowski
https://bit.ly/mastering-apache-spark
Ranked #91 in Spark contributors
https://stackoverflow.com/tags/apache-spark/topusers
https://twitter.com/jaceklaskowski
Agenda
- Day 1a — Just Enough Scala and Tools (sbt, IntelliJ IDEA)
- Day 1b — Spark SQL and Tools (spark-shell, Databricks)
- Day 2 — Advanced Spark SQL
- Day 3a — Spark Architecture and Core Concepts
- Day 3b — Monitoring using web UI
- Day 3c — Spark Streaming + Kafka
- Day 4a — Structured Streaming (Kafka, files)
- Day 4b — Spark MLlib
Mastering Apache Spark notebook featured in the Big Data course at Coursera!
Prerequisities (1 of 3)
- Some programming experience using modern programming language, e.g. Scala, Python, Java, F#
Prerequisities (2 of 3)
- Installed
- Java Platform, Standard Edition (Java SE) 8
- IntelliJ IDEA Community Edition with Scala plugin
- sbt
- PostgreSQL
- Apache Kafka
- Apache Hadoop 2.7.3 (not
Hadoop 3.0.0-alpha2)
Prerequisities (3 of 3)
- Downloaded
Questions?
- Read Mastering Apache Spark 2
- Follow @jaceklaskowski on twitter
- Upvote my questions and answers on StackOverflow
- Use Jacek's code at GitHub
- Read blog posts on Medium
- Upvote my answers on Quora
- Connect on LinkedIn
- Visit Jacek Laskowski's blog