Apache Spark 2
Ecosystem 5 Days (beta^2)
@jaceklaskowski / StackOverflow / GitHub / Mastering Apache Spark 2
- Jacek Laskowski is an independent consultant
- Specializing in Spark, Kafka, Mesos, DC/OS, Scala
- Among contributors to Spark 2.1, Spark 2.0 (since 1.6.0)
- Contact me at jacek@japila.pl or follow me on twitter at @JacekLaskowski <-- and learn more Spark!
- Delivering Development Services | Consulting | Training
- Leader of Warsaw Scala Enthusiasts and Warsaw Apache Spark
- Java Champion
https://github.com/jaceklaskowski
https://bit.ly/mastering-apache-spark
Among contributors to Apache Spark 1.6
Among contributors to Apache Spark 2
Among contributors to Apache Spark 2.1
Ranked #96 in Spark contributors
http://stackoverflow.com/users/1305344/jacek-laskowski
https://twitter.com/jaceklaskowski
Agenda - Day 1
- The Elements of Apache Spark (aka Why Spark)
-
My First Spark Application
- IntelliJ IDEA, sbt and spark-submit
- Deploying Spark Applications to a Cluster
- Spark Standalone
- Hadoop YARN
- Apache Mesos
Agenda - Day 2-5
- Apache Spark 2
- Spark SQL (incl. Structured Streaming)
- Spark MLlib
- Spark Streaming
- DC/OS & Apache Mesos
- Apache Kafka
- PostgreSQL
- Spark JobServer
- Hadoop YARN
Prerequisities
- Some programming experience using modern programming language (preferably on JVM)
- Scala, Python, Java, F#
- Installed
- Downloaded
- Apache Spark
- PostgreSQL
- Apache Hadoop 2.7.3 (not
Hadoop 3.0.0-alpha2) - Apache Kafka
- Willingness to ask PLENTY of questions
Questions?
- Read Mastering Apache Spark 2
- https://bit.ly/mastering-apache-spark
- Follow @jaceklaskowski on twitter
- Upvote my activities on StackOverflow
- Use Jacek's code at GitHub
- Read Jacek Laskowski @ Medium
- Visit Jacek Laskowski's blog