Day 6 / Apr 11 (Mon)¶

Welcome to Week 7¶

1. 18.04 (Mon) is a day-off
2. We're back in the office on 19.04 (Tue)
3. You, during 14-15.04 (Thu and Fri) work on a 2-day-long Scala assigment
   • Everybody works alone (but you can team up, too)
4. During that time I do code review of all your projects on GitHub
   • Sending you feedback in a shared Google Docs

Introduction: Spark Structured Streaming¶

Next 3 days are about Spark Structured Streaming with Scala and SQL.

1. We know Scala and Spark SQL already
2. SQL is yet to be learnt

Morning Exercise / Code Review¶

1. Select one or more solutions for a code review
   1. Clone the repo
   2. Execute tests (sbt test)
   3. Import the project to IntelliJ IDEA
   4. Take notes about interesting parts of the code
   5. Be ready with questions to the author
   6. (advanced) If you find a bug, write a failing test
2. Time: 45 mins

Self-Study: Reading Official Documentation¶

1. Structured Streaming Programming Guide
2. Scala examples

Demo: Writing Your First Spark Structured Streaming Application¶

Based on Quick Example.

Resources¶

1. Spark: The Definitive Guide
2. Learning Spark, 2^nd Ed