Mastering Apache Spark 2

https://bit.ly/mastering-apache-spark

Spark Structured Streaming

https://bit.ly/spark-structured-streaming

Gitbooks in Mobile Formats

Mobile formats available (PDF, Mobi, ePub) DRM-free

StackOverflow

StackOverflow

Agenda / Spark SQL

---

  Basic Aggregation

  Windowed Aggregation

  Multi-Dimensional Aggregation

...and 20 mins of our time went away

Agenda / From Batch to Streaming

---

  Spark Structured Streaming's Internals

...and 25 mins of our time went away

Agenda / Spark Structured Streaming Spark Streams

---

  Streaming Dataset API for aggregates

  groupBy Operator — Untyped Streaming Aggregation

  Demo: groupBy Streaming Aggregation with Append Output Mode

  groupByKey Operator — Streaming Aggregation

  window Function — Stream Time Windows

  withWatermark Operator — Event Time Watermark

  mapGroupsWithState Operator — Stateful Streaming Aggregation (with Explicit State Logic)

  flatMapGroupsWithState Operator — Arbitrary Stateful Streaming Aggregation (with Explicit State Logic)

...and 40 mins of our time went away