Agenda

1. Spark Streaming
2. StreamingContext
3. Stream Operators
4. Web UI

Heads-up

Spark Streaming had almost no notable changes between Spark 1.6 and Spark 2.0. Jacek believes that it will marked deprecated soon and replaced by the modern Structured Streaming.

Spark Streaming

1. Spark Streaming is the incremental stream processing framework for Apache Spark.
2. Switch to Mastering Apache Spark 2
   • Spark Streaming

StreamingContext

1. StreamingContext is the entry point for all Spark Streaming functionality.

   
           val ssc = new StreamingContext(sc, Seconds(5))
                 

2. Switch to Mastering Apache Spark 2
   • StreamingContext — The Entry Point to Spark Streaming

Stream Operators

1. Stream operators allow for transformations to the records from input DStreams and ultimately trigger computations using output operators.
2. Switch to Mastering Apache Spark 2
   • Stream Operators

Web UI

1. Streaming applications have their own web UI with Streaming Statistics Page
2. Switch to Mastering Apache Spark 2
   • web UI and Streaming Statistics Page

Recap

1. Spark Streaming
2. StreamingContext
3. Stream Operators
4. Web UI

Questions?

• Read Mastering Apache Spark 2
  • https://bit.ly/mastering-apache-spark
• Follow @jaceklaskowski on twitter
• Upvote my questions and answers on StackOverflow
• Use Jacek's code at GitHub
• Read Jacek Laskowski @ Medium
• Visit Jacek Laskowski's blog