spark-workshop

Exercise: Using foreach Operator (and ForeachWriter)

Module: Spark Structured Streaming

Duration: 30 mins

Steps

1. Develop a standalone Spark SQL application
   • Use IntelliJ IDEA
2. Read datasets from any source (e.g. rate, kafka, csv, socket)
3. Use foreach operator on streaming Dataset to process a data stream
   • Use DataStreamWriter and ForeachWriter scaladocs
4. Use sbt package and spark-submit

This site is open source. Improve this page.