Fundamentals of Spark SQL Application Development¶

Development of a Spark SQL application requires the following steps:

1. Setting up Development Environment (IntelliJ IDEA, Scala and sbt)
2. Specifying Library Dependencies
3. Creating SparkSession
4. Loading Data from Data Sources
5. Processing Data Using Dataset API
6. Saving Data to Persistent Storage
7. Deploying Spark Application to Cluster (using spark-submit)