KubernetesClientApplication¶

KubernetesClientApplication is a SparkApplication (Apache Spark) in Spark on Kubernetes in cluster deploy mode.

Creating Instance¶

KubernetesClientApplication takes no arguments to be created.

KubernetesClientApplication is created when:

• SparkSubmit is requested to launch a Spark application (for kubernetes in cluster deploy mode)

Starting Spark Application¶

start(
  args: Array[String],
  conf: SparkConf): Unit

start is part of the SparkApplication (Apache Spark) abstraction.

start parses the command-line arguments (args) and runs.

run¶

run(
  clientArguments: ClientArguments,
  sparkConf: SparkConf): Unit

run generates a custom Spark Application ID of the format:

spark-[randomUUID-without-dashes]

run creates a KubernetesDriverConf (with the given ClientArguments, SparkConf and the custom Spark Application ID).

run removes the k8s:// prefix from the spark.master configuration property (which has already been validated by SparkSubmit itself).

run creates a LoggingPodStatusWatcherImpl (with the KubernetesDriverConf).

run creates a KubernetesClient (with the master URL, the namespace, and others).

In the end, run creates a Client (with the KubernetesDriverConf, a new KubernetesDriverBuilder, the KubernetesClient, and the LoggingPodStatusWatcherImpl) and requests it to run.