Skip to content

KubernetesDriverBuilder

KubernetesDriverBuilder is used to build a specification of a driver pod (for a Spark application deployed in cluster deploy mode).

KubernetesDriverBuilder, Client and KubernetesClientApplication

Creating Instance

KubernetesDriverBuilder takes no arguments to be created.

KubernetesDriverBuilder is created when:

  • KubernetesClientApplication is requested to start

KubernetesDriverSpec

KubernetesDriverSpec is the following:

  • SparkPod
  • Driver Resources
  • System Properties

Building Pod Spec for Driver

buildFromFeatures(
  conf: KubernetesDriverConf,
  client: KubernetesClient): KubernetesDriverSpec

buildFromFeatures creates an initial driver pod specification.

With spark.kubernetes.driver.podTemplateFile configuration property defined, buildFromFeatures loads it (with the given KubernetesClient and the container name based on spark.kubernetes.driver.podTemplateContainerName configuration property) or defaults to an empty pod specification.

buildFromFeatures builds a KubernetesDriverSpec (with the initial driver pod specification).

In the end, buildFromFeatures configures the driver pod specification (with additional system properties and additional resources) through a series of the feature steps:

buildFromFeatures is used when:

  • Client is requested to run

Last update: 2021-01-27