DriverCommandFeatureStep¶

DriverCommandFeatureStep is a KubernetesFeatureConfigStep.

Creating Instance¶

DriverCommandFeatureStep takes the following to be created:

• KubernetesDriverConf

DriverCommandFeatureStep is created when:

• KubernetesDriverBuilder is requested to build a KubernetesDriverSpec from features

Configuring Pod¶

configurePod(
  pod: SparkPod): SparkPod

configurePod is part of the KubernetesFeatureConfigStep abstraction.

configurePod branches off based on the MainAppResource (of the KubernetesDriverConf):

• For JavaMainAppResource, configurePod configures a pod for Java with the primary resource (if defined) or uses spark-internal special value

• For PythonMainAppResource, configurePod configures a pod for Python with the primary resource

• For RMainAppResource, configurePod configures a pod for R with the primary resource

Configuring Pod for Java Application¶

configureForJava(
  pod: SparkPod,
  res: String): SparkPod

configureForJava builds the base driver container for the given SparkPod and the primary resource.

In the end, configureForJava creates another SparkPod (for the pod of the given SparkPod) and the driver container.

Configuring Pod for Python Application¶

configureForPython(
  pod: SparkPod,
  res: String): SparkPod

configureForPython...FIXME

Configuring Pod for R Application¶

configureForR(
  pod: SparkPod,
  res: String): SparkPod

configureForR...FIXME

Base Driver ContainerBuilder¶

baseDriverContainer(
  pod: SparkPod,
  resource: String): ContainerBuilder

baseDriverContainer renames the given primary resource if the MainAppResource is a JavaMainAppResource. Otherwise, baseDriverContainer leaves the primary resource as-is.

baseDriverContainer creates a ContainerBuilder (for the pod of the given SparkPod) and adds the following arguments (in that order):

1. driver
2. --properties-file with /opt/spark/conf/spark.properties
3. --class with the mainClass of the KubernetesDriverConf
4. the primary resource (possibly renamed if a MainAppResource)
5. appArgs of the KubernetesDriverConf

kubectl get po [driverPod] -o=jsonpath='{.spec.containers[0].args}'