KubernetesExecutorBuilder¶

KubernetesExecutorBuilder is used to build a pod spec of executors (when ExecutorPodsAllocator is requested to handle executor pods snapshots and finds executors to be requested from Kubernetes).

Creating Instance¶

KubernetesExecutorBuilder takes no arguments to be created (and could really be a Scala utility object).

KubernetesExecutorBuilder is created when:

• KubernetesClusterManager is requested to create a SchedulerBackend (and creates a ExecutorPodsAllocator)

Building Pod Spec for Executors¶

buildFromFeatures(
  conf: KubernetesExecutorConf,
  secMgr: SecurityManager,
  client: KubernetesClient): SparkPod

When defined, buildFromFeatures loads the pod spec from the pod template file (based on the spark.kubernetes.executor.podTemplateFile and spark.kubernetes.executor.podTemplateContainerName configuration properties). Otherwise, buildFromFeatures starts from an initial empty pod specification.

In the end, buildFromFeatures configures the executor pod specification through a series of the feature steps:

• BasicExecutorFeatureStep
• ExecutorKubernetesCredentialsFeatureStep
• MountSecretsFeatureStep
• EnvSecretsFeatureStep
• MountVolumesFeatureStep
• LocalDirsFeatureStep

buildFromFeatures is used when:

• ExecutorPodsAllocator is requested to handle executor pods snapshots (and requests missing executors from Kubernetes)