Skip to content

BasicExecutorFeatureStep

BasicExecutorFeatureStep is a KubernetesFeatureConfigStep.

Creating Instance

BasicExecutorFeatureStep takes the following to be created:

BasicExecutorFeatureStep is created when:

Executor Resources

BasicExecutorFeatureStep creates an ExecutorResourcesOrDefaults when created (cf. ResourceProfile) with the following:

The ExecutorResourcesOrDefaults is used for the following:

Executor Container Image Name

BasicExecutorFeatureStep asserts that spark.kubernetes.executor.container.image configuration property is defined or throws a SparkException:

Must specify the executor container image

Executor Pod Name Prefix

BasicExecutorFeatureStep requests the KubernetesExecutorConf for the resourceNamePrefix.

Configuring Pod

configurePod(
  pod: SparkPod): SparkPod

configurePod builds the name of the pod (with the executorPodNamePrefix and the executorId of the KubernetesExecutorConf):

[executorPodNamePrefix]-exec-[executorId]

configurePod configures a PodBuilder and a ContainerBuilder, and in the end, creates a SparkPod.

configurePod is part of the KubernetesFeatureConfigStep abstraction.

buildExecutorResourcesQuantities

buildExecutorResourcesQuantities(
  customResources: Set[ExecutorResourceRequest]): Map[String, Quantity]

For every ExecutorResourceRequest (in the given customResources), buildExecutorResourcesQuantities builds a Kubernetes resource name (based on the vendor and the resourceName of the ExecutorResourceRequest) and maps it to the amount (of the ExecutorResourceRequest).

Back to top