Skip to content

MountVolumesFeatureStep

MountVolumesFeatureStep is a KubernetesFeatureConfigStep.

Creating Instance

MountVolumesFeatureStep takes the following to be created:

MountVolumesFeatureStep is created when:

Configuring Driver Pod

configurePod(
  pod: SparkPod): SparkPod

configurePod is part of the KubernetesFeatureConfigStep abstraction.

configurePod constructs the volumes and volume mounts from the volumes (of the KubernetesConf) and creates a new SparkPod:

  • Adds the volumes to the pod specification

  • Adds the volume mounts to the container specification

constructVolumes

constructVolumes(
  volumeSpecs: Iterable[KubernetesVolumeSpec]): Iterable[(VolumeMount, Volume)]

constructVolumes creates Kubernetes VolumeMounts and Volumes for the given KubernetesVolumeSpecs.

VolumeMounts are built based on the following (4 of 5 properties):

Volumes are build based on the type of the volume (the remaining 5th property).

In the end, Volumes and VolumeMounts are wired together using volumeName.

Claim Name Placeholders

MountVolumesFeatureStep defines OnDemand and SPARK_EXECUTOR_ID as placeholders for runtime-replaceable parts of the claim name of a KubernetesPVCVolumeConf.

These placeholders allow for templating claim names to include parts to be replaced at deployment.

When constructVolumes MountVolumesFeatureStep replaces all OnDemands with the following (using resourceNamePrefix and executorId of the KubernetesConf):

Pod Replacement
driver [resourceNamePrefix]-exec-[executorId]-pvc-[volumeIndex]
executors [resourceNamePrefix]-driver-pvc-[volumeIndex]

When constructVolumes MountVolumesFeatureStep replaces all SPARK_EXECUTOR_IDs with executorId of the KubernetesConf.


Last update: 2021-03-16