Skip to content

KubernetesVolumeUtils

parseVolumesWithPrefix

parseVolumesWithPrefix(
  sparkConf: SparkConf,
  prefix: String): Seq[KubernetesVolumeSpec]

parseVolumesWithPrefix requests the SparkConf to get all properties with the given prefix.

parseVolumesWithPrefix extracts volume types and names (from the keys of the properties) and for every pair creates a KubernetesVolumeSpec with the following:

  • volumeName
  • mountPath based on [volumeType].[volumeName].mount.path key (in the properties)
  • mountSubPath based on [volumeType].[volumeName].mount.subPath key (in the properties) if available or defaults to an empty path
  • mountReadOnly based on [volumeType].[volumeName].mount.readOnly key (in the properties) if available or false
  • volumeConf with a KubernetesVolumeSpecificConf based on the properties and the volumeType and volumeName of the volume

parseVolumesWithPrefix is used when:

Extracting Volume Types and Names

getVolumeTypesAndNames(
  properties: Map[String, String]): Set[(String, String)]

getVolumeTypesAndNames splits the keys (in the given properties key-value collection) by . to a pair of a volume type and a name.

Extracting Volume Configuration

parseVolumeSpecificConf(
  options: Map[String, String],
  volumeType: String,
  volumeName: String): KubernetesVolumeSpecificConf

parseVolumeSpecificConf creates a KubernetesVolumeSpecificConf based on the given volumeType.

volumeType Keys
hostPath [volumeType].[volumeName].options.path
persistentVolumeClaim [volumeType].[volumeName].options.claimName
emptyDir [volumeType].[volumeName].options.medium
  [volumeType].[volumeName].options.sizeLimit

parseVolumeSpecificConf throws an IllegalArgumentException for unsupported volumeType:

Kubernetes Volume type `[volumeType]` is not supported

Last update: 2021-01-11