Skip to content

ExecutorPodsWatchSnapshotSource

Creating Instance

ExecutorPodsWatchSnapshotSource takes the following to be created:

ExecutorPodsWatchSnapshotSource is created when:

Starting

start(
  applicationId: String): Unit

start prints out the following DEBUG message to the logs:

Starting watch for pods with labels spark-app-selector=[applicationId], spark-role=executor.

start requests the KubernetesClient to watch pods with the following labels using ExecutorPodsWatcher:

  • spark-app-selector with the given applicationId
  • spark-role as executor

start is used when:

  • KubernetesClusterSchedulerBackend is requested to start

Logging

Enable ALL logging level for org.apache.spark.scheduler.cluster.k8s.ExecutorPodsWatchSnapshotSource logger to see what happens inside.

Add the following line to conf/log4j.properties:

log4j.logger.org.apache.spark.scheduler.cluster.k8s.ExecutorPodsWatchSnapshotSource=ALL

Refer to Logging.


Last update: 2021-01-10