Skip to content

PollRunnable¶

PollRunnable is a Java Runnable that ExecutorPodsPollingSnapshotSource schedules for full executor pod state snapshots from Kubernetes every spark.kubernetes.executor.apiPollingInterval in the Spark application.

PollRunnable is an internal class of ExecutorPodsPollingSnapshotSource with full access to its internals.

Creating Instance¶

PollRunnable takes the following to be created:

Application Id

PollRunnable is created when:

ExecutorPodsPollingSnapshotSource is requested to start

Starting Thread¶

run(): Unit

run prints out the following DEBUG message to the logs:

Resynchronizing full executor pod state from Kubernetes.

run requests the KubernetesClient for Spark executor pods with the following labels and values.

Label Name	Value
`spark-app-selector`	application Id
`spark-role`	`executor`
spark-exec-inactive	any value but `true`

In the end, run requests the ExecutorPodsSnapshotsStore to replace the snapshot.

Logging¶

PollRunnable uses org.apache.spark.scheduler.cluster.k8s.ExecutorPodsPollingSnapshotSource logger for logging.