ExecutorPodsWatchSnapshotSource is given KubernetesClient to a Kubernetes API server to watch for status updates of the executor pods of a given Spark application when started (that ExecutorPodsWatcher passes along to the ExecutorPodsSnapshotsStore).
ExecutorPodsWatchSnapshotSource takes the following to be created:
ExecutorPodsWatchSnapshotSource is created when:
KubernetesClusterManageris requested for a SchedulerBackend
watchConnection internal registry to be a "watch connection" to a Kubernetes API server to watch any status updates of the executor pods of a given Spark application (using ExecutorPodsWatcher).
watchConnection internal registry as an indication of whether it has been started already or not (and throws an
IllegalArgumentException when it has).
ExecutorPodsWatchSnapshotSource requests the
watchConnection to close and
nulls it when requested to stop.
start( applicationId: String): Unit
start prints out the following DEBUG message to the logs:
Starting watch for pods with labels spark-app-selector=[applicationId], spark-role=executor.
| ||the given |
| || |
start is used when:
KubernetesClusterSchedulerBackendis requested to start
ALL logging level for
org.apache.spark.scheduler.cluster.k8s.ExecutorPodsWatchSnapshotSource logger to see what happens inside.
Add the following line to
Refer to Logging.