ExecutorPodsSnapshotsStore¶

ExecutorPodsSnapshotsStore is an abstraction of executor pods snapshots stores that subscribers can subscribe to and be notified about single pod or full snapshots updates.

Contract¶

Registering Subscriber¶

addSubscriber(
  processBatchIntervalMillis: Long)(
  onNewSnapshots: Seq[ExecutorPodsSnapshot] => Unit): Unit

Registers a new subscriber to be notified about new ExecutorPodsSnapshots every processBatchIntervalMillis interval

Used when:

• ExecutorPodsAllocator is requested to start
• ExecutorPodsLifecycleManager is requested to start

Notifying Subscribers¶

notifySubscribers(): Unit

Used when:

• ExecutorPodsAllocator is requested to change the total expected executors

Full Executor Pod State Synchronization¶

replaceSnapshot(
  newSnapshot: Seq[Pod]): Unit

Used when:

• PollRunnable is requested to start (every spark.kubernetes.executor.apiPollingInterval when ExecutorPodsPollingSnapshotSource is started)

Stopping¶

stop(): Unit

Used when:

• KubernetesClusterSchedulerBackend is requested to stop

Single Executor Pod State Update¶

updatePod(
  updatedPod: Pod): Unit

Used when:

• ExecutorPodsWatcher is requested to eventReceived

Implementations¶

• ExecutorPodsSnapshotsStoreImpl