Skip to content

LoggingPodStatusWatcherImpl

LoggingPodStatusWatcherImpl is a LoggingPodStatusWatcher that monitors and logs the application status.

Creating Instance

LoggingPodStatusWatcherImpl takes the following to be created:

LoggingPodStatusWatcherImpl is created when:

  • KubernetesClientApplication is requested to start

watchOrStop

watchOrStop(
  sId: String): Unit

watchOrStop is part of the LoggingPodStatusWatcher abstraction.

watchOrStop uses spark.kubernetes.submission.waitAppCompletion configuration property to control whether to wait for the Spark application to complete (true) or merely print out the following INFO message to the logs:

Deployed Spark application [appName] with submission ID [sId] into Kubernetes

While waiting for the Spark application to complete, watchOrStop prints out the following INFO message to the logs:

Waiting for application [appName] with submission ID [sId] to finish...

Until podCompleted flag is true, watchOrStop waits spark.kubernetes.report.interval configuration property and prints out the following INFO message to the logs:

Application status for [appId] (phase: [phase])

Once podCompleted flag is true, watchOrStop prints out the following INFO messages to the logs:

Container final statuses:

[containersDescription]
Application [appName] with submission ID [sId] finished

When no pod is available, watchOrStop prints out the following INFO message to the logs:

No containers were found in the driver pod.

eventReceived

eventReceived(
  action: Action,
  pod: Pod): Unit

eventReceived is part of the Kubernetes' Watcher abstraction.

eventReceived brances off based on the given Action:

logLongStatus

logLongStatus(): Unit

logLongStatus prints out the following INFO message to the logs:

State changed, new state: [formatPodState|unknown]

hasCompleted

hasCompleted(): Boolean

hasCompleted is true when the phase is Succeeded or Failed.

hasCompleted is used when:

  • LoggingPodStatusWatcherImpl is requested to eventReceived (when an action is neither DELETED nor ERROR)

podCompleted Flag

LoggingPodStatusWatcherImpl turns podCompleted off when created.

Until podCompleted is on, LoggingPodStatusWatcherImpl waits the spark.kubernetes.report.interval configuration property and prints out the following INFO message to the logs:

Application status for [appId] (phase: [phase])

podCompleted turns podCompleted on when closeWatch.

closeWatch

closeWatch(): Unit

closeWatch turns podCompleted on.

closeWatch is used when:

Logging

Enable ALL logging level for org.apache.spark.deploy.k8s.submit.LoggingPodStatusWatcherImpl logger to see what happens inside.

Add the following line to conf/log4j.properties:

log4j.logger.org.apache.spark.deploy.k8s.submit.LoggingPodStatusWatcherImpl=ALL

Refer to Logging.


Last update: 2021-01-09