Skip to content

K8SSparkSubmitOperation

K8SSparkSubmitOperation is a SparkSubmitOperation (Apache Spark).

Killing Submission

kill(
  submissionId: String,
  conf: SparkConf): Unit

kill is part of the SparkSubmitOperation (Apache Spark) abstraction.

kill prints out the following message to standard error:

Submitting a request to kill submission [submissionId] in [spark.master]. Grace period in secs: [[getGracePeriod] | not set].

kill creates a KillApplication to execute it (with the input submissionId and SparkConf).

Displaying Submission Status

printSubmissionStatus(
  submissionId: String,
  conf: SparkConf): Unit

printSubmissionStatus is part of the SparkSubmitOperation (Apache Spark) abstraction.

printSubmissionStatus prints out the following message to standard error:

Submitting a request for the status of submission [submissionId] in [spark.master].

printSubmissionStatus creates a ListStatus to execute it (with the input submissionId and SparkConf).

Checking Whether Master URL Supported

supports(
  master: String): Boolean

supports is part of the SparkSubmitOperation (Apache Spark) abstraction.

supports is true when the input master starts with k8s:// prefix.

Executing Operation

execute(
  submissionId: String,
  sparkConf: SparkConf,
  op: K8sSubmitOp): Unit

execute...FIXME

execute is used for kill and printSubmissionStatus.


Last update: 2021-01-10