Skip to content


ExecutionListenerManager is the <> for QueryExecutionListeners that listen for execution metrics:

  • Name of the action (that triggered a query execution)

  • QueryExecution

  • Execution time of this query (in nanoseconds)

ExecutionListenerManager is available as[listenerManager] property of SparkSession (and[listenerManager] property of SessionState).

[source, scala]

scala> :type spark.listenerManager org.apache.spark.sql.util.ExecutionListenerManager

scala> :type spark.sessionState.listenerManager org.apache.spark.sql.util.ExecutionListenerManager

[[conf]] [[creating-instance]] ExecutionListenerManager takes a single SparkConf when created

While <>, ExecutionListenerManager reads[spark.sql.queryExecutionListeners] configuration property with QueryExecutionListeners and <> them.

[[spark.sql.queryExecutionListeners]] ExecutionListenerManager uses[spark.sql.queryExecutionListeners] configuration property as the list of QueryExecutionListeners that should be automatically added to newly created sessions (and registers them while <>).

[[methods]] .ExecutionListenerManager's Public Methods [cols="1,2",options="header",width="100%"] |=== | Method | Description

| <> a|

[source, scala]

register(listener: QueryExecutionListener): Unit

| <> a|

[source, scala]

unregister(listener: QueryExecutionListener): Unit

| <> a|

[source, scala]

clear(): Unit


ExecutionListenerManager is <> exclusively when BaseSessionStateBuilder is requested for[ExecutionListenerManager] (while SessionState is[built]).

[[listeners]] ExecutionListenerManager uses listeners internal registry for registered QueryExecutionListeners.

onSuccess Internal Method

  funcName: String,
  qe: QueryExecution,
  duration: Long): Unit


onSuccess is used when:

onFailure Internal Method

  funcName: String,
  qe: QueryExecution,
  exception: Exception): Unit


onFailure is used when:

=== [[withErrorHandling]] withErrorHandling Internal Method

[source, scala]

withErrorHandling(f: QueryExecutionListener => Unit): Unit


NOTE: withErrorHandling is used when ExecutionListenerManager is requested to <> and <>.

=== [[register]] Registering QueryExecutionListener -- register Method

[source, scala]

register(listener: QueryExecutionListener): Unit

Internally, register simply registers (adds) the input QueryExecutionListener to the listeners internal registry.

Last update: 2020-11-25