Skip to content

ExecutionListenerManager

ExecutionListenerManager is the <> for QueryExecutionListeners that listen for execution metrics:

  • Name of the action (that triggered a query execution)

  • QueryExecution

  • Execution time of this query (in nanoseconds)

ExecutionListenerManager is available as SparkSession.md#listenerManager[listenerManager] property of SparkSession (and SessionState.md#listenerManager[listenerManager] property of SessionState).

[source, scala]

scala> :type spark.listenerManager org.apache.spark.sql.util.ExecutionListenerManager

scala> :type spark.sessionState.listenerManager org.apache.spark.sql.util.ExecutionListenerManager


[[conf]] [[creating-instance]] ExecutionListenerManager takes a single SparkConf when created

While <>, ExecutionListenerManager reads StaticSQLConf.md#spark.sql.queryExecutionListeners[spark.sql.queryExecutionListeners] configuration property with QueryExecutionListeners and <> them.

[[spark.sql.queryExecutionListeners]] ExecutionListenerManager uses StaticSQLConf.md#spark.sql.queryExecutionListeners[spark.sql.queryExecutionListeners] configuration property as the list of QueryExecutionListeners that should be automatically added to newly created sessions (and registers them while <>).

[[methods]] .ExecutionListenerManager's Public Methods [cols="1,2",options="header",width="100%"] |=== | Method | Description

| <> a|

[source, scala]

register(listener: QueryExecutionListener): Unit

| <> a|

[source, scala]

unregister(listener: QueryExecutionListener): Unit

| <> a|

[source, scala]

clear(): Unit

|===

ExecutionListenerManager is <> exclusively when BaseSessionStateBuilder is requested for BaseSessionStateBuilder.md#listenerManager[ExecutionListenerManager] (while SessionState is BaseSessionStateBuilder.md#build[built]).

[[listeners]] ExecutionListenerManager uses listeners internal registry for registered QueryExecutionListeners.

onSuccess Internal Method

onSuccess(
  funcName: String,
  qe: QueryExecution,
  duration: Long): Unit

onSuccess...FIXME

onSuccess is used when:

onFailure Internal Method

onFailure(
  funcName: String,
  qe: QueryExecution,
  exception: Exception): Unit

onFailure...FIXME

onFailure is used when:

=== [[withErrorHandling]] withErrorHandling Internal Method

[source, scala]

withErrorHandling(f: QueryExecutionListener => Unit): Unit

withErrorHandling...FIXME

NOTE: withErrorHandling is used when ExecutionListenerManager is requested to <> and <>.

=== [[register]] Registering QueryExecutionListener -- register Method

[source, scala]

register(listener: QueryExecutionListener): Unit

Internally, register simply registers (adds) the input QueryExecutionListener to the listeners internal registry.


Last update: 2020-11-25