ExecutionListenerManager¶
ExecutionListenerManager
is the <QueryExecutionListeners
that listen for execution metrics:
-
Name of the action (that triggered a query execution)
-
Execution time of this query (in nanoseconds)
ExecutionListenerManager
is available as SparkSession.md#listenerManager[listenerManager] property of SparkSession
(and SessionState.md#listenerManager[listenerManager] property of SessionState
).
[source, scala]¶
scala> :type spark.listenerManager org.apache.spark.sql.util.ExecutionListenerManager
scala> :type spark.sessionState.listenerManager org.apache.spark.sql.util.ExecutionListenerManager
[[conf]] [[creating-instance]] ExecutionListenerManager
takes a single SparkConf
when created
While <ExecutionListenerManager
reads StaticSQLConf.md#spark.sql.queryExecutionListeners[spark.sql.queryExecutionListeners] configuration property with QueryExecutionListeners
and <
[[spark.sql.queryExecutionListeners]] ExecutionListenerManager
uses StaticSQLConf.md#spark.sql.queryExecutionListeners[spark.sql.queryExecutionListeners] configuration property as the list of QueryExecutionListeners
that should be automatically added to newly created sessions (and registers them while <
[[methods]] .ExecutionListenerManager's Public Methods [cols="1,2",options="header",width="100%"] |=== | Method | Description
| <
[source, scala]¶
register(listener: QueryExecutionListener): Unit¶
| <
[source, scala]¶
unregister(listener: QueryExecutionListener): Unit¶
| <
[source, scala]¶
clear(): Unit¶
|===
ExecutionListenerManager
is <BaseSessionStateBuilder
is requested for BaseSessionStateBuilder.md#listenerManager[ExecutionListenerManager] (while SessionState
is BaseSessionStateBuilder.md#build[built]).
[[listeners]] ExecutionListenerManager
uses listeners
internal registry for registered QueryExecutionListeners.
onSuccess Internal Method¶
onSuccess(
funcName: String,
qe: QueryExecution,
duration: Long): Unit
onSuccess
...FIXME
onSuccess
is used when:
-
DataFrameWriter
is requested to run a logical command (after it has finished with no exceptions) -
Dataset
is requested to withAction
onFailure Internal Method¶
onFailure(
funcName: String,
qe: QueryExecution,
exception: Exception): Unit
onFailure
...FIXME
onFailure
is used when:
-
DataFrameWriter
is requested to run a logical command (after it has reported an exception) -
Dataset
is requested to withAction
=== [[withErrorHandling]] withErrorHandling
Internal Method
[source, scala]¶
withErrorHandling(f: QueryExecutionListener => Unit): Unit¶
withErrorHandling
...FIXME
NOTE: withErrorHandling
is used when ExecutionListenerManager
is requested to <
=== [[register]] Registering QueryExecutionListener -- register
Method
[source, scala]¶
register(listener: QueryExecutionListener): Unit¶
Internally, register
simply registers (adds) the input QueryExecutionListener to the listeners internal registry.