Skip to content

SQLMetric

SQLMetric is an AccumulatorV2 (Spark Core) for performance metrics of physical operators.

Note

Use Details for Query page in SQL tab in web UI to see the SQL execution metrics of a structured query.

Creating Instance

SQLMetric takes the following to be created:

SQLMetric is created using the specialized utilities:

createAverageMetric

createAverageMetric(
  sc: SparkContext,
  name: String): SQLMetric

createAverageMetric creates a SQLMetric with average type and registers it with the given name.

createMetric

createMetric(
  sc: SparkContext,
  name: String): SQLMetric

createMetric creates a SQLMetric with sum type and registers it with the given name.

createNanoTimingMetric

createNanoTimingMetric(
  sc: SparkContext,
  name: String): SQLMetric

createNanoTimingMetric creates a SQLMetric with nsTiming type and registers it with the given name.

createSizeMetric

createSizeMetric(
  sc: SparkContext,
  name: String): SQLMetric

createSizeMetric creates a SQLMetric with size type and registers it with the given name.

createTimingMetric

createTimingMetric(
  sc: SparkContext,
  name: String): SQLMetric

createTimingMetric creates a SQLMetric with timing type and registers it with the given name.

Metric Types

SQLMetric is given a metric type to be created:

  • average
  • nsTiming
  • size
  • sum
  • timing

Posting Driver-Side Metric Updates

postDriverMetricUpdates(
  sc: SparkContext,
  executionId: String,
  metrics: Seq[SQLMetric]): Unit

postDriverMetricUpdates posts a SparkListenerDriverAccumUpdates event to LiveListenerBus when executionId is specified.

postDriverMetricUpdates is used when:


Last update: 2020-11-29
Back to top