CollectMetricsExec Physical Operator¶
CollectMetricsExec is a unary physical operator.
CollectMetricsExec takes the following to be created:
Collected metrics Accumulator¶
CollectMetricsExec registers an
AggregatingAccumulator accumulator under the name Collected metrics.
Executing Physical Operator¶
doExecute resets the Collected metrics Accumulator.
- A new per-partition
updater) is requested to
- The value of the accumulator is published only when a task is completed
- For every row, the per-partition
AggregatingAccumulatoris requested to add it (that updates ImperativeAggregates and TypedImperativeAggregates)
doExecute is part of the SparkPlan abstraction.