Skip to content

QueryStageExec Leaf Physical Operators

QueryStageExec is an extension of the LeafExecNode abstraction for leaf physical operators for Adaptive Query Execution.

Contract

Cancelling

cancel(): Unit

Cancels the stage materialization if in progress; otherwise does nothing.

Used when:

Materializing

doMaterialize(): Future[Any]

Used when:

Runtime Statistics

getRuntimeStatistics: Statistics

Statistics after stage materialization

Used when:

Unique ID

id: Int

Used when:

newReuseInstance

newReuseInstance(
  newStageId: Int,
  newOutput: Seq[Attribute]): QueryStageExec

Used when:

Physical Query Plan

plan: SparkPlan

The sub-tree of the main query plan of this query stage (that acts like a child operator, but QueryStageExec is a LeafExecNode and has no children)

Implementations

Result

_resultOption: AtomicReference[Option[Any]]

QueryStageExec uses a _resultOption transient volatile internal variable (of type AtomicReference) for the result of a successful materialization of the QueryStageExec operator (when preparing for query execution):

_resultOption is available using resultOption method. As AtomicReference is mutable that is enough to change the value.

_resultOption is set when AdaptiveSparkPlanExec physical operator is requested for the final physical plan.

resultOption is used when:

Computing Statistics

computeStats(): Option[Statistics]

computeStats uses the resultOption to access the underlying ReusedExchangeExec or Exchange physical operators.

If available, computeStats creates Statistics with the sizeInBytes as the dataSize performance metric of the physical operator.

Otherwise, computeStats returns no statistics.

computeStats is used when LogicalQueryStage logical operator is requested for the stats.

Materializing Query Stage

materialize(): Future[Any]

materialize prepares the operator for execution followed by doMaterialize.

Note

materialize is a final method and cannot be overriden.

materialize is used when AdaptiveSparkPlanExec physical operator is requested to getFinalPhysicalPlan.

Text Representation

generateTreeString(
  depth: Int,
  lastChildren: Seq[Boolean],
  append: String => Unit,
  verbose: Boolean,
  prefix: String = "",
  addSuffix: Boolean = false,
  maxFields: Int,
  printNodeId: Boolean,
  indent: Int = 0): Unit

generateTreeString is part of the TreeNode abstraction.

generateTreeString generateTreeString (the default) followed by another generateTreeString (with the depth incremented).


Last update: 2021-05-17
Back to top