Skip to content

QueryStageExec Leaf Physical Operators

QueryStageExec is an extension of the LeafExecNode abstraction for query stage operators.



cancel(): Unit

Cancels the stage materialization if in progress; otherwise does nothing.

Used when AdaptiveSparkPlanExec physical operator is requested to cleanUpAndThrowException


doMaterialize(): Future[Any]

Used when QueryStageExec is requested to materialize


id: Int

Unique ID

Used when CoalesceShufflePartitions adaptive physical optimization is executed


  newStageId: Int,
  newOutput: Seq[Attribute]): QueryStageExec

Used when AdaptiveSparkPlanExec physical operator is requested to reuseQueryStage


plan: SparkPlan

"Child" physical operator (but QueryStageExec is a LeafExecNode and has got no children)

Used when...FIXME



  depth: Int,
  lastChildren: Seq[Boolean],
  append: String => Unit,
  verbose: Boolean,
  prefix: String = "",
  addSuffix: Boolean = false,
  maxFields: Int,
  printNodeId: Boolean): Unit


generateTreeString is part of the TreeNode abstraction.

resultOption Internal Registry

resultOption: Option[Any] = None

resultOption is the result of materializing the QueryStageExec operator:

resultOption is None by default.

resultOption is set a value when AdaptiveSparkPlanExec physical operator is requested for the final physical query plan

resultOption is used when:

Computing Statistics

computeStats(): Option[Statistics]

computeStats uses the resultOption to access the underlying ReusedExchangeExec or Exchange physical operators.

If available, computeStats creates Statistics with the sizeInBytes as the dataSize performance metric of the physical operator.

Otherwise, computeStats returns no statistics.

computeStats is used when LogicalQueryStage logical operator is requested for the stats.

Materializing Query Stage

materialize(): Future[Any]

materialize prepares the query stage operator for execution followed by doMaterialize.


materialize is a final method and cannot be overriden.

materialize is used when AdaptiveSparkPlanExec physical operator is requested to getFinalPhysicalPlan.

Last update: 2020-09-07