Skip to content

ShuffleExchangeLike Physical Operators

ShuffleExchangeLike is an extension of the Exchange abstraction for physical operators that...FIXME

Contract

getShuffleRDD

getShuffleRDD(
  partitionSpecs: Array[ShufflePartitionSpec]): RDD[_]

RDD (Spark Core)

Used when:

  • CustomShuffleReaderExec physical operator is requested for the shuffleRDD

mapOutputStatisticsFuture

mapOutputStatisticsFuture: Future[MapOutputStatistics]

Used when:

numMappers

numMappers: Int

Used when:

Number of Partitions

numPartitions: Int

Used when:

runtimeStatistics

runtimeStatistics: Statistics

Used when:

ShuffleOrigin

shuffleOrigin: ShuffleOrigin

ShuffleOrigin

Used when:

Implementations

Submitting Shuffle Job

submitShuffleJob: Future[MapOutputStatistics]

submitShuffleJob executes a query with the mapOutputStatisticsFuture.

Final Method

submitShuffleJob is a Scala final method and may not be overridden in subclasses.

Learn more in the Scala Language Specification.

submitShuffleJob is used when:

  • ShuffleQueryStageExec adaptive leaf physical operator is requested for the shuffleFuture
Back to top