Skip to content

BufferedRowIterator

BufferedRowIterator is an abstraction of iterators that are code-generated at execution time in Whole-Stage Java Code Generation (using WholeStageCodegenExec unary physical operator).

Contract

Initializing

void init(
  int index,
  Iterator<InternalRow>[] iters)

Used when WholeStageCodegenExec unary physical operator is executed

Processing Next Row

void processNext()

Used when BufferedRowIterator is requested to hasNext

hasNext

boolean hasNext()

hasNext processNext when there are no rows in the currentRows buffer.

hasNext is true when the currentRows buffer has got any element. Otherwise, hasNext is false.

hasNext is used when WholeStageCodegenExec unary physical operator is executed (to generate an RDD[InternalRow] to mapPartitionsWithIndex over rows from up to two input RDDs).

currentRows Buffer

LinkedList<InternalRow> currentRows

currentRows is an internal buffer of InternalRow:

currentRows is used for hasNext and shouldStop.

Appending Row

void append(
  InternalRow row)

append simply adds the given InternalRow to the end of the currentRows buffer (thus the name append).

append is used...FIXME


Last update: 2020-11-14