Skip to content

AggregationIterator -- Generic Iterator of UnsafeRows for Aggregate Physical Operators

AggregationIterator is the base for <> of <> that...FIXME

Iterators are data structures that allow to iterate over a sequence of elements. They have a hasNext method for checking if there is a next element available, and a next method which returns the next element and discards it from the iterator.

[[implementations]] .AggregationIterator's Implementations [width="100%",cols="1,2",options="header"] |=== | Name | Description

| <> | Used when ObjectHashAggregateExec physical operator is executed

| <> | Used exclusively when SortAggregateExec physical operator is SortAggregateExec.md#doExecute[executed].

| TungstenAggregationIterator.md[TungstenAggregationIterator] a| Used when HashAggregateExec physical operator is executed

|===

[[internal-registries]] .AggregationIterator's Internal Registries and Counters [cols="1,2",options="header",width="100%"] |=== | Name | Description

| [[aggregateFunctions]] aggregateFunctions | spark-sql-Expression-AggregateFunction.md[Aggregate functions]

Used when...FIXME

| [[allImperativeAggregateFunctions]] allImperativeAggregateFunctions | spark-sql-Expression-ImperativeAggregate.md[ImperativeAggregate] functions

Used when...FIXME

| [[allImperativeAggregateFunctionPositions]] allImperativeAggregateFunctionPositions | Positions

Used when...FIXME

| [[expressionAggInitialProjection]] expressionAggInitialProjection | MutableProjection

Used when...FIXME

| generateOutput a| [[generateOutput]] Function used to generate an <> (i.e. (UnsafeRow, InternalRow) => UnsafeRow)

Used when:

  • ObjectAggregationIterator is requested for the <> and <>

  • SortBasedAggregationIterator is requested for the <> and <>

  • TungstenAggregationIterator is requested for the <> and <>

| [[groupingAttributes]] groupingAttributes | Grouping spark-sql-Expression-Attribute.md[attributes]

Used when...FIXME

| [[groupingProjection]] groupingProjection | UnsafeProjection

Used when...FIXME

| [[processRow]] processRow | (InternalRow, InternalRow) => Unit

Used when...FIXME |===

=== [[creating-instance]] Creating AggregationIterator Instance

AggregationIterator takes the following when created:

  • [[groupingExpressions]] Grouping expressions/NamedExpression.md[named expressions]
  • [[inputAttributes]] Input spark-sql-Expression-Attribute.md[attributes]
  • [[aggregateExpressions]] Aggregate expressions
  • [[aggregateAttributes]] Aggregate spark-sql-Expression-Attribute.md[attributes]
  • [[initialInputBufferOffset]] Initial input buffer offset
  • [[resultExpressions]] Result expressions/NamedExpression.md[named expressions]
  • [[newMutableProjection]] Function to create a new MutableProjection given expressions and attributes

AggregationIterator initializes the <>.

NOTE: AggregationIterator is a Scala abstract class and cannot be <> directly. It is created indirectly for the <>.

=== [[initializeAggregateFunctions]] initializeAggregateFunctions Internal Method

[source, scala]

initializeAggregateFunctions( expressions: Seq[AggregateExpression], startingInputBufferOffset: Int): Array[AggregateFunction]


initializeAggregateFunctions...FIXME

NOTE: initializeAggregateFunctions is used when...FIXME

=== [[generateProcessRow]] generateProcessRow Internal Method

[source, scala]

generateProcessRow( expressions: Seq[AggregateExpression], functions: Seq[AggregateFunction], inputAttributes: Seq[Attribute]): (InternalRow, InternalRow) => Unit


generateProcessRow...FIXME

NOTE: generateProcessRow is used when...FIXME

=== [[generateResultProjection]] generateResultProjection Method

[source, scala]

generateResultProjection(): (UnsafeRow, InternalRow) => UnsafeRow

generateResultProjection...FIXME

[NOTE]

generateResultProjection is used when:

  • AggregationIterator is <>

* TungstenAggregationIterator is requested for the <>


Last update: 2021-02-18