Skip to content

AggregateExpression

AggregateExpression is an unevaluable expression that acts as a container for an AggregateFunction.

Creating Instance

AggregateExpression takes the following to be created:

AggregateExpression is created using apply utility.

Creating AggregateExpression

apply(
  aggregateFunction: AggregateFunction,
  mode: AggregateMode,
  isDistinct: Boolean,
  filter: Option[Expression] = None): AggregateExpression

apply creates an AggregateExpression with a new autogenerated ExprId.

apply is used when:

Human-Friendly Textual Representation

toString: String

toString returns the following text:

[prefix][name]([args]) FILTER (WHERE [predicate])

toString converts the mode to a prefix.

mode prefix
Partial partial_
PartialMerge merge_
Final or Complete (empty)

toString requests the AggregateFunction for the toAggString (with the isDistinct flag).

In the end, toString adds FILTER (WHERE [predicate]) based on the optional filter expression.

Review Me

AggregateExpression contains the following:

  • [[aggregateFunction]] AggregateFunction
  • [[mode]] AggregateMode
  • [[isDistinct]] isDistinct flag indicating whether this aggregation is distinct or not (e.g. whether SQL's DISTINCT keyword was used for the aggregate function)
  • [[resultId]] ExprId

AggregateExpression is created when:

  • Analyzer is requested to resolve AggregateFunctions (and creates an AggregateExpression with Complete aggregate mode for the functions)

  • UserDefinedAggregateFunction is created with isDistinct flag disabled or enabled

  • AggUtils is requested to planAggregateWithOneDistinct (and creates AggregateExpressions with Partial and Final aggregate modes for the functions)

  • Aggregator is requested for a TypedColumn (using Aggregator.toColumn)

  • AggregateFunction is spark-sql-Expression-AggregateFunction.md#toAggregateExpression[wrapped in a AggregateExpression]

[[toString-prefixes]] .toString's Prefixes per AggregateMode [cols="1,2",options="header",width="100%"] |=== | Prefix | AggregateMode

| partial_ | Partial

| merge_ | PartialMerge

| (empty) | Final or Complete |===

[[properties]] .AggregateExpression's Properties [width="100%",cols="1,2",options="header"] |=== | Name | Description

| canonicalized | AggregateExpression with <> expression canonicalized with the special ExprId as 0.

| children | <> expression (for which AggregateExpression was created).

| dataType | DataType of AggregateFunction expression

| foldable | Disabled (i.e. false)

| nullable | Whether or not <> expression is nullable.

| [[references]] references a| AttributeSet with the following:

  • references of <> when <> is Partial or Complete

  • spark-sql-Expression-AggregateFunction.md#aggBufferAttributes[aggBufferAttributes] of <> when PartialMerge or Final

| resultAttribute a|

spark-sql-Expression-Attribute.md[Attribute] that is:

  • AttributeReference when <> is itself resolved

  • UnresolvedAttribute otherwise

| sql | Requests <> to generate SQL output (with <> flag).

| toString | <> followed by <>'s toAggString (with <> flag). |===

Back to top