Skip to content

First Aggregate Function Expression

First is a <> function expression that is <> when:

  • AstBuilder is requested to <>

  • <> standard function is used

  • first and first_value <> are used

[source, scala]

val sqlText = "FIRST (organizationName IGNORE NULLS)" val e = spark.sessionState.sqlParser.parseExpression(sqlText) scala> :type e org.apache.spark.sql.catalyst.expressions.Expression

import org.apache.spark.sql.catalyst.expressions.aggregate.AggregateExpression val aggExpr = e.asInstanceOf[AggregateExpression]

import org.apache.spark.sql.catalyst.expressions.aggregate.First val f = aggExpr.aggregateFunction scala> println(f.simpleString) first('organizationName) ignore nulls

[[evaluateExpression]] When requested to <> (and return the final value), First simply returns a <> (with first name and the <> of the <> expression).

[[catalyst-dsl]] TIP: Use <> operator from the Catalyst DSL to create an First aggregate function expression, e.g. for testing or Spark SQL internals exploration.

Catalyst DSL

first(e: Expression): Expression

first <> a First expression and requests it to <>.

[source, scala]

import org.apache.spark.sql.catalyst.dsl.expressions._ val e = first('orgName)

scala> println(e.numberedTreeString) 00 first('orgName, false) 01 +- first('orgName)() 02 :- 'orgName 03 +- false

import org.apache.spark.sql.catalyst.expressions.aggregate.AggregateExpression val aggExpr = e.asInstanceOf[AggregateExpression]

import org.apache.spark.sql.catalyst.expressions.aggregate.First val f = aggExpr.aggregateFunction scala> println(f.simpleString) first('orgName)()

=== [[creating-instance]] Creating First Instance

First takes the following when created:

  • [[child]] Child <>
  • [[ignoreNullsExpr]] ignoreNullsExpr flag <>
Back to top