WindowExpression Unevaluable Expression¶
WindowExpression
is an unevaluable expression that represents a window function (over some WindowSpecDefinition).
WindowExpression
is <
-
WindowSpec
is requested to <> (when < > operator is used) -
WindowsSubstitution logical evaluation rule is executed (with <
> logical operators with < > expressions) -
AstBuilder
is requested to <> in a SQL statement
NOTE: WindowExpression
can only be <
// Using Catalyst DSL
val wf = 'count.function(star())
val windowSpec = ???
NOTE: WindowExpression
is resolved in ExtractWindowExpressions, ResolveWindowFrame and ResolveWindowOrder logical rules.
import org.apache.spark.sql.catalyst.expressions.WindowExpression
// relation - Dataset as a table to query
val table = spark.emptyDataset[Int]
scala> val windowExpr = table
.selectExpr("count() OVER (PARTITION BY value) AS count")
.queryExecution
.logical // <1>
.expressions
.toList(0)
.children(0)
.asInstanceOf[WindowExpression]
windowExpr: org.apache.spark.sql.catalyst.expressions.WindowExpression = 'count() windowspecdefinition('value, UnspecifiedFrame)
scala> windowExpr.sql
res2: String = count() OVER (PARTITION BY `value` UnspecifiedFrame)
sqlParser
directly as in WithWindowDefinition.md#example[WithWindowDefinition Example] [[properties]] .WindowExpression's Properties [width="100%",cols="1,2",options="header"] |=== | Name | Description
| children
| Collection of two Expression.md[expressions], i.e. <WindowExpression
was created.
| dataType
| DataType of windowFunction
| foldable
| Whether or not <
| nullable
| Whether or not <
| sql
| "[windowFunction].sql OVER [windowSpec].sql"
| toString
| "[windowFunction] [windowSpec]"
|===
NOTE: WindowExpression
is subject to <
NOTE: Distinct window functions are not supported which is enforced at <
NOTE: An offset window function can only be evaluated in an ordered row-based window frame with a single offset which is enforced at <
=== [[catalyst-dsl]][[windowExpr]] Catalyst DSL -- windowExpr
Operator
[source, scala]¶
windowExpr( windowFunc: Expression, windowSpec: WindowSpecDefinition): WindowExpression
windowExpr operator in Catalyst DSL creates a <
[source, scala]¶
// FIXME: DEMO¶
Creating Instance¶
WindowExpression
takes the following when created:
- [[windowFunction]] Window function expression
- [[windowSpec]] WindowSpecDefinition expression