Window Utility Object -- Defining Window Specification¶
Window
utility object is a <
[[methods]] .Window API [cols="1m,3",options="header",width="100%"] |=== | Method | Description
| currentRow a| [[currentRow]]
[source, scala]¶
currentRow: Long¶
Value representing the current row that is used to define <
| orderBy a| [[orderBy]]
[source, scala]¶
orderBy(cols: Column*): WindowSpec orderBy(colName: String, colNames: String*): WindowSpec
Creates a <
| partitionBy a| [[partitionBy]]
[source, scala]¶
partitionBy(cols: Column*): WindowSpec partitionBy(colName: String, colNames: String*): WindowSpec
Creates a <
| rangeBetween a| [[rangeBetween]]
[source, scala]¶
rangeBetween(start: Column, end: Column): WindowSpec rangeBetween(start: Long, end: Long): WindowSpec
Creates a <start
(inclusive) to end
(inclusive). Both start
and end
are relative to the current row based on the actual value of the ORDER BY
expression(s).
| rowsBetween a| [[rowsBetween]]
[source, scala]¶
rowsBetween(start: Long, end: Long): WindowSpec¶
Creates a <start
(inclusive) to end
(inclusive). Both start
and end
are positions relative to the current row based on the position of the row within the partition.
| unboundedFollowing a| [[unboundedFollowing]]
[source, scala]¶
unboundedFollowing: Long¶
Value representing the last row in a partition (equivalent to "UNBOUNDED FOLLOWING" in SQL) that is used to define <
| unboundedPreceding a| [[unboundedPreceding]]
[source, scala]¶
unboundedPreceding: Long¶
Value representing the first row in a partition (equivalent to "UNBOUNDED PRECEDING" in SQL) that is used to define <
[source, scala]¶
import org.apache.spark.sql.expressions.Window import org.apache.spark.sql.functions.{currentRow, lit} val windowSpec = Window .partitionBy("orderId") .orderBy("time") .rangeBetween(currentRow, lit(1)) scala> :type windowSpec org.apache.spark.sql.expressions.WindowSpec
=== [[spec]] Creating "Empty" WindowSpec -- spec
Internal Method
[source, scala]¶
spec: WindowSpec¶
spec
creates an "empty" <UnspecifiedFrame
.
[NOTE]¶
spec
is used when:
- <
> operator is used (with no WindowSpec
)