Nondeterministic Expression Contract¶
Nondeterministic
is a <
Nondeterministic
expressions require explicit <
[[contract]] [source, scala]
package org.apache.spark.sql.catalyst.expressions
trait Nondeterministic extends Expression { // only required methods that have no implementation protected def initializeInternal(partitionIndex: Int): Unit protected def evalInternal(input: InternalRow): Any }
.Nondeterministic Contract [cols="1,2",options="header",width="100%"] |=== | Method | Description
| initializeInternal
| [[initializeInternal]] Initializing the Nondeterministic
expression
Used exclusively when Nondeterministic
expression is requested to <
| evalInternal
| [[evalInternal]] Evaluating the Nondeterministic
expression
Used exclusively when Nondeterministic
expression is requested to <
NOTE: Nondeterministic
expressions are the target of PullOutNondeterministic
logical plan rule.
[[implementations]] .Nondeterministic Expressions [cols="1,2",options="header",width="100%"] |=== | Expression | Description
| CurrentBatchTimestamp
| [[CurrentBatchTimestamp]]
| InputFileBlockLength
| [[InputFileBlockLength]]
| InputFileBlockStart
| [[InputFileBlockStart]]
| InputFileName
| [[InputFileName]]
| expressions/MonotonicallyIncreasingID.md[MonotonicallyIncreasingID] | [[MonotonicallyIncreasingID]]
| NondeterministicExpression
| [[NondeterministicExpression]]
| Rand
| [[Rand]]
| Randn
| [[Randn]]
| RDG
| [[RDG]]
| SparkPartitionID
| [[SparkPartitionID]] |===
[[internal-registries]] .Nondeterministic's Internal Properties (e.g. Registries, Counters and Flags) [cols="1,2",options="header",width="100%"] |=== | Name | Description
| [[deterministic]] Expression.md#deterministic[deterministic] | Always turned off (i.e. false
)
| [[foldable]] Expression.md#foldable[foldable] | Always turned off (i.e. false
)
| [[initialized]] initialized
| Controls whether a Nondeterministic
expression has been <
Turned off by default. |===
=== [[initialize]] Initializing Expression -- initialize
Method
[source, scala]¶
initialize(partitionIndex: Int): Unit¶
Internally, initialize
<
initialize
is used when InterpretedProjection and InterpretedMutableProjection
are requested to initialize
themselves.
=== [[eval]] Evaluating Expression -- eval
Method
[source, scala]¶
eval(input: InternalRow): Any¶
eval
is part of the Expression abstraction.
eval
is just a wrapper of <
Internally, eval
makes sure that the expression was <
eval
reports a IllegalArgumentException
exception when the internal <
requirement failed: Nondeterministic expression [name] should be initialized before eval.