InterpretedProjection¶
InterpretedProjection
is a Projection.md[Projection] that...FIXME
[[creating-instance]] [[expressions]] InterpretedProjection
takes expressions/Expression.md[expressions] when created.
[source, scala]¶
// HACK: Disable symbolToColumn implicit conversion // It is imported automatically in spark-shell (and makes demos impossible) // implicit def symbolToColumn(s: Symbol): org.apache.spark.sql.ColumnName trait ThatWasABadIdea implicit def symbolToColumn(ack: ThatWasABadIdea) = ack
import org.apache.spark.sql.catalyst.dsl.expressions._ val boundRef = 'hello.string.at(4)
import org.apache.spark.sql.catalyst.expressions.{Expression, Literal} val expressions: Seq[Expression] = Seq(Literal(1), boundRef)
import org.apache.spark.sql.catalyst.expressions.InterpretedProjection val ip = new InterpretedProjection(expressions) scala> println(ip) Row => [1,input[4, string, true]]
InterpretedProjection
is <
-
UserDefinedGenerator
is requested toinitializeConverters
-
ConvertToLocalRelation
logical optimization is executed (to transformProject
logical operators) -
HiveGenericUDTF
is evaluated -
ScriptTransformationExec
physical operator is executed
=== [[initialize]] Initializing Nondeterministic Expressions -- initialize
Method
[source, scala]¶
initialize(partitionIndex: Int): Unit¶
NOTE: initialize
is part of Projection.md#initialize[Projection Contract] to...FIXME.
initialize
requests Nondeterministic
expressions (in <partitionIndex
.