Skip to content


InterpretedProjection is a[Projection] that...FIXME

[[creating-instance]] [[expressions]] InterpretedProjection takes expressions/[expressions] when created.

[source, scala]

// HACK: Disable symbolToColumn implicit conversion // It is imported automatically in spark-shell (and makes demos impossible) // implicit def symbolToColumn(s: Symbol): org.apache.spark.sql.ColumnName trait ThatWasABadIdea implicit def symbolToColumn(ack: ThatWasABadIdea) = ack

import org.apache.spark.sql.catalyst.dsl.expressions._ val boundRef = '

import org.apache.spark.sql.catalyst.expressions.{Expression, Literal} val expressions: Seq[Expression] = Seq(Literal(1), boundRef)

import org.apache.spark.sql.catalyst.expressions.InterpretedProjection val ip = new InterpretedProjection(expressions) scala> println(ip) Row => [1,input[4, string, true]]

InterpretedProjection is <> when:

  • UserDefinedGenerator is requested to initializeConverters

  • ConvertToLocalRelation logical optimization is executed (to transform Project logical operators)

  • HiveGenericUDTF is evaluated

  • ScriptTransformationExec physical operator is executed

=== [[initialize]] Initializing Nondeterministic Expressions -- initialize Method

[source, scala]

initialize(partitionIndex: Int): Unit

NOTE: initialize is part of[Projection Contract] to...FIXME.

initialize requests Nondeterministic expressions (in <>) to[initialize] with the partitionIndex.

Last update: 2020-11-15