CreateNamedStruct Expression¶
CreateNamedStruct
is a <
[[prettyName]] CreateNamedStruct
uses named_struct for the <
[source, scala]¶
// Using Catalyst DSL import org.apache.spark.sql.catalyst.dsl.expressions._ val s = namedStruct("") scala> println(s) named_struct()
CreateNamedStruct
is registered in <named_struct
SQL function.
[source, scala]¶
import org.apache.spark.sql.catalyst.FunctionIdentifier val fid = FunctionIdentifier(funcName = "named_struct") val className = spark.sessionState.functionRegistry.lookupFunction(fid).get.getClassName scala> println(className) org.apache.spark.sql.catalyst.expressions.CreateNamedStruct
val q = sql("SELECT named_struct('id', 0)") // analyzed so the function is resolved already (using FunctionRegistry) val analyzedPlan = q.queryExecution.analyzed scala> println(analyzedPlan.numberedTreeString) 00 Project [named_struct(id, 0) AS named_struct(id, 0)#7] 01 +- OneRowRelation
val e = analyzedPlan.expressions.head.children.head import org.apache.spark.sql.catalyst.expressions.CreateNamedStruct assert(e.isInstanceOf[CreateNamedStruct])
CreateNamedStruct
is <
-
ScalaReflection, RowEncoder and
JavaTypeInference
are requested for a serializer of a type -
TimeWindowing and ResolveCreateNamedStruct logical resolution rules are executed
-
CreateStruct
is requested to <>
[[children]] [[creating-instance]] CreateNamedStruct
takes a collection of <
CreateNamedStruct
<
[source, scala]¶
// You could also use Seq("*") import org.apache.spark.sql.functions.lit val exprs = Seq("a", 1).map(lit).map(_.expr)
import org.apache.spark.sql.catalyst.expressions.CreateNamedStruct val createNamedStruct = CreateNamedStruct(exprs)
import org.apache.spark.sql.catalyst.expressions.codegen.{CodegenContext, ExprCode} val ctx = new CodegenContext // doGenCode is used when Expression.genCode is executed val ExprCode(code, _, _) = createNamedStruct.genCode(ctx)
// Helper methods def trim(code: String): String = { code.trim.split("\n").map(_.trim).filter(line => line.nonEmpty).mkString("\n") } def prettyPrint(code: String) = println(trim(code)) // END: Helper methods
scala> println(trim(code)) Object[] values_0 = new Object[1]; if (false) { values_0[0] = null; } else { values_0[0] = 1; } final InternalRow value_0 = new org.apache.spark.sql.catalyst.expressions.GenericInternalRow(values_0); values_0 = null;
[TIP]¶
Use namedStruct
operator from Catalyst DSL's expressions to create a CreateNamedStruct
expression.
[source, scala]¶
import org.apache.spark.sql.catalyst.dsl.expressions._ val s = namedStruct() scala> :type s org.apache.spark.sql.catalyst.expressions.Expression
import org.apache.spark.sql.catalyst.expressions.CreateNamedStruct assert(s.isInstanceOf[CreateNamedStruct])
val s = namedStruct("") scala> println(s) named_struct()
====
=== [[doGenCode]] Generating Java Source Code (ExprCode) For Code-Generated Expression Evaluation -- doGenCode
Method
[source, scala]¶
doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode¶
NOTE: doGenCode
is part of <
doGenCode
...FIXME