Skip to content

JsonToStructs Unary Expression

JsonToStructs is a unary expression with timezone support and CodegenFallback.

JsonToStructs is <> to represent spark-sql-functions.md#from_json[from_json] function.

import org.apache.spark.sql.functions.from_json
val jsonCol = from_json($"json", new StructType())

import org.apache.spark.sql.catalyst.expressions.JsonToStructs
val jsonExpr = jsonCol.expr.asInstanceOf[JsonToStructs]
scala> println(jsonExpr.numberedTreeString)
00 jsontostructs('json, None)
01 +- 'json

JsonToStructs is a <> expression.

[[FAILFAST]] [NOTE] ==== JsonToStructs uses <> in FAILFAST mode that simply fails early when a corrupted/malformed record is found (and hence does not support columnNameOfCorruptRecord JSON option). ====

[[properties]] .JsonToStructs's Properties [width="100%",cols="1,2",options="header"] |=== | Property | Description

| [[converter]] converter | Function that converts Seq[InternalRow] into...FIXME

| [[nullable]] nullable | Enabled (i.e. true)

| [[parser]] parser a| JacksonParser with <> and JSON options

| [[rowSchema]] rowSchema a| StructType that...FIXME

  • <> when of type StructType
  • StructType of the elements in <> when of type ArrayType |===

=== [[creating-instance]] Creating JsonToStructs Instance

JsonToStructs takes the following when created:

  • [[schema]] DataType
  • [[options]] Options
  • [[child]] Child expression
  • [[timeZoneId]] Optional time zone ID

JsonToStructs initializes the <>.

=== [[validateSchemaLiteral]] Parsing Table Schema for String Literals -- validateSchemaLiteral Method

[source, scala]

validateSchemaLiteral(exp: Expression): StructType

validateSchemaLiteral requests CatalystSqlParser to parseTableSchema for Literal of StringType.

For any other non-StringType types, validateSchemaLiteral reports a AnalysisException:

Expected a string literal instead of [expression]
Back to top