Skip to content

ParserInterface — SQL Parsers

ParserInterface is the abstraction of SQL parsers that can convert (parse) textual representation of SQL statements (SQL text) into Spark SQL's relational entities (e.g. Catalyst expressions, logical operators, table and function identifiers, table schema, and data types).

Accessing ParserInterface

ParserInterface is available as SessionState.sqlParser.

scala> :type spark
org.apache.spark.sql.SparkSession

scala> :type spark.sessionState.sqlParser
org.apache.spark.sql.catalyst.parser.ParserInterface

Contract

parseDataType

parseDataType(
  sqlText: String): DataType

Parses a SQL text to a DataType

Used when:

parseExpression

parseExpression(
  sqlText: String): Expression

Parses a SQL text to an Expression

Used in the following:

  • Dataset operators: <>, <> and <>

  • <> standard function

parseFunctionIdentifier

parseFunctionIdentifier(
  sqlText: String): FunctionIdentifier

Parses a SQL text to a FunctionIdentifier

Used when:

parseMultipartIdentifier

parseMultipartIdentifier(
  sqlText: String): Seq[String]

Parses a SQL text to a multi-part identifier

Used when:

parsePlan

parsePlan(
  sqlText: String): LogicalPlan

Parses a SQL text to a LogicalPlan

Used when:

parseRawDataType

parseRawDataType(
  sqlText: String): DataType

Used when...FIXME

parseTableIdentifier

parseTableIdentifier(
  sqlText: String): TableIdentifier

Parses a SQL text to a TableIdentifier

Used when:

parseTableSchema

parseTableSchema(
  sqlText: String): StructType

Parses a SQL text to a StructType

Used when:

Extensions

AbstractSqlParser is the base extension of the ParserInterface abstraction.


Last update: 2020-12-29