AbstractSqlParser — Base SQL Parsing Infrastructure¶

AbstractSqlParser is an extension of the ParserInterface abstraction for SQL parsers that delegate parsing to an AstBuilder.

AbstractSqlParser is the foundation of the SQL Parsing Framework.

AstBuilder¶

astBuilder: AstBuilder

AstBuilder for parsing SQL statements

Implementations¶

CatalystSqlParser for parsing canonical textual representation of data types and schema
SparkSqlParser that is the default SQL parser in SessionState

Setting Up SqlBaseLexer and SqlBaseParser for Parsing¶

parse[T](
  command: String)(
  toResult: SqlBaseParser => T): T

parse sets up ANTLR parsing infrastructure with SqlBaseLexer and SqlBaseParser (which are the ANTLR-specific classes of Spark SQL that are auto-generated at build time from the SqlBase.g4 grammar).

Internally, parse first prints out the following INFO message to the logs:

Parsing command: [command]

Tip

Enable INFO logging level for the custom AbstractSqlParsers to see the above and other INFO messages.

parse then creates and sets up a SqlBaseLexer and SqlBaseParser that in turn passes the latter on to the input toResult function where the parsing finally happens.

parse uses SLL prediction mode for parsing first before falling back to LL mode.

In case of parsing errors, parse reports a ParseException.

parse is used in all the parse methods.