DDLUtils Utility¶
DDLUtils
is a helper object that...FIXME
[[HIVE_PROVIDER]] DDLUtils
uses hive
value to denote hive/index.md[Hive data source] (Hive provider).
=== [[verifyPartitionProviderIsHive]] verifyPartitionProviderIsHive
Utility
[source, scala]¶
verifyPartitionProviderIsHive( spark: SparkSession, table: CatalogTable, action: String): Unit
verifyPartitionProviderIsHive
requests the given CatalogTable for the TableIdentifier that is in turn requested for the table name.
verifyPartitionProviderIsHive
throws an AnalysisException
when spark.sql.hive.manageFilesourcePartitions configuration property is disabled (false
) and the input CatalogTable
is a <
[action] is not allowed on [tableName] since filesource partition management is disabled (spark.sql.hive.manageFilesourcePartitions = false).
verifyPartitionProviderIsHive
throws an AnalysisException
when the tracksPartitionsInCatalog of the given CatalogTable
is disabled (false
) and the input CatalogTable
is a <
[action] is not allowed on [tableName] since its partition metadata is not stored in the Hive metastore. To import this information into the metastore, run `msck repair table [tableName]`
NOTE: verifyPartitionProviderIsHive
is used when AlterTableAddPartitionCommand
, AlterTableRenamePartitionCommand
, AlterTableDropPartitionCommand
, AlterTableSetLocationCommand
, TruncateTableCommand.md[TruncateTableCommand], DescribeTableCommand.md[DescribeTableCommand], and ShowPartitionsCommand
commands are executed.
isDatasourceTable Utility¶
isDatasourceTable(
table: CatalogTable): Boolean
isDatasourceTable
is positive (true
) when the provider of the input CatalogTable is not hive when defined. Otherwise, isDatasourceTable
is negative (false
).
isDatasourceTable
is used when:
-
HiveExternalCatalog
is requested to createTable (and saveTableIntoHive) -
HiveUtils
utility is used to hive/HiveUtils.md#inferSchema[inferSchema] -
AlterTableSerDePropertiesCommand
,AlterTableAddColumnsCommand
,LoadDataCommand
, ShowCreateTableCommand.md[ShowCreateTableCommand] commands are executed -
DDLUtils
utility is used to <> -
DataSourceAnalysis and FindDataSourceTable logical rules are executed
isHiveTable Utility¶
isHiveTable(
provider: Option[String]): Boolean
isHiveTable
is positive (true
) when the provider is hive when defined. Otherwise, isHiveTable
is negative (false
).
isHiveTable
is used when:
-
HiveAnalysis logical resolution rule is executed
-
DDLUtils
utility is used to isHiveTable -
HiveOnlyCheck
extended check rule is executed
=== [[verifyNotReadPath]] verifyNotReadPath
Utility
[source, scala]¶
verifyNotReadPath( query: LogicalPlan, outputPath: Path) : Unit
verifyNotReadPath
...FIXME