Skip to content

CatalogManager

Creating Instance

CatalogManager takes the following to be created:

CatalogManager is created when:

Default Session Catalog

defaultSessionCatalog: CatalogPlugin

CatalogManager is given a CatalogPlugin when created for the default session catalog.

defaultSessionCatalog is used as the delegate catalog when CatalogManager is requested to load a V2SessionCatalog.

defaultSessionCatalog is used as the fall-back catalog when CatalogManager is requested to load a custom V2CatalogPlugin.

Default Catalog Name

CatalogManager defines spark_catalog as the name of the default catalog (V2SessionCatalog).

spark_catalog is used as the default value of spark.sql.defaultCatalog configuration property.

Current Catalog Name

_currentCatalogName: Option[String]

_currentCatalogName is the name of the current catalog and is undefined by default (None).

_currentCatalogName can be changed using setCurrentCatalog.

Current CatalogPlugin

currentCatalog: CatalogPlugin

currentCatalog uses the current CatalogPlugin if defined or falls back on spark.sql.defaultCatalog configuration property.

currentCatalog is used when:

Current Namespace

currentNamespace: Array[String]

currentNamespace...FIXME

currentNamespace is used when:

  • ShowCurrentNamespaceExec physical command is executed

  • ResolveNamespace analysis rule is executed

  • GetCurrentDatabase analysis rule is executed

  • CatalogAndIdentifier extractor utility is requested to unapply

  • ViewHelper utility is requested to generateViewProperties

Setting Current Namespace

setCurrentNamespace(
  namespace: Array[String]): Unit

setCurrentNamespace...FIXME

setCurrentNamespace is used when SetCatalogAndNamespaceExec physical command is executed.

Changing Current Catalog Name

setCurrentCatalog(
  catalogName: String): Unit

setCurrentCatalog checks out whether the given catalog name is different from the currentCatalog's.

Only if the names are different, setCurrentCatalog makes it _currentCatalogName and "resets" _currentNamespace (None). In the end, setCurrentCatalog requests the SessionCatalog to setCurrentDatabase as default.

setCurrentCatalog is used when SetCatalogAndNamespaceExec physical command is executed.

Finding CatalogPlugin by Name

catalog(
  name: String): CatalogPlugin

catalog returns the v2 session catalog when the given name is spark_catalog.

Otherwise, catalog looks up the name in catalogs internal registry. When not found, catalog tries to load a CatalogPlugin by name (and registers it in catalogs internal registry).

catalog is used when:

isCatalogRegistered

isCatalogRegistered(
  name: String): Boolean

isCatalogRegistered...FIXME

isCatalogRegistered is used when Analyzer is requested to expandRelationName.

v2SessionCatalog

v2SessionCatalog: CatalogPlugin

v2SessionCatalog...FIXME

v2SessionCatalog is used when:

  • CatalogManager is requested to look up a CatalogPlugin by name

  • CatalogV2Util is requested to getTableProviderCatalog

  • CatalogAndIdentifier utility is requested to extract a CatalogPlugin and an identifier from a multi-part name (unapply)

loadV2SessionCatalog

loadV2SessionCatalog(): CatalogPlugin

loadV2SessionCatalog loads the default spark_catalog.

If it is of type CatalogExtension, loadV2SessionCatalog requests it to setDelegateCatalog with the defaultSessionCatalog.

loadV2SessionCatalog is used when:


Last update: 2020-12-28