FunctionRegistry -- Contract for Function Registries (Catalogs)¶
FunctionRegistry
is the <
[[contract]] [source, scala]
package org.apache.spark.sql.catalyst.analysis
trait FunctionRegistry { // only required properties (vals and methods) that have no implementation // the others follow def clear(): Unit def dropFunction(name: FunctionIdentifier): Boolean def listFunction(): Seq[FunctionIdentifier] def lookupFunction(name: FunctionIdentifier): Option[ExpressionInfo] def lookupFunction(name: FunctionIdentifier, children: Seq[Expression]): Expression def lookupFunctionBuilder(name: FunctionIdentifier): Option[FunctionBuilder] def registerFunction( name: FunctionIdentifier, info: ExpressionInfo, builder: FunctionBuilder): Unit }
.FunctionRegistry Contract [cols="1m,2",options="header",width="100%"] |=== | Property | Description
| clear | [[clear]] Used exclusively when SessionCatalog
is requested to reset
| dropFunction | [[dropFunction]] Used when...FIXME
| listFunction | [[listFunction]] Used when...FIXME
| lookupFunction a| [[lookupFunction]]
Used when:
-
FunctionRegistry
is requested to <> -
SessionCatalog
is requested to find a function by name, lookupFunctionInfo or reset -
HiveSessionCatalog
is requested to lookupFunction0
| lookupFunctionBuilder | [[lookupFunctionBuilder]] Used when...FIXME
| registerFunction a| [[registerFunction]]
Used when:
-
SessionCatalog
is requested to registerFunction or reset -
FunctionRegistry
is requested for a <> or < > -
SimpleFunctionRegistry
is requested toclone
|===
[[implementations]] NOTE: The one and only FunctionRegistry
available in Spark SQL is <
FunctionRegistry
is available through SessionState.md#functionRegistry[functionRegistry] property of a SessionState
(that is available as <SparkSession
).
[source, scala]¶
scala> :type spark org.apache.spark.sql.SparkSession
scala> :type spark.sessionState.functionRegistry org.apache.spark.sql.catalyst.analysis.FunctionRegistry
NOTE: You can register a new user-defined function using UDFRegistration.md[UDFRegistration].
[[attributes]] .FunctionRegistry's Attributes [width="100%",cols="1,2",options="header"] |=== | Name | Description
| [[builtin]] builtin
| <
[[expressions]] FunctionRegistry
manages function expression registry of <
.(Subset of) FunctionRegistry's Catalyst Expression to SQL Function Mapping [cols="1,1m",options="header",width="100%"] |=== | Catalyst Expression | SQL Function
| <
| IfNull
| [[ifnull]] ifnull
| Left
| [[left]] left
| <
| NullIf
| [[nullif]] nullif
| Nvl
| [[nvl]] nvl
| Nvl2
| [[nvl2]] nvl2
| <
| <
| Right
| [[right]] right
| <
=== [[expression]] expression
Internal Method
[source, scala]¶
expressionT <: Expression (implicit tag: ClassTag[T]): (String, (ExpressionInfo, FunctionBuilder))
expression
...FIXME
NOTE: expression
is used when...FIXME
=== [[SimpleFunctionRegistry]] SimpleFunctionRegistry
SimpleFunctionRegistry
is the default <
=== [[createOrReplaceTempFunction]] createOrReplaceTempFunction
Final Method
[source, scala]¶
createOrReplaceTempFunction(name: String, builder: FunctionBuilder): Unit¶
createOrReplaceTempFunction
...FIXME
NOTE: createOrReplaceTempFunction
is used exclusively when UDFRegistration
is requested to register an <registerPython
.
=== [[functionExists]] functionExists
Method
[source, scala]¶
functionExists(name: FunctionIdentifier): Boolean¶
functionExists
...FIXME
NOTE: functionExists
is used when...FIXME