Skip to content

FunctionRegistry -- Contract for Function Registries (Catalogs)

FunctionRegistry is the <> of <> (catalogs) of native and user-defined functions.

[[contract]] [source, scala]

package org.apache.spark.sql.catalyst.analysis

trait FunctionRegistry { // only required properties (vals and methods) that have no implementation // the others follow def clear(): Unit def dropFunction(name: FunctionIdentifier): Boolean def listFunction(): Seq[FunctionIdentifier] def lookupFunction(name: FunctionIdentifier): Option[ExpressionInfo] def lookupFunction(name: FunctionIdentifier, children: Seq[Expression]): Expression def lookupFunctionBuilder(name: FunctionIdentifier): Option[FunctionBuilder] def registerFunction( name: FunctionIdentifier, info: ExpressionInfo, builder: FunctionBuilder): Unit }

.FunctionRegistry Contract [cols="1m,2",options="header",width="100%"] |=== | Property | Description

| clear | [[clear]] Used exclusively when SessionCatalog is requested to reset

| dropFunction | [[dropFunction]] Used when...FIXME

| listFunction | [[listFunction]] Used when...FIXME

| lookupFunction a| [[lookupFunction]]

Used when:

| lookupFunctionBuilder | [[lookupFunctionBuilder]] Used when...FIXME

| registerFunction a| [[registerFunction]]

Used when:

  • SessionCatalog is requested to registerFunction or reset

  • FunctionRegistry is requested for a <> or <>

  • SimpleFunctionRegistry is requested to clone |===

[[implementations]] NOTE: The one and only FunctionRegistry available in Spark SQL is <>.

FunctionRegistry is available through SessionState.md#functionRegistry[functionRegistry] property of a SessionState (that is available as <> property of a SparkSession).

[source, scala]

scala> :type spark org.apache.spark.sql.SparkSession

scala> :type spark.sessionState.functionRegistry org.apache.spark.sql.catalyst.analysis.FunctionRegistry

NOTE: You can register a new user-defined function using UDFRegistration.md[UDFRegistration].

[[attributes]] .FunctionRegistry's Attributes [width="100%",cols="1,2",options="header"] |=== | Name | Description

| [[builtin]] builtin | <> with the <> registered. |===

[[expressions]] FunctionRegistry manages function expression registry of <> and the corresponding built-in/native SQL functions (that can be used in SQL statements).

.(Subset of) FunctionRegistry's Catalyst Expression to SQL Function Mapping [cols="1,1m",options="header",width="100%"] |=== | Catalyst Expression | SQL Function

| <> | [[cume_dist]] cume_dist

| IfNull | [[ifnull]] ifnull

| Left | [[left]] left

| <> | [[monotonically_increasing_id]] monotonically_increasing_id

| NullIf | [[nullif]] nullif

| Nvl | [[nvl]] nvl

| Nvl2 | [[nvl2]] nvl2

| <> | [[to_date]] to_date

| <> | [[to_timestamp]] to_timestamp

| Right | [[right]] right

| <> | [[struct]] struct |===

=== [[expression]] expression Internal Method

[source, scala]

expressionT <: Expression (implicit tag: ClassTag[T]): (String, (ExpressionInfo, FunctionBuilder))

expression...FIXME

NOTE: expression is used when...FIXME

=== [[SimpleFunctionRegistry]] SimpleFunctionRegistry

SimpleFunctionRegistry is the default <> that is backed by a hash map (with optional case sensitivity).

=== [[createOrReplaceTempFunction]] createOrReplaceTempFunction Final Method

[source, scala]

createOrReplaceTempFunction(name: String, builder: FunctionBuilder): Unit

createOrReplaceTempFunction...FIXME

NOTE: createOrReplaceTempFunction is used exclusively when UDFRegistration is requested to register an <>, <>, <> or registerPython.

=== [[functionExists]] functionExists Method

[source, scala]

functionExists(name: FunctionIdentifier): Boolean

functionExists...FIXME

NOTE: functionExists is used when...FIXME

Back to top