RewriteCorrelatedScalarSubquery Logical Optimization¶
RewriteCorrelatedScalarSubquery
is a base logical optimization that <
. FIXME
RewriteCorrelatedScalarSubquery
is part of the Operator Optimization before Inferring Filters fixed-point batch in the standard batches of the Logical Optimizer.
RewriteCorrelatedScalarSubquery
is simply a <Rule[LogicalPlan]
.
[source, scala]¶
import org.apache.spark.sql.catalyst.optimizer.RewriteCorrelatedScalarSubquery
// FIXME // Demo: Filter + Aggregate // Demo: Filter + UnaryNode
val plan = ??? val optimizedPlan = RewriteCorrelatedScalarSubquery(plan)
=== [[evalExpr]] evalExpr
Internal Method
[source, scala]¶
evalExpr(expr: Expression, bindings: Map[ExprId, Option[Any]]) : Option[Any]¶
evalExpr
...FIXME
NOTE: evalExpr
is used exclusively when RewriteCorrelatedScalarSubquery
is...FIXME
=== [[evalAggOnZeroTups]] evalAggOnZeroTups
Internal Method
[source, scala]¶
evalAggOnZeroTups(expr: Expression) : Option[Any]¶
evalAggOnZeroTups
...FIXME
NOTE: evalAggOnZeroTups
is used exclusively when RewriteCorrelatedScalarSubquery
is...FIXME
=== [[evalSubqueryOnZeroTups]] evalSubqueryOnZeroTups
Internal Method
[source, scala]¶
evalSubqueryOnZeroTups(plan: LogicalPlan) : Option[Any]¶
evalSubqueryOnZeroTups
...FIXME
NOTE: evalSubqueryOnZeroTups
is used exclusively when RewriteCorrelatedScalarSubquery
is requsted to <
=== [[constructLeftJoins]] constructLeftJoins
Internal Method
[source, scala]¶
constructLeftJoins( child: LogicalPlan, subqueries: ArrayBuffer[ScalarSubquery]): LogicalPlan
constructLeftJoins
...FIXME
NOTE: constructLeftJoins
is used exclusively when RewriteCorrelatedScalarSubquery
logical optimization is <
=== [[apply]] Executing Rule -- apply
Method
[source, scala]¶
apply(plan: LogicalPlan): LogicalPlan¶
apply
transforms the input spark-sql-LogicalPlan.md[logical plan] as follows:
. For Aggregate.md[Aggregate] operators, apply
...FIXME
. For Project.md[Project] operators, apply
...FIXME
. For Filter.md[Filter] operators, apply
...FIXME
apply
is part of the Rule abstraction.
=== [[extractCorrelatedScalarSubqueries]] Extracting ScalarSubquery Expressions with Children -- extractCorrelatedScalarSubqueries
Internal Method
[source, scala]¶
extractCorrelatedScalarSubqueriesE <: Expression: E
extractCorrelatedScalarSubqueries
finds all spark-sql-Expression-ExecSubqueryExpression-ScalarSubquery.md[ScalarSubquery] expressions with at least one spark-sql-Expression-ExecSubqueryExpression-ScalarSubquery.md#children[child] in the input expression
and adds them to the input subqueries
collection.
extractCorrelatedScalarSubqueries
traverses the input expression
down (the expression tree) and, every time a ScalarSubquery
with at least one child is found, returns the head of the output attributes of the spark-sql-Expression-ExecSubqueryExpression-ScalarSubquery.md#plan[subquery plan].
In the end, extractCorrelatedScalarSubqueries
returns the rewritten expression.
NOTE: extractCorrelatedScalarSubqueries
uses https://docs.scala-lang.org/overviews/collections/concrete-mutable-collection-classes.html[scala.collection.mutable.ArrayBuffer] and mutates an instance inside (i.e. adds ScalarSubquery
expressions) that makes for two output values, i.e. the rewritten expression and the ScalarSubquery
expressions.
NOTE: extractCorrelatedScalarSubqueries
is used exclusively when RewriteCorrelatedScalarSubquery
is <