Skip to content

RewriteCorrelatedScalarSubquery Logical Optimization

RewriteCorrelatedScalarSubquery is a base logical optimization that <> with the following operators:


RewriteCorrelatedScalarSubquery is part of the Operator Optimization before Inferring Filters fixed-point batch in the standard batches of the Logical Optimizer.

RewriteCorrelatedScalarSubquery is simply a <> for transforming <>, i.e. Rule[LogicalPlan].

[source, scala]

import org.apache.spark.sql.catalyst.optimizer.RewriteCorrelatedScalarSubquery

// FIXME // Demo: Filter + Aggregate // Demo: Filter + UnaryNode

val plan = ??? val optimizedPlan = RewriteCorrelatedScalarSubquery(plan)

=== [[evalExpr]] evalExpr Internal Method

[source, scala]

evalExpr(expr: Expression, bindings: Map[ExprId, Option[Any]]) : Option[Any]


NOTE: evalExpr is used exclusively when RewriteCorrelatedScalarSubquery is...FIXME

=== [[evalAggOnZeroTups]] evalAggOnZeroTups Internal Method

[source, scala]

evalAggOnZeroTups(expr: Expression) : Option[Any]


NOTE: evalAggOnZeroTups is used exclusively when RewriteCorrelatedScalarSubquery is...FIXME

=== [[evalSubqueryOnZeroTups]] evalSubqueryOnZeroTups Internal Method

[source, scala]

evalSubqueryOnZeroTups(plan: LogicalPlan) : Option[Any]


NOTE: evalSubqueryOnZeroTups is used exclusively when RewriteCorrelatedScalarSubquery is requsted to <>.

=== [[constructLeftJoins]] constructLeftJoins Internal Method

[source, scala]

constructLeftJoins( child: LogicalPlan, subqueries: ArrayBuffer[ScalarSubquery]): LogicalPlan


NOTE: constructLeftJoins is used exclusively when RewriteCorrelatedScalarSubquery logical optimization is <> (i.e. applied to <>, <> or <> logical operators with correlated scalar subqueries)

=== [[apply]] Executing Rule -- apply Method

[source, scala]

apply(plan: LogicalPlan): LogicalPlan

apply transforms the input[logical plan] as follows:

. For[Aggregate] operators, apply...FIXME

. For[Project] operators, apply...FIXME

. For[Filter] operators, apply...FIXME

apply is part of the Rule abstraction.

=== [[extractCorrelatedScalarSubqueries]] Extracting ScalarSubquery Expressions with Children -- extractCorrelatedScalarSubqueries Internal Method

[source, scala]

extractCorrelatedScalarSubqueriesE <: Expression: E

extractCorrelatedScalarSubqueries finds all[ScalarSubquery] expressions with at least one[child] in the input expression and adds them to the input subqueries collection.

extractCorrelatedScalarSubqueries traverses the input expression down (the expression tree) and, every time a ScalarSubquery with at least one child is found, returns the head of the output attributes of the[subquery plan].

In the end, extractCorrelatedScalarSubqueries returns the rewritten expression.

NOTE: extractCorrelatedScalarSubqueries uses[scala.collection.mutable.ArrayBuffer] and mutates an instance inside (i.e. adds ScalarSubquery expressions) that makes for two output values, i.e. the rewritten expression and the ScalarSubquery expressions.

NOTE: extractCorrelatedScalarSubqueries is used exclusively when RewriteCorrelatedScalarSubquery is <> (i.e. applied to a[logical plan]).

Last update: 2020-11-07