Skip to content

AQEOptimizer — Adaptive Logical Optimizer

AQEOptimizer is a logical optimizer (a RuleExecutor of the logical rules) for re-optimizing logical plans in Adaptive Query Execution.

Creating Instance

AQEOptimizer takes the following to be created:

AQEOptimizer is created when:

  • AdaptiveSparkPlanExec physical operator is created

Default Batches

Batch Name Strategy Rules
Demote BroadcastHashJoin Once DemoteBroadcastHashJoin
Eliminate Join to Empty Relation Once EliminateJoinToEmptyRelation

Batches

batches: Seq[Batch]

batches is part of the RuleExecutor abstraction.

batches uses the spark.sql.adaptive.optimizer.excludedRules configuration property for the rules to exclude from the default rules.

For excluded rules, batches prints out the following INFO message to the logs:

Optimization rule '[ruleName]' is excluded from the optimizer.

For batches with all rules excluded, batches prints out the following INFO message to the logs:

Optimization batch '[name]' is excluded from the optimizer as all enclosed rules have been excluded.
Back to top