ResolveJoinStrategyHints Logical Resolution Rule¶

ResolveJoinStrategyHints is a logical resolution rule to resolve UnresolvedHints logical operators with JoinStrategyHints.

ResolveJoinStrategyHints is a Catalyst rule for transforming logical plans (Rule[LogicalPlan]).

ResolveJoinStrategyHints is part of Hints batch of rules of Logical Analyzer.

Creating Instance¶

ResolveJoinStrategyHints takes the following to be created:

SQLConf

ResolveJoinStrategyHints is created when Logical Analyzer is requested for the batches of rules.

Executing Rule¶

apply(
  plan: LogicalPlan): LogicalPlan

apply traverses the given logical query plan to find UnresolvedHint operators with names that are the hintAliases of the supported JoinStrategyHints.

For UnresolvedHints with no parameters, apply creates a ResolvedHint (with HintInfo with just the name).

For UnresolvedHints with parameters, apply accepts two types of parameters:

Table Name (as a String)
Table Identifier (as a UnresolvedAttribute)

apply applyJoinStrategyHint to create a ResolvedHint.

apply is part of the Rule abstraction.

applyJoinStrategyHint Internal Method¶

applyJoinStrategyHint(
  plan: LogicalPlan,
  relationsInHint: Set[Seq[String]],
  relationsInHintWithMatch: mutable.HashSet[Seq[String]],
  hintName: String): LogicalPlan

applyJoinStrategyHint...FIXME

applyJoinStrategyHint is used when ResolveJoinStrategyHints logical rule is executed.

Example¶

// FIXME Review the example to use ResolveJoinStrategyHints and other hints

// Use Catalyst DSL to create a logical plan
import org.apache.spark.sql.catalyst.dsl.plans._
val plan = table("t1").join(table("t2")).hint(name = "broadcast", "t1", "table2")
scala> println(plan.numberedTreeString)
00 'UnresolvedHint broadcast, [t1, table2]
01 +- 'Join Inner
02    :- 'UnresolvedRelation `t1`
03    +- 'UnresolvedRelation `t2`

import org.apache.spark.sql.catalyst.analysis.ResolveHints.ResolveJoinStrategyHints
val resolver = new ResolveJoinStrategyHints(spark.sessionState.conf)
val analyzedPlan = resolver(plan)
scala> println(analyzedPlan.numberedTreeString)
00 'Join Inner
01 :- 'ResolvedHint (broadcast)
02 :  +- 'UnresolvedRelation `t1`
03 +- 'UnresolvedRelation `t2`

Last update: 2020-09-08