HashJoin — Hash-Based Join Physical Operators¶

HashJoin is an extension of the BaseJoinExec abstraction for hash-based join physical operators with support for Java code generation.

Contract¶

buildSide¶

buildSide: BuildSide

Preparing HashedRelation¶

prepareRelation(
  ctx: CodegenContext): HashedRelationInfo

Used when:

• HashJoin is requested to codegenInner, codegenOuter, codegenSemi, codegenAnti, codegenExistence

Implementations¶

• BroadcastHashJoinExec
• ShuffledHashJoinExec

join¶

join(
  streamedIter: Iterator[InternalRow],
  hashed: HashedRelation,
  numOutputRows: SQLMetric): Iterator[InternalRow]

join branches off per JoinType to create an joined rows iterator (off the rows from the input streamedIter and hashed):

• innerJoin for a InnerLike join

• outerJoin for a LeftOuter or a RightOuter join

• semiJoin for a LeftSemi join

• antiJoin for a LeftAnti join

• existenceJoin for a ExistenceJoin join

join creates a result projection.

In the end, for every row in the joined rows iterator join increments the input numOutputRows SQL metric and applies the result projection.

join reports an IllegalArgumentException for unsupported JoinType:

HashJoin should not take [joinType] as the JoinType

join is used when:

• BroadcastHashJoinExec and ShuffledHashJoinExec physical operators are executed

Generating Java Code for Anti Join¶

codegenAnti(
  ctx: CodegenContext,
  input: Seq[ExprCode]): String

codegenAnti...FIXME

codegenAnti is used when:

• BroadcastHashJoinExec physical operator is requested to codegenAnti (with the isNullAwareAntiJoin flag off)
• HashJoin is requested to doConsume