HashedRelation¶

HashedRelation is an extension of the KnownSizeEstimation abstraction for relations with values hashed by some key.

Contract¶

asReadOnlyCopy¶

asReadOnlyCopy(): HashedRelation

A read-only copy of this HashedRelation to be safely used in a separate thread

Used when BroadcastHashJoinExec physical operator is executed

close¶

close(): Unit

Used when ShuffledHashJoinExec physical operator is requested to buildHashedRelation

get¶

get(
  key: InternalRow): Iterator[InternalRow]
get(
  key: Long): Iterator[InternalRow]

Gets internal rows for the given key or null

Used when:

• HashJoin is requested to innerJoin, [outerJoin]/HashJoin.md#outerJoin), semiJoin, existenceJoin and antiJoin
• LongHashedRelation is requested to get a value for a key

getValue¶

getValue(
  key: InternalRow): InternalRow
getValue(
  key: Long): InternalRow

Gives the value internal row for the given key

Used when LongHashedRelation is requested to get a value for a key

keyIsUnique¶

keyIsUnique: Boolean

Used when BroadcastHashJoinExec physical operator is requested to multipleOutputForOneInput, codegenInner, codegenOuter, codegenSemi, codegenAnti, codegenExistence

keys¶

keys(): Iterator[InternalRow]

Used when SubqueryBroadcastExec physical operator is requested for relationFuture

Implementations¶

• LongHashedRelation
• UnsafeHashedRelation

Creating HashedRelation¶

apply(
  input: Iterator[InternalRow],
  key: Seq[Expression],
  sizeEstimate: Int = 64,
  taskMemoryManager: TaskMemoryManager = null): HashedRelation

apply creates a LongHashedRelation when the input key collection has a single expression of type long or a UnsafeHashedRelation otherwise.

apply is used when:

• ShuffledHashJoinExec physical operator is requested to build a HashedRelation for given internal rows
• HashedRelationBroadcastMode is requested to transform