Skip to content

StreamingDeduplicationStrategy Execution Planning Strategy

StreamingDeduplicationStrategy is an execution planning strategy that can plan streaming queries with Deduplicate logical operators (over streaming queries) to StreamingDeduplicateExec physical operators.

Tip

Learn more about Execution Planning Strategies in The Internals of Spark SQL online book.

StreamingDeduplicationStrategy is available using SessionState.

spark.sessionState.planner.StreamingDeduplicationStrategy