Skip to content

StreamingExecutionRelation Leaf Logical Operator

StreamingExecutionRelation is a leaf logical operator (i.e. LogicalPlan) that represents a streaming source in the logical query plan of a streaming Dataset.

The main use of StreamingExecutionRelation logical operator is to be a "placeholder" in a logical query plan that will be replaced with the real relation (with new data that has arrived since the last batch) or an empty LocalRelation when StreamExecution is requested to <>.

StreamingExecutionRelation is <> for a StreamingRelation in analyzed logical query plan (that is the execution representation of a streaming Dataset).

Note

Right after StreamExecution MicroBatchExecution.md#runStream-initializing-sources[has started running streaming batches] it initializes the streaming sources by transforming the analyzed logical plan of the streaming Dataset so that every StreamingRelation logical operator is replaced by the corresponding StreamingExecutionRelation.

StreamingExecutionRelation Represents Streaming Source At Execution

Note

StreamingExecutionRelation is also resolved (planned) to a StreamingRelationExec physical operator in StreamingRelationStrategy execution planning strategy only when explaining a streaming Dataset.

Creating Instance

StreamingExecutionRelation takes the following when created:

=== [[apply]] Creating StreamingExecutionRelation (based on a Source) -- apply Object Method

[source, scala]

apply(source: Source): StreamingExecutionRelation

apply creates a StreamingExecutionRelation for the input source and with the attributes of the schema of the source.

NOTE: apply seems to be used for tests only.