WriteBuilder¶
WriteBuilder
is an abstraction of write builders for batch and streaming.
Contract¶
Creating BatchWrite¶
BatchWrite buildForBatch()
BatchWrite for writing data to a batch source
Used when:
- CreateTableAsSelectExec, ReplaceTableAsSelectExec, AppendDataExec, OverwriteByExpressionExec, OverwritePartitionsDynamicExec, AtomicTableWriteExec physical commands are executed
Creating StreamingWrite¶
StreamingWrite buildForStreaming()
StreamingWrite
for writing data to a streaming source
Used when:
StreamExecution
stream execution engine (Spark Structured Streaming) is requested tocreateStreamingWrite
Implementations¶
- ConsoleTable (Spark Structured Streaming)
- ForeachWriterTable (Spark Structured Streaming)
- KafkaTable
- MemorySink (Spark Structured Streaming)
- FileWriteBuilder
- NoopWriteBuilder
- SupportsDynamicOverwrite
- SupportsOverwrite
- SupportsStreamingUpdate
- SupportsTruncate
- V1WriteBuilder
Last update: 2020-11-13