Table¶
Table
is an abstraction of logical structured data set:
- a directory or files on a file system
- a topic of Apache Kafka
- a table in a catalog
- others
Contract¶
Table Capabilities¶
Set<TableCapability> capabilities()
TableCapabilities of the table
Used when Table
is asked whether or not it supports a given capability
name¶
String name()
Name of the table
partitioning¶
Transform[] partitioning()
Partitions of the table (as Transforms)
Default: (empty)
Used when:
- ResolveInsertInto logical analysis rule is executed
DataFrameWriter
is requested to insertInto and save- DescribeTableExec physical operator is executed
properties¶
Map<String, String> properties()
Table properties
Default: (empty)
Used when:
- DescribeTableExec and
ShowTablePropertiesExec
physical operators are executed
schema¶
StructType schema()
StructType of the table
Used when:
DataSourceV2Relation
utility is used to create a DataSourceV2Relation logical operatorSimpleTableProvider
is requested to inferSchema- DataSourceV2Strategy execution planning strategy is executed
- DescribeTableExec physical operator is executed
FileDataSourceV2
is requested to inferSchema- (Spark Structured Streaming)
TextSocketTable
is requested for aScanBuilder
with a read schema - (Spark Structured Streaming)
DataStreamReader
is requested to load data
Implementations¶
- ConsoleTable (Spark Structured Streaming)
- FileTable
- ForeachWriterTable (Spark Structured Streaming)
- KafkaTable
- MemorySink (Spark Structured Streaming)
- MemoryStreamTable (Spark Structured Streaming)
- NoopTable
- RateStreamTable (Spark Structured Streaming)
- Sink (Spark Structured Streaming)
- StagedTable
- SupportsRead
- SupportsWrite
- TextSocketTable (Spark Structured Streaming)
- V1Table
Last update: 2020-11-13