Skip to content

DataSourceV2Relation Leaf Logical Operator

DataSourceV2Relation is a <> that represents a data scan (data reading) or data writing in the DataSource V2.

DataSourceV2Relation is <> (indirectly via <> helper method) exclusively when DataFrameReader is requested to "load" data (as a DataFrame).

[[creating-instance]] DataSourceV2Relation takes the following to be created:

  • [[source]] FIXME
  • [[output]] Output <> (Seq[AttributeReference])
  • [[options]] Options (Map[String, String])
  • [[tableIdent]] Optional TableIdentifier (default: undefined, i.e. None)
  • [[userSpecifiedSchema]] User-defined schema (default: undefined, i.e. None)

When used to represent a data scan (data reading), DataSourceV2Relation is planned (translated) to a <> with a <> physical operator (possibly under the <> operator) when DataSourceV2Strategy execution planning strategy is requested to plan a logical plan.

When used to represent a data write (with <> logical operator), DataSourceV2Relation is planned (translated) to a <> physical operator (with the <>) when DataSourceV2Strategy execution planning strategy is requested to plan a logical plan.

=== [[create]] Creating DataSourceV2Relation Instance -- create Factory Method

[source, scala]

create( source: DataSourceV2, options: Map[String, String], tableIdent: Option[TableIdentifier] = None, userSpecifiedSchema: Option[StructType] = None): DataSourceV2Relation


create requests the given FIXME to create a FIXME (with the given options and user-specified schema).

create finds the table in the given options unless the optional tableIdent is defined.

In the end, create <>.

NOTE: create is used exclusively when DataFrameReader is requested to "load" data (as a DataFrame).

=== [[computeStats]] Computing Statistics -- computeStats Method

[source, scala]

computeStats(): Statistics

NOTE: computeStats is part of the <> to compute a Statistics.

computeStats...FIXME

=== [[newWriter]] Creating DataSourceWriter -- newWriter Method

[source, scala]

newWriter(): DataSourceWriter

newWriter simply requests (delegates to) the <> to <>.

newWriter is used when DataSourceV2Strategy execution planning strategy is executed.


Last update: 2020-11-13