Skip to content

RowDataSourceScanExec Leaf Physical Operator

RowDataSourceScanExec is a DataSourceScanExec (and so indirectly a leaf physical operator) for scanning data from a BaseRelation.

RowDataSourceScanExec is a InputRDDCodegen.

Performance Metrics

Key Name (in web UI) Description
numOutputRows number of output rows Number of output rows

Creating Instance

RowDataSourceScanExec takes the following to be created:

RowDataSourceScanExec is created when:

Metadata

metadata: Map[String, String]

metadata is part of the DataSourceScanExec abstraction.

metadata marks the filter predicates that are included in the handled filters predicates with * (star).

Note

Filter predicates with * (star) are to denote filters that are pushed down to a relation (aka data source).

In the end, metadata creates the following mapping:

Executing Physical Operator

doExecute(): RDD[InternalRow]

doExecute is part of the SparkPlan abstraction.

doExecute...FIXME

createUnsafeProjection

createUnsafeProjection: Boolean

createUnsafeProjection is part of the InputRDDCodegen abstraction.

createUnsafeProjection is true.

Input RDD

inputRDD: RDD[InternalRow]

inputRDD is part of the InputRDDCodegen abstraction.

inputRDD is the RDD.

Back to top