RowDataSourceScanExec Leaf Physical Operator¶
RowDataSourceScanExec is a InputRDDCodegen.
|Key||Name (in web UI)||Description|
|numOutputRows||number of output rows||Number of output rows|
RowDataSourceScanExec takes the following to be created:
- Output Schema (Attributes)
- Required Schema (StructType)
- Data Source Filter Predicates
- Handled Data Source Filter Predicates
RowDataSourceScanExec is created when:
metadata: Map[String, String]
metadata is part of the DataSourceScanExec abstraction.
Filter predicates with
* (star) are to denote filters that are pushed down to a relation (aka data source).
In the end,
metadata creates the following mapping:
- ReadSchema with the required schema converted to catalog representation
- PushedFilters with the marked and unmarked filter predicates
Executing Physical Operator¶
doExecute is part of the SparkPlan abstraction.
createUnsafeProjection is part of the InputRDDCodegen abstraction.
inputRDD is part of the InputRDDCodegen abstraction.
inputRDD is the RDD.