ApplyColumnarRulesAndInsertTransitions Physical Optimization¶

ApplyColumnarRulesAndInsertTransitions is a physical query plan optimization.

ApplyColumnarRulesAndInsertTransitions is a Catalyst rule for transforming physical plans (Rule[SparkPlan]).

Creating Instance¶

ApplyColumnarRulesAndInsertTransitions takes the following to be created:

• SQLConf
• ColumnarRules

ApplyColumnarRulesAndInsertTransitions is created when:

• QueryExecution utility is requested for preparations optimizations
• AdaptiveSparkPlanExec physical operator is requested for adaptive optimization

Executing Rule¶

apply(
  plan: SparkPlan): SparkPlan

apply...FIXME

apply is part of the Rule abstraction.

Inserting ColumnarToRowExec Transitions¶

insertTransitions(
  plan: SparkPlan): SparkPlan

insertTransitions simply creates a ColumnarToRowExec physical operator for the given SparkPlan that supportsColumnar. The child of the ColumnarToRowExec operator is created using insertRowToColumnar.

Inserting RowToColumnarExec Transitions¶

insertRowToColumnar(
  plan: SparkPlan): SparkPlan

insertRowToColumnar simply creates a RowToColumnarExec physical operator for the given SparkPlan that does not supportsColumnar. The child of the RowToColumnarExec operator is created using insertTransitions. Otherwise, insertRowToColumnar requests the operator for child operators and insertRowToColumnar.