Change Description
The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.
New Features
- Optimized the execution plan and added three new rules: DeduplicateRightSideOfLeftSemiJoin, RewriteSelfJoinInInPredicate, and MergeSubqueryFilters.
- Added support for the NullType data type.
- Added the SubqueryBroadcastExec, CoalesceExecTransformer, and Limit Omni operators.
- Optimized operator functions: HashAggregator RollUp optimization, TableScan operator Parquet data read optimization, and Radix Sort for Sort operator.
- Optimized end expressions: adding the instr, startswith, and endswith functions, allowing conversion between the string type and int/long type, optimizing decimal data processing, and optimizing the expressions of the string type.
- Optimized functions in a Kerberos security cluster: operator acceleration in Spark local or Yarn mode, and ORC/Parquet data read in native mode.
- Added the spill function for Window and HashAggregator operators.
- Optimized NEON instructions, covering HashJoin, Sort, and Aggregator operators.
Modified Features
None
Removed Features
None
Parent topic: V1.4.0