Change History

2025-12-30

This issue is the twenty-first official release.

• OmniOperator 2.0.0:
  • Added the adaptation layer Gluten 1.3 for Spark, and added the concat_ws, regexp, regexp_replace, trim, and floor expressions for SparkExtension.
  • Added "error in opening zip file" Displayed When Running Gluten for Spark and "libabsl_xxx.so.2501.0.0 no such file" Displayed When Running Gluten for Spark.
• OmniAdvisor 2.0.0:
  • Added the notice on Spark SQL job data collection and security usage specifications in Architecture.
  • Added an important risk warning for disabling password complexity verification function in Initializing the Feature.
  • Deleted section "(Optional) Enabling the Background Data Management Page."
  • Added Environment Dependency Installation Failed and "Error: Failed building wheel for pyrfr" Displayed.
  • Update the OmniAdvisor 2.0 system framework package name. For details, see Environment Requirements.
• OmniStream 1.1.0:
  • SQL: task-level operator rollback mechanism
  • DataStream: checkpoint and restore mechanisms for the keyedcoprocess operator
• OmniStateStore 1.1.0:
  • Added support for interconnection with the Flink metric framework to implement some common metrics: Updated Metrics.
  • Added support for persistent storage of priority queues: Added the state.backend.ockdb.timer-service.factory configuration item to control the Flink timer storage location in Table 2.
  • Added support for KV separated storage: In Table 2, added the state.backend.ockdb.kv-separate.switch configuration item to control whether to enable KV separation, and the state.backend.ockdb.kv-separate.threshold configuration item to configure the threshold for enabling KV separation.
• Changed the document name from SQL Query Acceleration Feature Documentation to OmniRuntime Feature Documentation.

2025-09-30

This issue is the twentieth official release.

Released the OmniAdvisor 2.0 feature.

2025-06-30

This issue is the nineteenth official release.

• Added column-based write in Parquet format for Spark 3.3.1.
• Added support for CentOS 7.9.
• Added OmniStream Flink Native. This feature uses native code (C/C++) to implement open source software operators to accelerate the Flink engine execution and improve the search performance.
• Added the OmniStateStore feature, which improves the I/O performance of Flink in big data scenarios.

2025-03-30

This issue is the eighteenth official release.

• Added support for Spark 3.4.3 and Spark 3.5.2 in OmniOperator.
• Added support for Spark 3.4.3 in OmniMV.
• Added the OmniScheduler feature, which leverages the Yarn load scheduling algorithm to resolve cluster resource unbalance across Hadoop cluster nodes.

2024-12-30

This issue is the seventeenth official release.

• Adapted OmniOperator to the Hive engine and added operator and expression details.
• Added support for a minimum of 1 GB memory in the OmniShuffle Memory Fabric. Added support for mistake-proof deployment in External Shuffle Service (ESS) and Remote Shuffle Service (RSS) modes. Added a communication library with higher multi-thread performance in ESS mode.
• Added remote attestation support for OmniShield.
• Added the OmniRuntime public network address statement.

2024-09-30

This issue is the sixteenth official release.

Added the OmniShield feature. This feature provides the data source encryption and decryption functions. Data is computed in the TEE to ensure data security along the entire computing link.

2024-06-30

This issue is the fifteenth official release.

• Added support for Hive 3.1.0 in OmniOperator. The performance of Hive 3.1.0 is improved by 20%.
• Added support for flat-wide table query in the OmniOperator memory optimization. The performance is improved by 30% in the TPC-DS benchmark test of 3 TB data.
• Added support for RSS mode in OmniShuffle. The performance is improved by 10% compared with that of Celeborn.

2024-03-30

This issue is the fourteenth official release.

• Added support for Spark 3.3.1 in OmniOperator to improve performance.
• Added optimized NEON instructions in OmniShuffle, mainly for key read/write processes in the memory.
• Enhanced OmniAdvisor to support Spark applications.

2024-01-11

This issue is the thirteenth official release.

• Added the OmniAdvisor and OmniHBaseGSI features to OmniRuntime.
• OmniAdvisor uses AI algorithms to intelligently tune parameters of Spark and Hive tasks running in online systems throughout the whole process.
• OmniHBaseGSI uses an independent index table to store index data, accelerating conditional queries that include SingleColumnValueFilter.

2024-01-02

This issue is the twelfth official release.

Added the video help page to unveil the OmniRuntime feature.

2023-10-24

This issue is the eleventh official release.

• Upgraded the OmniOperator version to 1.3.0, with performance improvement from 20% to 30% and support for the Parquet format and combination with OmniShuffle.
• Added the OmniShuffle feature, which can be combined with OmniOperator to improve computing performance.

2023-07-12

This issue is the tenth official release.

• Integrated the OmniData, OmniOperator, and OmniMV features into the OmniRuntime Feature Guide.
• The operator pushdown feature is not updated in this version. For details, see Kunpeng BoostKit 22.0.0 OmniData User Guide.

2023-05-05

This issue is the ninth official release.

Resolved OmniOperator trouble tickets and updated the version to 22.0.0.SPC5. For details, see the OmniOperator Release Notes.

2022-12-30

This issue is the eighth official release.

• Added the documents: OmniData Release Notes, OmniData Version Mapping, OmniOperator Release Notes, OmniOperator Version Mapping, OmniCache Release Notes, OmniCache Version Mapping, and OmniCache User Guide.
• Updated the OmniData User Guide. For details, see the OmniData Release Notes.
• Updated the OmniOperator User Guide. For details, see the OmniOperator Release Notes.

2022-12-13

This issue is the seventh official release.

Fixed the vulnerabilities of third-party dependency software.

2022-09-13

This issue is the sixth official release.

Modified parameter names and added parameters for OmniData Hive.

2022-07-18

This issue is the fifth official release.

2022-07-08

This issue is the fourth official release.

• Upgraded HAF to 1.2.0 and removed HAF user dependency by adding its support for common user deployment.
• Added the support for UDFs and data statistics by OmniData and upgraded the OmniData version to 1.3.0.
• Added the OmniOperator User Guide.

2022-04-08

This issue is the third official release.

• Upgraded OmniData to 1.2.0.
• Upgraded the HAF to 1.0.1.

2021-12-31

This issue is the second official release.

Added the support for the HAF and Ceph access by OmniData.

2021-09-30

This issue is the first official release.