OmniMV

The OmniRuntime materialized view feature, called OmniMV, is designed for scenarios where SQL analysis tasks in a data warehouse have many identical subqueries. Repeated calculations on these subqueries waste a large number of compute resources and decrease the query efficiency.

OmniMV uses AI algorithms to recommend the optimal materialized view from historical SQL queries, automatically matches SQL statements with a materialized view in Spark, and replaces part of the SQL statement execution plan with the matched materialized view. This feature reduces repeated calculations and increases the query efficiency.

The replacement is implemented using the OmniMV plugin, which performs the following functions:

Pre-calculates and caches batch queries. Compared with the original query from the base table, the OmniMV plugin greatly improves the performance of the computing engine.
Recommends the optimal materialized view using deep learning and reinforcement learning algorithms.

Figure 1 OmniMV principle

OmniMV improves the computing performance of Spark by an average of 30% according to TPC-DS benchmark test cases.

Figure 2 OmniMV TPC-DS test result

OmniMV improves the computing performance of ClickHouse according to Star Schema Benchmark test cases.

Figure 3 OmniMV Star Schema Benchmark test result

Parent topic: Solution Features