我要评分
获取效率
正确性
完整性
易理解

Constraints

  • For Spark, Table 1 describes the scenarios and rules in which OmniMV does not allow rewriting or extracting materialized views.
  • For ClickHouse, Table 2 describes the scenarios and rules in which OmniMV does not allow rewriting or extracting materialized views.
Table 1 Scenarios and rules for Spark SQL

Scenario

Rule

Description

Rewriting is not supported.

Basic rules

  • When required data is queried, if the view condition does not match the query condition, rewriting is not supported. For example, when the query condition is c1>=2 and the view condition is c1>2, rewriting is not supported.
  • The In and Like statements require that the query and view conditions be completely matched. If not, rewriting is not supported. For example, when the query condition is in(2,3) and the view condition is in(1,2,3), rewriting is not supported.
  • When a view has been used in the SQL query statements, rewriting is not supported.

Rewriting is not supported.

Join rules

  • Only the Inner-Join type supports rewriting. Other Join types does not support rewriting in most scenarios. For example, the Outer-Join type can be rewritten only when the query condition is the same as the view condition.
  • isValidPlan is used to check whether the logical plan tree of the currently matched query and view meets the minimum unit requirement, which requires that only the following operators be included. If the requirement is not met, rewriting is not supported.
    • LogicalRelation
    • HiveTableRelation
    • Project
    • Filter
    • Join
    • SubqueryAlias

Rewriting is not supported.

Aggregate rules

  • The root node of the logical plan tree in the view must be the Aggregate operator. For example, the root nodes of order by and having are Sort and Filter respectively, and these views do not support rewriting.
  • The query must match the logical plan tree of the view. The root nodes on both sides must be the Aggregate operator. Then, isValidPlan is used to check whether the subtree meets the minimum unit requirement. If the subtree does not meet the requirement, rewriting is not supported.
  • If the query contains aggregate functions that do not exist in the view, rewriting is not supported.
  • The group by field in the query is a subset of the group by field in the view. Only the following aggregate functions (without distinct, except for min and max) can be rolled up. If the condition is not met, rewriting is not supported.
    • SUM
    • MIN
    • MAX
    • COUNT

Materialized views cannot be extracted.

The query logic is too simple or complex.

  • Materialized views cannot be extracted from single-table queries.
  • If multi-layer nested subqueries exist in a query, views can be extracted only from the innermost subqueries, but not from the outer subqueries.
  • Materialized views cannot be extracted from subqueries that contain temporary tables.
Table 2 Scenarios and rules on ClickHouse

Scenario

Rule

Description

Rewriting is not supported.

Engine-defined rules

  • WHERE must be a subset of GROUP BY in the PROJECTION definition.
  • GROUP BY must be a subset of GROUP BY in the PROJECTION definition.
  • SELECT must be a subset of SELECT in the PROJECTION definition.
  • When multiple projections are matched, the one that reads the least partitions is selected.
  • The number of returned data rows is less than the total number of data rows in the base table.
  • The query covers more than half of the partitions.

Materialized views cannot be extracted.

The query logic is too simple or complex.

A subquery for which PROJECTION can be created must be a subquery on a single table. That is, the FROM clause contains only one table and must contain the GROUP BY clause. In other scenarios, PROJECTION cannot be extracted.