Constraints
- The user-defined function (
UDF ) plugin supports only simple UDFs. It is used to execute UDFs written based on the Hive UDF framework. - Currently, 64-bit and 128-bit Decimal data types are supported. If the Decimal data exceeds 128 bits, an exception is thrown or null is returned. In this case, an issue that does not match native behaviors of the engine may occur. For example, during SUM or AVG aggregation, if the intermediate result exceeds Decimal 128 bits, the engine native behaviors are normal, but OmniOperator throws an exception or returns null based on the configuration. If AVG calculation is required for a field and the accumulated result may be too large, use other storage types such as Double.
- Different loads require different memory configurations. For example, for a TPC-DS 3 TB dataset, the recommended Spark Extension configuration requires that off-heap memory be greater than or equal to 20 GB so that all the 99 SQL statements can be successfully executed. During the execution, "MEM_CAP_EXCEEDED" may be reported in logs, but the execution result is not affected. If the off-heap memory is insufficient, the SQL execution result may be incorrect.
- Sort, Window, and HashAgg operators support the spill function, whereas BroadcastHash Join, ShuffledHash Join, and SortMerge Join do not support this function.
Parent topic: Feature Overview