Known Issues

DTS2024060329127

Minor

In the scenario where Spark OmniOperator executes INSERT statements and there is only one data partition, when Sort Merge Join is performed in 50 consecutive tables, the SMJ operator calls the new statement to apply for the vector memory when the off-heap memory is used up. As a result, a core dump occurs.

1. OmniOperator uses column-based processing and occupies more memory than native Spark which uses row-based processing. In addition, the resources applied by the SMJ operator during the calculation can only be released after the task is complete.
2. The problem occurs when INSERT statements are executed and there is only one data partition. Spark generates only one task. As a result, Sort Merge Join on 50 tables is executed in one task. In this case, the configured 38 GB off-heap memory is used up by the 50 consecutive SMJ operators during the calculation. When the new statement is used to apply for memory, a core dump occurs.

This problem occurs in a rare scenario. Spark jobs aim to leverage the concurrent processing capability of large clusters. In normal scenarios, the situation where a single task (single thread) executes JOIN for a large number of tables does not exist. This problem has not occurred in actual service scenarios and has little impact on customers.

1. Adjust the value of spark.memory.offHeap.size to increase the off-heap memory and trigger the service again.
2. Roll back to the native Spark process to trigger the service.

1. Add this troubleshooting case to the Feature Guide so that customers can prevent or locate the problem.
2. Resolve this problem in the next commercial release of Kunpeng BoostKit 24.0.0.