SQL 4 (CPU-intensive)

Purpose

SQL 4 is a CPU-intensive scenario. You can optimize Spark executor parameters and adjust I/O parameters for the optimal performance.

Procedure

Based on the actual environment, adjust the number of running cores and memory size specified by Spark-Test-Tool in the configuration file to achieve the optimal performance. For example, for the Kunpeng 920 5220 processor, the following executor parameters are recommended for SQL 4.

• Open the script/spark-default.conf file in the tool directory and add the following configuration items:

  1 2 3 4

  yarn.executor.num 42 yarn.executor.cores 6 spark.executor.memory 15G spark.driver.memory 36G