Spark Automatic Tuning Example

This section uses Spark as an example to demonstrate an automatic tuning process, from obtaining the template to applying the result. The latency is shortened from 123.132 seconds to 52 seconds under the default Spark configuration, presenting a performance increase of over 130%.

The actual performance increase is subject to factors such as server hardware configuration and other applications.

Obtain the template file.
Select a configuration template file for the application that you want to run.
1

devkit kat template -g -o /opt/template
The -g parameter displays the interactive user interface. Set the basic parameters required by a template file. The -o /opt/template parameter indicates the path to the generated template file.

Convert a simple template file to a custom template file.

A custom template file contains all task and application commands, which facilitate adjustment of the overall tuning task process.

       
            devkit kat template -c /opt/template/template_Spark_20250528_025149/task_Spark.yaml

Command output:

       
            [2025-05-28 02:56:24 UTC] [KAT] [ info  ] - /opt/template/template_Spark_20250528_025149/task_Spark.yaml has been converted to /opt/template/template_Spark_20250528_025149/task_Spark_custom.yaml

The custom template file has the following content:

Set mandatory parameters in the simple template file.
Set the required parameters in the template file. The simple template file has the following content:

In the example, the IP address and host name are not specified, which indicates that the local host is used. A total of 20 rounds of tuning are performed. In the test tool options, 23a is selected as the test case, the database name is tpcds_orc_hive_3, the Spark execution file path is /opt/tools/installed/spark-3.1.1-bin-hadoop3.2/bin, and the Spark deployment mode is yarn. Retain the default values for all parameters in the application parameter template file.
Enable automatic tuning.
Use the completed simple template file to start automatic tuning.
```
devkit kat train -t /opt/template/template_Spark_20250528_025149/task_Spark.yaml -p /opt/template/template_Spark_20250528_025149/param_Spark.yaml 
-o /home/template/DevKit-CLI-x.x.x-Linux-Kunpeng/
```
Figure 1 Automatic tuning result

The Spark performance metric is latency and the baseline performance is 126 seconds. After 20 rounds of automatic tuning, the optimal performance reaches 53 seconds, representing a performance increase of 137%. The directory for storing the tuning result file is provided at the end of the command output.
Check the tuning result.
The parameter set generated after automatic tuning is generated in the tool directory, which is /home/template/DevKit-CLI-x.x.x-Linux-Kunpeng/train-20250520-091026 in the example. You can select a parameter set based on the tuning result to adjust application parameters.

Use the tuning result.

devkit kat use -i /home/template/DevKit-CLI-x.x.x-Linux-Kunpeng/train-20250520-091026

Go to the interactive user interface.

View the function manual.
Type man and press Enter to view the complete introduction to the Kunpeng AutoTuner, or type man -m and press Enter to view the manual directory and select the content that you want to view.

Figure 2 Manual directory
View the performance test data.
Type show and press Enter to view the automatic tuning statistics.

Figure 3 Tuning result

Type show top 5 and press Enter to view the five rounds with the best performance data.

Figure 4 Top 5 performance data
Perform a test again.
Before assigning values to parameters, you can retest the rounds with good performance improvement for multiple times to check whether the tuned parameters are stable. A smaller value of CV(%) indicates higher performance data consistency and more stable parameters in a round. That means you can consider using the parameters of that round to the actual application.

In a normal test scenario, if the value of CV(%) is less than 1%, the parameter tuning effect is considered stable. If the value is within the range of 1% to 5%, the effect is less stable. If the value is within the range of 5% to 10%, the effect is unstable and you need to determine whether to accept it. If the value exceeds 10%, the tuning effect of the current round is unstable and you are advised to solve the stability problem (caused by environmental or human factors).

Generally, A/B testing is used to ensure stable test results. For example, to retest the result of the round (assume it is round A) whose performance ranks first, you can run test 0,A 5 to alternately test five groups of the baseline round and highest-performing for 10 times.

Type test 11 5 and press Enter to retest the round (the 11th round of automatic tuning) that delivers the best performance data for five times.

Figure 5 Retest data

In the example, after the specified round has been tested for five times, the final CV(%) value is only 3.94%, indicating that the performance data obtained using the parameters of this round is consistent and the tuned parameters are stable. Therefore, these parameters can be used in the application and system.
View the importance of all parameters.
Type show importance and press Enter.

Figure 6 Importance of all parameters

A higher value in the importance(%) column indicates stronger correlation between the parameter and the tuning effect.
Use the optimal parameters inferred from parameter importance analysis to perform tests.
Type ibesttest and press Enter.

Figure 7 Using the optimal parameters for tests
Resume automatic training.
Type train -r 5 and press Enter to perform the next five rounds of automatic tuning on the use screen.

Figure 8 Resuming automatic training

Apply the parameters of a specified round to the application and system.

If you are satisfied with the performance data of the 11th round, type set 11 and press Enter to set the parameters of the 11th round of automatic tuning to the application and system.

If the value assignment is successful, the following information is displayed:

       
            [2025-05-19 06:55:48 UTC] [KAT] [message] - set params of round 11 success!

Exit the interactive user interface.
Type exit and press Enter.

Parent topic: Kunpeng AutoTuner