Spark Automatic Tuning Example

This section uses Spark as an example to demonstrate an automatic tuning process, from obtaining the template to applying the result. The latency is shortened from 123.132 seconds to 52 seconds under the default Spark configuration, presenting a performance increase of over 130%.

The actual performance increase is subject to factors such as server hardware configuration and other applications.

1. Obtain the template file.

   Select a configuration template file for the application that you want to run.

   1	devkit kat template -g -o /opt/template

   

   The -g parameter displays the interactive user interface. Set the basic parameters required by a template file. The -o /opt/template parameter indicates the path to the generated template file.

2. Convert a simple template file to a custom template file.

   A custom template file contains all task and application commands, which facilitate adjustment of the overall tuning task process.

   1	devkit kat template -c /opt/template/template_Postgresql_20250526_025149/task_Spark_System.json

   Command output:

   1	[2025-05-16 02:56:24 UTC] [KAT] [ info ] - /opt/template/template_Postgresql_20250526_025149/task_Spark_System.json has been converted to /opt/template/template_Postgresql_20250526_025149/task_Spark_System_custom.json

   The custom template file has the following content:

   

3. Set mandatory parameters in the simple template file.

   Set the required parameters in the template file. The simple template file has the following content:

   

   

   In the example, if the IP address and host name are not specified, the local host is used. In the test tool options, 23a is selected as the test case, the database name is tpcds_orc_hive_3, the Spark execution file path is /opt/tools/installed/spark-3.1.1-bin-hadoop3.2/bin, and the Spark deployment mode is yarn. Retain the default values for all parameters in the application parameter template file.

4. Enable automatic tuning.

   Use the completed simple template file to start automatic tuning.

   1	devkit kat train -t /opt/template/template_Postgresql_20250526_025149/task_Spark_System.json -p /opt/template/template_Postgresql_20250526_025149/param_Spark_System.json -o /home/template/DevKit-CLI-x.x.x-Linux-Kunpeng/

   Figure 1 Tuning result summary
   

   

   In the example, the baseline performance is 123.132 seconds. After 50 rounds of automatic tuning, the optimal performance reaches 52.662 seconds, representing a performance increase of 133.82%. The directory for storing the tuning result file is provided at the end of the command output.

5. Check the tuning result.

   The parameter set generated after automatic tuning is generated in the tool directory, which is /home/template/DevKit-CLI-x.x.x-Linux-Kunpeng/train-20250526-152341 in the example. You can select a parameter set based on the tuning result to adjust application parameters.

6. Use the tuning result.

   1	devkit kat use -i /home/template/DevKit-CLI-x.x.x-Linux-Kunpeng/train-20250526-152341

   Go to the interactive user interface.

7. View the performance test data.

   Type show and press Enter to view the automatic tuning statistics.

   Figure 2 Automatic tuning statistics
   

   Type top 5 and press Enter to view the five rounds with the best performance data.

   Figure 3 Top 5 performance data
   
8. Perform a test again.

   Before assigning values to parameters, you can retest the rounds with good performance improvement for multiple times to check whether the tuned parameters are stable. A smaller value of CV(%) indicates higher performance data consistency and more stable parameters in a round. That means you can consider using the parameters of that round to the actual application.

   In a normal test scenario, if the value of CV(%) is less than 1%, the parameter tuning effect is considered stable. If the value is within the range of 1% to 5%, the effect is less stable. If the value is within the range of 5% to 10%, the effect is unstable and you need to determine whether to accept it. If the value exceeds 10%, the tuning effect of the current round is unstable and you are advised to solve the stability problem (caused by environmental or human factors).

   

   Generally, A/B testing is used to ensure stable test results. For example, to retest the result of the round (assume it is round A) whose performance ranks first, you can run test 0,A 5 to alternately test five groups of the baseline round and highest-performing for 10 times.

   Type test r1 10 and press Enter to retest the round whose performance ranks first for 10 times.

   Figure 4 Retest
   
   

   In the example, after the specified round has been tested for 10 times, the final CV(%) value is only 1.96%, indicating that the performance data obtained using the parameters of this round is consistent and the tuned parameters are stable. Therefore, these parameters can be used in the application and system.

9. View the importance of all parameters.

   Type iglobal and press Enter.

   Figure 5 Importance of all parameters
   

   A higher value in the importance(%) column indicates a stronger correlation between the parameter and the tuning effect.

10. View the importance of tuning parameters in the round whose performance ranks first.

    If you are uncertain about the performance data of the round whose performance ranks first, type iround r1 and press Enter to check the importance of parameters in this round.

    Figure 6 Importance of parameters in the round whose performance ranks first
    
11. Use the optimal parameters inferred from parameter importance analysis to perform tests.

    Type ibesttest and press Enter.

    Figure 7 Using the optimal parameters for tests
    
12. Resume automatic training.

    Type train -r 5 and press Enter to perform the next five rounds of automatic tuning on the use screen.

    Figure 8 Resuming automatic training
    
13. Apply the parameters of a specified round to the application and system.

    If you are satisfied with the performance data of the r1 round, type set r1 and press Enter to set the parameters of the first round of automatic tuning to the application and system.

    If the following information is displayed, the value assignment is successful:

    [2025-05-19 07:55:48 UTC] [KAT] [message] - set params of r1 success!

14. Exit the interactive user interface.

    Type exit and press Enter.