Using the Training Result

Command Function

Uses the report result of automatic tuning.

Syntax

1

devkit kat use [-h] -i <dir> [-l {0,1,2,3}]

Parameter Description

1. You can append r plus a number to a subcommand to specify the round of ranking by performance. For example:

   show r1 # Display the result data of the round whose performance ranks first.

   set r1 # Apply the task parameters of the round whose performance ranks first to the application and system.

   test r1 # Use the parameters of the round whose performance ranks first to perform the performance test again.

2. You can append a single number to a subcommand to indicate a round. For example:

   show 5 # Display the data of the fifth round of automatic tuning.

   set 5 # Apply the task parameters of the fifth round to the application and system.

   test 5 # Use the task parameters of the fifth round to perform the performance test again.

Example

View the automatic tuning result report. (The last command output in Enabling Automatic Tuning shows the result directory.)

1

devkit kat use -i /opt/template/template_xxxx/train-20241219-215721

Go to the interactive user interface.

• Viewing the example

  Type example and press Enter.

  Command output:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66

  Welcome to KAT examples. Here is a simple case help you get quick start of "KAT use". Following are some conceptions you should know. [STEPS] "Use" runs the original training steps Which means it will reuse the steps of "train". Please ensure the cmds is still OK (1),At beginning, Case will be load the same as train. It will cost some time If it is a remote case. (2),If you use "set" or "test" for the first time. Next steps will be called: "initialize" --> "assign_params.backup_cmd" (3),If "set", next steps will be called: "assign_params.assign_cmd" (4),If "test", next steps will be called repeatedly: "assign_params.assign_cmd" --> "round_prepare" --> "run_test" --> "round_cleanup" (5),If "initialize" called , "finalize" will be called when "exit" (6),If you do not want to set a group parameter, please confirm "no" when "exit" to call "assign_params.recover" These are simple principle of invoking the command here. Details refer to "README path and chapter" [ROUND] To make this clear, Here are a trained case with the following data. Total round : 10 Baseline performance: 5.0 -------------------------------------------------------------------------------- | Rank | Round | Performance | Improvement (%) | -------------------------------------------------------------------------------- | 1 | round 5 | 10.0 | 100.00 | | 2 | round 3 | 9.0 | 80.00 | | 3 | round 2 | 8.0 | 60.00 | | 4 | round 4 | 7.0 | 40.00 | | 5 | round 1 | 6.0 | 20.00 | -------------------------------------------------------------------------------- Brief: The train result consisted of rounds. Parameters used in the train process corresponding to each round. Selecting rounds is selecting parameters. Usage: Here are some simple usage of round: (1), Use a single number to represent a round, such as "5" represent line 1 in table above, following command you can use: show 5 # show the information of round 5 set 5 # assign the parameters of round 5 test 5 # retest the performance of round 5 (2), Use r/rank prefix to specify the round ranked by performance. Such as r1 represent line 1 in table above.Follow command you can try: show r1 # show the information of round 5 which ranks 1 set r1 # assign the parameters of round which ranks 1 test r1 # retest the performance of round 5 which ranks 1 Specially, you can specify sequence when use "test". (1), single numbers or rank numbers separated by "," test 1,2,r1,r2 # test sequence: round 1, 2, 5 ,3 (2), Use top or "-" to specify consecutive rounds or ranks: test 1-3,r1-2,top3 # test sequence: round 1, 2, 3, 5, 3, 5, 3, 2 (3), Specify repetitions using an additional number: test 1,2 3 # test sequence: round 1, 2, 1, 2, 1, 2

• Obtaining help information

  Type help and press Enter. You can type a subcommand next to help to obtain the help information about the subcommand.

  1 2 3 4 5 6 7 8 9 10 11 12 13

  Available Commands: example - Show examples for use exit - Exit the program help - Show this help message log - change the log level reset - reset params set - set the config of round n. show - show round <n> information. test - Retest round sequence. test <n> times. top - show top N performance For more usage, please refer to "help <command>". e.g., help test

• Viewing the performance test data

  Type show 1 and press Enter to view the parameters and performance data of the first round of automatic tuning. Type show and press Enter to view the performance data of all rounds.

  Command output:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

  [2024-12-31 07:54:51 UTC] [KAT] [message] - ======================== info of Round 1 =================================== [2024-12-31 07:54:51 UTC] [KAT] [message] - # Performance: 144.9 [2024-12-31 07:54:51 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 07:54:51 UTC] [KAT] [message] - ---------------------- Params --------------------------------- [2024-12-31 07:54:51 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 07:54:51 UTC] [KAT] [message] - Postgresql: [2024-12-31 07:54:51 UTC] [KAT] [message] - autovacuum_vacuum_cost_delay = 96 [2024-12-31 07:54:51 UTC] [KAT] [message] - autovacuum_vacuum_cost_limit = 1994 [2024-12-31 07:54:51 UTC] [KAT] [message] - autovacuum_vacuum_scale_factor = 0.07744139964226633 [2024-12-31 07:54:51 UTC] [KAT] [message] - autovacuum_vacuum_threshold = 714 [2024-12-31 07:54:51 UTC] [KAT] [message] - bgwriter_delay = 2215 ... ... ... [2024-12-31 07:54:51 UTC] [KAT] [message] - parallel_leader_participation = on [2024-12-31 07:54:51 UTC] [KAT] [message] - vacuum_cleanup_index_scale_factor = 0.8960460592061281 [2024-12-31 07:54:51 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 07:54:51 UTC] [KAT] [message] - ---------------------- Run Info -------------------------------- [2024-12-31 07:54:51 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 07:54:51 UTC] [KAT] [message] - Run record: 1 [2024-12-31 07:54:51 UTC] [KAT] [message] - Result: success [2024-12-31 07:54:51 UTC] [KAT] [message] - Performance: 144.9 [2024-12-31 07:54:51 UTC] [KAT] [message] - Run time: 2024-12-31 07:36:24.048227 [2024-12-31 07:54:51 UTC] [KAT] [message] - Finish time: P2024-12-31 07:36:24.048238 [2024-12-31 07:54:51 UTC] [KAT] [message] - [2024-12-31 07:54:51 UTC] [KAT] [message] - ================================================================================

• Viewing the data of the two rounds whose performance ranks first and second

  Type top 2 and press Enter.

  Command output:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

  [2024-12-31 08:27:59 UTC] [KAT] [message] - ============================== Auto Tuning Report ============================== [2024-12-31 08:27:59 UTC] [KAT] [message] - Total round : 5 [2024-12-31 08:27:59 UTC] [KAT] [message] - Total run : 6 [2024-12-31 08:27:59 UTC] [KAT] [message] - Fail times : 0 [2024-12-31 08:27:59 UTC] [KAT] [message] - Tuning Direction : high [2024-12-31 08:27:59 UTC] [KAT] [message] - Performance description: tpmC, transactions per minute [2024-12-31 08:27:59 UTC] [KAT] [message] - Baseline performance : 141.87 [2024-12-31 08:27:59 UTC] [KAT] [message] - Top 2 performance: [2024-12-31 08:27:59 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:27:59 UTC] [KAT] [message] - | Rank | Round | Performance | Improvement (%) | [2024-12-31 08:27:59 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:27:59 UTC] [KAT] [message] - | 1 | round 5 | 174.94 | 23.31 | [2024-12-31 08:27:59 UTC] [KAT] [message] - | 2 | round 1 | 159.93 | 12.73 | [2024-12-31 08:27:59 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:27:59 UTC] [KAT] [message] - Note: [2024-12-31 08:27:59 UTC] [KAT] [message] - The performance value is the return value of the run test step, [2024-12-31 08:27:59 UTC] [KAT] [message] - Performance improvement (%) = [2024-12-31 08:27:59 UTC] [KAT] [message] - (round performance - baseline performance) / baseline performance * 100 [2024-12-31 08:27:59 UTC] [KAT] [message] - Case package locate: /home/kat_show/DevKit-CLI-24.0.T50-Linux-Kunpeng/train-20241231-161557 [2024-12-31 08:27:59 UTC] [KAT] [message] - ================================================================================

• Performing the performance test again for a specified round

  Type test 1 and press Enter.

  

  • When using the test subcommand, you can specify multiple rounds of data and use the parameters of these rounds to perform the performance test again. Separate the rounds using commas (,). For example:

    test 1,2,r1,r2 # Use the first and second rounds of tuning and the two rounds whose performance ranks first and second to perform the performance test again.

  • You can append top or n-m to the test command to specify multiple rounds. For example:

    test 1-3,top3 # The first, second, and third rounds of automatic tuning and the three rounds with the best performance data

  • You can append the specific rounds to the test command and then a space and a number after the specific rounds to specify the number of repeated performance tests for these rounds. For example:

    test 1,2 3 # Repeat the first and second rounds of automatic tuning for three times.

  Command output:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

  [2024-12-31 08:28:35 UTC] [KAT] [message] - ================================================================================ [2024-12-31 08:28:35 UTC] [KAT] [message] - Kunpeng Auto Tuner - task set up [2024-12-31 08:28:35 UTC] [KAT] [message] - start time: 2024-12-31 08:28:35 [2024-12-31 08:28:35 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:28:48 UTC] [KAT] [message] - --------- Kunpeng Auto Tuner - Step <params prepare> start ----------------- [2024-12-31 08:29:47 UTC] [KAT] [message] - ======================== info of Round 1 =================================== [2024-12-31 08:29:47 UTC] [KAT] [message] - # Performance: 159.93 [2024-12-31 08:29:47 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:29:47 UTC] [KAT] [message] - ---------------------- Params --------------------------------- [2024-12-31 08:29:47 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:29:47 UTC] [KAT] [message] - Postgresql: [2024-12-31 08:29:47 UTC] [KAT] [message] - autovacuum_vacuum_cost_delay = 96 [2024-12-31 08:29:47 UTC] [KAT] [message] - autovacuum_vacuum_cost_limit = 1994 [2024-12-31 08:29:47 UTC] [KAT] [message] - autovacuum_vacuum_scale_factor = 0.07744139964226633 [2024-12-31 08:29:47 UTC] [KAT] [message] - autovacuum_vacuum_threshold = 714 ... ... ... [2024-12-31 08:29:47 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:29:47 UTC] [KAT] [message] - ---------------------- Run Info -------------------------------- [2024-12-31 08:29:47 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:29:47 UTC] [KAT] [message] - Run record: 1 [2024-12-31 08:29:47 UTC] [KAT] [message] - Result: success [2024-12-31 08:29:47 UTC] [KAT] [message] - Performance: 159.93 [2024-12-31 08:29:47 UTC] [KAT] [message] - Run time: 2024-12-31 08:19:20.023751 [2024-12-31 08:29:47 UTC] [KAT] [message] - Finish time: P2024-12-31 08:19:20.023762 [2024-12-31 08:29:47 UTC] [KAT] [message] - [2024-12-31 08:29:47 UTC] [KAT] [message] - ================================================================================ [2024-12-31 08:29:47 UTC] [KAT] [message] - --------- Kunpeng Auto Tuner - Step <round set up> start ----------------- [2024-12-31 08:29:47 UTC] [KAT] [message] - --------- Kunpeng Auto Tuner - Step <assign params> start ----------------- [2024-12-31 08:30:20 UTC] [KAT] [message] - --------- Kunpeng Auto Tuner - Step <run test> start ----------------- [2024-12-31 08:30:41 UTC] [KAT] [message] - ### Run test successfully. Performance of this round is [127.89] ### [2024-12-31 08:30:41 UTC] [KAT] [message] - --------- Kunpeng Auto Tuner - Step <round tear down> start ----------------- [2024-12-31 08:30:43 UTC] [KAT] [message] - ================================================================================

• Changing the output log level

  Type log 1 and press Enter. The output log level is changed to INFO, which is valid only for the current login on the interactive user interface.

• Applying the parameters of the first round of automatic tuning to the application and system

  Type set 1 and press Enter.

  Command output:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

  [2024-12-31 08:31:37 UTC] [KAT] [message] - ======================== info of Round 1 =================================== [2024-12-31 08:31:37 UTC] [KAT] [message] - # Performance: 127.89 [2024-12-31 08:31:37 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:31:37 UTC] [KAT] [message] - ---------------------- Params --------------------------------- [2024-12-31 08:31:37 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:31:37 UTC] [KAT] [message] - Postgresql: [2024-12-31 08:31:37 UTC] [KAT] [message] - autovacuum_vacuum_cost_delay = 96 [2024-12-31 08:31:37 UTC] [KAT] [message] - autovacuum_vacuum_cost_limit = 1994 ... ... ... [2024-12-19 13:46:17 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-19 13:46:17 UTC] [KAT] [message] - ---------------------- Run Info -------------------------------- [2024-12-19 13:46:17 UTC] [KAT] [message] - -------------------------------------------------------------------------------- ... ... ... [2024-12-31 08:31:37 UTC] [KAT] [message] - ================================================================================ [2024-12-31 08:31:37 UTC] [KAT] [message] - --------- Kunpeng Auto Tuner - Step <assign params> start ----------------- [2024-12-31 08:32:10 UTC] [KAT] [message] - set params of 1 success!

• Resetting task parameters

  Type reset and press Enter.

  Command output:

  1	[2024-12-31 08:33:11 UTC] [KAT] [message] - --------- Kunpeng Auto Tuner - Step <params recover> start -----------------

• Exiting from the interactive user interface

  Type exit and press Enter.

  Command output:

  1 2 3 4 5 6 7 8

  [2024-12-31 08:35:02 UTC] [KAT] [message] - Finish using /home/kat_show/DevKit-CLI-24.0.T50-Linux-Kunpeng/train-20241231-161557, goodbye! [2024-12-31 08:35:02 UTC] [KAT] [message] - ================================================================================ [2024-12-31 08:35:02 UTC] [KAT] [message] - Kunpeng Auto Tuner - task tear down [2024-12-31 08:35:02 UTC] [KAT] [message] - start time: 2024-12-31 08:35:02 [2024-12-31 08:35:02 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2024-12-31 08:35:02 UTC] [KAT] [message] - --------- Kunpeng Auto Tuner - Step <params recover> start ----------------- [2024-12-31 08:35:06 UTC] [KAT] [message] - --------- Kunpeng Auto Tuner - Step <tear down> start ----------------- [2024-12-31 08:35:07 UTC] [KAT] [message] - ================================================================================