Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

test

Command Function

Uses the parameters of a specified round to perform the test again.

Syntax

1
test [h] <n1,n2,...> <n> test round n1,n2,... and test <n> times

You can specify the number of retests for a round. For example, to use the parameters in the first round of automatic tuning for five retests, run the test 1 5 command.

Example

Test twice in the first round.

1
test 1 2

Command output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
[2025-05-19 07:21:42 UTC] [KAT] [message] - ================================================================================
[2025-05-19 07:21:42 UTC] [KAT] [message] -           Kunpeng Auto Tuner - test 1
[2025-05-19 07:21:42 UTC] [KAT] [message] - --------- kunpeng automatic tuning - Step    <round set up>    start -----------
[2025-05-19 07:21:42 UTC] [KAT] [message] - --------- kunpeng automatic tuning - Step   <assign params>    start -----------
[2025-05-19 07:22:03 UTC] [KAT] [message] - --------- kunpeng automatic tuning - Step      <run test>      start -----------
[2025-05-19 07:22:14 UTC] [KAT] [message] - --------- Run test successfully.
[2025-05-19 07:22:14 UTC] [KAT] [message] - --------- Performance     :       61.58
[2025-05-19 07:22:14 UTC] [KAT] [message] - --------- kunpeng automatic tuning - Step  <round tear down>   start -----------
[2025-05-19 07:22:16 UTC] [KAT] [message] - ================================================================================
[2025-05-19 07:22:16 UTC] [KAT] [message] -           Kunpeng Auto Tuner - test 2
[2025-05-19 07:22:16 UTC] [KAT] [message] - --------- kunpeng automatic tuning - Step    <round set up>    start -----------
[2025-05-19 07:22:16 UTC] [KAT] [message] - --------- kunpeng automatic tuning - Step   <assign params>    start -----------
[2025-05-19 07:22:37 UTC] [KAT] [message] - --------- kunpeng automatic tuning - Step      <run test>      start -----------
[2025-05-19 07:22:50 UTC] [KAT] [message] - --------- Run test successfully.
[2025-05-19 07:22:50 UTC] [KAT] [message] - --------- Performance     :        56.2
[2025-05-19 07:22:50 UTC] [KAT] [message] - --------- kunpeng automatic tuning - Step  <round tear down>   start -----------
[2025-05-19 07:22:52 UTC] [KAT] [message] - ================================ Detail Report =================================
[2025-05-19 07:22:52 UTC] [KAT] [message] - Actual test times : 2
[2025-05-19 07:22:52 UTC] [KAT] [message] - Fail times        : 0
[2025-05-19 07:22:52 UTC] [KAT] [message] - --------------------------------------------------------------------------------
[2025-05-19 07:22:52 UTC] [KAT] [message] - |  No.  |  Round  | Status | Performance | Performance(before) |
[2025-05-19 07:22:52 UTC] [KAT] [message] - --------------------------------------------------------------------------------
[2025-05-19 07:22:52 UTC] [KAT] [message] - |   1   |    1    |   S    |    61.58    |       103.32        |
[2025-05-19 07:22:52 UTC] [KAT] [message] - |   2   |    1    |   S    |    56.2     |       103.32        |
[2025-05-19 07:22:52 UTC] [KAT] [message] - --------------------------------------------------------------------------------
[2025-05-19 07:22:52 UTC] [KAT] [message] - ================================  Total Report =================================
[2025-05-19 07:22:52 UTC] [KAT] [message] - Tested rounds     : 1
[2025-05-19 07:22:52 UTC] [KAT] [message] - Tuning Direction : high
[2025-05-19 07:22:52 UTC] [KAT] [message] - Performance description: tpmC, transactions per minute
[2025-05-19 07:22:52 UTC] [KAT] [message] - Baseline performance  : 38.01
[2025-05-19 07:22:52 UTC] [KAT] [message] - --------------------------------------------------------------------------------
[2025-05-19 07:22:52 UTC] [KAT] [message] - |  Round  | Times | Performance | Performance(before) | Change (%) |  CV(%) |
[2025-05-19 07:22:52 UTC] [KAT] [message] - --------------------------------------------------------------------------------
[2025-05-19 07:22:52 UTC] [KAT] [message] - |    1    |   2   |    58.89    |       103.32        |   -43.00   |  4.57  |
[2025-05-19 07:22:52 UTC] [KAT] [message] - --------------------------------------------------------------------------------
[2025-05-19 07:22:52 UTC] [KAT] [message] - WARNING:
[2025-05-19 07:22:52 UTC] [KAT] [message] -     The "Change (%)" in red indicates the result of that round changes
[2025-05-19 07:22:52 UTC] [KAT] [message] - greatly compared with the previous training result. Check that the test
[2025-05-19 07:22:52 UTC] [KAT] [message] - environment is stable.
[2025-05-19 07:22:52 UTC] [KAT] [message] - ================================================================================

Before assigning values to parameters, you can retest the rounds with good performance improvement for multiple times to check whether the tuned parameters are stable. A smaller value of CV(%) indicates higher performance data consistency and more stable parameters in a round.

In a normal test scenario, if the value of CV(%) is less than 1%, the parameter tuning effect is considered stable. If the value is within the range of 1% to 5%, the effect is less stable. If the value is within the range of 5% to 10%, the effect is unstable and you need to determine whether to accept it. If the value exceeds 10%, the tuning effect of the current round is unstable and you are advised to solve the stability problem (caused by environmental or human factors).

Generally, A/B testing is used to ensure stable test results. For example, to retest the result of the round (assume it is round A) whose performance ranks first, you can run test 0,A 5 to alternately test five groups of the baseline round and highest-performing for 10 times.