鲲鹏社区首页
中文
注册
开发者
我要评分
文档获取效率
文档正确性
内容完整性
文档易理解
在线提单
论坛求助

开启自动调优

鲲鹏自动调优训练模式,执行后可以根据业务场景指标自动调优参数空间里的参数。

命令功能

开始自动调优,任务的参数配置根据业务场景指标自动调优。

命令格式

1
devkit kat train [-h] [-l {0,1,2,3}] -t <file> [-p <file>] [-o <dir>]

参数说明

表1 参数说明

参数

参数选项

说明

-h/--help

-

可选参数,获取帮助信息。

-l/--log-level

0/1/2/3

可选参数,设置日志级别,默认为2。
说明:

新增功能采用更合理设计,默认等级调整为2(WARNING)。

  • 0:日志级别为DEBUG。
  • 1:日志级别为INFO。
  • 2:日志级别为WARNING。
  • 3:日志级别为ERROR。

-t/--task-cfg

-

必选参数,指定任务的配置文件路径。

-p/--param-cfg

-

可选参数,指定应用参数的配置文件路径。

说明:

只开启Kunpeng uarch参数时,-p参数可以不指定,其他情况下必须指定-p参数。

Kunpeng uarch为微架构参数。

-o/--output

-

可选参数,设置任务结果文件输出目录,若不指定则在当前目录生成格式为“train-YMD-HMS”的文件夹,不可指定已存在目录。

使用示例

以PostgreSQL应用为例。

  1. 开启自动调优。
    devkit kat train -t /opt/template/template_PostgreSQL_20250721_091524/task_PostgreSQL.yaml -p /opt/template/template_PostgreSQL_20250721_091524/param_PostgreSQL.yaml -o /opt/template/train

    任务参数已在/opt/template/template_PostgreSQL_20250721_091524/task_PostgreSQL.yaml中配置,应用参数在/opt/template/template_PostgreSQL_20250721_091524/param_PostgreSQL.yaml中配置;结果将以性能数据降序显示。

    返回信息如下:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    [2025-07-22 01:58:00 UTC] [KAT] [message] - ================================== Round Info =================================
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] -         Case        |    Round     | Performance  | Difference   |Improvement (%)
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] -       Baseline      |      0       |    85.61     |      --      |      --      
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] -         Best        |      1       |    115.75    |    30.14     |       35.21  
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] -        Worst        |      5       |    86.97     |     1.36     |        1.59  
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Note:
    [2025-07-22 01:58:00 UTC] [KAT] [message] -     This table compares performance across the baseline, best, and worst rounds. 
    [2025-07-22 01:58:00 UTC] [KAT] [message] - The gap between the best and worst rounds indicates the impact of the parameter 
    [2025-07-22 01:58:00 UTC] [KAT] [message] - space on overall performance.
    [2025-07-22 01:58:00 UTC] [KAT] [message] - ================================== Param Diff =================================
    [2025-07-22 01:58:00 UTC] [KAT] [message] -  Group |   Parameter   |Baseline |  Best   |  Worst  | Importance |    Bound    
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Postgre|  wal_buffers  |   --    | 127736  | 147579  |   4.79%    |[-1, 262144] 
    [2025-07-22 01:58:00 UTC] [KAT] [message] -   SQL  |               |         |         |         |            |             
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Postgre|wal_receiver_ti|   --    |  5948   |  9023   |   4.57%    | [0, 12000]  
    [2025-07-22 01:58:00 UTC] [KAT] [message] -   SQL  |     meout     |         |         |         |            |             
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Postgre|autovacuum_anal|   --    |0.3773385|0.1809016|   4.44%    |[0.001, 0.5] 
    [2025-07-22 01:58:00 UTC] [KAT] [message] -   SQL  |yze_scale_facto|         |945195332|030090553|            |             
    [2025-07-22 01:58:00 UTC] [KAT] [message] -        |       r       |         |         |         |            |             
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Postgre|backend_flush_a|   --    |   190   |   97    |   4.30%    |  [0, 256]   
    [2025-07-22 01:58:00 UTC] [KAT] [message] -   SQL  |     fter      |         |         |         |            |             
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Postgre|maintenance_wor|   --    |   21    |   11    |   4.15%    |   [1, 64]   
    [2025-07-22 01:58:00 UTC] [KAT] [message] -   SQL  |     k_mem     |         |         |         |            |             
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Note:
    [2025-07-22 01:58:00 UTC] [KAT] [message] -     This table shows the parameter differences between the baseline round and the 
    [2025-07-22 01:58:00 UTC] [KAT] [message] - best and worst rounds throughout the training. The table is sorted by parameter 
    [2025-07-22 01:58:00 UTC] [KAT] [message] - importance, displaying the top 5 parameters or parameters whose importance sum 
    [2025-07-22 01:58:00 UTC] [KAT] [message] - exceeds 60%. Items marked with "--" indicate parameter values cannot be obtained. 
    [2025-07-22 01:58:00 UTC] [KAT] [message] - ============================== Auto Tuning Report ==============================
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Total round             : 5
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Total run               : 6
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Fail times              : 0
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Tuning direction        : high
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Algorithm               : HPO
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Performance description : tpmC, transactions per minute
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Parameter groups        : PostgreSQL
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Baseline performance    : 85.610
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Top 10 performance :
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] - |  Rank  |     Round     |  Performance   |  Improvement  (%) |
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] - |   1    |   round  1    |    115.750    |       35.21       |
    [2025-07-22 01:58:00 UTC] [KAT] [message] - |   2    |   round  2    |    103.820     |       21.27       |
    [2025-07-22 01:58:00 UTC] [KAT] [message] - |   3    |   round  3    |     91.310     |        6.66       |
    [2025-07-22 01:58:00 UTC] [KAT] [message] - |   4    |   round  4    |     87.040     |        1.67       |
    [2025-07-22 01:58:00 UTC] [KAT] [message] - |   5    |   round  5    |     86.970     |        1.59       |
    [2025-07-22 01:58:00 UTC] [KAT] [message] - --------------------------------------------------------------------------------
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Note:
    [2025-07-22 01:58:00 UTC] [KAT] [message] - The performance value is the return value of the run test step,
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Performance improvement (%) =
    [2025-07-22 01:58:00 UTC] [KAT] [message] -     (round performance - baseline performance) / baseline performance * 100
    [2025-07-22 01:58:00 UTC] [KAT] [message] - Case package locate: /opt/template/train/train-20250722-015800
    

    自动调优后的参数集生成在“/opt/template/train/train-20250722-015800”目录下,可根据调优结果选择参数集进行应用参数的调整。

    示例中性能指标为tpmC,基线性能为86.610,使用自动调优5轮后,最优性能数据为115.750,较基线性能提升35.21%;并在回显结尾处提供调优结果文件保存目录,可在后续调优结果处理步骤使用。

    在PostgreSQL的数据库性能测试中,tpmC是一个源自TPC-C基准测试的关键性能指标。

  2. 查看自动调优后的参数集。
    ls /opt/template/train/train-20250722-015800

    返回信息如下:

    1
    kat_run.log  param_cfg.yaml  task_cfg.yaml  train.db
    

    该目录中包含执行日志kat_run.log和自动调优后的数据文件train.db,param_cfg.yaml和task_cfg.yaml是复制的模板文件。