开启自动调优
鲲鹏自动调优训练模式,执行后可以根据业务场景指标自动调优参数空间里的参数。
命令功能
开始自动调优,任务的参数配置根据业务场景指标自动调优。
命令格式
1 | devkit kat train [-h] [-l {0,1,2,3}] -t <file> [-p <file>] [-o <dir>] |
参数说明
参数 |
参数选项 |
说明 |
---|---|---|
-h/--help |
- |
可选参数,获取帮助信息。 |
-l/--log-level |
0/1/2/3 |
可选参数,设置日志级别,默认为2。
说明:
新增功能采用更合理设计,默认等级调整为2(WARNING)。
|
-t/--task-cfg |
- |
必选参数,指定任务的配置文件路径。 |
-p/--param-cfg |
- |
可选参数,指定应用参数的配置文件路径。 说明:
只开启Kunpeng uarch参数时,-p参数可以不指定,其他情况下必须指定-p参数。 Kunpeng uarch为微架构参数。 |
-o/--output |
- |
可选参数,设置任务结果文件输出目录,若不指定则在当前目录生成格式为“train-YMD-HMS”的文件夹,不可指定已存在目录。 |
使用示例
以PostgreSQL应用为例。
- 开启自动调优。
devkit kat train -t /opt/template/template_PostgreSQL_20250721_091524/task_PostgreSQL.yaml -p /opt/template/template_PostgreSQL_20250721_091524/param_PostgreSQL.yaml -o /opt/template/train
任务参数已在/opt/template/template_PostgreSQL_20250721_091524/task_PostgreSQL.yaml中配置,应用参数在/opt/template/template_PostgreSQL_20250721_091524/param_PostgreSQL.yaml中配置;结果将以性能数据降序显示。
返回信息如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62
[2025-07-22 01:58:00 UTC] [KAT] [message] - ================================== Round Info ================================= [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Case | Round | Performance | Difference |Improvement (%) [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Baseline | 0 | 85.61 | -- | -- [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Best | 1 | 115.75 | 30.14 | 35.21 [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Worst | 5 | 86.97 | 1.36 | 1.59 [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Note: [2025-07-22 01:58:00 UTC] [KAT] [message] - This table compares performance across the baseline, best, and worst rounds. [2025-07-22 01:58:00 UTC] [KAT] [message] - The gap between the best and worst rounds indicates the impact of the parameter [2025-07-22 01:58:00 UTC] [KAT] [message] - space on overall performance. [2025-07-22 01:58:00 UTC] [KAT] [message] - ================================== Param Diff ================================= [2025-07-22 01:58:00 UTC] [KAT] [message] - Group | Parameter |Baseline | Best | Worst | Importance | Bound [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Postgre| wal_buffers | -- | 127736 | 147579 | 4.79% |[-1, 262144] [2025-07-22 01:58:00 UTC] [KAT] [message] - SQL | | | | | | [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Postgre|wal_receiver_ti| -- | 5948 | 9023 | 4.57% | [0, 12000] [2025-07-22 01:58:00 UTC] [KAT] [message] - SQL | meout | | | | | [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Postgre|autovacuum_anal| -- |0.3773385|0.1809016| 4.44% |[0.001, 0.5] [2025-07-22 01:58:00 UTC] [KAT] [message] - SQL |yze_scale_facto| |945195332|030090553| | [2025-07-22 01:58:00 UTC] [KAT] [message] - | r | | | | | [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Postgre|backend_flush_a| -- | 190 | 97 | 4.30% | [0, 256] [2025-07-22 01:58:00 UTC] [KAT] [message] - SQL | fter | | | | | [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Postgre|maintenance_wor| -- | 21 | 11 | 4.15% | [1, 64] [2025-07-22 01:58:00 UTC] [KAT] [message] - SQL | k_mem | | | | | [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Note: [2025-07-22 01:58:00 UTC] [KAT] [message] - This table shows the parameter differences between the baseline round and the [2025-07-22 01:58:00 UTC] [KAT] [message] - best and worst rounds throughout the training. The table is sorted by parameter [2025-07-22 01:58:00 UTC] [KAT] [message] - importance, displaying the top 5 parameters or parameters whose importance sum [2025-07-22 01:58:00 UTC] [KAT] [message] - exceeds 60%. Items marked with "--" indicate parameter values cannot be obtained. [2025-07-22 01:58:00 UTC] [KAT] [message] - ============================== Auto Tuning Report ============================== [2025-07-22 01:58:00 UTC] [KAT] [message] - Total round : 5 [2025-07-22 01:58:00 UTC] [KAT] [message] - Total run : 6 [2025-07-22 01:58:00 UTC] [KAT] [message] - Fail times : 0 [2025-07-22 01:58:00 UTC] [KAT] [message] - Tuning direction : high [2025-07-22 01:58:00 UTC] [KAT] [message] - Algorithm : HPO [2025-07-22 01:58:00 UTC] [KAT] [message] - Performance description : tpmC, transactions per minute [2025-07-22 01:58:00 UTC] [KAT] [message] - Parameter groups : PostgreSQL [2025-07-22 01:58:00 UTC] [KAT] [message] - Baseline performance : 85.610 [2025-07-22 01:58:00 UTC] [KAT] [message] - Top 10 performance : [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - | Rank | Round | Performance | Improvement (%) | [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - | 1 | round 1 | 115.750 | 35.21 | [2025-07-22 01:58:00 UTC] [KAT] [message] - | 2 | round 2 | 103.820 | 21.27 | [2025-07-22 01:58:00 UTC] [KAT] [message] - | 3 | round 3 | 91.310 | 6.66 | [2025-07-22 01:58:00 UTC] [KAT] [message] - | 4 | round 4 | 87.040 | 1.67 | [2025-07-22 01:58:00 UTC] [KAT] [message] - | 5 | round 5 | 86.970 | 1.59 | [2025-07-22 01:58:00 UTC] [KAT] [message] - -------------------------------------------------------------------------------- [2025-07-22 01:58:00 UTC] [KAT] [message] - Note: [2025-07-22 01:58:00 UTC] [KAT] [message] - The performance value is the return value of the run test step, [2025-07-22 01:58:00 UTC] [KAT] [message] - Performance improvement (%) = [2025-07-22 01:58:00 UTC] [KAT] [message] - (round performance - baseline performance) / baseline performance * 100 [2025-07-22 01:58:00 UTC] [KAT] [message] - Case package locate: /opt/template/train/train-20250722-015800
自动调优后的参数集生成在“/opt/template/train/train-20250722-015800”目录下,可根据调优结果选择参数集进行应用参数的调整。
示例中性能指标为tpmC,基线性能为86.610,使用自动调优5轮后,最优性能数据为115.750,较基线性能提升35.21%;并在回显结尾处提供调优结果文件保存目录,可在后续调优结果处理步骤使用。
在PostgreSQL的数据库性能测试中,tpmC是一个源自TPC-C基准测试的关键性能指标。
- 查看自动调优后的参数集。
ls /opt/template/train/train-20250722-015800
返回信息如下:
1
kat_run.log param_cfg.yaml task_cfg.yaml train.db
该目录中包含执行日志kat_run.log和自动调优后的数据文件train.db,param_cfg.yaml和task_cfg.yaml是复制的模板文件。