AutoFDO

Command Function

Improves performance based on the user's configuration.

Syntax

devkit advisor kfdo [-h | --help] {-i INPUT_CONFIG_PATH | --input INPUT_CONFIG_PATH} [-l {0,1,2,3} | --log-level {0,1,2,3}] [--set-timeout TIMEOUT]

Parameter Description

The tool provides six configuration file samples (all.json, run_sample_opt.json, sample_opt.json, opt.json, run_sample.json, and sample.json). Choose the most sample appropriate to your requirements.

The samples are stored in /root/CLI/DevKit-CLI-xxx-Linux-Kunpeng/advisor/devkitplugins/affinity/tools/compileopt/templates/.

/root/CLI/DevKit-CLI-xxx-Linux-Kunpeng/ is the installation path of the command line tool.

Example

In this example, AutoFDO is executed in the /home/test/redis directory and the config.json file is stored in /home/test/redis. Replace the example directories with the actual ones.

ls -al /home/test /redis/
-rw-r--r--  1 test     test         817 Jul 15 20:43 config.json
-rw-r--r--  1 test      test    4189897 Jul 11 08:59 redis-7.2.5.zip   
devkit advisor kfdo -i /home/test/config.json

The following information is displayed and a report is generated:

['/home/test/affinity/devkitplugins/dist/advisor/devkitplugins/affinity/tools/compileopt/bin/kfdo', '--input', '/home/test/redis/config.json', '-l', '1']
begin to run kfdo tool
[#######...                                                              ]
[running] reading configuration file
[waiting] building baseline software
[waiting] using perf to collect sample profiles
[waiting] using profile to optimize binary program
[##############...                                                       ]
[   ✔   ] reading configuration file
[running] building baseline software
[waiting] using perf to collect sample profiles
[waiting] using profile to optimize binary program
...
[#######################################################################]
[   ✔   ] reading configuration file
[   ✔   ] building baseline software
[   ✔   ] using perf to collect sample profiles
[   ✔   ] using profile to optimize binary program
finish run kfdo tool

config.json File Description

{
    /**
     * The value of the stages parameter is all, run_sample_opt, sample_opt, opt, run_sample, or sample.
     *
     * The all option indicates that all steps will be performed, including step 1 (completing basic compilation), step 2 (running the program), step 3 (collecting program data), and step 4 (optimizing the compilation).
     * The run_sample_opt option indicates that three steps will be performed, including step 2 (running the program), step 3 (collecting program data), and step 4 (optimizing the compilation).
     * The sample_opt option indicates that two steps will be performed, including step 3 (collecting program data) and step 4 (optimizing the compilation).
     * The opt option indicates that only step 4 will be performed, that is, optimizing the compilation.
     * The run_sample option indicates that two steps will be performed, including step 2 (running the program) and step 3 (collecting program data). The profile.afdo file is generated in the ${workspace}/devkit_compiler directory, and is used for afdo_file during compilation optimization.
     * The sample option indicates only step 3 (collecting program data) will be performed. The profile.afdo file is generated in the ${workspace}/devkit_compiler directory, and is used for afdo_file during compilation optimization.
     */
    "stages": "all",
    /**
    * The value of the compiler_type parameter is gcc or bisheng.
     *
     * gcc:
     *    1. In the step of collecting program data, use create_gcov to create the profile.afdo file.
     *    2. In the steps of completing basic compilation and optimizing the compilation, the used compiler is GCC for openEuler or the GCC (7.x to 10.x)
     * bisheng:
     *    1. In the step of collecting program data, use create_llvm_prof to create the profile.afdo file.
     *    2. In the steps of completing basic compilation and optimizing the compilation, the used compiler is the BiSheng Compiler (2.4 to 4.0)
     */
    "compiler_type" : "bisheng",
    // Basic directory for running commands
    "workspace" : "/home/workspace/",
    // Step 1: Complete basic compilation.
    "baseline_build" : {
        // Environment variables
        "env":{
            // Environment variable CC, which indicates the specific compiler command.
            // "CC": "clang"
        },
        // Compilation command. Separate multiple commands with semicolons (;).
       "command" : "cd redis-6.2.14;make"
    },
    // Step 2: Run the program.
    "load_program" : [
        {
            // Environment variables
            "env":{
                // "LD_LIBRARY_PATH": "/home/workspace/local/lib64"
            },
            // Start command. Only one command is supported.
            "command" : "/home/workspace/redis-6.2.14/src/redis-server /home/workspace/redis.conf",
            // Stop command. Only one command is supported. This command is executed after the collection is complete. If this command is not configured, the process group used to start the program will be used to stop the program by default.
            "stop_command":"/bin/bash /home/workspace/stop.sh",
            // Interval between the current time to the time to start the next application, in seconds.
            "interval": 5
        },
        {
            // Application pressure test
            // Environment variables
            "env":{
                // "LD_LIBRARY_PATH": "/home/workspace/local/lib64"
            },
            // Start command. Only one command is supported.
            "command" : "/home/workspace/redis-6.2.14/src/redis-benchmark -t set -n 1000000",
            // Stop command. Only one command is supported. This command is executed after the collection is complete. If this command is not configured, the process group used to start the program will be used to stop the program by default.
            "stop_command":"",
            // Interval between the current time to the time to start the next application.
            "interval": 0
        }
    ],
    // Step 3: Collect program data.
    "perf_sample" : {
        // Feature of the startup command for the sampled program. You can run the ps -e -o pid,command command to query the feature.
        "app_names":["redis-server"],
        // Collection duration, in seconds.
        "duration" : 10,
        // Collection latency, in seconds.
        "delay" : 0,
        // Use create_llvm_prof or create_gcov to generate the binary file required by profile.afdo.
        "create_afdo":{
            "binary":  "/home/workspace/redis-6.2.14/src/redis-server"
        }
    },
    // Step 4: Optimize the compilation.
    "optimize_build" : {
        // Environment variables
        "env":{
            // Environment variable CC, which indicates the specific compiler command.
            //  "CC": "clang"
        },
        // The afdo_file configuration takes effect only when stages is set to opt.
        //     You can run create_llvm_prof or create_gcov to generate the data used to optimize the collection in compilation optimization.
        // "afdo_file": "/home/workspace/devkit_compiler/profile.afdo",
        // Compilation command. Separate multiple commands with semicolons (;).
        "command" : "cd redis-6.2.14_opt;make"
    }
}