BiSheng Compiler

Use BiSheng Compiler to perform profile-guided optimization (PGO) on the OceanBase database to improve its execution efficiency and performance. The process includes preparing the OceanBase source code, configuring the compiler, setting optimization options, collecting PGO data, and tuning based on the collected PGO data.

1. Take OceanBase 3.1.5 as an example. Download the OceanBase source code, go to the source code directory, and initialize and compile the source code.

   git clone https://github.com/oceanbase/oceanbase oceanbase-3.1.5_CE
   cd oceanbase-3.1.5_CE
   bash build.sh release --init --make

   

   You can also download the OceanBase 3.1.5 source code from https://github.com/oceanbase/oceanbase/archive/refs/tags/v3.1.5_CE.zip and save it to the target path.

   Wait until the compilation is complete.

2. Download and decompress BiSheng Compiler to a directory, for example, /home.

   Download the source code of BiSheng Compiler 4.0.0 at https://www.hikunpeng.com/developer/devkit/download/bishengcompiler

3. Modify the CMake configuration file to adapt to BiSheng Compiler.
   1. Modify the cmake/Env.cmake file.
      1. Open the cmake/Env.cmake file.

         cd oceanbase-3.1.5_CE
         vim cmake/Env.cmake

      2. Press i to enter the insert mode, set DEVTOOLS_DIR to the path of BiSheng Compiler. Add the path of the Clang header file to BUILD_OPT. In the following configurations, change x.x.x to the actual Clang version.

         ob_define(DEVTOOLS_DIR "/home/BiShengCompiler4.0")
         set(BUILD_OPT "${BUILD_OPT} -I${DEVTOOLS_DIR}lib/clang/x.x.x/include/")

      3. Press Esc, type :wq!, and press Enter to save the file and exit.
   2. Modify the ./deps/easy/CMakeLists.txt and /deps/oblib/src/CMakeLists.txt files.
      1. Open the cmake/Env.cmake file.
      2. Press i to enter the insert mode. Modify line 46 in the ./deps/easy/CMakeLists.txt file and line 27 in the /deps/oblib/src/CMakeLists.txt file. Delete the -Werror option to prevent compilation warnings from being treated as errors.
      3. Press Esc, type :wq!, and press Enter to save the files and exit.
4. Configure compilation options to support PGO.
   1. Open the cmake/Env.cmake file.

      vim cmake/Env.cmake

   2. Press i to enter the insert mode and add optimization options for CMAKE_CXX_FLAGS and CMAKE_C_FLAGS.
      Find CMAKE_CXX_FLAGS and CMAKE_C_FLAGS and set them as follows:

      set(CMAKE_CXX_FLAGS "${BUILD_OPT} -mcpu=tsv110 -fuse-ld=lld -Wno-register -Wl,-q -no-pie -flto=thin -mllvm -enable-aggressive-inline -Wl,-mllvm,-enable-aggressive-inline -mllvm -aggressive-inline-level=3 -Wl,-mllvm,-aggressive-inline-level=3 ")
      set(CMAKE_C_FLAGS "${BUILD_OPT} -mcpu=tsv110 -fuse-ld=lld -Wno-register -Wl,-q -no-pie -flto=thin -mllvm -enable-aggressive-inline -Wl,-mllvm,-enable-aggressive-inline -mllvm -aggressive-inline-level=3 -Wl,-mllvm,-aggressive-inline-level=3 ")

   3. Enable instrumentation compilation of PGO.
      Find CMAKE_CXX_FLAGS and CMAKE_C_FLAGS, add the -fprofile-generate and -Wno-backend-plugin options for them, and specify the path for storing sampling data (for example, /opt/profile). The following is a modification example:

      set(CMAKE_CXX_FLAGS "${BUILD_OPT} -fprofile-generate=/opt/profile -Wno-backend-plugin")
      set(CMAKE_C_FLAGS "${BUILD_OPT} -fprofile-generate=/opt/profile -Wno-backend-plugin")

   4. Press Esc, type :wq!, and press Enter to save the file and exit.
5. Recompile the OceanBase source code to generate the instrumented OceanBase.

   cd oceanbase-3.1.5_CE
   bash build.sh release  --make

   After the compilation is complete, the instrumented OBServer binary file is generated.

6. Profile the performance.
   1. Replace the OBServer in the OceanBase cluster with the instrumented version and start the cluster.
   2. Run a test model, such as sysbench or TPC-C, to test the OceanBase cluster and generate performance data.
   3. Stop the OBServer process and collect the generated profile data files.

      kill `pidof observer`

      After the preceding command is executed, the profile data file (for example, xxx.profraw) is generated.

   4. Use the llvm-profdata tool to merge the xxx.profraw file into the xxx.profdata file.

      llvm-profdata merge xxx.profraw -output=xxx.profdata

7. Use the profile data for optimized compilation.
   Modify the cmake/Env.cmake file in the OceanBase source code.

   1. Open the cmake/Env.cmake file.

      cd oceanbase-3.1.5_CE
      vim cmake/Env.cmake

   2. Press i to enter the insert mode, change the compilation options of CMAKE_CXX_FLAGS and CMAKE_C_FLAGS from -fprofile-generate to -fprofile-use, and specify the path of the xxx.profdata file. The following is a modification example:

      set(CMAKE_CXX_FLAGS "${BUILD_OPT} -fprofile-use=/opt/profile/xxx.profdata -Wno-backend-plugin")
      set(CMAKE_C_FLAGS "${BUILD_OPT} -fprofile-use=/opt/profile/xxx.profdata -Wno-backend-plugin ")

   3. Press Esc, type :wq!, and press Enter to save the file and exit.
8. Recompile OceanBase.

   cd oceanbase-3.1.5_CE
   bash build.sh release  --make

   After the compilation is complete, the optimized OBServer binary file is generated. Replace the OBServer in the OceanBase cluster with the optimized version and start the cluster.