Compiling Code

Compile all source code to obtain the doris_be binary file. Use the obtained doris_be to replace all the doris_be files in the cluster BEs to gain better performance.

1. Go to the /opt/tools/installed/ directory. Download and decompress the Doris-2.1.2-rc04 source package.

   1 2 3 4

   cd /opt/tools/installed wget https://github.com/apache/doris/archive/refs/tags/2.1.2-rc04.zip mv 2.1.2-rc04.zip ./doris-2.1.2-rc04.zip unzip doris-2.1.2-rc04.zip

2. Download and decompress the pre-compiled third-party library doris-thirdparty-prebuild-arm64.tar.xz and move it to the Doris file directory /thirdparty.

   1 2 3 4 5

   cd /opt/tools/installed wget https://github.com/apache/doris-thirdparty/releases/download/automation/doris-thirdparty-prebuild-arm64.tar.xz tar -xvf doris-thirdparty-prebuild-arm64.tar.xz cp installed/lib64/libjemalloc_doris_pic.a installed/lib64/libjemalloc_arrow.a mv installed /opt/tools/installed/doris-2.1.2-rc04/thirdparty/

3. Download and decompress the third-party source package doris-thirdparty-source.tgz, move it to the Doris file directory /thirdparty, and decompress bitshuffle-0.5.1.tar.gz.

   1 2 3 4 5 6

   cd /opt/tools/installed wget https://github.com/apache/doris-thirdparty/releases/download/automation/doris-thirdparty-source.tgz tar -xzf doris-thirdparty-source.tgz mv src /opt/tools/installed/doris-2.1.2-rc04/thirdparty/ cd /opt/tools/installed/doris-2.1.2-rc04/thirdparty/src tar -xvf bitshuffle-0.5.1.tar.gz

4. Download the Bitshuffle source code and patch, decompress the patch to the source code path, obtain the modified bitshuffle_core.c, and replace thirdparty/src/bitshuffle-0.5.1/src/bitshuffle_core.c with the modified bitshuffle_core.c.
   1. Go to the /opt/tools/installed directory. Download and decompress the Bitshuffle source package.

      1 2 3

      cd /opt/tools/installed wget https://github.com/kiyo-masui/bitshuffle/archive/0.5.1.tar.gz tar -xvf 0.5.1.tar.gz

   2. Download the Bitshuffle patch and move it to the project path of the Bitshuffle source code.

      1 2 3

      cd /opt/tools/installed/ wget https://gitee.com/kunpengcompute/boostkit-bigdata/releases/download/24.0.RC5-doris_neon_optimization/bitshuffle.patch mv bitshuffle.patch /opt/tools/installed/bitshuffle-0.5.1/

   3. Apply the optimized bitshuffle.patch.

      1 2	cd /opt/tools/installed/bitshuffle-0.5.1/ patch -p1 < bitshuffle.patch

   4. Replace thirdparty/src/bitshuffle-0.5.1/src/bitshuffle_core.c with the modified bitshuffle_core.c.

      1	cp /opt/tools/installed/bitshuffle-0.5.1/src/bitshuffle_core.c /opt/tools/installed/doris-2.1.2-rc04/thirdparty/src/bitshuffle-0.5.1/src/bitshuffle_core.c

5. Modify thirdparty/build-thirdparty.sh to make it compile Bitshuffle only.
   1. Open the file.

      1	vi /opt/tools/installed/doris-2.1.2-rc04/thirdparty/build-thirdparty.sh

   2. Press i to enter the insert mode. Comment out lines 1796 to 1869, and add line 1871 to set packages to bitshuffle.

      1	packages=(bitshuffle)

      

      

   3. Press Esc, type :wq!, and press Enter to save the file and exit.
6. Use GCC 12 to compile the third-party Bitshuffle.

   1 2 3

   export DORIS_TOOLCHAIN=gcc cd /opt/tools/installed/doris-2.1.2-rc04/thirdparty sh build-thirdparty.sh

7. Use Clang 17 to compile a BE.
   1. Download the development package of the binutils tool.

      1	yum install binutils-devel.aarch64

   2. Add environment variables.

      1	export DORIS_TOOLCHAIN=clang

   3. Open the build.sh file.

      1 2	cd /opt/tools/installed/doris-2.1.2-rc04 vi build.sh

   4. Press i to enter insert mode. Then, specify the submodule versions on lines 505 and 506 to ensure that the submodules are compatible with the Doris main code.

      update_submodule "be/src/apache-orc" "apache-orc" "https://github.com/apache/doris-thirdparty/archive/d7975612de8b9711a284af9d7cba5248c49f73b3.tar.gz"
      update_submodule "be/src/clucene" "clucene" "https://github.com/apache/doris-thirdparty/archive/ef95e67ae3123409f006072194f742a079603159.tar.gz"

      

   5. Add compilation option --gcc-install-dir=/opt/tools/installed/gcc-12.3.1-2024.05-aarch64-linux/lib64/gcc/aarch64-linux-gnu/12.3.1 to line 550.

      1	-DEXTRA_CXX_FLAGS="${EXTRA_CXX_FLAGS} --gcc-install-dir=/opt/tools/installed/gcc-12.3.1-2024.05-aarch64-linux/lib64/gcc/aarch64-linux-gnu/12.3.1"

      

   6. Press Esc, type :wq!, and press Enter to save the file and exit.
   7. Compile a BE.

      1	STRIP_DEBUG_INFO=ON sh build.sh --clean --be

8. Obtain doris_be compiled by Clang 17.

   After BE compilation is complete, the binary file doris_be is obtained.

   1	cd /opt/tools/installed/doris-2.1.2-rc04/be/output/lib