Generating a BC File

A bitcode (BC) file is an intermediate representation (IR) binary generated by compiling source code using Low Level Virtual Machine (LLVM). It contains intermediate code for program compilation and is more compact and efficient than the source file. Based on the LLVM framework, the Kunpeng DevKit uses BC files to improve the program analysis efficiency and accuracy.

LLVM is an open source compiler infrastructure that provides a set of tools and libraries for building compilers, interpreters, debuggers, and optimizers of different languages.

During vectorization check, do not specify the CC, CXX, and FC variables when using a project build tool to generate BC files.

To ensure complete functions, you are advised to use the Clang tool provided by the Kunpeng DevKit as the compilation tool.
If the BC file fails to be generated, modify the source code based on the error information in compiler logs or tool logs. For details about how to rectify errors, see Compatibility in the BiSheng Compiler User Guide.

Prerequisites

For projects used to generate BC files, ensure that LLVM can be used locally to compile and generate dynamic library (.so) files or executable files. (C/C++ compilers are Clang and Clang++, whereas the Fortran compiler is Flang.)

Clang is an open source front-end compiler that is used to compile programming languages such as C, C++, Objective C, and Objective C++ into machine code.

Method 1 (Recommended)

Before you start, check that the Affinity Analyzer CLI tool has been installed.

Modify the build file.
- Replace the compilation command: If the build file contains gcc, g++, gfortran, or ld, replace them with gclang, gclang++, gflang, or llvm-link.
  
  llvm-link does not support library files. If the ld command includes a library file, replace the ld command with gclang, gclang++, or gflang based on the programming language.
- Adjust the compile options: Set the tuning level to -O0 and add the compilation option -g -fno-inline-functions.
Use an SSH tool to remotely log in to the Linux CLI.
Create a BC file.
Replace the example parameter values in the command with the ones that suit your requirements.
```
devkit advisor bc-gen -i /home/test -c make -o /home/test/bc_files --set-timeout 10 -l 1
```
- -i: Path of the source code folder for which a BC file is to be generated. This parameter is mandatory.
- -c: Builds source code. This parameter is mandatory.
- -o: Path for storing the generated BC file. You must have the write permission on the path. This parameter is optional.
- --set-timeout: Timeout interval of a task, in minutes. If the execution duration exceeds the timeout interval, the task exits. This parameter is optional.
- -l: Log level, which is 1 by default. This parameter is optional. 0: DEBUG; 1: INFO; 2: WARNING; 3: ERROR.

Method 2

Use the third-party tool gllvm to generate a BC file. gllvm is an LLVM-based compiler toolchain that compiles C/C++ code into BC files. It can simplify the BC file generation process.

The gllvm tool uses the Go programming language and the Clang compiler. Visit https://github.com/SRI-CSL/gllvm/blob/master/README.md to learn about how to use the tool. Version 1.3.0 is recommended.
When using gllvm for build, use the gclang, gclang++, or gflang compiler, and add the compile option -g -fno-inline-functions.

Check that the current system environment meets the requirements for building the program to be tested.

Add the paths of the Clang compiler provided by the Kunpeng DevKit and the compiled gllvm tool to the environment variables.

export PATH=/path/DevKit/devkitplugins/affinity/tools/weakconsistency/staticcodeanalyzer/llvm-tools/bin:$PATH
export PATH=/path/DevKit/devkitplugins/affinity/tools/weakconsistency/staticcodeanalyzer/gllvm-tools:$PATH
export LD_LIBRARY_PATH=/path/DevKit/devkitplugins/affinity/tools/weakconsistency/staticcodeanalyzer/llvm-tools/lib:$LD_LIBRARY_PATH

/path indicates the DevKit installation directory. Replace it with the actual directory.
If the libstdc++ version in the system is earlier than that in /home/devkit/lib, set the following environment variable:
```
export LD_LIBRARY_PATH=/home/devkit/lib:$LD_LIBRARY_PATH
```

Configure gllvm's dependency on Clang and Flang.

export LLVM_CC_NAME="clang"
export LLVM_CXX_NAME="clang++"
export LLVM_FC_NAME="flang"
export LLVM_LINK_NAME="llvm-link"

Optimization level: Change all optimization levels of the test project to -O0.

Build: Specify gclang as the C compiler, gclang++ as the C++ compiler, and gflang as the Fortran compiler. Add the compile option -g -fno-inline-functions. This operation varies according to the build tool and parameters of the test project. Other build steps remain unchanged. The following examples are provided for your reference:

For a project built using a script without specified parameters, such as jemalloc, run the following command to specify the compiler types:

[root@localhost jemalloc-5.2.1]# CC="gclang" CXX="gclang++" CFLAGS="-g -fno-inline-functions" CXXFLAGS="-g -fno-inline-functions" ./autogen.sh
autoconf
./configure --enable-autogen
checking for xsltproc.. /usr/bin/xsltproc
checking	for gcc..	gclang

For a project built using a script with specified parameters, such as incubator-brpc, run the following command to specify the compiler types based on incubator-brpc's parameter requirements:

[root@localhost incubator-brpc-0.9.6]# sh config_brpc.sh --headers=/usr/include --libs=/usr/lib64 --cc="gclang" --cxx="gclang++" CFLAGS="-g -fno-inline-functions" CXXFLAGS="-g -fno-inline-functions"
objcopy: st4rvLWw: Failed to find link section for section 8
objcopy: st4rvLWw: Failed to find link section for section 8

For a project built using a configure script, such as sqlite, run the following command to specify the compiler types:

[root@localhost sqlite-version-3.32.1]# CC="gclang" CXX="gclang++" ./configure CFLAGS="-g -fno-inline-functions" CXXFLAGS="-g -fno-inline-functions"
checking build system type... aarch64-unknown-Linux-gnu
checking host system type... aarch64-unknown-linux-gnu
checking for gcc... gclang

If the compiler type is specified in the project configuration file, you need to change the configuration file. For example, for bwa built using the Make tool, set CC in the first line of the Makefile to gclang, and then run the build command.

Modifying the Makefile:

#CC=                        gcc
CC=                         gclang

bwa:

[root@localhost bwa-0.7.17]# make CFLAGS="-g -fno-inline-functions" CXXFLAGS="-g -fno-inline-functions"
gclang   -c -g -fno-inline-functions -DHAVE_PTHREAD -DUSE_MALLOC_WRAPPERS utils.c -o utils.o
objcopy: stKUUevi: Failed to find link section for section 15
objcopy: stKUUevi: Failed to find link section for section 15
gclang   -c -g -fno-inline-functions -DHAVE_PTHREAD -DUSE_MALLOC_WRAPPERS kthread.c -o kthread.o
objcopy: stzqwREn: Failed to find link section for section 14

Generate a BC file: After the build is complete, generate a dynamic library (.so file) or an executable file, and use the get-bc tool of gllvm to generate a BC file. Take jemalloc as an example:

[root@localhost lib]# ls
libjemalloc.a libjemalloc_pic.a libjemalloc.so libjemalloc.so.2
[root@localhost lib]# get-bc libjemalloc.so.2
Bitcode file extracted to: libjemalloc.so.2.bc.
[root@localhost lib]# ls
libjemalloc.a libjemalloc_pic.a libjemalloc.so libjemalloc.so.2 libjemalloc.so.2.bc
[root@localhost lib]#

Package the BC file.
1. Create a directory.
```
mkdir bcfile
```
2. Copy the BC file to the directory.
```
cp libjemalloc.so.2.bc bcfile/
```
3. Package the BC file.
```
tar -czvf bcfile.tar.gz bcfile
```

Method 3

Use the project building tool to generate BC files, such as Make and CMake. Before the build, modify related configuration files, such as the Makefile.

Replace the compilation command: If the build file contains gcc, g++, gfortran, or ld, replace them with clang, clang++, flang, or llvm-link.
Adjust the compilation options: Set the tuning level to -O0 and add the compilation option -flto -g -fno-inline-functions.

After the preceding operations are complete, build the BC file.

Before building a project, replace the GCC compiler with an LLVM tool.
During the build, replace or add specified compile options to prevent the loss of some instruction information that affects the accuracy of analysis results.
Change the generation target file to a BC file.

For example, use Make to build a project:

Check that the Makefile file exists in the project.
```
[root@localhost test]# ls
main.c Makefile test.c
```

Modify the compile command and replace the compile options.

Original Makefile:

objects=main.o test.o
exe=case

CFLAGS = -O2 -DAM_CPU_NUMBER=96 -DMAX_PARALLEL_NUMBER=1

all: $(objects)

main.o: ./main.c
        gcc $(CFLAGS) -c $< -o main.o
test.o: ./test.c
        gcc $(CFLAGS) -c $< -o test.o

all: $(exe)
case: main.o test.o
        gcc -lpthread main.o test.o -o case

clean:
       rm -f *.o

New Makefile:

objects=main.o test.o
exe=case.bc

CFLAGS = -O0 -flto -g -fno-inline-functions -DAM_CPU_NUMBER=96 -DMAX_PARALLEL_NUMBER=1

all: $(objects)

main.o: ./main.c
        clang $(CFLAGS) -c $< -o main.o
test.o: ./test.c
        clang $(CFLAGS) -c $< -o test.o

all: $(exe)
case.bc: main.o test.o
        llvm-link main.o test.o -o case.bc

clean:
       rm -f *.o

In the new Makefile, the build tool is changed to Clang, the target file link and generation tool is changed to llvm-link.

Complete the project build.

[root@localhost test]# make
clang -O0 -flto -g -fno-inline-functions -DAM_CPU_NUMBER=96 -DMAX_PARALLEL_NUMBER=1 -c main.c -o main.o
clang -O0 -flto -g -fno-inline-functions -DAM_CPU_NUMBER=96 -DMAX_PARALLEL_NUMBER=1 -c test.c -o test.o
llvm-link main.o test.o -o case.bc
[root@localhost test]# ls
case.bc main.c main.o Makefile test.c test.o

The generated BC file is case.bc.

Package the BC file.
1. Create a directory.
```
mkdir bcfile
```
2. Copy the BC file to the directory.
```
cp case.bc bcfile/
```
3. Package the BC file.
```
tar -czvf bcfile.tar.gz bcfile
```

Parent topic: Common Operations