Adapting TensorFlow for KDNN

This section describes how to adapt the TensorFlow matrix operators (MatMul and FusedMatMul, which correspond to the Gemm operator of KDNN) to KDNN. Follow the instructions in this section. Misoperations may introduce undefined behaviors. Exercise caution when performing this operation.

1. Install basic software.

   yum install gcc g++ zip python vim tar wget unzip 

2. Install Bazel by following the instructions in Installing Bazel in the TensorFlow Porting Guide. Bazel is required for TensorFlow compilation. For TensorFlow 2.15.0, download Bazel 6.5.0.
3. Download open source TensorFlow 2.15.0 from GitHub.

   git clone -b v2.15.0 https://github.com/tensorflow/tensorflow.git 

4. Download the optimization patch from GitCode and integrate it into the open source TensorFlow directory.

   git clone -b v2.15.0-2512 https://gitcode.com/boostkit/tensorflow.git sra-tensorflow 
   cp /path/to/sra-tensorflow/0001-boostsra-tensorflow.patch /path/to/tensorflow/ 
   cd /path/to/tensorflow && patch -p1 < 0001-boostsra-tensorflow.patch

5. Install the dependencies.

   yum install patchelf perl python3-devel
   pip3 install numpy==1.24.3
   pip3 install certifi==2023.7.22
   pip3 install requests==2.31.0
   pip3 install grpcio==1.59.0
   pip3 install packaging
   pip3 install wheel
   export C_INCLUDE_PATH=/usr/include/python3.9:$C_INCLUDE_PATH
   export CPLUS_INCLUDE_PATH=/usr/include/python3.9:$CPLUS_INCLUDE_PATH

   In the preceding commands, /usr/include/python3.9 is the directory where Python.h is stored. Replace it with the actual directory in the compilation environment.

6. Prepare the KDNN header file and static library.

   cd /path/to/tensorflow/third_party/KDNN
   cp -r /usr/local/kdnn/include .
   mkdir -p src && cp /usr/local/kdnn/lib/threadpool/libkdnn.a src

7. Go to the KDNN directory and apply the header file patch to enable TensorFlow's exception handling feature.

   patch -p0 < tensorflow_kdnn_include_adapter.patch

8. Return to the TensorFlow root directory and configure the build options. For details about how to configure compilation options, see Installation from Source Code in the TensorFlow Porting Guide.

   cd ../..
   ./configure

9. Start the build.

   export TF_PYTHON_VERSION=3.9
   bazel --output_user_root=../output build -c opt --define=enable_kdnn=true //tensorflow/tools/pip_package:build_pip_package

   ../output indicates the specified build output directory.

10. Install the pip package.

    ./bazel-bin/tensorflow/tools/pip_package/build_pip_package ./output-kdnn
    pip3 install ./output-kdnn/tensorflow-2.15.0-cp39-cp39-linux_aarch64.whl