Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Generating a Full ScaNN Library

KScaNN depends on the open source ScaNN optimized for Kunpeng to provide full functionality. Therefore, after installing KScaNN, obtain the source code of the ScaNN optimized for Kunpeng to compile a full ScaNN Python installation package and dynamic library files.

Procedure

  1. Obtain the KScaNN software package from GitCode and extract the archive to obtain the binary RPM package.

    The software package structure is as follows.

    1
2
3
    ├── boostsra-kscann-xxxx.aarch64.rpm
├── Kunpeng BoostKit User License Agreement 2.0 (in Chinese).txt
└── Kunpeng BoostKit User License Agreement 2.0.txt

    boostkit-sra_kscann-xxxx.aarch64.rpm contains the header files, dynamic library files, and static library files of KScaNN. xxxx indicates the version of the KScaNN software package.

  2. Install the RPM package.
    1
    rpm -ivh boostsra-kscann-xxxx.aarch64.rpm

    After the installation is complete, the environment variable LD_LIBRARY_PATH is automatically added to /etc/profile, that is, the directories /usr/local/sra_kscann/lib/ and /usr/local/sra_kscann/lib/sve where the KScaNN dynamic and static library files are stored.

    In the preceding command, xxxx indicates the version.

  3. Download the open source ScaNN source code optimized for Kunpeng from Gitee. The tag is v2.2.0. Save the file to a path accessible to a compilation computer, such as /path/to/kscann-build.
    mkdir /path/to/kscann-build && cd /path/to/kscann-build
git clone --branch v2.2.0 --single-branch https://gitee.com/openeuler/sra_scann_adapter.git
  4. Obtain the open source code from the google-research repository, which includes the ScaNN v1.2.10 code. Assume that the code is stored in /path/to/kscann-build.
    cd /path/to/kscann-build
git clone https://github.com/google-research/google-research.git
cd google-research
git reset --hard 03c4c851a28dffe0244c65089e68d6cbf73c730b
cp -r scann/ ../sra_scann_adapter
  5. Integrate the patch file 0001-boostsra-kscann-adapter.patch required for enabling KScaNN in ScaNN v1.2.10.
    cd /path/to/kscann-build/sra_scann_adapter
patch -p1 < 0001-boostsra-kscann-adapter.patch
  6. ScaNN depends on the AVX2KI library. Therefore, perform the following steps to install the AVX2KI RPM package.
    1. Before installing AVX2KI, obtain the software package from Huawei technical support and verify the software package to ensure that it is consistent with the original software package on the website. Decompress the ZIP file to obtain the RPM installation package.

      Before using the software package, read and agree to Kunpeng BoostKit User License Agreement 2.0.

    2. Obtain the verification tool and guide from the Huawei enterprise website.
    3. Verify the installation package integrity by following instructions in the OpenPGP Signature Verification Guide obtained in 6.b.
    4. Install the RPM package of AVX2KI.
      1
      rpm -ivh boostkit-ksl-xxxx.aarch64.rpm

      After the installation is complete, the environment variable LD_LIBRARY_PATH is automatically added to /etc/profile, that is, the /usr/local/ksl/lib directory where the AVX2KI dynamic library file is located.

      In the preceding command, xxxx indicates the version.

    5. Run the source command or log in to the terminal again for the AVX2KI environment variable to take effect.
      1
      source /etc/profile
    6. Check whether the environment variable LD_LIBRARY_PATH contains the AVX2KI installation path /usr/local/ksl/lib.
      1
      env | grep LD_LIBRARY_PATH

      If the variable contains the installation path, the installation is successful.

      After the installation, the target files are generated in the installation path (the default path is /usr/local/ksl), where the include folder contains the AVX2KI header files, and the lib folder contains the AVX2KI dynamic library files.

  7. Install OpenJDK 11.
    1
    yum install java-11-openjdk java-11-openjdk-devel

    Check whether OpenJDK 11 is successfully installed.

    1
    java -version

    If the command output contains "openjdk version 11. x.x", the installation is successful. An earlier OpenJDK version may cause an error during Bazel compilation.

    If the preceding command output shows that there is a version conflict, perform the following steps to configure the environment variable:

    1
    find / -name java
    Configure the environment variable. Assume that the installation directory is /usr/lib/jvm/java-11-openjdk-11.0.23.9-2.oe2203sp3.aarch64.
    1
2
    export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-11.0.23.9-2.oe2203sp3.aarch64
export PATH=/usr/lib/jvm/java-11-openjdk-11.0.23.9-2.oe2203sp3.aarch64/bin:$PATH
  8. Install Bazel 5.4.0.
    1. Download the Bazel 5.4.0 source code. Assume that the path is /path/to/bazel.
      1
      wget https://github.com/bazelbuild/bazel/releases/download/5.4.0/bazel-5.4.0-dist.zip --no-check-certificate
    2. Decompress and compile Bazel.
      1
2
3
      unzip bazel-5.4.0-dist.zip -d bazel-5.4.0
cd bazel-5.4.0
env EXTRA_BAZEL_ARGS="--tool_java_runtime_version=local_jdk" bash ./compile.sh
    3. After the build is successful, the generated executable file bazel is stored in /path/to/bazel/bazel-5.4.0/output. Configure the path in the environment variable PATH for subsequent ScaNN compilation.
      1
      export PATH=/path/to/bazel/bazel-5.4.0/output:$PATH
    4. Perform the verification.
      1
      bazel --version

      If the command output contains "bazel 5.4.0- (@non-git)", the installation is successful.

  9. Install GCC 12.3.1 and GCC-C++ 12.3.1.
    1. Contact Huawei technical support to obtain the GCC.
    2. Install GCC for openEuler according to Installing GCC for openEuler in the GCC for openEuler User Guide.
      export CC=/opt/aarch64/compiler/gcc-12.3.1-2025.06-aarch64-linux/bin/gcc
export CXX=/opt/aarch64/compiler/gcc-12.3.1-2025.06-aarch64-linux/bin/g++
  10. Install the compilation dependencies.
    1
    yum install python python3-devel python3-pip rsync libomp libuuid-devel

    Confirm that KScaNN is in /usr/local/sra_recall/lib/kscann and AVX2KI is in /usr/local/ksl/lib.

  11. Configure a proper pip repository.
    1. Run the following commands to install the dependencies:
      1
2
      cd /path/to/scann
sh project.sh --prepare
    2. Specify the KScaNN installation directory.
      export KSCANN_PATH=/usr/local/sra_kscann
    3. Run the following command to build the Python installation package of ScaNN:
      1
      sh project.sh --build_whl
    4. The resulting Python installation package scann-1.2.10-cp39-cp39-linux_aarch64.whl is located in /path/to/kscann-build/scann.
    5. Run the following command to build the dynamic library file of ScaNN:
      1
      sh project.sh --build_scann_cc_sve
      The resulting dynamic library file libscann_cc.so is located in /path/to/kscann-build/scann.

      project.sh contains the following compilation options. You can select them as required:

      • --prepare: Installs Python dependencies.
      • --build_whl: Builds the full Python installation package of ScaNN.
      • --build_scann_cc_sve: Builds the full ScaNN dynamic library file optimized with SVE instructions.
      • --build_scann_cc_neon: Builds the full ScaNN dynamic library file optimized with NEON instructions.
      • --build_scann_cc_sve_milvus: Builds the full ScaNN dynamic library file optimized with SVE instructions for Milvus scenarios.
      • --build_scann_cc_neon_milvus: Builds the full ScaNN dynamic library file optimized with NEON instructions for Milvus scenario.

      If the message "unable to find valid certification path to requested target" is displayed during environment setup, solve this problem by following instructions in The Certificate Path Cannot Be Found During ScaNN Compilation in the ScaNN Porting Guide.

  12. Install the WHL file.
    1
    pip install scann-1.2.10-cp39-cp39-linux_aarch64.whl
  13. Run the pip show command to check whether ScaNN 1.2.10 has been installed.
    1
    pip show scann

    If the command output contains "Name: scann Version: 1.2.10", the installation is successful.

Uninstalling the RPM Package

If you do not need KScaNN, uninstall it. During uninstallation, service flows that are being executed are affected. You are advised to stop the service flows before uninstalling KScaNN. Using parameters supported by the RPM package manager but not documented in this guide may result in undefined behavior. Proceed with caution.

  1. Run the rpm -e command to uninstall the RPM package.
    1
    rpm -e boostsra-kscann
  2. Confirm that the installation directory /usr/local/sra_kscann is deleted.
  3. Confirm that the /etc/profile file does not contain environment variables related to /usr/local/sra_kscann.

Uninstalling the WHL File

  1. Run the pip uninstall command to uninstall the WHL file.
    1
    pip uninstall scann
  2. Run the pip show command to check whether ScaNN 1.2.10 has been uninstalled.
    1
    pip show scann

    If the command output reads "WARNING: Package(s) not found: scann", the uninstallation is successful.