Generating a Complete ScaNN Library

Procedure

1. Download the source code of open source ScaNN. Save the file to a path, for example, /path/to/scann/sra_scann_adapter, that can be accessed by the compilation device.
   Address for obtaining the source package: https://gitee.com/openeuler/sra_scann_adapter. The tag is v1.0.0.

   1	git clone --branch v1.0.0 --single-branch https://gitee.com/openeuler/sra_scann_adapter.git

2. ScaNN depends on the AVX2KI library. Therefore, perform the following steps to install the AVX2KI RPM package.
   1. Before installing AVX2KI, obtain the software package from the official website and verify the software package to ensure that it is consistent with the original software package on the website.

      Obtain the software digital certificate and software package from the Kunpeng community.

      Decompress the BoostKit-ksl_2.4.0.zip file to obtain the RPM installation package.

      Download the software package at the Kunpeng community.

      

      Before using the software package, read and agree to Kunpeng BoostKit User License Agreement 2.0.

   2. Obtain the verification tool and guide at:

      https://support.huawei.com/enterprise/en/tool/pgp-verify-TL1000000054

   3. Verify the installation package integrity by following instructions in the OpenPGP Signature Verification Guide obtained in 2.

      1	rpm -ivh boostkit-ksl-2.4.0-1.aarch64.rpm

      After the installation is complete, the environment variable LD_LIBRARY_PATH is automatically added to /etc/profile, that is, the /usr/local/ksl/lib directory where the AVX2KI dynamic library file is located.

   4. Run the source command or log in to the terminal again for the AVX2KI environment variable to take effect.

      1	source /etc/profile

   5. Check whether the environment variable LD_LIBRARY_PATH contains the AVX2KI installation path /usr/local/ksl/lib.

      1	env | grep LD_LIBRARY_PATH

      If the variable contains the installation path, the installation is successful.

      After the installation, the target files are generated in the installation path (the default path is /usr/local/ksl), where the include folder contains the AVX2KI header files, and the lib folder contains the AVX2KI dynamic library files.

3. Install OpenJDK 11.

   1	yum install java-11-openjdk java-11-openjdk-devel

   Check whether OpenJDK 11 is successfully installed.

   1	java -version

   If the command output contains "openjdk version '11. x.x'", the installation is successful. An earlier OpenJDK version may cause an error during Bazel compilation.

   

   If the preceding command output shows that there is a version conflict, perform the following steps to configure the environment variable:

   1	find / -name java

   Configure the environment variable. Assume that the installation directory is /usr/lib/jvm/java-11-openjdk-11.0.23.9-2.oe2203sp3.aarch64.

   1 2	export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-11.0.23.9-2.oe2203sp3.aarch64 export PATH=/usr/lib/jvm/java-11-openjdk-11.0.23.9-2.oe2203sp3.aarch64/bin:$PATH

4. Install Bazel 5.4.0.
   1. Download the Bazel 5.4.0 source code. Assume that the path is /path/to/bazel.

      1	wget https://github.com/bazelbuild/bazel/releases/download/5.4.0/bazel-5.4.0-dist.zip --no-check-certificate

   2. Decompress and compile Bazel.

      1 2 3

      unzip bazel-5.4.0-dist.zip -d bazel-5.4.0 cd bazel-5.4.0 env EXTRA_BAZEL_ARGS="--tool_java_runtime_version=local_jdk" bash ./compile.sh

   3. After the build is successful, the generated executable file bazel is stored in /path/to/bazel/bazel-5.4.0/output. Configure the path in the environment variable PATH for subsequent ScaNN compilation.

      1	export PATH=/path/to/bazel/bazel-5.4.0/output:$PATH

   4. Perform the verification.

      1	bazel --version

      If the command output contains "bazel 5.4.0- (@non-git)", the installation is successful.

5. Install GCC 12.3.1 and GCC-C++ 12.3.1.

   1 2 3

   yum install gcc-toolset-12-gcc gcc-toolset-12-gcc-c++ export PATH=/opt/openEuler/gcc-toolset-12/root/usr/bin/:$PATH export LD_LIBRARY_PATH=/opt/openEuler/gcc-toolset-12/root/usr/lib64/:$LD_LIBRARY_PATH

6. Install the dependencies.

   1	yum install python python3-devel python3-pip rsync

   Confirm that KScaNN is in /usr/local/sra_recall/lib/kscann and AVX2KI is in /usr/local/ksl/lib.

   Configure a proper pip source and run the following command to build the Python installation package of ScaNN:

   1 2	cd sra_scann_adapter sh project.sh -pb

   The built Python installation package scann-1.2.10-cp39-cp39-linux_aarch64.whl is stored in /path/to/scann/sra_scann_adapter/scann.

   

   The preceding command for building a Python installation package consists of two steps: -p is used to set up the environment, and -b is used to build the package.

   If the message "unable to find valid certification path to requested target" is displayed during environment setup, solve this problem by following instructions in Error "unable to find valid certification path to requested target" Reported When Compiling ScaNN.

7. Install the WHL file.

   1	pip install scann-1.2.10-cp39-cp39-linux_aarch64.whl

   Run the pip show command to check whether ScaNN of version 1.2.10 has been installed.

   1	pip show scann

   If the command output contains "Name: scann Version: 1.2.10", the installation is successful.