Examples

Obtaining Test Code

Obtain the test framework code from the source code obtained in 3. Assume that the source code is stored in /path/to/scann/sra_scann_adapter and the test framework code is stored in /path/to/scann/sra_scann_adapter/ann-benchmarks.

Obtain datasets.

1
2
3

cd /path/to/scann/sra_scann_adapter/ann-benchmarks
mkdir data && cd data
wget http://ann-benchmarks.com/sift-128-euclidean.hdf5 --no-check-certificate

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

├── ann-benchmarks
      ├── data                                                      // Stores the datasets.
            └── sift-128-euclidean.hdf5
      ├── ann_benchmarks
            └── algorithms
                  └── scann
                        └── cpp_test
                              └── config-sift-128-euclidean.config  // Dataset configuration file
            └── test_cpp.sh                                         // Test script
├── scann
            ├── CMakeLists.txt                                      // Compilation configuration file
            ├── eval.cpp                                            // Test code
            └── cmdline.h                                           // Header file of the command line argument parsing library
project.sh                                                             // Compilation script

Procedure

1. Check that libscann_cc.so has been compiled in 12.
2. Install CMake.

   yum install cmake

3. Install the dependencies.

   yum install numactl numactl-devel hdf5 hdf5-devel gtest-devel gcc-toolset-12-libstdc++-static

4. Install the Python dependencies.

   1 2 3

   cd /path/to/scann/sra_scann_adapter/ann-benchmarks pip install -r requirements.txt pip install treelite==4.2.1 tl2cgen

5. Install protobuf.

   wget https://github.com/protocolbuffers/protobuf/archive/refs/tags/v3.21.9.tar.gz --no-check-certificate
   tar -xzf v3.21.9.tar.gz
   cd protobuf-3.21.9
   mkdir build && cd build
   cmake .. -Dprotobuf_BUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/usr/local/protobuf-3.21.9
   make install
   export PATH=/usr/local/protobuf-3.21.9/bin:$PATH
   export LD_LIBRARY_PATH=/usr/local/protobuf-3.21.9/lib:$LD_LIBRARY_PATH

6. Install Abseil.

   wget https://storage.googleapis.com/mirror.tensorflow.org/github.com/abseil/abseil-cpp/archive/fb3621f4f897824c0dbe0615fa94543df6192f30.tar.gz --no-check-certificate
   tar -xvzf fb3621f4f897824c0dbe0615fa94543df6192f30.tar.gz
   cd abseil-cpp-fb3621f4f897824c0dbe0615fa94543df6192f30
   mkdir build && cd build
   cmake .. && make -j
   make install

7. Install Eigen.

   git clone https://gitlab.com/libeigen/eigen.git
   cd eigen
   git checkout 33d0937c6bdf5ec999939fb17f2a553183d14a74
   mkdir build && cd build
   cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/eigen-3.3.7
   make -j && make install

8. Build the executable file.

   cd /path/to/scann/sra_scann_adapter
   sh project.sh --build_eval_cmake_sve

   

   project.sh contains the following compilation options. You can select them as required:

   • --prepare: Installs Python dependencies.
   • --build_eval_cmake_sve: Builds the executable file of the SVE instruction version using libscann_cc.so through cmake.
   • --build_eval_cmake_neon: Builds the executable file of the NEON instruction version using libscann_cc.so through cmake.
   • --build_eval_bazel_sve: Builds the executable file of the SVE instruction version using the source code through bazel.
   • --build_eval_bazel_neon: Builds the executable file of the NEON instruction version using the source code through bazel.
9. Run the test script.

   1	sh test_cpp.sh

   

   During the test, dataset-related parameters (such as index construction and search policy) are controlled by the configuration file. The configuration file path is as follows:

   ./algorithms/scann/cpp_test/config-*.config (The asterisk (*) is a wildcard, covering all configuration files that start with config- and end with .config, such as config-deep-image-96-angular.config.)

   You can adjust the index_save_or_load parameter in the configuration file to control the index processing mode during the test. The details are as follows:

   • save: Builds the search index from scratch and saves it to a specified path, enabling direct loading in subsequent runs.
   • load: Skips the index-building step and reads the saved index file from the specified path for search. This option is applicable to the index reuse verification.
   • Other values (not save or load): Builds the search index from scratch but does not save it. The index is only effective for this process.

   The execution result is as follows: