Installation Guide

Compiling and Installing hnswlib

Verified Environments

To use hnswlib smoothly and securely, ensure that your environment is one of the verified environments.

Table 1 Verified hnswlib environments

Compilation and Installation

Download the open-source code of hnswlib from GitHub, download the hnswlib patch code from GitCode, and apply the patch.

1. Obtain the open-source code. Assume that the installation path is /path/to.

   cd /path/to
   git clone -b v0.8.0 https://github.com/nmslib/hnswlib.git

2. Download the hnswlib patch file. Assume that the installation path is /path/to.

   git clone https://gitcode.com/boostkit/hnswlib.git hnswlib-patch

3. Apply the patch.

   cd hnswlib
   patch -p1 < ../hnswlib-patch/0001-hnswlib_0.8.0-optimize-fp32_fp16_dis_idsort_prefetch.patch

   The directory structure after the patch is enabled is as follows:

   ├── hnswlib/                  // Core library code directory
   │   ├── bruteforce.h          // Implementation of exhaustive search
   │   ├── hnswalg.h             // Core implementation of the HNSW algorithm
   │   ├── hnswlib.h             // Main header file, defining core APIs
   │   ├── safe_memory.h         // Tool class for memory safe operations
   │   ├── space_ip.h            // Implementation of the IP distance space
   │ ├── space_l2.h            // Implementation of the L2 distance space
   │   ├── stop_condition.h      // Implementation of the stop condition
   │   └── visited_list_pool.h   // Implementation of the visited list pool
   ├── examples/                 // Test code directory
   ├── tests/                    // Test code directory
   ├── python_bindings/          // Python APIs
   ├── .gitignore
   ├── ALGO_PARAMS.md            // HNSW algorithm parameter description
   ├── CMakeLists.txt            // CMake build file
   ├── LICENSE
   ├── Makefile                  // Makefile build file
   ├── MANIFEST.in
   ├── pyproject.toml            // Python project configuration file
   ├── README.md                 # Project description
   ├── setup.py                  // Python project installation script
   └── TESTING_RECALL.md         // Description of the test recall rate

4. Install GCC 12.3.1 and GCC-C++ 12.3.1.

   yum install gcc-toolset-12-gcc gcc-toolset-12-gcc-c++
   export PATH=/opt/openEuler/gcc-toolset-12/root/usr/bin/:$PATH
   export LD_LIBRARY_PATH=/opt/openEuler/gcc-toolset-12/root/usr/lib64/:$LD_LIBRARY_PATH

5. Install OpenMP, CMake, and Make.

   yum install cmake libgomp make

6. Install the HDF5 library.

   yum install hdf5-devel

Verifying Compatibility

This section describes how to verify the compatibility of open-source hnswlib with the Kunpeng platform. This section provides details on how to obtain the open-source hnswlib code, compile the test script, and execute the test script.

Test Source Code Directory

Assume that the directory where the program runs is /path/to/hnswlib. The complete directory structure is as follows:

├── examples         // Function test files
├── hnswlib           // hnswlib header files
├── python_bindings    // Python API files
├── tests             // Function test files
├── ALGO_PARAMS.md
├── CMakeLists.txt     // CMakeLists.txt file for function tests
├── LICENSE
├── Makefile           // Makefile file for performance tests
├── MANIFEST.in
├── pyproject.toml
├── README.md
├── setup.py
└── TESTING_RECALL.md

Compatibility Verification Method

hnswlib is an open-source library that implements the HNSW algorithm. It uses a hierarchical greedy search strategy to quickly locate nearest neighbors and is widely used for efficient approximate nearest-neighbor (ANN) search. The following describes how to verify the compatibility of hnswlib.

1. Download the open-source hnswlib code.

   git clone -b v0.8.0 https://github.com/nmslib/hnswlib.git
   cd /path/to/hnswlib

2. Modify CMakeLists.txt.

   1. Edit CMakeLists.txt.

      vim CMakeLists.txt

   2. Change -march=native to -march=armv8.2-a+dotprod.

   3. Press Esc, type :wq!, and press Enter to save the file and exit.

3. Add a test script.

   vim run.sh

   The content of the script file is as follows:

   rm -rf build
   mkdir build
   cd build
   cmake ..
   make
   cd ../tests/cpp
   python update_gen_data.py
   cd ../../build
   ./epsilon_search_test
   ./example_epsilon_search
   ./example_filter
   ./example_mt_filter
   ./example_mt_replace_deleted
   ./example_mt_search
   ./example_multivector_search
   ./example_replace_deleted
   ./example_search
   ./multiThreadLoad_test
   ./multiThread_replace_test
   ./multivector_search_test
   ./searchKnnCloserFirst_test
   ./searchKnnWithFilter_test
   ./test_updates
   ./test_updates update

4. Run the script to execute the test code.

   sh run.sh

   The following figure shows the test result: