Best Practices

This document describes how to test the optimized hnswlib on the Kunpeng platform. The test depends on the Kunpeng optimization patch file 0001-hnswlib_0.8.0-optimize-fp32_fp16_dis_idsort_prefetch.patch.

Testing Example of the Optimized hnswlib

The following example uses the fashion-mnist-784-euclidean.hdf5 dataset.

Install hnswlib as described in Installation Guide.
Obtain a test program. Assume that the path is /path/to/sra_test.
```
git clone https://gitcode.com/openeuler/sra_test.git
```

Create a data folder and obtain the test dataset.

cd /path/to/sra_test
mkdir data && cd data
wget http://ann-benchmarks.com/fashion-mnist-784-euclidean.hdf5 --no-check-certificate

Compile the executable file.

make hnswlib_test          // FP32 data type, using NEON vectorization instructions.
make hnswlib_fp16_test     // FP16 data type, using NEON vectorization instructions.

The complete directory structure is as follows:

├── configs                                                   // Stores configuration files for the algorithm and dataset.
│      └── hnswlib_fp16/
│            └── hnswlib_fp16_fashion-mnist-784-euclidean.config
├── data                                                      // Stores the datasets.
│      └── fashion-mnist-784-euclidean.hdf5
├── include                                                   // Stores header files of the test framework.
├── indexes/
│      └── hnswlib_fp16/                                       // Stores the built index, which needs to be manually created.
│            └── fashion.hnswlib                               // Built index, generated when the executable file hnswlib_fp16_test runs and the save_or_load parameter in the dataset configuration file is set to save.
├── scripts/
│ └── build.sh                                            // Script file
├── src                                                       // Stores source files of the test framework.
├── get_result.py
├── LICENSE
├── Makefile
├── README.md
├── test.sh                                                   // Script for performing performance tests
├── test_multi-numas.sh                                       // Script for performing performance tests on multiple NUMA nodes
├── hnswlib_fp16_test                                         // Executable file generated after compilation
└── hnswlib_test

During the first compilation, as prompted by the command line, enter the hnswlib path and the macros to be enabled. The script automatically saves the corresponding configuration in the build folder. In subsequent compilations, you can directly modify the configuration in the config_hnswlib.sh or config_hnswlib_fp16.sh file under the build folder and run hnswlib_test or hnswlib_fp16_test again.
To compile hnswlib_test, enable the macro -DUSE_NEON.
To compile hnswlib_fp16_test, enable the macros -DUSE_NEON and -DUSE_FP16.

Run the test scripts.
```
sh test.sh hnswlib                // FP32 data type, using NEON vectorization instructions.
sh test.sh hnswlib_fp16           // FP16 data type, using NEON vectorization instructions.
```
The test results are stored in the output folder. Pay attention to the QPS metric in the test results and ensure that the recall rate is greater than 0.99.
- The /configs folder contains the configuration information for different datasets. index_path is the path for saving the index, and save_or_load is the option for building or loading the graph index.
- When save_or_load is set to save, the graph construction mode is used. In this case, the constructed graph index is saved to /index_path.
- When save_or_load is set to load, the loading mode is used. In this case, the graph index is loaded from /index_path for subsequent search.
- During the first run, execute the performance test script in save mode on a single NUMA node and save the graph index.
- In subsequent runs, if the values of k_f and efc do not need to be changed, you can run the script in load mode across all 4 NUMA nodes to obtain performance data.
- If the values of k_f and efc need to be changed, you need to reconstruct the graph in save mode.

Performance Optimization Suggestions

Select a proper data type. When high precision is not required, preferentially use FP16 to reduce memory footprint and improve performance.
Adjust parameters. Adjust the M, efc, and efs parameters. For details, see section Tuning Guide for Performance Test Parameters.
Perform batch queries. Process queries in batches as much as possible to reduce thread creation overhead.

Tuning Guide for Performance Test Parameters

Assume that the path to the configuration file for parameter tuning is /path/to/sra_test/configs. The directory structure of the folder is as follows:

├── configs/
      ├── hnswlib/
      ├── hnswlib_fp16/
            └── hnswlib_fp16_fashion-mnist-784-euclidean.config

The content of the config file is as follows:

# HNSWLIB
k_f = 16
efs = 30
efc = 200
metric = L2
nloop = 3
num_threads = 80
top_k = 10
batch_mode = false
batch_size = 100
save_or_load = save
index_path = indexes/hnswlib_fp16/fashion.hnswlib

Table 1 Parameters to be tuned describes the parameters to be tuned.

Table 1 Parameters to be tuned

Parameter	Description	Tuning Suggestion
k_f	Maximum number of connections per node in an HNSW graph.	A smaller value reduces the index size but may decrease search precision. A larger value improves search precision but increases the index size and construction time.
efs	Number of candidate neighbors per node during a search.	A smaller value accelerates the search speed but may lower search precision. A larger value improves search precision but increases search latency.
efc	Number of candidate neighbors per node during index construction.	A smaller value speeds up index construction but may degrade index quality. A larger value improves index quality but increases construction time. Using a large value (e.g., 200 to 800) during construction achieves better index quality.

Tune the above parameters when save_or_load is set to save. It is recommended that a single NUMA node is used for index construction. The tuning objective is to identify the parameter combination that achieves the highest QPS while maintaining a recall rate greater than 0.99.

(Optional) UT Test

Run the following commands to check whether the compilation and execution environments are normal and whether the algorithm functions correctly.

Create a unit test (UT) execution script.
```
cd /path/to/hnswlib
vim run.sh
```
The content of the run.sh script is as follows:
```
rm -rf build
mkdir build
cd build
cmake ..
make

./distance_calculation_neon_test
```
Paste the preceding content to the script, press Esc, enter :wq, and press Enter to save the file and exit.
Run the UT test.
```
sh run.sh
```