Overview

The Milvus database supports Scalable Nearest Neighbors (ScaNN). ScaNN is an open source algorithm library released by Google for efficient vector similarity retrieval. Based on the InVerted File Product Quantization (IVFPQ) principle, ScaNN uses 4-bit SIMD deep optimization on x86 and anisotropic quantization loss function optimization to achieve high retrieval performance. However, the ScaNN algorithm supported by Milvus is not from Google, but extended from IVFPQFastScan in Faiss. According to the performance curve provided by ANN-Benchmarks, the ScaNN algorithm outperforms Faiss-IVFPQFastScan and surpasses Milvus-HNSW in terms of precision. Interconnecting with Google's ScaNN algorithm can significantly improve the query performance.

However, due to the architectural differences of Kunpeng processors, the advantages in hardware-software collaboration of the ScaNN algorithm cannot be fully realized on Kunpeng servers. To address this, the Kunpeng Scalable Nearest Neighbors (KScaNN) optimization feature is introduced to enhance the performance of the ScaNN algorithm on these servers. KScaNN is a vector retrieval algorithm that utilizes inverted indexing and is deeply optimized for the Kunpeng processor architecture in terms of index layout, algorithm flow, and computation flow, with the goal of fully exploiting the processor capabilities. The KScaNN interfaces extend and modify the open source ScaNN interfaces, offering comprehensive retrieval capabilities comparable to those of open source ScaNN.

This feature applies KScaNN as a patch file to the open source Milvus database to provide graph index functionality. For details, see Installation and Usage Description.