Feature Overview

This document describes the optimization principles, installation, and usage of the vector instruction optimization feature of the Milvus database.

Milvus is an industry-leading vector database with high performance and scalability. It provides powerful data modeling functions. When the Hierarchical Navigable Small World (HNSW) or Scalable Nearest Neighbors (ScaNN) algorithm is used to test the GIST dataset on Milvus, it is found that the hotspot function for calculating the similarity between two vectors accounts for nearly 90% of the CPU usage. Therefore, optimizing this hotspot function is expected to greatly improve the database performance. For such operations that involve a large quantity of loops and simple mathematical calculations, using the SVE instructions and the prefetch can be effective in acceleration.

• SVE

  Scalable Vector Extension (SVE) is an instruction extension developed by Arm to improve the performance of computing-intensive applications using vectorization. Different from the NEON instructions, SVE is scalable in that the length of the vector register is not limited by a fixed length and can be scaled as required. SVE also introduces vector predicate operations, allowing some elements to be computed in the same vector operation. Such flexibility improves code portability and efficiency.

• PF

  Prefetch (PF) is a technology used to enhance the computer system performance by reducing the idle time of a processor when it waits for memory data. By loading data to the cache in advance, prefetch can significantly lower the memory access latency and improve the overall system performance.

SVE and PF together implement the vector instruction optimization feature of the Milvus database. The Milvus-HNSW algorithm improves queries per second (QPS) performance by 20% on the ANN-Benchmarks GIST dataset with a recall value greater than 0.99 and the configuration of 16 vCPUs and 64 GB memory. The Milvus-ScaNN algorithm also improves QPS performance by 20% on the ANN-Benchmarks GIST dataset with a recall value greater than 0.95.