BoostSRA
Kunpeng Recall Algorithm Library
A recall algorithm library optimized for the Kunpeng platform. It optimizes the instruction set architecture and memory access mechanism of the Kunpeng processor at the bottom layer, improving the computing efficiency and throughput of the recall algorithm. It is especially suitable for high-concurrency recall scenarios.
- KBest
A proprietary, efficient graph-based search algorithm. It provides the search capability benchmarking against Faiss HNSW.
- KScaNN
An inverted index-based vector retrieval algorithm. It uses the Kunpeng architecture to deeply optimize index layout, algorithmic logic, and computing process to fully unlock the chip potential.
- KVecTurbo
A proprietary vector retrieval acceleration component. It quantifies and compresses high-dimensional vectors to quickly obtain the near neighbors of a query. In addition, KVecturbo uses the SIMD instructions to accelerate distance calculation for multidimensional Nearest Neighbor Search (NNS). It can work with the openGauss vector database.
- KNewPfordelta
An integer compression algorithm optimized by Kunpeng based on the open-source PForDelta algorithm. It is designed for efficient compression and fast decompression of inverted indexes.
- Kunpeng hnswlib
It delivers FP16 support by deeply optimizing the open-source Hierarchical Navigable Small World library (hnswlib) for the Kunpeng Arm platform.
- Kunpeng Faiss
Based on the open-source Faiss library, the IVFFlat, IVFPQ, HNSW, PQFS, and IVFPQFS indexes are optimized through vectorization, dimension-interleaved table lookups, and vector filtering compression to accelerate performance in similarity search and clustering. In addition, FP16 support has been added for the HNSW index type.
- Kunpeng RaBitQ
Intrusive modifications are made to the open-source RaBitQ codebase. This extends its support to the AArch64 architecture, introducing performance optimizations and functional enhancements. The optimizations include FP16 precision optimization, NEON SIMD vectorization, assembly-level Lookup Table (LUT) acceleration, Spilling with Orthogonality-Amplified Residuals (SOAR) spilled vector assignment, and ML-based adaptive nprobe.
- Kunpeng DiskANN
Based on the open-source DiskANN algorithm library, the disk retrieval performance has been enhanced through NEON SIMD vectorization, data layout optimization (adjacency list in-memory), asynchronous I/O pipelining, and reranking queue reduction.
- Embedding Lookup
The Embedding Lookup module within the open-source Monolith large-scale real-time recommendation system has been deeply adapted and optimized. By leveraging key techniques including compiler option tuning, spinlock optimization, memory alignment optimization, and Arm SIMD vectorization, this optimization reduces table lookup latency and boosts online inference performance on the Kunpeng Arm platform.
Kunpeng Retrieval Library
Kunpeng Inference Acceleration Kit
TensorFlow Inference Optimization
- TensorFlow inference optimization
A high-performance inference acceleration extension based on open-source TensorFlow. It focuses on efficient execution in SRA inference scenarios. It significantly improves throughput and cuts latency for model inference through in-depth enhancements in graph optimization, operators, and runtime, providing top performance for AI applications based on Kunpeng CPUs.
- Kunpeng TensorRT-LLM
Based on open-source TensorRT-LLM, it focuses on efficient execution in Large Language Model (LLM) inference scenarios. Through deep performance enhancements including kernel optimization, memory access optimization, and parameter tuning, it increases inference throughput and reduces latency.