Introduction

This document describes how to install vLLM on Kunpeng servers, including version compatibility, installation preparations, and basic verification methods.

The unique PagedAttention mechanism of vLLM introduces operating system virtual memory management to KV cache. Combined with continuous batch processing and other technologies, vLLM achieves a 3 to 24-fold increase in throughput in the Hugging Face Transformers benchmark test, while reducing GPU memory usage by more than 40%. Deploying vLLM on Kunpeng servers leverages the native Arm64 architecture compatibility to achieve smooth migration from the x86 platform without any code changes. Additionally, the Kunpeng DevKit and vLLM-Ascend plugin can be used to deeply adapt to Ascend AI processors and implement operator-level fusion and optimization.