What's New

1

Migrated the CMF to the GitCode open-source platform.

Open-source software packages and documents are provided on GitCode.

Kunpeng BoostKit for AI CMF Feature Documentation

2

Added the data parallel routing plugin vLLM-Router.

• Open-source software packages and documents are provided on GitCode.
• vLLM-Router is a routing plugin based on the vLLM open-source community. It aims to support data parallel deployment and provide high-performance request routing and load balancing capabilities.

Kunpeng BoostKit for AI vLLM-Router Feature Documentation

1

Added the Cache Management Framework (CMF).

CMF is developed based on the Kunpeng hardware platform and consists of a kernel-mode driver and a command line tool. It modifies hardware registers to control the allocation of system resources such as the L2 cache and L3 cache.

Kunpeng BoostKit for AI CMF Feature Documentation

1

Updated the DeepSeek 70B Inference Model Tuning Guide (Kunpeng 920 + Atlas 800I A2 Inference Server).

Updated the MindIE Turbo version to 2.0.RC2 and added the model quantization performance tuning methods.

DeepSeek 70B Inference Model Tuning Guide (Kunpeng 920 + Atlas 800I A2 Inference Server)

1

Added the DeepSeek 70B Inference Model Tuning Guide (Kunpeng 920 + Atlas 800I A2 Inference Server).

Details the deployment procedures for vLLM, vLLM-Ascend, and MindIE Turbo frameworks on Atlas 800I A2 inference servers running on Kunpeng 920 processors, covering both execution and tuning techniques for the DeepSeek 70B model.

DeepSeek 70B Inference Model Tuning Guide (Kunpeng 920 + Atlas 800I A2 Inference Server)

2

Added the DeepSeek 70B Inference Model Tuning Guide (Kunpeng 920 + Atlas 300I Duo Inference Card).

Details the deployment of the DeepSeek 70B model in an environment with Kunpeng 920 and two Atlas 300I Duo inference cards, along with performance tuning procedures.

DeepSeek 70B Inference Model Tuning Guide (Kunpeng 920 + Atlas 300I Duo Inference Card)