Introduction to CMF

Project Introduction

This document explains the concept and architecture of the Cache Management Framework (CMF) and provides instructions on how to compile, install, and use it.

CMF is developed based on the Kunpeng hardware platform. It consists of a kernel-space driver and a command line tool. The kernel-space driver interacts with the command line tool through ioctl. The command line tool parses user command line parameters and sends the parsed parameters to the kernel-space driver through the user-space API. Then, the driver verifies the validity of the parameters. After the parameters pass the validity check, the hardware access module queries and sets hardware registers.

CMF modifies hardware registers to control the allocation of system resources such as the L2 cache and L3 cache.

Architecture

Figure 1 CMF architecture

Application Scenarios

Ascend 800I A3 inference servers running on the new Kunpeng 920 processor model may experience slow operator delivery, leading to a high NPU idle rate and a long decoding latency. CMF modifies hardware registers to adjust the L2 cache allocation in the system. This reduces the decoding latency of the Qwen2 1.5B model by 7%.

In addition, CMF provides APIs that allow external applications to interact with the driver to view and modify the hardware resource allocation policies.

Constraints

This function is supported only on physical machines.
The driver must be loaded by the root user.
The L2 I-cache and D-cache must be configured with 2-way or greater associativity.
Power status switching operations are not allowed on driver-occupied cores, as they can cause a reset. These operations include restarting, enabling Local Peripheral Interrupts (LPIs), and activating low-power features in non-high-performance mode.
Hardware registers can be read and written through the command line tool only after the kernel-space driver is successfully loaded.

Directory Structure

cmf/
├── drv                        # Source code directory of the kernel-space driver
│   ├── CMakeListx.txt         # Defining compilation rules and dependencies of the driver module
│   ├── cman_drv.c             # Main logic of the driver, which implements the ioctl interface for communication with the user space
│   ├── cman_drv.h             # Driver header file, which defines the internal data structure and function declaration
│   ├── hw_regs_access.c       # Encapsulating the implementation of read and write operations on hardware registers
│   ├── hw_regs_access.h       # Header file of the hardware access interface, which defines the register operation functions
│   ├── Makefile               # Used with the kernel build system to compile the driver module
│   ├── platform_info.c        # Implementation of obtaining and parsing the current CPU platform information
│   └── platform_info.h        # Platform information interface header file, which defines the platform query functions
├── uapi                       # User-space API and tool source code
│   ├── CMakeListx.txt         # Defining the compilation rules of the user-space library and command line tool
│   ├── cmanuapi.c             # User-space API implementation, which encapsulates the ioctl calling logic
│   ├── cmanuapi.h             # User-space API header file, which is called by external programs
│   └── devcman.cpp            # Main program of the command line tool, which parses parameters and calls APIs
├── docs/en                    # Documentation description
│   ├── release_notes.md       # Release notes
│   ├── introduction.md        # Introduction
│   ├── installation_guide.md  # Installation guide
│   ├── menu_cmf.md            # Documentation guide
│   ├── user_guide.md          # User guide
│   └── best_practices.md      # Best practices
├── CMakeListx.txt             # Project build configuration file
├── LICENSE                    # Open-source license file
├── CC-BY                      # Open-source document license file
└── README_en.md                  # Project description

Release Notes

The CMF release notes include the CMF version and the corresponding hardware and software version requirements. For details, see Release Notes.

Documents

Resource Type	Resource Name	Resource Description
Document	Release Notes	Provides basic information and feature updates of each CMF version.
Document	Installation Guide	Describes how to compile and install CMF.
Document	User Guide	Provides a quick start guide for CMF.
Document	Best Practices	Provides best practices of using CMF.

Environment Requirements

To use CMF smoothly, ensure that your environment is one of the verified environments provided in this section. For details, see Installation Guide.

User Guide

start includes the command parameter description and usage. For details, see User Guide.

Contribution Statement

We welcome your contributions to the community. If you have any questions/suggestions or want to provide feedback on feature requirements and bug reports, you can submit issues. For details, see the contribution guideline. You are also welcome to share insights in the Discussions. Thank you for your support.

License

This project is licensed under GPL-2.0. For details, see LICENSE. The documents of this project are licensed under CC-BY 4.0. For details, see LICENSE.

Acknowledgments

CMF is jointly developed by the following Huawei department:

Kunpeng Computing BoostKit Development Dept Thank you to everyone in the community for your PRs. We warmly welcome contributions to CMF!