Introduction

This document describes how to develop FlashAttention on Kunpeng servers, including source code build, installation, and GPU path verification.

FlashAttention is a Compute Unified Device Architecture (CUDA) acceleration library for Transformer attention computation. It significantly accelerates model training and inference while maintaining equivalent precision. When large language models (LLMs) are deployed on a Kunpeng server, the Ascend AI accelerator in the heterogeneous ecosystem of the server has natively supported and deeply optimized the FlashAttention fusion operator through the Compute Architecture for Neural Networks (CANN) heterogeneous compute architecture.