Introduction

This document describes how to install and verify FlashAttention on Kunpeng servers.

FlashAttention is a Compute Unified Device Architecture (CUDA) acceleration library for Transformer attention computation. It significantly accelerates model training and inference while maintaining equivalent precision. When large language models (LLMs) are deployed on a Kunpeng server, the Ascend AI accelerator in the heterogeneous ecosystem of the server has natively supported and deeply optimized the FlashAttention fusion operator through the Compute Architecture for Neural Networks (CANN) heterogeneous compute architecture.