Introduction

This document describes how to develop Transformers on Kunpeng servers, including source code build, installation, and basic function verification based on the PyTorch Compute Unified Device Architecture (CUDA) backend.

Transformers provides a unified API and thousands of pre-trained models, covering the entire lifecycle from model loading, training, and fine-tuning to inference and deployment. It natively supports three mainstream deep learning frameworks: PyTorch, TensorFlow, and JAX. Deploying Transformers on Kunpeng servers not only enables deep hardware-based acceleration for core operators such as multi-head attention through the Ascend Compute Architecture for Neural Networks (CANN) ecosystem, but also allows seamless package porting by leveraging the AArch64 precompilation optimization of the open-source community.