Introduction to TensorRT-LLM

Latest Updates

[2026-06-30]: The TensorRT-LLM optimization patches were released on the Gitcode platform, focuses on efficient execution in Large Language Model (LLM) inference scenarios.

Project Introduction

This project is based on the open-source TensorRT-LLM and focuses on efficient execution in Large Language Model (LLM) inference scenarios. Through deep performance enhancements including kernel optimization, memory access optimization, and parameter tuning, it significantly improves both inference throughput and latency performance.

Directory Structure

tensorrt-llm
├── 001-boostsra-tensorrtllm-1.0.0-optimize_kernel.patch     // TensorRT-LLM patch file
├── LICENSE                                                  // License file
├── README_en.md                                             // Open-source repository introduction
└── docs                                                     // Documentation

Version Description

For details about the Kunpeng TensorRT-LLM version updates, see Release Notes.

Documents

Resource Type	Resource Name	Resource Description
Document	Release Notes	Provides basic information and feature updates of each Kunpeng TensorRT-LLM release.
Document	Feature Introduction	Introduces the key capabilities and performance optimizations of Kunpeng TensorRT-LLM based on the open-source TensorRT-LLM.
Document	Installation Guide	Provides compilation and installation guidance for Kunpeng TensorRT-LLM.

Disclaimer

This code repository contributes to the TensorRT-LLM community. It strictly adheres to the coding style and methods, as well as security design of the native open-source software. Any vulnerability and security issues of the software shall be resolved by the corresponding upstream communities according to their response mechanisms. Please pay attention to the notifications and version updates released by the upstream communities. The Kunpeng computing community does not assume any responsibility for software vulnerabilities and security issues.

License

This project is licensed under Apache License 2.0. For details, see the LICENSE.

The documentation of this project is released under the CC-BY 4.0 license. For details, see the LICENSE.

Contribution Statement

We welcome your contributions to the community. If you have any questions/suggestions or want to provide feedback on feature requirements and bug reports, you can submit issues. For details, see Contribution Guideline. You are also welcome to share insights in Discussions. Thank you for your support.

Acknowledgments

TensorRT-LLM is jointly developed by the following Huawei department:

Kunpeng Computing BoostKit Development Dept

Thank you for every PR from the community. We welcome your contributions to TensorRT-LLM!