Introduction to zstd

Latest Updates

[2026-06-30]: Released the Kunpeng zstd optimized based on open-source zstd 1.5.7, covering the optimization paths for both block and streaming compression/decompression.

Project Introduction

Overview

Zstandard (zstd) is a fast, lossless compression algorithm designed for real-time compression scenarios. It delivers fast compression and decompression while maintaining a competitive compression ratio. zstd provides a C library and a command-line tool, making it suitable for scenarios such as file compression, memory block compression, network streaming, and log archiving.

This repository is a Kunpeng-optimized version of zstd, built upon upstream zstd 1.5.7 for enhanced performance. It focuses on common execution paths, including block compression, block decompression, streaming compression, and streaming decompression. The optimizations do not modify the user-facing APIs, ensuring that applications can still be developed and integrated using the public C APIs of open-source zstd.

Core Modules

The core capabilities of zstd are divided into the following modules:

Block compression module: Compresses contiguous data in memory in a single operation, suitable for data blocks with a known input size.
Block decompression module: Decompresses a complete zstd frame in a single operation, suitable for data with a known or boundable output size.
Context reuse module: Minimizes repetitive allocation overhead by reusing compression or decompression contexts, suitable for high-frequency invocation scenarios.
Streaming compression module: Supports chunked inputs and outputs, suitable for streaming data such as files, networks, and pipes.
Streaming decompression module: Continuously decodes zstd frames based on the input stream and writes the decompressed results into the caller's output buffer.
Error handling and tool interfaces: Provides auxiliary capabilities such as return value error checking, error text conversion, and compression upper-bound estimation.

Core External API Functions

The following are common public APIs of zstd, covering block compression/decompression, context reuse, streaming compression/decompression, and error handling capabilities.

Module	Function/Method	Description
Block compression	`ZSTD_compress`	Compresses an input memory block into a single zstd frame.
Block compression	`ZSTD_compressBound`	Calculates the worst-case output buffer size required for a single compression operation.
Block decompression	`ZSTD_decompress`	Decompresses a complete zstd frame into an output buffer provided by the caller.
Block decompression	`ZSTD_getFrameContentSize`	Reads the original content size from the zstd frame header.
Context reuse	`ZSTD_createCCtx` / `ZSTD_freeCCtx`	Creates and frees a compression context.
Context reuse	`ZSTD_compressCCtx`	Performs block compression using an explicit compression context.
Context reuse	`ZSTD_createDCtx` / `ZSTD_freeDCtx`	Creates and frees a decompression context.
Context reuse	`ZSTD_decompressDCtx`	Performs block decompression using an explicit decompression context.
Streaming compression	`ZSTD_compressStream2`	Uses the streaming API to continuously compress input data.
Streaming decompression	`ZSTD_decompressStream`	Uses the streaming API to continuously decompress input data.
Error handling	`ZSTD_isError`	Determines whether the return value of a zstd API indicates an error.
Error handling	`ZSTD_getErrorName`	Converts an error return value into a human-readable string.

Directory Structure

The project directory structure is as follows:

# Documentation directory
README_EN.md                         # Project introduction
LICENSE                           # BSD license for code
COPYING                           # GPLv2 alternative license description
docs/
├── LICENSE                       # Document license
└── en/
    ├── api_reference.md          # API reference
    ├── installation_guide.md     # Installation guide
    ├── menu_zstd.md              # Document menu
    ├── quick_start.md            # Quick start
    └── release_notes.md          # Release notes

# Code directory (after fetching code from the dev_1.5.7_FOR_KP branch)
zstd/
├── lib/                          # libzstd library source code and public header files
│   ├── zstd.h                    # zstd public C API header file
│   ├── compress/                 # Compression implementation
│   ├── decompress/               # Decompression implementation
│   ├── common/                   # Common data structures and tool functions
│   └── dictBuilder/              # Dictionary training and build capabilities
├── programs/                     # Source code for the zstd command-line tool
├── examples/                     # API usage examples
├── tests/                        # Test cases
├── build/                        # Build configurations for CMake, Meson, etc.
├── contrib/                      # Extension tools and integration examples
└── Makefile                      # Top-level Makefile

Release Notes

For details about the feature changes in each release of the zstd version optimized for Kunpeng, see Release Notes.

Environment Deployment

For details about the compilation environment, dependency obtaining, and installation procedure of Kunpeng zstd, see Installation Guide.

Quick Start

For details about how to install Kunpeng zstd and quickly get started with the zstd command-line interface and C APIs, see Quick Start.

Documentation

Document Name	Description
Release Notes	Provides basic information and feature updates of each release of Kunpeng zstd.
Quick Start	Provides quick start examples for the Kunpeng-optimized zstd command-line interface and C APIs.
API Reference	Provides descriptions and definitions of common public APIs of zstd.
Installation Guide	Provides guidance on how to obtain, compile, and install the zstd code optimized for Kunpeng.

Disclaimer

To zstd Users

This software is intended solely for debugging and development. You are responsible for any risks and should carefully review the following information:
- This code repository contributes to the zstd open-source project solely for performance optimization of certain zstd paths on Kunpeng processors. It strictly adheres to the coding style and methods, as well as security design of the original open-source software. Any vulnerability and security issues of the software shall be resolved by the corresponding upstream communities according to their response mechanisms. Please pay attention to the notifications and version updates released by the upstream communities. The Kunpeng computing community does not assume any responsibility for software vulnerabilities and security issues.
- Data processing and deletion: Users are responsible for managing and deleting any data generated while using this software. You are advised to promptly delete any related data after use to prevent information leaks.
- Data confidentiality and transmission: Users understand and agree not to share or transmit any data generated by this software. Neither the software nor its developers are responsible for any information leaks, data breaches, or other negative consequences.
- User input security: Users are responsible for the security of any commands they enter and for any risks or losses resulting from improper input. The software and its developers are not liable for issues caused by incorrect command usage.
Disclaimer scope: This disclaimer applies to all individuals and entities using this software. By using the software, you acknowledge and accept this statement and assume all risks and responsibilities arising from its use. If you do not agree, please stop using the software immediately.
Before using this software, please read and understand the preceding disclaimer. If you have any questions, contact the developer.

To Data Owners

If you do not want your dataset to be mentioned in the zstd optimization repository, or if you wish to update its description, please submit an issue on GitCode. We will delete or update your description according to your request. Thank you for your understanding and support.

License

zstd is dual-licensed under BSD and GPLv2. For details, see LICENSE and COPYING.
The documents of zstd are licensed under CC-BY 4.0. For details, see LICENSE.

Contribution Statement

We welcome your contributions to the community. If you have any questions/suggestions or want to provide feedback on feature requirements and bug reports, you can submit issues. For details, see Contribution Guideline. You are also welcome to share insights in Discussions. Thank you for your support.

Acknowledgments

Kunpeng zstd is jointly developed by the following Huawei department:

Kunpeng Computing BoostKit Development Dept

Thank you to everyone in the community for your PRs. We warmly welcome contributions to zstd!