What's New

1

Migrated documents.

Migrated the documents for the basic acceleration features and application acceleration features to the Kunpeng BoostKit BoostSDS documentation site.

Kunpeng BoostKit BoostSDS Documentation

1

Updated the document.

Enhanced the performance of RocksDB 6.1.2.

Metadata Acceleration Feature Guide

1

Updated the documents.

Adapted software to enhance the reliability and maintainability.

• BoostIO Feature Guide
• BoostIO API Reference
• BoostIO Release Notes

1

Removed some documents.

This feature is no longer iterated or maintained, and related documents are removed.

Global Cache

1

Updated the document.

Supports RocksDB hardware offload.

Metadata Acceleration Feature Guide

2

Updated the document.

Extracted EC object metadata in Ceph object storage to reduce the proportion of object metadata.

Ceph Object Storage Tuning Guide

3

Added the SPDK I/O Acceleration Feature Guide.

Uses the SPDK technology to accelerate I/O flushing in Ceph all-flash storage and reduce Ceph I/O latency.

SPDK I/O Acceleration Feature Guide

4

Added SPDK deployment instructions during Ceph deployment in a container.

Applied a plugin to the Ceph network framework AsyncMessage to support the UCX network framework, which enables full RDMA in Ceph all-flash storage.

Ceph Object Storage Deployment Guide (Ubuntu 22.04 + openEuler 22.03 + Ceph 17.2.8) (Container)

1

Updated documents.

Added options in the configuration file. Users can enable the bcache QoS policy control, writeback policy control, and GC policy control functions of the smart write cache as required.

• Smart Write Cache Feature Guide
• Smart Write Cache Release Notes

2

Updated documents.

• Optimized encoding and decoding at the 6+3 ratio and 64-byte strip size (version 2).
• Supports zstd for compressing the metadata of 10 billion objects in Ceph object storage. This algorithm is optimized based on zstd level 5.

• KSAL Developer Guide
• KSAL Release Notes

3

Added the EC Turbo Tuning Guide.

Added Ceph all-flash storage support for EC Turbo.

EC Turbo Tuning Guide

4

Added the HDFS Tuning Guide.

Added KSAL EC algorithm support for HDFS.

HDFS Tuning Guide

1

Added documents about the RDMA network acceleration feature.

The Ceph RDMA network acceleration feature is released. By applying the UCX RDMA network framework in Ceph, the RDMA capability is available in Ceph all-flash storage.

• RDMA Network Acceleration Feature Guide (Container)
• RDMA Network Acceleration Feature Guide (Physical Machine)
• RDMA Network Acceleration Release Notes

2

Added BoostIO documents.

The BoostIO feature is released. It uses memory and drive resources on the compute side to build distributed multi-tier cache. The write cache improves service write performance and data reliability, and the read cache improves the read cache hit ratio, thus improving service read performance.

• BoostIO Feature Guide
• BoostIO API Reference
• BoostIO Release Notes

3

Added documents about the KAE-enabled SPDK feature.

As the virtual device layer, the SPDK block device (bdev) interconnects with underlying virtual and physical devices. By enabling KAE compression, encryption, and decryption in the bdev, all SPDK devices can be supported.

KAE-enabled SPDK Feature Guide

4

Updated documents.

The EC algorithm of the Kunpeng Storage Acceleration Library (KSAL) supports encoding and decoding at K+1, K+2, K+3, K+4 (2 ≤ K ≤ 25), and 28+3 ratios and 64- and 4096-byte strip sizes.

• KSAL Developer Guide
• KSAL Release Notes

5

Updated documents.

The SSD fault prediction and slow drive detection capabilities are added to the Kunpeng Storage Maintenance Library (KSML).

• KSML Developer Guide
• KSML Release Notes

6

Removed some documents.

This feature is no longer iterated or maintained, and related documents are removed.

SDS KAE MD5 digest algorithm

7

Removed some documents.

This feature is no longer iterated or maintained, and related documents are removed.

SDS KAE zlib compression

8

Removed some documents.

This feature is no longer iterated or maintained, and related documents are removed.

SDS Smart I/O Prefetch User Guide

1

Added Kunpeng Storage Maintenance Library (KSML) documents.

KSML provides HDD fault prediction and slow drive detection algorithms to predict storage media exceptions in advance and improve storage system stability.

• KSML Developer Guide
• KSML Release Notes

2

Updated documents.

The Kunpeng Storage Acceleration Library (KSAL) supports 8+2/8+3/25+4 EC encoding, delivering 50%+ higher encoding performance than open-source versions.

• KSAL Developer Guide
• KSAL Release Notes

1

Updated documents.

The EC algorithm in the Kunpeng Storage Acceleration Library (KSAL) supports encoding and decoding at a stripe size of 64 bytes. The KSAL software package name and the link for obtaining the software package are updated.

• KSAL Release Notes
• KSAL Developer Guide

1

Added the KSAL documents.

The KSAL feature is released. KSAL is a Huawei-developed storage algorithm library that uses algorithms optimized based on Kunpeng instead of mainstream open-source algorithms to improve storage performance. Currently, it consists of the EC, CRC16 T10DIF, and CRC32C algorithms.

• KSAL Release Notes
• KSAL Developer Guide

1

Added the Global Cache client documents.

Added the Global Cache client documents for all-flash scenarios.

• Global Cache Client Deployment Guide
• Global Cache Client Tuning Guide

2

Updated documents.

Restructured the Global Cache Feature Guide and Semi-automatic Compilation and Deployment Guide into the Feature Guide and Deployment Guide.

• Global Cache Deployment Guide
• Global Cache Feature Guide

1

Updated documents.

Fixed the security vulnerabilities related to the third-party component OpenSSL, c-ares, and gRPC.

• Global Cache Patch Release Notes
• Global Cache Feature Guide
• Global Cache Semi-Automatic Compilation and Deployment Guide

1

Updated documents.

Fixed the security vulnerabilities related to the third-party component OpenSSL used in Global Cache.

• Global Cache Patch Release Notes
• Global Cache Feature Guide
• Global Cache Semi-Automatic Compilation and Deployment Guide

1

Updated documents.

Supports all-flash models.

• Global Cache Feature Guide
• Global Cache Tuning Guide

1

Updated the document.

Fixed the Global Cache problem that the write performance in a scenario with the block size being 128 KB or 256 KB does not meet the requirement.

Global Cache Feature Guide

1

Updated documents.

Updated documents related to I/O passthrough, compression algorithms, smart write cache, and global cache.

• Global Cache Feature Guide
• I/O Passthrough Tool User Guide
• Smart Write Cache Feature Guide
• Compression Algorithm Feature Guide

1

Added Kunpeng BoostKit for SDS documents of earlier versions.

Added the Version Mapping, guide pages for historical versions, and documents of historical versions.

Guide pages for historical versions

1

Updated SDS Global Cache documents.

• Added typical configurations of the Global Cache server and client in specific scenarios and a serviceability sub-feature (capacity expansion) in the feature list to the Technical White Paper.
• Added "Updating the TLS Certificate", "For the globalcache Process", "For the ccm_main Process", "Index CLI Commands", and "Process D Status Detection" to the Feature Guide, and adjusted some troubleshooting cases.
• Added "Disabling Ceph BlueFS Buffer Read" to the Tuning Guide.
• Modified test case IDs in the Acceptance Test Guide and added a single image performance test case and the test records.

Kunpeng BoostKit for SDS Global Cache Documentation

2

Added the Global Cache Semi-Automatic Compilation and Deployment Guide.

This document supplements the Feature Guide. It describes how to simplify software compilation, Global Cache installation, and Global Cache server and client deployment. You can use the provided scripts to complete semi-automatic software compilation and install and deploy software on the server and client nodes.

Global Cache Semi-Automatic Compilation and Deployment Guide

1

Integrated documents of the same components and features.

Documents with repeated contents are optimized and simplified to provide better reading experience and more convenient guidance.

• Ceph 14.2.8 Porting Guide (CentOS 7.6 & openEuler 20.03)
• Ceph-Ansible Deployment Guide (CentOS 7.6 & openEuler 20.03)
• Ceph Block Storage Deployment Guide (CentOS 7.6 & openEuler 20.03)
• Ceph Object Storage Deployment Guide (CentOS 7.6 & openEuler 20.03)
• Ceph File Storage Deployment Guide (CentOS 7.6 & openEuler 20.03)
• Data Compaction Feature Guide
• Smart Write Cache Feature Guide
• Bcache User Guide
• EC Turbo Feature Guide

2

Optimized the document structure.

The installation, deployment, tuning, and feature guides are displayed by basic acceleration feature, application acceleration feature, and open source enablement, facilitating locating of required documents.

All documents of the solution

1

Added the open-source BeeGFS deployment guide.

BeeGFS is both a network file system and a parallel file system, complying with the POSIX standard. The client communicates with the storage server via network. BeeGFS supports the native verbs interface and TCP/IP-based connection or connection via RDMA-capable networks like InfiniBand (IB), RDMA over Converged Ethernet (RoCE), and Omni-Path (OPA). By increasing the number of servers in the system, BeeGFS aggregates their capacity and performance in a single namespace.

BeeGFS 7.2 Deployment Guide (CentOS 7.6)

2

Added the open-source FastDFS deployment guide.

FastDFS is an open-source high-performance distributed file system (DFS). It provides functions including file storage, file synchronization, file access, high capacity, and load balancing. It is mainly used to store massive data and is especially suitable for online services based on small- and medium-sized files (recommended range: 4 KB < file_size < 500 MB).

FastDFS 6.06 Deployment Guide (CentOS 7.6)

3

Added the open-source MinIO deployment guide.

MinIO is a lightweight unstructured distributed file storage service. It is compatible with Amazon S3 cloud storage service interfaces and is suitable for storing large-capacity unstructured data, such as images, videos, log files, backup data, and container/VM images. An object file can be of any size, ranging from several KB to 5 TB.

MinIO 2020-12-12T08-39-07Z Deployment Guide (CentOS 7.6)

4

Added the open-source Lustre deployment guide.

Lustre is a cluster-oriented storage architecture. It is an open-source cluster (parallel) file system based on the Linux platform and provides file system interfaces compatible with POSIX. Lustre features high scalability and high performance. It supports tens of thousands of client systems, PB-scale storage capacity, and hundreds of GB aggregated I/O throughput.

Lustre 2.13.0 Deployment Guide (CentOS 8.0)

1

Added the new feature "data compaction."

The data compaction algorithm of Kunpeng BoostKit for SDS is deployed on the open-source distributed storage cluster Ceph to eliminate data waste caused by zero padding. In addition, combined with functions including data encapsulation, space allocation based on block counting, granularity-based traffic diversion, batch submission, and batch callback, the data compaction algorithm improves the data reduction ratio and overall system IOPS, which reduces costs and improves performance.

• Data Compaction Feature Guide (CentOS 7.6)
• Data Compaction Feature Guide (openEuler 20.03)

2

Added the new feature "global cache."

Kunpeng BoostKit for SDS provides a global cache, which is a distributed cache system deployed on an open-source Ceph storage cluster for I/O acceleration. Global Cache improves IOPS performance of distributed storage systems and shortens access delays without increasing hardware costs.

Global Cache User Guide (openEuler 20.03)