EC Turbo

Overview

The EC feature of open source Ceph is mainly used for whole object read/write operations in object storage (as shown in Figure 1), and it does not support partial read/write which will cause severe read/write amplification in block storage. In this case, the read/write performance of small blocks is only a fraction of that in the replication solution. In addition, the number of cross-network accesses increases during the stripe read/write process of EC. As a result, the read/write performance deteriorates. EC Turbo supports partial read/write (as shown in Figure 2), which effectively mitigates read/write amplification and improves read/write performance.

Figure 1 Only full-stripe read/write allowed in open source Ceph

Figure 2 Partial read/write with EC Turbo enabled

Technical Principles

In Ceph, the minimum storage unit size for an EC stripe is 4 KB. When a user reads or writes only 4 KB data, open source Ceph does not support partial modification. As a result, when a small data block is read or written, read/write amplification is severe. The EC Turbo feature enables partial modification of an EC stripe. Only required data blocks are read, reducing read/write amplification. When data to be read exceeds the 4 KB block size, cross-stripe read is required. In this case, an 8 KB operation is amplified into two 4 KB drive read operations. The performance is only half that of one local 8 KB read operation in the replication solution. EC Turbo enlarges the local storage unit size for an EC stripe (for example, to 1 MB) and supports modification of small stripes. For small I/Os such as 8 KB or 32 KB read, only one local read operation is required, similar to the replication solution. This improves read performance. In addition, operator pushdown is used to reduce network transmission overhead, and refined striping strategies are used to eliminate unnecessary padding zeros in EC stripes. These methods reduce read/write amplification.

Figure 3 EC Turbo row-to-column store

Expected Results

The EC performance of open source Ceph is only about 40% of the replication solution. By reducing read/write amplification during the I/O process, EC Turbo improves the read/write performance of EC to over 80% of the replication solution.

Parent topic: Features