Using the SPDK Compression Feature

To use the SPDK compression feature, configure the zlib driver of DPDK and then create a compression bdev after initialization.

Allocate hugepage memory.

Allocate huge pages as required. The following uses 40,000 2 MB huge pages as an example.

echo 0 >/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
echo 20000 >/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
echo 20000 >/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages

Set the environment variable to load the zlib dynamic library of KAE for workload offload.

export LD_LIBRARY_PATH=/usr/local/lib:/usr/local/kaezip/lib:/usr/local/lib64:$LD_LIBRARY_PATH

Manage the NVMe drive used to store data.

export PCI_ALLOWED="0000:84:00.0"
./scripts/setup.sh

"0000:84:00.0" indicates the PCI number of the NVMe drive. Change it to the actual one that can be queried by running the following command:

./scripts/setup.sh status

In the following example, values in the BDF column are NVMe PCI numbers.

[root@ceph1 spdk_ksal]# ./scripts/setup.sh status
Type     BDF             Vendor Device NUMA    Driver           Device     Block devices
NVMe     0000:84:00.0    19e5   3714   1       vfio-pci         -          -
NVMe     0000:86:00.0    19e5   3714   1       nvme             nvme1      nvme1n1

Start the SPDK process.
```
./build/bin/nvmf_tgt --wait-for-rpc
```
Configure the zlib driver of DPDK.
```
./scripts/rpc.py bdev_compress_set_pmd -p 3
./scripts/rpc.py compressdev_zlib_module_set_wbits --wbits 31
```
- bdev_compress_set_pmd -p specifies the compression driver. The value 3 indicates the zlib driver.
- compressdev_zlib_module_set_wbits --wbits sets the bit width for zlib compression.
  If the bit width is set to a value ranging from –15 to –8, the deflate-raw format is used. In this case, the compression workload cannot be offloaded to KAE on a Kunpeng 920 processor, and will be executed by the CPU. Such workload can be offloaded to KAE on a Kunpeng 920 high-performance processor.
  If the bit width is set to a value ranging from 8 to 15, the zlib compression format is used.
  If the bit width is set to a value ranging from 25 to 31, the gzip compression format is used.
Initialize the framework.
```
./scripts/rpc.py framework_start_init
```

Use the managed NVMe drive to create a bdev.

./scripts/rpc.py bdev_nvme_attach_controller -b NVMe1 -t PCIe -a 0000:84:00.0

Create a compression bdev.
```
./scripts/rpc.py bdev_compress_create -b NVMe1n1 -p /dev/pmem0n1 -l 512
```
- -b specifies the bdev for storing compressed data. NVMe1n1 is the name of the bdev created in step 7.
- -p specifies the persistent memory drive for storing metadata, and /dev/pmem0n1 is the name of the persistent memory drive.
Test the compression bdev.
1. Mount the NVMe bdev.
```
./scripts/rpc.py nvmf_create_transport -t TCP -u 16384 -m 8 -c 8192
./scripts/rpc.py nvmf_create_subsystem nqn.2017-06.io.spdk:cnode2 -a -s SPDK00000000000001 -d SPDK_Controller1
./scripts/rpc.py nvmf_subsystem_add_ns nqn.2017-06.io.spdk:cnode2 COMP_NVMe1n1
./scripts/rpc.py nvmf_subsystem_add_listener nqn.2017-06.io.spdk:cnode2 -t TCP -a 96.10.57.104 -s 4521
nvme discover -t tcp -a 96.10.57.104 -s 4521
nvme connect -t tcp -n "nqn.2017-06.io.spdk:cnode2" -a 96.10.57.104 -s 4521 -g -G
```
  Running nvme commands requires nvme-cli. Run the following command to install it:
  
  yum install nvme-cli
  
  The nvme discover and nvme connect commands mount the SPDK compression bdev as a local NVMe drive. -t tcp indicates that the TCP protocol is used. -a 96.10.57.104 indicates the IP address of the SPDK host (the local IP address in this example). -s 4521 indicates that port 4521 is used. You can change it to another unoccupied port.
  
  In the nvme connect command, -n "nqn.2017-06.io.spdk:cnode2" is the name of the created NVMe-oF subsystem.
2. Perform a fio test.
  If the fio dependency is not installed, run the following command to install it:
  yum install fio
  
  You can run the fdisk -l command to query the NVMe drive letter. SPDK_Controller1 in the command output is the virtual drive letter created using SPDK.
  The following figure shows the output of the fdisk -l command for a common NVMe drive.
  
  The following figure shows the output of the fdisk -l command for a mounted NVMe drive.
```
fio -filename=/dev/nvme0n1 -direct=1 -iodepth=64 -thread -rw=randwrite -ioengine=libaio -bs=4k -size=10G -numjobs=1 -group_reporting -name=mytest --verify_pattern=0x12345678 -verify=pattern -do_verify=1
fio -filename=/dev/nvme0n1 -direct=1 -iodepth=64 -thread -rw=randread -ioengine=libaio -bs=4k -size=10G -numjobs=1 -group_reporting -name=mytest --verify_pattern=0x12345678 -verify=pattern -do_verify=1
```

Parent topic: Feature Use