Using the SPDK Crypto Feature

This feature enables the crypto feature at the SPDK bdev layer and uses OpenSSL for encryption and decryption. Encryption and decryption workloads can be offloaded to KAE to improve performance and reduce CPU consumption. The current version supports symmetric encryption algorithms AES_CBC, AES_CTR, SM4_CBC, and SM4_CTR and asymmetric encryption algorithm RSA.

1. Allocate hugepage memory.

   Allocate huge pages as required. The following uses 40,000 2 MB huge pages as an example.

   echo 0 >/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
   echo 20000 >/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
   echo 20000 >/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages

2. Start the SPDK process.

   ./build/bin/nvmf_tgt --wait-for-rpc

3. Set the encryption driver to OpenSSL.

   ./scripts/rpc.py bdev_crypto_set_driver -d crypto_openssl

4. (Optional) Set the OpenSSL engine to KAE.

   ./scripts/rpc.py bdev_cryptodev_set_engine -e crypto_engine_kae

5. Initialize the framework.

   ./scripts/rpc.py framework_start_init

6. Create an encrypted bdev. The following are examples of encryption algorithms supported by the current version. You can select one based on the actual application scenario.

   Create a bdev encrypted using AES_CBC.

   

   For command details, refer to official SPDK documentation or run the -h command.

   [root@ceph1 spdk]# ./scripts/rpc.py bdev_aio_create -h
   usage: rpc.py [options] bdev_aio_create [-h] filename name [block_size]
   
   positional arguments:
     filename    Path to device or file (ex: /dev/sda)
     name        Block device name
     block_size  Block size for this bdev
   
   optional arguments:
     -h, --help  show this help message and exit

   ./scripts/rpc.py bdev_aio_create /dev/sda sda
   ./scripts/rpc.py bdev_crypto_create sda crypto_aes_cbc crypto_openssl 0123456789123456 -c AES_CBC

   Create a bdev encrypted using AES_CTR.

   ./scripts/rpc.py bdev_aio_create /dev/sdb sdb
   ./scripts/rpc.py bdev_crypto_create sdb crypto_aes_ctr crypto_openssl 0123456789123456 -c AES_CTR

   Create a bdev encrypted using SM4_CBC.

   ./scripts/rpc.py bdev_aio_create /dev/sdc sdc
   ./scripts/rpc.py bdev_crypto_create sdc crypto_sm4_cbc crypto_openssl 0123456789123456 -c SM4_CBC

   Create a bdev encrypted using SM4_CTR.

   ./scripts/rpc.py bdev_aio_create /dev/sdd sdd
   ./scripts/rpc.py bdev_crypto_create sdd crypto_sm4_ctr crypto_openssl 0123456789123456 -c SM4_CTR

   

   Before creating an asymmetrically encrypted bdev, you need to create an RSA key. The current version supports only 4096-bit RSA keys.

   openssl genrsa -out prikey.pem 4096

   Create a bdev encrypted using CRT-RSA.

   ./scripts/rpc.py bdev_aio_create /dev/sde sde
   ./scripts/rpc.py bdev_crypto_create sde crypto_rsa_crt crypto_openssl prikey.pem -c RSA -k2 CRT

   Create a bdev encrypted using NED-RSA.

   ./scripts/rpc.py bdev_aio_create /dev/sdf sdf
   ./scripts/rpc.py bdev_crypto_create sdf crypto_rsa_ned crypto_openssl prikey.pem -c RSA -k2 NED

7. Mount the NVMe bdev.
   

   This step uses crypto_aes_cbc created in step 6 as an example to describe how to mount a NVMe bdev.

   For command details, refer to official SPDK documentation or run the -h command.

   [root@ceph1 spdk]# ./scripts/rpc.py nvmf_create_subsystem -h
   usage: rpc.py [options] nvmf_create_subsystem [-h] [-t TGT_NAME] [-s SERIAL_NUMBER] [-d MODEL_NUMBER] [-a] [-m MAX_NAMESPACES] [-r] nqn
   
   positional arguments:
     nqn                   Subsystem NQN (ASCII)
   
   optional arguments:
     -h, --help            show this help message and exit
     -t TGT_NAME, --tgt_name TGT_NAME
                           The name of the parent NVMe-oF target (optional)
     -s SERIAL_NUMBER, --serial-number SERIAL_NUMBER
                           Format: 'sn' etc Example: 'SPDK00000000000001'
     -d MODEL_NUMBER, --model-number MODEL_NUMBER
                           Format: 'mn' etc Example: 'SPDK Controller'
     -a, --allow-any-host  Allow any host to connect (don't enforce allowed host NQN list)
     -m MAX_NAMESPACES, --max-namespaces MAX_NAMESPACES
                           Maximum number of namespaces allowed
     -r, --ana-reporting   Enable ANA reporting feature

   ./scripts/rpc.py nvmf_create_transport -t TCP -u 16384 -m 8 -c 8192
   ./scripts/rpc.py nvmf_create_subsystem nqn.2024-08.io.spdk:crypto_aes_cbc -a -s SPDK00000000000001 -d SPDK_Controller1
   ./scripts/rpc.py nvmf_subsystem_add_ns nqn.2024-08.io.spdk:crypto_aes_cbc crypto_aes_cbc
   ./scripts/rpc.py nvmf_subsystem_add_listener nqn.2024-08.io.spdk:crypto_aes_cbc -t TCP -a 90.90.82.112 -s 4420
   nvme discover -t tcp -a 90.90.82.112 -s 4420
   nvme connect -t tcp -n "nqn.2024-08.io.spdk:crypto_aes_cbc" -a 90.90.82.112 -s 4420 -g -G

8. Test the read and write performance.

   fio -filename=/dev/nvme0n1 -direct=1 -iodepth=64 -thread -rw=randwrite -ioengine=libaio -bs=4k -size=10G -numjobs=1 -group_reporting -name=mytest --verify_pattern=0x12345678 -verify=pattern -do_verify=1
   fio -filename=/dev/nvme0n1 -direct=1 -iodepth=64 -thread -rw=randread -ioengine=libaio -bs=4k -size=10G -numjobs=1 -group_reporting -name=mytest --verify_pattern=0x12345678 -verify=pattern -do_verify=1