KAE MD5 Digest Algorithm Tuning

Environment Preparations

Before installing the KAE, you need to apply for and install a license.

License application guide:

https://support.huawei.com/enterprise/en/doc/EDOC1100068118/69417574

Installation guide:

https://support.huawei.com/enterprise/en/doc/EDOC1100048786/ba20dd15

Download the acceleration engine installation package and developer Guide.

https://gitee.com/mirrors_kunpengcompute/KAE/tree/v1.3.6-bata

Installing the Accelerator Engine

• The developer guide describes how to install and use all modules of the accelerator engine. After reading the guide, select an appropriate installation mode.
• If the CentOS 7.6 and OpenSSL 1.0.2k are used, you need to download the corresponding libkae engine software package. The installation method of the package is the same as that in the developer guide.

Install the accelerator engine as instructed by the developer guide.

Changing the Default Number of Accelerator Queues

The default number of hardware accelerator queues is 256. When the number of accelerator queues remaining (which can be obtained in 4) is 0 or a small value during service running, you can change the number of queues to 512 or 1024 to maximize the accelerator performance.

1. Uninstall hisi_sec2.

   1	rmmod hisi_sec2

2. Set the default accelerator queue parameter pf_q_num=512.

   1	vi /etc/modprobe.d/hisi_sec2.conf

   1	options hisi_sec2 uacce_mode=2 enable_sm4_ctr=1 pf_q_num=512

3. Load hisi_sec2.

   1	modprobe hisi_sec2

4. Check the hardware accelerator queue.

   1	cat /sys/class/uacce/hisi_sec2-*/attrs/available_instances

   The change is successful if the following information is displayed.

   

Adapting Ceph to the Accelerator

Currently, the Ceph mainline version supports the configuration of the OpenSSL external engine using the configuration file. However, this feature is not available in the released Ceph versions (by 15.2.3). To use MD5 hardware acceleration in these versions, you need to modify the Ceph source code. For details about the modification method, see the latest patch that has been incorporated into the mainline versions.

https://github.com/ceph/ceph/pull/33964/

The following uses Ceph 14.2.8 as an example to describe how to adapt Ceph to MD5 hardware acceleration.

1. Obtain the source code.

   URL: https://download.ceph.com/tarballs/

   

   After the source package is downloaded, save it to the /home directory on the server.

2. Obtain the adaptation patch. Download the patch that enables the OpenSSL external engine of Ceph 14.2.8. After downloading the patch, save it to the /home directory.

   Download the patch at https://mirrors.huaweicloud.com/kunpeng/archive/kunpeng_solution/storage/Patch/.

   

3. Go to the /home directory, decompress the source package and go to the directory generated after decompression.

   1	cd /home && tar -zxvf ceph-14.2.8.tar.gz && cd ceph-14.2.8/

4. Apply the patch in the root directory of the source code.

   1	patch -p1 < ../ceph-14.2.8-common-rgw-add-openssl-engine-support.patch

   

5. After modifying the source code, compile Ceph.

   For details about the compilation procedure, see Ceph 14.2.8 Compilation Guide (CentOS 7.6 & openEuler 20.03 & openEuler 22.03).

6. Install Ceph.
7. Modify the systemd permissions. In Ceph, the RGW process service is managed by systemd. To enable systemd to access hardware acceleration devices, you need to modify the configuration on each RGW node as follows:
   1. Open the configuration file.

      1	vi /usr/lib/systemd/system/ceph-radosgw@.service

      Change PrivateDevices=yes to PrivateDevices=no.

   2. Make the modification take effect.

      1	systemctl daemon-reload

8. Use KAE to accelerate RGW digest computing. Add the following OpenSSL engine options to the global section in the ceph.conf file on the node where RGW is deployed:

   1	openssl_engine_opts = "engine_id=kae,dynamic_path=/usr/local/lib/engines-1.1/libkae.so,KAE_CMD_ENABLE_ASYNC=0,default_algorithms=DIGESTS,init=1"

   dynamic_path is the default installation path of libkae.so in the libkae engine software package. default_algorithms=DIGESTS indicates that this engine is used for digest algorithms including MD5. After completing the configuration, synchronize the configuration to all nodes where the RGW is deployed.

9. Restart RGW for the settings to take effect. Run the following command on each node where the RGW is deployed:

   1	systemctl restart ceph-radosgw.target