Enabling KAE Hardware Offload

After the compression feature is enabled in HDFS, network and drive data volume is reduced. However, the computing consumption increases, which may cause the end-to-end bandwidth to decrease in some scenarios. In this case, Kunpeng Accelerator Engine (KAE) hardware offload can be used to reduce CPU consumption and improve the bandwidth.

1. Install and deploy KAE. For details, see Installation Using Source Code.
2. Enable KAE in HDFS.
   Run the following command on agent1 to agent9:

   cp /usr/local/kaezip/lib/lib* /usr/local/hadoop/lib/native

3. (Optional) Disable KAE in HDFS.
   Run the following commands on agent1 to agent9:

   rm -rf /usr/local/hadoop/lib/native/libkaezip.*
   rm -rf /usr/local/hadoop/lib/native/libz.*

   

   This step is used to disable the KAE feature. If you require this feature in HDFS, skip this step.

4. Restart HDFS.
   Run the following commands on the server1 node:

   /usr/local/hadoop/sbin/stop-all.sh
   /usr/local/hadoop/sbin/start-all.sh
   hdfs dfsadmin -safemode leave