NIC Tuning

Modifying NIC Parameters

1. Adjust rx_buff.

   The default value of rx_buff of the Hi1822 NIC is 2 KB. When 64 KB packets are aggregated, multiple non-contiguous memory blocks are required, resulting in low memory usage. This parameter can be set to 2, 4, 8 or 16 KB to reduce non-contiguous memory blocks and improve memory usage.

   1. Query the rx_buff parameter.

      cat /sys/bus/pci/drivers/hinic/module/parameters/rx_buff

   2. Change the value of rx_buff to 8 KB.

      add this to /etc/modprobe.d/hinic.conf
      options hinic rx_buff=8
      rmmod hinic
      modprobe hinic

   3. Check the modification result.

      cat /sys/bus/pci/drivers/hinic/module/parameters/rx_buff

2. Change the size of the ring buffer.

   The maximum ring buffer size of the Hi1822 NIC is 4096. The default value is 1024. You can increase the size of the NIC ring buffer. The following example command sets the ring buffer size to 4096.

   ethtool -G enp4s0 rx 4096 tx 4096

3. Enable LRO.

   Same as the Hi 82599 NIC, the Hi1822 NIC supports large receive offload (LRO). You are advised to enable this function and set the LRO parameter of the Hi1822 properly.

   ethtool -K enp4s0 lro on
   hinicadm lro -i hinic0 -t <TIME> # Indicates that packets are aggregated and sent after the specified maximum time (in microseconds). The default value is 16 μs.
   hinicadm lro -i hinic0 -n <NUM> # Indicates that packets are aggregated and sent after the maximum number of packets is received. The default value is 4. In this example, the value is set to 8.

Binding NIC Interrupts to Cores

To help the service network improve the capability of receiving and transmitting packets, binding cores manually to fix interrupts is a better option than using irqbalance of the kernel to schedule NIC interrupts on all cores.

To bind a NIC software interrupt to a core, perform the following steps:

1. Disable the irqbalance service.

   systemctl stop irqbalance
   systemctl disable irqbalance

2. Query the NUMA node to which the NIC belongs.

   cat /sys/class/net/<Network port name>/device/numa_node

3. Query the CPU cores that correspond to the NUMA node.

   lscpu

4. Query the interrupt ID corresponding to the NIC.

   cat /proc/interrupts | grep <NIC name> | awk -F ':' '{print $1}'

5. Bind the software interrupt to the core corresponding to the NUMA node.

   echo <core number> > /proc/irq/<Interrupt ID>/smp_affinity_list