High Network IOPS or Traffic

Binding NIC Interrupts to CPU Cores

1. Disable irqbalance.

   Before binding cores to NICs, disable irqbalance.

   1. Stop the irqbalance service. (The setting will be invalid after the system restarts.)

      systemctl stop irqbalance.service

   2. Disable the irqbalance service. (The setting will be permanently valid.)

      systemctl disable irqbalance.service

   3. Check the irqbalance service status.

      systemctl status irqbalance.service

2. Query the PCI device number of the NIC (for example, enp131s0).

   ethtool -i enp131s0

   The command output is as follows:

   driver:hinic
   version:2.3.2.1
   firmware-version:2.3.2.1

3. Query the NUMA node to which the PCIe NIC belongs.

   lspci -vvvs <bus-info>

   The command output is as follows:

   

4. View the core range corresponding to the NUMA node. In this example, the NUMA node can be bound to CPU cores 48 to 71.

   lscpu

   

5. Bind the interrupts to 16 cores on the NUMA node. In this example, there are 16 queues for the 1822 NIC and the interrupts are bound to cores 48 to 63 on NUMA node 2.

   The script content is as follows:

   #!/bin/bash
   irq_list=(`cat /proc/interrupts | grep enp131s0 | awk -F: '{print $1}'`)
   cpunum=48 # Change the value to the first CPU core of the node.
   for irq in ${irq_list[@]}
   do
   echo $cpunum > /proc/irq/$irq/smp_affinity_list
   echo `cat /proc/irq/$irq/smp_affinity_list`
   (( cpunum+=1 ))
   done

   Edit the script.

   bash smartIrq.sh

6. Check whether the CPU core binding is successful.

   The script content is as follows:

   #!/bin/bash
   # NIC name
   intf=$1
   log=irqSet-`date "+%Y%m%d-%H%M%S"`.log
   # Number of available CPUs
   cpuNum=$(cat /proc/cpuinfo |grep processor -c)
   # RX and TX interrupt lists
   irqListRx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}')
   irqListTx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}')
   # Bind the RX interrupt requests (IRQs).
   for irqRX in ${irqListRx[@]}
   do
   cat /proc/irq/${irqRX}/smp_affinity_list
   done
   # Bind the TX IRQs.
   for irqTX in ${irqListTx[@]}
   do
   cat /proc/irq/${irqTX}/smp_affinity_list
   done

   Check whether the CPU core binding is successful.

   sh irqCheck.sh enp131s0

   The command output is as follows: