NIC Interrupts Binding to CPU Cores

Binding NIC interrupts to several fixed CPU cores can reduce performance loss caused by frequent CPU scheduling and save time for processing services. The following core binding strategy has been tested for multiple times. It greatly improves small-block read/write performance in EC mode, without affecting large-block read/write.

1. Disable irqbalance.

   1 2 3

   systemctl stop irqbalance.service systemctl disable irqbalance.service systemctl status irqbalance.service

2. Create a directory for storing the core binding script.

   1	mkdir /home/bind_net

3. Check the NIC names of the public network and cluster network.

   1

   ip a

   

   • In this example, the NIC name corresponding to the public network (192.168.3.166) is enp1330f1, and that corresponding to the cluster network (192.168.4.166) is enp1330f2.

     

4. Query the interrupts of the NICs corresponding to the public network and cluster network.

   1 2	cat /proc/interrupts |grep enp133s0f1 | awk '{print $1}' | awk -F ':' '{print $1}' cat /proc/interrupts |grep enp133s0f2 | awk '{print $1}' | awk -F ':' '{print $1}'

   

   • In the preceding commands, enp133s0f1 and enp133s0f2 correspond to the NIC names of the public network and cluster network, respectively. If no information is displayed after the commands are run, the interrupts cannot be obtained by NIC name. In this case, you need to query the interrupts by NIC bus information.
   • Run the ethtool -i NIC_name command to obtain the bus information of the NIC.

     

     As shown in the preceding figure, the bus information of enp133s0f0np0 is 0000:85:00.0. Use the bus information instead of the NIC name to query the interrupt.

     1	cat /proc/interrupts |grep 0000:85:00.0 | awk '{print $1}' | awk -F ':' '{print $1}'

5. Create a core binding script.
   1. Create bind_net_intterrupt.sh.

      1	vim bind_net_intterrupt.sh

   2. Enter the following content:

      1 2 3 4 5

      arr=(0 1 2 3 32 33 34 35 64 65 66 67 96 97 98 99) idx=0;for i in `cat /proc/interrupts |grep enp133s0f1 | awk '{print $1}' | awk -F ':' '{print $1}'`;do echo ${arr[$idx]} > /proc/irq/${i}/smp_affinity_list;((idx=(idx+1)%16));echo $idx;done arr=(0 1 2 3 32 33 34 35 64 65 66 67 96 97 98 99) idx=0;for i in `cat /proc/interrupts | grep enp133s0f2 | awk '{print $1}' | awk -F ':' '{print $1}'`;do echo ${arr[$idx]} > /proc/irq/${i}/smp_affinity_list;((idx=(idx+1)%16));echo $idx;done

      

      • Replace enp133s0f1 and enp133s0f2 in this script based on the results in 4. If the interrupts can be obtained by NIC name, no modification is required. Otherwise, replace the NIC names with the corresponding NIC bus information.
      • The arr array lists the first four CPU cores of each NUMA node. Bind all NIC interrupts to the listed CPU cores.
6. Run the core binding script.

   1	sh bind_net_intterrupt.sh

7. Execute the 4 KB block write test case. For details, see Test Procedure.

   

   

   The network tasks preferentially run on the first four CPU cores of each NUMA node.