Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Binding NIC Interrupts to Cores

To help the service network improve the capability of receiving and transmitting packets, binding cores manually to fix interrupts is a better option than using irqbalance of the kernel to schedule NIC interrupts on all cores.

Before binding NIC interrupts to CPU cores, disable irqbalance.

Method

  1. Stop the irqbalance service.
    1
    		 
    systemctl stop irqbalance.service
  2. Disable the irqbalance service.
    1
    		 
    systemctl disable irqbalance.service
  3. Check whether the irqbalance service is disabled.
    1
    		 
    systemctl status irqbalance.service

    If the status is inactive, the irqbalance service is disabled.

  4. Check the PCI device number of the NIC, assuming that the current NIC name is enp131s0.
    1
    		 
    ethtool -i enp131s0

  5. Check the NUMA node to which the PCIe NIC connects. Replace bus-info in the following command with the value obtained in step 4.
    1
    		 
    lspci -vvvs <bus-info>

  6. Query the core range corresponding to the NUMA node. In this example, the core range allows binding NUMA nodes 48 to 63.
    1
    		 
    lscpu

  7. Create a smartIrq.sh script.
    1. Create smartIrq.sh.
      vi smartIrq.sh
    2. Press i to enter the insert mode and add the following content to the file: 
      1
2
3
4
5
6
7
8
9
      		 
      #!/bin/bash
irq_list=(`cat /proc/interrupts | grep enp131s0 | awk -F: '{print $1}'`)
cpunum=48  # Change the value to the first core of the node.
for irq in ${irq_list[@]}
do
echo $cpunum > /proc/irq/$irq/smp_affinity_list
echo `cat /proc/irq/$irq/smp_affinity_list`
(( cpunum+=1 ))
done

      enp131s0 indicates the name of the NIC used to bind the queue. Change it as required.

    3. Press Esc, type :wq!, and press Enter to save the file and exit.
  8. Bind interrupts to cores. The Hi1822 NIC has 16 queues. Bind the interrupts to the 16 cores of the NUMA node to which the NIC connects. In this example, bind to cores 48 to 63 corresponding to NUMA node 2.
    1
    		 
    bash smartIrq.sh
  9. Create an irqCheck.sh script.
    1. Create an irqCheck.sh script.
      vi irqCheck.sh
    2. Press i to enter the insert mode and add the following content to the file: 
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
      		 
      #!/bin/bash
# NIC name
intf=$1
log=irqSet-`date "+%Y%m%d-%H%M%S"`.log
# Number of available CPUs
cpuNum=$(cat /proc/cpuinfo |grep processor -c)
# RX and TX interrupt lists
irqListRx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}')
irqListTx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}')
# Bind the RX interrupt requests (IRQs).
for irqRX in ${irqListRx[@]}
do
cat /proc/irq/${irqRX}/smp_affinity_list
done
# Bind the TX IRQs.
for irqTX in ${irqListTx[@]}
do
cat /proc/irq/${irqTX}/smp_affinity_list
done
    3. Press Esc, type :wq!, and press Enter to save the file and exit.
  10. Use the created script to check whether the core binding is successful. Assume that the current NIC name is enp131s0.
    1
    		 
    sh irqCheck.sh enp131s0

    The following is an example of successful core binding:

Parent topic: OS Tuning