Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Binding NIC Interrupts to Cores

Purpose

Compared with using irqbalance of the kernel to schedule NIC interrupts on all cores, using manual core binding to fix interrupts can effectively improve the capability of the service network to receive and send packets.

Procedure

  1. Disable irqbalance.

    Before binding cores to NICs, disable irqbalance.

    • Stop the irqbalance service. The setting will be invalid after the system restarts. 
      1
      		 
      systemctl stop irqbalance.service
    • Disable the irqbalance service. This setting is permanently valid. 
      1
      		 
      systemctl disable irqbalance.service
    • Check whether the irqbalance service is disabled.
      If the service is disabled, the status is inactive. 
      1
      		 
      systemctl status irqbalance.service
  2. Check the PCI device number of the NIC, assuming that the current NIC name is enp131s0.
    1
    		 
    ethtool -i enp131s0

  3. Check the NUMA node to which the PCIe NIC connects.
    1
    		 
    lspci -vvvs <bus-info>

  4. Check the core range corresponding to a NUMA node. For example, the core range can be 48 to 63.
    1
    		 
    lscpu

  5. Bind interrupts to cores. The Hi1822 NIC has 16 queues. Bind the interrupts to the 16 cores of the NUMA node to which the NIC connect (for example, cores 48 to 63 corresponding to NUMA node 1).
    1. Create a script. 
      1
      		 
      vi smartIrq.sh
    2. Press i to enter the insert mode and add the following content to the file: 
      1
2
3
4
5
6
7
8
9
      		 
      #!/bin/bash
irq_list=(`cat /proc/interrupts | grep enp131s0 | awk -F: '{print $1}'`)
cpunum=48 # Change the value to the first core of the node.
for irq in ${irq_list[@]}
do
echo $cpunum > /proc/irq/$irq/smp_affinity_list
echo `cat /proc/irq/$irq/smp_affinity_list`
(( cpunum+=1 ))
done
    3. Press Esc, type :wq!, and press Enter to save the file and exit.
    4. Bind interrupts to cores. 
      1
      		 
      bash smartIrq.sh
  6. Run the following command to check whether the core binding is successful.
    1. Create a script. 
      1
      		 
      vi irqCheck.sh
    2. Press i to enter the insert mode and add the following content to the file: 
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
      		 
      #!/bin/bash
# NIC name
intf=$1
log=irqSet-`date "+%Y%m%d-%H%M%S"`.log
# Number of available CPUs
cpuNum=$(cat /proc/cpuinfo |grep processor -c)
# RX and TX interrupt lists
irqListRx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}')
irqListTx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}')
# Bind the RX interrupt requests (IRQs).
for irqRX in ${irqListRx[@]}
do
cat /proc/irq/${irqRX}/smp_affinity_list
done
# Bind the TX IRQs.
for irqTX in ${irqListTx[@]}
do
cat /proc/irq/${irqTX}/smp_affinity_list
done
    3. Press Esc, type :wq!, and press Enter to save the file and exit.
    4. Check whether the core binding is successful.
      If the IRQ is displayed, the cores are successfully bound. 
      1
      		 
      sh irqCheck.sh enp131s0
Parent topic: Hardware Tuning