Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Binding NIC Interrupts to Cores

Purpose

Compared with using the kernel's irqbalance to schedule NIC interrupts across all cores, manually binding interrupts to cores can effectively improve the service network throughput.

Procedure

  1. Disable irqbalance.

    Before binding NIC interrupts to cores, disable irqbalance.

    • Stop the irqbalance service. The service will be invalid after the system restarts. 
      1
      		 
      systemctl stop irqbalance.service
    • Disable the irqbalance service. This setting is permanently valid. 
      1
      		 
      systemctl disable irqbalance.service
    • Check whether the irqbalance service is disabled.
      If the service is disabled, the status is inactive. 
      1
      		 
      systemctl status irqbalance.service
  2. Check the PCI device number of the NIC, assuming that the current NIC name is enp131s0.
    1
    		 
    ethtool -i enp131s0

  3. Check the NUMA node to which the PCIe NIC connects.
    1
    		 
    lspci -vvvs <bus-info>

  4. Check the core range corresponding to a NUMA node. For example, you can bind interrupts to cores 48 to 63.
    1
    		 
    lscpu

  5. Bind interrupts to cores. The Hi1822 NIC has 16 queues. Bind the interrupts to the 16 cores of the NUMA node to which the NIC connect (for example, cores 48 to 63 corresponding to NUMA node 1).
    1. Create a script. 
      1
      		 
      vi smartIrq.sh
    2. Press i to enter the insert mode and add the following content to the file: 
      1
2
3
4
5
6
7
8
9
      		 
      #!/bin/bash
irq_list=(`cat /proc/interrupts | grep enp131s0 | awk -F: '{print $1}'`)
cpunum=48 # Change the value to the first core of the node.
for irq in ${irq_list[@]}
do
echo $cpunum > /proc/irq/$irq/smp_affinity_list
echo `cat /proc/irq/$irq/smp_affinity_list`
(( cpunum+=1 ))
done
    3. Press Esc, type :wq!, and press Enter to save the file and exit.
    4. Bind interrupts to cores. 
      1
      		 
      bash smartIrq.sh
  6. Run the following script to check whether interrupt binding to cores is successful.
    1. Create a script. 
      1
      		 
      vi irqCheck.sh
    2. Press i to enter the insert mode and add the following content to the file: 
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
      		 
      #!/bin/bash
# NIC name
intf=$1
log=irqSet-`date "+%Y%m%d-%H%M%S"`.log
# Number of available CPUs
cpuNum=$(cat /proc/cpuinfo |grep processor -c)
# RX and TX interrupt lists
irqListRx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}')
irqListTx=$(cat /proc/interrupts | grep ${intf} | awk -F':' '{print $1}')
# Bind the RX interrupt requests (IRQs).
for irqRX in ${irqListRx[@]}
do
cat /proc/irq/${irqRX}/smp_affinity_list
done
# Bind the TX IRQs.
for irqTX in ${irqListTx[@]}
do
cat /proc/irq/${irqTX}/smp_affinity_list
done
    3. Press Esc, type :wq!, and press Enter to save the file and exit.
    4. Check whether interrupt binding to cores is successful.
      If the IRQ is displayed, interrupts have been successfully bound to cores. 
      1
      		 
      sh irqCheck.sh enp131s0
Parent topic: Hardware Tuning