Enabling the Network Multipathing RPS Enhancement Feature
- Disable irqbalance.
systemctl stop irqbalance
- The network multipathing module depends on the hisi_l3t.ko module. Check whether the module has been loaded.
lsmod | grep hisi_l3t
If the command output is empty, load hisi_l3t.ko.
modprobe hisi_l3t
- Configure the NIC information and enable the network multipathing RPS enhancement feature.
insmod /lib/modules/5.10.0-301.0.0.204.aarch64/kernel/net/oenetcls/oenetcls.ko mode=0 appname="redis-server" ifname="eth0#eth1" strategy=1 debug=0 match_ip_flag=1 rxq_multiplex_limit=1 lo_rps_policy=2 rps_policy=2
mode specifies the optimization mode. The value 0 indicates the ntuple mode and the value 1 indicates the flow mode. The default value is 0. When the load of cores corresponding to interrupts is relatively low, this feature can work with the domain-based scheduling feature to implement cluster affinity.
appname specifies the name of the process for which the multipathing feature takes effect. The maximum name length is 16 characters. Multiple process names can be input, which are separated by number signs (#). For example, if the process names are IO_Context and redis-server, the value of appname can be IO_Con#redis-server. Its default value is redis-server.
ifname specifies the interface name of the NIC where the multipathing feature takes effect. Multiple interface names can be input, which are separated by number signs (#). For example, the value can be eth1#eth2.
strategy: interrupt-core binding policy. The value can be 0 (default), 1, 2, 3, or other values. If other values are passed, the default policy 0 is used.
The options are as follows:
- 0: default policy. The NIC queue interrupts are evenly distributed to different NUMA nodes, and different NICs use different cores.
- 1: cluster-based even distribution policy. The NIC queue interrupts are evenly distributed to different clusters.
- 2: NUMA-based even distribution policy. The NIC queue interrupts are evenly distributed to different NUMA nodes. Different from the default policy, this policy allows different NICs to use the same core.
- 3: user-defined NIC interrupt-core binding policy. After the multipathing module is loaded, the interrupt information is automatically read.
debug indicates the debugging switch. If debug is set to 1, debugging logs are generated. The value can be dynamically changed between 0 and 1. The default value is 0.
match_ip_flag controls whether the NIC uses the destination IP address to assign incoming data packets to specific queues. The value can be dynamically changed between 0 and 1. The change takes effect only for newly started processes.
rxq_multiplex_limit is used only when mode is 0. It specifies the number of TCP data flows that can multiplex each queue during NIC receiving rule configuration. For example, if rxq_multiplex_limit is set to 4, four rules can be configured for a single queue. This parameter is applicable to scenarios where the number of NIC queues is less than the number of TCP flows. The value ranges from 1 to 64, and the default value is 1.
lo_rps_policy indicates the receive packet steering (RPS) switch for local loopback interfaces. RPS is used to distribute data packets received by NICs to multiple CPU cores for processing. This leverages the multi-core advantage and prevents a single CPU from becoming a bottleneck. The default value is 0.
The options are as follows:
- 0: disables RPS for local loopback interfaces.
- 1: distributes software interrupts within the NUMA node where the current flow is located.
- 2: distributes software interrupts within the cluster node where the current flow is located.
rps_policy indicates the RPS switch for NIC interfaces. RPS is used to distribute data packets received by NICs to multiple CPU cores for processing. This leverages the multi-core advantage and prevents a single CPU from becoming a bottleneck. The default value is 0.
The options are as follows:
- 0: disables RPS for NIC interfaces.
- 1: distributes software interrupts within the NUMA node where the current flow is located.
- 2: distributes software interrupts within the cluster node where the current flow is located.