Driver Errors Reported During Service Execution

Symptom

When a service operation is performed, an error message is displayed, as shown in Figure 1.

Possible Cause

The RCache size registered by the UCX service exceeds the hardware buffer size limit. You can increase the hardware buffer size by adding the configuration in step 1 or bonding network interfaces in step 2.

Solution

1. Configure the following information on the node:

   1 2 3

   mst start mlxconfig -d 85:00.0 -y s PF_LOG_BAR_SIZE=8 reboot

   

   85:00.0 indicates the PCIe number of the NIC. You can run the lspci | grep Mellanox command to query.

2. Bond network interfaces.
   1. Create a bond device.

      1	nmcli con add type bond ifname bond_01 mode 4

   2. Set an IP address for bond_01.

      1 2	nmcli connection modify bond-bond_01 ipv4.addresses 10.5.5.131/24 nmcli connection modify bond-bond_01 ipv4.method manual

   3. Add slave network interfaces. In this example, the slave network interfaces are ens7f1 and ens8f1. Change them as required.

      1 2	nmcli con add type bond-slave ifname ens7f1 master bond-bond_01 nmcli con add type bond-slave ifname ens8f1 master bond-bond_01

   4. Modify the BONDING_OPTS configuration.
      1. Open the /etc/sysconfig/network-scripts/ifcfg-bond-bond_01 file.

         1	vim /etc/sysconfig/network-scripts/ifcfg-bond-bond_01

      2. Press i to enter the insert mode and modify BONDING_OPTS.

         1 2	BONDING_OPTS="mode=4 miimon=100 xmit_hash_policy=layer3+4" MTU=4200

         

      3. Press Esc to exit the insert mode. Type :wq! and press Enter to save the file and exit.
   5. Enable the network device.

      1 2 3

      ifdown bond-bond_01 ifup bond-bond_01 systemctl restart NetworkManager

   6. On the switch, configure the port corresponding to the network interfaces to work in trunk mode. For details about the switch model and operation commands, contact the network IT owner. The following commands are for reference.
      • Configure the port to work in trunk mode.

        Switch(config)# interface GigabitEthernet1/0/1
        Switch(config-if)# switchport mode trunk
        Switch(config-if)# switchport trunk allowed vlan 10,20

        The commands configure GigabitEthernet1/0/1 to work in trunk mode and allow traffic of VLAN 10 and VLAN 20.

      • If multiple VLANs are connected between switches:

        Switch(config)# interface GigabitEthernet1/0/2
        Switch(config-if)# switchport mode trunk
        Switch(config-if)# switchport trunk allowed vlan 10-30

        The commands configure GigabitEthernet1/0/2 to work in trunk mode and allow traffic of VLAN 10 to VLAN 30.