Direct Access of cVMs to SR-IOV Devices on OpenStack

OpenStack supports SR-IOV only for NICs.

1. Configure NIC VFs on the compute node.
   1. Check the maximum number of VFs supported by the NIC. ${device_name} indicates the NIC name.

      cat /sys/class/net/${device_name}/device/sriov_totalvfs

   2. Enter the number of VFs ${vf_num}. The value cannot be greater than the maximum number of VFs supported by the NIC.

      echo ${vf_num} > /sys/class/net/${device_name}/device/sriov_numvfs

2. Configure device information on the compute node.
   1. Modify the /etc/nova/nova.conf file to configure the name of the NIC to be directly accessed, that is, ${device_name}. You can customize ${network_name}.

      [pci]
      passthrough_whitelist = { "devname": "${device_name}", "physical_network": "${network_name}"}

   2. After the configuration is complete, restart the nova-compute service.

      systemctl restart openstack-nova-compute

3. Install and configure neutron-sriov-agent on the compute node.
   1. Install neutron-sriov-agent and start it.

      yum install openstack-neutron-sriov-nic-agent -y
      systemctl enable neutron-sriov-nic-agent
      systemctl start neutron-sriov-nic-agent

   2. Modify the /etc/neutron/plugins/ml2/sriov_agent.ini file to configure the SR-IOV agent.

      [DEFAULT]
      log_dir = /var/log/neutron
      [sriov_nic]
      physical_device_mappings = ${network_name}:${device_name}
      [securitygroup]
      firewall_driver = neutron.agent.firewall.NoopFirewallDriver

   3. Restart the neutron-sriov-nic-agent service.

      systemctl restart neutron-sriov-nic-agent

   4. Check that the neutron-sriov-nic-agent service is running properly.

      openstack network agent list --agent-type nic

4. Configure the nova-scheduler controller node.
   1. Add the PciPassthroughFilter to the /etc/nova/nova.conf file.

      [filter_scheduler]
      enabled_filters = ...,PciPassthroughFilter
      available_filters = nova.scheduler.filters.all_filters

   2. Restart the nova-scheduler service.

      systemctl restart openstack-nova-scheduler.service

5. Configure the neutron-server controller node.
   1. Open the /etc/neutron/plugins/ml2/ml2_conf.ini file of neutron-server. Add the sriovnicswitch driver, and set the flat range to ${network_name}. This name must match the configuration on the compute node.

      [ml2]
      mechanism_drivers = openvswitch,sriovnicswitch
      
      [ml2_type_flat]
      flat_networks = ${network_name}

   2. Restart neutron-server.

      systemctl restart neutron-server

6. Create a network and an SR-IOV passthrough cVM.
   1. Create a network and specify the physical network ${network_name}.

      openstack network create --provider-physical-network ${network_name} \
      --provider-network-type flat \
      sriov-net

   2. Create an address pool and specify the IP address prefix and prefix range.

      openstack subnet pool create \
      --pool-prefix xxx.xxx.x.x/x \
      --default-prefix-length 24 \
      --share \
      shared-default-subnetpool-v4

   3. Create a subnet on the created network sriov-net.

      openstack subnet create --network sriov-net \
      --subnet-pool shared-default-subnetpool-v4 \
      sriov-subnet

   4. Obtain net_id and create an SR-IOV network port.

      net_id=$(openstack network show sriov-net -c id -f value)
      openstack port create --network ${net_id} --vnic-type direct sriov-port

   5. Obtain the port ID.

      port_id=$(openstack port show sriov-port -c id -f value)

   6. Use the created port to create a cVM.

      openstack server create --flavor cca-flavor --image openEuler-image \
      --nic port-id=$port_id \
      sriov_server

      

      If multiple VFs need to be directly connected, you can create more ports and specify multiple ports when creating a cVM.

      openstack port create --network ${net_id} --vnic-type direct sriov-port1
      openstack port create --network ${net_id} --vnic-type direct sriov-port2
      port_id1=$(openstack port show sriov-port1 -c id -f value)
      port_id2=$(openstack port show sriov-port2 -c id -f value)
      
      openstack server create --flavor cca-flavor --image openEuler-image \
      --nic port-id=$port_id1 \
      --nic port-id=$port_id2 \
      sriov_server

   7. After deploying the cVM, you can view the corresponding NIC in the console.

      ip a