Binding the VMs to Cores

Purpose

Multiple VMs are running on a server, and each VM provides different services. As a result, resources are occupied to different extents. To prevent interference from adjacent VMs, processes of different VMs need to be isolated. In addition, OpenStack processes are used to ensure network and storage services. Therefore, OpenStack processes need to be isolated from other processes.

Procedure

Method 1: Use OpenStack for Configuration

1. Set the value of vcpu_pin_set in the /etc/nova/nova.conf file to the CPU core list reserved for the customer system process. For example, you can perform the following settings:

   vcpu_pin_set = "0-3"

   

2. Restart the nova service.

   1	systemctl restart openstack-nova-compute.service

   

3. Set flavor to 1v1 core binding and FLAVOR-NAME to the name of the created instance type.

   1	openstack flavor set FLAVOR-NAME --property hw:cpu_policy=dedicated

   

   The flavor can be configured based on the actual service scenario. The possible values of the cpu_policy option and their descriptions are as follows:

   • shared (default): Physical CPUs are not exclusively used and a vCPU can float among different physical CPUs.
   • dedicated: Physical CPUs are exclusively used and the vCPU of a VM is strictly bound to a physical CPU.

4. Create a VM by using this flavor.
   

   vCPUs cannot be explicitly specified in OpenStack core binding mode.

Method 2: Use virsh for Configuration (Recommended)

vCPUs cannot be explicitly specified by using method 1. If you need to specify vCPUs based on service requirements, use the virsh mode to explicitly bind vCPUs to cores. The procedure is as follows:

1. Run the following command to query the VM instance name and the compute node to which the VM belongs. xxx indicates the VM name.

   1	nova show xxx

   

2. Run the following command to edit the XML configuration file of the VM on compute1. instance-xxx indicates the instance name queried in step 1.

   1	virsh edit instance-000008f1

3. Add the following settings to the XML file.

   <domain type='kvm'>
   ...
   <vcpu placement = 'static' cpuset='4-7'>4</vcpu>
   <cputune>
   <vcpupin vcpu='0' cpuset='4'/>
   <vcpupin vcpu='1' cpuset='5'/>
   <vcpupin vcpu='2' cpuset='6'/>
   <vcpupin vcpu='3' cpuset='7'/>
   <emulatorpin cpuset='4-7'/>
   </cputune>
   ...
   </domain>

4. Restart the VM.

   1 2	virsh shutdown instance-000008f1 virsh start instance-000008f1

Recommended Policies

• Scenario 1

  When multiple VMs exist in the environment, the performance of 1v1 core binding is better than that of range-based core binding.

  cpuset='4-7' controls the qemu-kvm thread and other working threads to use only cores 4 to 7. If this parameter is not configured, the VM threads float on any cores, causing more cross-die and cross-chip loss.

  vcpupin binds each CPU thread to a core. If vcpupin is not used to bind CPU threads, the threads may switch between cores 4 to 7, causing extra overheads.

• Scenario 2

  When VMs have high requirements on memory bandwidth, the performance of CPU core binding across CPU clusters is better than that of CPU core binding in the same cluster.

  Kunpeng 920 series processors provides two super CPU clusters (SCCLs). Each SCCL contains six to eight CPU clusters, and each CPU cluster contains four cores. During core binding, the four cores in the same CPU cluster compete with each other. As a result, the memory bandwidth bottleneck occurs in the L3 cache.

  Therefore, you are advised to bind vCPUs to cores in multiple CPU clusters in the virtualization environment to avoid the memory bandwidth bottleneck caused by multiple cores in the same CPU cluster competing the L3 cache.

  vCPUs of VMs are bound to cores across CPU clusters. When the load is light, the competition on the L3 cache is reduced, and the memory bandwidth can be used to the maximum extent.

  Competition for L3 cache tags can be dramatically reduced and the memory bandwidth and CPU computing performance can be improved by performing CPU binding on as many CPU clusters as possible.

  The following is an example of cross-cluster core binding:

  1 2 3 4 5 6 7 8 9 10 11 12

  <domain type='kvm'> ... <vcpu placement = 'static' cpuset='4,8,12,16'>4</vcpu> <cputune> <vcpupin vcpu='0' cpuset='4'/> <vcpupin vcpu='1' cpuset='8'/> <vcpupin vcpu='2' cpuset='12'/> <vcpupin vcpu='3' cpuset='16'/> <emulatorpin cpuset='4,8,12,16'/> </cputune> ... </domain>