Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Configuring the NUMA Affinity

Purpose

VMs can be bound to cores by socket or NUMA. However, the performance for CPU cores in different NUMA nodes to access the same memory varies. The memory access latency in descending order is as follows: cross-CPU, cross-NUMA, and intra-NUMA. Therefore, avoid cross-NUMA and cross-chip memory access when applications are running. A good NUMA core binding policy can better reduce the access latency.

Procedure

Method 1: Use OpenStack for Configuration

  1. Set flavor to NUMA affinity, use one NUMA node, and set FLAVOR-NAME to the name of the created instance type.
    1
    		 
    openstack flavor set FLAVOR-NAME --property hw:numa_nodes=1
    • If the value of numa_nodes is set to 1, the scheduler selects a compute node on which single NUMA node meets the VM flavor configuration requirements.
    • If the value of numa_nodes is greater than 1, the scheduler selects a compute node no which the number of NUMA nodes and the resource in the NUMA nodes meet the VM flavor configuration requirements.
  2. Create a VM by using the flavor.

Method 2: Use virsh for Configuration (Recommended)

  1. View the NUMA node information on the physical machine.
    1
    		 
    numactl -H

    If the numactl tool is not installed, run the following command to install it:

    yum install numactl

  2. The following command output indicates that CPUs 0 to 23 are on NUMA node 0 and CPUs 24 to 47 are on NUMA node 1.

  3. Edit the XML configuration file of the VM on the compute node and perform core binding on the same die. For details about how to query instances, see Binding the VMs to Cores. The following is an example of the configuration file:
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
    		 
    <domain type = 'KVM'>
...
<vcpu placement = 'static' cpuset='4-7'>4</vcpu>
<cputune>
<vcpupin vcpu='0' cpuset='4'/>
<vcpupin vcpu='1' cpuset='5'/>
<vcpupin vcpu='2' cpuset='6'/>
<vcpupin vcpu='3' cpuset='7'/>
<emulatorpin cpuset='4-7'/>
</cputune>
...
<numatune>
<memmode cellid='0' mode='strict' nodeset='0'/>
</numatune>
...
<cpu mode='host-passthrough' check='none'>
<topology sockets='4' cores='1' threads='1'/>
<numa>
<cell id='0' cpus='0-3' memory='83886088' unit='KiB'/>
</numa>
</cpu>
...
</domain>

Recommended Policy

The non-cross-die + strict mode is recommended for memory allocation.

To improve VM performance, set <numatune> and <cputune> so that the vCPUs and their memory are in the same physical NUMA node.

For the memory allocation mode, the strict mode is preferred. The strict mode does not allow cross-NUMA memory allocation. The preferred mode preferentially allocates memory from the specified NUMA. If the memory is insufficient, the memory is allocated from another NUMA.

Parent topic: VM Tuning