Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Setting 1:1 Core Binding and Same-Die Memory Access

vCPUs of a VM can be bound to physical CPU cores in the same processor or cores in the same NUMA node. Avoid cross-die and cross-chip memory access to prevent performance deterioration. By default, vCPUs of different VMs may run on the same physical core, which causes CPU resource competition and frequent VMID changes. As a result, L1 TLB flushing frequently occurs and the TLB miss rate is high, causing performance deterioration.

  1. Query the NUMA node information in the Linux system.
    1
    		 
    numactl -H

  2. Edit the VM XML file in the Linux system, bind each vCPU to a core, and ensure that the cores are in the same die (NUMA). The following is an example:
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    		 
    <domain type = 'KVM'>
...
  <vcpu placement = 'static' cpuset='4-7'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='4'/>
    <vcpupin vcpu='1' cpuset='5'/>
    <vcpupin vcpu='2' cpuset='6'/>
    <vcpupin vcpu='3' cpuset='7'/>
    <emulatorpin cpuset='4-7'/>
  </cputune>
...
</domain>
  3. Configure one NUMA channel (NUMA 0) for the VM.
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
    		 
    <domain type = 'KVM'>
...
  <vcpu placement = 'static' cpuset='4-7'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='4'/>
    <vcpupin vcpu='1' cpuset='5'/>
    <vcpupin vcpu='2' cpuset='6'/>
    <vcpupin vcpu='3' cpuset='7'/>
    <emulatorpin cpuset='4-7'/>
  </cputune>
...
  <numatune>
    <memory mode='strict' nodeset='0'/>
  </numatune>
...
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='1'/>
    <numa>
      <cell id='0' cpus='0-3' memory='8388608' unit='KiB'/>
    </numa>
  </cpu>
...
</domain>
    • strict mode prohibits memory allocation across NUMA nodes. In preferred mode, memory is allocated first from the designated NUMA node, with other nodes used only if the primary node lacks sufficient memory. For Proof of Concept (PoC) testing and benchmarking, strict mode is preferred as it can boost performance by around 5%. <numa> defines the NUMA topology within the VM, with cpus specifying vCPU numbers and memory defining the memory size for each virtual node (vnode).
    • <numa> defines the NUMA topology within the VM, with cpus specifying vCPU numbers and memory defining the memory size for each virtual node (vnode).
    • To further optimize VM performance, configure <numatune> and <cputune> to ensure vCPUs and their associated memory reside on the same physical NUMA node (same-NUMA allocation).
    • In <numatune>, cellid matches cell id in <numa>. The mode parameter supports three options: strict (memory allocation is restricted to the specified node, failing if insufficient), preferred (memory is primarily allocated from the specified node, with fallback to others if needed), and interleave (memory is allocated in an interleaved fashion from the specified nodes). The nodeset parameter designates the physical NUMA nodes.