Testing the Functionality

Prerequisites

A VM has been created. Supported VM specifications include but are not limited to 2 vCPUs with 8 GB memory, 4 vCPUs with 8 GB memory, 4 vCPUs with 16 GB memory, 8 vCPUs with 16 GB memory, 16 vCPUs with 32 GB memory, and 32 vCPUs with 64 GB memory.

Limiting the Memory Bandwidth

1. Edit the VM XML file.

   1	virsh edit <vm name>

2. Use the following XML configurations to limit the memory bandwidth of node 0 to 60%.

   1 2 3 4 5 6 7 8 9

   <domain type='kvm'> ...... <cputune> <memorytune vcpus='0-31'> <node id='0' bandwidth='60'/> </memorytune> </cputune> ...... </domain>

3. Start the VM.

   1	virsh start <vm name>

4. Check whether the corresponding MPAM control group is created in /sys/fs/resctrl. The control group name is determined based on the VM ID, name, and restricted vCPUs.
   1. Check whether the corresponding control group exists.

      ll /sys/fs/resctrl

      

   2. Check whether the content of the schemata file of this control group matches the XML configurations.

      cat /sys/fs/resctrl/<group name>/schemata

      

5. Enter the VM, use LMbench to test the bandwidth, and check whether the bandwidth changes in different configurations.
   

   • The upper limit of memory bandwidth varies with VM specifications. MPAM limits bandwidth based on the theoretical upper limit, not the actual limit that can be reached by a VM. This test focuses on the change trend.
   • The concurrency (-P) and memory page size parameters in the following commands need to be adjusted based on the VM specifications. The value of -P should be the same as the number of vCPUs.

   The following uses a VM with 32 vCPUs and 64 GB memory as an example.

   1. Bind the VM to NUMA node 0, set the memory bandwidth limitation to 50, and start and enter the VM. Then run the test command to test the memory bandwidth:

      1 2 3 4 5 6 7 8 9 10

      <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> ...... <vcpupin vcpu='30' cpuset='30'/> <vcpupin vcpu='31' cpuset='31'/> </cputune> <numatune> <memnode cellid='0' mode='strict' nodeset='0'/> </numatune>

      1	/opt/lmbench/bin/bw_mem -P 32 -N 5 512M rd

      

   2. Repeat the preceding operations, change the memory bandwidth limitation to 80, and restart the VM. Then run the same test command to test the memory bandwidth again. Check whether the memory bandwidth increases significantly.

      1 2	virsh shutdown <vm name> virsh start <vm name>

      

Limiting the L3 Cache

To limit the L3 cache using libvirt and ensure exclusive access to cache lines, you need to adjust the cache line usage of other control groups (including the default control group) to free up enough cache lines.

1. Modify the L3 cache data of the default control group to set the cache lines corresponding to the two highest-order mask bits as idle.

   1	echo "L3:1=3ffffff" > /sys/fs/resctrl/schemata

2. Edit the VM XML file.

   1	virsh edit <vm name>

3. Use the following XML configurations to limit the L3 cache of node 0 to 4 MiB, which corresponds to two cache lines of the physical machine.

   1 2 3 4 5 6 7 8 9 10

   <domain type='kvm'> ...... <cputune> <cachetune vcpus='0-15'> <cache id='0' level='3' type='both' size='4' unit='MiB'/> <cache id='0' level='3' type='priority' size='2'/> </cachetune> </cputune> ...... </domain>

4. Start the VM.

   1	virsh start <vm name>

5. Check whether the corresponding MPAM control group is created in /sys/fs/resctrl. The control group name is determined based on the VM ID, name, and restricted vCPUs.
   1. Check whether the corresponding control group exists.

      1	ll /sys/fs/resctrl

      

   2. Check whether the content of the schemata file of this control group matches the XML configurations.

      1	cat /sys/fs/resctrl/<group name>/schemata

      

6. View the statistics of the control group, and check whether the limitation takes effect and whether the data size (in bytes) is under the limit set in the XML file.

   1 2	cd /sys/fs/resctrl/<group name>/mon_data/mon_L3_01 watch -n 1 grep . *