Configuring the Kubernetes Topology Manager

After configuring the Kubernetes Topology Manager, check whether the application is restricted to run on one NUMA node to verify the configuration.

1. Install Docker.

   yum install docker

2. Deploy a Kubernetes cluster. For details, see Deploying the Kubernetes Cluster.
3. Modify the configuration file.
   1. Open the configuration file.

      vim /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf

      The default content of the configuration file is as follows:

      # Note: This dropin only works with kubeadm and kubelet v1.11+ 
      [Service] 
      Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf" 
      Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml" 
      # This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically 
      EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env 
      # This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use 
      # the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file. 
      EnvironmentFile=-/etc/sysconfig/kubelet 
      ExecStart= 
      ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

   2. Press i to enter the insert mode and modify the configuration file as follows:

      # Note: This dropin only works with kubeadm and kubelet v1.11+ 
      [Service] 
      Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf" 
      Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml" 
      # This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically 
      EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env 
      # This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use 
      # the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file. 
      EnvironmentFile=-/etc/sysconfig/kubelet 
      # Modification 1: Add the two lines of the ExecStartPre configuration to the file.
      ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/cpuset/system.slice/kubelet.service 
      ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/hugetlb/system.slice/kubelet.service 
      ExecStart= 
      # Modification 2: Add parameters such as --kube-reserved, --cpu-manager-policy, --feature-gates, and --topology-manager-policy to the end of the ExecStart configuration.
      ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS --kube-reserved=cpu=2,memory=250Mi --cpu-manager-policy=static --feature-gates=CPUManager=true --topology-manager-policy=single-numa-node

   3. Press Esc to exit the insert mode. Type :wq! and press Enter to save the file and exit.
4. Delete the CPU management status file cpu_manager_state.

   rm -f /var/lib/kubelet/cpu_manager_state

5. Restart the kubelet service.

   systemctl daemon-reload && systemctl restart kubelet

   Check the kubelet service status.

   systemctl status kubelet

   If the following information is displayed, the service is started successfully:

   

6. Modify the YAML file for creating an application. For example, the example.yaml file.
   1. Open the file.

      vi example.yaml

   2. Press i to enter the insert mode and add the following content. Select proper CPU and memory configuration values based on the CPU and memory specifications of the node to be deployed.

      resources:
        limits:
              cpu: 16
              memory: 64Gi
        requests:
              cpu: 16
              memory: 64Gi

      

      To make the single-numa-node mode take effect in a pod, set the cpu and memory values under the resources > limits field of the pod. In addition, the two values must be the same as the cpu and memory values under requests. If the value of requests is not specified, it is the same as the value of limits by default.

   3. Press Esc to exit the insert mode. Type :wq! and press Enter to save the file and exit.
7. Redeploy the application on the management node.

   kubectl apply -f *.ymal

8. Check the usage of pods and NUMA nodes.
   1. View the application container.

      docker ps | grep tp

      

      tp indicates the keyword for searching for the application container.

      In the expected result, 678044fcecaf indicates the application container ID.

      

   2. View the CpusetCpus used by the pod and check whether the application is restricted to run on one NUMA node.

      docker inspect xxx | grep Cpuset

      

      xxx indicates the application container ID specified in 8.a.

      You can see that the pod is restricted to the cores 2 to 11.

      

      Check the distribution of cores on the NUMA node of the physical machine.

      lscpu

      The distribution range includes the range of the NUMA nodes where the pod is located. It can be confirmed that the application is restricted to only one NUMA node and Kubernetes Topology Manager has been successfully configured.