Enabling NUMA Awareness

Purpose

NUMA is enabled in Yarn 3.1.0 to improve the cluster resource scheduling efficiency.

Procedure

NUMA awareness is added to Yarn 3.1.0. During container startup, this function reads the CPU count and memory capacity of each NUMA node in the system. It uses the numactl command to specify the CPU and membind ranges for the container. This reduces cross-NUMA data access and improves task performance. To enable NUMA awareness, perform the following steps:

1. Install numactl.

   1	yum install numactl.aarch64 -y

2. Modify the $HADOOP_HOME/etc/hadoop/yarn-site.xml file as follows to enable NUMA awareness.

   <property>
       <name>yarn.nodemanager.numa-awareness.enabled</name>
       <value>true</value>
   </property>
   <property>
       <name>yarn.nodemanager.numa-awareness.read-topology</name>
       <value>true</value>
   </property>