System Tuning

Optimizing the OS Configuration

• Purpose

  Adjust the system configuration to maximize the hardware performance.

• Procedure
  Table 1 lists the optimization items.

  Table 1 OS configuration parameters

  Parameter	Description	Suggestion	Configuration Method
  vm.swappiness	The swap partition is the virtual memory of the system. It is not recommended because it will degrade system performance.	Default value: 60 Symptom: The performance deteriorates significantly when the swap partition is used. Suggestion: Disable the swap partition and set this parameter to 0.	Run the following command: 1 sudo sysctl vm.swappiness=0
  MTU	Maximum size of a data packet allowed by a NIC. After the value is increased, the number of network packets can be reduced and the efficiency can be improved.	Default value: 1500 bytes Symptom: Run the ip addr command to view the value. Suggestion: Set the MTU size to 9000 bytes.	Run the following command: 1 vim /etc/sysconfig/network-scripts/ifcfg-$(Interface) Add MTU="9000". NOTE: ${Interface} indicates the interface name. After the configuration is complete, restart the network service. 1 service NetworkManager restart
  pid_max	A tunable parameter provided by the kernel, which is used to limit the maximum number of processes that can be allocated by the system. The default value of pid_max is 32768, which is sufficient in normal cases. However, when heavy workloads are being processed, this value may be insufficient and cause memory allocation failure.	Default value: 32768 Symptom: Run the cat /proc/sys/kernel/pid_max command to view the value. Suggestion: Set the maximum number of threads that can be generated in the system to 4194303.	Run the following command: 1 echo 4194303 > /proc/sys/kernel/pid_max
  file_max	Maximum number of files that can be opened by all processes in the system. In addition, some programs can call setrlimit to set the limit on each process. If the system generates a large number of errors indicating that file handles are used up, increase the value of this parameter.	Default value: 13291808 Symptom: Run the cat /proc/sys/fs/file-max command to view the value. Suggestion: Set the value of this parameter to the one queried by running the cat /proc/meminfo | grep MemTotal | awk '{print $2}' command.	Run the following command: 1 echo ${file-max} > /proc/sys/fs/file-max NOTE: ${file-max} indicates the value displayed after cat /proc/meminfo | grep MemTotal | awk '{print $2}' is run.
  read_ahead	Linux readahead means that the Linux kernel prefetches a certain area of ​​the specified file and loads it into the page cache. As a result, when the area is accessed subsequently, blocks caused by page faults will not occur. Reading data from memory is much faster than from drives. Therefore, the readahead feature can effectively reduce the number of drive seeks and the I/O waiting time of applications. It is one of the important methods for optimizing the drive read I/O performance.	Default value: 128 KB Symptom: Run the /sbin/blockdev --getra /dev/sdb to view the value. Suggestion: Change the value to 8192 KB. This improves the drive read efficiency by pre-reading and recording data to random access memory (RAM).	Run the following command: 1 /sbin/blockdev --setra <sectors> /dev/sdb NOTE: /dev/sdb is used as an example. <sectors> indicates the readahead size, and you are advised to set it to 8192. You need to modify this parameter for all data drives.
  I/O_Scheduler	The Linux I/O scheduler is a component of the Linux kernel. You can adjust the scheduler to optimize system performance.	Default value: CFQ Symptom: The Linux I/O scheduler needs to be configured based on storage devices to achieve the optimal system performance. Suggestion: Set the I/O scheduling policy to deadline for HDDs and none for SSDs.	Run the following command: 1 2 echo deadline > /sys/block/sdb/queue/scheduler echo none > /sys/class/block/nvme0n1/queue/scheduler NOTE: /dev/sdb and /dev/nvme0n1 are used as an example. You need to modify this parameter for all data drives.
  nr_requests	A block device I/O scheduling parameter provided by the kernel. It controls the I/O request queue depth of each block device (such as an HDD or SSD). If the Linux system receives a large number of read requests, the default number of request queues may be insufficient. To deal with this problem, you can dynamically adjust the default number of request queues in the /sys/block/hda/queue/nr_requests file.	Default value: 128 (1023 for SSDs) Symptom: Adjusting the nr_requests parameter can increase the drive throughput. Suggestion: Set the number of drive request queues to 512 (1023 for SSDs).	Run the following command: 1 2 echo 512 > /sys/block/sdb/queue/nr_requests echo 1023 > /sys/block/nvme0n1/queue/nr_requests NOTE: /dev/sdb and /dev/nvme0n1 are used as an example. You need to modify this parameter for all data drives.

Optimizing the Network Performance

Purpose

This test uses the 25GE Ethernet adapter (Hi1822) with four ports, SFP+. It is used as an example to describe how to optimize the NIC parameters for the optimal performance.

Procedure

The optimization methods include adjusting NIC parameters and binding NIC interrupts to the corresponding local physical CPU. Table 2 describes the optimization items.

• Adjusting rx_buff
  1. Go to the /etc/modprobe.d directory.

     1	cd /etc/modprobe.d

  2. Create the hinic.conf file.

     1	vim /etc/modprobe.d/hinic.conf

     Add the following information to the file:

     1	options hinic rx_buff=8

  3. Reload the driver.

     1 2	rmmod hinic modprobe hinic

  4. Verify that the value of rx_buff is changed to 8.

     1	cat /sys/bus/pci/drivers/hinic/module/parameters/rx_buff

• Adjusting ring_buffer
  1. Change the buffer size from the default value 1024 to 4096.

     1	ethtool -G <NIC_name> rx 4096 tx 4096

  2. Check the current buffer size.

     1	ethtool -g <NIC_name>

• Enabling LRO
  1. Enable the LRO function for a NIC.

     1	ethtool -K <NIC_name> lro on

  2. Check whether the function is enabled.

     1	ethtool -k <NIC_name> | grep large-receive-offload

In addition to optimizing the preceding parameters, you need to bind the NIC software interrupts to CPU cores.

1. Disable the irqbalance service.
2. Query the NUMA node to which the NIC belongs.

   1	cat /sys/class/net/ <Interface_name> /device/numa_node

3. Query the CPU cores within the NUMA node.

   1

   lscpu

4. Query the NIC interrupt index. (For the IN200 NIC, the reference interface name is hns3. For other NICs, run the ip a command to query the interface name or contact technical support from the NIC vendor.)

   Generally, you can run the following command to query the NIC interrupt index:

   1	cat /proc/interrupts | grep <Interface_name> | awk -F ':' '{print $1}'

   In some cases, the preceding query command is not supported. You can obtain the interrupt index based on NIC bus information as follows.

   • Use ethtool to obtain NIC information.

     1	ethtool -i <Interface_name>

   • Obtain the interrupt index based on bus information of the NIC.

     1	cat /proc/interrupts |grep <bus-info> | awk '{print $1}' | awk -F ':' '{print $1}'

5. Bind the software interrupt to a core within the NUMA node.

   1	echo <Core index> > /proc/irq/ <Interrupt_index> /smp_affinity_list