System Tuning
Optimizing the OS Configuration
- Purpose
Adjust the system configuration to maximize the hardware performance.
- ProcedureTable 1 lists the optimization items.
Table 1 OS configuration parameters Parameter
Description
Suggestion
Configuration Method
vm.swappiness
The swap partition is the virtual memory of the system. Do not use the swap partition because it will deteriorate system performance.
Default value: 60
Symptom: The performance deteriorates significantly when the swap partition is used.
Suggestion: Disable the swap partition and set this parameter to 0.
Run the following command:
1sudo sysctl vm.swappiness=0
MTU
Maximum size of a data packet that can pass through a NIC. After the value is increased, the number of network packets can be reduced and the efficiency can be improved.
Default value: 1500 bytes
Symptom: Run the ip addr command to view the value.
Suggestion: Set the maximum size of a data packet that can pass through a NIC to 9000 bytes.
- Run the following command:
1vim /etc/sysconfig/network-scripts/ifcfg-$(Interface)
Add MTU="9000".NOTE:${Interface} indicates the network port name.
- After the configuration is complete, restart the network service.
1service network restart
pid_max
The default value of pid_max is 32768, which is sufficient in normal cases. However, when heavy workloads are being processed, this value is insufficient and may cause memory allocation failure.
Default value: 32768
Symptom: Run the cat /proc/sys/kernel/pid_max command to view the value.
Suggestion: Set the maximum number of threads that can be generated in the system to 4194303.
Run the following command:
1echo 4194303 > /proc/sys/kernel/pid_max
file_max
Maximum number of files that can be opened by all processes in the system. In addition, some programs can call the setrlimit interface to set the limit on each process. If the system generates a large number of errors indicating that file handles are used up, increase the value of this parameter.
Default value: 13291808
Symptom: Run the cat /proc/sys/fs/file-max command to view the value.
Suggestion: Set the maximum number of files that can be opened by all processes in the system to the value displayed after the cat /proc/meminfo | grep MemTotal | awk '{print $2}' command is run.
Run the following command:
1echo ${file-max} > /proc/sys/fs/file-max
NOTE:${file-max} is the value displayed after the cat /proc/meminfo | grep MemTotal | awk '{print $2}' is run.
read_ahead
Linux readahead means that the Linux kernel prefetches a certain area of the specified file and loads it into the page cache. As a result, when the area is accessed subsequently, block caused by page fault will not occur.
Reading data from memory is much faster than reading data from drives. Therefore, the readahead feature can effectively reduce the number of drive seeks and the I/O waiting time of the applications. It is one of the important methods for optimizing the drive read I/O performance.
Default value: 128 KB
Symptom: Readahead can effectively reduce the number of drive seeks and the I/O waiting time of the applications. Run the /sbin/blockdev --getra /dev/sdb to view the value.
Suggestion: Change the value to 8192 KB. Improve the drive read efficiency by pre-reading and recording the data to random access memory (RAM).
Run the following command:
1/sbin/blockdev --setra /dev/sdb
NOTE:/dev/sdb is used as an example. You need to modify this parameter for all data drives.
I/O_Scheduler
The Linux I/O scheduler is a component of the Linux kernel. You can adjust the scheduler to optimize system performance.
Default value: CFQ
Symptom: The Linux I/O scheduler needs to be configured based on different storage devices for the optimal system performance.
Suggestion: Set the I/O scheduling policy to deadline for HDDs and noop for SSDs.
Run the following command:
1echo deadline > /sys/block/sdb/queue/scheduler
NOTE:/dev/sdb is used as an example. You need to modify this parameter for all data drives.
nr_requests
If the Linux system receives a large number of read requests, the default number of request queues may be insufficient. To deal with this problem, you can dynamically adjust the default number of request queues in the /sys/block/hda/queue/nr_requests file.
Default value: 128
Symptom: Increase the drive throughput by adjusting the nr_requests parameter.
Suggestion: Set the number of drive request queues to 512.
Run the following command:
1echo 512 > /sys/block/sdb/queue/nr_requests
NOTE:/dev/sdb is used as an example. You need to modify this parameter for all data drives.
- Run the following command:
NUMA Affinity Tuning
- Procedure
Evenly allocate network and storage resources to NUMA nodes.
- Purpose
In this example, 12 NVMe SSDs and four network ports are evenly allocated to four NUMA nodes.
The NVMe SSD numbers range from 0 to 11, and the network port names are enps0f0, enps0f1, enps0f2, and enps0f3.
1 2
for i in {0..11}; do echo `expr ${i} / 3` > /sys/class/block/nvme${i}n1/device/device/numa_node; done for j in {0..3}; do echo ${j} > /sys/class/net/enps0f${j}/device/numa_node; done