Adjusting the Memory Page Size

Principles

The translation lookaside buffer (TLB) is the high-speed buffer in the CPU for the page table that stores the mapping between the page address of the virtual address and the page address of the physical address. A higher TLB hit ratio indicates better page table query performance.

A row in the TLB is the mapping of a page, that is, a row manages the memory of a page size.

Memory size managed by the TLB = Number of TLB rows x Memory page size

The number of TLB rows of the same CPU is fixed. Therefore, the larger the memory page is, the larger the managed memory is, and the higher the TLB hit ratio is in the same service scenario.

Before and after the modification, run the following command to check the TLB hit ratio ($PID indicates the process ID):

1	perf stat -p $PID -d -d -d

The command output contains the following information, in which 1.21% and 0.59% indicate the data miss ratio and instruction miss ratio respectively.

1
2
3
4

1,090,788,717      dTLB-loads                #  520.592 M/sec
13,213,603      dTLB-load-misses             #    1.21% of all dTLB cache hits
669,485,765      iTLB-loads                 #  319.520 M/sec
3,979,246      iTLB-load-misses              #  0.59% of all iTLB cache hits

Modification Method

• Set the memory page size of the kernel.
• Set memory huge page.
• Set transparent huge page.

Setting the Memory Page Size of the Kernel

To change the memory page size of the Linux kernel, you need to recompile the kernel after modifying the kernel compilation options. The procedure is as follows:

• Run the make menuconfig command:
• Set Page size to 64 KB:

  Kernel Features-->Page size(64KB)

• Compile and install kernel

Setting Memory Huge Pages

• Set during kernel startup

  Set the following kernel startup parameters by referring to kernel-parameters:

  Parameter	Description
  hugepages	Defines the number of huge pages configured in the kernel during startup.
  hugepagesz	Defines the size of huge pages configured in the kernel during startup.
  default_hugepagesz	Defines the default size of huge pages configured in the kernel during startup.

• Set during running

  Set the number of 2 MB huge pages on the node to 20.

  1	# echo 20 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

Setting Transparent Huge Page

Transparent huge page (THP) is an upgraded version of the preceding huge pages. When transparent huge page is used, the kernel automatically allocates huge pages to processes. Therefore, you do not need to manually reserve huge pages.

Set /sys/kernel/mm/transparent_hugepage/enabled as follows:

For example, to disable transparent huge page, run the following command:

1

# echo never > /sys/kernel/mm/transparent_hugepage/enabled

Many other memory parameters of the OS can be adjusted. For details, see Common OS Memory Parameters.