How to check Transparent HugePage status in Linux?

Transparent Hugepages (THP) is an advanced memory management feature in the Linux kernel that optimizes system performance by reducing the overhead associated with managing memory resources. THP achieves this by utilizing larger memory pages, called hugepages, which can significantly decrease the number of Translation Lookaside Buffer (TLB) entries required for address translation. In this article, we will also discuss how to check Transparent HugePage status on your system.

In traditional Linux memory management, the kernel divid

In traditional Linux memory management, the kernel divides the physical memory into small fixed-size pages, typically 4KB in size. While this granularity provides flexibility, it can also result in a higher demand on the TLB, which can lead to performance bottlenecks. Hugepages, which come in sizes of 2MB or 1GB, mitigate this issue by allowing more memory to be mapped with fewer TLB entries.

THP operates transparently to applications, meaning that no modifications to application code or manual configuration are required to benefit from this feature. With THP enabled, the Linux kernel automatically allocates hugepages when possible and falls back to smaller pages when needed. This dynamic and adaptive approach allows for improved performance while maintaining compatibility with a wide range of applications and workloads.

In this article, we will delve deeper into the inner workings of Transparent Hugepages, discussing the advantages and potential drawbacks, as well as exploring practical use cases and configuration options for fine-tuning this powerful feature.

 

Different types of HugePages

In Linux, hugepages are larger memory pages that help improve performance by reducing the overhead associated with managing and translating memory addresses. There are two primary types of hugepages:

 

1. Static or Explicit Hugepages (nr_hugepages)

Static hugepages are explicitly reserved during system boot or runtime and require manual configuration. They remain allocated and fixed in size throughout the system's uptime. Static hugepages are typically used in performance-critical applications, such as databases and virtualization platforms, where the benefits of using larger memory pages outweigh the additional complexity of manual configuration.

To use static hugepages, you need to configure the vm.nr_hugepages sysctl parameter during boot time. Applications must also be modified to use these hugepages by employing specific system calls, such as mmap() or shmget(), with appropriate flags, like MAP_HUGETLB or SHM_HUGETLB.

There are two common sizes for static hugepages:

• 2MB: Known as regular hugepages, these are the most commonly used size and offer a balance between granularity and reduced TLB overhead.
• 1GB: Known as gigantic hugepages, these are available on systems with a large amount of memory and provide even greater TLB efficiency but may suffer from higher internal fragmentation.

 

2. Transparent Hugepages (AnonHugePages)

Transparent Hugepages (THP) is a memory management feature in the Linux kernel that aims to improve system performance and efficiency by leveraging larger memory pages called hugepages. This overview will discuss the following aspects of THP:

• Memory Management in Linux: In Linux, memory is divided into fixed-size chunks called pages. The default page size is 4KB, and the kernel uses a data structure called the Translation Lookaside Buffer (TLB) to manage and translate virtual-to-physical memory addresses. The use of small pages can lead to TLB inefficiencies and performance bottlenecks when dealing with large memory regions.
• Hugepages: Hugepages are memory pages with significantly larger sizes, typically 2MB or 1GB. By using hugepages, the kernel can map larger memory regions using fewer TLB entries, reducing the overhead associated with TLB management and improving overall system performance.
• Transparent Allocation: THP operates transparently to applications, which means that it automatically allocates hugepages without requiring any changes to the application code or manual configuration. The kernel dynamically and adaptively allocates hugepages when possible and falls back to smaller pages when needed.
• Advantages: Some key benefits of THP include reduced TLB overhead, improved memory management efficiency, and better overall system performance. THP is particularly beneficial for applications with large memory footprints, such as databases, virtualization, and high-performance computing workloads.
• Potential Drawbacks: Despite its advantages, THP may introduce some challenges, such as increased memory fragmentation, longer garbage collection pauses, or higher memory consumption in specific scenarios. These issues can be mitigated by tuning THP configurations or selectively enabling/disabling THP for specific applications.
• Configuration and Tuning: THP can be configured and tuned through the sysfs filesystem. Users can enable/disable THP system-wide or for specific processes, control the allocation policy (always, madvise, or never), and monitor the number of hugepages allocated.

 

 

Check HugePage Status

The /proc/meminfo file provides information about the total number of persistent hugetlb pages in the kernel's huge page pool. It also displays default huge page size and information about the number of free, reserved and surplus huge pages in the pool of huge pages of default size.

 

Explicit Huge Pages (nr_hugepages):

To check the current configuration of static hugepages, you can use the /proc/meminfo file or the sysctl command. Here's how:

# grep -i ^Huge /proc/meminfo
HugePages_Total:   50226
HugePages_Free:    50226
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

Here,

• HugePages_Total is the size of the pool of huge pages and is configured using /proc/sys/vm/nr_hugepages
• HugePages_Free is the number of huge pages in the pool that are not yet allocated.
• HugePages_Rsvd is short for "reserved," and is the number of huge pages for which a commitment to allocate from the pool has been made, but no allocation has yet been made. Reserved huge pages guarantee that an application will be able to allocate a huge page from the pool of huge pages at fault time.
• HugePages_Surp is short for "surplus," and is the number of huge pages in the pool above the value in /proc/sys/vm/nr_hugepages. The maximum number of surplus huge pages is controlled by /proc/sys/vm/nr_overcommit_hugepages.
• Hugepagesize is the default hugepage size (in Kb).

Alternatively we can use sysctl command:

sysctl vm.nr_hugepages

 

Transparent Huge Pages (AnonHugePages):

To check the status and usage of Transparent Huge Pages, you can use the /proc/meminfo and /sys/kernel/mm/transparent_hugepage files. Here's how:

# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]

Here,

• "always" means that an application requesting THP will stall on allocation failure and directly reclaim pages and compact memory in an effort to allocate a THP immediately. This may be desirable for virtual machines that benefit heavily from THP use and are willing to delay the VM start to utilise them.
• "madvise" will enter direct reclaim like "always" but only for regions that are have used madvise(MADV_HUGEPAGE). This is the default behaviour.
• "never" should be self-explanatory.

To check the THP defrag status, use the following command:

~]# cat /sys/kernel/mm/transparent_hugepage/defrag
[always] madvise never

This command will show the current THP defrag status, which can be "always", "madvise", "defer", "defer+madvise", or "never".

The AnonHugePages value in /proc/meminfo represents the current memory usage by Transparent Huge Pages (THP) in your Linux system. The value is displayed in kilobytes (kB).

~]# grep -i anonhugepages /proc/meminfo
AnonHugePages: 884736 kB

In this  example, AnonHugePages: 884736 kB indicates that 884,736 kilobytes of memory are being used by Transparent Huge Pages.
This value provides insight into how much memory is being utilized by THP, which can help assess the impact of THP on the system's performance and whether it's beneficial for our specific workloads.

 

How to check if static hugepage is disabled?

To check if nr_hugepages (static hugepages) is disabled or not, you can look at the total number of allocated static hugepages in the system. If the value is 0, it indicates that static hugepages are not preallocated.

You can check the total number of static hugepages using the /proc/meminfo file or the sysctl command:

• In the below snippets as you see the value of HugePages_Total and nr_hugepages is 0 which means that explicit hugepage is disabled on my system.
• If you see a non-zero value here then it means that the provided amount of memory is reserved in the kernel for explicit transparent hugepage.

# cat /sys/devices/system/node/node*/meminfo | fgrep Huge
Node 0 AnonHugePages:         0 kB
Node 0 HugePages_Total:     0
Node 0 HugePages_Free:      0
Node 0 HugePages_Surp:      0

# grep -i huge /proc/meminfo
AnonHugePages:    174080 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

# sysctl -a | grep nr_hugepages
vm.nr_hugepages = 0

 

 

Summary

Hugepages in Linux is a memory management technique that optimizes system performance by using larger memory pages, typically 2MB or 1GB, instead of the standard 4KB pages. By employing hugepages, the Linux kernel reduces the overhead associated with the Translation Lookaside Buffer (TLB), a cache responsible for managing and translating virtual-to-physical memory addresses. This optimization is particularly beneficial for workloads with large memory footprints, such as databases, virtualization, and high-performance computing.

There are two types of hugepages in Linux: Explicit Huge Pages (static hugepages) and Transparent Huge Pages (THP). Explicit Huge Pages are manually configured and allocated by the user, either during boot time or at runtime, using kernel boot parameters or sysctl commands. Applications must be modified to utilize these explicitly allocated hugepages. In contrast, Transparent Huge Pages are dynamically and automatically managed by the kernel, requiring no manual intervention or application code changes. THP provides an adaptive and efficient solution for a wide range of workloads.

While hugepages offer numerous benefits, including reduced TLB overhead, improved memory management efficiency, and enhanced overall system performance, they can also introduce some challenges. These may include increased memory fragmentation, longer garbage collection pauses, or higher memory consumption in specific scenarios. However, these issues can often be mitigated by fine-tuning the hugepage configurations or selectively enabling/disabling hugepages for particular applications.

 

References:
How to monitor, enable and disable transparent hugepages in Linux