Huge Pages
Also known as: hugepages, large pages, transparent huge pages, THP, 2MB pages, 1GB pages
Memory pages of 2 MB or 1 GB rather than the default 4 KB — reducing TLB misses for large working sets (databases, JVM heaps) by increasing the coverage each TLB entry provides, at the cost of internal fragmentation.
- Primary domain
- Systems Software
- Sub-category
- Kernels, Operating Systems & Device Drivers
In simple terms
The CPU’s TLB caches virtual-to-physical address translations. With 4 KB pages and a 64-entry TLB, only 256 KB of address space can be covered without a TLB miss. A database with a 64 GB buffer pool will miss the TLB constantly — each miss adds 20–100 cycles to memory access. Huge pages solve this: a 2 MB page gives each TLB entry 512× more coverage; a 1 GB page gives 262,144× more. A 64 GB buffer pool needs 16,777,216 entries with 4 KB pages but only 64 entries with 1 GB pages — easily fitting in the TLB. The result: 10–30% performance improvement for memory-intensive workloads.
The Visual Map
graph TD
subgraph TLB4K ["TLB with 4 KB pages"]
TLB1["L1 dTLB: 64 entries\n64 × 4 KB = 256 KB covered\nDatabase buffer pool: 64 GB\n→ 99.99% TLB miss rate"]
Miss1["TLB miss = page table walk\n20-100 cycles each\nhundreds of millions per second"]
TLB1 --> Miss1
end
subgraph TLB2M ["TLB with 2 MB huge pages"]
TLB2["L1 dTLB: 64 entries\n64 × 2 MB = 128 MB covered\nMost JVM heaps fit → low miss rate"]
TLB3["1 GB pages:\n64 × 1 GB = 64 GB covered\n64 GB DB buffer pool fits in 64 TLB entries"]
TLB2 --> TLB3
end
TLB4K -->|"hugepages / mmap(MAP_HUGETLB)"| TLB2M
More detail
Page sizes on x86-64:
- 4 KB (default): standard page size. Fine for most applications with small working sets.
- 2 MB: “large pages” or “huge pages”. Supported by all modern CPUs. Require 2 MB-aligned, contiguous physical memory.
- 1 GB: “gigantic pages”. Requires CPU support (
pdpe1gbCPUID bit, available since Intel Nehalem/2009). Reserves the full gigabyte as a contiguous physical region — allocated only at boot or withhugetlbfs.
Why TLB coverage matters: with a 64-entry L1 dTLB and 4 KB pages, only 256 KB of address space is covered. A database or JVM heap is gigabytes — almost every heap access misses the L1 TLB and pays 6–15 cycles for the L2 TLB, or 100–200 cycles for a full page table walk on an L2 miss.
Linux huge pages configuration:
Static huge pages (preallocated, no compaction jitter):
echo 1024 > /proc/sys/vm/nr_hugepages # allocate 1024 × 2 MB = 2 GB
grep -i huge /proc/meminfo # verify allocation
Processes access them via mmap(MAP_HUGETLB) or shmget(SHM_HUGETLB). PostgreSQL uses huge_pages = on to mmap its shared buffer pool.
Transparent Huge Pages (THP): Linux automatically promotes 4 KB pages to 2 MB when a 2 MB-aligned contiguous allocation is found. Zero application changes required. However:
- THP compaction: the kernel must occasionally compact memory to create contiguous 2 MB ranges — this causes latency spikes (20–100ms pauses). For latency-sensitive applications (Redis, MongoDB, HFT), THP is often disabled.
Best practice for latency-sensitive apps: disable THP and use static huge pages for specific allocations:
echo never > /sys/kernel/mm/transparent_hugepage/enabled
For the JVM: -XX:+UseLargePages flag — JVM uses huge pages for the heap and code cache. Databases (PostgreSQL: huge_pages = on, Oracle: use_large_pages = TRUE) pre-register the buffer pool as huge pages.
Huge pages are one of the highest-leverage kernel parameters for database and JVM workloads. For PostgreSQL with large shared buffer pools, enabling huge pages typically yields 5–30% throughput improvement with near-zero code changes.
Under the Hood
Simulating TLB coverage — showing why huge pages reduce miss rates for large working sets:
def tlb_miss_rate(working_set_mb: float, page_kb: float, tlb_entries: int) -> float:
"""
Estimate TLB hit rate: fraction of accesses covered by the TLB.
Assumes uniform random access over the working set.
"""
covered_mb = (tlb_entries * page_kb) / 1024
if working_set_mb <= covered_mb:
return 0.0 # entire working set fits in TLB
# fraction of working set covered by TLB
hit_rate = covered_mb / working_set_mb
return 1.0 - hit_rate
configs = [
("4 KB pages", 4, 64),
("2 MB pages", 2048, 32), # TLBs have fewer 2MB entries
("1 GB pages", 1024*1024, 4),
]
workloads_mb = [1, 64, 256, 1024, 4096, 16384, 65536]
print("TLB miss rates by page size and working set size:")
print(f"{'Working set':>14} " + " ".join(f"{n:>15}" for n, _, _ in configs))
print("-" * 70)
for ws in workloads_mb:
row = f"{ws/1024 if ws >= 1024 else ws:>10}"
row += "GB" if ws >= 1024 else "MB"
row += " "
for name, page_kb, entries in configs:
mr = tlb_miss_rate(ws, page_kb, entries)
row += f" {mr*100:>13.1f}%"
print(row)
Engineering Trade-offs
Static huge pages vs. THP:
- Static pages: no compaction jitter, must be preallocated at boot (or via sysctl). The application or a library (jemalloc, tcmalloc, PostgreSQL) must explicitly request huge pages via
mmap(MAP_HUGETLB). - THP: zero code changes, but compaction can cause latency spikes of 20–100ms. Production latency-sensitive workloads (Redis, MongoDB, HFT) always disable THP.
Memory fragmentation: huge pages require contiguous physical pages — 2 MB pages need 512 contiguous 4 KB pages. On a long-running, fragmented system, huge page allocation may fail even if enough total free memory exists. Allocate huge pages at boot before memory becomes fragmented.
Internal fragmentation: a process that allocates a 5 KB object with huge pages gets a 2 MB page — 2043 KB wasted per allocation. Huge pages are appropriate for large, stable allocations (database buffer pools, JVM heaps), not for many small allocations.
NUMA interaction: on multi-socket systems, huge pages should be allocated on the NUMA node that will use them:
numactl --membind=0 --cpubind=0 ./app
A huge page allocated on node 1 but accessed from node 0 crosses the inter-socket interconnect — eliminating the TLB win with a NUMA penalty.
Real-world examples
- PostgreSQL tuning guides universally recommend
huge_pages = onwith pre-allocated huge pages for large servers (>16 GB shared_buffers). - Redis explicitly recommends
echo never > /sys/kernel/mm/transparent_hugepage/enabledin its documentation to avoid THP-induced latency spikes. - Oracle: DBA best practices require 1 GB huge pages for SGA > 32 GB.
- HFT firms: 1 GB huge pages on order book servers; TLB misses in hot order-matching code reduced from ~15% to under 0.1% of cycles.
Common misconceptions
- “Transparent Huge Pages are always better than default.” THP causes latency spikes from memory compaction — explicitly bad for Redis, MongoDB, and any latency-sensitive workload. Static huge pages are preferred for predictable latency.
- “Huge pages waste memory.” Internal fragmentation wastes up to 2 MB per huge-page allocation for tiny allocations. For large contiguous allocations (database pools, JVM heaps), fragmentation is negligible.
Try it yourself
Calculate TLB coverage and miss rate for different page sizes and working sets:
python3 - <<'EOF'
configs = [
("4 KB pages", 4, 64, 15), # (name, page_kb, entries, miss_cycles)
("2 MB pages", 2048, 32, 15),
("1 GB pages", 1048576, 4, 15),
]
print("Coverage and miss penalty for a 32 GB database buffer pool:")
print(f"{'Config':<16} {'Covered':>10} {'Miss rate':>12} {'Penalty/access'}")
print("-" * 58)
WS_GB = 32
WS_MB = WS_GB * 1024
for name, page_kb, entries, miss_cycles in configs:
covered_mb = entries * page_kb / 1024
miss_rate = max(0.0, 1 - covered_mb / WS_MB)
penalty_ns = miss_rate * miss_cycles * 0.3 # ~0.3 ns per cycle at 3 GHz
print(f"{name:<16} {covered_mb:>8,.0f} MB {miss_rate*100:>10.2f}% {penalty_ns:>8.2f} ns avg")
EOF
Learn next
- Memory pool — huge pages eliminate TLB misses; memory pools eliminate allocator latency; together they remove the two main memory-related latency sources for low-latency applications
- NUMA awareness — huge pages should be NUMA-local: allocate them on the socket that will access them, otherwise the inter-socket penalty negates the TLB benefit
- Cache-line alignment — the cache-level complement to huge-page-level TLB optimisation: after eliminating TLB misses, align data within huge pages to cache lines to eliminate cache miss overhead
Relationships
- Requires
Neighborhood
A visual companion to the relationships above. Click any node to visit that topic.