Memory Pool
Also known as: object pool, arena allocator, pool allocator
Pre-allocating a block of memory and handing out fixed-size chunks from it — trading flexibility for speed and predictability by sidestepping the general-purpose allocator.
- Primary domain
- Systems Software
- Sub-category
- Computer Architecture, Embedded Systems & Real-Time Computing
In simple terms
Calling the general-purpose allocator (malloc/new) for every object is like going to the warehouse every time you need a screw. A memory pool instead grabs one big crate up front and hands out screws from it instantly. You allocate a large block once, then carve fixed-size pieces out of it with almost no work — and often hand them all back at once when you’re done.
The Visual Map
graph TD
subgraph GenAlloc ["General-purpose malloc"]
M["malloc(64)"] -->|"global lock\nbest-fit search\nmaybe mmap for new page"| Heap["Heap\n(fragmented)"]
M2["malloc(64) again"] -->|"unpredictable: 20ns-1ms"| Heap
Heap -->|"free()\nmay defer to OS\ncan cause TLB shootdown"| Back["OS"]
end
subgraph Pool ["Memory Pool / Arena"]
Init["arena_create(1MB)\n(one-time cost)"] --> Slab["1 MB contiguous block\n(all cache-friendly)"]
A1["pool_alloc()\nbump pointer += 64\n1-3 ns, no lock"] --> Slab
A2["pool_alloc() again\nbump pointer += 64"] --> Slab
A3["pool_alloc()..."] --> Slab
Reset["arena_reset()\npointer = base\nall freed at once"] --> Slab
end
More detail
A pool (sometimes called an arena or slab, depending on the flavour) reserves a contiguous region of memory and manages it itself:
- Fixed-size pools keep a free list of identical chunks. Allocation pops the head of the list; deallocation pushes it back. Both are O(1) with no searching and no fragmentation, because every chunk is interchangeable.
- Arena/bump allocators just advance a pointer through the block on each allocation and free everything at once by resetting the pointer. Individual frees are free — you don’t do them. Perfect for data with a shared lifetime, like everything produced during one request or one frame.
The payoffs over a general allocator:
- Speed: no global locks, no best-fit search. A bump allocator is
ptr += size; return old_ptr— 1–3 ns instead of 20–100 ns formalloc. - Predictability: no unbounded pauses — critical for real-time work.
malloccan mmap new pages (thousands of ns) or trigger GC in managed runtimes. - Locality: objects sit contiguously, so they share cache lines and play well with cache-line alignment.
- No external fragmentation: pools don’t fragment like a general heap; the slab is used compactly.
The costs: flexibility and discipline — pools assume a known size class or lifetime, can waste memory if oversized, and require you to reason about ownership yourself rather than leaning on a general allocator or garbage collection.
In low-latency systems the general allocator is a frequent culprit for tail latency: it takes a global lock, may fault in new pages, and can stall unpredictably. A pool replaces that with a bounded, branch-light operation, so worst-case timing becomes knowable.
Under the Hood
A fully-functional arena allocator and fixed-size pool in Python:
class Arena:
"""Bump-pointer arena: O(1) alloc, O(1) reset, no individual free."""
def __init__(self, capacity: int):
self._buf = bytearray(capacity)
self._cap = capacity
self._pos = 0
self._allocs = 0
def alloc(self, size: int, align: int = 8):
# align up
if self._pos % align:
self._pos += align - (self._pos % align)
if self._pos + size > self._cap:
raise MemoryError("arena full")
start = self._pos
self._pos += size
self._allocs += 1
return memoryview(self._buf)[start:start+size]
def reset(self):
"""Free all allocations at once: O(1)."""
self._pos = 0
self._allocs = 0
class FixedPool:
"""Fixed-size pool: O(1) alloc/free via free-list."""
def __init__(self, chunk_size: int, count: int):
self._chunk = chunk_size
self._free = list(range(count)) # free slot indices
self._used = {}
def alloc(self) -> int:
if not self._free:
raise MemoryError("pool exhausted")
slot = self._free.pop()
self._used[slot] = True
return slot
def free(self, slot: int):
del self._used[slot]
self._free.append(slot)
arena = Arena(4096)
pool = FixedPool(chunk_size=64, count=32)
# Allocate objects using arena
v1 = arena.alloc(64)
v2 = arena.alloc(64)
print(f"Arena: allocated {arena._allocs} objects, used {arena._pos} / {arena._cap} bytes")
# Use pool for fixed-size objects
slots = [pool.alloc() for _ in range(5)]
print(f"Pool: 5 slots allocated: {slots}")
pool.free(slots[2])
reclaimed = pool.alloc()
print(f"Pool: freed slot {slots[2]}, reallocated as slot {reclaimed}")
# Reset arena (free everything in O(1))
arena.reset()
print(f"Arena after reset: {arena._pos} bytes used (O(1) free)")
Engineering Trade-offs
Arena vs. fixed-size pool:
- Arena/bump: fastest allocation (1–3 cycles), no fragmentation, but no individual free — everything freed together. Use for per-frame or per-request temporary data.
- Fixed-size pool: O(1) alloc and free via free list; good for objects with individual lifetimes but the same size (packets, connections, messages). Can’t handle variable sizes without multiple pools.
malloc vs. pool under profiling: in a tight loop making thousands of malloc/free calls per second, the allocator lock and fragmentation management can consume 5–25% of CPU time. A pool eliminates this entirely, dropping allocation to 1–2 cycles.
Avoiding false sharing: pool objects are contiguous in memory, which can cause false sharing if different threads write to adjacent objects. Pad pool objects to 64 bytes (cache line size) or maintain per-thread pools to avoid coherence traffic.
Memory wastage: a fixed-size pool pre-allocates all capacity up front. If the maximum concurrent objects is overestimated by 10×, 90% of the pool sits idle. Size pools to realistic peak usage, not theoretical maximum.
Interaction with virtual memory: pools bypass the OS allocator, but still use virtual address space. Pre-fault the pool on startup (touch every page) to avoid page-fault stalls during production use.
Real-world examples
- Game engines reset a per-frame arena every frame instead of freeing thousands of temporary objects individually.
- High-throughput network servers pool fixed-size packet/connection buffers to avoid allocating on the hot path.
- Embedded and real-time systems forbid general
mallocentirely and use pools so timing is deterministic. - Database engines manage their page cache as a pool of fixed-size buffers — PostgreSQL’s shared_buffers is a static pool of 8 KB pages.
Common misconceptions
- “Pools are just a micro-optimisation.” Their bigger value is predictability — bounded, lock-free allocation matters more than raw speed in real-time code. The latency reduction is often 10–100×, not 10%.
- “A pool means you never think about memory again.” The opposite: you take on ownership and sizing decisions the general allocator used to make for you. Oversizing wastes memory; undersizing causes exhaustion at runtime.
Try it yourself
Benchmark bump-allocator vs. Python list-append (proxy for general allocation):
python3 - <<'EOF'
import time
N = 500_000
# Simulate general allocation: append to a list (new allocation per item)
t0 = time.perf_counter_ns()
general = []
for i in range(N):
general.append({"id": i, "value": i * 2}) # new dict per iteration
t1 = time.perf_counter_ns()
# Simulate pool: pre-allocate slots, reuse them
pool = [{"id": 0, "value": 0}] * N
t2 = time.perf_counter_ns()
for i in range(N):
pool[i]["id"] = i
pool[i]["value"] = i * 2
t3 = time.perf_counter_ns()
gen_ms = (t1 - t0) / 1e6
pool_ms = (t3 - t2) / 1e6
print(f"{N:,} object allocations:")
print(f" General (new dict each): {gen_ms:.1f} ms ({N/(gen_ms/1000):>10,.0f} allocs/s)")
print(f" Pool (pre-allocated) : {pool_ms:.1f} ms ({N/(pool_ms/1000):>10,.0f} allocs/s)")
print(f" Pool speedup: {gen_ms/pool_ms:.1f}x (larger in C/C++ without Python overhead)")
EOF
Learn next
- Garbage collection — the managed-language approach to memory reclamation: automatic but unpredictable pauses; memory pools are the manual alternative when GC pause times are unacceptable
- Cache-line alignment — pool objects are contiguous in memory; aligning each object to a cache line boundary prevents false sharing between threads accessing adjacent pool slots
- Lock-free programming — high-performance lock-free allocators (jemalloc, tcmalloc, mimalloc) use per-thread pool strategies internally to make allocation itself lock-free
Read this in a learning path
All paths →This topic is part of a learning path. Start in context to keep prev/next and progress tracking.
Relationships
- Requires
Neighborhood
A visual companion to the relationships above. Click any node to visit that topic.