Memory: allocators, guards and the cgroup

Why resident size was the wrong number

The question

Of 17 attempts in this theme, which ones survived two benchmark runs, the correctness suite and the gates?

adopted

The tantivy "heap" budget is not the whole story: 64 MB of arena turned into ~350 MB of resident memory with three JSON fields (columnar/fast-field writers live outside the budget). 32 MB costs ~8 % indexing speed and halves the peak.

Attempts17

Kept9

Discarded or crashed4

Best kept score37,100.2

Every run in this theme

Every run in this theme
#	score	writes/s	reads/s	verdict	what changed
0005	11,322.5	18,516	6,924	keep	HARNESS: container now pinned to 1 core (cpuset) instead of CFS quota. ENGINE: keyword field ignore_above 64 (index 226->106 MB); no delete_term for fresh ids (recovery…
0013	—	14,215	9,727	crash	INVALID (same cause: ran on the 0001 code): mimalloc global allocator; re-test later
0015	—	27,951	7,143	discard	mimalloc (valid re-test): writes +13% (28k/s) but rebuild peak anon 502 MB vs 408 MB glibc at 400k docs; at the 560k docs it produces the 768 MB container OOMs;…
0053	37,100.2	29,688	46,364	keep	tantivy index heap 32 -> 48 MB (default): runs 37100 / 37000 (mean 37.05k vs 35.3k, +5%), reads 46.4k both runs, RSS ~300 MB. Rebuild replays only the tail after a…
0054	36,917.2	29,183	46,702	discard	tantivy heap 64 MB: runs 36917 / 37106 (mean 37.0k) = same as 48 MB; 48 stays (less rebuild memory)
0071d	—	—	—	info	CONTAINER SOAK (768 MB, snapshots every 50): ~3 min at ~29k writes/s observed, server RSS 400-640 MB (peaks during 0.8-1.2 GB snapshot packs), no OOM, kv 2.2 GB, index…
0072b	—	12,881	—	info	CONTAINER SOAK 10 min (768 MB): OOM-killed after ~4 min of writes (RSS 230 -> 713 MB), snapshots grew to 2.4 GB packed in 30 s each every ~15 s of writes; sealed…
0073	34,839.4	28,732	42,245	keep	SAFETY: memory guard (--max-rss-mb 700: writers sleep 20 ms per request while RSS is above it; status.memory_guard) + snapshots only when segments since the last one are…
0073b	—	19,261	2,856	info	CONTAINER SOAK 10 min with memory guard + growth-proportional snapshots: 11.56M docs at 19.3k/s, 0 errors, no OOM (guard on twice at 713-749 MB), reads 2.9k/s over 11.5M…
0077	33,874.7	27,784	41,300	keep	SAFETY: the rebuild's segment replay pauses while RSS is over --max-rss-mb (indexer/flushes catch up), same guard as live writes. Bench unchanged
0081	30,712.7	28,080	33,594	keep	SAFETY: memory guard compares anon + dirty (statm resident-shared + cgroup file_dirty/writeback; io log now has anon_mb/dirty_mb) instead of resident size, which counts…
0083	30,984.5	28,389	33,817	keep	SAFETY: the snapshot (commit, stage, pack, upload, bucket GC, prune) runs on its own tokio task, one at a time; the seal loop keeps sealing, uploading segments and…
0084	33,414.5	27,548	40,531	keep	SAFETY: the rebuild's memory pause is bounded (10 s per segment, then continue with a warning): the 0083b soak rebuild hung forever because the rebuilt engine's own…
0085c	—	—	—	info	REBUILD MEMORY, same 3.8M-doc state (919 MB snapshot + ~700k-record tail), one core: FRESH PROCESS peaks at 259 MB heap (heaptrack; 101 MB fjall memtable copies, 43 MB…
0086	29,891.5	25,875	34,532	discard	MEMORY: mallopt(M_ARENA_MAX, 1). Bench: writes -7% (25.9k), peak rss 385 (better), old-engine residue 5 MB (vs 30-104) BUT the 3.8M-doc in-process rebuild ballooned to…
0087	35,081.1	34,283	35,900	keep	MEMORY+SPEED: jemalloc (tikv-jemallocator, background_thread, 1 s dirty/muzzy decay, narenas 2) as the global allocator. Writes +22% (34.3k/s both runs, p99 4.6 ms),…
0106	35,682.2	34,682	36,711	keep	OBSERVABILITY: /metrics now exposes hanudb_index_lag, hanudb_bucket_lag_txn/segments, kv_flushed_txn, index_txn, snapshot_seq, rss_anon_mb, memory_guard, index_segments,…

What we found

The tantivy "heap" budget is not the whole story: 64 MB of arena turned into ~350 MB of resident memory with three JSON fields (columnar/fast-field writers live outside the budget). 32 MB costs ~8 % indexing speed and halves the peak.
Inside a cgroup, dirty page cache counts against the memory limit: writing or extracting a 3 GB snapshot faster than the disk writes back gets the process killed even with a small heap. fdatasync every 32 MB on both paths bounds it.
The allocator decides how much of a rebuild you keep paying for: with glibc, an in-process engine swap (drop, wipe, rebuild 4.9 M docs) settled at 585 MB anon where a fresh process needs 229 MB, because freed pages stay in per-thread arenas (malloc_trim recovered 60 MB; one arena made it worse). jemalloc with a 1 s decay settles at 245 MB, and writes got 22 % faster on top (0087). mimalloc had lost earlier because its rebuild peak was compared with glibc's understated live number.
Never run a minute-long job inline in a control loop. The snapshot pack + upload sat in the seal loop, so for 40–130 s per snapshot nothing was sealed or uploaded (bucket-durable writes waited), the memory guard did not run, and requests timed out. Moving it to its own task (0083) turned 213 k timeouts into 0 and let anon memory stay flat. Corollary: the guard must live on a loop that never blocks.
Resident size is the wrong number for an OOM guard: it includes the clean, mmapped index files, which the kernel drops under pressure. What kills a process is anon + dirty page cache. The guard (0081) reads statm resident − shared plus the cgroup's file_dirty/file_writeback; on the bench that is 394 MB of pressure for 464 MB resident. And every path that writes gigabytes locally must fsync as it goes: the third one found (uploading a 3.2 GB snapshot to a file:// bucket through object_store) OOM-killed a 10-minute soak.
Sustained writes on one core need backpressure: when snapshot packing or merges hog the core, sealed memtables and index buffers pile up until the cgroup kills the process. A memory guard (sleep writers above 700 MB RSS) turned a 4-minute OOM into a clean 10-minute soak; and snapshot cost has to be proportional to growth, or a 2 GB database is re-packed every 15 seconds.

hanudbmemoryautonomous loop