active13 September 2026 → 14 September 2026 · 6 parts

HanuDB, tuned while I slept

A document database improved by an autonomous agent: one change at a time, two benchmark runs, hard gates, keep or discard.

HanuDBRustobject storagebenchmarksnegative results

In one breath145 logged runs. The benchmark score went from 9,046.8 to 40,116.4. The loop kept 74 changes, discarded 27, crashed 3 times, and logged 41 unscored soaks and checks. Several of its findings reversed what a person would assume.

Summary

The rule was simple: the human is asleep; never stop and never ask. A fixed benchmark ran the server inside a container shaped like the smallest free virtual machine — one core, 768 MB — and scored the geometric mean of writes and reads per second. Any change that broke recovery from the bucket, blew the memory limit or cost too many bucket operations scored zero.

Each idea got two runs because identical builds differ by eight per cent. A change was kept only if the mean of both beat the baseline by more than three per cent, or made the code simpler at the same speed. The parts below group every run by what it was trying to fix, with the lessons it left behind — including the ones that went the wrong way.

Benchmark score of each kept experiment — higher is betterscore

The protocol, in order

01Writes: the log, group commit and the diskWith two durability watermarks, every "how far are we" number must follow the smaller one. After 0091 the current.log staleness floor (meta.next_txn) followed the index watermark, so after a SIGKILL the records the kv had not flushed yet were dropped as…adopted 02Reads: filters, sort and full-text on one coreOn one core, every background thread competes with request handling: moving work off the write lock onto a spare thread (0003) made throughput worse. Blocking writers briefly is free; extra CPU is not.adopted 03Memory: allocators, guards and the cgroupThe tantivy "heap" budget is not the whole story: 64 MB of arena turned into ~350 MB of resident memory with three JSON fields (columnar/fast-field writers live outside the budget). 32 MB costs ~8 % indexing speed and halves the peak.adopted 04Recovery: snapshots, rebuilds and the bucketProcess discipline: a set -e bug in keep.sh (empty experiments/ made ls | wc fail) meant the baseline snapshot never advanced; a later discard reset the tree to day-one code and three experiments measured the wrong program. The scripts now verify…adopted 05Product: what an application needs from it22 of 22 changes kept.adopted 06Soaks: ten minutes, fourteen million documentsDocker --cpus=1 (CFS quota) is a bad stand-in for a small VM: 50 ms throttling stalls and ±8 % noise. Pin a core instead.insight