Two wall-clock fixes from a linux-kernel (8.5M nodes) profiling run:
1. Extraction back-pressure futility latch: when a full collect+nap cycle ends
still over the RSS budget, the resident floor (accumulated graph + retained
sources) -- not in-flight transients -- holds the memory, so napping again on
every subsequent pull reclaims nothing and only idles workers (measured: one
full cycle per pull, ~390 s at 79% avg CPU across 63k pulls). Latch bp_futile
after the first futile cycle, proceed with the designed soft overshoot, and
re-arm via the cheap per-pull probe once RSS drains under budget. WARN
mem.backpressure.futile exactly once per latch event (0->1 transition).
2. Supervised-worker fast exit: the index worker piecemeal-freed a multi-GB
graph (and store) right before process death (measured: minutes of free()
in the tail). In worker role, skip cbm_pipeline_free and _Exit after the
response is written and flushed; the OS reclaims memory wholesale. In-process
paths (tests, kill switch, degrade) free normally, keeping ASan/LSan
meaningful.
Reproduce-first: pipeline_backpressure_futile_nap_disengages (64-file fixture >
MIN_FILES_FOR_PARALLEL so the parallel gate actually runs, CBM_MEM_BUDGET_MB=1,
deterministic nap-cycle counter; engagement guard cycles>=1 prevents a vacuous
pass). RED on the old gate: 64 cycles (one per pull) > bound cores+2. GREEN with
the latch. Suites: pipeline 216, mcp 140, subprocess 17 -- all pass.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel
1322dba099