mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-07 08:43:35 +08:00
Martin Vogel
2ee0c68d4a
Replaces the prior Robin-Hood open-addressing implementation with Verstable (https://github.com/JacksonAllan/Verstable), a 2024 state- of-the-art C hash table. The public API in foundation/hash_table.h is UNCHANGED — CBMHashTable is now an opaque type and all 14 production callers (graph_buffer, registry, pipeline passes, service_patterns, watcher, semantic, etc.) get the new impl transparently. str_intern.c keeps its specialised (ptr+len) hash — different semantics, not a clean swap target. Why: per-sub-phase profiling pinpointed cbm_pipeline_lsp_target_node and cbm_registry_resolve as the dominant resolve_calls cost. Both ultimately hit cbm_ht_get; both run millions of times on a kubernetes index. Verstable's key innovations target exactly this: * 4-bit hash-fragment metadata in a uint16_t sidecar array lets chain walks skip most key comparisons WITHOUT touching the bucket payload (cache wins on hot lookups). * Quadratic probing with per-bucket displacement chains (not linear Robin-Hood psl). * Tombstone-free deletion that moves at most one existing key. Vendored at internal/cbm/vendored/verstable/verstable.h (2069 lines, 84KB, MIT). Wrapped as a const char* → void* template inside hash_table.c. The header struct is opaque (forward declaration only); audited the entire tree — no caller accesses internals or takes sizeof(CBMHashTable), so opacity is safe. Test refactor: test_hash_table.c had two impl-specific tests: * ht_robin_hood_bounded_psl — Robin-Hood specific; removed. ht_stress_10000 covers the functional equivalent. * ht_power_of_two_capacity — renamed ht_create_small_capacity, replaced struct probes with a functional insert+lookup check. ht_create_capacity_zero/one similarly de-internalised. Benchmark on kubernetes (12 workers, 20,650 files): - parallel_resolve wall: 36,376 -> 27,217 ms (-25.2%) - pipeline.done total: 61,485 -> 51,745 ms (-15.8%) - registry_build: ~1,000 -> 779 ms (faster — registry uses cbm_ht_set heavily during build) - All 3,615 tests pass; hash_table suite 100% green - ASan/UBSan/SEGV: none - Peak RSS: ~6.83 GB - Graph: nodes 297,082 -> 297,102 (+20, 0.007%), edges 1,155,444 -> 1,154,025 (-1,419, -0.12%) — within tolerance; minor delta likely from hash-iteration-order differences affecting tie-breaks in resolution. Sanity (zod/serilog): nodes exact, edges within 0.1%. Cumulative trajectory from broken-state baseline ~35 min on K8s -> 51.7s wall = ~40x total speedup.