mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-11-06 08:35:21 +08:00
Martin Vogel
75a0df688f
Previously cbm_pipeline_pass_lsp_cross ran as a separate sequential pass that re-read every source file from disk and re-parsed every tree-sitter tree on a single thread — a 50x regression on the kubernetes Go monorepo (extract 10s, lsp_cross 9+ min when killed). Three changes: 1. CBMFileResult gains const char *source / int source_len. The parallel extract worker copies source bytes into result->arena (per-file 100MB cap, project-wide 2GB atomic cap) so the fused cross-LSP step can run without disk I/O. Source pointer is borrowed by downstream readers; arena owns the storage. 2. Cross-LSP work moves into the parallel resolve worker. Each worker processes its file with the existing parallel infrastructure (worker pool + work stealing). Slab is reclaimed after each per-file cross-LSP call to bound peak slab memory. The previously separate lsp_cross pass entry function is kept for the sequential pipeline path; the parallel path no longer invokes it. 3. CBMModuleDefIndex (gopls 'package summary' pattern) — an inverted index module_qn -> list of def indices built once in O(D) before resolve_worker fires. Per file, cbm_pxc_filter_defs_for_file slices all_defs down to the file's own_module + imported modules only, typically 50-100x smaller than the global ~110k defs on kubernetes. Drops the per-file registry build from O(all_defs) to O(relevant). Other plumbing: cbm_parallel_resolve signature gains all_defs, def_count, def_modules, module_def_index params. pipeline.c builds the index between registry_build and parallel_resolve and frees it after. pipeline_incremental and tests/test_parallel updated to match. Helpers in pass_lsp_cross.c exposed via the header: cbm_pxc_has_cross_lsp, cbm_pxc_collect_all_defs, cbm_pxc_run_one(_ts), cbm_pxc_ts_modes, cbm_pxc_build_module_def_index, cbm_pxc_filter_defs_for_file. Kubernetes wall-time: 35 min -> 6.24 min (~5.6x) before the followup perf commits in this series.