Previously cbm_pipeline_pass_lsp_cross ran as a separate sequential
pass that re-read every source file from disk and re-parsed every
tree-sitter tree on a single thread — a 50x regression on the
kubernetes Go monorepo (extract 10s, lsp_cross 9+ min when killed).
Three changes:
1. CBMFileResult gains const char *source / int source_len. The
parallel extract worker copies source bytes into result->arena
(per-file 100MB cap, project-wide 2GB atomic cap) so the fused
cross-LSP step can run without disk I/O. Source pointer is
borrowed by downstream readers; arena owns the storage.
2. Cross-LSP work moves into the parallel resolve worker. Each worker
processes its file with the existing parallel infrastructure
(worker pool + work stealing). Slab is reclaimed after each
per-file cross-LSP call to bound peak slab memory. The previously
separate lsp_cross pass entry function is kept for the sequential
pipeline path; the parallel path no longer invokes it.
3. CBMModuleDefIndex (gopls 'package summary' pattern) — an inverted
index module_qn -> list of def indices built once in O(D) before
resolve_worker fires. Per file, cbm_pxc_filter_defs_for_file slices
all_defs down to the file's own_module + imported modules only,
typically 50-100x smaller than the global ~110k defs on kubernetes.
Drops the per-file registry build from O(all_defs) to O(relevant).
Other plumbing: cbm_parallel_resolve signature gains all_defs,
def_count, def_modules, module_def_index params. pipeline.c builds
the index between registry_build and parallel_resolve and frees it
after. pipeline_incremental and tests/test_parallel updated to match.
Helpers in pass_lsp_cross.c exposed via the header: cbm_pxc_has_cross_lsp,
cbm_pxc_collect_all_defs, cbm_pxc_run_one(_ts), cbm_pxc_ts_modes,
cbm_pxc_build_module_def_index, cbm_pxc_filter_defs_for_file.
Kubernetes wall-time: 35 min -> 6.24 min (~5.6x) before the followup
perf commits in this series.
Martin Vogel
75a0df688f