Match sequential CALLS property capacity to the parallel pass for edge JSON emitted through calls_emit_edge so changed-file overlay/exact paths preserve later call arguments.
Require THROWS/RAISES source nodes to be callable scopes through a shared pipeline helper, preventing sequential file/module fallback edges from diverging from parallel full-index behavior.
Add regression coverage for the eighth CALLS arg and top-level raise sequential/parallel parity. Validated with focused canaries, full pipeline and parallel suites, source-safety, product build, guarded FastAPI matrix, and an isolated guard-removal experiment that still failed active equality.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Share the route path/handler scanner between sequential and parallel call passes, add FastAPI/Starlette websocket route-registration suffixes, and let sequential unresolved route-registration suffix calls emit Route registration facts when the existing route-literal guard accepts the path.
Add a sequential/parallel parity canary for FastAPI websocket decorators. Validation: make -f Makefile.cbm -j16 build/c/test-runner; CBM_ONLY_SUITE=parallel build/c/test-runner; CBM_ONLY_SUITE=pipeline CBM_ONLY_TEST=fast_route build/c/test-runner; make -f Makefile.cbm -j16 cbm; bash scripts/check-source-safety.sh; git diff --check.
FastAPI matrix evidence: guarded production remains safe by full fallback in /private/tmp/cbm-fastapi-matrix-websocket-parity-current-20260704T2215.json. Forced exact cap64 now preserves the previously missing tests/test_ws_router websocket Route nodes and 9 route-registration edges, but still fails canonical edge parity; keep scoped-LSP full fallback until the remaining edge-attribution mismatch is fixed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Parallel resolve used an unresolved route-registration suffix fallback for callees such as .get and .post. When no route literal was present, that fallback could fall through to a normal CALLS edge using the source node as the target, which fabricated self-calls for ordinary receiver accessors like ec.get(e.type).
Return without emitting when the suffix-only fallback has no route literal, while preserving resolved route API calls and unresolved route registrations that do have a valid route path. Add a parallel resolver canary that first proves extraction sees ec.get and then asserts no self CALLS edge is emitted.
Validation: build/c/test-runner rebuilt; CBM_ONLY_SUITE=parallel CBM_ONLY_TEST=parallel_unresolved_route_suffix_does_not_emit_self_call passed; CBM_ONLY_SUITE=parallel passed 27 tests; lint-source-safety passed; git diff --check passed; make -f Makefile.cbm cbm built and signed; MCP self-dogfood overlay matrix passed 5/5 with cleanup.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Keep the shared LSP override index semantically aligned with the linear resolver. C++ textual callees using :: or -> now match dotted LSP QNs, and allowed original-text reason joins cover function-pointer and implicit-call strategies without falling off the indexed fast path.
Use the project cbm_strdup compatibility wrapper for indexed keys and retain the existing incomplete-index fallback to the linear helper when key allocation or formatting fails.
Add focused parallel-suite tests for C++ segment matching, reason-gated joins, indexed/linear parity, and overlong-key fallback.
Validation: git diff --check; bash scripts/check-source-safety.sh; make -j8 -f Makefile.cbm build/c/test-runner; CBM_ONLY_SUITE=parallel ./build/c/test-runner; CBM_ONLY_SUITE=c_lsp ./build/c/test-runner; CBM_ONLY_SUITE=pipeline ./build/c/test-runner.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Replace the plain cbm_init() static guard with a C11 atomic one-shot state machine so parallel extraction callers wait for allocator/library initialization instead of racing the guard.
Make the macOS mach timebase cache use the same release/acquire publication pattern while leaving the Linux, BSD, and Windows timing paths unchanged.
Add a parallel cbm_init() idempotence regression registered first in the parallel suite.
Validation: make -f Makefile.cbm build/c/test-runner; CBM_ONLY_SUITE=parallel ./build/c/test-runner; CBM_ONLY_SUITE=platform ./build/c/test-runner; bash scripts/check-source-safety.sh; git diff --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Add a 66-file Python fixture that forces the production pipeline across the sequential and parallel branches. The test runs CBM_WORKERS=1, 2, 4, and default workers against isolated SQLite stores, then checks integrity, project root persistence, file-hash count, representative qualified names, and node/edge/type-count parity.
Use bounded formatting and existing test/platform helpers so fixture paths fail closed on truncation and cleanup removes SQLite sidecars without adding stdout-sensitive debug output.
Validation: make -f Makefile.cbm build/c/test-runner && CBM_ONLY_SUITE=parallel ./build/c/test-runner && bash scripts/check-source-safety.sh
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
A call carrying enough long arguments drove append_args_json()'s running
position past the fixed CBM_SZ_2K `props` stack buffer in
emit_normal_calls_edge(): format_call_arg() returns snprintf's *untruncated*
length, so `pos += (size_t)n` could exceed `bufsize`, after which the
trailing `buf[pos] = '\0'` (and `buf[pos++] = ']'`) wrote out of bounds. The
stack canary caught it as SIGABRT, so full-repo indexing of large TypeScript
codebases crashed the server in the parallel resolve pass
(emit_service_edge -> emit_normal_calls_edge -> finalize_and_emit ->
append_args_json). Confirmed with AddressSanitizer:
stack-buffer-overflow WRITE at pass_parallel.c:1124, 'props' (2048 B).
Fix: when an argument does not fully fit, roll back to before its separator
and stop appending (atomic field, matching append_json_string's behaviour),
so `pos` can never advance past the buffer.
Add regression test parallel_args_json_no_overflow: indexes a fixture whose
single call carries 60 long string args (args JSON well past 2 KB); under the
ASan test build it aborts without this fix and passes with it.
Signed-off-by: Andrius Skerla <1492322+rainder@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add scripts/check-no-test-skips.sh (run from lint) which fails the lint phase on any plain SKIP() or direct tf_skip_count manipulation; only SKIP_PLATFORM() (for genuinely platform-specific tests) is tolerated. Add FAIL() and SKIP_PLATFORM() helpers to the test framework and convert the remaining SKIP()/perf-gated skips across the suite into pass-or-fail assertions, so a suite that cannot meet its preconditions reports a red failure instead of a silent skip.
emit_grpc_edge produced two classes of bad output:
1. Wrong service name: the suffix list stripped "ServiceClient"/"ServiceGrpc"
before "Client"/"Grpc", so FooServiceClient collapsed to "Foo" instead of
the proto-declared "FooService" — breaking cross-repo Route matching.
2. Phantom Routes: extract_grpc_service_method returned true for any
"<recv>.<method>" call, so ordinary receiver vars (_provider.GetGroup,
_builder.AddSomeService) became __grpc__provider/... Routes matching no
.proto anywhere.
Strip only the trailing stub/client token (Client/Stub/Grpc/BlockingStub/…),
preserving "Service", and require that a recognized suffix was actually present
before emitting — the suffix match is the gRPC stub-type signal. Plain receiver
vars no longer yield Routes.
extract_grpc_service_method is no longer static (declared in pipeline_internal.h)
so it is unit-testable.
The gopls 'package summary' pattern, applied properly: build the cross-
LSP CBMTypeRegistry ONCE per language at pipeline time (in a dedicated
cross_lsp_arena), share it READ-ONLY across all resolve workers via the
CBMCrossLspRegistries struct. Per-file work then becomes just tree-
sitter parse + AST walk + O(1) hash lookups — zero per-file registry
rebuilds for languages with a pre-built registry.
Each LSP gains two new public functions:
CBMTypeRegistry *cbm_<lang>_build_cross_registry(arena, defs, def_count)
— filters all_defs to its language, init + stdlib + registration
loop + cbm_registry_finalize, returns finalized registry.
void cbm_run_<lang>_lsp_cross_with_registry(arena, source, ...,
CBMTypeRegistry *reg, ...)
— skips the per-file build AND the per-file Phase 1b/1c AST scans
(those mutate the registry, which would race on a shared read-
only reg). Trade-off: a few file-local type aliases discovered
only by Phase 1b are missed; in practice the per-file extract
LSP already captured them via embedded_types/field_defs/etc.
Tested with parallel_python_lsp_override_cross_file_emits_lsp_
strategy_edges — fused + filtered + prebuilt path stays correct.
CBMLSPDef gains a CBMLanguage lang field set in pxc_build_lsp_def so
the per-language build functions can filter the global all_defs[].
cbm_parallel_resolve signature now takes CBMCrossLspRegistries* instead
of a single CBMTypeRegistry*. The resolve_worker dispatches:
cbm_pxc_registry_for_lang(lang) returns the right pre-built reg if any;
the worker then switch()es to the matching _with_registry variant.
Languages without a _with_registry variant fall through to the existing
filter + per-file build path.
Kubernetes (Go-heavy, 12k files, 139k defs) measured perf:
parallel_resolve: 5.51 min (strdup-removal baseline) -> 1.86 min
= 2.96x speedup
parallel.resolve.lsp_cross.slow lines: 1,258 -> 2
(-99.8%; the 2 remaining are Python files, not Go)
Peak RSS: 6.29 GB -> 6.23 GB (slight decrease — pre-built registry
shared, no per-file scratch)
All 3,616 tests pass; zod/serilog graphs exact match (per-file path
unaffected).
Cumulative trajectory from the original broken state on kubernetes:
35 min -> 19 min (finalize) -> 6.24 min (module-def filter) ->
5.51 min (strdup-removal) -> 1.86 min (Tier 2 Go) = 18.8x total.
Languages wired so far: Go, Python. C/C++, TS/JS, PHP, C# still use the
per-file filter+build path until their _with_registry variants land in
follow-up commits.
Previously cbm_pipeline_pass_lsp_cross ran as a separate sequential
pass that re-read every source file from disk and re-parsed every
tree-sitter tree on a single thread — a 50x regression on the
kubernetes Go monorepo (extract 10s, lsp_cross 9+ min when killed).
Three changes:
1. CBMFileResult gains const char *source / int source_len. The
parallel extract worker copies source bytes into result->arena
(per-file 100MB cap, project-wide 2GB atomic cap) so the fused
cross-LSP step can run without disk I/O. Source pointer is
borrowed by downstream readers; arena owns the storage.
2. Cross-LSP work moves into the parallel resolve worker. Each worker
processes its file with the existing parallel infrastructure
(worker pool + work stealing). Slab is reclaimed after each
per-file cross-LSP call to bound peak slab memory. The previously
separate lsp_cross pass entry function is kept for the sequential
pipeline path; the parallel path no longer invokes it.
3. CBMModuleDefIndex (gopls 'package summary' pattern) — an inverted
index module_qn -> list of def indices built once in O(D) before
resolve_worker fires. Per file, cbm_pxc_filter_defs_for_file slices
all_defs down to the file's own_module + imported modules only,
typically 50-100x smaller than the global ~110k defs on kubernetes.
Drops the per-file registry build from O(all_defs) to O(relevant).
Other plumbing: cbm_parallel_resolve signature gains all_defs,
def_count, def_modules, module_def_index params. pipeline.c builds
the index between registry_build and parallel_resolve and frees it
after. pipeline_incremental and tests/test_parallel updated to match.
Helpers in pass_lsp_cross.c exposed via the header: cbm_pxc_has_cross_lsp,
cbm_pxc_collect_all_defs, cbm_pxc_run_one(_ts), cbm_pxc_ts_modes,
cbm_pxc_build_module_def_index, cbm_pxc_filter_defs_for_file.
Kubernetes wall-time: 35 min -> 6.24 min (~5.6x) before the followup
perf commits in this series.
Per-file LSP (run from cbm_extract_file) only sees one file's defs, so
callees whose receiver type comes from an imported module stayed
unresolved — even though Go/C/Py/TS already had cross-file entry points
(cbm_run_X_lsp_cross). Those entry points existed but no pipeline pass
ever invoked them. PHP didn't even have a cross-file entry point.
This change lands the missing pieces:
1. cbm_run_php_lsp_cross + cbm_batch_php_lsp_cross — PHP cross-file
resolver mirroring py_lsp_cross / ts_lsp_cross. Reuses PHPLSPContext,
php_lsp_init, php_lsp_add_use, php_lsp_process_file. Handles PHP-
specific labels (Class / Interface / Trait / Enum / Type) and uses
def_module_qn so cross-file return types qualify against the def's
own module rather than the importing module's.
2. pass_lsp_cross.c — new pipeline pass. For every cached file result,
builds a project-wide CBMLSPDef[] (CBMDefinition → CBMLSPDef
converter), per-file import map (from gbuf IMPORTS edges), then
dispatches to the matching cbm_run_X_lsp_cross. Output strings get
copied into the per-file arena (with O(N²) memory bug avoided by
running each file inside a fresh scratch arena that gets destroyed
after the LSP call — the LSP allocates its registry + stdlib +
project defs in scratch, not in the caller's persistent arena).
3. Wired into both pipelines:
- sequential: definitions → k8s → lsp_cross → calls → ...
- parallel: parallel_extract → registry_build → lsp_cross →
parallel_resolve → ...
4. Updated tests/test_parallel.c run_parallel helper to mirror the new
parallel ordering so existing cross-file Python/PHP tests now exercise
pass_lsp_cross.
Languages covered: Go, C/C++/CUDA, Python, TS/JS/JSX/TSX, PHP. PHP cross
file is new. The other four had functions but no pipeline integration
until this change.
Verified on a 1670-file FastAPI index: pass_lsp_cross processes 1050
Python files (525 non-LSP files skipped), collects 4070 project defs in
9.2s, and parallel_resolve subsequently reports lsp_overrides=568 — the
LSP-strategy attribution gain over per-file alone.
Per-LSP emit functions dedup by (caller_qn, callee_qn) within their own
run; pxc_append_results dedups again at append time so cross-file
entries duplicating per-file output get dropped instead of inflating
result->resolved_calls.
Per-file py_lsp emits resolved_calls.callee_qn as the raw import-module
path (e.g. `greeter.Greeter` from `from greeter import Greeter`) rather
than the project-qualified QN the gbuf stores (`<project>.greeter.Greeter`).
Before this change, the LSP match succeeded (lsp_overrides counter
incremented) but the downstream cbm_gbuf_find_by_qn lookup missed
silently and the edge was dropped — the failure mode the new
lsp_overrides telemetry was designed to surface.
The LSP layer can't tell in-project imports (qualify) from external
imports (don't qualify, e.g. `os.path`) without consulting the gbuf,
which is built downstream. So normalise at the consumer instead: try
the LSP-emitted QN as-is first; on miss, retry with
`<project>.<callee_qn>`. If that also misses, drop the edge — same as
before, target is external/unindexed.
New helper cbm_pipeline_lsp_target_node in lsp_resolve.h is shared by
both pipelines (sequential pass_calls.c and parallel pass_parallel.c)
so they continue to admit identical sets of LSP overrides. The helper
also canonicalises res.qualified_name to the gbuf node's QN, so
downstream edge property serialisation shows the project-qualified
form even when fallback resolution kicked in.
Cross-file regression test parallel_python_lsp_override_cross_file_-
emits_lsp_strategy_edges pins the two-file Greeter scenario that
originally exposed the bug.
The sequential pass_calls and the parallel cbm_parallel_extract pipelines
each carried their own copy of the LSP-resolved-call lookup with
divergent confidence floors and different match semantics. Most production
indexing runs the parallel path, so projects produced different
CALLS-edge attributions depending on which pipeline mode kicked in:
pass_calls.c find_lsp_resolution floor 0.6 (CBM_LSP_CONFIDENCE_FLOOR)
pass_parallel lsp_override_resolution_pp floor 0.5 (hardcoded)
Both pipelines correctly invoke cbm_run_*_lsp through cbm_extract_file,
so result->resolved_calls is populated identically — the bug was at the
consumer.
Changes
-------
- New src/pipeline/lsp_resolve.h: static-inline cbm_pipeline_find_lsp_resolution
and CBM_LSP_CONFIDENCE_FLOOR (0.6f). Single source of truth for the
match rule (caller_qn equality + callee_qn short-name match + floor),
with rationale and language-coverage notes inline.
- pass_calls.c: include lsp_resolve.h, drop the local copy, rename the
call site to cbm_pipeline_find_lsp_resolution.
- pass_parallel.c: include lsp_resolve.h, drop lsp_override_resolution_pp,
use cbm_pipeline_find_lsp_resolution at resolve_file_calls. Add an
int lsp_overrides counter to resolve_worker_state_t (with adjusted
cache-line padding for the new field), aggregate it per-worker,
and emit it in the parallel.resolve.done log line so divergence
between pipelines becomes observable in production telemetry.
- tests/test_parallel.c: parallel_python_lsp_override_emits_lsp_strategy_edges
builds a single-file Python scenario (Greeter class + main() in one
file), runs the parallel pipeline, walks every CALLS edge, asserts
at least one carries an "lsp_*" strategy in its properties_json.
Pins both halves of the wiring at once: the shared find helper IS
consulted in the parallel path, AND the LSP-attributed strategy
survives all the way to the gbuf edge. Single-file rather than
cross-file because per-file py_lsp's resolved_calls callee_qn uses
the import module path rather than the project-qualified QN, so
cross-file LSP edges currently get dropped by cbm_gbuf_find_by_qn —
a separate cross-file QN-mismatch bug to track elsewhere.
Full suite: 3250 passed, 0 failed (was 3249; +1 new regression test).
- cbm_mkdtemp: _mktemp + _mkdir on Windows
- cbm_setenv/cbm_unsetenv: _putenv_s on Windows
- cbm_pipe: _pipe on Windows
- Replace bare POSIX calls in 12 test files with compat wrappers
- All test files now include compat.h
Enforce WarningsAsErrors: '*' with zero suppressions of entire check categories.
All non-FP warnings fixed at source; true false positives get per-instance NOLINT
with reasoning. Fixes include: braces around statements (1679), isolate declarations
(75), explicit casts for multilevel pointer conversions, missing includes, function
parameter documentation via NOLINT, and proper NOLINTBEGIN/NOLINTEND nesting.
Complete C implementation of the indexing pipeline (src/), parallel worker
pool, graph buffer with merge support, SQLite writer, and 1893-test suite.
Linter setup: zero warnings from clang-tidy (all checks enabled, no NOLINT
suppressions), cppcheck, and clang-format. All issues fixed at source —
proper headers for external linkage, GROW_ARRAY macro restructured to
eliminate type parameter, null-deref paths guarded, named intermediates
for suspicious-argument checks.
Andrew Hundt