Closes#856: native `fetch()` calls (Node 18+ built-in, and the identical
browser/WHATWG API — no static way to tell them apart, and no need to:
both make a real outbound HTTP request) were invisible to cross-repo
intelligence. A bare `fetch(url)` has no import and typically no local
definition, so registry resolution comes back empty and the call was
silently dropped instead of becoming an HTTP_CALLS edge.
The obvious fix — add "fetch" to the http_libraries substring table in
service_patterns.c — is wrong: that table is consulted by an early,
unconditional check in both pass_calls.c and pass_parallel.c that runs
before/regardless of whether the callee resolves to a real local
definition (by design, so `axios.get(url)` still classifies even when the
registry mis-binds bare `get` to an unrelated local method). That's safe
for "axios"/"requests" because nobody names their own function that; it
is not safe for "fetch", which collides with a plausible local identifier
(`function fetch(){}` or `const fetch = () => {}`).
Instead, cbm_service_pattern_is_global_fetch() is a new, narrow, exact-
match check consulted only in the *empty-resolution* fallback in both
files — the existing #523 path for unindexed external libraries. By the
time that branch runs, the registry has already had its chance to
resolve "fetch" to a local/imported definition; only a genuine miss
reaches the new check. This also means a member call like `repo.fetch()`
is naturally excluded (its callee_name is "repo.fetch", not the bare
"fetch" this checks for).
pass_parallel.c's empty-resolution branch calls the low-level
emit_http_async_service_edge() directly rather than going through
emit_service_edge(), which re-derives its own classification from
res->qualified_name via cbm_service_pattern_match() — a call with
"fetch" would come back CBM_SVC_NONE there and silently fall through to
a plain CALLS edge.
Three new tests in test_pipeline.c:
- bare fetch() -> HTTP_CALLS, sequential path (< 50 files)
- bare fetch() -> HTTP_CALLS, forced through the parallel resolver
(>= 50 files)
- a local `function fetch(){}` shadowing the global -> plain CALLS to
the local definition, zero HTTP_CALLS (the false-positive this PR
must not introduce), bundled with a `repo.fetch()` member-call check
in the first test
Verified end-to-end, not just by inspection: prod build clean under
-Wall -Wextra -Werror; test build clean under ASan+UBSan; full suite
5936 passed, 1 skipped (pre-existing, unrelated), 0 failed; clang-format
clean on all touched files; lint-cppcheck's pre-existing failures
(extract_defs.c, compat_regex.c) reproduced identically on unmodified
main via stash/pop, confirming they predate this change.
Refs #592.
Signed-off-by: Alexandros Pappas <11921291+apappas1129@users.noreply.github.com>
Imports: per-grammar parsers for hare/pascal/powershell/scheme/racket/starlark/tcl/teal/zsh + css/html/cmake/bitbake/kconfig/gn/just/nix/jsonnet/pkl/nickel/thrift/capnp/d/tablegen/crystal/fsharp/ada; node-extraction for D/PowerShell/Pascal/Julia/F#; INHERITS for those. Pipeline edges: register Variable/Field defs for WRITES/READS resolution; CONFIGURES pass consuming env_accesses; DEPENDS_ON from go.mod/requirements.txt; SQS dot-form QN. Golden updates: julia +Class, just +Function (correct new extraction).
Format drift had accumulated across recent commits (the dry-run lint enforces clang-format-20, which had not run on these commits yet). No functional changes — whitespace/layout only, verified by a clean cppcheck + clang-format + NOLINT lint-ci pass.
cbm_service_pattern_match runs once per resolved CALL edge inside
emit_service_edge — that's 7 pattern lists × ~30 patterns × strstr
= ~210 strstrs per call. On kubernetes (~600k resolved CALL edges)
the SAME resolved QN ("fmt.Errorf", "context.Context.Done",
"errors.New", etc) repeats thousands of times across files. Caching
turns the linear pattern-list scan into one hash lookup after the
first miss.
API additions in service_patterns.h:
void cbm_service_pattern_cache_begin(void);
void cbm_service_pattern_cache_end(void);
Cache is TLS (no contention) scoped to the resolve_worker's lifetime
in the parallel_resolve phase. Capacity 8192 entries per worker —
covers typical project QN diversity without rehashing.
Encoding: enum value + 1 in the void* slot (0/NULL means "miss").
Two helpers svc_enum_to_ptr / svc_ptr_to_enum hide the trick.
Benchmark on kubernetes (12 workers, 20,650 files):
- parallel_resolve wall: 98,520 -> 90,104 ms (-8.5% vs Chunk 1
baseline, -12.8% vs original 103,281 baseline)
- resolve_calls CPU: 992,427 -> 885,663 ms (-10.8% vs Chunk 1,
-5.4% vs original — Chunk 1 added overhead that Chunk 2 recovers
AND surpasses)
- resolve_usages CPU: 11,977 -> 10,944 ms (-8.6%)
- total_loop CPU: 1,005,409 -> 897,561 ms (-10.7%)
- Tests: 3616/0
- Graph: 297,082 -> 297,080 nodes (-2), 1,153,482 -> 1,156,097 edges
(within 0.2%, semantic-preserving)
- zod/serilog sanity: edges within +/-3 of baseline
- Peak RSS: 7.46 -> 7.75 GB (+290 MB for 12 × 8192-entry caches)
Cumulative trajectory: original broken state ~35 min K8s ->
1.92 min wall (-49× total). The per-file is_import_reachable cache
(Chunk 1) + the per-worker service-pattern cache (Chunk 2) together
save ~13s of resolve wall time. Next chunks target the gbuf_insert_
edge hot path and remaining strstr scans.
Pipeline: gRPC/GraphQL/tRPC client libs in service_patterns, edge emitters
in pass_parallel, __grpc__ Route nodes from .proto service defs in
pass_route_nodes, cross-repo matchers in pass_cross_repo. Also adds MQTT,
NATS, Dapr async patterns.
Protobuf extraction: service as Class, rpc as Function with rpc_name
fallback in func_name_node. Route nodes created by line-range matching
rpc Functions to enclosing service Classes.
Tested: Online Boutique protos produce 16 gRPC Route nodes.
Route registration detection now works for unresolved local variables
(app.include_router, app.mount) by matching callee name suffix when
QN resolution fails. Scans ALL arguments for URL paths — handles
prefix=, path=, route= keyword args across frameworks.
New route_reg_suffixes: include_router, mount, add_url_rule,
register_blueprint, use, register, add_route, add_api_route.
Infrastructure bindings: YAML/JSON extraction captures topic→URL
pairs from subscription configs (push_endpoint, uri). Creates
INFRA_MAPS edges linking async topic Routes to endpoint URLs.
Improved match_infra_routes: matches prefix Routes from include_router,
supports service-only matching for generic "/" handlers, correct
Cloud Run hostname parsing for new project-number format.
Alexandros Pappas