Passing an unrecognised project name triggered a buffer overflow while
building the available-projects error list: collect_db_project_names did
snprintf(out+offset, out_sz-offset, ...) with no bound, so once the
listed names exceeded the 4 KB projects buffer, (out_sz - offset)
underflowed (unsigned) to a huge size and snprintf wrote past the buffer.
Fixed in 21c73b5 (PR #352) with an offset>=out_sz bounds check + clamp.
This test fills a temp CBM_CACHE_DIR with 40 long-named .db files
(~5 KB of names) and issues a bad-project search_graph call; the test
build runs under -fsanitize=address so the old underflow would abort here.
Closes#235.
A C++ function template with a defaulted type param and an `auto`
parameter, called with matching arg count, previously segfaulted in the
definitions pass (unbound `U` flowed through type substitution as NULL).
Fixed by the cbm_type_substitute NULL-preservation guard; this locks in
a reproduction so it can't regress.
Closes#312.
The CBM_WORKERS env tests called bare setenv/unsetenv, which are
POSIX-only and undeclared on the MSYS2 Windows test build. Switch to the
cbm_setenv/cbm_unsetenv compat wrappers (a no-op #define to setenv on
POSIX) and include foundation/compat.h. Fixes the test-windows build.
- cbm_arena_alloc returns NULL on a NULL arena (both arena.c copies)
instead of dereferencing it — defense-in-depth against the NULL-arena
type-allocation path that could crash the LSP type layer.
- discover: add "vendored" to the always-skip directory list.
Distilled from #374, taking the parts not already on main. The PR's
ts_lsp tuple-arena allocation, the C/C++ template formal-count clamp,
and the type_rep unbound-param preservation it also carried all landed
independently in v0.7.0 / via #322 / #360, so only these two
defensive improvements remain. Relates to #390.
Inside a container, sysconf(_SC_NPROCESSORS_ONLN) and sysinfo() report
the host's CPU count and RAM, not the cgroup's effective quota — so
cbm_default_worker_count over-provisions workers and the mmap budget can
exceed the cgroup memory cap, driving OOMKills (#363).
detect_system_linux now reads the cgroup limits:
- cbm_detect_cgroup_cpus: cgroup v2 cpu.max, then v1 cpu.cfs_quota_us /
cpu.cfs_period_us → ceil(quota/period); -1 when unlimited.
- cbm_detect_cgroup_mem: cgroup v2 memory.max, then v1
memory.limit_in_bytes; 0 when unlimited or the v1 near-ULLONG_MAX
sentinel.
- Effective value is min(cgroup, host), guarding against mis-mounted
cgroups that report more than the host.
Helpers read via a bounded read_small_file (fopen "re", capped fread)
and are exposed through system_info_internal.h for unit tests that drive
them against a fake cgroup tree. The test teardown uses opendir/unlink/
rmdir (no shell spawn).
Distilled from #365 onto current main (unioned the test additions with
the CBM_WORKERS tests from #364, and replaced the test-cleanup
system("rm -rf") with a shell-free recursive remove). Closes#363
together with #364.
Adds a CBM_WORKERS env knob (clamped to [1, 256], invalid values warned
and ignored) that explicitly sets the parallel-indexing worker count.
In containers, sysconf(_SC_NPROCESSORS_ONLN) reports host CPUs rather
than the cgroup's effective quota, so a 1-vCPU pod on a 16-core node
otherwise spawns ~16 workers — the dominant OOMKill driver. When unset,
behaviour is unchanged.
From #364.
Root cause: unbound template parameters flowed through cbm_type_substitute
as NULL, and call-expression return handling then dereferenced the NULL
substituted type. Reproduced on MangoHud's src/hud_elements.cpp.
- type_rep: cbm_type_substitute now returns the original param `t` (not
NULL) when a type arg is unbound, in both the TYPE_PARAM and named-param
paths — unbound params stay intact instead of poisoning the type.
- c_lsp: guard `if (!ret) return cbm_type_unknown();` before unwrapping
the substituted return type; guard the substitute result before use; add
template_function call-node resolution (qualified/scoped/plain names).
- extract_defs: factor template-inner lookup into find_cpp_template_inner_node,
which recurses into nested template_declarations and also handles
field_declaration; reuse it from unwrap_template_inner.
- tests: clsp_nocrash_template_function_multi_param_nested_call (the nested
reproducer) and typerep_substitute_unbound_param_preserved.
Distilled from #360 onto current main. The formal-parameter-count clamp
that overlapped this PR already landed via #322, so only the additive
crash fixes are taken here. Contributes to the C++ stability cluster (#390;
relates to #215/#312).
Fixes the OOB read in c_resolve_pending_template_calls when a call site has more arguments than the parsed function signature (issue #215). Verified locally: build clean, all 3,617 tests pass including the new clsp_nocrash_template_extra_call_args regression.
Route Claude Code config paths (skills, .mcp.json, .claude.json,
settings.json, hook scripts) and agent detection through
CLAUDE_CONFIG_DIR-aware helpers, falling back to ~/.claude. Hook command
strings written to settings.json keep the legacy tilde form when the env
var is unset, so existing configs stay portable across HOME values. Prints
a one-line migration nudge when CLAUDE_CONFIG_DIR is set and a legacy
~/.claude tree still exists.
Adds cli_detect_agents_finds_claude_via_env and isolates CLAUDE_CONFIG_DIR
in the existing detection tests so the runner env can't leak in.
Distilled from #321 onto current main (adapts to the v0.7.0 non-blocking
augmenter hook signature, which the original branch predated). Closes#320.
GCC's preprocessor doesn't short-circuit 'defined(__has_feature) && __has_feature(...)' — it tries to parse the call syntax of the undefined token and errors with 'missing binary operator before token "("'. Modern GCC and Clang both define __SANITIZE_ADDRESS__ under -fsanitize=address, so use that single check.
Free the result before the time-budget assert (no leak on miss) and scale the budget ×10 when ASan/UBSan is active, so the perf benches run cleanly in the release CI (skip_perf=false) instead of only being gated by CBM_SKIP_PERF in dry-run.
cbm_mkdtemp rewrites the template in place to the translated %TEMP% path on Windows, which overflowed the 23-byte 'char tmpl[]' literal (ASan stack-buffer-overflow). Use a 256-byte buffer as the cbm_mkdtemp contract requires (matching the other test callers).
Four tests hardcoded POSIX /tmp/ paths that don't exist on Windows: test_path_alias used raw mkdtemp("/tmp/...") and the store pragma/checkpoint tests built /tmp/*.db directly. Switch to cbm_mkdtemp (translates /tmp/ -> %TEMP% on Windows) and cbm_tmpdir() (matching test_store_bulk). build/smoke/soak-windows already pass; this makes the Windows unit-test job green too.
The py/cs LSP benchmark tests have time-budget asserts (<150ms / <200ms) that ASan+UBSan blow on CI (750-900ms on arm64), so they bailed before cbm_free_result and LeakSanitizer flagged the leaked arena/parser. Gate them behind CBM_SKIP_PERF (set by the dry-run) like the incremental perf tests, so they run only when perf is enabled.
Fix Windows (msys2-clang) compile errors in test code by using the existing compat shims: setenv/unsetenv -> cbm_setenv/cbm_unsetenv (test_store_pragmas, test_ts_lsp), strndup -> cbm_strndup and 2-arg mkdir -> cbm_mkdir (test_path_alias), and guard the utimensat mtime-bump in test_pipeline behind #ifndef _WIN32.
Replaces the prior Robin-Hood open-addressing implementation with
Verstable (https://github.com/JacksonAllan/Verstable), a 2024 state-
of-the-art C hash table. The public API in foundation/hash_table.h
is UNCHANGED — CBMHashTable is now an opaque type and all 14
production callers (graph_buffer, registry, pipeline passes,
service_patterns, watcher, semantic, etc.) get the new impl
transparently. str_intern.c keeps its specialised (ptr+len) hash —
different semantics, not a clean swap target.
Why: per-sub-phase profiling pinpointed cbm_pipeline_lsp_target_node
and cbm_registry_resolve as the dominant resolve_calls cost. Both
ultimately hit cbm_ht_get; both run millions of times on a kubernetes
index. Verstable's key innovations target exactly this:
* 4-bit hash-fragment metadata in a uint16_t sidecar array lets
chain walks skip most key comparisons WITHOUT touching the
bucket payload (cache wins on hot lookups).
* Quadratic probing with per-bucket displacement chains (not linear
Robin-Hood psl).
* Tombstone-free deletion that moves at most one existing key.
Vendored at internal/cbm/vendored/verstable/verstable.h (2069 lines,
84KB, MIT). Wrapped as a const char* → void* template inside
hash_table.c. The header struct is opaque (forward declaration only);
audited the entire tree — no caller accesses internals or takes
sizeof(CBMHashTable), so opacity is safe.
Test refactor: test_hash_table.c had two impl-specific tests:
* ht_robin_hood_bounded_psl — Robin-Hood specific; removed.
ht_stress_10000 covers the functional equivalent.
* ht_power_of_two_capacity — renamed ht_create_small_capacity,
replaced struct probes with a functional insert+lookup check.
ht_create_capacity_zero/one similarly de-internalised.
Benchmark on kubernetes (12 workers, 20,650 files):
- parallel_resolve wall: 36,376 -> 27,217 ms (-25.2%)
- pipeline.done total: 61,485 -> 51,745 ms (-15.8%)
- registry_build: ~1,000 -> 779 ms (faster — registry uses
cbm_ht_set heavily during build)
- All 3,615 tests pass; hash_table suite 100% green
- ASan/UBSan/SEGV: none
- Peak RSS: ~6.83 GB
- Graph: nodes 297,082 -> 297,102 (+20, 0.007%), edges 1,155,444 ->
1,154,025 (-1,419, -0.12%) — within tolerance; minor delta likely
from hash-iteration-order differences affecting tie-breaks in
resolution. Sanity (zod/serilog): nodes exact, edges within 0.1%.
Cumulative trajectory from broken-state baseline ~35 min on K8s ->
51.7s wall = ~40x total speedup.
The gopls 'package summary' pattern, applied properly: build the cross-
LSP CBMTypeRegistry ONCE per language at pipeline time (in a dedicated
cross_lsp_arena), share it READ-ONLY across all resolve workers via the
CBMCrossLspRegistries struct. Per-file work then becomes just tree-
sitter parse + AST walk + O(1) hash lookups — zero per-file registry
rebuilds for languages with a pre-built registry.
Each LSP gains two new public functions:
CBMTypeRegistry *cbm_<lang>_build_cross_registry(arena, defs, def_count)
— filters all_defs to its language, init + stdlib + registration
loop + cbm_registry_finalize, returns finalized registry.
void cbm_run_<lang>_lsp_cross_with_registry(arena, source, ...,
CBMTypeRegistry *reg, ...)
— skips the per-file build AND the per-file Phase 1b/1c AST scans
(those mutate the registry, which would race on a shared read-
only reg). Trade-off: a few file-local type aliases discovered
only by Phase 1b are missed; in practice the per-file extract
LSP already captured them via embedded_types/field_defs/etc.
Tested with parallel_python_lsp_override_cross_file_emits_lsp_
strategy_edges — fused + filtered + prebuilt path stays correct.
CBMLSPDef gains a CBMLanguage lang field set in pxc_build_lsp_def so
the per-language build functions can filter the global all_defs[].
cbm_parallel_resolve signature now takes CBMCrossLspRegistries* instead
of a single CBMTypeRegistry*. The resolve_worker dispatches:
cbm_pxc_registry_for_lang(lang) returns the right pre-built reg if any;
the worker then switch()es to the matching _with_registry variant.
Languages without a _with_registry variant fall through to the existing
filter + per-file build path.
Kubernetes (Go-heavy, 12k files, 139k defs) measured perf:
parallel_resolve: 5.51 min (strdup-removal baseline) -> 1.86 min
= 2.96x speedup
parallel.resolve.lsp_cross.slow lines: 1,258 -> 2
(-99.8%; the 2 remaining are Python files, not Go)
Peak RSS: 6.29 GB -> 6.23 GB (slight decrease — pre-built registry
shared, no per-file scratch)
All 3,616 tests pass; zod/serilog graphs exact match (per-file path
unaffected).
Cumulative trajectory from the original broken state on kubernetes:
35 min -> 19 min (finalize) -> 6.24 min (module-def filter) ->
5.51 min (strdup-removal) -> 1.86 min (Tier 2 Go) = 18.8x total.
Languages wired so far: Go, Python. C/C++, TS/JS, PHP, C# still use the
per-file filter+build path until their _with_registry variants land in
follow-up commits.
Previously cbm_pipeline_pass_lsp_cross ran as a separate sequential
pass that re-read every source file from disk and re-parsed every
tree-sitter tree on a single thread — a 50x regression on the
kubernetes Go monorepo (extract 10s, lsp_cross 9+ min when killed).
Three changes:
1. CBMFileResult gains const char *source / int source_len. The
parallel extract worker copies source bytes into result->arena
(per-file 100MB cap, project-wide 2GB atomic cap) so the fused
cross-LSP step can run without disk I/O. Source pointer is
borrowed by downstream readers; arena owns the storage.
2. Cross-LSP work moves into the parallel resolve worker. Each worker
processes its file with the existing parallel infrastructure
(worker pool + work stealing). Slab is reclaimed after each
per-file cross-LSP call to bound peak slab memory. The previously
separate lsp_cross pass entry function is kept for the sequential
pipeline path; the parallel path no longer invokes it.
3. CBMModuleDefIndex (gopls 'package summary' pattern) — an inverted
index module_qn -> list of def indices built once in O(D) before
resolve_worker fires. Per file, cbm_pxc_filter_defs_for_file slices
all_defs down to the file's own_module + imported modules only,
typically 50-100x smaller than the global ~110k defs on kubernetes.
Drops the per-file registry build from O(all_defs) to O(relevant).
Other plumbing: cbm_parallel_resolve signature gains all_defs,
def_count, def_modules, module_def_index params. pipeline.c builds
the index between registry_build and parallel_resolve and frees it
after. pipeline_incremental and tests/test_parallel updated to match.
Helpers in pass_lsp_cross.c exposed via the header: cbm_pxc_has_cross_lsp,
cbm_pxc_collect_all_defs, cbm_pxc_run_one(_ts), cbm_pxc_ts_modes,
cbm_pxc_build_module_def_index, cbm_pxc_filter_defs_for_file.
Kubernetes wall-time: 35 min -> 6.24 min (~5.6x) before the followup
perf commits in this series.
parse_ts_type_text in ts_lsp.c builds a 16-slot CBMType* members[] for
union (|) and intersection (&) types. The in-loop branch correctly
caps additions at mc<15, but the trailing post-loop append that
collects the final member after the last separator was unconditional,
and the subsequent members[mc] = NULL sentinel could then write to
members[16] — one past the array end.
Caught by UBSan on the zod codebase (a TS union with >=16 members
exists). Non-deterministic crash signature under pthread-parallel
extraction (12 workers): exit 134 / SIGABRT with no stack, because
libc's allocator detected the heap corruption and called abort().
Single-fixture-file test suite never exercised the parallel path or
a wide-enough union, so existing ASan/UBSan coverage missed it.
Fix: mirror the in-loop guard on the trailing append (cap at mc<15).
Add regression test tslsp_union_many_members_no_overflow that
constructs a 20-member string-literal union — extracting must not
crash. UBSan: clean. Full suite: 3616 passed, 0 failed.
The cli_upsert_claude_hook_fresh / _existing tests still asserted the
old matcher 'Grep|Glob|Read', missed by the previous hook-augment
commit. Update them to mirror what scripts/smoke-test.sh now enforces:
exact matcher value '"Grep|Glob"' present AND 'Glob|Read' absent.
The second assertion regress-protects against re-introducing Read
into the matcher (issue #362).
Add a C# language LSP resolver mirroring the existing Go/C/PHP/Python
modules: type evaluation, using-directive resolution, method lookup,
and a corpus-seeded .NET BCL stdlib data table.
- internal/cbm/lsp/cs_lsp.{c,h}: resolver implementation
- internal/cbm/lsp/generated/cs_stdlib_data.c: BCL type/method tables
- wire CBM_LANG_CSHARP dispatch in cbm.c and lsp_all.c
- register cs_lsp + cs_lsp_bench suites in the test harness
- tests/test_cs_lsp.c, tests/test_cs_lsp_bench.c
(cherry picked from commit 8f677c67a0c9d7829cd6ac27393de2d0aaaac534)
cbm_store_checkpoint() invoked SQLITE_CHECKPOINT_TRUNCATE, the most
aggressive mode. When two cbm-mcp processes share a cache dir, one
process's TRUNCATE can shrink files while another has them mmap'd,
raising SIGBUS on macOS. PASSIVE never blocks readers and never
ftruncate()s either file; SQLite still autocheckpoints in PASSIVE
mode at 1000-page boundaries, so reclamation is unaffected for
single-process users.
Recommended by SQLite docs for shared databases:
https://www.sqlite.org/pragma.html#pragma_wal_checkpoint
Hard-coded `PRAGMA mmap_size = 67108864` in configure_pragmas() left
no path for users running multiple cbm-mcp instances against the
same cache to opt out of memory-mapped I/O. On macOS, when one
instance's checkpoint or reindex truncates the DB file under
another instance's live mmap, accessing the now-missing pages
raises SIGBUS, taking the process down.
Setting CBM_SQLITE_MMAP_SIZE=0 reverts to read()/pread() I/O,
which returns recoverable SQLITE_IOERR instead of crashing the
process.
- Default unchanged (67108864 / 64 MB). No behavior change for
single-instance users.
- Malformed values (non-numeric, partial-numeric) fall back to
the default rather than failing the store open.
- Negative values clamp to 0.
- New tests: tests/test_store_pragmas.c covers all five resolver
paths plus an integration smoke that opens a file-backed store
with mmap disabled.
Empirical evidence: 9 SIGBUS crash reports collected on macOS
arm64 v0.6.0 in a 14-hour window, all signature 'cluster_pagein
past EOF' with stacks bottoming in SQLite btree code under the
watcher thread's incremental-index pipeline.
The Windows search path landed in 82a9052 wraps shell args in cmd.exe-
level "powershell -Command \"...'%s'...\"". PowerShell's single quotes
hold the inner '%s' interpolation, but a literal " inside the user-
supplied value can close the cmd.exe outer quote. With ' ; | & $ `
already blocked, that wouldn't reach RCE on its own — but < and > were
unblocked, so a quote-break followed by cmd.exe redirection (e.g.
*">C:\evil.txt") would expose a file-write primitive.
Block " < > unconditionally. The validator's contract is "safe inside
single quotes for shell interpolation"; on POSIX these aren't strictly
necessary (single quotes hold), but on Windows the cmd.exe→powershell
wrapping makes them load-bearing, and unconditional blocking keeps the
validator simple — no ifdef-laddered policy and no surprises if a future
caller invokes it from a different shell context.
Three tests added to tests/test_security.c covering each new char.
When pattern contains spaces and is not already a regex, convert to a
regex with '.*' between words: "foo bar baz" → "foo.*bar.*baz".
This matches all terms in order on a line instead of requiring the
exact multi-word substring.
Metacharacters in user input are escaped so the conversion is safe
for arbitrary text.
Includes test: search_code_multi_word verifies that a two-word query
matches a line containing both words non-contiguously.
Per-file LSP (run from cbm_extract_file) only sees one file's defs, so
callees whose receiver type comes from an imported module stayed
unresolved — even though Go/C/Py/TS already had cross-file entry points
(cbm_run_X_lsp_cross). Those entry points existed but no pipeline pass
ever invoked them. PHP didn't even have a cross-file entry point.
This change lands the missing pieces:
1. cbm_run_php_lsp_cross + cbm_batch_php_lsp_cross — PHP cross-file
resolver mirroring py_lsp_cross / ts_lsp_cross. Reuses PHPLSPContext,
php_lsp_init, php_lsp_add_use, php_lsp_process_file. Handles PHP-
specific labels (Class / Interface / Trait / Enum / Type) and uses
def_module_qn so cross-file return types qualify against the def's
own module rather than the importing module's.
2. pass_lsp_cross.c — new pipeline pass. For every cached file result,
builds a project-wide CBMLSPDef[] (CBMDefinition → CBMLSPDef
converter), per-file import map (from gbuf IMPORTS edges), then
dispatches to the matching cbm_run_X_lsp_cross. Output strings get
copied into the per-file arena (with O(N²) memory bug avoided by
running each file inside a fresh scratch arena that gets destroyed
after the LSP call — the LSP allocates its registry + stdlib +
project defs in scratch, not in the caller's persistent arena).
3. Wired into both pipelines:
- sequential: definitions → k8s → lsp_cross → calls → ...
- parallel: parallel_extract → registry_build → lsp_cross →
parallel_resolve → ...
4. Updated tests/test_parallel.c run_parallel helper to mirror the new
parallel ordering so existing cross-file Python/PHP tests now exercise
pass_lsp_cross.
Languages covered: Go, C/C++/CUDA, Python, TS/JS/JSX/TSX, PHP. PHP cross
file is new. The other four had functions but no pipeline integration
until this change.
Verified on a 1670-file FastAPI index: pass_lsp_cross processes 1050
Python files (525 non-LSP files skipped), collects 4070 project defs in
9.2s, and parallel_resolve subsequently reports lsp_overrides=568 — the
LSP-strategy attribution gain over per-file alone.
Per-LSP emit functions dedup by (caller_qn, callee_qn) within their own
run; pxc_append_results dedups again at append time so cross-file
entries duplicating per-file output get dropped instead of inflating
result->resolved_calls.
Per-file py_lsp emits resolved_calls.callee_qn as the raw import-module
path (e.g. `greeter.Greeter` from `from greeter import Greeter`) rather
than the project-qualified QN the gbuf stores (`<project>.greeter.Greeter`).
Before this change, the LSP match succeeded (lsp_overrides counter
incremented) but the downstream cbm_gbuf_find_by_qn lookup missed
silently and the edge was dropped — the failure mode the new
lsp_overrides telemetry was designed to surface.
The LSP layer can't tell in-project imports (qualify) from external
imports (don't qualify, e.g. `os.path`) without consulting the gbuf,
which is built downstream. So normalise at the consumer instead: try
the LSP-emitted QN as-is first; on miss, retry with
`<project>.<callee_qn>`. If that also misses, drop the edge — same as
before, target is external/unindexed.
New helper cbm_pipeline_lsp_target_node in lsp_resolve.h is shared by
both pipelines (sequential pass_calls.c and parallel pass_parallel.c)
so they continue to admit identical sets of LSP overrides. The helper
also canonicalises res.qualified_name to the gbuf node's QN, so
downstream edge property serialisation shows the project-qualified
form even when fallback resolution kicked in.
Cross-file regression test parallel_python_lsp_override_cross_file_-
emits_lsp_strategy_edges pins the two-file Greeter scenario that
originally exposed the bug.
The sequential pass_calls and the parallel cbm_parallel_extract pipelines
each carried their own copy of the LSP-resolved-call lookup with
divergent confidence floors and different match semantics. Most production
indexing runs the parallel path, so projects produced different
CALLS-edge attributions depending on which pipeline mode kicked in:
pass_calls.c find_lsp_resolution floor 0.6 (CBM_LSP_CONFIDENCE_FLOOR)
pass_parallel lsp_override_resolution_pp floor 0.5 (hardcoded)
Both pipelines correctly invoke cbm_run_*_lsp through cbm_extract_file,
so result->resolved_calls is populated identically — the bug was at the
consumer.
Changes
-------
- New src/pipeline/lsp_resolve.h: static-inline cbm_pipeline_find_lsp_resolution
and CBM_LSP_CONFIDENCE_FLOOR (0.6f). Single source of truth for the
match rule (caller_qn equality + callee_qn short-name match + floor),
with rationale and language-coverage notes inline.
- pass_calls.c: include lsp_resolve.h, drop the local copy, rename the
call site to cbm_pipeline_find_lsp_resolution.
- pass_parallel.c: include lsp_resolve.h, drop lsp_override_resolution_pp,
use cbm_pipeline_find_lsp_resolution at resolve_file_calls. Add an
int lsp_overrides counter to resolve_worker_state_t (with adjusted
cache-line padding for the new field), aggregate it per-worker,
and emit it in the parallel.resolve.done log line so divergence
between pipelines becomes observable in production telemetry.
- tests/test_parallel.c: parallel_python_lsp_override_emits_lsp_strategy_edges
builds a single-file Python scenario (Greeter class + main() in one
file), runs the parallel pipeline, walks every CALLS edge, asserts
at least one carries an "lsp_*" strategy in its properties_json.
Pins both halves of the wiring at once: the shared find helper IS
consulted in the parallel path, AND the LSP-attributed strategy
survives all the way to the gbuf edge. Single-file rather than
cross-file because per-file py_lsp's resolved_calls callee_qn uses
the import module path rather than the project-qualified QN, so
cross-file LSP edges currently get dropped by cbm_gbuf_find_by_qn —
a separate cross-file QN-mismatch bug to track elsewhere.
Full suite: 3250 passed, 0 failed (was 3249; +1 new regression test).
Resolved 5 conflicts where the php-lsp branch and main both added LSP
infrastructure but with different APIs:
- Makefile.cbm: kept both TEST_PHP_LSP_SRCS and main's py_lsp/scope/
type_rep/stack_overflow test source variables in ALL_TEST_SRCS.
- internal/cbm/cbm.c: kept both #include lsp/php_lsp.h and lsp/py_lsp.h;
the per-language dispatch already supports stacking PHP after PYTHON.
- internal/cbm/lsp_all.c: kept both lsp/php_lsp.c + php_stdlib_data.c
AND lsp/py_lsp.c + python_stdlib_data.c in the compilation unit.
- src/pipeline/pass_calls.c: took main's version entirely.
find_lsp_resolution + resolve_single_call(ctx, call, lsp_calls, ...)
is language-agnostic — it matches any CBMResolvedCall whose caller_qn
hits the textual call's enclosing function and whose callee_qn
short-name hits the textual callee. PHP LSP populates the same
result->resolved_calls array via cbm_run_php_lsp, so the unified
resolver handles PHP for free. Dropped the php-branch's
qn_short_name + lsp_override_resolution helpers as superseded.
Locked confidence floor at main's CBM_LSP_CONFIDENCE_FLOOR = 0.6f
(was 0.5f on the php branch).
- tests/test_main.c: kept all suite externs and RUN_SUITE invocations
for php_lsp + py_lsp/py_lsp_bench/py_lsp_stress/py_lsp_scale +
scope + type_rep.
Auto-merged: src/pipeline/pass_parallel.c.
Brings in 52 commits from main since the branch was forked at c768b61
(#207 watcher follow-up): #208 through #281, dependabot bumps, Pine
Script, AUR docs, INHERITS edges, get_architecture wiring, codeql
revert, the entire python-lsp subsystem.
Suite: 3249 passed, 0 failed (was 2971 on main; +278 new tests from
php-lsp matches the branch's claimed test count).
Follow-up to #281 (handle_get_architecture) plus a CodeQL workflow
upgrade developed in parallel; bundling into one commit because the
test suite had to land alongside both.
mcp.c — handle_get_architecture
- NULL-coerce every const-char* field in the architecture sections
via `x ? x : ""`, matching the rest of mcp.c (search_graph, etc.).
Without this, a NULL field becomes a missing JSON key instead of
an empty string; yyjson_mut_obj_add_str returns false on NULL and
silently no-ops, so an inconsistent omission could surprise callers.
- Serialize two more architecture aspects that #281 left on the
floor: services (cbm_service_link_t: from/to/type/count) and
clusters (cbm_cluster_info_t: id/label/members/cohesion plus the
top_nodes / packages / edge_types string arrays). The store-side
computation populates these for aspects=["all"] / explicit names,
so dropping them in the serializer was data loss.
tests/test_mcp.c
- New tool_get_architecture_emits_populated_sections regression test.
Uses a minimal inline fixture (single Function node tagged with
"is_entry_point": true) since arch_entry_points reads that flag
out of properties_json. Asserts the response contains both an
"entry_points" array and the function name — neither would appear
before #281 because handle_get_architecture never called
cbm_store_get_architecture.
- extract_text_content drilled too shallow: it pulled "content" only
from the JSON root, so it worked for cbm_mcp_handle_tool but
silently fell through to the raw response for cbm_mcp_server_handle
(where content lives under .result.content). Added a fallback that
checks .result.content; both unwrappers tested by existing fixtures.
ci(codeql)
- Run on pull_request to main, not just push; surfaces findings on
the PR instead of after merge.
- Pull custom queries from ./codeql via `queries: +./codeql`.
- Capture SARIF output and fail the job on any error-level finding,
using jq to enumerate rule id, file:line, and message text in the
GitHub Actions error annotation. Warnings are still reported as
before; only errors block.
Full suite: 2842 passed, 0 failed.
Phase 5f, 4ac, 4ad: closing the long-tail gaps toward LSP-server parity.
- @phpstan-type / @psalm-type / @phan-type aliases parsed from class
docblocks and applied during PHPDoc type resolution. References to
the alias name in @var/@param/@return are rewritten to the aliased
type. Aliases live on PHPLSPContext (per-file).
- @phpstan-import-type accepted without crash (cross-file alias
resolution deferred — same root cause as Composer PSR-4).
- @template-covariant T and @template-contravariant T now register T
as a proper type-param (previously skipped along with
@template-extends).
- Closure binding: \\Closure::bind($f, $obj) and $f->bindTo($obj),
when invoked with an inline closure literal as the first arg, walk
the closure body with $this rebound to the second arg's class.
Resolves the Laravel-macroable / VarDumper / encapsulation patterns.
- @return ($x is string ? int : bool) conditional return-type syntax
accepted without crash (parsed leftmost segment as fallback).
- Long PHPDoc blocks with mixed @param/@return/@throws/@internal
parsed without misattributing tags.
Stdlib unchanged.
Tests: 248 → 278 covering:
- phpstan-type alias basic / via @param / union / array-shape
- psalm-type alternate spelling
- phpstan-import-type (no-crash)
- Closure::bind / ::bindTo / static-bind on arrow fn
- conditional return / template-covariant / -contravariant (no-crash)
- class-string<T>, int<0,100>, literal-string (no-crash on phpstan
types we don't fully model)
- 8 realistic patterns: repository, event listener, console command,
form handler, mail sender, cached service, logger w/ context,
Carbon chains in methods, Eloquent model.
Total tests: 3061 → 3091 / 0 failed.
Audit for O(n^2) hot paths after Round 10. The dedup loop in
py_emit_resolved_call scanned ALL prior entries linearly, making
emission O(N) and total file work O(N^2). On a synthetic 2000-class
fixture with 8000 calls, that dominated time at 5083 ms.
Cap the scan window at the last 256 entries. Most duplicate
emissions come from the same expression being walked twice (resolver
+ emitter passes), so they're temporally adjacent — a 256-entry
window catches every common case while keeping the dedup O(1) per
emission.
Scale measurements (test_py_lsp_scale.c added):
| Classes | Calls | Resolved | Before | After |
|--------:|------:|---------:|-------:|------:|
| 100 | 400 | 300 | 55ms | 57ms |
| 500 | 2000 | 1500 | 333ms | 323ms |
| 2000 | 8000 | 6000 | 5083ms | 1804ms|
Ratio for 20x input: 91.8x -> 31.4x (linear would be 20x; quadratic
would be 400x). No remaining quadratic hot path in code added by
this branch.
Other hot paths surveyed:
- py_register_def -> cbm_registry_lookup_type: O(types) per def at
registration, O(D x types) for D defs. Pre-existing linear scan in
type_registry.c; hash-indexing it would be a cross-language
optimization (Go and C++ LSPs benefit too).
- py_lookup_attribute / py_lookup_field: O(types) per call site.
Same pre-existing linear scan. With ~900 stdlib types + per-file
defs, accounts for ~75µs/call at 2000-class scale.
- MODULE submodule prefix scan: O(funcs) per module-attribute access,
early-exits on first match. Bounded by registry size.
- lambda / dict-literal registries: linear, but bounded by per-file
N which is small.
Bench unchanged at 100% / 12ms (52/52 calls, 178 LOC).
When a trait method declares `: self` as its return type, flattening
that method into a using class now substitutes the trait's QN with the
using class's QN in the return type. This fixes fluent chains like:
trait T { public function tap(): self { return $this; } }
class C { use T; public function ok(): int { return 1; } }
$c->tap()->ok(); // resolves through to C.ok now
Implementation:
- flatten_trait_into_class scans the source method's signature for
NAMED(trait_qn) in the return type and rewrites it to
NAMED(using_class_qn) when registering the flattened method.
- php_lsp_collect_class_fields now does TWO passes — pass 0 for
trait/interface declarators (so their declared-return-types get
re-parsed before any consumer), pass 1 for class/enum (which may
`use Trait;` and need the trait's signatures already up-to-date).
Total tests: 3013 / 0 failed (200 php_lsp).
Round 10. Closes the last documented gaps from Round 9 plus 21 new
real-world stress tests covering SQLAlchemy / Pydantic / dataclass /
TypedDict / Protocol / generator / async patterns.
Lambda call-site inference:
- PyLSPContext gains a per-context lambda registry. `fn = lambda x:
body` records the lambda's AST node bound to fn. At call site
fn(arg), we push a sub-scope, bind lambda params from arg types,
and walk the body. Edges emitted under a synthetic <lambda> child
of the enclosing function so traces attribute correctly.
Dict-literal dispatch table:
- `funcs = {"a": foo, "b": bar}` records per-literal-key target QNs
in CBMDictLiteralEntry[]. funcs["a"]() emits an edge directly to
foo's QN with strategy=lsp_dict_dispatch.
Async with / async for:
- `async with X() as f` looks up __aenter__ before __enter__.
__aenter__'s declared return type wins; the manager type is the
fallback (async __aenter__s typically return self).
- `async for x in iter` falls back to __anext__'s return type when
the iterable's TEMPLATE form isn't recognized as AsyncIterator /
AsyncGenerator.
Callable / next / iter:
- `f()` where f resolves to CBM_TYPE_CALLABLE returns the callable's
return type. TEMPLATE("Callable", [..., R]) returns R.
- factory()() / (lambda)() etc — when the function expression is
itself a call / parenthesized / lambda, evaluate it and use its
return type.
- `next(it)` returns py_iterable_element_type(type-of-it).
- `iter(x)` is identity.
*args / **kwargs typed-parameter wrapping:
- `def f(*args: T)` -> args is TEMPLATE("tuple", [T]).
- `def f(**kwargs: T)` -> kwargs is TEMPLATE("dict", [str, T]).
- Detected by walking typed_parameter > list_splat_pattern /
dictionary_splat_pattern wrappers in py_bind_parameters.
Tuple / list / set / dict literal expressions:
- py_eval_expr_type now builds structured TUPLE / TEMPLATE types
from element types of the literal. `(Foo(), Bar())` -> TUPLE([Foo,
Bar]); `[Foo()]` -> TEMPLATE("list", [Foo]); `{"k": Foo()}` ->
TEMPLATE("dict", [str, Foo]). Without this, unpacking lost element
precision.
Match sequence pattern:
- list_pattern wraps each element in another case_pattern; capture
appears as dotted_name -> identifier; splat as splat_pattern.
Element-binder walks both wrappers and binds capture/splat to the
subject's iterable element type / list[elem].
Stress test suite expanded from 22 to 43 patterns. All 43 hard-
asserted as PASS:
- typing helpers: NamedTuple, TypedDict (incl. total=False), Protocol
(nominal + structural), ABC + abstractmethod, Mapped[T] (SQLAlchemy
2.0), Pydantic-style BaseModel
- control flow: with-as via __enter__, except-as, post-early-return
narrowing, async with via __aenter__, async for via __anext__
- containers: tuple unpack from function return + literal,
dict.items() comprehension k/v unpack, list slice, dict.get
default, recursive self-referencing types via forward refs
- decorators: factory + functools.wraps, classmethod chained to
method, property setter
- inheritance: Self in chain, diamond MRO with C3, multi-class match
- functional: lambda call-site inference, function-as-dict-value
dispatch, callable return-value, next(iter), generator delegation
(yield from), iterator next, double-star **kwargs, isinstance |
branches, module-level const, dataclass + default_factory
Bench: 178 LOC, 52 calls, **52 resolved (100%)**, 11.25 ms (ASan +
UBSan). Total 2941 / 0. Zero remaining KNOWN GAPs in the stress
suite — every probed pattern resolves.
Pins the bug-fix from #279 with a Java class declaring both extends and
implements. Asserts that base_classes contains:
- the superclass name (DefaultDiagramTool)
- every implements interface (ILinkTool, Closeable)
- bare type names only — no 'extends' / 'implements' keyword text
leaking into any entry
Without the fix, the field loop returned on the first match (only the
extends parent emitted) and cbm_node_text on the field returned the full
literal 'extends Bar' / 'implements Baz, Qux' string.
Round 9 closes 2 of the 3 remaining stress gaps from Round 8:
TypedDict literal-key subscript:
- d["foo"] where d: TD with class-body annotation `foo: Foo` returns
the field's type. Routes through the existing class-body-annotation
field registry (Round 4): TypedDict subclasses already register their
fields the same way. The subscript branch in py_eval_expr_type now
detects NAMED container + literal-string key and looks up the field
via py_lookup_field. Strips both single and double quotes.
Early-return narrowing (post-block flow):
- if not isinstance(x, T): return / raise / break / continue —
applies the *positive* narrow to the enclosing scope after the
if-block when the consequence terminates.
- if x is None: return — narrows x to non-None afterwards.
- py_block_terminates detects return/raise/break/continue as the last
statement of the body block. Subsequent code in the function sees
the narrowed type.
Match sequence pattern (tentative):
- list_pattern / tuple_pattern handler added: bind capture-pattern
elements to the subject's iterable element type; bind splat-pattern
rest to list[elem]. Also walks more wrapper levels (case_pattern ->
pattern -> _simple_pattern). Test still flags as KNOWN GAP because
tree-sitter's pattern-wrapper structure for the bracket-form `[a, b]`
in match/case differs in some grammar versions; documented as
remaining gap pending grammar inspection.
Stress test suite: 22/22 PASS. Hard-asserted patterns: namedtuple,
typeddict, protocol, abc, with-as, except-as, isinstance-early-return,
tuple-unpack, dict.items, slice, decorator-factory, property-setter,
Self chain, diamond MRO, recursive types, nested closure, long method
chains, generator delegation, async-gen for. Documented gaps:
function-as-dict-value, match sequence pattern, lambda inference.
Bench unchanged at 100% / 11.02 ms; total 2920 / 0.
Round 8. Stress test suite (test_py_lsp_stress.c) added with 22 advanced
patterns; bench fixture grew to 178 lines covering the full surface.
with-as binding via __enter__:
- Look up __enter__ on the context manager's NAMED type. Use its
return type when known; fall back to the manager type itself
(matches the practical case where __enter__ returns self). Handles
tree-sitter Python's wrapping of `X as y` in an as_pattern node
inside with_item, plus the flat field-only shape.
except-as binding:
- Bind exception alias to the caught type. Resolves through
except_clause's as_pattern wrapper and the as_pattern_target
identifier wrapper.
Tuple unpacking:
- Both `a, b = f()` and `for k, v in d.items()` destructure the RHS
element-by-element when the rhs is TUPLE or TEMPLATE("tuple", ...).
*rest binds the trailing slice as list[elem]. Shared helper
py_bind_for_target used by for-statement, comprehension for_in_clause,
and assignment LHS.
Slice subscript:
- `lst[1:3]` returns the same TEMPLATE container — list[T][1:3] is
list[T], dict[K, V][1:3] is dict[K, V] (rare but consistent).
Container TypeVar substitution:
- Without a constraint solver we can't generally substitute K, V into
registered method return types. Hand-rolled the most-impactful
cases: dict.items/keys/values/get/pop/copy, list.copy/__iter__/pop,
set.copy. These return properly parameterized ItemsView[K, V] /
KeysView[K] / ValuesView[V] / Optional[V] / etc. so subsequent
iteration and unpacking yield typed elements.
ItemsView / KeysView / ValuesView / dict_items / dict_keys / dict_values
added to py_iterable_element_type. ItemsView[K, V] iterates as
tuple[K, V], unpacked correctly by py_bind_for_target.
Bench: 178 LOC, 52 calls, **52 resolved (100%)**, 11.59 ms (ASan + UBSan).
22 new stress tests, 8 of 10 documented gaps closed in this round.
Remaining gaps documented inline (typeddict_subscript, post-early-return
narrowing, function-as-dict-value, match sequence pattern, lambda
inference). All 2898 prior tests stay green; total 2920 / 0.
Phase 4m–w in one push, plus 25 new tests (100 → 125, all green).
New capabilities:
- Generic @template substitution:
@template T on a class records T as a type-param;
@return T on a method gets parsed as NAMED("T");
when receiver is TEMPLATE("Container", [User]), method-call return
types are substituted T → User so chains like
`$container->get()->name()` resolve through to the element type.
- Closure use(...) capture: typed variables in `use ($a, $b)` clauses
of `function () use (...) {}` are inherited from the parent scope.
- Multi-conjunction narrowing: `if ($x instanceof A && $y instanceof B)`
narrows BOTH $x and $y in the if-body. Up to 8 simultaneous narrowings.
- is_a($x, Foo::class) narrowing — both the ::class and string forms.
- Subscript inference: $arr[$k] on `array<T>` returns T;
on `array<K, V>` returns V;
on a NAMED ArrayAccess class consults registered offsetGet.
- Enum case access: Suit::Hearts evaluates to NAMED(Suit) so
Suit::Hearts->label() resolves to Suit::label.
- @extends ParentClass<X>: appends ParentClass to embedded_types so
the inheritance walk picks up methods from generic ancestors.
- PHPDoc @return on methods with no declared return type updates
the registry signature.
- foreach key=>value: handles tree-sitter-php's `pair` node so
`foreach ($users as $idx => $user)` binds $user to the element type.
- Class-level @template-extends, @template-implements parsed without
crashing.
- Don't override TEMPLATE/NAMED bindings with `null` literal —
preserves @var-typed variables across `$x = null` initialization.
Stdlib expansion (Phase 5d):
- Symfony Console: SymfonyStyle, InputInterface, OutputInterface
- Doctrine ORM: EntityManagerInterface, QueryBuilder chain, Query,
EntityRepository
- Guzzle HTTP: Client and ClientInterface (returning PSR-7 responses)
- Twig: Environment, TemplateWrapper
- Eloquent Model: forwarded static methods (where, whereIn, etc.)
so `User::where()->first()` chains resolve immediately.
Test coverage: 100 -> 125 tests; total 2913 -> 2938 / 0 failed.
Major parity push. Bench 79% -> 98% (51/52 calls resolved) on a 178-line
fixture exercising the full parity surface.
Annotation wrapping:
- Type-wrapper annotations now unwrap to the inner type:
ClassVar[T], Final[T], InitVar[T], ReadOnly[T], Required[T],
NotRequired[T], Annotated[T, ...] (drop metadata),
Mapped[T] / WriteOnlyMapped[T] / DynamicMapped[T] (SQLAlchemy 2.0),
Type[T] / type[T]. Same logic in both py_resolve_annotation
(runtime, scope-aware) and py_parse_type_text_qn (registry-build,
scope-free).
Builtin / template attribute dispatch:
- obj_type=BUILTIN("str"|"int"|"list"|"dict"|...) now resolves
attribute access via "builtins.<typename>" in the typeshed-derived
registry. obj.split(...) on str, list.append(), dict.get() etc all
emit edges with strategy=lsp_builtin_method.
- obj_type=TEMPLATE("dict"|"list"|...) routes the same way; receiver
is the unparameterized container's typeshed entry.
Bare-name builtin fallback:
- Bare-name calls (range, len, list, dict, str, int, print, etc.) fall
through to a "builtins" module symbol lookup, then to a builtins
type lookup for class constructors. Strategies: lsp_builtin /
lsp_builtin_constructor.
Emission deduplication:
- py_emit_resolved_call dedupes by (caller_qn, callee_qn). When two
resolution paths converge on the same edge (expression typing then
call emission, or attribute walk through wrappers), keep one entry
with the higher confidence. Previously the same call could emit
twice — the bench showed >100% ratios as evidence.
Bench fixture expanded from 136 to 178 lines:
- Adds @property access chains, Mapped[Annotated[int, ...]], Final
fields, generator iteration with method dispatch on element type,
multi-class match/case, dict comprehension over .values(), instance
attribute typing through super().__init__, classmethod chained with
Self return.
All 2898 prior tests stay green; bench at 98% in 11.79 ms (ASan +
UBSan).
Round 6:
CBMRegisteredFunc gains a `flags` bitfield (CBMFuncFlags) and a
`decorator_qns` NULL-terminated list. py_register_def populates them
from CBMDefinition.decorators by short-name match — so `@property`,
`@functools.cache`, `@abc.abstractmethod`, `@typing.overload`, and
`@final` all light up regardless of qualified vs bare reference. The
flags aren't consumed by resolution yet (still on roadmap: emit
distinct edge for @property attribute access vs method call); they
are recorded so future overload selection / property modeling can
use them without another schema change.
The struct extension is at-tail and existing call sites that memset
to zero stay correct; non-Python LSPs see flags=0 / decorator_qns=NULL
and behave identically.
Three new verification tests prove that generators, dataclasses, and
@property already work end-to-end through the Round 4-5 infrastructure
without further code changes:
- Generator[Foo, ...] iterables in for-loop bind loop var to Foo via
py_iterable_element_type's existing TEMPLATE handling.
- @dataclass class Point: x: Foo (class-body annotated assignment) is
picked up by Round 4's third-pass field registrar; p.x.method()
resolves through the field type.
- @property def thing(self) -> Foo: c.thing returns Foo because
attribute access on NAMED routes through py_func_return_type which
yields the getter's annotated return.
All 2895 prior tests stay green; 3 new tests pass. Total 2898 / 0.
Bench unchanged at 95% (the small bench fixture doesn't exercise
the new patterns; will expand in Round 7).
Round 5 parity push:
Subscript value typing — py_eval_expr_type's subscript branch resolves
indexing against the container's TEMPLATE type:
- TEMPLATE("dict"|"Mapping"|"defaultdict"|"OrderedDict"|...)[K] -> V
- TEMPLATE("list"|"set"|"frozenset"|"Sequence"|"Iterable"|...)[i] -> elem
- TEMPLATE("tuple", [A, B, ...])[literal_int] -> args[idx]
- Non-literal index returns UNION of all element types.
This unlocks the `self.cache[k].method()` pattern that's everywhere in
real Python. Also fixes `items[0].method()` for typed lists.
super().__init__(...) constructor edge — super().<attr> already worked
for regular methods; __init__ is special-cased because parent classes
typically don't register __init__ with a useful return type. Strategy
"lsp_super_init" with confidence 0.85.
Operator dunders — a + b looks up __add__ on a's type and uses its
return type. Same for *, -, /, //, %, **, <<, >>, &, |, ^, @. Falls
back to left-operand type when registry has no info (correct for the
common int+int / str+str / list+list cases).
3 new tests; all 2892 prior tests stay green. Bench unchanged at 95%
(fixture doesn't exercise these patterns directly — Round 6 will
expand it).
Round 4 / final parity push. Tracks self.x = expr from __init__ and
class-body annotated assignments (`x: Foo`) as registered fields on
the enclosing class type.
py_register_instance_field mutates the registry's CBMRegisteredType
field_names / field_types arrays in place. Re-registering the same
name overwrites; first-seen wins for capture order. Arena-allocated,
not a hot path.
py_lookup_field walks the alias / embedded chain with the same
CBM_LSP_MAX_LOOKUP_DEPTH cap as method lookup, so inherited fields
resolve through MRO without stack-overflow risk on pathological
hierarchies.
Field-fallback wired into py_eval_expr_type's attribute branch:
when method lookup misses, try field lookup. Returns the field's
type so downstream attribute / call resolution chains naturally.
Same fallback applied to UNION (single-match heuristic).
py_process_class now does three passes:
1. Class-body annotated assignments (`x: int = ...`) → fields.
2. __init__ / __post_init__ → self.x = expr fields.
3. All other methods.
This guarantees fields are registered before methods that reference
them, regardless of source order.
Annotated assignment binding fix: py_resolve_annotation is now called
on assignment annotations too, so `self.cache: dict[str, str] = {}`
binds with the proper TEMPLATE type instead of NAMED("dict[str, str]").
Bench result on the 136-line fixture: **95%** (36/38 calls resolved,
8.06 ms). Up from 79%. The 5% gap covers cases that need a full
constraint solver — see Final Stopping Point doc.
2 new tests; all 2890 prior tests stay green. Total 2892 / 0.
Round 3 parity push:
- match/case (PEP 634): when the case pattern is class_pattern (wrapped
in a case_pattern), narrow the match subject identifier to NAMED(class)
inside the case body. Push/pop scope per case. Capture sub-patterns
bind to UNKNOWN (field-type extraction is a v1.1 task — would require
field type info in the registry).
- await expr: identity passthrough. async def f() -> T registers with
return T, so `await f()` returning T is the resolved-call shape we
want without modeling Coroutine[Any, Any, T] explicitly.
- Expanded test_py_lsp_bench.c fixture from 65 lines to 136 lines
exercising every parity feature: dotted-module walks, dataclass +
Self chaining, multi-inheritance with super(), Optional/Union
annotations + narrowing (isinstance / is not None / walrus), generic
list[T] / dict[K, V] / defaultdict comprehensions, classmethod /
staticmethod, match/case dispatch, async/await, common stdlib
(logging.getLogger, json.loads, pathlib.Path, functools.lru_cache).
Bench result on the new fixture: 38 calls, 30 resolved, 79%, 7.50 ms.
Remaining 21% includes patterns where intermediate types lose precision
(e.g. super() constructor, dict subscript), instance attribute typing
(self.x where x isn't annotated at class scope), and decorator
return-type substitution (lru_cache).
All 2888 prior tests stay green; 2 new tests + bench expansion. Total
2890 / 0.
Round 2/3 parity push. Three connected gap fixes that share an annotation-
parser and a generic-aware iterable model.
Annotation parser upgrade:
- py_resolve_annotation now turns subscripted forms into structured types
rather than stripping. `Optional[Foo]` -> UNION(NAMED(Foo), None).
`Union[A, B]` -> UNION. `list[Foo]` / `dict[K, V]` / `Iterable[T]` ->
TEMPLATE with arg list. `X | Y` (PEP 604) handled at depth zero.
- New py_parse_type_text_qn variant: same parser without scope lookup,
used by py_register_def at registry-build time. Bare class names are
qualified to <module_qn>.<name> so registry lookups match.
- py_register_def prefers d->return_type (full subscripted text) over
d->return_types[0] when brackets are present — extract_defs's array
form strips the subscript, losing element types like Optional[Foo].
Narrowing:
- if isinstance(x, T): bind x as NAMED(T) inside the consequence body
(block-scoped via push/pop).
- if x is not None: strip None from x's UNION inside the consequence;
py_match_is_none recognizes "is not" as the single tree-sitter token
it actually is, walks through parens / walrus to find the underlying
identifier.
- Else branch unchanged — narrow's negation (not-T, including-None) is
not modeled in v1.
Walrus + truthy:
- if (x := compute()) is not None: walrus binds x in the enclosing scope
(PEP 572) so the body sees x. py_bind_walrus_in walks the cond AST
looking for named_expression / assignment_expression nodes.
Comprehension + for-loop element typing:
- New py_iterable_element_type maps TEMPLATE(list/set/Iterable/...) to
arg[0]; TEMPLATE(dict) to key arg; TEMPLATE(tuple, ...) to UNION of
args.
- Comprehensions push their own scope and bind for_in_clause loop vars
to the iterable's element type before walking body / filter clauses.
for_statement does the same.
UNION attribute access fallback (used when narrowing can't run):
- obj_type=UNION & exactly one non-None NAMED member has the attr:
emit edge with strategy=lsp_method_union, confidence 0.85.
7 new test_py_lsp.c cases. All 2880 prior tests stay green; 8 new tests
land. Bench unchanged (small fixture, no narrowing exercised there).
Phase 4g+ / 4l / 5b additions toward LSP-server type-resolution parity:
- Generic template parsing in PHPDoc: parse `Collection<User>`,
`array<int, User>`, `iterable<T>`, `list<T>`, etc. Bind variables
to CBM_TYPE_TEMPLATE; method dispatch on a TEMPLATE receiver
resolves against the template's base class.
- Foreach element-type propagation: `foreach ($users as $u)` where
$users: array<User> binds $u to User. Two-arg array<K,V> takes
V. Falls back to Iterator::current() return type for NAMED types
that implement the iterator interface.
- Negative narrowing: `if (!($x instanceof Foo)) return;` (or throw)
narrows $x to Foo for the rest of the enclosing scope.
- Foreach iterable can be an arbitrary expression (variable_name,
member call, function call) — fixed earlier-too-narrow filter.
- PHPDoc @var/@param accept generic syntax: type token now spans
angle brackets so `array<User> $users` parses correctly.
- Stdlib expansion (Laravel + Symfony): full Eloquent Builder
method chain (where/orderBy/limit/get/first/...), Eloquent Model
static factories, Illuminate\Support\Collection chains
(map/filter/sort/...), Symfony HttpFoundation Request/Response,
Carbon date methods, PSR-7 with*() builders.
Test suite: 81 -> 100 tests, all passing. Total: 2913 / 0.
Adds the resolver passes that close most of the gap to a real PHP language
server's type inference, while staying in-process and PHP-runtime-free.
New capabilities:
- Type narrowing: instanceof, is_string/int/float/bool/array/object/...
is_callable, is_iterable, is_numeric. Active inside if/elseif bodies
and after assert(...) (sequential narrowing)
- Property type tracking: typed property declarations, constructor
property promotion (public X $foo), and constructor-body inference
($this->bar = $bar where $bar is typed)
- Late-static-binding chain depth: full ancestor walk with cycle
detection for $obj->method() and Class::method() resolution
- Trait flattening: methods from `use Trait;` inside a class body are
re-registered with the using class as receiver; basic `as` aliasing
is honored
- PHPDoc class-level tags: @property TYPE $name binds a virtual field
on the class; @method TYPE name() registers a virtual method
(Eloquent-style dynamic dispatch)
- Match expression result type: union over arm result expressions,
first concrete type wins
- Ternary result type: prefers the consequent, falls back to alternative
- Variadic parameter binding: $args bound to `array`
- `use Vendor\\Helper as H;` two-name form correctly maps H -> Vendor.Helper
- Static-call unindexed fallback: when class is resolved from `use`
but methods unindexed, emit php_static_unindexed for the bridge
- Stdlib expansion: full Eloquent Builder + Model + Collection method
chains, Symfony HttpFoundation Request/Response, Carbon date/time,
PSR-7 with*() builders
Test suite expanded from 16 to 81 tests (2829 -> 2893 total).
Martin Vogel