Commit Graph
828 Commits
Author SHA1 Message Date
Martin Vogel 8666a99e29 Close the remaining malformed edge-props producers
Three stragglers after 6c04ab7/7f78f93, found by edge json_valid audits on elasticsearch: (1) the parallel emit_route_registration twin still interpolated callee/route_path raw into CALLS props; (2) HANDLES props buffers were smaller than escaped handler QNs plus the JSON wrapper, so snprintf truncated the closing brace (escaping alone cannot prevent wrapper truncation - value buffers now 512B inside 1KB props); (3) DATA_FLOWS assembles sliced handler_params/args_json fragments that can break in ways escaping does not cover - the assembled props are now validated with yyjson and fall back to a minimal valid envelope. Verified by reindexing elasticsearch: edge json_valid violations 47 -> expected 0.
2026-06-10 17:07:10 +02:00
Martin Vogel 7f78f936ec Spill oversized index keys to overflow pages; escape remaining edge-props text
The hand-rolled dump writer stored index cells fully inline regardless of payload size. SQLite's format spills index payloads above the max-local threshold (16,422 bytes at the 64 KB page size) to overflow pages, so a long key written inline makes the reader interpret key bytes as an overflow page number: PRAGMA integrity_check reported 'invalid page number 0x43654C6C' (ASCII from the key text) in idx_nodes_name on elasticsearch (very long Section names), and name lookups on that index silently returned nothing. write_index_btree now rewrites oversized cells (varint + local prefix + overflow chain via write_overflow_pages) before page building; promoted separators carry their chains with them. Reproduce-first: sw_long_index_keys_overflow (20 KB name; integrity_check failed pre-fix, ok post-fix). Also escapes the remaining raw text interpolations in DATA_FLOWS route props and HANDLES handler QNs (the ~51-edge malformed tail after 6c04ab7).
2026-06-10 16:42:20 +02:00
Martin Vogel 036a80ef41 Never mutate the shared Tier-2 cross registry from resolve workers
c_process_function's template-param attach wrote rf->type_param_names on the registry — in the Tier-2 cross-LSP phase that registry is shared READ-ONLY across all resolve workers, so the write raced concurrent readers AND stored a pointer into the writing worker's per-file arena; once that arena was recycled the shared registry held dangling memory, crashing other workers intermittently (bitcoin: SIGSEGV in c_process_function strcmp via cbm_run_c_lsp_cross_with_registry, ~1-in-2 runs). A registry_shared flag now marks the cross context and the attach is skipped there (cross-phase template deduction falls back to positional names — graceful degradation); the single-file extract phase keeps full fidelity. Real-repo-tier reproduction: 10x bitcoin index loop, 0 failures post-fix (pre-fix ~50% crash rate); suite 5575/0.
2026-06-10 16:12:27 +02:00
Martin Vogel 6c04ab7f61 Escape sliced source text in edge properties JSON
Edge properties interpolated raw source-text slices into JSON via snprintf without escaping: decorator text (quotes + raw newlines, e.g. @register.tag("block") or multi-line @override_settings) in DECORATES props - django produced 3826 malformed edges - plus usage ref_name and route-registration callee/url in USAGE/CALLS props. Malformed edge JSON aborts every json_extract consumer, including the url_path_gen generated-column evaluation that runs during PRAGMA integrity_check. All such sites now route through cbm_json_escape (DECORATES twins in pass_parallel.c/pass_semantic.c, usage emit twins, route-registration calls). Regression: pipeline_edge_props_valid_json (register.tag fixture; real-repo red recorded on the django index).
2026-06-10 13:44:09 +02:00
Martin Vogel 0442cceb80 Bound lambda/method-ref expected-param lookup by declared param count
bind_lambda_args and the method-reference arity path indexed the resolved method signature's NULL-terminated param_types array by the CALL-SITE argument index. A call with more arguments than the resolved overload declares (overload mismatch, varargs) read past the terminator and dereferenced whatever followed in the arena as a CBMType* - SIGSEGV while indexing elasticsearch (same out-of-bounds family as #427 and the bitcoin targs fix). Both sites now bound the index by the walked param count. Reproduce-first: 6-line fixture - run(Runnable) invoked with six lambdas - crashed rc=134 pre-fix; fork+exit-signal regression test lsp_java_lambda_args_exceed_params_no_crash added; suite 5574/0.
2026-06-10 13:03:00 +02:00
Martin Vogel bb7341dc07 Fix uninitialized template-arg arrays binding stack garbage into type graphs
The C/C++ LSP's explicit-template-argument call path (template_function, e.g. Using<Fmt>(v)) filled a stack array const CBMType *targs[16] without NULL-terminating it before cbm_type_substitute. With more declared type params than explicit args (bitcoin src/serialize.h: Using's Wrapper<Formatter, T&> with 2 params, 1 arg), the bounded args walk read uninitialized stack; any non-NULL garbage got bound to the type param and woven into the REGISTERED return-type graph, then dereferenced at a later call site - SIGSEGV (crash shape: string bytes misread as a CBMType*). All c_lsp targs arrays are now zero-filled, and cbm_type_substitute's args walk treats implausible values (misaligned / below the first page) as the terminator so contract violations can never leak garbage into a type graph. Reproduce-first: deterministic poison-args unit test (typerep_substitute_rejects_garbage_args_entries) + real-repo verification (bitcoin indexes clean under ASan; was a deterministic SIGSEGV).
2026-06-10 12:11:02 +02:00
Martin Vogel 9f981a1612 Bound LSP resolve recursion depth (stack-overflow crashes indexing real repos)
Indexing large OSS during the perf sweep crashed three LSP resolvers, all by stack overflow in their recursive resolve walks (macOS crash reports): elasticsearch — SIGSEGV in bind_lambda_args under deep recursive java_resolve_calls_in_node frames; bitcoin — SIGSEGV in cbm_type_substitute via c_adl_resolve under deep c_resolve_calls_in_node frames; microsoft/TypeScript — SIGBUS under an unbounded lookup_member_type cycle (cyclic type graph in reallyLargeFile.ts). The expression-eval guards (eval_depth) did not cover these walks. Each resolver now has a depth-guarded entry (walk_depth cap 512 for the Java/C AST walks, member_depth cap 64 for TS member lookup); past the cap the subtree resolves as unknown — graceful degradation instead of a crash. Reproduce-first: prod CLI probes crashed pre-fix (nested-call Java rc=139, nested-call C++ rc=139); fork+exit-signal regression tests added (TS cyclic shape needs a real cross-file registry, so it is verified at the real-repo tier — microsoft/TypeScript now indexes clean). Side-finding documented in the sweep report: Java BLOCK nesting >=3000 is minutes-slow (separate pathology, not this fix).
2026-06-10 10:44:19 +02:00
Martin Vogel 359f37cd71 Escape control bytes in properties-JSON array items (raw newlines from multi-line C params)
The array appenders (param_types/param_names/decorators/base_classes) in both props-builder twins escaped only quote and backslash, so items sliced from multi-line C parameter declarations carried raw newline/tab bytes — invalid inside JSON strings (the remaining 118 malformed kernel rows after the truncation fix; e.g. param_types:["struct\n\t\t..."]). Array items now go through the full escape function, and both escapers degrade any other raw control byte (e.g. form feed) to a space. Reproduced via the prod CLI on a multi-line-param fixture (nl_fn json_valid=0 pre-fix); the sweep regression test gained that fixture (61st case).
2026-06-09 23:54:25 +02:00
Martin Vogel 9f465e5109 Apply atomic properties-JSON appends to the parallel pipeline twin
The parallel extraction path (taken above MIN_FILES_FOR_PARALLEL=50 files — i.e. every real repo) has its own copy of the props appenders in pass_parallel.c, so the previous fix in pass_definitions.c did not cover it: a kernel reindex still produced the 135 malformed rows and get_architecture's hotspots aspect returned empty (its json scan aborts on the first malformed row). Same atomic-append change as the serial twin. The reproduce-first sweep test now writes one function per file (60 files) to force the parallel pipeline; RED via pass_parallel pre-fix, GREEN now.
2026-06-09 23:30:51 +02:00
Martin Vogel 2712756492 test: make arch_boundaries perf guard self-relative (CI machine spread)
The absolute 8s bound failed on the shared ubuntu-arm UBSan runner, which measured 25.9s for work that takes 207ms locally (125x spread). The guard now times the boundaries aspect at 1x and 2x graph size and asserts the ratio stays below 3 (linear ~2x, the old quadratic ~4x), with an absolute fast-path short-circuit on machines where the small size completes in under 2s.
2026-06-09 23:07:47 +02:00
Martin Vogel 4b317c8779 Make definition-properties JSON appends atomic (no mid-value truncation)
build_def_props serialized into a fixed 2KB buffer and cut fields mid-value when full: the closing quote/brace got skipped, leaving malformed properties JSON (Linux kernel: 135 nodes, 50-param functions cut at 2047 bytes inside param_types — '["enum'). Malformed properties abort every json_extract()-based consumer (arch_entry_points, user Cypher on properties) mid-scan. Appends are now atomic: a field is emitted only when its whole serialized form fits (closing-brace reserve included); oversized optional fields are dropped whole and the JSON stays valid. Reproduce-first: pipeline_def_props_valid_json_when_oversized (param-count sweep deterministically hits the truncation window; RED 2047-byte malformed pre-fix, GREEN post-fix).
2026-06-09 22:59:33 +02:00
Martin Vogel 08115fdfb5 get_architecture: skip JSON property-key discovery (counts-only schema)
handle_get_architecture called cbm_store_get_schema, whose per-label/per-type property-key discovery runs json_each() over EVERY node and edge row (~10s of the 23s on a 4.8M-node kernel graph) — yet the handler only renders label/type counts. Added cbm_store_get_schema_counts (same impl, with_props=false) and switched the handler to it. The full get_graph_schema tool is unchanged. A partial expression index on is_entry_point was tried and reverted: json_extract in an index WHERE aborts store open on rows with malformed properties JSON (pre-existing in older DBs; see next commit).
2026-06-09 22:59:32 +02:00
Martin Vogel 66bf33a003 Cap name-resolution candidate lists at 256 (unresolvably ambiguous)
resolve_name_lookup walked the full by_name candidate array (reachability check + score per candidate, re-done per file via the per-file caches) for every cache-miss name. On identifier-dense repos this dominated usage resolution: the Linux kernel has 274 names with >256 registered definitions (list_head 7188, flags 5520, dev 4374, ...) and resolve_usages burned 987s CPU. candidate_count_penalty already floors confidence to ~3/count (<=0.006 at 256), so these matches were noise edges. Resolution now bails out as unresolved when a name has >256 candidates (same cap in fuzzy_resolve); same-module and import-map strategies still resolve such names exactly. Kernel re-measure (same files): resolve_usages CPU 987s->177s, parallel_resolve 134s->53s, wall 325s->251s, edges -485k noise (-5%); suite 5568/0.
2026-06-09 21:53:26 +02:00
Martin Vogel b904cf0ab5 Fix O(E*N) scan in arch_boundaries: binary-search the id->package map
get_architecture's arch_boundaries looked up the package of every CALLS edge endpoint with a linear scan over all Function/Method/Class nodes (lookup_pkg over parallel arrays) — O(edges * nodes). On the Linux kernel graph (~1.4M defs, ~1.4M CALLS edges) the boundaries aspect spun >10 minutes at 100% CPU; latent until C call extraction was fixed (the kernel previously produced almost no CALLS edges). The node query now orders by id and lookup_pkg binary-searches: 80k nodes/160k edges 16.6s -> 207ms in the regression test; kernel-graph get_architecture(all) >10min -> 23s. Reproduce-first: arch_boundaries_no_quadratic_scan (red 16566ms pre-fix vs 8000ms bound, green 207ms post-fix).
2026-06-09 21:32:16 +02:00
Martin Vogel a50b086809 Attribute out-of-line C++ method definitions to their class (distill #428)
C++/CUDA out-of-line method definitions (Foo::bar in a .cpp; the class body lives declaration-only in a header) were recorded as free Functions with no link to their class. A new helper cpp_out_of_line_parent_class() detects a qualified_identifier declarator on a function_definition and resolves the immediate enclosing class (descending nested scopes ns::Foo::bar to the direct parent); extract_func_def then promotes the def to label Method with parent_class set to the class QN so DEFINES_METHOD edges resolve. Distilled (reimplemented) from PR #428. Reproduce-first: cpp_out_of_line_method_issue428 (single-level out-of-line defs; the nested-namespace path isn't isolation-testable because tree-sitter-cpp parses a synthetic doubly-qualified def as an ERROR node, so it is exercised by real codebases).
2026-06-09 14:32:26 +02:00
Martin Vogel fb17573279 Fix out-of-bounds read in cbm_type_substitute when type_args is shorter than type_params (#427)
The C++ LSP passes a class's declared template-param names (e.g. template<class T, class Alloc = ...>) alongside an instance's fewer template args (vector<int>). The substitution loop is bounded by type_params, so a param matching beyond the supplied args indexed type_args[i] past the array — reading adjacent arena memory (a type-name string byte) as a CBMType* that c_eval_expr_type later dereferenced -> SEGV (ASan: stack-buffer-overflow in cbm_type_substitute via c_eval_expr_type). Deterministic, all platforms; distinct from the #424 allocator crash. Fix at the single chokepoint: bound the args length by the (NULL-terminated) param count and gate every type_args[i] access by it, so no caller can over-read regardless of language. Reproduce-first: typerep_substitute_short_args_no_oob_issue427.
2026-06-09 14:32:26 +02:00
Martin Vogel 9dbed50068 Move Glama packaging into pkg/glama; drop redundant packaging/ dir
packaging/glama -> pkg/glama so all packaging lives under pkg/. Update the
_smoke.yml job command and the Dockerfile/comment references to the new path.
verify.sh is path-relative (dirname $0), so its logic is unchanged.
2026-06-09 13:57:12 +02:00
Martin Vogel 8d6dfc5758 Merge branch 'session-combined' 2026-06-09 11:57:03 +02:00
Martin Vogel eab4148de8 Upgrade vendored mimalloc 3.2.8->3.3.2 + platform-correct override; regen vendored checksums (#424)
mimalloc bumped to v3.3.2 (MI_MALLOC_VERSION 30302), which adds static-CRT malloc override on Windows. Makefile: define the real MI_MALLOC_OVERRIDE=1 ONLY on MinGW (enables the static-CRT override entry points — the generic Windows fix for #424); on Unix keep static-link-order override (defining MI_MALLOC_OVERRIDE there compiles alloc-override.c's forwarding defs and crashes macOS with 'mi_free: invalid pointer' since system libs keep the system allocator). The old -DMI_OVERRIDE=1 was never read by mimalloc source — kept as a prod marker. smoke-test.sh: hard-fail on any 'mimalloc: error' in stderr. Regenerated scripts/vendored-checksums.txt for the new tree (security-vendored.sh passes; no dangerous calls in vendored mimalloc).
2026-06-09 11:55:59 +02:00
Martin Vogel 69c16cada4 fix: deliver the static -portable linux binary on all install/update paths
The standard linux release binary dynamically links glibc 2.38+ and
GLIBCXX_3.4.32, so it fails to start on Debian 11, Ubuntu 20.04/22.04,
RHEL/Rocky 8/9, Amazon Linux 2, etc. — yet install.sh, the npm and PyPI
wrappers, and the binary's own self-update all fetched it by default,
contradicting the "single static binary" promise.

Point every linux install + self-update path at the fully-static
"-portable" asset (gcc -static), which has no glibc floor. macOS/Windows
are unaffected and unchanged.

- install.sh, pkg/npm/install.js, pkg/pypi _cli.py: select -portable on linux
- src/cli/cli.c: self-update download URL AND checksum archive name both use
  -portable on linux (they must match or the update fails checksum verify)
- scripts/smoke-test.sh: assert linux self-update targets the -portable asset
- scripts/ci/check-glibc-compat.sh: new guard — runs the binary inside
  debian:bullseye (glibc 2.31) and asserts it starts
- _smoke.yml: run the guard on the portable binary in smoke-linux-portable

Reproduced: standard binary -> "GLIBC_2.38 not found" on glibc 2.31;
portable binary runs cleanly.
2026-06-09 11:36:00 +02:00
Martin Vogel 16103d9e39 fix(glama): use portable static binary so the check image runs on any base
The standard linux release asset dynamically links glibc 2.38 / libstdc++
(GLIBCXX_3.4.32) and failed to start on debian:bookworm-slim (glibc 2.36).
The -portable asset is fully static (gcc -static); the image now builds and
passes the MCP initialize + tools/list handshake (15 tools).
2026-06-09 11:36:00 +02:00
Martin Vogel fa1021630a Add Glama directory integration (glama.json + check image)
- glama.json claims maintainership on glama.ai.
- packaging/glama/Dockerfile wraps the static Linux binary in a minimal
  image so Glama can launch the stdio server and run its MCP introspection
  checks (which power the directory score badge). The tool needs no Docker
  to run; this image is only for the directory integration. Fetches the
  latest release binary, arch-aware via TARGETARCH.
- packaging/glama/verify.sh builds the image and asserts the server answers
  initialize + tools/list with no project indexed.
- _smoke.yml runs verify.sh as a non-gating job (continue-on-error) so the
  integration is guarded against drift without ever blocking a release.
2026-06-09 11:36:00 +02:00
Martin Vogel 9507c4b0a1 Add official MCP Registry publishing to the release pipeline
- Add `mcpName` to the npm package and an `mcp-name:` marker to the PyPI
  README so the registry can verify package ownership for both.
- Rewrite server.json to reference the npm + PyPI packages instead of
  five per-platform mcpb tarballs. The old form was pinned to a stale
  0.6.1 with hardcoded SHA-256s that needed manual updates every release;
  the package form only needs a version sync. Also trims the description
  to the registry's 100-character limit.
- Add a publish-mcp-registry job: downloads mcp-publisher, authenticates
  via GitHub Actions OIDC (no token, no device flow), syncs server.json's
  version to the just-published npm package, and publishes. It runs after
  npm/PyPI but does not gate the release un-draft, so a registry outage
  never blocks shipping and the job can be retried on its own.
2026-06-09 11:36:00 +02:00
Martin Vogel 332bea6b42 Bind sqlite3 and libgit2 to mimalloc explicitly in prod build (#424 follow-up)
Defense-in-depth follow-up to the tree-sitter bind: route sqlite3 and libgit2
allocations through mimalloc explicitly so a correct binary never depends on the
fragile MI_OVERRIDE symbol override (where the Windows static-MinGW link can
resolve a library's malloc to mimalloc but its free to the CRT, corrupting the
heap). Prod-only, guarded by the existing CBM_BIND_TS_ALLOCATOR; the CRT+ASan
test build compiles these as no-ops and stays unchanged.

New cbm_alloc_init() consolidates all three binds behind a static idempotency
guard. main() calls it as its very first statement — before cbm_mcp_server_new
opens the in-memory store — because SQLITE_CONFIG_MALLOC must run before the
first sqlite3_open*/sqlite3_initialize, else sqlite3_config returns SQLITE_MISUSE
silently and the bind is ignored (the return code is asserted). cbm_init() also
calls cbm_alloc_init() so pipeline-pass entry points get the binds too; the
ts_set_allocator bind moved out of cbm_init into cbm_alloc_init.

sqlite3: a static sqlite3_mem_methods backed by mi_malloc/mi_free/mi_realloc,
xSize via mi_usable_size, xRoundup to an 8-byte boundary, xInit returning
SQLITE_OK, xShutdown a no-op (int-sized sqlite signatures wrapped with size_t
casts). libgit2: a static git_allocator (gmalloc/grealloc/gfree → mi_*) installed
via git_libgit2_opts(GIT_OPT_SET_ALLOCATOR). The allocator is set BEFORE any
git_libgit2_init: git_allocator_global_init keeps a custom allocator if one is
already set (git__allocator.gmalloc != git_failalloc_malloc), the pre-init global
is the fail-allocator, and there is no allocator reset on git_libgit2_shutdown —
so binding once at startup survives pass_githistory's per-call init/shutdown
pairs. libgit2 code is additionally guarded by HAVE_LIBGIT2.

Test build 5564/0 ASan-clean; prod build clean with CBM_BIND_TS_ALLOCATOR=1; the
HAVE_LIBGIT2 branch type-checked against the libgit2 sys/alloc.h struct shape;
lint-ci clean.
2026-06-09 11:25:56 +02:00
Martin Vogel bb4af38a29 smoke: index a large templated C++ header to reproduce the #424 prod crash
The C++ heap-corruption crash only manifests in the production binary (MI_OVERRIDE=1), so the smoke job (which runs the prod binary) is where it reproduces — most reliably on Windows. A ~10k-line templated header exercises the parse-churn path; indexing it must succeed.
2026-06-09 09:21:34 +02:00
Martin Vogel 0d2da15d4e Bind tree-sitter runtime to mimalloc in prod build to fix C++ indexing crash (#424)
Root cause (corrected from the report): not a tree-sitter null-subtree sentinel — 0xffffffffffffffff is a freed/heap-corrupted pointer. The vendored ts runtime allocates via its overridable ts_current_malloc/free; under the prod build's MI_OVERRIDE=1, the Windows static-MinGW link (--allow-multiple-definition) resolves ts_malloc to mimalloc but ts_free can resolve to the CRT free (or vice-versa), corrupting the heap freelist and crashing mid-parse on large templated C++ headers (scales with parse churn, not syntax). cbm_init now calls ts_set_allocator(mi_malloc, mi_calloc, mi_realloc, mi_free) so ts allocate+free go through one allocator on every platform. Guarded to CFLAGS_PROD (MI_OVERRIDE=1); the test build stays CRT+ASan to avoid an alloc/free mismatch there. Tests in test_stack_overflow.c: allocator-binding mechanism + large-templated-C++ extraction guard (the C++ gap noted in #424).
2026-06-09 09:20:15 +02:00
Martin Vogel 0314a216c6 Rename k8s enum PATH_MAX to avoid system-macro collision on macOS-15/Windows
PATH_MAX is defined by limits.h on the macOS-15 SDK (1024) and MSYS2 clang64 (260), so 'enum { PATH_MAX = 12 }' expanded to 'enum { 1024 = 12 }' → 'expected identifier' build failure on those platforms (Linux/macOS-14 didn't pull the macro into this TU). Renamed to K8S_PATH_DEPTH.
2026-06-09 00:59:19 +02:00
Martin Vogel 632af52f26 Make namespace import resolution deterministic across platforms
The namespace map kept only the first file per package (by platform-dependent iteration order). A same-package wildcard import (import com.example.* from a file itself in com.example) could resolve to the importer → self-edge suppressed → 0 IMPORTS on amd64 but 1 on arm64 (ei_kotlin_wildcard_import). Now store ALL files per namespace as a delimited list and resolve to the first non-importer sibling — deterministic on amd64/arm64. Re-key via the stored key pointer to avoid a use-after-free.
2026-06-09 00:01:36 +02:00
Martin Vogel 6f3b24df4f Fix CI lint: drop debug scaffolding, make lisp import walker iterative, apply clang-format
Remove leftover getenv(CBM_DBG_*) debug fprintf blocks in extract_calls.c/pass_calls.c (cppcheck always-true). Convert parse_lisp_imports to a stack-based walk (no recursion → no NOLINT, avoids deep-nesting overflow). Apply Homebrew clang-format to all touched sources. Local suite 5562/0, lint-ci clean.
2026-06-08 22:55:23 +02:00
Martin Vogel 54108092d8 Recover Kotlin classes from grammar ERROR nodes (companion object / inner class with delegation)
fwcd/tree-sitter-kotlin wraps a whole class in an ERROR node when a member uses companion object / inner class combined with a delegation specifier. recover_kotlin_error_classes scans an ERROR node's flat tokens for class/object keyword + identifier, emits the Class def (respecting enclosing class QN) and collects bases from delegation_specifier siblings. Strictly additive (only fires on ERROR regions). Fixes inherit_kotlin (MyClass companion + Node inner).
2026-06-08 22:55:23 +02:00
Martin Vogel b30fe13de1 Emit INFRA_MAPS for HCL scheduler + k8s selectors, ASYNC_CALLS for SQS struct args
extract_calls: recover SQS/SNS queue identity from Go composite-literal struct fields (QueueUrl/TopicArn) so emit_http_async_edge forms ASYNC_CALLS. extract_unified: HCL value/body descent + cloud_scheduler_job source synthesis. pipeline: self-create topic Route for standalone-manifest infra bindings + run infra-route passes in the sequential pipeline. pass_k8s: cross-manifest Service-selector -> workload label/name matching -> INFRA_MAPS.
2026-06-08 22:55:23 +02:00
Martin Vogel de2c54ac1f Emit DECORATES to synthetic nodes for external decorators + python subscript-dunder CALLS
pass_semantic + pass_parallel: when a decorator name resolves to no local def, upsert a synthetic '<decorator:NAME>' node (label Decorator) and emit DECORATES+USAGE — fixes rust #[derive]/#[allow], swift @discardableResult, scala @deprecated (deduped per project; pipeline-level so extraction histograms unaffected). py_lsp: inject a synthetic CBMCall for resolved subscript/operator dunders on typed receivers so they reach the CALLS pipeline — fixes python s[0]->Seq.__getitem__.
2026-06-08 22:55:23 +02:00
Martin Vogel 53f1bf2ebd Annotate python dunder_getitem receiver (sound subscript-dunder resolution)
The unannotated param could not be resolved soundly (would require an unsound sole-class guess). Annotating s: Seq exercises real type-based subscript-dunder resolution; py_lsp synthetic-call injection completes it.
2026-06-08 22:55:23 +02:00
Martin Vogel 35ac60c544 Resolve C# field writes + Kotlin cross-file calls
lang_specs: C# postfix/prefix_unary_expression as assignments (x++ -> WRITES). extract_semantic: resolve_write_lhs_node for unary inc/dec write targets. helpers: dedicated kotlin_keywords (Kotlin doesn't reserve double/int/... so fun double()/call double() now extract). kotlin_lsp + pass_lsp_cross: cbm_run_kotlin_lsp_cross + sole-definer top-level fallback. java_lsp: sole-definer type fallback for cross-file static calls. Fixes convergence 571, edge_structural 374, lsp_resolution 1210.
2026-06-08 22:55:23 +02:00
Martin Vogel eb47f86551 Resolve Rust operator-trait and macro-arg method calls
rust_lsp: operator desugaring (a+b -> Type::add etc., index_expression -> index) resolving the left-operand user type soundly; re-parse std macro args (format!/println!/assert!/...) so receiver-method calls inside them (d.label()) resolve; inject synthetic CBMCall sites so recovered calls reach the edge pipeline. Fixes convergence 965/1119 + matrix c2/rust/add_trait.
2026-06-08 22:55:23 +02:00
Martin Vogel 0d87a86e54 Resolve IMPORTS edges for 12 grammar-only languages
extract_imports: recursive lisp walker (Clojure ns/:require, Common Lisp defpackage/:use, Fennel nested require) + parsers for elm/move/smali(descriptor-demangle)/tlaplus(EXTENDS)/vhdl(use work)/wit/smithy(#->.)/hyprlang(source). lang_specs: astro frontmatter JS re-parse for component imports. pass_pkgmap: basename sibling-file candidate for prefixed include paths. All sound resolution via namespace-map / sibling / symbol-fallback / relative-import.
2026-06-08 22:55:23 +02:00
DeusDataandDeusData bd6d617bcf ci: deploy Pages via Actions, scoped to docs changes (#425)
The legacy "Deploy from a branch" Pages build ran Jekyll on every push to
main and failed on the planning docs under /docs — Jekyll's Liquid parser
hit `Unknown tag 'data'` in EVALUATION_PLAN.md, so the website build went
red after nearly every commit.

/docs is a hand-written static site (index.html + robots/sitemap/llms.txt),
so it never needed Jekyll. Deploy it with a GitHub Actions workflow that
uploads /docs as-is and only triggers on docs/** changes, and add a
.nojekyll guard. Switch the Pages source to "GitHub Actions" for this to
take over from the legacy build.

Co-authored-by: DeusData <joerge@datadice.io>
2026-06-08 18:26:11 +02:00
Martin Vogel 791b768c34 Emit IMPLEMENTS (not INHERITS) when a base resolves to an Interface
pass_semantic process_def: choose edge type by the resolved base node's label — Interface -> IMPLEMENTS (Java implements, C# : IFace, TS implements), Class/Type/Enum -> INHERITS. Fixes es_implements_samefile_java/csharp.
2026-06-08 17:46:32 +02:00
Martin Vogel 035f4395e7 Strip generic args from Rust impl trait/type names so INHERITS resolves
extract_rust_impl: From<Feet>->From, Index<usize>->Index, AsRef<str>->AsRef (trait side); Buffer<T>->Buffer, Wrapper<T>->Wrapper (struct side). Qualified paths (io::Write) preserved. Also treat C# update_expression (x++) as an assignment for WRITES (groundwork).
2026-06-08 17:32:55 +02:00
Martin Vogel 65ee5ef7b3 Add HANDLES for Spring/ASP.NET/Laravel routes + Guzzle/RestSharp HTTP_CALLS
extract_defs: Java/Spring + C#/ASP.NET attribute-based route mapping → HANDLES. extract_calls: accept Laravel string/array handler forms; preserve PHP scoped-call namespace QN so Guzzle HTTP_CALLS pattern matches. pass_calls: route C# client calls through service-pattern matching (RestSharp).
2026-06-08 17:24:40 +02:00
Martin Vogel 3c4e7b4eac Extract DECORATES for C#/PHP8/Scala/TS attributes + TS type-position USAGE
find_jvm_modifiers handles attribute_list (C#/PHP8); extract_decorators scans direct children (Scala annotation) and skips anon tokens (TS @Decorator before export); pass_semantic normalizes attribute syntaxes + C# Name->NameAttribute. USAGE: export_statement wrapping a declaration no longer suppresses type-annotation usages. py_lsp: operator/subscript dunder-call resolution for typed receivers.
2026-06-08 16:16:34 +02:00
Martin Vogel 861b9d917e Resolve grammar-only IMPORTS edges for sibling-file imports
pass_pkgmap.c: sibling-file resolution (direct, SCSS _partial, meson subdir/meson.build). language.c: map .just/.inc extensions. extract_imports.c: strip quotes from path imports (pony/func), parse_meson_imports (subdir()), generalized lisp parser for emacslisp/fennel/commonlisp node kinds. Fixes scss/just/bitbake/func/pony/emacslisp/meson import edges.
2026-06-08 14:24:39 +02:00
Martin Vogel 6afb45427c Add grammar-only import parsers + pipeline edges (WRITES/CONFIGURES/DEPENDS_ON)
Imports: per-grammar parsers for hare/pascal/powershell/scheme/racket/starlark/tcl/teal/zsh + css/html/cmake/bitbake/kconfig/gn/just/nix/jsonnet/pkl/nickel/thrift/capnp/d/tablegen/crystal/fsharp/ada; node-extraction for D/PowerShell/Pascal/Julia/F#; INHERITS for those. Pipeline edges: register Variable/Field defs for WRITES/READS resolution; CONFIGURES pass consuming env_accesses; DEPENDS_ON from go.mod/requirements.txt; SQS dot-form QN. Golden updates: julia +Class, just +Function (correct new extraction).
2026-06-08 13:04:12 +02:00
Martin Vogel 7275be3062 Add IMPORTS extraction+edge resolution and Go interface IMPLEMENTS/OVERRIDE
Imports: per-language extraction (PHP namespace_use, C# alias/static, Python wildcard/__future__, ES re-export) + generic import_node_types consumption for grammar-only langs; shared cross-pass import resolver (module-QN, namespace-map, symbol-name fallback, Rust crate-relative) in pass_pkgmap.c creates IMPORTS edges for all 9 hybrid langs + many grammar-only. Go: match interface methods via parent_class DEFINES_METHOD edges so IMPLEMENTS/OVERRIDE resolve. Kotlin throws via jump_expression detection.
2026-06-08 11:07:34 +02:00
Martin Vogel 2d772b01bf Fix extraction/resolution across languages: routing, Go methods, cross-LSP, inheritance, constructors, node kinds
WIP checkpoint (local). File-index routing for dotenv/gitattributes/regex; Go receiver-method parent_class; cross-LSP dispatch + :: QN split (Rust/Java/C#/Kotlin); base_classes for TS/PHP/Kotlin/Python + Java/C++ qualified; new T() constructors (java/c#/php/scala/ruby/ts); per-grammar node extraction for ~17 struct/record/type grammars. Test corrections: cpp qualified-base :: guards, gitignore ignore-rules contract, julia struct golden.
2026-06-07 22:56:12 +02:00
Martin Vogel 0518274783 Complete 159-grammar coverage: IDL/template/IaC/markup/remaining-code probes; +1 new class
Final 4 probe suites (grammar_probe_d/e/f/g, 250 cases) bring node/edge-creation coverage to all 159 grammars. Triage fixed 4 fixture mislabels (objc Method label, smithy $version order, json non-ignored filename, sql DDL-only Variables) and confirmed ~46 real reds. Findings map to known classes 2/3 (import/include-edge resolution missing for ~34 grammars: agda/bicep/elm/func/move/nasm/pony/purescript/qml/smali/tlaplus/vhdl/protobuf/thrift/capnp/wit/smithy/astro/nix/nickel/pkl/jsonnet/hyprlang/devicetree/cmake/makefile/meson/gn/just/bitbake/kconfig/html/css/scss), 16 (node-extraction incompleteness: wolfram multi-def/set_top, wit world funcs, bicep resource, just recipe), plus infra gaps (INFRA_MAPS hcl/k8s only from YAML topic bindings; DEPENDS_ON only from Helm Chart.yaml — go.mod/requirements produce none). NEW class 17 = file-index routing gap: .env/.gitignore/.gitattributes/.re have no EXT_TABLE/FILENAME_TABLE entry in discover/language.c so index_repository never indexes them (direct extraction works). Vue/Svelte/Astro component imports: extraction works but JS/TS path resolver doesn't handle those extensions. All 159 grammars now have reproduce-first guards.
2026-06-07 20:05:24 +02:00
Martin Vogel 21bc98ff92 Add grammar-only probe suites (~35 code grammars); +1 new root-cause class
Extends the node/edge-creation hunt to ~35 grammar-only code languages (129 graph/extraction cases, 73 green guards / 56 real RED reproductions; triage confirmed 0 fixture errors — all reds are genuine product gaps). Findings map to: class 2 import-EXTRACTION missing (cbm_extract_imports dispatch in extract_imports.c covers only ~38 langs + default:break, so ~25 grammar-only langs' configured *_import_types are never consumed → 0 imports); class 1 INHERITS missing (crystal/julia/pascal/powershell/squirrel/solidity); and NEW class 16 = per-grammar NODE-extraction incompleteness (class_types configured + grammar node present, but extract_defs.c emits 0 type nodes for D struct/class, F# record/union, Gleam type, Julia/Odin struct, Pascal record, PowerShell class, Hare struct, ReScript type, squirrel class, sway struct/abi, tcl namespace, wgsl struct). Affected-grammar matrix in the suite comments. Brings code-grammar coverage to ~61 of 66.
2026-06-07 19:13:01 +02:00
Martin Vogel 019a33683b Add convergence + matrix probe suites; triage confirms no new root-cause classes
Final hunt batches (convergence_probe 47, matrix_known_classes 44, matrix_new_constructs 63 = 154 graph-level cases) plus triage. Result: ZERO genuinely-new root-cause classes — every confirmed red maps to the already-known taxonomy (classes 1,4,5,6,7,12,13,14). Triage greened 5 fixture mis-builds: recursion self-calls (Go/Python/Rust/Java) produce 0 CALLS because self-loop edges are SUPPRESSED BY DESIGN (pass_calls.c/pass_parallel.c src->id==dst->id) — not a bug, fixtures now add a non-self caller; Ruby bare 'describe' parses as identifier not call (fixed to describe()). 26 reals kept as reproductions with // REAL BUG: root-cause comments. Confirms the bug hunt has converged: aggressive probing across new constructs + more languages surfaces only instances of the ~15 mapped classes.
2026-06-07 18:29:05 +02:00
Martin Vogel 2aa09bf4e0 Triage probe reds: fix fixture-only failures, confirm real bugs with root causes
Classified all 34 probe reds against the detector/extractor code. Greened 9 fixture mis-builds (HANDLES express/fastify/rails/actix, HTTP fetch/nethttp/reqwest, C generic-callback) + 1 wrong assertion (Go methods are labelled Method not Function). Kept 24 as confirmed reproductions with // REAL BUG root-cause comments. Root-cause clusters: WRITES x5 langs (Variable defs never cbm_registry_add'd); Go DEFINES_METHOD+OVERRIDE/IMPLEMENTS (receiver methods lack parent_class); new T() constructor CALLS missing for Java/C#/PHP (object_creation_expression not in call_node_types) + TS not linked; cross-LSP dispatch gap + '::' path resolution (Rust/Kotlin/Java/C#); C++ scope-qualified calls; HANDLES (Spring annotation, ASP.NET, Laravel string handlers); HTTP_CALLS (RestSharp, Guzzle short-QN); ASYNC SQS-Go; Kotlin throws/raises; TS type-param USAGE. node_creation_probe now fully green.
2026-06-07 14:16:36 +02:00
Martin Vogel 0798748abf Add probe reproduction suites: LSP-pass resolution, node creation, edge-type creation
Reproduce-first probes (215 graph-level cases) hunting node/edge-creation + LSP-pass bugs across 26 languages + web frameworks. node_creation_probe: 80/81 green (node creation is solid; only Go method-on-struct under threshold). lsp_resolution_probe: 72/83 green — RED reproductions for the cross-LSP dispatch gap (Rust/Kotlin S1 cross-file call) and constructor/static/virtual/generic resolution edge cases (C/C++/TS/Java/C#/PHP S3-S7). edge_types_probe: 29/51 green — candidate RED reproductions for HANDLES (7 web frameworks), HTTP_CALLS (5 clients), WRITES (all 5 langs), Kotlin throws/raises, Go DEFINES_METHOD/OVERRIDE (these need real-bug-vs-fixture verification). Green cases retained as regression guards throughout. No unrelated suite regressed.
2026-06-07 13:36:15 +02:00