Commit Graph
23 Commits
Author SHA1 Message Date
Martin Vogel d1143fbdce feat(lsp): TypeScript / JavaScript / JSX / TSX hybrid LSP resolver
Adds clean-room ts_lsp resolver covering single- and cross-file type
resolution for TS, JS, JSX, TSX. Mirrors the structure and entry-point
pattern of the existing Go and C/C++ LSP hybrids (single-file
cbm_run_ts_lsp + batch cbm_run_ts_lsp_cross), with TS-specific surface:

- Hash-table indexed type registry (FNV-1a, lazy via cbm_registry_finalize)
- Partial structural relater for NAMED / BUILTIN / LITERAL / TUPLE /
  UNION / INTERSECTION / FUNC / TEMPLATE / TYPE_PARAM / ALIAS
- Conditional types T extends U ? X : Y with union distribution
- 'infer X' constraint solver, keyof / typeof / T[K] type-position parsing
- Flow-sensitive narrowing (instanceof / typeof / discriminated unions),
  optional chaining, non-null assertion
- TS utility-type passthrough (Partial, Pick, ReturnType, Awaited, etc.)
- Function overload resolution by argument types
- Async iterables + 'for await ... of'
- Curated DOM stdlib subset + comprehensive base stdlib
- Mode flags: js_mode (.js/.jsx with JSDoc inference), jsx_mode (JSX
  expressions), dts_mode (.d.ts ambient declarations, no resolved calls)

Touches existing files only with additive changes:
- type_rep.h/.c: new TS-specific CBMTypeKind values + constructors
- type_registry.h/.c: TS-specific fields, hash indexes, cbm_registry_finalize
- cbm.c: dispatch for JS/TS/TSX languages
- lsp_all.c, Makefile.cbm, tests/test_main.c: include + suite registration
- README.md: feature list mention

291 ts_lsp test cases. CBM_LSP_DISABLED=1 disables the resolver
(matches CBM_LSP_DEBUG / CBM_LSP_DISABLED knobs in go_lsp / c_lsp).
2026-05-10 14:44:50 +02:00
Martin Vogel e6f4aa74bd Merge origin/main into worktree-php-lsp-integration
Resolved 5 conflicts where the php-lsp branch and main both added LSP
infrastructure but with different APIs:

- Makefile.cbm: kept both TEST_PHP_LSP_SRCS and main's py_lsp/scope/
  type_rep/stack_overflow test source variables in ALL_TEST_SRCS.
- internal/cbm/cbm.c: kept both #include lsp/php_lsp.h and lsp/py_lsp.h;
  the per-language dispatch already supports stacking PHP after PYTHON.
- internal/cbm/lsp_all.c: kept both lsp/php_lsp.c + php_stdlib_data.c
  AND lsp/py_lsp.c + python_stdlib_data.c in the compilation unit.
- src/pipeline/pass_calls.c: took main's version entirely.
  find_lsp_resolution + resolve_single_call(ctx, call, lsp_calls, ...)
  is language-agnostic — it matches any CBMResolvedCall whose caller_qn
  hits the textual call's enclosing function and whose callee_qn
  short-name hits the textual callee. PHP LSP populates the same
  result->resolved_calls array via cbm_run_php_lsp, so the unified
  resolver handles PHP for free. Dropped the php-branch's
  qn_short_name + lsp_override_resolution helpers as superseded.
  Locked confidence floor at main's CBM_LSP_CONFIDENCE_FLOOR = 0.6f
  (was 0.5f on the php branch).
- tests/test_main.c: kept all suite externs and RUN_SUITE invocations
  for php_lsp + py_lsp/py_lsp_bench/py_lsp_stress/py_lsp_scale +
  scope + type_rep.

Auto-merged: src/pipeline/pass_parallel.c.

Brings in 52 commits from main since the branch was forked at c768b61
(#207 watcher follow-up): #208 through #281, dependabot bumps, Pine
Script, AUR docs, INHERITS edges, get_architecture wiring, codeql
revert, the entire python-lsp subsystem.

Suite: 3249 passed, 0 failed (was 2971 on main; +278 new tests from
php-lsp matches the branch's claimed test count).
2026-05-09 17:14:14 +02:00
Martin Vogel 50b218e5d5 docs(benchmark): final PHP-LSP numbers — 278 tests, ~10k LoC 2026-05-09 16:06:20 +02:00
Martin Vogel 65964b4267 docs(benchmark): refresh PHP numbers — 248 tests, 47k CALLS edges 2026-05-09 15:02:49 +02:00
Martin Vogel 7a77a178ec docs(py_lsp): final results — 43/43 stress patterns PASS, zero gaps
BENCHMARK_PYTHON.md updated through Round 10. The stress-test suite
went from 22 patterns (19 hard / 3 KNOWN GAP) to 43 patterns (all
hard-asserted PASS). Bench held at 100% (52/52) across the surface
expansion.

Patterns added in Round 10 cover the framework / real-world cases
that distinguish parity-with-LSP from passing-toy-fixtures:
SQLAlchemy 2.0 Mapped[T], Pydantic BaseModel, async with / async
for, Callable[..., R], next(iter), **kwargs annotation, dict-literal
dispatch tables, lambda call-site inference, recursive types,
classmethod-to-method chains, dataclass default_factory, nested
match patterns.

The original Phase 11 plan called this category "compiler-grade".
The actual line where compiler-rebuild starts is now narrower:
constraint solving for ParamSpec, code-flow narrowing across
basic blocks, custom metaclass __call__, and full descriptor
protocol. None of those are needed for what's resolved on the
bench fixture or the stress suite.
2026-05-09 14:16:53 +02:00
Martin Vogel 97764d48b4 docs(py_lsp): final results table through Round 9
Updates BENCHMARK_PYTHON.md to capture the full trajectory:
- Bench: 81% (65 LOC) -> 79% (136 LOC) -> 95% (136 LOC) -> 98%
  (178 LOC) -> 100% (178 LOC, 52/52)
- 22 stress-test patterns probed; 19 hard-asserted, 3 documented gaps
- Documented gaps: function-as-dict-value, match sequence pattern,
  lambda parameter inference

The hard-asserted patterns include the full LSP surface a user would
expect: NamedTuple, TypedDict, Protocol, ABC, with-as, except-as,
post-early-return narrowing, tuple unpacking, dict.items
comprehension, slice typing, decorator factories, property setters,
Self chains, diamond MRO, recursive types, closures, generator
delegation, async-gen for, long fluent chains.
2026-05-09 12:49:02 +02:00
Martin Vogel 3ec56b7bff docs(benchmark): refresh PHP-LSP numbers post-Phase-4t/v 2026-05-09 11:45:45 +02:00
Martin Vogel 0be2ac338c docs(py_lsp): final results table + stopping-point doc
BENCHMARK_PYTHON.md ends the parity push with the in-process bench
results table (81% -> 79% -> 95% across rounds), an explicit
stopping-point list of features that would require building a
constraint solver / code-flow engine / plugin system, and an
"achievable next steps" list for future iterations that stay below
the compiler-rebuild line.

The 81% -> 79% jump going from a 65-line fixture to 136 lines is
deliberate: the smaller fixture was biased toward the resolver's
strengths. The 79% -> 95% jump comes from instance-attribute typing
(Round 4) which closed the largest single gap on real-world code
shapes.

Total: 11 LSP commits on this branch (Phases 0-11 plus Rounds 1-4),
2892 / 0 passing, 95% resolution on the parity fixture.
2026-05-09 03:39:05 +02:00
Martin Vogel ee380eb551 docs(benchmark): record Phase-4 PHP-LSP outcome on laravel/framework
Edge counts (laravel/framework):
  - Total edges:  196,979 (baseline) -> 152,614 (-22%)
  - CALLS edges: ~83,000 (baseline) -> 50,150 (-40%)

The drop is from suppressing name-fallback misroutes when the receiver
is statically typed to a vendor class that isn't indexed. The Phase-4
capabilities (generic templates, narrowing, property tracking, full
parent-walk method dispatch, expanded stdlib) cover most of phpactor's
type-resolution surface for in-project receivers.

100 unit tests, all passing. Total project tests: 2913 / 0 failed.
2026-05-09 02:55:05 +02:00
Martin Vogel a8c9ace4a5 feat(py_lsp): Phase 11 benchmark methodology + integration bench
Final phase of Python LSP integration. Adds:

- docs/BENCHMARK_PYTHON.md — methodology document covering target
  metrics (per-file LSP overhead < 10 ms, resolution ratio ≥ 40% on
  application code, ≥ 70% with stdlib, < 5% spot-check false positive
  rate), the declared accepted-failure-class ceiling (metaclass
  __call__, __getattr__/__getattribute__, dynamic setattr, runtime
  importlib, monkey-patching, eval/exec, descriptors beyond the
  standard property/classmethod/staticmethod, dynamic type() classes),
  and instructions for running scripts/benchmark-index.sh against the
  largest Python repos in datadice (falkemedia: 232 files, api-hub:
  188 files).

- tests/test_py_lsp_bench.c — in-process benchmark wired into
  scripts/test.sh. Runs cbm_extract_file on a 65-line Python fixture
  covering imports, dataclasses, classmethods, single-inheritance with
  super(), typed parameters, attribute chains, and stdlib calls.
  Asserts soft floor (resolved >= calls/4) and hard time budget
  (< 50 ms under sanitizers). Initial run reports:

    bench: 65 lines, 16 calls, 13 resolved (81%), 2.45 ms

  81% beats the 40% application-code target and the 70%
  application-plus-stdlib target. 2.45 ms under ASan + UBSan is well
  inside the 10 ms (no-sanitizer) production budget.

This wraps the 11-phase plan documented in PYTHON_LSP_PLAN.md. Total
test count: 2874 / 0 (100% pass). Repository-level benchmarks via
scripts/benchmark-index.sh remain a manual operation against a fresh
production binary; methodology is now in place for when that runs.
2026-05-09 00:47:00 +02:00
Martin Vogel b8926fa52b docs(benchmark): record PHP-LSP attribution-correctness outcome
PHP score moves from 9/12 (75%) to 10/12 (83%); the two remaining
PARTIALs (Q6 search recall, Q8 trace tool disambiguation) are non-LSP
issues tracked separately per PHP_LSP_PRE_FLIGHT.md §3 and §4.1.

Headline change is a graph-correctness win on laravel/framework: the
collide-set $prompt->value() misroute is gone, and unindexed-vendor
receiver calls no longer produce wrong name-fallback edges (total
edges drop ~14% from 196,979 to 169,268, trading recall for precision).
2026-05-08 23:33:26 +02:00
Martin Vogel b24f34aaa0 docs(php-lsp): plan PHP Light Semantic Pass and pre-flight findings
Pre-flight against laravel/framework establishes that the PHP graph has
systemic call-attribution drift on short names shared between a global
function and a method (e.g. value(), make(), get()). Spot-check shows
~20-30% of typed-receiver method calls misroute to the global helper
because the existing extractor cannot resolve receiver type.

Plan adds internal/cbm/lsp/php_lsp.{c,h} as a third sibling to go_lsp.c
and c_lsp.c. In-process C type resolver, no PHP runtime, no new
dependencies. Uses already-vendored tree-sitter-php grammar and the
existing CBMArena / CBMScope / CBMType / CBMTypeRegistry primitives.

Phase 1 success metric is collide-set attribution correctness, not
benchmark tier alone. Q6 (text search) and Q8 (trace tool
disambiguation) tracked as separate tickets.
2026-05-08 22:17:25 +02:00
Martin Vogel babbebc022 Move TEST_PLAN.md to docs/ 2026-03-26 15:14:57 +01:00
Martin Vogel f52376b982 Update: 2586 tests, 66 languages everywhere, tre vendored hash 2026-03-26 11:53:35 +01:00
Martin Vogel c38853f999 Remove Go-era cruft, update THIRD_PARTY.md for pure C project
- Delete cmd/codebase-memory-mcp/assets/ — skills and codex instructions
  are embedded as C string literals in src/cli/cli.c since v0.5.0
- Move BENCHMARK.md to docs/ to reduce root clutter
- Fix setup scripts: replace stale go build with scripts/build.sh
- THIRD_PARTY.md: replace Go module dependencies with vendored C
  libraries (sqlite3, mimalloc, mongoose, yyjson, xxhash, tre),
  fix CGo reference to static compilation
2026-03-22 01:09:12 +01:00
Martin Vogel f6b9b39c60 Fix snippet tests: align with simplified get_code_snippet handler
The snippet handler now uses exact QN + suffix matching only (no fuzzy/auto-resolve).
Updated 3 tests to match: fuzzy_suggestions, fuzzy_last_segment, auto_resolve_enabled.
2026-03-20 13:45:17 +01:00
Martin Vogel 78e1df2021 Refine messaging: seconds for avg repos, clarify AST (64 langs) vs LSP hybrid (Go/C/C++) 2026-03-19 23:44:35 +01:00
Martin Vogel d1ad61fec3 Update README + website: performance-first messaging, graph UI, 8 agents
- Lead with speed (3-min Linux kernel) and plug-and-play static binaries
- Highlight tree-sitter + LSP hybrid quality (Go, C, C++)
- Add graph UI screenshot to README and website hero
- Update agent list to 8 (add Antigravity, KiloCode, Aider)
- Remove all stale Go build references (go build, CGO_ENABLED, golangci-lint)
- Update architecture section to reflect C source layout
- Update comparison table: Visual web UI now Yes
- Fix stats: 14 tools, 64 languages, 8 agents
- Condense feature list and usage examples
2026-03-19 23:35:30 +01:00
Martin Vogel 3ff579c670 Update language count to 64, add Wolfram to language lists 2026-03-07 18:27:31 +01:00
Martin Vogel ced21194ac Fix Magma import/call extraction, resolve file-path imports in linker
- Add field('path', ...) to Magma load_statement grammar rule so
  parse_generic_imports() finds the import path via field lookup
  instead of the broken text fallback (which only extracted 1 per file)
- Fix passImports() to resolve file-path imports (e.g. "utils.mag",
  "lib/helpers.h") via fqn.ModuleQN() when raw path doesn't match
  any node QN — general fix benefiting any file-path-based import
- Add TestMagmaImport_Regression and TestMagmaCall_Regression
- Update language count 59 → 63 in README, docs/index.html, marketing
2026-03-07 12:55:56 +01:00
Martin Vogel fe20c998f2 Update landing page and README with latest stats
- Languages: 35 -> 59
- Add Gemini CLI, VS Code, Zed to supported clients list
- Update benchmark description to match v7 methodology
- Update meta tags and comparison table
2026-03-06 20:03:00 +01:00
Martin Vogel 8397e4d00f Add robots.txt and sitemap.xml for search engine indexing 2026-03-02 23:15:48 +01:00
Martin Vogel 9e50803ca1 Add CONTRIBUTING.md and GitHub Pages landing page
- CONTRIBUTING.md: build from source, run tests, PR guidelines, language fix workflow
- docs/index.html: SEO-optimized landing page with benchmark data, feature grid, comparison table
2026-03-02 23:13:24 +01:00