Files
codebase-memory-mcp/docs
Martin Vogel a8c9ace4a5 feat(py_lsp): Phase 11 benchmark methodology + integration bench
Final phase of Python LSP integration. Adds:

- docs/BENCHMARK_PYTHON.md — methodology document covering target
  metrics (per-file LSP overhead < 10 ms, resolution ratio ≥ 40% on
  application code, ≥ 70% with stdlib, < 5% spot-check false positive
  rate), the declared accepted-failure-class ceiling (metaclass
  __call__, __getattr__/__getattribute__, dynamic setattr, runtime
  importlib, monkey-patching, eval/exec, descriptors beyond the
  standard property/classmethod/staticmethod, dynamic type() classes),
  and instructions for running scripts/benchmark-index.sh against the
  largest Python repos in datadice (falkemedia: 232 files, api-hub:
  188 files).

- tests/test_py_lsp_bench.c — in-process benchmark wired into
  scripts/test.sh. Runs cbm_extract_file on a 65-line Python fixture
  covering imports, dataclasses, classmethods, single-inheritance with
  super(), typed parameters, attribute chains, and stdlib calls.
  Asserts soft floor (resolved >= calls/4) and hard time budget
  (< 50 ms under sanitizers). Initial run reports:

    bench: 65 lines, 16 calls, 13 resolved (81%), 2.45 ms

  81% beats the 40% application-code target and the 70%
  application-plus-stdlib target. 2.45 ms under ASan + UBSan is well
  inside the 10 ms (no-sanitizer) production budget.

This wraps the 11-phase plan documented in PYTHON_LSP_PLAN.md. Total
test count: 2874 / 0 (100% pass). Repository-level benchmarks via
scripts/benchmark-index.sh remain a manual operation against a fresh
production binary; methodology is now in place for when that runs.
2026-05-09 00:47:00 +02:00
..
2026-03-26 15:14:57 +01:00