Files
codebase-memory-mcp/scripts
Andrew Hundt fc56bb2398 feat(benchmarks): measure list-project scaling
Add --list-projects-scaling to scripts/benchmark-incremental-speed.py. The mode indexes one minimal fast-mode seed, snapshots and rekeys valid SQLite project databases, and starts a fresh MCP server for every strictly increasing project count.

Record canonical response bytes, deterministic byte/4 token estimates, MCP envelope bytes, call latency, post-call resident memory, fixture disk bytes, transport survival, and server/thread reap status. Label post-call RSS as non-peak and state that the experiment cannot attribute combined multi-tool response size.

Gate fixture creation at scripts/benchmark-incremental-speed.py:540-573 against --list-project-fixture-max-mb plus a max(2 GiB, 5% free-space) reserve. Write completed or failed JSON atomically and remove the temporary output path in a finally block. Auto-created work roots are removed while --keep-work-root preserves them.

Add count validation, snapshot rekeying, and disk-budget tests in tests/test_benchmark_incremental_speed.py:16-76. The 1→2 integration smoke used optimized binary SHA-256 379585ab9be20eade9edf6aff2e1cea318aceb1659aa3f6cbfa3f51bc8dc5fb4, returned 1/1 and 2/2 projects, kept transport alive, reaped both servers, and removed its work root.

Verification:
- uv run python -m unittest tests.test_benchmark_incremental_speed tests.test_benchmark_campaign tests.test_summarize_benchmark_results: 73/73 passed.
- uv run ruff check scripts/benchmark-incremental-speed.py tests/test_benchmark_incremental_speed.py: passed.
- make -f Makefile.cbm cbm: built the -O2 production binary.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-16 03:44:53 -04:00
..