Files
codebase-memory-mcp/scripts
Andrew Hundt 8bd307e8df test(benchmark): score query quality against response cost
Record canonical JSON payload bytes separately from CLI/MCP framing in scripts/benchmark-incremental-speed.py and attach the deterministic utf8_bytes_div_4_ceil token estimate to every query oracle.

Require every applicable marker, changed-path, architecture, and route oracle to pass. Render quality checks, query latency, response size, indexing latency, and peak RSS in scripts/summarize-benchmark-results.py; mark Pareto candidates only after correctness passes and all comparison axes are present.

Validated by 10 unittest cases across tests/test_benchmark_incremental_speed.py and tests/test_summarize_benchmark_results.py plus py_compile and git diff --check.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-14 11:41:49 -04:00
..