Files
codebase-memory-mcp/scripts
Andrew Hundt 73468dbabb test(benchmark): rank expected query results
Extend score_quality_oracles() in scripts/benchmark-incremental-speed.py with result rank, returned count, reciprocal rank, hit@1, and hit@5 for each applicable marker, changed-path, architecture, and route criterion.

Use mean reciprocal rank as the report quality/Pareto axis while retaining the all-applicable binary gate and passed/applicable counts. Render hit@1 and hit@5 beside exact response bytes, token estimates, query latency, incremental latency, and peak RSS.

Validated by 21 campaign/measurement/report unittest cases, including a rank-2 expected result with MRR 0.5, plus ruff check and git diff --check.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-14 12:09:22 -04:00
..