Add score_ranked_relevance() with a bounded cutoff and explicit positive relevance judgments. Compute first-relevant reciprocal rank, Hit@1, Hit@5, DCG, ideal DCG, and nDCG@k in O(k*j) time for cutoff k and judgment count j.
Extend score_quality_oracles() to accept graded judgment objects while preserving existing tuple-based substring expectations. Missing judged evidence contributes zero; probes without judgments remain N/A. Keep strict task and canonical-graph gates independent from descriptive retrieval scores.
Aggregate nDCG@5 and judgment counts in summarize-benchmark-results.py without hiding MRR or Hit@k. Generated methodology links NIST's TREC QA reciprocal-rank definition and the TREC Complex Answer Retrieval graded-ranking evaluation.
Tests: uv run python -m unittest tests.test_benchmark_campaign tests.test_benchmark_incremental_speed tests.test_summarize_benchmark_results (62 passed)
Static: uv run ruff check scripts/benchmark-incremental-speed.py scripts/summarize-benchmark-results.py tests/test_benchmark_incremental_speed.py tests/test_summarize_benchmark_results.py
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
Andrew Hundt
c988570d34