Files
codebase-memory-mcp/scripts
Andrew Hundt 66c2185a4b fix(benchmarks): reject semantic freshness policy mismatches
The capability-quality report could label a semantic lifecycle PASS even when pair_lifecycle.incremental_policy.policy_conformance_met was false. This occurred for upstream-main in both SIMILAR_TO and SEMANTICALLY_RELATED smoke cells: the post-edit judged pair was present, but the whole incremental graph was non-canonical and the recorded stale_on_incremental policy did not conform.

Make summarize_group reject non-disabled semantic cells with REJECT: freshness policy and emit a concrete finding that binds the post-edit pair result to the canonical-graph comparison. Preserve expected deferred-freshness and intentional capability-off decisions.

Also copy execution_order, execution_block, and execution_position from the immutable campaign cell into derived report inputs. This prevents paired_interleaved campaigns from being mislabeled grouped after report materialization while leaving raw result JSON immutable.

Tests: uv run python -m unittest tests.test_summarize_benchmark_results tests.test_benchmark_campaign tests.test_benchmark_incremental_speed (111 passed); uv run ruff check scripts/summarize-benchmark-results.py scripts/run-benchmark-campaign.py tests/test_summarize_benchmark_results.py tests/test_benchmark_campaign.py; git diff --cached --check.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-16 15:51:31 -04:00
..