Files
codebase-memory-mcp/scripts
Andrew Hundt 1e567d4a25 fix(benchmark): reject harness-error result files
Require validate_result() in scripts/run-benchmark-campaign.py to reject top-level error reports, missing boolean derived.passed verdicts, and artifacts without non-empty cases or measurement objects before complete.json is written.

Keep accepted exit code 1 available for valid quality/performance gate failures while preventing caught RuntimeError reports from becoming resumable measurements.

Validated by 20 campaign/measurement/report unittest cases, including test_harness_error_report_is_not_marked_complete, and ruff check.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-14 12:03:56 -04:00
..