Files
codebase-memory-mcp/docs
Andrew Hundt 7df50e1eb6 test(benchmark): retain resumable campaign evidence
Add scripts/run-benchmark-campaign.py with SHA-256 cell identities over revisions, binaries, release flags, capabilities, transports, scenarios, repetitions, harnesses, commands, environments, and timeouts.

Retain timestamped command/stdout/stderr/result/attempt artifacts; validate binary and result hashes before atomically writing complete.json. Reject live locks, record stale-lock recovery, enforce disk headroom, archive exact plan bytes and environment snapshots, and classify missing, corrupt, duplicate-attempt, and unplanned run directories in immutable manifests.

Regenerate reports atomically from validated completion records only. docs/BENCHMARK_CAMPAIGN.md documents default, rank_enabled=false, and all-optional-capabilities-off cells plus release-build metadata requirements.

Validated by 19 unittest cases and ruff check across the campaign, measurement, and report scripts; git diff --check passed.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-14 11:51:35 -04:00
..
2026-06-28 12:39:13 -04:00
2026-06-28 12:39:13 -04:00