Files
codebase-memory-mcp/scripts
Andrew Hundt b3a7dc5dc1 feat(benchmark): automate versioned resumable campaigns
Previously scripts/run-benchmark-campaign.py required callers to hand-build binaries, author a plan or matrix spec, and choose a campaign root. Timestamp-only artifact names did not distinguish source time from execution time, repeated preparation rebuilt candidates, and absolute checkout paths prevented portable resume.

Add no-argument quick and --full campaign presets, peeled commit/tree identities, detached candidate worktree reuse, canonical Makefile.cbm CFLAGS_PROD capture, streamed production build logs, binary hashing, clean tracked-tree checks, and tamper-aware candidate cache records. Name new campaign evidence with v0001 definition, runset content ID, explicit commit time, and explicit generation time; identity_version=2 normalizes canonical path mappings while legacy cells keep their prior identity document.

scripts/benchmark-incremental-speed.py now uses explicit sortable UTC filenames for retained failure and scaling evidence. tests/test_benchmark_campaign.py covers tag peeling, worktree reuse, stale binary rebuild, quick/full matrix construction, path remapping, bounded names, clean-tree policy, and legacy identity compatibility.

Verification: uv run python -m unittest tests.test_benchmark_campaign tests.test_benchmark_incremental_speed tests.test_summarize_benchmark_results tests.test_autotune (159 passed); ruff check on all three changed files; git diff --check; bash scripts/check-source-safety.sh.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-19 17:49:09 -04:00
..