Files
codebase-memory-mcp/scripts
Andrew Hundt e6126468fa scripts: rename benchmark entry point to run-benchmark-experiments.py
Move the full implementation from run-benchmark-campaign.py to
run-benchmark-experiments.py and reduce the old filename to a
backwards-compatible shim that loads the new module by path and
re-exports every public name, so direct imports
(tests/test_benchmark_campaign.py, scripts/autotune.py,
scripts/summarize-benchmark-results.py) and existing invocations keep
working with byte-identical behavior.

- Add --experiment-root and --allow-temporary-experiment-root, keeping
  --campaign-root and --allow-temporary-campaign-root as accepted
  aliases on both entry points.
- Add --candidate-ref LABEL=REF override and an upstream/main ->
  origin/main -> main fallback chain so --quick/--full keep working
  after the api-consolidation branch is merged and its pinned refs no
  longer resolve; explicitly requested refs remain fail-closed.
- Say experiment instead of campaign in --help text, runtime error
  text, and generated-report boilerplate (summarize-benchmark-results.py);
  keep the campaign_version manifest key, the campaigns composition key,
  and the .worktrees/benchmark-campaign/ directory so every retained
  runset stays valid.
- Move docs/BENCHMARK_CAMPAIGN.md content to docs/BENCHMARK_EXPERIMENTS.md
  and leave a pointer stub so existing links keep resolving.
- Add tests/test_benchmark_experiments_shim.py (25 subtests) proving the
  shim re-exports the implementation and both CLIs accept old and new
  flags; update the temporary-root error-text assertion.

Verified: ruff clean on all touched files; 171 passed, 1 skipped across
the four benchmark test files; --help output identical between the two
entry points except the program name.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-21 21:54:29 -04:00
..