Files
codebase-memory-mcp/scripts
Andrew Hundt d08ecbe594 fix(benchmark): reject boolean numeric experiment fields
scripts/run-benchmark-experiments.py now distinguishes decoded JSON integers and finite numbers from booleans in validate_plan, validate_cell, and expand_matrix_spec. This rejects true/false schema versions, identity versions, repetitions, timeouts, frontier sizes, exact caps, and accepted exit codes while retaining positive fractional direct-cell timeouts.

tests/test_benchmark_experiments.py covers each rejected boolean field and cross-checks build_automatic_spec capability declarations against benchmark-incremental-speed.py resolve_config_overrides to prevent the two profile maps from drifting.

Verification: 185 passed, 1 skipped, 34 subtests passed in the benchmark Python set; 7,228 passed, 1 skipped in make -f Makefile.cbm test; ruff format --check and git diff --check passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-22 16:33:49 -04:00
..