Files
codebase-memory-mcp/scripts
Andrew Hundt b2d236490e feat(benchmarks): score Git and HTTP graph capabilities
scripts/benchmark-incremental-speed.py previously measured Git-history and HTTP-link runtime profiles without task-quality oracles, leaving enabled/disabled cost cells unable to demonstrate user-visible graph evidence.

Add deterministic, network-free fixtures that reuse FILE_CHANGES_WITH and HTTP_CALLS from the production pipeline. The Git fixture records four fixed-date co-change commits. The HTTP fixture places a Python client and source-discovered Ktor route in distinct service namespaces so only the optional HTTP linker produces the direct caller-to-handler edge.

Query existing relationship schemas with bounded results and require the declared paths, counterpart, count, caller, handler, and method/confidence evidence in the quality judgments. Register both cases in --capability-quality and cover fixture identity plus oracle queries in tests/test_benchmark_incremental_speed.py.

Optimized binary SHA-256 2f2ceeb052f78bcdfc2b64b6175cf016315f3c3b9e28af4ec1487624e2523806 passed Git and HTTP enabled smokes; the HTTP disabled control missed the direct fetch_order-to-configureRouting edge as required. Every isolated work root was removed.

Verification: 139 benchmark/campaign/report unittests passed; Ruff, scripts/check-source-safety.sh, git diff --check, and optimized MCP enabled/disabled smoke cases passed.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>
2026-07-16 21:05:27 -04:00
..