Andrew Hundt 33ef7249fe fix(benchmark): separate default response cost from quality JSON ...

Route run_cli_tool_call() and run_mcp_tool_call() through a default-format request for elapsed_ms, response_bytes, and response_token_estimate, then issue format=json only for quality scoring. Record quality_probe_elapsed_ms and quality_response_bytes so the second request cannot inflate the user-facing latency or replace TOON payload sizing.

This fixes the pilot JSONDecodeError: Expecting value at line 1 column 1 when search_graph returned its default TOON content. Document the two-request instrumentation contract in docs/BENCHMARK_CAMPAIGN.md.

Validated with 23 focused pytest cases across tests/test_benchmark_incremental_speed.py, tests/test_summarize_benchmark_results.py, and tests/test_benchmark_campaign.py; Ruff and git diff --check pass.

Signed-off-by: Andrew Hundt <ATHundt@gmail.com>

2026-07-14 13:42:37 -04:00

..

ci

fix: deliver the static -portable linux binary on all install/update paths

2026-06-09 11:36:00 +02:00

git-hooks

Adopt the Developer Certificate of Origin for all commits

2026-06-12 14:31:55 +02:00

hooks

Add time-bomb + MCP file read audit, tracked pre-commit hooks

2026-03-21 16:43:48 +01:00

audit-grammar-security.sh

Add 89 new tree-sitter grammars (66 to 155 languages)

2026-04-15 21:28:51 +02:00

audit-license-provenance.py

Run the gate self-test and provenance audit in the security workflow

2026-06-12 13:24:07 +02:00

autotune.py

fix(autotune): fix CWD bug, race condition, and update CBM ground truth

2026-03-26 05:37:40 -04:00

benchmark-incremental-speed.py

fix(benchmark): separate default response cost from quality JSON

2026-07-14 13:42:37 -04:00

benchmark-index.sh

Fix C++ SEGV: NULL deref in LSP type resolver on large header files

2026-03-26 16:19:10 +01:00

benchmark-search-graph.sh

fix(scripts): use /usr/bin/env bash shebangs (NixOS has no /bin/bash)

2026-07-03 20:11:49 +02:00

build.sh

fix(scripts): use /usr/bin/env bash shebangs (NixOS has no /bin/bash)

2026-07-03 20:11:49 +02:00

check-dco.sh

Adopt the Developer Certificate of Origin for all commits

2026-06-12 14:31:55 +02:00

check-lsp-originality.sh

chore(lsp): add Perl reference to originality-check manifest

2026-07-09 23:43:41 +02:00

check-no-test-skips.sh

Forbid test skips and convert existing skips to hard failures

2026-06-05 21:53:25 +02:00

check-nolint-whitelist.sh

fix(scripts): use /usr/bin/env bash shebangs (NixOS has no /bin/bash)

2026-07-03 20:11:49 +02:00

check-source-safety.sh

fix(mcp): reuse portable path checks

2026-07-01 21:38:26 -04:00

clean.sh

fix(scripts): use /usr/bin/env bash shebangs (NixOS has no /bin/bash)

2026-07-03 20:11:49 +02:00

clone-bench-repos.sh

Cross-service communication discovery + RAM-first incremental indexing

2026-03-28 13:59:42 +01:00

embed-frontend.sh

fix(scripts): use /usr/bin/env bash shebangs (NixOS has no /bin/bash)

2026-07-03 20:11:49 +02:00

env.sh

fix(scripts): use /usr/bin/env bash shebangs (NixOS has no /bin/bash)

2026-07-03 20:11:49 +02:00

extract_nomic_vectors.py

Parallelize post-passes, fix mode filtering + semantic edge quality

2026-04-06 12:41:30 +02:00

gen-py-stdlib.py

feat(py_lsp): Round 1 parity push (parent-modules, cast, Self, fwd-refs)

2026-05-09 01:39:28 +02:00

gen-third-party-notices.sh

Extend the notices bundle to the UI dependencies and the Glama image

2026-06-12 10:51:11 +02:00

gen-ui-licenses.py

Resolve npm through the shell for Windows compatibility

2026-06-12 18:16:43 +02:00

generate-lang-code.py

Add 89 new tree-sitter grammars (66 to 155 languages)

2026-04-15 21:28:51 +02:00

install-git-hooks.sh

Adopt the Developer Certificate of Origin for all commits

2026-06-12 14:31:55 +02:00

license-gate-check-npm.py

Resolve npm through the shell for Windows compatibility

2026-06-12 18:16:43 +02:00

license-gate-check.py

Bundle third-party notices into release archives; extend release tooling

2026-06-12 02:17:39 +02:00

license-gate.sh

Run the gate self-test and provenance audit in the security workflow

2026-06-12 13:24:07 +02:00

license-policy.json

Exempt the provenance-audit tool from the ScanCode pass

2026-06-12 15:57:08 +02:00

lint.sh

fix(scripts): use /usr/bin/env bash shebangs (NixOS has no /bin/bash)

2026-07-03 20:11:49 +02:00

new-languages.json

Drop the nim grammar and refresh language counts

2026-06-12 02:17:39 +02:00

repro.sh

fix(scripts): use /usr/bin/env bash shebangs (NixOS has no /bin/bash)

2026-07-03 20:11:49 +02:00

run-benchmark-campaign.py

fix(benchmark): reject harness-error result files

2026-07-14 12:03:56 -04:00

security-allowlist.txt

Merge pull request #865 from DeusData/chore/remove-libgit2

2026-07-05 01:55:17 +02:00

security-audit.sh

Exempt the graph-UI server from the raw-network-call audit

2026-06-12 02:28:05 +02:00

security-fuzz-random.sh

Fix 29 CodeQL alerts: command injection, snprintf overflow, TOCTOU races

2026-03-21 22:18:31 +01:00

security-fuzz.sh

Separate perf tests from CI, fix cross-platform build issues

2026-04-02 14:52:14 +02:00

security-install.sh

fix(install): respect $CLAUDE_CONFIG_DIR in install/uninstall/update

2026-05-30 15:22:37 +02:00

security-network.sh

Add 8-layer security test suite + hardening

2026-03-20 18:19:15 +01:00

security-strings.sh

fix(ci): allow-list MSYS2 toolchain URL in binary string audit

2026-06-24 03:15:36 +02:00

security-ui.sh

Replace the graph-UI HTTP server with a first-party implementation

2026-06-12 02:17:25 +02:00

security-vendored.sh

Replace the graph-UI HTTP server with a first-party implementation

2026-06-12 02:17:25 +02:00

setup-windows.ps1

Remove Go-era cruft, update THIRD_PARTY.md for pure C project

2026-03-22 01:09:12 +01:00

setup.sh

fix(install): respect $CLAUDE_CONFIG_DIR in install/uninstall/update

2026-05-30 15:22:37 +02:00

smoke-invariants.sh

merge(upstream): integrate main through Laravel route fixes

2026-07-14 01:47:43 -04:00

smoke-test.sh

fix(lifecycle): cancel and reap auto-index process trees

2026-07-14 13:14:43 -04:00

soak-test.sh

merge(upstream): integrate main through Laravel route fixes

2026-07-14 01:47:43 -04:00

summarize-benchmark-results.py

test(benchmark): rank expected query results

2026-07-14 12:09:22 -04:00

test_mcp_rapid_init.py

fix(mcp): address QA round 1 findings

2026-03-21 12:37:28 -05:00

test-source-safety.sh

fix(mcp): reuse portable path checks

2026-07-01 21:38:26 -04:00

test-windows.ps1

fix(writer): open the graph DB wide-safe for non-ASCII cache paths

2026-07-12 19:57:47 +02:00

test.sh

fix(supervisor): wrap only marked host binaries; watchdog in worker mode

2026-07-04 14:38:25 +02:00

vendor-grammar.sh

fix(scripts): use /usr/bin/env bash shebangs (NixOS has no /bin/bash)

2026-07-03 20:11:49 +02:00

vendored-checksums.txt

Replace the graph-UI HTTP server with a first-party implementation

2026-06-12 02:17:25 +02:00