mirror of https://github.com/DeusData/codebase-memory-mcp.git synced 2026-11-08 08:53:43 +08:00

Files

T

Andrew Hundt 603e516faf refactor(benchmarks): consolidate canonical assets under benchmarks

Move incremental_speed.py, run_experiments.py, summarize_results.py, fact_comparisons.py, autotune.py, active v2 schemas, terminology.json, configuration spelling data, and shell harnesses into benchmarks/.

Keep scripts/_benchmark_compat.py plus historical Python and shell filenames as thin compatibility frontends. Preserve docs/schema/benchmark-facts-v1.schema.json because retained v1 bundles embed that exact URI; accept the former v2 URI only while loading retained results.

Regenerate docs/BENCHMARK_TERMINOLOGY.md and src/foundation/profile_terms_generated.h from benchmarks/terminology.json. Update CONTRIBUTING.md, MAINTAINERS.md, docs/BENCHMARK_EXPERIMENTS.md, docs/EVALUATION_PLAN.md, and tests to use canonical paths.

Verification: 205 passed, 1 skipped, 34 subtests passed; Ruff E4/E7/E9/F passed; comparisons-v1 schema validation passed for 42 retained bundles and 91 comparisons; generator --check, bash -n, git diff --check, and scripts/check-source-safety.sh passed.
Signed-off-by: Andrew Hundt <ATHundt@gmail.com>

2026-07-23 20:29:05 -04:00

schema

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

semantic-pairs-v1

feat(benchmarks): verify semantic freshness after source edits

2026-07-16 15:16:33 -04:00

autotune.py

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

clone_repositories.sh

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

config-spellings-v1.json

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

fact_comparisons.py

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

generate_terminology.py

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

incremental_speed.py

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

index.sh

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

README.md

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

run_experiments.py

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

search_graph.sh

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

summarize_results.py

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

terminology.json

refactor(benchmarks): consolidate canonical assets under benchmarks

2026-07-23 20:29:05 -04:00

README.md

Benchmark tooling

This directory owns the benchmark implementations, active schemas, machine-readable terminology, configuration-spelling compatibility data, and source fixtures.

Primary entry points:

incremental_speed.py: measure indexing lifecycles and emit canonical fact tables.
run_experiments.py: build and execute immutable, resumable experiment matrices.
summarize_results.py: render quality-gated Markdown from retained result JSON.
fact_comparisons.py: derive parity, capability-delta, and lifecycle tables from canonical facts.
autotune.py: run the isolated PageRank tuning experiment.

schema/ contains schemas for records emitted by current tooling. terminology.json defines every normative fact, step, join, and formula identifier. The generated human view remains in docs/BENCHMARK_TERMINOLOGY.md, and the full workflow is documented in docs/BENCHMARK_EXPERIMENTS.md.

Files under scripts/ with historical benchmark names are compatibility frontends, not independent implementations. docs/schema/benchmark-facts-v1.schema.json is the only schema intentionally left outside this directory because retained v1 bundles embed that frozen URI.