mirror of
https://github.com/zhaoxuya520/reverse-skill.git
synced 2026-10-02 09:14:34 +08:00
test(coherence): fail on routes with no benchmark case
verify-routing-coherence.ps1 already rejects benchmark cases that point at a removed route, but nothing enforced the reverse. The benchmark meta says to add a case whenever a route is added, yet a new route could ship with zero coverage. Add the symmetric check: every routing.json route must have at least one routing-benchmark.json case. Passes today (44/44).
This commit is contained in:
@@ -10,6 +10,7 @@ Versioning follows [Semantic Versioning](https://semver.org/).
|
||||
- **CI runs remaining unwired suites** — `test-p0-friction.ps1` on the Windows leg of `routing-tests` (Windows PowerShell 5.1); `case-review/tests/test_review_case.py` in the Linux `case-contract` job. `test-workflow-title-safety.ps1` was already wired.
|
||||
- **Binary Ninja route and skill** — added `binary-ninja-reverse` for HLIL/MLIL/LLIL, Python API, and an explicitly enabled loopback community MCP bridge; Binary Ninja remains a manual commercial dependency.
|
||||
- **Optional Codex adapter plugin** — added `plugins/reverse-skill/` without changing the client-neutral core or auto-registering MCP servers.
|
||||
- **Route↔benchmark coverage gate** — `verify-routing-coherence.ps1` now fails when any `routing.json` route has no `routing-benchmark.json` case, enforcing the benchmark meta's "add a case when you add a route" rule (the reverse of the existing ghost-expect check).
|
||||
|
||||
### Fixed
|
||||
- **IDA MCP HTTP stall** — `run-supervisor.py` patches stock `idalib_supervisor` onto `ThreadingHTTPServer` and accepts Streamable HTTP GET `/mcp` (patch failure is skipped, supervisor still starts). Keep-alive deadlock is **time since last healthy `tools/list`**, not process `CreationDate`; `open.ps1` holds `opening.lock` so in-flight opens are never `-Force`d. New `recover.ps1` is an immediate `-Force` path. Never `taskkill`s `ida.exe`.
|
||||
|
||||
@@ -76,6 +76,10 @@ if (Test-Path -LiteralPath $benchJson) {
|
||||
$rjIds = @($rjRoutes | ForEach-Object { $_.Name })
|
||||
$ghostExpect = @($bjCases | Where-Object { $_.expect -notin $rjIds })
|
||||
if ($ghostExpect.Count -eq 0) { Ok 'benchmark expects all exist in routing.json' } else { Bad "benchmark ghost expects: $(($ghostExpect | Select-Object -First 5).expect -join ',')" }
|
||||
# 反向:每个 routing.json 路由至少要有一条 benchmark 用例(meta 要求新增路由同步加用例)
|
||||
$expectIds = @($bjCases | ForEach-Object { $_.expect } | Select-Object -Unique)
|
||||
$uncoveredRoutes = @($rjIds | Where-Object { $_ -notin $expectIds })
|
||||
if ($uncoveredRoutes.Count -eq 0) { Ok 'benchmark covers every routing.json route' } else { Bad "routes with no benchmark case: $($uncoveredRoutes -join ',')" }
|
||||
}
|
||||
} else {
|
||||
Bad 'skills/tests/routing-benchmark.json missing'
|
||||
|
||||
Reference in New Issue
Block a user