Files
headroom/wiki/learn.md
71663b44e2 fix(learn): pin the learned verbosity level on a cache-mode proxy (#4033)
## Description

`headroom learn --verbosity --apply` hot-enables the output shaper on a
running proxy and prints `level N is live now`. On a proxy in **cache
mode**, which is the default, that claim is false. The proxy never reads
`verbosity.json` there: `resolve_verbosity_level` returns the startup
level (`cache_mode_default`, L2) unless `HEADROOM_VERBOSITY_LEVEL` pins
one. A level that can change mid-conversation would bust the prefix
cache (#3773). So after `--apply` the shaper is on, but at L2 instead of
the learned level, while the CLI reports the learned level as live.

This PR makes `--apply` either do what it says or say what it did:

- **Cache mode, no pin:** the learned level is pinned through
`HEADROOM_VERBOSITY_LEVEL` in the same `/admin/runtime-env` update. That
is exactly the stable-pin path #3773 allows in cache mode.
- **An existing `HEADROOM_VERBOSITY_LEVEL`:** left alone, and the CLI
says which level stays in effect. The one exception is a pin this CLI
provably set. When the proxy confirms a pin (its `applied` map),
`--apply` records `{port, pid, started_at, level}` in
`verbosity_pin.json`. `started_at` is `timestamp - uptime_seconds` from
`/health`, because pids are reused. A later `--apply` replaces the pin
only while that same proxy process still holds that exact value. Hot
overrides die with the process, so a restarted proxy's pin, a reused
pid, or a value changed since stays the operator's.
- **Mode unreadable:** the shaper is still enabled, but the CLI no
longer claims the level is live.
- **Token mode:** unchanged. The proxy reads `verbosity.json` per
request, so no pin is added (a pin would also shadow the verbosity
autotune controller).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/cli/learn.py`:
- New `_query_proxy_verbosity()` reads `config.mode` and
`config.runtime_env.HEADROOM_VERBOSITY_LEVEL` from the local proxy's
`/health`. The proxy includes that config block for loopback callers.
The level is read the way `OutputShaperSettings.from_env` reads it
(parse, default on garbage, clamp to 0–4), so `9` or `04` compare and
print as the level the proxy steers at.
- `_proxy_process()`, `_record_pin()` and `_pin_is_ours()` keep the
ownership record described above. Recording is best-effort: an `OSError`
no longer aborts `--apply`; the CLI reports the activation and notes the
pin wasn't recorded. The record is written with `newline="\n"`.
- `_activate_output_shaper()` returns `(status, port, applied)`, so a
pin the proxy applied while reporting the shaper "blocked" is still
recorded.
- `_activate_output_shaper()` takes an optional `verbosity_level` and
sends it in the same runtime-env request.
- The `--apply` messages distinguish the four cases above. The restart
hints now include `HEADROOM_VERBOSITY_LEVEL=<level>` for cache mode.
- The "learned level is used automatically" wording on the blocked and
no-proxy paths no longer claims cache mode reads the level.
- Those hints no longer prescribe `HEADROOM_ROLLOUT_CHANNEL=beta`:
`proxy_output_shaper` is available on every channel, and beta would also
enable unrelated features. "Blocked" now points at
`HEADROOM_DISABLE_FEATURES`, the only thing that refuses the hot enable.
- `tests/test_cli_learn.py`:
- CLI tests run `learn --verbosity --apply` against a stubbed local
proxy: cache-mode pin, token mode, an operator pin, an unreadable mode,
recording and then replacing this CLI's own pin, and keeping pins it
cannot prove it set. That last case covers an operator pin equal to the
previously learned level, a record from a restarted proxy, and a value
changed since. Also covered: out-of-range pins, re-pinning the same
level, and a proxy that disables the shaper. Each asserts both the
runtime-env body sent and the message printed.
- Further cases: a reused pid, a record without a start time, a pin
applied while the shaper is blocked, a failed record write, and a proxy
without a pid.
- Fallback branches: a non-numeric pin (read as the default level), a
`/health` reply without a config object, no usable proxy start time
(zero or missing uptime, missing or bad timestamp), and a record that
isn't a JSON object. Every new line and branch in
`headroom/cli/learn.py` is covered (`pytest --cov=headroom.cli.learn
--cov-branch`).
- A `windows_newline` test asserts that the ownership record is written
with `newline="\n"`.
- `tests/test_offline_egress_chokepoint.py`: `cli/learn.py` now has two
loopback egress sites (`/health` and `/admin/runtime-env`), recorded in
the allowlist with the same wording as `cli/wrap.py`.
- The existing `--all --apply` test made a real POST to whatever
listened on `127.0.0.1:8787`, which on a developer machine is their own
proxy. It now runs against a stubbed `urlopen`.
- `README.md`, `wiki/learn.md`: state that cache mode does not read the
learned level, and what `--apply` does there. `wiki/learn.md` no longer
says the shaper needs the beta channel.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# Before: origin/main's headroom/cli/learn.py with the new tests
$ pytest tests/test_cli_learn.py -k verbosity_apply -q
FAILED tests/test_cli_learn.py::test_verbosity_apply_pins_the_level_on_a_cache_mode_proxy
FAILED tests/test_cli_learn.py::test_verbosity_apply_leaves_a_token_mode_proxy_reading_the_profile
FAILED tests/test_cli_learn.py::test_verbosity_apply_does_not_override_an_explicit_level
FAILED tests/test_cli_learn.py::test_verbosity_apply_does_not_claim_live_when_the_mode_is_unknown
4 failed, 22 deselected in 1.04s

# Review round 3: the previous head (e10141c3) with this round's tests
$ pytest tests/test_cli_learn.py -q
FAILED tests/test_cli_learn.py::test_activate_output_shaper_reports_effective_rollout_decision[True-live]
FAILED tests/test_cli_learn.py::test_activate_output_shaper_reports_effective_rollout_decision[False-blocked]
FAILED tests/test_cli_learn.py::test_activate_output_shaper_handles_malformed_response
FAILED tests/test_cli_learn.py::test_verbosity_apply_records_the_pin_it_sets
FAILED tests/test_cli_learn.py::test_verbosity_apply_replaces_the_pin_an_earlier_apply_set
FAILED tests/test_cli_learn.py::test_verbosity_apply_keeps_a_pin_it_cannot_prove_it_set[no-start-time]
FAILED tests/test_cli_learn.py::test_verbosity_pin_record_is_written_with_lf
FAILED tests/test_cli_learn.py::test_verbosity_apply_records_a_pin_the_proxy_applied_while_the_shaper_is_disabled
FAILED tests/test_cli_learn.py::test_verbosity_apply_reports_activation_when_the_pin_cannot_be_recorded
FAILED tests/test_cli_learn.py::test_verbosity_apply_does_not_record_a_pin_without_a_process_identity
10 failed, 31 passed in 0.74s

# After: this branch
$ pytest tests/test_cli_learn.py -q
50 passed in 0.71s

# Offline-egress coverage (failed on e10141c3 in CI shard 3)
$ pytest tests/test_offline_egress_chokepoint.py -k test_every_egress_site_is_guarded_or_allowlisted -q
1 passed, 142 deselected in 2.63s

# Related suites (learn, verbosity, output shaper, runtime-env), with branch coverage of the changed file
$ pytest tests/test_*learn*.py tests/test_*verbosity*.py tests/test_*output_shaper*.py tests/test_*runtime_env*.py -q --cov=headroom.cli.learn --cov-branch
207 passed, 3 warnings in 7.83s
# lines/branches added by this PR in headroom/cli/learn.py that are not covered: none

$ ruff check headroom/cli/learn.py tests/test_cli_learn.py
All checks passed!
$ ruff format --check headroom/cli/learn.py tests/test_cli_learn.py
2 files already formatted
```

## Real Behavior Proof

- Environment: macOS (Apple Silicon), Python 3.11. A real `headroom
proxy --port 8799` in default cache mode from this checkout, with an
isolated `HEADROOM_WORKSPACE_DIR` and `--anthropic-api-url` pointed at a
closed local port, so no request left the machine. `headroom learn
--verbosity --apply` ran over real Claude Code transcripts and learned
level 3. Before = `origin/main`'s `headroom/cli/learn.py`; after = this
branch. A fresh proxy was used for each run.
- Exact command / steps: Start the proxy. Run `HEADROOM_PORT=8799
headroom learn --verbosity --apply --project <repo>`. Send one
`/v1/messages` request through the proxy. Read `/health`
`config.runtime_env` and the proxy's `OutputShaper(...)` log line.
- Observed result: before, the CLI says level 3 is live while the proxy
steers at L2. After, the level is pinned and the proxy steers at L3:

  ```text
  ===== BEFORE (origin/main learn.py), proxy in default cache mode
  >> Recommended verbosity level: 3 (confidence: high)
✓ Output shaper enabled on the running proxy (port 8799); level 3 is
live now (while HEADROOM_VERBOSITY_LEVEL is unset).
To keep it on across restarts: export HEADROOM_ROLLOUT_CHANNEL=beta and
HEADROOM_OUTPUT_SHAPER=1 before `headroom wrap ...`.
proxy /health: {"mode": "cache", "HEADROOM_OUTPUT_SHAPER": "1",
"HEADROOM_VERBOSITY_LEVEL": null}
  proxy log: OutputShaper(L2/cache_mode_default)

  ===== AFTER (this branch), proxy in default cache mode
  >> Recommended verbosity level: 3 (confidence: high)
✓ Output shaper enabled on the running proxy (port 8799); level 3 is
live now (pinned with HEADROOM_VERBOSITY_LEVEL: cache mode does not read
the learned level).
Conversations already open re-cache their prompt once, because the
steering text changed.
To keep it on across restarts: export HEADROOM_OUTPUT_SHAPER=1
HEADROOM_VERBOSITY_LEVEL=3 before `headroom wrap ...` or `headroom
proxy`.
  --- verbosity_pin.json: {"port": 8799, "pid": 25345, "level": 3}
  --- second --apply against the same proxy:
✓ Output shaper enabled on the running proxy (port 8799); level 3 is
live now (pinned with HEADROOM_VERBOSITY_LEVEL: cache mode does not read
the learned level).
proxy /health: {"mode": "cache", "HEADROOM_OUTPUT_SHAPER": "1",
"HEADROOM_VERBOSITY_LEVEL": "3"}
  proxy log: OutputShaper(L3/env_pinned)
  ```
- Not tested: a real token-mode proxy, one with an operator-set
`HEADROOM_VERBOSITY_LEVEL`, and a second `--apply` that learns a
*different* level (all covered by the CLI tests against a stubbed proxy;
the real run above re-learns the same level 3); multi-worker proxies,
which reject `/admin/runtime-env` before and after this change; Windows
(the record write is covered by a `windows_newline` test).

## Runtime Rollout Safety

- Rollout-managed feature(s): `proxy_output_shaper` (unchanged). This PR
only changes what the `headroom learn` CLI sends to and prints about a
running proxy.
- Minimum rollout channel: stable.
- Stable/default behavior changed: only `learn --verbosity --apply`
against a running cache-mode proxy with no `HEADROOM_VERBOSITY_LEVEL`.
It now also pins that variable to the learned level, which is what the
CLI already claimed. The proxy itself is unchanged.
- Kill switch / disable path: run `learn --verbosity` without `--apply`;
or POST `{"HEADROOM_VERBOSITY_LEVEL": ""}` / restart the proxy to drop
the hot override.
- Unsafe override required: no.
- Qualification impact: none expected.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- Type checking was not run locally.
- Pin ownership: `/health` cannot say whether `HEADROOM_VERBOSITY_LEVEL`
came from startup env or a hot override, so `--apply` keeps its own
record. The process start time is derived from `/health` rather than
adding an instance id to the proxy, which keeps this PR CLI-only. The
one remaining ambiguity is an operator hot-POSTing the *same* value to
the *same* proxy process after `--apply` pinned it. That pin would be
treated as ours, but its value is the one `--apply` set anyway.
- Locally, 5 OTLP/Langfuse tests in `test_offline_egress_chokepoint.py`
fail on both `origin/main` and this branch because the dev venv lacks
`opentelemetry-exporter`; they pass in CI.


## Maintainer follow-up at 8c3eba8919

Ownership now compares the exact raw `HEADROOM_VERBOSITY_LEVEL` value
from the same health snapshot as process identity. A changed operator
value is preserved even when it clamps to the same effective level as a
previous CLI pin (for example, CLI4 followed by operator8). The display
still reports the effective clamped level. The new regression fails on
the preceding head and passes here; existing CLI ownership/restart/error
behavior remains covered.

Fresh validation:140 CLI/verbosity/output-shaper/runtime-env tests pass
with one existing warning; Ruff lint/format and Linux-platform mypy
over566files pass. Actual proxy/CLI proof additionally covers learning3,
re-learning1, preserving an operator8 pin, and the
changed-raw/same-clamped-level ownership case. The last case fails
before this follow-up and passes afterward.

Exact commands and full runnable Windows/Python3.12 script: [maintainer
evidence](https://github.com/headroomlabs-ai/headroom/pull/4033#issuecomment-6063120923).
Save the script as `verify_learn_pin.py` in the checkout root and run
`python verify_learn_pin.py` with proxy dependencies installed.
Synthetic Claude transcripts are processed by the real analyzer; no live
LLM judge or provider request is made. Hosted CI and fresh review must
qualify this new head before approval.

---------

Co-authored-by: Kenneth Wong <17061293+KennethWKZ@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-08 10:48:59 -05:00

255 lines
10 KiB
Markdown

# Headroom Learn
Offline failure learning for coding agents. Analyzes past conversations, finds what went wrong, correlates it with what eventually worked, and writes specific project-level learnings that prevent the same mistakes next session.
## Quick Start
```bash
# See recommendations for current project (dry-run, no changes)
headroom learn
# Write recommendations to CLAUDE.local.md (gitignored, personal default)
headroom learn --apply
# Write to the shared team file instead
headroom learn --apply --target CLAUDE.md
# Analyze a specific project
headroom learn --project ~/my-project --apply
# Analyze all projects
headroom learn --all --apply
```
## How It Works
```
Past Sessions → Plugin → Analyzer → Writer → Agent-native context file
│ │ │
│ │ └─ Writes marker-delimited sections
│ │ (replaced on re-run, not duplicated)
│ │
│ └─ LLM-based analysis: finds failure patterns,
│ success correlations, and actionable rules
│
└─ Plugin reads agent-specific logs:
• Claude Code: ~/.claude/projects/*.jsonl
• Codex: ~/.codex/sessions/*.json
• Gemini CLI: ~/.gemini/tmp/*/chats/session-*.json
```
### Success Correlation
The core innovation. Instead of cataloging failures ("Read failed 5 times"), Headroom finds what the model did to fix each failure:
- **Failed**: `Read axion-formats/src/main/java/.../FirstClassEntity.java`
- **Then succeeded**: `Read axion-scala-common/src/main/scala/.../FirstClassEntity.scala`
- **Learning**: "`FirstClassEntity` is at `axion-scala-common/`, not `axion-formats/`"
This produces specific, actionable corrections — not generic advice.
## What It Learns
### 1. Environment Facts → CLAUDE.md
Which runtime commands work vs fail.
```markdown
### Environment
- **Python**: use `uv run python` (not `python3` — modules not available outside venv)
```
### 2. File Path Corrections → CLAUDE.md
Wrong paths the model keeps guessing, with the correct locations.
```markdown
### File Path Corrections
- `axion-common/src/.../AxionSparkConstants.scala`
→ actually at `axion-spark-common/src/.../AxionSparkConstants.scala`
```
### 3. Search Scope → CLAUDE.md
Which directories to search in (narrow paths fail, broader ones work).
```markdown
### Search Scope
- Don't search `axion-model/` → use `axion/` (the repo root)
```
### 4. Command Patterns → CLAUDE.md
How commands should (and shouldn't) be run.
```markdown
### Command Patterns
- **user_prefers_manual**: User rejected gradle 18 times — show the command, don't execute
- **python_runtime**: Use `uv run python` not `python3` (ModuleNotFoundError)
```
### 5. Known Large Files → CLAUDE.md
Files that need `offset`/`limit` with Read.
```markdown
### Known Large Files
- `proxy/server.py` (~8000 lines) — always use offset/limit
```
### 6. Retry Prevention → MEMORY.md
Specific suggestions derived from actual corrections.
### 7. Permission Notes → MEMORY.md
Commands repeatedly rejected — model should suggest them to the user instead.
## Where Learnings Go
| Pattern | Claude Code | Codex | Gemini CLI |
|---------|-------------|-------|-----------|
| Environment, paths, commands | **CLAUDE.local.md** (default) or `CLAUDE.md` (with `--target CLAUDE.md`) | **AGENTS.md** | **GEMINI.md** |
| Retry patterns, permissions | **MEMORY.md** | **instructions.md** | **GEMINI.md** |
Output files are agent-native: Claude Code writes to `CLAUDE.local.md` by default (gitignored, personal); pass `--target CLAUDE.md` for the shared team file. Codex uses `AGENTS.md`, Gemini uses `GEMINI.md`. The same learnings, written to the format each agent reads.
## Marker-Based Updates
Headroom manages a clearly-delimited section in each file:
```markdown
<!-- headroom:learn:start -->
## Headroom Learned Patterns
*Auto-generated by `headroom learn` — do not edit manually*
...
<!-- headroom:learn:end -->
```
On re-run, only the content between markers is replaced. Your existing file content is preserved.
## Architecture (Plugin System)
Headroom Learn uses a plugin architecture where each agent is a self-contained plugin:
```
Plugin Registry (auto-discovered)
├── ClaudeCodePlugin → Analyzer (LLM) → ClaudeCodeWriter → CLAUDE.md / MEMORY.md
├── CodexPlugin → Analyzer (LLM) → CodexWriter → AGENTS.md / instructions.md
├── GeminiPlugin → Analyzer (LLM) → GeminiWriter → GEMINI.md
└── (your plugin) → Analyzer (LLM) → (your writer) → (your file)
```
**Plugins** bundle scanning, detection, and writing for one agent. Built-in plugins are auto-discovered from `headroom.learn.plugins.*`. External plugins register via the `headroom.learn_plugin` entry point.
**The Analyzer** is shared — it uses an LLM (Sonnet, GPT-4o, or Gemini Flash) to find patterns. Same analysis for any agent.
### Adding Support for a New Agent
1. Create `headroom/learn/plugins/myagent.py`
2. Implement `LearnPlugin` + `ConversationScanner` (scanner + writer + detection)
3. Add `plugin = MyAgentPlugin()` at module scope
4. Done — `headroom learn --agent myagent` works automatically
Or install an external plugin: `pip install headroom-learn-cursor` (registers via entry point).
## CLI Reference
```
headroom learn [OPTIONS]
Options:
--project PATH Project directory (default: current directory)
--all Analyze all discovered projects (mutually exclusive with --project)
--apply Write recommendations (default: dry-run)
--target TEXT Context file to write (default: CLAUDE.local.md for Claude Code)
--main-only Write only to the main context file, skip MEMORY.md
--agent [auto|claude|codex|gemini|grok|opencode]
Which agent to analyze (default: auto-detect)
--model TEXT LLM for analysis (default: auto from API keys or CLI)
--workers / -j INTEGER Parallel analysis workers (min 1, default: auto)
--verbosity Analyze verbosity level instead of failure patterns
--llm-judge Use an LLM to score verbosity quality (requires --verbosity)
```
### Verbosity learning (`--verbosity`)
`headroom learn --verbosity` analyzes past sessions to infer the ideal output verbosity level for your project and writes a `verbosity.json` profile.
**Important**: the output shaper is **off by default** (`HEADROOM_OUTPUT_SHAPER=1`
turns it on; it is available on every rollout channel). Running `--verbosity --apply` will either:
- Hot-enable the output shaper on an eligible running proxy (`POST /admin/runtime-env`), OR
- Print instructions to set `HEADROOM_OUTPUT_SHAPER=1` before `headroom wrap ...`
A proxy in cache mode (the default) does not read `verbosity.json`: a level that
changes mid-conversation would bust the prompt cache, so it steers at its startup
level. On a cache-mode proxy `--apply` therefore also pins
`HEADROOM_VERBOSITY_LEVEL` to the learned level, and records the pin (port,
proxy pid, level) in `verbosity_pin.json`. A `HEADROOM_VERBOSITY_LEVEL` that is
already set is left alone, and `--apply` says so. The one exception is a pin that
record proves an earlier `--apply` set: the same proxy process still holding the
same raw value. That pin is replaced. A changed raw pin remains operator-owned
even if it clamps to the same effective level as the previous CLI pin.
To keep the shaper on across proxy restarts, export `HEADROOM_OUTPUT_SHAPER=1` before starting the proxy, plus `HEADROOM_VERBOSITY_LEVEL=<level>` in cache mode.
**Flag interactions**:
- `--all` and `--project` are mutually exclusive
- `--llm-judge` requires `--verbosity`
- `--verbosity --all --apply` is rejected (verbosity persists a single global level)
### Supported Agents
| Agent | Scanner | Writer | Output Files |
|-------|---------|--------|-------------|
| **Claude Code** | Reads `~/.claude/projects/*.jsonl` | ClaudeCodeWriter | CLAUDE.md, MEMORY.md |
| **OpenAI Codex** | Reads `~/.codex/sessions/*.json` | CodexWriter | AGENTS.md, instructions.md |
| **Gemini CLI** | Reads `~/.gemini/tmp/*/chats/session-*.json` | GeminiWriter | GEMINI.md |
| **Grok CLI** | Reads `~/.grok/sessions/<workspace>/<session-id>/updates.jsonl` | GrokWriter | GROK.md |
| **OpenCode** | Reads the OpenCode SQLite DB at `~/.local/share/opencode/opencode.db` (or `opencode-local.db`; override with `HEADROOM_OPENCODE_DB`) | CodexWriter (shared) | AGENTS.md, instructions.md |
## LLM Backend Selection
`headroom learn` needs an LLM to analyze your sessions. It picks one automatically using this priority:
| Priority | Source | Example |
|----------|--------|---------|
| 1 | `--model` flag | `headroom learn --model gpt-4o` |
| 2 | API key env var | `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY` |
| 3 | `HEADROOM_LEARN_CLI` env var | `export HEADROOM_LEARN_CLI=gemini` |
| 4 | Auto-detect installed CLIs | Checks PATH for `claude`, `gemini`, `codex` |
### Using without an API key
If you use Claude Code, Gemini CLI, or Codex via subscription (no raw API key), `headroom learn` can call them directly:
```bash
# Auto-detects claude in PATH — no API key needed
headroom learn
# Explicitly select a CLI backend
headroom learn --model gemini-cli
# Pin a CLI via environment variable
export HEADROOM_LEARN_CLI=codex
headroom learn
```
Valid values for `HEADROOM_LEARN_CLI`: `claude`, `gemini`, `codex`, `agy`.
`agy` (Antigravity CLI) is never auto-detected. It has no mode that starts a session without tools, so instructions embedded in analyzed session text could make it write files or run shell commands with your permissions. To use it anyway, opt in explicitly:
```bash
export HEADROOM_LEARN_CLI=agy
export HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1
headroom learn
```
Without `HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1`, selecting `agy` (via `HEADROOM_LEARN_CLI` or `--model agy-cli`) fails before any analysis runs.
## Real-World Results
Tested on 67,583 tool calls across 23 projects:
| Metric | Value |
|--------|-------|
| Failure rate | 7.5% (5,066 failures) |
| Corrections extracted | 164 per project (avg) |
| Specific path corrections | 22 (axion project) |
| Search scope corrections | 24 (axion project) |
| Command patterns learned | 5 (axion project) |
| Estimated preventable waste | ~27 MB across corpus |