fix(memory): ignore leading cd prefix when pairing Bash error recoveries (#3776)

## Description

`TrafficLearner` pairs unrelated Bash commands into `Learned: error
recovery` rules when both commands start with the same `cd <dir> &&`
prefix. Coding agents add that prefix to almost every Bash call, so most
Bash recoveries written to `MEMORY.md` are bogus. Two examples written
to my `MEMORY.md`:

```
- Command `cd "C:\...\repo" && grep -rn "shopping-vintage" ... -l .` fails (timeout).
  Use `cd "C:\...\repo" && cat components/places/categoryIcons.ts && ...` instead.
- Command `cd "C:\...\repo" && npx vitest run "...SubmitPlaceForm.test.tsx" 2>&1 | tail -40` fails (runtime_error).
  Use `cd "C:\...\repo" && git add -A && git commit -F - <<'EOF' ...` instead.
```

The second rule tells the agent to commit instead of running the tests.

Two defects compound:

1. **Matcher false positives.** `_commands_related_as_retry` requires
the same first binary, plus either ≤40% edit distance or a shared
substantive token. When both commands share a `cd <dir> &&` prefix, the
first binary is always `cd` and the directory path is always a shared
token of ≥5 characters, so any two such commands pass.
2. **Hash collapse inflates evidence.** `_normalize_bash_for_hash` cuts
at the first `&&`, so every cd-prefixed pair hashes to
`error_recovery|Bash|cd <dir>|cd <dir>`. Unrelated pairs in one project
merge into a single row whose evidence is the sum of all of them. That
is how they get past the `min_evidence` gate.

This is a follow-up to #326, which introduced the retry matcher and the
evidence gate. I found no existing issue or PR for this.

Closes # (no existing issue)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `_strip_leading_cd()` in `headroom/memory/traffic_learner.py`. It
removes leading `cd <dir> &&` / `cd <dir>;` segments, including quoted
paths, backslash-escaped spaces (`cd /home/dev/my\ project && ...`) and
repeated segments. A bare `cd <dir>` with no chain is left unchanged.
- Apply it in `_commands_related_as_retry` before the binary,
edit-distance and token-overlap checks.
- Apply it in `_normalize_bash_for_hash`, so the hash key reflects the
command that actually ran.
- Tests in `tests/test_memory/test_traffic_learner.py`:
- `test_shared_cd_prefix_alone_is_not_retry` and
`test_strips_leading_cd` fail on `main` and pass with this change.
- `test_cd_prefixed_retry_still_matches` guards against rejecting real
retries.
- **Changed expectation:** `test_cuts_at_first_chain` previously
asserted `_normalize_bash_for_hash("cd /tmp && ls") == "cd /tmp"`. That
assertion encoded the bug. It now uses `make build && ls`, and the cd
case moved to `test_strips_leading_cd`, which asserts `"cd /tmp && ls"`
normalizes to `"ls"`.

## Testing

- [ ] Unit tests pass (`pytest`) (see Additional Notes)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) (see Additional Notes)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_memory/test_traffic_learner.py -k "CommandsRelatedAsRetry or NormalizeBashForHash or RefineErrorRecovery or NormalizedHash"
=============== 35 passed, 122 deselected, 39 warnings in 0.80s ===============

# New tests on main (fix reverted), full file:
FAILED tests/test_memory/test_traffic_learner.py::TestCommandsRelatedAsRetry::test_shared_cd_prefix_alone_is_not_retry
FAILED tests/test_memory/test_traffic_learner.py::TestNormalizeBashForHash::test_strips_leading_cd

$ ruff check .
All checks passed!

$ mypy headroom/memory/traffic_learner.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.14, headroom 0.38.0, Claude Code via
`headroom proxy --learn --memory`.
- Exact command / steps: Read every persisted `traffic_learner` Bash
`error_recovery` row from my own `memory.db` (read-only). Run
`_commands_related_as_retry` and `_normalize_bash_for_hash` on each
row's `failed_cmd` / `success_cmd`, first with `main` and then with this
branch.
- Observed result: 72 of 96 Bash rows had a cd prefix on both sides, and
`main` accepted all 72 as retries; on this branch 16 are accepted. On
`main` those rows also shared hash keys (e.g. 6 unrelated pairs on one
`cd <repo>|cd <repo>` key); on this branch every row gets its own key.
- Not tested: A full proxy session on Linux or macOS. I also did not
check how long existing polluted rows take to disappear from a live
`MEMORY.md` after upgrading.

## Runtime Rollout Safety

- Rollout-managed feature(s): None
- Minimum rollout channel: dev
- Stable/default behavior changed: Yes, narrowly. Bash error-recovery
pairing and hash keys ignore a leading `cd <dir>` segment. Nothing else
changes.
- Kill switch / disable path: N/A (existing `--learn` opt-in is
unchanged)
- Unsafe override required: No
- Qualification impact: None
- Rollback path: Revert commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes (see
Additional Notes)
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- **Unit tests.** On my Windows machine, 45 tests in
`test_traffic_learner.py` already fail on `main`. Examples are
`TestProjectForPattern` (POSIX path fixtures) and
`TestUserAuthoredPreferenceFiltering`. The set of failing tests is
identical before and after this change, apart from the new tests. I have
not run the suite on Linux/macOS and am relying on CI for that.
- **Type checking.** `mypy headroom` reports one error on Windows, in
`headroom/fileperms.py:71` (`os.fchmod` does not exist on Windows). That
file is untouched here. The changed module passes mypy.
- **Documentation.** N/A, this is an internal matcher fix.
- **Out of scope.** 16 remaining accepted pairs include some that are
still questionable, where commands share a binary and a file name, e.g.
`git status --short lib/x.ts` → `git add lib/x.ts && git commit`. That
is a broader weakness of the token-overlap rule for compound commands.
- **Existing hash keys move.** Any Bash command with a leading `cd
<dir>` segment now normalizes to a different hash key, so rows persisted
under the old key are not re-keyed. Their accumulated evidence counts
are orphaned and learning for those commands restarts from zero. This is
intentional: the old keys merged unrelated pairs, so their evidence was
inflated.
- **Existing rows.** Rows already persisted with the collapsed hash are
not rewritten. They split into separate hashes on load, so most should
fall below `min_evidence`, and the 21-day hard floor retires the rest. I
did not add render-time re-validation for Bash rows, because
`test_bash_recoveries_not_revalidated` documents that as intentional.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Trîggered
2026-09-30 22:41:13 -05:00
committed by GitHub
co-authored by JD Davis
parent 8dbbd1d896
commit 143a38d56f
2 changed files with 42 additions and 2 deletions
+16 -1
View File
@@ -62,6 +62,10 @@ _BASH_VOLATILE_SUFFIX_RE = re.compile(
r"|\s+2>&1|\s+2>/dev/null)+\s*$"
)
# Leading `cd <dir> && ` / `cd <dir>; ` prefixes. Agents prepend them to most
# commands, so they carry no signal about which operation was retried.
_BASH_LEADING_CD_RE = re.compile(r"""^\s*cd\s+(?:"[^"]*"|'[^']*'|(?:\\.|[^\s\\])+)\s*(?:&&|;)\s*""")
# Agent harnesses can encode orchestration metadata as user-role messages.
# These prefixes identify whole messages that are not authored by the user.
_HARNESS_USER_PREFIXES = (
@@ -194,7 +198,7 @@ def _normalize_bash_for_hash(cmd: str) -> str:
if not cmd:
return ""
# Drop paging, line-context flags, and redirections that vary between runs.
trimmed = _BASH_VOLATILE_SUFFIX_RE.sub("", cmd).strip()
trimmed = _BASH_VOLATILE_SUFFIX_RE.sub("", _strip_leading_cd(cmd)).strip()
# Cut at the first pipe or && so we hash the primary command, not the tail.
for sep in (" | ", " && "):
idx = trimmed.find(sep)
@@ -204,6 +208,15 @@ def _normalize_bash_for_hash(cmd: str) -> str:
return trimmed
def _strip_leading_cd(cmd: str) -> str:
"""Remove leading `cd <dir> &&` / `cd <dir>;` segments from a Bash command."""
prev = None
while prev != cmd:
prev = cmd
cmd = _BASH_LEADING_CD_RE.sub("", cmd, count=1)
return cmd
# =============================================================================
# Error Classification (reused from learn/scanner.py patterns)
# =============================================================================
@@ -372,6 +385,8 @@ def _commands_related_as_retry(failed: str, success: str) -> bool:
arguments, and gets rejected. Genuine retries (extra flag, single
arg edit) pass via the edit-distance path.
"""
failed = _strip_leading_cd(failed)
success = _strip_leading_cd(success)
if not failed or not success or failed == success:
return False
bin_a = _bash_first_binary(failed)
+26 -1
View File
@@ -116,6 +116,23 @@ class TestCommandsRelatedAsRetry:
)
assert not _commands_related_as_retry(failed, success)
def test_shared_cd_prefix_alone_is_not_retry(self):
# Agents prefix most commands with the same `cd <repo> &&`; that shared
# prefix used to satisfy both the binary and the token-overlap checks.
cd = 'cd "C:\\Users\\dev\\my-project" && '
assert not _commands_related_as_retry(
cd + "npx vitest run src/form.test.tsx", cd + "git add -A && git commit -F -"
)
assert not _commands_related_as_retry(
"cd /home/dev/my-project && sed -n '90,130p' app/actions.ts",
"cd /home/dev/my-project && cat components/icons.ts",
)
def test_cd_prefixed_retry_still_matches(self):
assert _commands_related_as_retry(
"cd /home/dev/proj && cargo build", "cd /home/dev/proj && cargo build --release"
)
def test_empty_or_equal_commands_rejected(self):
assert not _commands_related_as_retry("", "ls")
assert not _commands_related_as_retry("ls", "")
@@ -1930,7 +1947,15 @@ class TestNormalizeBashForHash:
def test_cuts_at_first_chain(self):
# && boundary collapses to just the primary command
assert _normalize_bash_for_hash("cd /tmp && ls") == "cd /tmp"
assert _normalize_bash_for_hash("make build && ls") == "make build"
def test_strips_leading_cd(self):
# A `cd` prefix is not the primary command; keeping it collapsed every
# cd-prefixed recovery in a project onto one hash key.
assert _normalize_bash_for_hash("cd /tmp && ls") == "ls"
assert _normalize_bash_for_hash('cd "C:\\a b"; cd sub && cargo check') == "cargo check"
assert _normalize_bash_for_hash(r"cd /home/dev/my\ project && cargo check") == "cargo check"
assert _normalize_bash_for_hash("cd /tmp") == "cd /tmp"
class TestParseIsoTimestamp: