mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(read-lifecycle): keep Reads fresh after a failed Edit or Write (#4461)
## Description `ReadLifecycleManager` counted every `Edit`/`Write` tool_use as a change to the file, so an Edit whose tool_result had `is_error: true` still turned earlier Reads of that file STALE and replaced their content with a marker, although the file never changed. This collects the tool_use ids whose tool_result is an error and leaves them out of the edit operations. Fixes #4460 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Add `ReadLifecycleManager._failed_tool_use_ids` (Anthropic `tool_result` blocks with `is_error` set) and skip those ids when building the edit operations in `_build_file_operation_index`. - Add two tests to `tests/test_transforms/test_read_lifecycle.py`: a failed Edit leaves the Read fresh; a later successful Edit still makes it stale. ## Testing Rebased onto upstream/main `3fef15d3` (2026-10-24): the new tests pass (25 passed); with only the source change reverted, 1 fails. The real-behavior run below was done on `e995eb9f`. Review round 1 (6aa14c9d, Devin findings): only a `<tool_use_error>` (pre-execution validation) keeps earlier Reads fresh; any other error, such as an interrupted call that may have partly run, still counts as an edit; an error result whose `tool_use_id` is not a string is ignored instead of raising. `tests/test_transforms/test_read_lifecycle.py`: 28 passed; on the previous head the two new tests fail, and on main the fresh-read tests fail. - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # new tests, source change reverted E AssertionError: assert 1 == 0 FAILED tests/test_transforms/test_read_lifecycle.py::TestAnthropicFormat::test_failed_edit_does_not_make_read_stale 1 failed, 24 passed # with the fix tests/test_transforms/test_read_lifecycle.py: 25 passed tests/test_read_maturation.py tests/test_read_maturation_handler_nobust.py tests/test_transforms: 510 passed, 78 skipped, 1 failed (the 1 failure is test_kompress_compressor.py::TestLazyImports::test_unloadable_native_dep_degrades_instead_of_raising; it fails the same way with the source change reverted) ruff check: All checks passed! ruff format --check: 2 files already formatted ``` ## Real Behavior Proof - Environment: Ubuntu (WSL2, Linux 6.18), Python 3.12.15, upstream maine995eb9fplus this change. Real proxy on 127.0.0.1:18980 started with `HEADROOM_MODE=token` (the default cache mode keeps earlier turns in the frozen prefix) and `ANTHROPIC_TARGET_API_URL=http://127.0.0.1:18981`, a local stub upstream that logs each tool_result it receives. The proxy log banner shows `Anthropic: http://127.0.0.1:18981`, so no request left the machine. `headroom/_core.abi3.so` was taken from the official headroom-ai 0.41.0 wheel (untracked, not committed). - Exact command / steps: start the stub (`stub.py 18981 stub.log`), start the proxy (`HEADROOM_MODE=token ANTHROPIC_TARGET_API_URL=http://127.0.0.1:18981 python -P -m headroom.proxy.server --host 127.0.0.1 --port 18980 --no-cache --no-rate-limit`), then run `client.py 18980`: one `POST /v1/messages` with Read `/src/app.py` (6,980-char result), Edit `/src/app.py` whose tool_result has `is_error: true`, and two more turns. Repeated with the source change reverted (before), with it applied (after), and with a successful Edit result (control). - Observed result: before, the upstream received the earlier Read result replaced by a stale marker (151 chars) after an Edit whose tool_result had is_error=True; after, the Read result arrives unchanged (6980 chars); the control (Edit result without is_error) is still replaced. Stub log: ```text # before (change reverted): upstream receives the Read result replaced STUB POST /v1/messages bytes=1494 STUB msg[2] tool_result toolu_r1 is_error=None len=151 head='[Read content stale: /src/app.py was modified after this read — re-read the file for current content. Retrieve' STUB msg[4] tool_result toolu_e1 is_error=True len=69 head='<tool_use_error>String to replace not found in file.</tool_use_error>' # after (this change): upstream receives the Read result unchanged STUB POST /v1/messages bytes=8206 STUB msg[2] tool_result toolu_r1 is_error=None len=6980 head='def handler_0(request):\n return process(request, 0)\n\ndef handler_1(request):\n return process(request, 1)' STUB msg[4] tool_result toolu_e1 is_error=True len=69 head='<tool_use_error>String to replace not found in file.</tool_use_error>' # control (this change, Edit result without is_error): still replaced STUB POST /v1/messages bytes=1460 STUB msg[2] tool_result toolu_r1 is_error=None len=151 head='[Read content stale: /src/app.py was modified after this read — re-read the file for current content. Retrieve' STUB msg[4] tool_result toolu_e1 is_error=None len=51 head='The file /src/app.py has been updated successfully.' ``` Scripts and raw outputs: `/var/tmp/cb-pub/proof/fix-read-lifecycle-failed-edit/` (WSL): `stub.py`, `client.py`, `run.sh`, `before.out`, `after.out`, `control.out`. - Not tested: a real Anthropic upstream (stub only); OpenAI-format conversations (tool messages carry no `is_error`, so behavior there is unchanged); streaming requests; cache mode; the Rust core is the 0.41.0 wheel's, not one built from this checkout; mypy; the full test suite. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: only for Anthropic conversations containing an Edit/Write whose tool_result has `is_error`; earlier Reads of that file are no longer replaced by a stale marker - Kill switch / disable path: `headroom proxy --no-read-lifecycle` - Unsafe override required: no - Qualification impact: none known - Rollback path: revert the commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes Not included: `MultiEdit`/`NotebookEdit` are still missing from `_MUTATING_TOOL_NAMES` (`NotebookEdit` also uses `notebook_path`, which the path extraction does not read); that is a separate change. Only the listed test files were run, not the whole suite. Produced with AI assistance (Claude Code) operated by me (HetCreep); the reproduction was run on my machine on a clone of the repository. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
cddd2fd60e
commit
4c3799fd81
@@ -96,6 +96,16 @@ class ReadLifecycleResult:
|
||||
ccr_hashes: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def _rejected_before_running(content: Any) -> bool:
|
||||
"""True when a tool_result's text is a ``<tool_use_error>`` validation error."""
|
||||
if isinstance(content, list):
|
||||
content = next(
|
||||
(b.get("text") for b in content if isinstance(b, dict) and b.get("type") == "text"),
|
||||
None,
|
||||
)
|
||||
return isinstance(content, str) and content.lstrip().startswith("<tool_use_error>")
|
||||
|
||||
|
||||
class ReadLifecycleManager:
|
||||
"""Event-driven Read lifecycle management.
|
||||
|
||||
@@ -240,6 +250,7 @@ class ReadLifecycleManager:
|
||||
Groups all Read/Edit/Write operations by file_path for lifecycle analysis.
|
||||
"""
|
||||
file_ops: dict[str, list[FileOperation]] = defaultdict(list)
|
||||
failed_ids = self._failed_tool_use_ids(messages)
|
||||
|
||||
for tc_id, (name, file_path, offset, limit) in tool_metadata.items():
|
||||
if not file_path:
|
||||
@@ -247,7 +258,8 @@ class ReadLifecycleManager:
|
||||
|
||||
if name in _READ_TOOL_NAMES:
|
||||
operation = "read"
|
||||
elif name in _MUTATING_TOOL_NAMES:
|
||||
elif name in _MUTATING_TOOL_NAMES and tc_id not in failed_ids:
|
||||
# A failed Edit/Write left the file unchanged, so earlier Reads stay valid.
|
||||
operation = "edit"
|
||||
else:
|
||||
continue
|
||||
@@ -271,6 +283,33 @@ class ReadLifecycleManager:
|
||||
|
||||
return dict(file_ops)
|
||||
|
||||
@staticmethod
|
||||
def _failed_tool_use_ids(messages: list[dict[str, Any]]) -> set[str]:
|
||||
"""Ids of Anthropic tool_use blocks that were rejected before they ran.
|
||||
|
||||
Only an is_error tool_result whose text is a ``<tool_use_error>`` (the
|
||||
harness's pre-execution validation, e.g. "String to replace not found")
|
||||
proves the file was not touched. Any other error, such as a call that was
|
||||
interrupted and "may have partially or fully completed", still counts as
|
||||
an edit.
|
||||
"""
|
||||
failed: set[str] = set()
|
||||
for msg in messages:
|
||||
content = msg.get("content")
|
||||
if not isinstance(content, list):
|
||||
continue
|
||||
for block in content:
|
||||
if not (
|
||||
isinstance(block, dict)
|
||||
and block.get("type") == "tool_result"
|
||||
and block.get("is_error")
|
||||
):
|
||||
continue
|
||||
tool_use_id = block.get("tool_use_id")
|
||||
if isinstance(tool_use_id, str) and _rejected_before_running(block.get("content")):
|
||||
failed.add(tool_use_id)
|
||||
return failed
|
||||
|
||||
def _find_tool_call_msg_index(
|
||||
self, messages: list[dict[str, Any]], tool_call_id: str
|
||||
) -> int | None:
|
||||
|
||||
@@ -142,6 +142,7 @@ def make_anthropic_tool_result(tool_call_id: str, content: str) -> dict:
|
||||
|
||||
LARGE_CONTENT = "x" * 2000 # Well above min_size_bytes
|
||||
SMALL_CONTENT = "tiny" # Below min_size_bytes
|
||||
REJECTED_EDIT = "<tool_use_error>String to replace not found in file.</tool_use_error>"
|
||||
|
||||
|
||||
# =============================================================================
|
||||
@@ -436,6 +437,94 @@ class TestAnthropicFormat:
|
||||
user_msg = result.messages[1]
|
||||
assert user_msg["content"][0]["content"] == LARGE_CONTENT
|
||||
|
||||
def test_failed_edit_does_not_make_read_stale(self):
|
||||
"""A tool_result with is_error means the edit did not happen."""
|
||||
mgr = ReadLifecycleManager(ReadLifecycleConfig(enabled=True))
|
||||
|
||||
failed = make_anthropic_tool_result("e1", REJECTED_EDIT)
|
||||
failed["content"][0]["is_error"] = True
|
||||
messages = [
|
||||
make_anthropic_read("r1", "/src/app.py"),
|
||||
make_anthropic_tool_result("r1", LARGE_CONTENT),
|
||||
make_anthropic_edit("e1", "/src/app.py"),
|
||||
failed,
|
||||
]
|
||||
|
||||
result = mgr.apply(messages)
|
||||
assert result.reads_stale == 0
|
||||
assert result.reads_fresh == 1
|
||||
assert result.messages[1]["content"][0]["content"] == LARGE_CONTENT
|
||||
|
||||
def test_successful_edit_after_failed_edit_still_makes_read_stale(self):
|
||||
mgr = ReadLifecycleManager(ReadLifecycleConfig(enabled=True))
|
||||
|
||||
failed = make_anthropic_tool_result("e1", REJECTED_EDIT)
|
||||
failed["content"][0]["is_error"] = True
|
||||
messages = [
|
||||
make_anthropic_read("r1", "/src/app.py"),
|
||||
make_anthropic_tool_result("r1", LARGE_CONTENT),
|
||||
make_anthropic_edit("e1", "/src/app.py"),
|
||||
failed,
|
||||
make_anthropic_edit("e2", "/src/app.py"),
|
||||
make_anthropic_tool_result("e2", "edit success"),
|
||||
]
|
||||
|
||||
result = mgr.apply(messages)
|
||||
assert result.reads_stale == 1
|
||||
|
||||
def test_interrupted_edit_still_makes_read_stale(self):
|
||||
"""An error that is not a pre-run rejection may have changed the file."""
|
||||
mgr = ReadLifecycleManager(ReadLifecycleConfig(enabled=True))
|
||||
|
||||
failed = make_anthropic_tool_result(
|
||||
"e1",
|
||||
"Interrupted: the response that made this tool call was stopped while "
|
||||
"the call was running; it may have partially or fully completed.",
|
||||
)
|
||||
failed["content"][0]["is_error"] = True
|
||||
messages = [
|
||||
make_anthropic_read("r1", "/src/app.py"),
|
||||
make_anthropic_tool_result("r1", LARGE_CONTENT),
|
||||
make_anthropic_edit("e1", "/src/app.py"),
|
||||
failed,
|
||||
]
|
||||
|
||||
result = mgr.apply(messages)
|
||||
assert result.reads_stale == 1
|
||||
|
||||
def test_rejected_edit_with_text_block_content_keeps_read_fresh(self):
|
||||
mgr = ReadLifecycleManager(ReadLifecycleConfig(enabled=True))
|
||||
|
||||
failed = make_anthropic_tool_result("e1", "")
|
||||
failed["content"][0]["content"] = [{"type": "text", "text": REJECTED_EDIT}]
|
||||
failed["content"][0]["is_error"] = True
|
||||
messages = [
|
||||
make_anthropic_read("r1", "/src/app.py"),
|
||||
make_anthropic_tool_result("r1", LARGE_CONTENT),
|
||||
make_anthropic_edit("e1", "/src/app.py"),
|
||||
failed,
|
||||
]
|
||||
|
||||
result = mgr.apply(messages)
|
||||
assert result.reads_stale == 0
|
||||
assert result.messages[1]["content"][0]["content"] == LARGE_CONTENT
|
||||
|
||||
def test_error_result_with_non_string_tool_use_id_is_ignored(self):
|
||||
mgr = ReadLifecycleManager(ReadLifecycleConfig(enabled=True))
|
||||
|
||||
bad = make_anthropic_tool_result("e1", REJECTED_EDIT)
|
||||
bad["content"][0]["tool_use_id"] = ["e1"]
|
||||
bad["content"][0]["is_error"] = True
|
||||
messages = [
|
||||
make_anthropic_read("r1", "/src/app.py"),
|
||||
make_anthropic_tool_result("r1", LARGE_CONTENT),
|
||||
bad,
|
||||
]
|
||||
|
||||
result = mgr.apply(messages)
|
||||
assert result.reads_fresh == 1
|
||||
assert result.messages[1]["content"][0]["content"] == LARGE_CONTENT
|
||||
|
||||
|
||||
class TestCCRStoreIntegration:
|
||||
"""Lifecycle manager stores originals in CCR."""
|
||||
|
||||
Reference in New Issue
Block a user