mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
## Description Item 3 of #4478, the quoted-pattern case that #4521 left open. Read detection scanned the raw command text for shell syntax without regard to quoting: `_segment_is_read` treated any `>` as a redirect (`content_router.py:893` on `e4a60ed7`) and split the pipeline at the first `|`, and `_command_segments` split on `;`, `&&` and `||`. A quiet sed whose script holds one of these characters, such as `sed -n '/if x > 0/,/return/p' a.py` or the GNU alternation `sed -n '/def \|class /p' a.py`, was therefore not detected as a file read and lost read protection. The three scans now run on a copy of the command in which quoted, backslash-escaped and commented characters are masked, and the match positions are mapped back onto the original text. Two shell rules keep word boundaries intact (review round 2): a backslash-newline is a line continuation and is masked as two spaces, so a redirect or `tee` on the next line still counts as a write; and an unquoted `#` at the start of a word masks the rest of its line without changing quote state, so a quote mark inside a comment cannot hide the next line's separators. Unquoted operators behave as before. Refs #4478 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/transforms/content_router.py`: new `_mask_quoted(command)`, which returns a same-length string with characters inside `'...'` / `"..."`, backslash-escaped characters and comment text replaced by `_` (the quote characters are kept; a backslash-newline becomes two spaces; a comment's newline stays visible). `_segment_is_read` runs its redirect/`tee` check and its pipeline split on the masked string; `_command_segments` finds `;` / `&&` / `||` on the masked string and slices the original. - `tests/test_transforms/test_read_command_detection.py`: `test_quoted_shell_operators_are_arguments`, 25 cases. Quoted `>`, `>>`, `|`, `;`, `&&` and `tee` inside a sed script or a file name are reads; a real redirect, pipe-to-`tee` or separator next to a quoted script still counts; `bash -lc "cat foo.py > copy.py"` is still not a read (the inner command is checked on its own, as before). Round 2 adds: continued `>`, `>>` and `| tee` are writes; a comment with an unmatched `"` or `'` followed by `ls; cat foo.py` / `true; cat foo.py` on the next line is a read; `cat foo.py # > not a redirect` is a read; a `#` inside a word is not a comment. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) (run on `headroom/transforms/content_router.py` only, see below) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # fail before (source ate4a60ed7, new test kept) tests/test_transforms/test_read_command_detection.py: 9 failed, 104 passed # round 2: the new review cases against the round-1 head639bfa9e7 failed, 115 passed # pass after (head50798e5a) tests/test_transforms/test_read_command_detection.py: 122 passed # wider set, same environment: # tests/test_transforms + tests/test_*content_router*.py + tests/test_router*.pye4a60ed7: 27 failed, 930 passed, 96 skipped (round 1 comparison)639bfa9e: 25 failed, 941 passed, 96 skipped50798e5a: 18 failed, 948 passed, 96 skipped # the 18 are the same tests on every side (the extras are the new cases); they # come from the local `_core` build not matching this tree (Rust parity # fixtures, minification), not from this change # ruff check / ruff format --check: All checks passed! / 2 files already formatted # mypy headroom/transforms/content_router.py: Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Ubuntu (WSL2) on Windows 11, Python 3.12.15, `headroom` Python source at `e4a60ed7` and at this branch, offline. - Exact command / steps: `ContentRouter.apply` on a three-message transcript (user, an assistant `Bash` tool call, a 3,890-byte Python tool result), with `HEADROOM_PROTECT_READS=1`, `HEADROOM_SHELL_RESULT_MIN_CHARS=2000` and the router config used by `test_quiet_sed_output_reaches_read_protection` (`enable_kompress=False`, `enable_code_aware=False`). Checked whether `router:read_protected` is in `transforms_applied` for each command. - Observed result: on `e4a60ed7` the three quoted-operator commands lose read protection; on this branch they keep it, and both controls are unchanged (table below). For the review-round cases, real `bash -c` in a temporary directory agrees with `_is_read_command` on this head for all 7 (continued `>`, `>>` and `| tee` write `copy.py`; the three multiline-comment commands and `cat foo.py # > not a redirect` print the file); on `639bfa9e` 6 of the 7 disagreed with bash. | command | `e4a60ed7` | this branch | |---|---|---| | `sed -n '1,120p' foo.py` (control) | protected | protected | | `sed -n '/if value > 0/,/return 0/p' foo.py` | not protected | protected | | `sed -n '/def \|class /p' foo.py` | not protected | protected | | `sed -n '/x;y/p' foo.py` | not protected | protected | | `sed -n '1,120p' foo.py > out.txt` (control, real redirect) | not protected | not protected | - Not tested: a run in which an unprotected read is actually compressed. With Kompress and code-aware compression off, the tool result came back byte-identical on both sides, so this proof shows the classification only. Not tested: the full test suite and `mypy headroom` as a whole (only `content_router.py`). Heredoc handling (`<<`) is unchanged and still a whole-command bailout. ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: N/A - Stable/default behavior changed: Yes, narrowly: commands whose only `>`, `|`, `;`, `&&` or `||` sits inside quotes, after a backslash or inside a comment are now classified by the rest of the command, so a quiet sed or `cat` of that kind gets read protection when `HEADROOM_PROTECT_READS` is on. - Kill switch / disable path: None added; read protection as a whole is still controlled by `HEADROOM_PROTECT_READS`. - Unsafe override required: No - Qualification impact: None expected - Rollback path: revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes #4478 had four items. Items 1 and 2 were fixed by #4488, item 4 by #4583, and the quiet-option half of item 3 by #4521, whose description notes that quoted redirect parsing remained separate. This PR covers that remainder. The `|` and `;` cases are the same cause as the `>` case reported in the issue (a quote-blind scan of the same command), so they are fixed together rather than left for a later PR. Review round 2 (head `50798e5a`) addresses the two masking regressions found by Devin and Copilot and confirmed by JerrettDavis: backslash-newline continuations and quote marks inside comments. Produced with AI assistance (Claude Code) operated by me (HetCreep); the reproduction was run on my machine against a clone of the repository. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ### Maintainer requalification Re-reviewed the author update against upstream maine4a60ed72f. The two earlier actual Git Bash counterexamples now classify correctly: continued redirect writes an exact copy and is not a read; a quote inside a comment no longer hides the next-line source read. All 132 focused read-detection and Gemini tests pass; touched-source Ruff/format and whitespace checks pass. The only maintainer change is the shared Gemini ASGI request test fixture from #4644; hosted failure was the same four independently verified main failures, with no contributor production logic changed. Addressed Devin/Copilot threads resolved after inspecting author fixes. Exact-head hosted CI remains pending. ### Maintainer current-main requalification Exact head `2461a910e31c5140d160942927c1c7b3fa203cc1`, upstream main `ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`. Cleanly integrated upstream/mainea59ac3eefand re-reviewed the additive content-router overlap with main's confident HTML classification. Actual isolated tree-sitter0.26.0:133 read-command/HTML tests pass,94 unrelated cases deselected,1 HTML assertion fails. A control with byte-identical main Python/transforms/test/lock inputs reproduces that same unrecoverable_sections assertion using the same existing development native extension, so it is not a finding against this PR. Read-command cases pass; prior actual Git Bash continued-redirection/comment-quote evidence remains scoped to unchanged shell parser inputs. Ruff and whitespace pass. No contributor behavior rewritten, fresh native build or cloud runtime claim. No new PR correctness finding; final approval awaits exact-head hosted checks. Nothing merged. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. ### Maintainer current-main requalification Exact head `69a6bd592932467162d9433601b0c2bd1c4c4728`, upstream main `c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`. Final post-Dependabot requalification against upstream/mainc01cf07d94. All six Dependabot updates are now merged. Relative to this branch's freshly tested prepared tree, the final main sync changes only the three intended Cargo.lock records; Python/source/test inputs are unchanged. Re-reviewed the integration and confirmed complete native source/lock/toolchain objects match the tested combined build and exact fresh native DLL used for this branch's local tests. Focused fresh-native qualification: 134 passed, 94 deselected in 5.40s. Selected real dependency overlays and native DLL paths are recorded in final4646-fresh-native-tests.log; the DLL provenance/input hashes are in final-native-artifact-proof.json. No redundant rerun is implied after this Cargo-only merge. Whitespace passes. Prior source review and scoped real-behavior evidence remain applicable to unchanged contributor inputs. No new source finding in the reviewed scope. Final formal approval awaits exact-head hosted qualification. The earlier HTML assertion with the old primary extension now passes with the fresh combined DLL; the same case also passes on a byte-identical-main Python-source control. No production workaround or assertion suppression was required. Maintainers handle hosted checks; no external-provider claim, contributor semantic rewrite or non-Dependabot merge. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
245 lines
9.8 KiB
Python
245 lines
9.8 KiB
Python
"""Tests for `_is_read_command` — which shell commands count as file reads.
|
|
|
|
Read detection drives read protection: a command classified as a file read has
|
|
its output passed verbatim instead of lossy-compressed, because the agent needs
|
|
exact bytes to build a patch from. Chained commands used to fall through: only
|
|
the first program was inspected, so `wc -l a.py && sed -n '1,60p' a.py` was
|
|
classified as a `wc` run and the file content was word-dropped by Kompress.
|
|
"""
|
|
|
|
import json
|
|
|
|
import pytest
|
|
|
|
from headroom.tokenizers.estimator import EstimatingTokenCounter
|
|
from headroom.transforms.content_router import (
|
|
ContentRouter,
|
|
ContentRouterConfig,
|
|
_is_read_command,
|
|
)
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
"cat foo.py",
|
|
"head -80 foo.py",
|
|
"tail -n 50 foo.py",
|
|
"sed -n '1,60p' foo.py",
|
|
"sed -ne '1,60p' foo.py",
|
|
"sed -En '1,60p' foo.py",
|
|
"sed --quiet '1,60p' foo.py",
|
|
"sed --silent '1,60p' foo.py",
|
|
"sed --quiet --expression='1,60p' foo.py",
|
|
"timeout 30 sed -ne '1,60p' foo.py",
|
|
"sed -n --follow-symlinks '1,60p' foo.py",
|
|
"sed -a -n '1,60p' foo.py", # BSD sed.
|
|
"sed -n -e '1,60p' foo.py -l", # BSD line buffering (no argument).
|
|
"sed -l -n '1,60p' foo.py",
|
|
"sed -ln '1,60p' foo.py",
|
|
"sed -l 80 -n '1,60p' foo.py", # GNU line length takes an argument.
|
|
"sed -nl80 '1,60p' foo.py",
|
|
"cd /repo && cat foo.py",
|
|
"sudo cat foo.py",
|
|
"rtk cat foo.py",
|
|
'bash -lc "cat foo.py"',
|
|
# The `-c` argument is not the first token after the shell.
|
|
'bash --norc -lc "cat foo.py"',
|
|
],
|
|
)
|
|
def test_simple_reads_are_detected(command: str) -> None:
|
|
assert _is_read_command(command) is True
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
# The reported case: a read batched behind an unrelated first program.
|
|
"wc -l a.py b.py && sed -n '1,60p' c.py",
|
|
"grep -n X foo.py | head -40; head -80 bar.py",
|
|
"ls -la || cat foo.py",
|
|
"echo start; cat foo.py",
|
|
# Bare `sed` is a stream edit, but the `cat` segment is still a read.
|
|
"sed 's/a/b/' foo.py && cat -n bar.py",
|
|
# A read in the first segment survives a non-read tail.
|
|
"cat foo.py && pytest -q",
|
|
# Trailing and doubled separators produce empty segments, which are skipped.
|
|
"cat foo.py;",
|
|
"ls -la; ; cat foo.py",
|
|
# A write in a SIBLING segment does not unprotect the read segment: the
|
|
# output still carries the exact bytes of foo.py.
|
|
"cat foo.py && echo done > marker",
|
|
"echo done > marker && cat foo.py",
|
|
"sed -n '1,60p' foo.py; touch out >> log",
|
|
],
|
|
)
|
|
def test_chained_reads_are_detected(command: str) -> None:
|
|
assert _is_read_command(command) is True
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"command",
|
|
[
|
|
# Derived output — stays compressible.
|
|
"grep -n x foo.py",
|
|
"pytest -q",
|
|
"ls -la",
|
|
# A pipeline stage consumes the previous stage's output, not a file:
|
|
# `head` here truncates grep output, it does not read bar.py.
|
|
"grep -n x foo.py | head -40",
|
|
# Writes are never reads.
|
|
"cat > foo.py",
|
|
# The write lives in the read segment itself — a pipeline is one segment,
|
|
# so `tee` is still visible after the split.
|
|
"cat foo.py | tee bar.py",
|
|
"cat foo.py > copy.py && echo done",
|
|
"sed -n '1,60p' foo.py >> out.txt",
|
|
"sed 's/a/b/' foo.py",
|
|
"sed -n1,60p foo.py", # Invalid: a script cannot be attached to -n.
|
|
"sed -ni '1,60p' foo.py",
|
|
"sed -n -i '1,60p' foo.py",
|
|
"sed -n -I .bak '1,60p' foo.py", # BSD in-place editing.
|
|
"sed -n -I '' '1,60p' foo.py",
|
|
"sed -n -l -i '' '1,60p' foo.py", # BSD -l must not hide in-place flags.
|
|
"sed -n -l -I .bak '1,60p' foo.py",
|
|
"sed -in '1,60p' foo.py",
|
|
"sed --quiet --in-place '1,60p' foo.py",
|
|
"sed --quiet --in-place=.bak '1,60p' foo.py",
|
|
"sed -e 's/a/ -n /' foo.py", # Script text is not a quiet option.
|
|
"sed -e ' -n ' foo.py",
|
|
"sed -- -n foo.py", # Option parsing ends at --.
|
|
# Lockfiles are regenerated, never byte-patched.
|
|
"cat uv.lock",
|
|
# `-n` from a sibling segment must not qualify a bare `sed`.
|
|
"sed 's/a/b/' foo.py && ls -n",
|
|
# A segment that is only an env assignment resolves to no program.
|
|
"FOO=1",
|
|
# A shell invocation with no `-c` argument runs a script, not a read.
|
|
"bash --norc",
|
|
"bash script.sh",
|
|
],
|
|
)
|
|
def test_non_reads_are_not_detected(command: str) -> None:
|
|
assert _is_read_command(command) is False
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("command", "expected"),
|
|
[
|
|
# Shell operators inside a quoted sed script are part of the argument (#4478).
|
|
("sed -n '/x >/p' foo.py", True),
|
|
("sed -n '/x >> y/p' foo.py", True),
|
|
('sed -n "/a > b/p" foo.py', True),
|
|
("sed -n '/a|b/p' foo.py", True),
|
|
("sed -n '/a;b/p' foo.py", True),
|
|
("sed -n '/a && b/p' foo.py", True),
|
|
("sed -n '/ tee /p' foo.py", True),
|
|
("cat 'a > b.py'", True),
|
|
# An escaped operator is a literal argument too.
|
|
(r"cat foo.py \> out", True),
|
|
# Real operators next to a quoted script still count.
|
|
("sed -n '/x >/p' foo.py > out.txt", False),
|
|
("sed -n '/a|b/p' foo.py | tee out.txt", False),
|
|
("sed -n '/a;b/p' foo.py; echo done", True),
|
|
("echo '>' && sed 's/a/b/' foo.py", False),
|
|
# The -c argument of a shell is quoted at the outer level; the inner
|
|
# command is checked on its own.
|
|
('bash -lc "cat foo.py > copy.py"', False),
|
|
('bash -lc "cat foo.py | tee copy.py"', False),
|
|
('bash -lc "cat foo.py"', True),
|
|
# A backslash-newline is a line continuation: the write on the next line
|
|
# still counts (review of #4646).
|
|
("cat foo.py \\\n> copy.py", False),
|
|
("cat foo.py \\\n>> copy.py", False),
|
|
("cat foo.py | \\\ntee copy.py", False),
|
|
("sed -n '/x >/p' foo.py \\\n> out.txt", False),
|
|
("sed -n '1,60p' \\\nfoo.py", True),
|
|
# A comment ends at the newline; its quote marks do not open a quote that
|
|
# would hide the next line's separators (review of #4646).
|
|
('echo start # " unclosed in comment\nls; cat foo.py', True),
|
|
("echo ok # author's note\ntrue; cat foo.py", True),
|
|
("cat foo.py # > not a redirect", True),
|
|
# A hash inside a word is not a comment.
|
|
("cat foo#bar.py > out.txt", False),
|
|
],
|
|
)
|
|
def test_quoted_shell_operators_are_arguments(command: str, expected: bool) -> None:
|
|
assert _is_read_command(command) is expected
|
|
|
|
|
|
def test_heredoc_body_is_not_split_into_a_bogus_read() -> None:
|
|
"""A `;` inside a heredoc body must not produce a segment that looks like a
|
|
read — the command as a whole writes a file."""
|
|
command = "cat > foo.py <<EOF\nprint(1); head -5 bar.py\nEOF"
|
|
assert _is_read_command(command) is False
|
|
|
|
|
|
@pytest.mark.parametrize("flag", ["-n", "-ne", "-En", "--quiet", "--silent"])
|
|
@pytest.mark.parametrize("comment", ["", " # avoid -i here", " # --unknown", " # author's note"])
|
|
def test_quiet_sed_output_reaches_read_protection(flag: str, comment: str, monkeypatch) -> None:
|
|
monkeypatch.setenv("HEADROOM_PROTECT_READS", "1")
|
|
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "2000")
|
|
# Sixty source lines exceed the small-shell shortcut and reach read protection.
|
|
content = "".join(
|
|
f"def compute_{i}(value):\n"
|
|
" return value * 2 # Preserve this source exactly for a later patch.\n"
|
|
for i in range(30)
|
|
)
|
|
assert len(content) >= 2000
|
|
messages = [
|
|
{"role": "user", "content": "Read the module."},
|
|
{
|
|
"role": "assistant",
|
|
"content": None,
|
|
"tool_calls": [
|
|
{
|
|
"id": "read-1",
|
|
"type": "function",
|
|
"function": {
|
|
"name": "Bash",
|
|
"arguments": json.dumps({"command": f"sed {flag} '1,60p' foo.py{comment}"}),
|
|
},
|
|
}
|
|
],
|
|
},
|
|
{"role": "tool", "tool_call_id": "read-1", "content": content},
|
|
]
|
|
router = ContentRouter(
|
|
ContentRouterConfig(
|
|
enable_code_aware=False,
|
|
enable_kompress=False,
|
|
protect_analysis_context=False,
|
|
protect_recent_reads_fraction=0.0,
|
|
)
|
|
)
|
|
result = router.apply(messages, EstimatingTokenCounter(), protect_recent=0)
|
|
assert "router:shell_small_result_verbatim" not in result.transforms_applied
|
|
assert "router:read_protected" in result.transforms_applied
|
|
assert result.messages[2]["content"] == content
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("command", "expected"),
|
|
[
|
|
("sed -n '1,60p' foo.py # avoid -i here", True),
|
|
("sed -n '1,60p' foo.py # --unknown", True),
|
|
("sed -n '1,60p' foo.py # author's note", True),
|
|
("sed -n '1,60p' foo.py # unclosed \"quote", True),
|
|
("sed -n '1,60p' 'foo#bar.py'", True),
|
|
('sed -n "1,60p" "foo#bar.py"', True),
|
|
("sed -n '1,60p' foo#bar.py", True),
|
|
(r"sed -n '1,60p' foo\#bar.py", True),
|
|
("sed -n '/# marker/p' foo.py", True),
|
|
# Literal hashes must not hide real in-place options that follow them.
|
|
("sed -n '1,60p' foo#bar.py -i", False),
|
|
("sed -n '1,60p' 'foo#bar.py' -i", False),
|
|
(r"sed -n '1,60p' foo\#bar.py -i", False),
|
|
(r"sed -n '1,60p' foo\ #bar.py -i", False),
|
|
("sed -n '/# marker/p' foo.py -i", False),
|
|
("sed -n '1,60p' foo.py -i # --quiet", False),
|
|
("sed -n 'unterminated # quote", False),
|
|
],
|
|
)
|
|
def test_sed_shell_comments_and_literal_hashes(command: str, expected: bool) -> None:
|
|
assert _is_read_command(command) is expected
|