Commit Graph
1 Commits
Author SHA1 Message Date
1fb4cbaea0 fix(proxy): astgrep disk verify truncation (#3192)
## Description

`_detect_truncation()` currently relies on client-specific truncation
banners in `tool_result` text. This works for clients that emit a
recognizable line-count banner, but not for clients such as Cursor
(byte-based, direction-reversed truncation) and OpenCode (placeholder +
external file pointer).

Because `headroom proxy` runs alongside the client it wraps, it can
optionally verify the returned content against the actual file on disk.
If the `tool_result` is a strict prefix of the file, we can identify
truncation without depending on client-specific output formatting.

This adds disk verification as an opt-in fallback: the existing
banner-based detection always runs first, and disk verification is only
attempted when no banner is found. All disk-verification failures or
ambiguous cases return `UNKNOWN` rather than guessing.

Following review, the verifier is also constrained to a trusted,
confined workspace root rather than trusting caller-supplied input
directly — see the last four bullets under Changes Made.

**Current scope:** the session-token workspace binding this fallback
requires is wired for `headroom wrap claude` and plugin-loaded `headroom
wrap opencode` only. Cursor remains unsupported/not tested — it has no
header-injection surface today to transmit an authoritative session
token, so disk verification stays `UNKNOWN` for it; extending Cursor
support needs a separate, safe transport design.

Related to #2430
Closes #2421

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made
- Added optional disk-based Read truncation verification behind
`HEADROOM_VERIFY_TRUNCATION_ON_DISK`.
- Disk verification compares the visible `tool_result` against the file
on disk and distinguishes `COMPLETE`, `TRUNCATED`, and `UNKNOWN`.
- Disk verification only runs when the existing `_detect_truncation()`
banner check finds no truncation marker, avoiding unnecessary file I/O
when the client already provides a signal.
- Added a `current_cwd` contextvar populated from `x-headroom-cwd` so
relative Read paths can be resolved against the client's working
directory.
- Registered `HEADROOM_VERIFY_TRUNCATION_ON_DISK` as a hot-reloadable
runtime setting.
- Gated verification on a new `is_current_request_trusted` contextvar,
set from `is_loopback_host(request.client.host)` — a server-observed
fact, not the caller-supplied header — so `x-headroom-cwd` is never
sufficient authority on its own.
- Confined all path resolution strictly under the canonicalized `cwd`
(`.resolve()` + `relative_to()`), rejecting `..` traversal,
absolute-path escapes, and post-resolution symlink escapes.
- Required the resolved target to be a regular file and under a byte cap
(new `HEADROOM_VERIFY_TRUNCATION_MAX_BYTES` knob, default 5MB), both
checked via `fstat` on the open fd before any read.
- Opened with `O_NONBLOCK` so a non-regular/special file can't park the
read indefinitely; disk I/O already runs off the event loop via the
existing compression thread pool.
- Added unit and integration tests covering successful verification,
missing/unreadable paths, relative paths, content races, feature gating,
banner precedence, and the adversarial cases below.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_tool_result_interceptors.py -v
51 passed in 2.21s
(includes TestVerifyReadAgainstDisk::* [14 cases, incl. absolute-path escape, `..`
traversal, symlink escape, sibling-prefix boundary, non-regular file, oversized
file, and untrusted-request-touches-no-disk], plus the end-to-end disk-verification
and banner-precedence cases)

$ pytest tests/test_proxy_loopback_gating.py -k trusted
16 passed
(includes test_record_headroom_stack_binds_trusted_true_for_loopback_client and
..._false_for_non_loopback_client — proves a spoofed x-headroom-cwd from a
non-loopback peer does not bind trust)

$ ruff check headroom/proxy/interceptors/astgrep.py headroom/proxy/project_context.py \
    headroom/proxy/server.py headroom/proxy/runtime_env.py \
    tests/test_tool_result_interceptors.py tests/test_proxy_loopback_gating.py
All checks passed!

$ mypy headroom/proxy/interceptors/astgrep.py headroom/proxy/project_context.py \
    headroom/proxy/server.py headroom/proxy/runtime_env.py
Success: no issues found in 4 source files
```

## Real Behavior Proof

- Environment: Mac OS local dev machine, no live proxy/network involved.
Exercised the real interceptor pipeline (`apply_to_messages`) directly
against an actual file on disk, using the same fixture and cwd-binding
path a live proxy request would use.
- Exact command / steps: wrote a 9-function Python fixture to a temp
file, built a client-truncated `tool_result` (first 8 functions, cut
mid-file, **no truncation banner text anywhere**), bound
`x-headroom-cwd` to the temp directory via `set_current_cwd()`, then
called `apply_to_messages()` once with
`HEADROOM_VERIFY_TRUNCATION_ON_DISK` unset and once with it set to `1`.
- Observed result: With disk verification disabled, the existing
ast-grep interceptor produced its normal header without indicating that
the input was truncated. With HEADROOM_VERIFY_TRUNCATION_ON_DISK=1, disk
verification detected that the 50-line tool_result was a strict prefix
of the 59-line file and produced:
 ```text
=== HEADROOM_VERIFY_TRUNCATION_ON_DISK unset (default) ===
[headroom: outlined by ast-grep — 8 definition(s); bodies elided.
Re-read the file with a line range to see a specific body.]

=== HEADROOM_VERIFY_TRUNCATION_ON_DISK=1, x-headroom-cwd bound to the
real directory ===
[headroom: outlined by ast-grep — 8 definition(s) in the visible
portion; input was truncated upstream (showing through line 50 of 59
total). Bodies elided. Re-read remaining lines to see more.]
```

**Trust-gate behavior:** same fixture and disk layout, `HEADROOM_VERIFY_TRUNCATION_ON_DISK=1`, but the request context's trust flag left at its default (`False`, simulating a non-loopback caller) instead of being marked trusted.

```text
=== trusted context (loopback) ===
[headroom: outlined by ast-grep — 8 definition(s) in the visible
portion; input was truncated upstream ...]

=== untrusted context (non-loopback) — same file, same cwd header, same
tool_result ===
[headroom: outlined by ast-grep — 8 definition(s); bodies elided.
Re-read the file with a line range to see a specific body.]
(no disk read attempted — verified via a call-recording wrapper around
os.open asserting zero invocations)
```
Also verified at the HTTP-middleware level with `TestClient(..., client=("100.90.0.5", 12345))`: a request carrying a spoofed `x-headroom-cwd` header from a non-loopback client IP does not bind trust, regardless of header content.

- Not tested: a live `headroom proxy` session with a real coding-agent client; the WebSocket/Codex responses-WS path.

## Runtime Rollout Safety

- Rollout-managed feature(s): None. The feature is gated by `HEADROOM_VERIFY_TRUNCATION_ON_DISK`
- Minimum rollout channel: N/A. `--intercept-tool-results` must already be enabled for the ast-grep interceptor to run.
- Stable/default behavior changed: no
- Kill switch / disable path: Unset `HEADROOM_VERIFY_TRUNCATION_ON_DISK`; the setting is hot-reloadable and does not require a restart.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the commit, or unset the env var on a running proxy (takes effect immediately, no restart).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)

## Additional Notes

this branch is based on `fix/astgrep-truncation-marker` #2430 not main


<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3192" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-08 10:52:09 -05:00