Files
headroom/tests/test_proxy_workspace_registration_middleware.py
1fb4cbaea0 fix(proxy): astgrep disk verify truncation (#3192)
## Description

`_detect_truncation()` currently relies on client-specific truncation
banners in `tool_result` text. This works for clients that emit a
recognizable line-count banner, but not for clients such as Cursor
(byte-based, direction-reversed truncation) and OpenCode (placeholder +
external file pointer).

Because `headroom proxy` runs alongside the client it wraps, it can
optionally verify the returned content against the actual file on disk.
If the `tool_result` is a strict prefix of the file, we can identify
truncation without depending on client-specific output formatting.

This adds disk verification as an opt-in fallback: the existing
banner-based detection always runs first, and disk verification is only
attempted when no banner is found. All disk-verification failures or
ambiguous cases return `UNKNOWN` rather than guessing.

Following review, the verifier is also constrained to a trusted,
confined workspace root rather than trusting caller-supplied input
directly — see the last four bullets under Changes Made.

**Current scope:** the session-token workspace binding this fallback
requires is wired for `headroom wrap claude` and plugin-loaded `headroom
wrap opencode` only. Cursor remains unsupported/not tested — it has no
header-injection surface today to transmit an authoritative session
token, so disk verification stays `UNKNOWN` for it; extending Cursor
support needs a separate, safe transport design.

Related to #2430
Closes #2421

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made
- Added optional disk-based Read truncation verification behind
`HEADROOM_VERIFY_TRUNCATION_ON_DISK`.
- Disk verification compares the visible `tool_result` against the file
on disk and distinguishes `COMPLETE`, `TRUNCATED`, and `UNKNOWN`.
- Disk verification only runs when the existing `_detect_truncation()`
banner check finds no truncation marker, avoiding unnecessary file I/O
when the client already provides a signal.
- Added a `current_cwd` contextvar populated from `x-headroom-cwd` so
relative Read paths can be resolved against the client's working
directory.
- Registered `HEADROOM_VERIFY_TRUNCATION_ON_DISK` as a hot-reloadable
runtime setting.
- Gated verification on a new `is_current_request_trusted` contextvar,
set from `is_loopback_host(request.client.host)` — a server-observed
fact, not the caller-supplied header — so `x-headroom-cwd` is never
sufficient authority on its own.
- Confined all path resolution strictly under the canonicalized `cwd`
(`.resolve()` + `relative_to()`), rejecting `..` traversal,
absolute-path escapes, and post-resolution symlink escapes.
- Required the resolved target to be a regular file and under a byte cap
(new `HEADROOM_VERIFY_TRUNCATION_MAX_BYTES` knob, default 5MB), both
checked via `fstat` on the open fd before any read.
- Opened with `O_NONBLOCK` so a non-regular/special file can't park the
read indefinitely; disk I/O already runs off the event loop via the
existing compression thread pool.
- Added unit and integration tests covering successful verification,
missing/unreadable paths, relative paths, content races, feature gating,
banner precedence, and the adversarial cases below.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_tool_result_interceptors.py -v
51 passed in 2.21s
(includes TestVerifyReadAgainstDisk::* [14 cases, incl. absolute-path escape, `..`
traversal, symlink escape, sibling-prefix boundary, non-regular file, oversized
file, and untrusted-request-touches-no-disk], plus the end-to-end disk-verification
and banner-precedence cases)

$ pytest tests/test_proxy_loopback_gating.py -k trusted
16 passed
(includes test_record_headroom_stack_binds_trusted_true_for_loopback_client and
..._false_for_non_loopback_client — proves a spoofed x-headroom-cwd from a
non-loopback peer does not bind trust)

$ ruff check headroom/proxy/interceptors/astgrep.py headroom/proxy/project_context.py \
    headroom/proxy/server.py headroom/proxy/runtime_env.py \
    tests/test_tool_result_interceptors.py tests/test_proxy_loopback_gating.py
All checks passed!

$ mypy headroom/proxy/interceptors/astgrep.py headroom/proxy/project_context.py \
    headroom/proxy/server.py headroom/proxy/runtime_env.py
Success: no issues found in 4 source files
```

## Real Behavior Proof

- Environment: Mac OS local dev machine, no live proxy/network involved.
Exercised the real interceptor pipeline (`apply_to_messages`) directly
against an actual file on disk, using the same fixture and cwd-binding
path a live proxy request would use.
- Exact command / steps: wrote a 9-function Python fixture to a temp
file, built a client-truncated `tool_result` (first 8 functions, cut
mid-file, **no truncation banner text anywhere**), bound
`x-headroom-cwd` to the temp directory via `set_current_cwd()`, then
called `apply_to_messages()` once with
`HEADROOM_VERIFY_TRUNCATION_ON_DISK` unset and once with it set to `1`.
- Observed result: With disk verification disabled, the existing
ast-grep interceptor produced its normal header without indicating that
the input was truncated. With HEADROOM_VERIFY_TRUNCATION_ON_DISK=1, disk
verification detected that the 50-line tool_result was a strict prefix
of the 59-line file and produced:
 ```text
=== HEADROOM_VERIFY_TRUNCATION_ON_DISK unset (default) ===
[headroom: outlined by ast-grep — 8 definition(s); bodies elided.
Re-read the file with a line range to see a specific body.]

=== HEADROOM_VERIFY_TRUNCATION_ON_DISK=1, x-headroom-cwd bound to the
real directory ===
[headroom: outlined by ast-grep — 8 definition(s) in the visible
portion; input was truncated upstream (showing through line 50 of 59
total). Bodies elided. Re-read remaining lines to see more.]
```

**Trust-gate behavior:** same fixture and disk layout, `HEADROOM_VERIFY_TRUNCATION_ON_DISK=1`, but the request context's trust flag left at its default (`False`, simulating a non-loopback caller) instead of being marked trusted.

```text
=== trusted context (loopback) ===
[headroom: outlined by ast-grep — 8 definition(s) in the visible
portion; input was truncated upstream ...]

=== untrusted context (non-loopback) — same file, same cwd header, same
tool_result ===
[headroom: outlined by ast-grep — 8 definition(s); bodies elided.
Re-read the file with a line range to see a specific body.]
(no disk read attempted — verified via a call-recording wrapper around
os.open asserting zero invocations)
```
Also verified at the HTTP-middleware level with `TestClient(..., client=("100.90.0.5", 12345))`: a request carrying a spoofed `x-headroom-cwd` header from a non-loopback client IP does not bind trust, regardless of header content.

- Not tested: a live `headroom proxy` session with a real coding-agent client; the WebSocket/Codex responses-WS path.

## Runtime Rollout Safety

- Rollout-managed feature(s): None. The feature is gated by `HEADROOM_VERIFY_TRUNCATION_ON_DISK`
- Minimum rollout channel: N/A. `--intercept-tool-results` must already be enabled for the ast-grep interceptor to run.
- Stable/default behavior changed: no
- Kill switch / disable path: Unset `HEADROOM_VERIFY_TRUNCATION_ON_DISK`; the setting is hot-reloadable and does not require a restart.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the commit, or unset the env var on a running proxy (takes effect immediately, no restart).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)

## Additional Notes

this branch is based on `fix/astgrep-truncation-marker` #2430 not main


<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3192" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-08 10:52:09 -05:00

143 lines
5.3 KiB
Python

"""End-to-end test of workspace-root registration through the real HTTP
middleware, including isolation between concurrent sessions on one proxy."""
from __future__ import annotations
import json
import os
from fastapi import FastAPI
from fastapi.testclient import TestClient
from headroom import paths
from headroom.proxy.server import ProxyConfig, create_app
_TEST_PORT = 18787
def _make_app() -> FastAPI:
return create_app(
ProxyConfig(
optimize=False,
cache_enabled=False,
rate_limit_enabled=False,
cost_tracking_enabled=False,
log_requests=False,
ccr_inject_tool=False,
ccr_handle_responses=False,
ccr_context_tracking=False,
image_optimize=False,
port=_TEST_PORT,
)
)
def _app_with_registered_cwd_probe() -> FastAPI:
"""Test-only route reading get_registered_cwd() after the middleware runs."""
from headroom.proxy.project_context import get_registered_cwd
app = _make_app()
@app.get("/__test/registered-cwd")
def _probe() -> dict[str, str | None]:
return {"registered_cwd": get_registered_cwd()}
# A catch-all passthrough route registered by create_app() would
# otherwise shadow this path -- Starlette matches routes in
# registration order, not by specificity.
app.router.routes.insert(0, app.router.routes.pop())
return app
def _write_marker(pid: int, session_token: str, cwd: str) -> None:
d = paths.proxy_clients_dir(_TEST_PORT)
d.mkdir(parents=True, exist_ok=True)
(d / f"{pid}.json").write_text(
json.dumps({"pid": pid, "session_token": session_token, "cwd": cwd}),
encoding="utf-8",
)
def test_valid_session_token_resolves_registered_cwd(tmp_path, monkeypatch):
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path))
project_dir = tmp_path / "project"
project_dir.mkdir()
_write_marker(os.getpid(), "tok-real", str(project_dir))
client = TestClient(_app_with_registered_cwd_probe())
resp = client.get("/__test/registered-cwd", headers={"x-headroom-session-token": "tok-real"})
assert resp.json() == {"registered_cwd": str(project_dir)}
def test_opencode_registration_marker_resolves_through_real_middleware(tmp_path, monkeypatch):
"""Closes the seam the reviewer flagged: proves there's no schema drift
between what opencode()'s wrap command actually writes
(wrap._register_proxy_client -- the same function claude() calls, not
a hand-rolled test marker) and what resolve_registered_cwd() reads,
using the lowercase header casing the OpenCode transport plugin sends
(vs. claude's title-case) over the real HTTP middleware."""
from headroom.cli import wrap as wrap_mod
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path))
project_dir = tmp_path / "opencode-project"
project_dir.mkdir()
monkeypatch.chdir(project_dir)
wrap_mod._register_proxy_client(_TEST_PORT, session_token="tok-opencode")
client = TestClient(_app_with_registered_cwd_probe())
resp = client.get(
"/__test/registered-cwd", headers={"x-headroom-session-token": "tok-opencode"}
)
assert resp.json() == {"registered_cwd": str(project_dir)}
def test_missing_or_wrong_token_resolves_to_none(tmp_path, monkeypatch):
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path))
project_dir = tmp_path / "project"
project_dir.mkdir()
_write_marker(os.getpid(), "tok-real", str(project_dir))
client = TestClient(_app_with_registered_cwd_probe())
no_token = client.get("/__test/registered-cwd")
assert no_token.json() == {"registered_cwd": None}
wrong_token = client.get(
"/__test/registered-cwd", headers={"x-headroom-session-token": "tok-guessed"}
)
assert wrong_token.json() == {"registered_cwd": None}
def test_spoofed_cwd_header_alone_never_resolves_anything(tmp_path, monkeypatch):
"""A spoofed x-headroom-cwd with no matching registration must never
surface as the registered root."""
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path))
client = TestClient(_app_with_registered_cwd_probe())
resp = client.get("/__test/registered-cwd", headers={"x-headroom-cwd": "/etc"})
assert resp.json() == {"registered_cwd": None}
def test_two_concurrent_sessions_on_one_shared_proxy_stay_isolated(tmp_path, monkeypatch):
"""Token A must resolve only project A's cwd, never project B's."""
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path))
project_a = tmp_path / "project-a"
project_b = tmp_path / "project-b"
project_a.mkdir()
project_b.mkdir()
import subprocess
import sys
child = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(30)"])
try:
_write_marker(os.getpid(), "tok-a", str(project_a))
_write_marker(child.pid, "tok-b", str(project_b))
client = TestClient(_app_with_registered_cwd_probe())
resp_a = client.get("/__test/registered-cwd", headers={"x-headroom-session-token": "tok-a"})
resp_b = client.get("/__test/registered-cwd", headers={"x-headroom-session-token": "tok-b"})
assert resp_a.json() == {"registered_cwd": str(project_a)}
assert resp_b.json() == {"registered_cwd": str(project_b)}
finally:
child.kill()
child.wait()