fix(router): keep small shell results verbatim (#3590) (#4426)

## Description

Fixes #3590. Small shell tool results (default: shorter than 2000
characters) now bypass lossy compression. Byte-lossless folds such as
search grouping may still apply. File reads and searches were already
protected; shell output is deliberately compressible, but a short `ls
-la`, `git status`, or JSON dump is not worth the accuracy loss
described in #3590.

The real proxy-level before/after evidence is in Real Behavior Proof. A
live Claude Code session is still untested.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added `HEADROOM_SHELL_RESULT_MIN_CHARS` (default `2000`) and
`_is_small_shell_result()`, applied to OpenAI `role:"tool"`, legacy
`role:"function"`, top-level text-block tool messages, and Anthropic
`tool_result` string/list-form paths after the lossless search fold and
before lossy compression.
- Normalized `ContentRouterConfig.bash_tool_names` once and extended it
with the shell aliases already in `headroom.learn._shared` (`shell`,
`local_shell`, `run_shell_command`, `execute_command`, `exec_command`,
`terminal`, `run_command`, `run_terminal_command`).
- Measure a top-level text-block shell result using the combined text
length, so a large result cannot bypass compression merely by splitting
into small blocks.
- Added boundary coverage for both tool/function block shapes, positive
threshold overrides and custom case-insensitive shell names; documented
the combined-length contract in `docs/content/docs/proxy.mdx`.
- Updated the existing proxy-level Bash regression to assert both sides
of the 2000-character floor, fixing the hosted shard-4 failure. Added
kill-switch, malformed/empty-value and diagnostic coverage.
- Carried dependency security remediation requested in review: certifi
and the OpenTelemetry family, three patched Rust crates, and patched npm
locks for docs, SDK, OpenCode and E2E tooling. These use existing
packages, explicit security floors and targeted dependency overrides;
libc platform constraints remain intact.

## Testing

- [x] Hosted CI passes on the current head, including all four Python
test shards
- [x] Changed Python tests passed Ruff in the preceding regression
revision; no Python runtime source changed in this dependency follow-up
- [ ] Full local Python type check passes (Windows/test-double baseline
limitations)
- [x] Regression and dependency-interface coverage added
- [x] Real proxy proof and isolated CLI smoke evidence recorded

### Test Output

Latest follow-up: Windows, Python 3.11 / Node 22.20.0 for the OpenCode
suite, and portable official Node 24.22.1 for E2E CLI qualification. The
portable Node archive's SHA-256 matched nodejs.org's published checksum.

```text
Python shell-floor/web-tool/router/alias/tracing + OpenClaw wrapper/bridge selection:
119 passed, 2 warnings in 10.37s

uv lock --check
Resolved 262 packages

uv tree --locked --python-version 3.10 --package filelock
filelock v4.1.0
uv tree --locked --python-version 3.12 --package filelock
filelock v4.2.4

FileLock 4.2.4 process smoke:
contention times out, release permits next process

cd plugins/opencode
npm test
Test Files  4 passed (4)
Tests       60 passed (60)
npm run typecheck
> node node_modules/@typescript/native/bin/tsc --noEmit
npm run build
DTS Build success
npm audit --package-lock-only --audit-level=low --registry=https://registry.npmjs.org
found 0 vulnerabilities

After merging current upstream main (including #4453):

cd e2e/wrap/tools
node node_modules/openclaw/openclaw.mjs --version
OpenClaw 2026.10.6
npm audit --package-lock-only --audit-level=low --registry=https://registry.npmjs.org
2 vulnerabilities (1 moderate, 1 high)
# limited to npm's bundled copies:
# GHSA-rxp7-6hxx-x8q5 — node_modules/npm/node_modules/ip-address 10.7.2
# GHSA-4qv8-2p9j-2685 and GHSA-xjj6-f33q-929c — node_modules/npm/node_modules/undici 6.29.0
# exact scoped exceptions expire 2026-11-15; no OpenClaw path is excepted

production-lock pip-audit:
No known vulnerabilities found
```

- The Python native extension remains the existing 0.41.0 Windows
wheel's `_core.pyd`; all Python source under test comes from this PR
checkout. No native rebuild or Torch/model inference qualification is
claimed.
- The 3 new OpenCode compatibility tests exercise the actual upstream
`tool.schema` interface and Effect `Primitive.fileParse` TOML
success/error channels, mocking only filesystem I/O. They accompany the
explicit Zod/TOML overrides.
- E2E CLI checks used isolated `OPENCLAW_STATE_DIR` /
`OPENCLAW_CONFIG_PATH`, with no model request or modification of the
user's normal OpenClaw configuration. E2E dependencies were installed
with lifecycle scripts disabled. The full Docker/model E2E suite was not
run locally.
- Earlier unchanged docs/SDK lock remediation passed: SDK **351 passed,
33 skipped** plus typecheck/build; docs typecheck/build with 185 static
pages; both npm audits were zero. The previous head also passed the
hosted Rust security audit.
- Full local Python suite was not rerun. The earlier remote-timeout
float-boundary failure remains outside this change.

## Real Behavior Proof

- Environment: Windows 10 x64, Python 3.11.0, Headroom source `main` at
`e995eb9` (0.41.1-dev), native `_core` from the 0.41.0 wheel because no
Rust toolchain is available locally. `HEADROOM_DISABLE_KOMPRESS=1` keeps
the strategy deterministic; the guard sits before strategy selection, so
the same path applies when Kompress is enabled.
- Exact command / steps: local harness (not committed) `python
proof-3590/run_proof.py` starts the source-tree proxy in `--mode token`
with `HEADROOM_MIN_TOKENS=0` and a local OpenAI-compatible upstream, and
sends a chat request containing a Bash `ps aux` tool result (1719 chars)
followed by later turns. The upstream records the transformed request
body.
- Observed result: `HEADROOM_SHELL_RESULT_MIN_CHARS=0` produced
`compressed=1`, `router:tabular:0.85`, 802 -> 697 tokens (saved 105),
and upstream received a 1455-char schema representation. Default `2000`
produced `shell_small_result: 1`, `compressed=0`, 802 -> 802 tokens, and
upstream received the original 1719-char output byte-for-byte.
- Review follow-up proof: `python proof-3590/run_block_review.py` ran
the real proxy against the same local upstream without replacing its
compressor. A 1719-character shell result split into two text blocks
stayed unchanged; a 3438-character result split into two 1719-character
blocks remained compressible. The previous PR revision incorrectly
protected both large-result fragments. This harness is local, not part
of the PR.
- Not tested: a live `headroom wrap claude` session; the ML Kompress
path; Linux/macOS; the full suite (one pre-existing baseline failure).

## Runtime Rollout Safety

- Rollout-managed feature(s): none; ContentRouter default path.
- Minimum rollout channel: default proxy.
- Stable/default behavior changed: yes. Shell results between
`min_chars_for_block_compression` (500) and
`HEADROOM_SHELL_RESULT_MIN_CHARS` (2000) now stay verbatim instead of
taking lossy compression. At ~1500 chars (~375 tokens), this keeps
roughly 325 extra tokens per result for the life of the context.
- Kill switch / disable path: `HEADROOM_SHELL_RESULT_MIN_CHARS=0`.
- Unsafe override required: no.
- Qualification impact: compression ratio may decrease for small shell
outputs; no API or correctness changes.
- Rollback path: revert this commit or set
`HEADROOM_SHELL_RESULT_MIN_CHARS=0`.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — no UI changes.

## Additional Notes

This follows the [dependency-remediation
review](https://github.com/headroomlabs-ai/headroom/pull/4426#pullrequestreview-5636487191).
It does not disable audits, ignore advisories, reduce audit severity, or
use `npm audit fix --force`.

- **Python:** certifi/OpenTelemetry/Rust fixes from the prior revision
remain. This follow-up raises setuptools to **84.1.0** for the Windows
wheel namespace traversal advisory and uses FileLock **4.2.4** on Python
≥3.11. The universal lock remains resolvable with the published Python
≥3.10 contract. **Python 3.10 now selects FileLock 4.1.0, but it remains
below the 4.2.1 security floor** because fixed releases require ≥3.11;
this is not presented as all-interpreter remediation.
- **OpenCode:** pin security overrides **Zod 4.6.7** and **TOML 5.0.1**
without downgrading the plugin or replacing its pinned Effect version.
TOML crosses a major version (4.3.0 → 5.0.1); its Node ≥20 floor and
consumed `parse(content)` API are unchanged. The added tests validate
the exercised schema and Effect parsing contracts, not every possible
TOML consumer. This override needs normal maintainer dependency review.
- **E2E:** the original remediation moved OpenClaw to 2026.10.5; the
upstream sync now brings **OpenClaw 2026.10.6** with **Undici 8.12.3**
and a Node floor of **≥24.22.1 <25 or ≥26.12.1**. The only remaining raw
npm findings are npm's published bundled copies, handled by the exact,
expiring E2E-only exception policy described below.
- Dependency packages and runtime floors were checked against the
upstream registry metadata. Locks use public registries and retain
existing libc platform constraints.

### Security qualification after upstream sync

Merge commit `e09d37c9274c812f26e994d01a623542a4a7ec31` brings this
branch to upstream `main` through `a616232`, including
[#4453](https://github.com/headroomlabs-ai/headroom/pull/4453). That
upstream change establishes the repository's current security policy for
the dependency cases that previously blocked this PR:

- **Production Python lock:** Python ≥3.11 resolves Pillow 13.0.0 and
FileLock 4.2.4. The current lock passes `uv lock --check`; an equivalent
export of the production `all` set passes `pip-audit 2.10.1 --no-deps
--disable-pip` with **no known vulnerabilities**.
- **Published-package resolution:** FastEmbed still caps Pillow below 13
when ordinary pip resolves package metadata. Upstream now disables image
decoding on Pillow below 13 and separately audits the pip-resolved set,
accepting only Pillow findings covered by that tested decoder guard.
This is a bounded runtime mitigation, not a claim that the older Pillow
release has no advisories.
- **E2E npm lock:** OpenClaw 2026.10.6 uses fixed Undici 8.12.3. Raw npm
audit now reports only npm 12.2.1's bundled Undici 6.29.0 and ip-address
10.7.2 copies. Because npm does not allow overrides to replace its
published bundled contents and no newer release is available, upstream
accepts only the three exact advisory/path/version pairs in
`e2e/wrap/tools/npm-audit-exceptions.json`, only for the non-shipping
E2E harness, and only until **2026-11-15**. The workflow fails on any
extra finding, version/path mismatch, stale exception or expiry.
- **Python 3.10:** the lock now selects FileLock 4.1.0, but it remains
below the 4.2.1 security floor because fixed releases require Python
≥3.11. This supported-interpreter limitation is documented by the
upstream policy and is not represented as universal remediation.

The earlier OpenCode standalone-wheel mismatch remains fixed by
`dee73be75cf81bb67a73573861f1b3ed4067a2bf`: typecheck, build, all 60
plugin tests and the standalone build passed locally, both generated
artifacts matched their Python-shipped paths byte-for-byte, and the
shipped bundle imported with the expected `headroom` id and both
`server`/`setup` entry points.

### Current merge status

- Current head `ab93d7917a6660c704069bb87b6c02f9c744cebc` is mergeable;
all router/documentation inline review threads are resolved.
- JerrettDavis approved this head on 2026-10-24, superseding the earlier
security-related change requests.
- All 12 pull-request workflows on this head completed successfully,
including
[CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40122163824)
(all four Python test shards),
[Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40122163777),
wheel builds/import smoke tests, OpenCode, docs, and Docker E2E checks.
The workflows are no longer awaiting approval.
- The PR remains open, awaiting maintainer merge.
- A live Claude Code/model session, the ML Kompress path, a native
rebuild and the full Docker/model E2E suite remain unverified locally;
the hosted results above are separate from the recorded local
qualification.

Implemented and verified with AI assistance.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
ShiYuan Xiong
2026-10-28 23:31:00 -05:00
committed by GitHub
co-authored by JerrettDavis
parent 4d2dd62b7c
commit 4ee616453f
9 changed files with 709 additions and 14 deletions
+1
View File
@@ -320,6 +320,7 @@ Fine-grained control over what gets compressed and how hard. Most users pick a [
| `--mode` / `HEADROOM_MODE` | `cache` | `cache` compresses only the newest delta (prefix-cache safe); `token` maximizes removal. |
| `--target-ratio` / `HEADROOM_TARGET_RATIO` | unset | Keep-ratio for ML text compression; lower = more aggressive (e.g. `0.10`). |
| `HEADROOM_MIN_TOKENS` | `500` | Minimum block size before a tool output is compressed. |
| `HEADROOM_SHELL_RESULT_MIN_CHARS` | `2000` | Shell tool results shorter than this many characters bypass lossy compression. For top-level text blocks, the threshold uses their combined text length. Byte-lossless folds such as search grouping may still apply. Applies to `bash`, `shell`, `local_shell`, and aliases such as `run_shell_command`/`execute_command`; set `0` to restore lossy compression. |
| `HEADROOM_COMPRESS_USER_MESSAGES` | profile | Compress text inside user-role messages. The `coding` profile turns this on; set `0` to keep user text verbatim under it. Anthropic `tool_result` blocks compress either way. No `--compress-user-messages` flag on `headroom proxy`; the direct module entry point (`python -m headroom.proxy.server`) does accept one. |
| `HEADROOM_COMPRESS_SYSTEM_MESSAGES` | unset | Compress system prompts. Off by default to keep the hottest cache prefix stable. |
| `HEADROOM_PROTECT_RECENT` | profile | Never compress the N most recent turns. |
+10 -2
View File
@@ -10,7 +10,8 @@ PATH. `npm ci` does not support global installs, so diagnostics must also use
`npm list --prefix /opt/wrap-tools`.
After editing a CLI version in `package.json`, regenerate the lockfile using
Node 22 and npm 11.16.0 on Linux:
Node 24.22.1+ within the Node 24 line and npm 11.16.0 on Linux (the pinned
OpenClaw version no longer supports Node 22):
```sh
cd e2e/wrap/tools
@@ -34,4 +35,11 @@ to make a build pass.
These are existing test tools, not new product dependencies. Codex is maintained
by OpenAI; OpenClaw and its transitive dependencies retain their upstream
maintainers. Their install surface includes registry downloads, native optional
packages and lifecycle scripts, as before. Top-level CLI versions are unchanged.
packages and lifecycle scripts, as before. OpenClaw 2026.10.6 and its fixed
Undici override follow the upstream #4453 security repair.
The raw audit still reports npm 12.2.1's bundled ip-address and Undici copies.
`npm-audit-exceptions.json` records the upstream-approved E2E-only risk
acceptance, restricted to exact advisory/path/version combinations and expiring
on 2026-11-15. Security CI rejects other findings and expired or stale entries;
these exceptions are not a claim that the contained packages are patched.
+127 -2
View File
@@ -2113,6 +2113,30 @@ def _is_small_mcp_result(tool_name: Any, text: Any) -> bool:
)
def _shell_result_min_chars() -> int:
"""Below this many characters a shell tool result is never lossily compressed."""
try:
return max(0, int(os.environ.get("HEADROOM_SHELL_RESULT_MIN_CHARS", "2000")))
except ValueError:
return 2000
def _is_small_shell_result(tool_name: Any, text: Any, bash_tool_names: frozenset[str]) -> bool:
"""True when ``text`` is a short shell result that must not be lossily compressed.
A 200-token ``ls -la``, ``git status``, or JSON dump is cheap to keep and
expensive to get wrong: lossy compression can leave a numbers-only blob
that still looks structured, so the model treats it as data instead of
retrying the command (issue #3590). Large build/test logs still take the
lossy path. ``HEADROOM_SHELL_RESULT_MIN_CHARS=0`` restores the old
behaviour.
"""
if not isinstance(text, str) or not text:
return False
name = str(tool_name or "").lower()
return name in bash_tool_names and len(text) < _shell_result_min_chars()
@dataclass
class ContentRouterConfig:
"""Configuration for intelligent content routing.
@@ -2317,8 +2341,22 @@ class ContentRouterConfig:
# BUT a read-only *search* run through them (grep/rg/git grep) yields byte-
# losslessly foldable output — folded instead of lossy-compressed. See
# ``_bash_search_fold``. Config so new harness tool names / search programs
# can be added without code changes.
bash_tool_names: frozenset[str] = frozenset({"bash", "shell", "local_shell"})
# can be added without code changes. The aliases mirror
# ``headroom.learn._shared._TOOL_NAME_MAP`` so the same harnesses are
# covered here.
bash_tool_names: frozenset[str] = frozenset(
{
"bash",
"shell",
"local_shell",
"run_shell_command",
"execute_command",
"exec_command",
"terminal",
"run_command",
"run_terminal_command",
}
)
bash_search_commands: frozenset[str] = frozenset(
{"grep", "egrep", "fgrep", "rg", "ripgrep", "ag", "ack"}
)
@@ -2349,6 +2387,12 @@ class ContentRouterConfig:
# Default False; the proxy enables it in token mode.
search_group_by_file: bool = False
def __post_init__(self) -> None:
# Normalize once so the per-message shell check stays a set lookup.
self.bash_tool_names = frozenset(
name.lower() for name in self.bash_tool_names if isinstance(name, str)
)
@dataclass
class _PerRequestRuntimeState:
@@ -6311,6 +6355,7 @@ class ContentRouter(Transform):
"ccr_retrieve": 0,
"user_msg": 0,
"small": 0,
"shell_small_result": 0,
"recent_code": 0,
"analysis_ctx": 0,
"ratio_too_high": 0,
@@ -6600,6 +6645,36 @@ class ContentRouter(Transform):
_diag[i] = "compressed:bash_lossless_search"
continue
# Small shell results stay verbatim (#3590). A short `ls`,
# `git status`, or JSON dump is cheap to keep; lossy
# compression can leave a numbers-only blob that still looks
# structured, so the model treats it as data instead of
# retrying. Large build/test logs still take the lossy path.
if _is_small_shell_result(tool_name, content, self.config.bash_tool_names):
result_slots[i] = message
transforms_applied.append("router:shell_small_result_verbatim")
route_counts["shell_small_result"] = (
route_counts.get("shell_small_result", 0) + 1
)
if collect_diagnostics:
_diag[i] = "protected:shell_small_result"
continue
# Legacy OpenAI `role:"function"` carries the tool name on the
# message itself, not in a tool_call_id. Resolve it directly so the
# small-shell floor also covers this wire shape (#3590 review).
if role == "function":
function_name = message.get("name", "")
if _is_small_shell_result(function_name, content, self.config.bash_tool_names):
result_slots[i] = message
transforms_applied.append("router:shell_small_result_verbatim")
route_counts["shell_small_result"] = (
route_counts.get("shell_small_result", 0) + 1
)
if collect_diagnostics:
_diag[i] = "protected:shell_small_result"
continue
# Read protection (ROLE / SHAPE-AGNOSTIC). An observation produced by
# a file read command (cat/sed/head/…) is passed VERBATIM so the agent
# keeps exact bytes to patch — regardless of how THIS harness labels
@@ -7569,6 +7644,21 @@ class ContentRouter(Transform):
any_compressed = False
role = message.get("role", "")
# Top-level text blocks are fragments of one tool result. Measure the
# whole result so splitting a large log cannot activate the small-result
# floor for every fragment. Sum lengths without copying the payload.
shell_result_chars = (
sum(
len(block["text"])
for block in content_blocks
if isinstance(block, dict)
and block.get("type") == "text"
and isinstance(block.get("text"), str)
)
if role in ("tool", "function")
else 0
)
# Role-based gate for `text` blocks. Tool/function roles are tool
# outputs and compress freely; assistant defaults to skip (cache
# safety) with explicit opt-in; unknown roles default to skip.
@@ -7767,6 +7857,18 @@ class ContentRouter(Transform):
any_compressed = True
continue
# Small shell results stay verbatim (#3590), same contract as
# the string-form path above. Build/test logs above the floor
# still go through lossy compression.
if _is_small_shell_result(tool_name, tool_text, self.config.bash_tool_names):
new_blocks.append(block)
transforms_applied.append("router:shell_small_result_verbatim")
if route_counts is not None:
route_counts["shell_small_result"] = (
route_counts.get("shell_small_result", 0) + 1
)
continue
# Protection: failed tool calls / error outputs stay verbatim
# (issue #847). `is_error` is Anthropic's explicit failure
# flag and suffices alone; the indicator scan catches error
@@ -7864,6 +7966,29 @@ class ContentRouter(Transform):
if route_counts is not None:
route_counts["ccr_retrieve"] = route_counts.get("ccr_retrieve", 0) + 1
continue
# A normalized tool/function message can carry its output
# as top-level text blocks rather than a tool_result
# wrapper. Apply the same small-shell floor here (#3590
# review).
_shell_name = (
(tool_name_map or {}).get(_msg_tool_id, "")
if role == "tool"
else message.get("name", "")
)
if (
_is_small_shell_result(
_shell_name, block.get("text", ""), self.config.bash_tool_names
)
and shell_result_chars < _shell_result_min_chars()
):
new_blocks.append(block)
transforms_applied.append("router:shell_small_result_verbatim")
if route_counts is not None:
route_counts["shell_small_result"] = (
route_counts.get("shell_small_result", 0) + 1
)
continue
text_content = block.get("text", "")
if isinstance(text_content, str) and (
len(text_content) > min_chars or self._has_lossless_fold(text_content)
+2 -2
View File
@@ -47,8 +47,8 @@
},
"overrides": {
"esbuild": "^0.28.1",
"zod": "^4.6.6",
"toml": "^5.0.1"
"zod": "4.6.7",
"toml": "5.0.1"
},
"license": "Apache-2.0"
}
@@ -0,0 +1,62 @@
import { tool } from "@opencode-ai/plugin";
import { Effect, FileSystem, Path } from "effect";
import { Primitive } from "effect/unstable/cli";
import { describe, expect, it } from "vitest";
function parseConfig(content: string): Promise<unknown> {
const fs = FileSystem.makeNoop({
exists: () => Effect.succeed(true),
stat: () => Effect.succeed({ type: "File" } as FileSystem.File.Info),
readFileString: () => Effect.succeed(content),
});
return Effect.runPromise(
Primitive.fileParse({ format: "toml" }).parse("/headroom/config.toml").pipe(
Effect.provideService(FileSystem.FileSystem, fs),
Effect.provide(Path.layer),
),
);
}
describe("patched OpenCode dependency compatibility", () => {
it("builds and validates a tool with the upstream Zod schema interface", () => {
const schema = tool.schema.object({
hash: tool.schema.string().regex(/^[a-f0-9]{24}$/i),
});
const retrieve = tool({
description: "Retrieve an original tool result",
args: schema.shape,
async execute(args) {
return args.hash;
},
});
expect(schema.parse({ hash: "a".repeat(24) })).toEqual({ hash: "a".repeat(24) });
expect(schema.safeParse({ hash: "invalid" }).success).toBe(false);
expect(retrieve.args.hash).toBe(schema.shape.hash);
});
it("parses typed TOML through Effect's actual file parser", async () => {
const content = `
name = "headroom"
enabled = true
ports = [8787, 8788]
[proxy]
host = "127.0.0.1"
ratio = 0.5
[[models]]
name = "coding"
[[models]]
name = "review"
`;
await expect(parseConfig(content)).resolves.toEqual({
name: "headroom",
enabled: true,
ports: [8787, 8788],
proxy: { host: "127.0.0.1", ratio: 0.5 },
models: [{ name: "coding" }, { name: "review" }],
});
});
it("keeps malformed TOML on Effect's parse-error channel", async () => {
await expect(parseConfig("[unclosed")).rejects.toThrow("Failed to parse '.toml'");
});
});
+8 -3
View File
@@ -54,7 +54,7 @@ dependencies = [
"litellm>=1.96.2,<2.0",
"click>=8.3.3", # CLI framework; PYSEC-2026-2132 fix (command injection in click.edit())
"rich>=13.0.0", # Rich terminal output
"opentelemetry-api>=1.24.0", # Safe no-op OTEL API for instrumentation
"opentelemetry-api>=1.45.1", # CVE-2026-114603: bounded, linear-time tracestate parsing
# AST-aware code slicing (CodeCompressor); binary wheel. 0.44.1 is excluded:
# that PyPI release was a compromised supply-chain build shipping an
# info-stealer `sg.exe` (Trojan:Win64/Lazy!MTB) alongside the real binary
@@ -414,6 +414,8 @@ Changelog = "https://github.com/headroomlabs-ai/headroom/blob/main/CHANGELOG.md"
# transitively; the floor pins below ensure uv resolves to patched versions.
[tool.uv]
constraint-dependencies = [
# PYSEC-2026-4320 — remove distrusted root certificates from the CA bundle.
"certifi>=2026.10.19",
# GHSA-5239-wwwm-4pmq (Low) — transitive via rich; fix at 2.20.0
"pygments>=2.20.0",
# GHSA-4xgf-cpjx-pc3j (Medium) — transitive via mcp; fix at 2.14.2
@@ -429,8 +431,11 @@ constraint-dependencies = [
"lxml-html-clean>=0.4.5",
# CVE-2026-5241 (High) — direct optional dep for proxy/ml/voice; fix at 5.5.0
"transformers>=5.5.0",
# PYSEC-2026-3447 — transitive dependency; fix at 83.0.0
"setuptools>=83.0.0",
# GHSA-xmj3-9gf7-h52w — Windows wheel namespace path traversal.
"setuptools>=84.1.0",
# CVE-2026-112940/112913/112907. The patched series requires Python 3.11;
# Python 3.10 retains the legacy locked version; it is not remediated by this marker.
"filelock>=4.2.1; python_version >= '3.11'",
# PYSEC-2026-3545/3546/3547 — transitive HTTP/WebSocket parser fixes;
# CVE-2026-111063/111064, GHSA-2g87-pp6c-29x5; fix at 3.14.4
"aiohttp>=3.14.4",
@@ -0,0 +1,483 @@
"""Small shell results must stay verbatim through the lossy router path.
Issue #3590: ``headroom wrap claude`` with a low ``HEADROOM_MIN_TOKENS``
compressed a short ``ls -la`` result into a numbers-only blob. The output
still looked structured, so the model treated it as data instead of
retrying. File reads and searches are already protected; shell output is
deliberately compressible, but a small result is not worth the accuracy
cost. Large build/test logs must keep going through the lossy path.
"""
from __future__ import annotations
from collections.abc import Callable
import pytest
from headroom.config import TransformResult
from headroom.providers import OpenAIProvider
from headroom.tokenizer import Tokenizer
from headroom.transforms.content_router import (
CompressionStrategy,
ContentRouter,
ContentRouterConfig,
ContentType,
RouterCompressionResult,
RoutingDecision,
_is_small_shell_result,
_shell_result_min_chars,
)
def _tokenizer() -> Tokenizer:
provider = OpenAIProvider()
return Tokenizer(provider.get_token_counter("gpt-4o"), "gpt-4o")
def _lossy_result(original: str) -> RouterCompressionResult:
"""Stand-in for the ML compressor, including its CCR recovery marker."""
return RouterCompressionResult(
compressed="128 448 09:26 1760 09:28 11980 16767 2394 1262\n<<ccr:deadbeef>>",
original=original,
strategy_used=CompressionStrategy.KOMPRESS,
strategy_chain=["kompress"],
routing_log=[
RoutingDecision(
ContentType.PLAIN_TEXT,
CompressionStrategy.KOMPRESS,
original_tokens=400,
compressed_tokens=12,
)
],
)
def _fake_compress(content: str, **kwargs: object) -> RouterCompressionResult:
return _lossy_result(content)
def _conversation(
tool_name: str, tool_output: str, *, list_form: bool = False
) -> list[dict[str, object]]:
"""A conversation old enough that the tool result is compressible."""
tool_content: object = [{"type": "text", "text": tool_output}] if list_form else tool_output
messages: list[dict[str, object]] = [
{"role": "system", "content": "You are a coding agent."},
{"role": "user", "content": "run the command"},
{
"role": "assistant",
"content": [
{
"type": "tool_use",
"id": "tool_1",
"name": tool_name,
"input": {"command": "ls -la"},
}
],
},
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "tool_1",
"content": tool_content,
}
],
},
]
for i in range(10):
messages.append({"role": "assistant", "content": f"step {i}"})
messages.append({"role": "user", "content": f"continue {i}"})
return messages
def _route(tool_name: str, tool_output: str, *, list_form: bool = False) -> TransformResult:
router = ContentRouter(ContentRouterConfig())
router.compress = _fake_compress # type: ignore[assignment]
return router.apply(
_conversation(tool_name, tool_output, list_form=list_form),
_tokenizer(),
target_ratio=0.4,
)
def _openai_conversation(tool_name: str, tool_output: str) -> list[dict[str, object]]:
messages: list[dict[str, object]] = [
{"role": "system", "content": "You are a coding agent."},
{"role": "user", "content": "run the command"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {"name": tool_name, "arguments": "{}"},
}
],
},
{"role": "tool", "tool_call_id": "call_1", "content": tool_output},
]
for i in range(10):
messages.append({"role": "assistant", "content": f"step {i}"})
messages.append({"role": "user", "content": f"continue {i}"})
return messages
def _route_openai(tool_name: str, tool_output: str) -> TransformResult:
router = ContentRouter(ContentRouterConfig())
router.compress = _fake_compress # type: ignore[assignment]
return router.apply(
_openai_conversation(tool_name, tool_output), _tokenizer(), target_ratio=0.4
)
def _tail(messages: list[dict[str, object]]) -> list[dict[str, object]]:
for i in range(10):
messages.append({"role": "assistant", "content": f"step {i}"})
messages.append({"role": "user", "content": f"continue {i}"})
return messages
def _function_conversation(tool_name: str, tool_output: str) -> list[dict[str, object]]:
"""Legacy OpenAI shape: role:function with the tool name on the message."""
messages: list[dict[str, object]] = [
{"role": "system", "content": "You are a coding agent."},
{"role": "user", "content": "run the command"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {"name": tool_name, "arguments": "{}"},
}
],
},
{"role": "function", "name": tool_name, "content": tool_output},
]
return _tail(messages)
def _route_function(tool_name: str, tool_output: str) -> TransformResult:
router = ContentRouter(ContentRouterConfig())
router.compress = _fake_compress # type: ignore[assignment]
return router.apply(
_function_conversation(tool_name, tool_output), _tokenizer(), target_ratio=0.4
)
def _text_block_conversation(tool_name: str, tool_output: str) -> list[dict[str, object]]:
"""OpenAI shape: a tool message whose content is a list of text blocks."""
messages: list[dict[str, object]] = [
{"role": "system", "content": "You are a coding agent."},
{"role": "user", "content": "run the command"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {"name": tool_name, "arguments": "{}"},
}
],
},
{
"role": "tool",
"tool_call_id": "call_1",
"content": [{"type": "text", "text": tool_output}],
},
]
return _tail(messages)
def _route_text_block(tool_name: str, tool_output: str) -> TransformResult:
router = ContentRouter(ContentRouterConfig())
router.compress = _fake_compress # type: ignore[assignment]
return router.apply(
_text_block_conversation(tool_name, tool_output), _tokenizer(), target_ratio=0.4
)
def _ls_output(rows: int = 18) -> str:
files = "\n".join(
f"-rw-r--r-- 1 user staff {1000 + i:>6} Oct 22 09:{i:02d} report_{i:02d}_final_v2.md"
for i in range(rows)
)
return (
"total 128\n"
"drwxr-xr-x 12 user staff 384 Oct 22 09:26 .\n"
"drwxr-xr-x 5 user staff 160 Oct 22 09:25 ..\n"
f"{files}\n"
)
def _padded_output(length: int) -> str:
"""A shell-like payload of exactly ``length`` characters."""
base = "line: some structured shell output\n"
return (base * (length // len(base) + 1))[:length]
def test_small_bash_tool_result_stays_verbatim() -> None:
"""A short shell result keeps its structure instead of becoming numbers."""
output = _ls_output()
assert len(output) < 2000
result = _route("Bash", output)
assert result.messages[3]["content"][0]["content"] == output
assert "router:shell_small_result_verbatim" in result.transforms_applied
def test_large_bash_tool_result_is_still_compressed() -> None:
"""The guard must not disable compression for large build/test logs."""
output = _ls_output(rows=80)
assert len(output) > 2000
result = _route("Bash", output)
assert result.messages[3]["content"][0]["content"] != output
assert "router:tool_result:kompress" in result.transforms_applied
def test_small_non_shell_tool_result_is_still_compressed() -> None:
"""The guard is scoped to shell tools, not every small tool result."""
output = "a short reference payload\n" * 30
result = _route("SomeOtherTool", output)
assert result.messages[3]["content"][0]["content"] != output
assert "router:tool_result:kompress" in result.transforms_applied
def test_small_openai_tool_result_stays_verbatim() -> None:
"""The OpenAI role:tool shape carries the same guard as Anthropic blocks."""
output = _ls_output()
result = _route_openai("Bash", output)
assert result.messages[3]["content"] == output
assert "router:shell_small_result_verbatim" in result.transforms_applied
def test_shell_result_min_chars_can_disable_the_guard(monkeypatch: pytest.MonkeyPatch) -> None:
"""The escape hatch restores the previous behaviour for operators."""
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "0")
output = _ls_output()
result = _route("Bash", output)
assert result.messages[3]["content"][0]["content"] != output
assert "router:tool_result:kompress" in result.transforms_applied
def test_invalid_shell_result_min_chars_falls_back(monkeypatch: pytest.MonkeyPatch) -> None:
"""A non-numeric override keeps the documented default instead of crashing."""
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "junk")
assert _shell_result_min_chars() == 2000
def test_zero_shell_result_min_chars_is_legitimate(monkeypatch: pytest.MonkeyPatch) -> None:
"""Zero is a real value ("no floor"), not an invalid one."""
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "0")
assert _shell_result_min_chars() == 0
def test_negative_shell_result_min_chars_disables_the_guard(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "-1")
assert _shell_result_min_chars() == 0
result = _route("Bash", _ls_output())
assert "router:shell_small_result_verbatim" not in result.transforms_applied
@pytest.mark.parametrize("text", [None, 123, ""])
def test_empty_or_non_string_values_are_not_small_shell_results(text: object) -> None:
assert not _is_small_shell_result("Bash", text, frozenset({"bash"}))
def test_shell_result_floor_boundary_is_exclusive() -> None:
"""The floor is "shorter than", so exactly 2000 chars stays compressible."""
at_floor = _padded_output(2000)
below_floor = _padded_output(1999)
at_result = _route("Bash", at_floor)
below_result = _route("Bash", below_floor)
assert at_result.messages[3]["content"][0]["content"] != at_floor
assert below_result.messages[3]["content"][0]["content"] == below_floor
assert "router:shell_small_result_verbatim" in below_result.transforms_applied
@pytest.mark.parametrize(
"tool_name",
["Bash", "shell", "local_shell", "run_shell_command", "execute_command"],
)
def test_recognized_shell_aliases_stay_verbatim(tool_name: str) -> None:
"""Harness aliases from the shared tool-name map get the same protection."""
output = _ls_output()
result = _route(tool_name, output)
assert result.messages[3]["content"][0]["content"] == output
assert "router:shell_small_result_verbatim" in result.transforms_applied
def test_unrecognized_shell_like_name_is_still_compressed() -> None:
"""A similar-looking name outside the configured set is not protected."""
output = _ls_output()
result = _route("run_remote_shell", output)
assert result.messages[3]["content"][0]["content"] != output
assert "router:tool_result:kompress" in result.transforms_applied
def test_small_list_form_tool_result_stays_verbatim() -> None:
"""The Anthropic list-form content wrapper is flattened and protected."""
output = _ls_output()
result = _route("Bash", output, list_form=True)
assert result.messages[3]["content"][0]["content"] == [{"type": "text", "text": output}]
assert "router:shell_small_result_verbatim" in result.transforms_applied
def test_small_legacy_function_result_stays_verbatim() -> None:
"""Legacy role:function resolves the shell name from message["name"]."""
output = _ls_output()
result = _route_function("Bash", output)
assert result.messages[3]["content"] == output
assert "router:shell_small_result_verbatim" in result.transforms_applied
def test_small_top_level_text_block_stays_verbatim() -> None:
"""A tool message normalized to top-level text blocks gets the guard."""
output = _ls_output()
result = _route_text_block("Bash", output)
assert result.messages[3]["content"] == [{"type": "text", "text": output}]
assert "router:shell_small_result_verbatim" in result.transforms_applied
@pytest.mark.parametrize("role", ["tool", "function"])
@pytest.mark.parametrize("length", [1999, 2000, 2400])
def test_split_shell_result_uses_total_text_length(role: str, length: int) -> None:
"""The floor applies to the whole result, not each provider text fragment."""
messages = _text_block_conversation("Bash", "")
if role == "function":
messages[3] = {"role": "function", "name": "Bash"}
blocks = [
{"type": "text", "text": _padded_output(length // 2)},
{"type": "text", "text": _padded_output(length - length // 2)},
]
messages[3]["content"] = blocks
router = ContentRouter(ContentRouterConfig())
router.compress = _fake_compress # type: ignore[assignment]
result = router.apply(messages, _tokenizer(), target_ratio=0.4)
assert (result.messages[3]["content"] == blocks) is (length < 2000)
assert ("router:shell_small_result_verbatim" in result.transforms_applied) is (length < 2000)
@pytest.mark.parametrize("route", [_route_openai, _route_function, _route_text_block])
def test_shell_floor_override_applies_to_wire_shapes(
monkeypatch: pytest.MonkeyPatch, route: Callable[[str, str], TransformResult]
) -> None:
"""A positive override protects below the floor and compresses at it."""
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "1000")
below = route("Bash", _padded_output(999))
at = route("Bash", _padded_output(1000))
assert "router:shell_small_result_verbatim" in below.transforms_applied
assert "router:shell_small_result_verbatim" not in at.transforms_applied
def test_custom_shell_names_are_normalized_and_protected() -> None:
"""An operator's custom mixed-case shell name gets the same floor."""
router = ContentRouter(ContentRouterConfig(bash_tool_names=frozenset({"MyShell"})))
router.compress = _fake_compress # type: ignore[assignment]
output = _ls_output()
result = router.apply(_openai_conversation("mYsHeLl", output), _tokenizer(), target_ratio=0.4)
assert result.messages[3]["content"] == output
assert "router:shell_small_result_verbatim" in result.transforms_applied
@pytest.mark.parametrize("route", [_route_openai, _route_function, _route_text_block])
def test_non_shell_results_remain_compressible_in_all_wire_shapes(
route: Callable[[str, str], TransformResult],
) -> None:
output = _ls_output()
result = route("SomeOtherTool", output)
assert "router:shell_small_result_verbatim" not in result.transforms_applied
assert result.messages[3]["content"] not in (output, [{"type": "text", "text": output}])
@pytest.mark.parametrize("route", [_route_openai, _route_function, _route_text_block])
def test_shell_floor_can_be_disabled_in_all_wire_shapes(
monkeypatch: pytest.MonkeyPatch, route: Callable[[str, str], TransformResult]
) -> None:
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "0")
output = _ls_output()
result = route("Bash", output)
assert "router:shell_small_result_verbatim" not in result.transforms_applied
assert result.messages[3]["content"] not in (output, [{"type": "text", "text": output}])
@pytest.mark.parametrize("conversation", [_openai_conversation, _function_conversation])
def test_small_shell_result_diagnostics_explain_why_it_was_preserved(
conversation: Callable[[str, str], list[dict[str, object]]],
) -> None:
output = _ls_output()
router = ContentRouter(ContentRouterConfig())
router.compress = _fake_compress # type: ignore[assignment]
result = router.apply(
conversation("Bash", output), _tokenizer(), target_ratio=0.4, collect_diagnostics=True
)
decision = next(item for item in result.message_decisions if item.message_index == 3)
assert decision.action == "protected:shell_small_result"
assert decision.tokens_before == decision.tokens_after
assert result.messages[3]["content"] == output
@pytest.mark.parametrize("shape", ["anthropic", "tool", "function"])
def test_small_shell_blocks_without_optional_route_counters(shape: str) -> None:
"""Direct block callers can omit counters without losing the shell guard."""
output = _ls_output()
if shape == "anthropic":
message = _conversation("Bash", output)[3]
elif shape == "function":
message = {
"role": "function",
"name": "Bash",
"content": [{"type": "text", "text": output}],
}
else:
message = _text_block_conversation("Bash", output)[3]
transforms: list[str] = []
router = ContentRouter(ContentRouterConfig())
router.compress = _fake_compress # type: ignore[assignment]
result = router._process_content_blocks(
message=message,
content_blocks=message["content"],
context="",
transforms_applied=transforms,
excluded_tool_ids=set(),
ccr_retrieve_tool_ids=set(),
tool_name_map={"tool_1": "Bash", "call_1": "Bash"},
route_counts=None,
messages_from_end=20,
)
assert result is message
assert transforms == ["router:shell_small_result_verbatim"]
+12 -3
View File
@@ -2,6 +2,8 @@
from __future__ import annotations
import pytest
from headroom.config import DEFAULT_EXCLUDE_TOOLS
from headroom.proxy.server import HeadroomProxy, ProxyConfig
from headroom.transforms.content_detector import ContentType
@@ -159,7 +161,11 @@ def test_web_tool_results_skip_cross_turn_dedup() -> None:
assert second["content"] == payload # type: ignore[index]
def test_bash_remains_compressible() -> None:
@pytest.mark.parametrize("repetitions, expected_calls", [(100, 0), (200, 1)])
def test_bash_compression_respects_small_result_floor(
monkeypatch: pytest.MonkeyPatch, repetitions: int, expected_calls: int
) -> None:
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "2000")
router = _router()
calls = 0
@@ -177,8 +183,11 @@ def test_bash_remains_compressible() -> None:
)
router.compress = fake_compress # type: ignore[method-assign]
payload = "bash output " * 100
payload = "bash output " * repetitions
result = router.apply(_messages("Bash", payload), _Tokenizer())
assert calls == 1
assert calls == expected_calls
assert "router:excluded:tool" not in result.transforms_applied
assert ("router:shell_small_result_verbatim" in result.transforms_applied) is (
expected_calls == 0
)
Generated
+4 -2
View File
@@ -23,13 +23,15 @@ resolution-markers = [
[manifest]
constraints = [
{ name = "aiohttp", specifier = ">=3.14.4" },
{ name = "certifi", specifier = ">=2026.10.19" },
{ name = "cryptography", specifier = ">=50.0.0" },
{ name = "filelock", marker = "python_full_version >= '3.11'", specifier = ">=4.2.1" },
{ name = "gitpython", specifier = ">=3.1.58" },
{ name = "langsmith", specifier = ">=0.9.0" },
{ name = "lxml-html-clean", specifier = ">=0.4.5" },
{ name = "pydantic-settings", specifier = ">=2.14.2" },
{ name = "pygments", specifier = ">=2.20.0" },
{ name = "setuptools", specifier = ">=83.0.0" },
{ name = "setuptools", specifier = ">=84.1.0" },
{ name = "transformers", specifier = ">=5.5.0" },
]
overrides = [{ name = "pillow", specifier = ">=13.0.0" }]
@@ -1765,7 +1767,7 @@ requires-dist = [
{ name = "openai", marker = "extra == 'proxy'", specifier = ">=2.14.0" },
{ name = "openpyxl", marker = "extra == 'dev'", specifier = ">=3.1.0" },
{ name = "openpyxl", marker = "extra == 'spreadsheet'", specifier = ">=3.1.0" },
{ name = "opentelemetry-api", specifier = ">=1.24.0" },
{ name = "opentelemetry-api", specifier = ">=1.45.1" },
{ name = "opentelemetry-exporter-otlp-proto-http", marker = "extra == 'dev'", specifier = ">=1.24.0" },
{ name = "opentelemetry-exporter-otlp-proto-http", marker = "extra == 'otel'", specifier = ">=1.24.0" },
{ name = "opentelemetry-sdk", marker = "extra == 'dev'", specifier = ">=1.24.0" },