mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description Fixes #3590. Small shell tool results (default: shorter than 2000 characters) now bypass lossy compression. Byte-lossless folds such as search grouping may still apply. File reads and searches were already protected; shell output is deliberately compressible, but a short `ls -la`, `git status`, or JSON dump is not worth the accuracy loss described in #3590. The real proxy-level before/after evidence is in Real Behavior Proof. A live Claude Code session is still untested. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `HEADROOM_SHELL_RESULT_MIN_CHARS` (default `2000`) and `_is_small_shell_result()`, applied to OpenAI `role:"tool"`, legacy `role:"function"`, top-level text-block tool messages, and Anthropic `tool_result` string/list-form paths after the lossless search fold and before lossy compression. - Normalized `ContentRouterConfig.bash_tool_names` once and extended it with the shell aliases already in `headroom.learn._shared` (`shell`, `local_shell`, `run_shell_command`, `execute_command`, `exec_command`, `terminal`, `run_command`, `run_terminal_command`). - Measure a top-level text-block shell result using the combined text length, so a large result cannot bypass compression merely by splitting into small blocks. - Added boundary coverage for both tool/function block shapes, positive threshold overrides and custom case-insensitive shell names; documented the combined-length contract in `docs/content/docs/proxy.mdx`. - Updated the existing proxy-level Bash regression to assert both sides of the 2000-character floor, fixing the hosted shard-4 failure. Added kill-switch, malformed/empty-value and diagnostic coverage. - Carried dependency security remediation requested in review: certifi and the OpenTelemetry family, three patched Rust crates, and patched npm locks for docs, SDK, OpenCode and E2E tooling. These use existing packages, explicit security floors and targeted dependency overrides; libc platform constraints remain intact. ## Testing - [x] Hosted CI passes on the current head, including all four Python test shards - [x] Changed Python tests passed Ruff in the preceding regression revision; no Python runtime source changed in this dependency follow-up - [ ] Full local Python type check passes (Windows/test-double baseline limitations) - [x] Regression and dependency-interface coverage added - [x] Real proxy proof and isolated CLI smoke evidence recorded ### Test Output Latest follow-up: Windows, Python 3.11 / Node 22.20.0 for the OpenCode suite, and portable official Node 24.22.1 for E2E CLI qualification. The portable Node archive's SHA-256 matched nodejs.org's published checksum. ```text Python shell-floor/web-tool/router/alias/tracing + OpenClaw wrapper/bridge selection: 119 passed, 2 warnings in 10.37s uv lock --check Resolved 262 packages uv tree --locked --python-version 3.10 --package filelock filelock v4.1.0 uv tree --locked --python-version 3.12 --package filelock filelock v4.2.4 FileLock 4.2.4 process smoke: contention times out, release permits next process cd plugins/opencode npm test Test Files 4 passed (4) Tests 60 passed (60) npm run typecheck > node node_modules/@typescript/native/bin/tsc --noEmit npm run build DTS Build success npm audit --package-lock-only --audit-level=low --registry=https://registry.npmjs.org found 0 vulnerabilities After merging current upstream main (including #4453): cd e2e/wrap/tools node node_modules/openclaw/openclaw.mjs --version OpenClaw 2026.10.6 npm audit --package-lock-only --audit-level=low --registry=https://registry.npmjs.org 2 vulnerabilities (1 moderate, 1 high) # limited to npm's bundled copies: # GHSA-rxp7-6hxx-x8q5 — node_modules/npm/node_modules/ip-address 10.7.2 # GHSA-4qv8-2p9j-2685 and GHSA-xjj6-f33q-929c — node_modules/npm/node_modules/undici 6.29.0 # exact scoped exceptions expire 2026-11-15; no OpenClaw path is excepted production-lock pip-audit: No known vulnerabilities found ``` - The Python native extension remains the existing 0.41.0 Windows wheel's `_core.pyd`; all Python source under test comes from this PR checkout. No native rebuild or Torch/model inference qualification is claimed. - The 3 new OpenCode compatibility tests exercise the actual upstream `tool.schema` interface and Effect `Primitive.fileParse` TOML success/error channels, mocking only filesystem I/O. They accompany the explicit Zod/TOML overrides. - E2E CLI checks used isolated `OPENCLAW_STATE_DIR` / `OPENCLAW_CONFIG_PATH`, with no model request or modification of the user's normal OpenClaw configuration. E2E dependencies were installed with lifecycle scripts disabled. The full Docker/model E2E suite was not run locally. - Earlier unchanged docs/SDK lock remediation passed: SDK **351 passed, 33 skipped** plus typecheck/build; docs typecheck/build with 185 static pages; both npm audits were zero. The previous head also passed the hosted Rust security audit. - Full local Python suite was not rerun. The earlier remote-timeout float-boundary failure remains outside this change. ## Real Behavior Proof - Environment: Windows 10 x64, Python 3.11.0, Headroom source `main` at `e995eb9` (0.41.1-dev), native `_core` from the 0.41.0 wheel because no Rust toolchain is available locally. `HEADROOM_DISABLE_KOMPRESS=1` keeps the strategy deterministic; the guard sits before strategy selection, so the same path applies when Kompress is enabled. - Exact command / steps: local harness (not committed) `python proof-3590/run_proof.py` starts the source-tree proxy in `--mode token` with `HEADROOM_MIN_TOKENS=0` and a local OpenAI-compatible upstream, and sends a chat request containing a Bash `ps aux` tool result (1719 chars) followed by later turns. The upstream records the transformed request body. - Observed result: `HEADROOM_SHELL_RESULT_MIN_CHARS=0` produced `compressed=1`, `router:tabular:0.85`, 802 -> 697 tokens (saved 105), and upstream received a 1455-char schema representation. Default `2000` produced `shell_small_result: 1`, `compressed=0`, 802 -> 802 tokens, and upstream received the original 1719-char output byte-for-byte. - Review follow-up proof: `python proof-3590/run_block_review.py` ran the real proxy against the same local upstream without replacing its compressor. A 1719-character shell result split into two text blocks stayed unchanged; a 3438-character result split into two 1719-character blocks remained compressible. The previous PR revision incorrectly protected both large-result fragments. This harness is local, not part of the PR. - Not tested: a live `headroom wrap claude` session; the ML Kompress path; Linux/macOS; the full suite (one pre-existing baseline failure). ## Runtime Rollout Safety - Rollout-managed feature(s): none; ContentRouter default path. - Minimum rollout channel: default proxy. - Stable/default behavior changed: yes. Shell results between `min_chars_for_block_compression` (500) and `HEADROOM_SHELL_RESULT_MIN_CHARS` (2000) now stay verbatim instead of taking lossy compression. At ~1500 chars (~375 tokens), this keeps roughly 325 extra tokens per result for the life of the context. - Kill switch / disable path: `HEADROOM_SHELL_RESULT_MIN_CHARS=0`. - Unsafe override required: no. - Qualification impact: compression ratio may decrease for small shell outputs; no API or correctness changes. - Rollback path: revert this commit or set `HEADROOM_SHELL_RESULT_MIN_CHARS=0`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A — no UI changes. ## Additional Notes This follows the [dependency-remediation review](https://github.com/headroomlabs-ai/headroom/pull/4426#pullrequestreview-5636487191). It does not disable audits, ignore advisories, reduce audit severity, or use `npm audit fix --force`. - **Python:** certifi/OpenTelemetry/Rust fixes from the prior revision remain. This follow-up raises setuptools to **84.1.0** for the Windows wheel namespace traversal advisory and uses FileLock **4.2.4** on Python ≥3.11. The universal lock remains resolvable with the published Python ≥3.10 contract. **Python 3.10 now selects FileLock 4.1.0, but it remains below the 4.2.1 security floor** because fixed releases require ≥3.11; this is not presented as all-interpreter remediation. - **OpenCode:** pin security overrides **Zod 4.6.7** and **TOML 5.0.1** without downgrading the plugin or replacing its pinned Effect version. TOML crosses a major version (4.3.0 → 5.0.1); its Node ≥20 floor and consumed `parse(content)` API are unchanged. The added tests validate the exercised schema and Effect parsing contracts, not every possible TOML consumer. This override needs normal maintainer dependency review. - **E2E:** the original remediation moved OpenClaw to 2026.10.5; the upstream sync now brings **OpenClaw 2026.10.6** with **Undici 8.12.3** and a Node floor of **≥24.22.1 <25 or ≥26.12.1**. The only remaining raw npm findings are npm's published bundled copies, handled by the exact, expiring E2E-only exception policy described below. - Dependency packages and runtime floors were checked against the upstream registry metadata. Locks use public registries and retain existing libc platform constraints. ### Security qualification after upstream sync Merge commit `e09d37c9274c812f26e994d01a623542a4a7ec31` brings this branch to upstream `main` through `a616232`, including [#4453](https://github.com/headroomlabs-ai/headroom/pull/4453). That upstream change establishes the repository's current security policy for the dependency cases that previously blocked this PR: - **Production Python lock:** Python ≥3.11 resolves Pillow 13.0.0 and FileLock 4.2.4. The current lock passes `uv lock --check`; an equivalent export of the production `all` set passes `pip-audit 2.10.1 --no-deps --disable-pip` with **no known vulnerabilities**. - **Published-package resolution:** FastEmbed still caps Pillow below 13 when ordinary pip resolves package metadata. Upstream now disables image decoding on Pillow below 13 and separately audits the pip-resolved set, accepting only Pillow findings covered by that tested decoder guard. This is a bounded runtime mitigation, not a claim that the older Pillow release has no advisories. - **E2E npm lock:** OpenClaw 2026.10.6 uses fixed Undici 8.12.3. Raw npm audit now reports only npm 12.2.1's bundled Undici 6.29.0 and ip-address 10.7.2 copies. Because npm does not allow overrides to replace its published bundled contents and no newer release is available, upstream accepts only the three exact advisory/path/version pairs in `e2e/wrap/tools/npm-audit-exceptions.json`, only for the non-shipping E2E harness, and only until **2026-11-15**. The workflow fails on any extra finding, version/path mismatch, stale exception or expiry. - **Python 3.10:** the lock now selects FileLock 4.1.0, but it remains below the 4.2.1 security floor because fixed releases require Python ≥3.11. This supported-interpreter limitation is documented by the upstream policy and is not represented as universal remediation. The earlier OpenCode standalone-wheel mismatch remains fixed by `dee73be75cf81bb67a73573861f1b3ed4067a2bf`: typecheck, build, all 60 plugin tests and the standalone build passed locally, both generated artifacts matched their Python-shipped paths byte-for-byte, and the shipped bundle imported with the expected `headroom` id and both `server`/`setup` entry points. ### Current merge status - Current head `ab93d7917a6660c704069bb87b6c02f9c744cebc` is mergeable; all router/documentation inline review threads are resolved. - JerrettDavis approved this head on 2026-10-24, superseding the earlier security-related change requests. - All 12 pull-request workflows on this head completed successfully, including [CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40122163824) (all four Python test shards), [Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40122163777), wheel builds/import smoke tests, OpenCode, docs, and Docker E2E checks. The workflows are no longer awaiting approval. - The PR remains open, awaiting maintainer merge. - A live Claude Code/model session, the ML Kompress path, a native rebuild and the full Docker/model E2E suite remain unverified locally; the hosted results above are separate from the recorded local qualification. Implemented and verified with AI assistance. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JerrettDavis
parent
4d2dd62b7c
commit
4ee616453f
@@ -320,6 +320,7 @@ Fine-grained control over what gets compressed and how hard. Most users pick a [
|
||||
| `--mode` / `HEADROOM_MODE` | `cache` | `cache` compresses only the newest delta (prefix-cache safe); `token` maximizes removal. |
|
||||
| `--target-ratio` / `HEADROOM_TARGET_RATIO` | unset | Keep-ratio for ML text compression; lower = more aggressive (e.g. `0.10`). |
|
||||
| `HEADROOM_MIN_TOKENS` | `500` | Minimum block size before a tool output is compressed. |
|
||||
| `HEADROOM_SHELL_RESULT_MIN_CHARS` | `2000` | Shell tool results shorter than this many characters bypass lossy compression. For top-level text blocks, the threshold uses their combined text length. Byte-lossless folds such as search grouping may still apply. Applies to `bash`, `shell`, `local_shell`, and aliases such as `run_shell_command`/`execute_command`; set `0` to restore lossy compression. |
|
||||
| `HEADROOM_COMPRESS_USER_MESSAGES` | profile | Compress text inside user-role messages. The `coding` profile turns this on; set `0` to keep user text verbatim under it. Anthropic `tool_result` blocks compress either way. No `--compress-user-messages` flag on `headroom proxy`; the direct module entry point (`python -m headroom.proxy.server`) does accept one. |
|
||||
| `HEADROOM_COMPRESS_SYSTEM_MESSAGES` | unset | Compress system prompts. Off by default to keep the hottest cache prefix stable. |
|
||||
| `HEADROOM_PROTECT_RECENT` | profile | Never compress the N most recent turns. |
|
||||
|
||||
@@ -10,7 +10,8 @@ PATH. `npm ci` does not support global installs, so diagnostics must also use
|
||||
`npm list --prefix /opt/wrap-tools`.
|
||||
|
||||
After editing a CLI version in `package.json`, regenerate the lockfile using
|
||||
Node 22 and npm 11.16.0 on Linux:
|
||||
Node 24.22.1+ within the Node 24 line and npm 11.16.0 on Linux (the pinned
|
||||
OpenClaw version no longer supports Node 22):
|
||||
|
||||
```sh
|
||||
cd e2e/wrap/tools
|
||||
@@ -34,4 +35,11 @@ to make a build pass.
|
||||
These are existing test tools, not new product dependencies. Codex is maintained
|
||||
by OpenAI; OpenClaw and its transitive dependencies retain their upstream
|
||||
maintainers. Their install surface includes registry downloads, native optional
|
||||
packages and lifecycle scripts, as before. Top-level CLI versions are unchanged.
|
||||
packages and lifecycle scripts, as before. OpenClaw 2026.10.6 and its fixed
|
||||
Undici override follow the upstream #4453 security repair.
|
||||
|
||||
The raw audit still reports npm 12.2.1's bundled ip-address and Undici copies.
|
||||
`npm-audit-exceptions.json` records the upstream-approved E2E-only risk
|
||||
acceptance, restricted to exact advisory/path/version combinations and expiring
|
||||
on 2026-11-15. Security CI rejects other findings and expired or stale entries;
|
||||
these exceptions are not a claim that the contained packages are patched.
|
||||
|
||||
@@ -2113,6 +2113,30 @@ def _is_small_mcp_result(tool_name: Any, text: Any) -> bool:
|
||||
)
|
||||
|
||||
|
||||
def _shell_result_min_chars() -> int:
|
||||
"""Below this many characters a shell tool result is never lossily compressed."""
|
||||
try:
|
||||
return max(0, int(os.environ.get("HEADROOM_SHELL_RESULT_MIN_CHARS", "2000")))
|
||||
except ValueError:
|
||||
return 2000
|
||||
|
||||
|
||||
def _is_small_shell_result(tool_name: Any, text: Any, bash_tool_names: frozenset[str]) -> bool:
|
||||
"""True when ``text`` is a short shell result that must not be lossily compressed.
|
||||
|
||||
A 200-token ``ls -la``, ``git status``, or JSON dump is cheap to keep and
|
||||
expensive to get wrong: lossy compression can leave a numbers-only blob
|
||||
that still looks structured, so the model treats it as data instead of
|
||||
retrying the command (issue #3590). Large build/test logs still take the
|
||||
lossy path. ``HEADROOM_SHELL_RESULT_MIN_CHARS=0`` restores the old
|
||||
behaviour.
|
||||
"""
|
||||
if not isinstance(text, str) or not text:
|
||||
return False
|
||||
name = str(tool_name or "").lower()
|
||||
return name in bash_tool_names and len(text) < _shell_result_min_chars()
|
||||
|
||||
|
||||
@dataclass
|
||||
class ContentRouterConfig:
|
||||
"""Configuration for intelligent content routing.
|
||||
@@ -2317,8 +2341,22 @@ class ContentRouterConfig:
|
||||
# BUT a read-only *search* run through them (grep/rg/git grep) yields byte-
|
||||
# losslessly foldable output — folded instead of lossy-compressed. See
|
||||
# ``_bash_search_fold``. Config so new harness tool names / search programs
|
||||
# can be added without code changes.
|
||||
bash_tool_names: frozenset[str] = frozenset({"bash", "shell", "local_shell"})
|
||||
# can be added without code changes. The aliases mirror
|
||||
# ``headroom.learn._shared._TOOL_NAME_MAP`` so the same harnesses are
|
||||
# covered here.
|
||||
bash_tool_names: frozenset[str] = frozenset(
|
||||
{
|
||||
"bash",
|
||||
"shell",
|
||||
"local_shell",
|
||||
"run_shell_command",
|
||||
"execute_command",
|
||||
"exec_command",
|
||||
"terminal",
|
||||
"run_command",
|
||||
"run_terminal_command",
|
||||
}
|
||||
)
|
||||
bash_search_commands: frozenset[str] = frozenset(
|
||||
{"grep", "egrep", "fgrep", "rg", "ripgrep", "ag", "ack"}
|
||||
)
|
||||
@@ -2349,6 +2387,12 @@ class ContentRouterConfig:
|
||||
# Default False; the proxy enables it in token mode.
|
||||
search_group_by_file: bool = False
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# Normalize once so the per-message shell check stays a set lookup.
|
||||
self.bash_tool_names = frozenset(
|
||||
name.lower() for name in self.bash_tool_names if isinstance(name, str)
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _PerRequestRuntimeState:
|
||||
@@ -6311,6 +6355,7 @@ class ContentRouter(Transform):
|
||||
"ccr_retrieve": 0,
|
||||
"user_msg": 0,
|
||||
"small": 0,
|
||||
"shell_small_result": 0,
|
||||
"recent_code": 0,
|
||||
"analysis_ctx": 0,
|
||||
"ratio_too_high": 0,
|
||||
@@ -6600,6 +6645,36 @@ class ContentRouter(Transform):
|
||||
_diag[i] = "compressed:bash_lossless_search"
|
||||
continue
|
||||
|
||||
# Small shell results stay verbatim (#3590). A short `ls`,
|
||||
# `git status`, or JSON dump is cheap to keep; lossy
|
||||
# compression can leave a numbers-only blob that still looks
|
||||
# structured, so the model treats it as data instead of
|
||||
# retrying. Large build/test logs still take the lossy path.
|
||||
if _is_small_shell_result(tool_name, content, self.config.bash_tool_names):
|
||||
result_slots[i] = message
|
||||
transforms_applied.append("router:shell_small_result_verbatim")
|
||||
route_counts["shell_small_result"] = (
|
||||
route_counts.get("shell_small_result", 0) + 1
|
||||
)
|
||||
if collect_diagnostics:
|
||||
_diag[i] = "protected:shell_small_result"
|
||||
continue
|
||||
|
||||
# Legacy OpenAI `role:"function"` carries the tool name on the
|
||||
# message itself, not in a tool_call_id. Resolve it directly so the
|
||||
# small-shell floor also covers this wire shape (#3590 review).
|
||||
if role == "function":
|
||||
function_name = message.get("name", "")
|
||||
if _is_small_shell_result(function_name, content, self.config.bash_tool_names):
|
||||
result_slots[i] = message
|
||||
transforms_applied.append("router:shell_small_result_verbatim")
|
||||
route_counts["shell_small_result"] = (
|
||||
route_counts.get("shell_small_result", 0) + 1
|
||||
)
|
||||
if collect_diagnostics:
|
||||
_diag[i] = "protected:shell_small_result"
|
||||
continue
|
||||
|
||||
# Read protection (ROLE / SHAPE-AGNOSTIC). An observation produced by
|
||||
# a file read command (cat/sed/head/…) is passed VERBATIM so the agent
|
||||
# keeps exact bytes to patch — regardless of how THIS harness labels
|
||||
@@ -7569,6 +7644,21 @@ class ContentRouter(Transform):
|
||||
any_compressed = False
|
||||
role = message.get("role", "")
|
||||
|
||||
# Top-level text blocks are fragments of one tool result. Measure the
|
||||
# whole result so splitting a large log cannot activate the small-result
|
||||
# floor for every fragment. Sum lengths without copying the payload.
|
||||
shell_result_chars = (
|
||||
sum(
|
||||
len(block["text"])
|
||||
for block in content_blocks
|
||||
if isinstance(block, dict)
|
||||
and block.get("type") == "text"
|
||||
and isinstance(block.get("text"), str)
|
||||
)
|
||||
if role in ("tool", "function")
|
||||
else 0
|
||||
)
|
||||
|
||||
# Role-based gate for `text` blocks. Tool/function roles are tool
|
||||
# outputs and compress freely; assistant defaults to skip (cache
|
||||
# safety) with explicit opt-in; unknown roles default to skip.
|
||||
@@ -7767,6 +7857,18 @@ class ContentRouter(Transform):
|
||||
any_compressed = True
|
||||
continue
|
||||
|
||||
# Small shell results stay verbatim (#3590), same contract as
|
||||
# the string-form path above. Build/test logs above the floor
|
||||
# still go through lossy compression.
|
||||
if _is_small_shell_result(tool_name, tool_text, self.config.bash_tool_names):
|
||||
new_blocks.append(block)
|
||||
transforms_applied.append("router:shell_small_result_verbatim")
|
||||
if route_counts is not None:
|
||||
route_counts["shell_small_result"] = (
|
||||
route_counts.get("shell_small_result", 0) + 1
|
||||
)
|
||||
continue
|
||||
|
||||
# Protection: failed tool calls / error outputs stay verbatim
|
||||
# (issue #847). `is_error` is Anthropic's explicit failure
|
||||
# flag and suffices alone; the indicator scan catches error
|
||||
@@ -7864,6 +7966,29 @@ class ContentRouter(Transform):
|
||||
if route_counts is not None:
|
||||
route_counts["ccr_retrieve"] = route_counts.get("ccr_retrieve", 0) + 1
|
||||
continue
|
||||
|
||||
# A normalized tool/function message can carry its output
|
||||
# as top-level text blocks rather than a tool_result
|
||||
# wrapper. Apply the same small-shell floor here (#3590
|
||||
# review).
|
||||
_shell_name = (
|
||||
(tool_name_map or {}).get(_msg_tool_id, "")
|
||||
if role == "tool"
|
||||
else message.get("name", "")
|
||||
)
|
||||
if (
|
||||
_is_small_shell_result(
|
||||
_shell_name, block.get("text", ""), self.config.bash_tool_names
|
||||
)
|
||||
and shell_result_chars < _shell_result_min_chars()
|
||||
):
|
||||
new_blocks.append(block)
|
||||
transforms_applied.append("router:shell_small_result_verbatim")
|
||||
if route_counts is not None:
|
||||
route_counts["shell_small_result"] = (
|
||||
route_counts.get("shell_small_result", 0) + 1
|
||||
)
|
||||
continue
|
||||
text_content = block.get("text", "")
|
||||
if isinstance(text_content, str) and (
|
||||
len(text_content) > min_chars or self._has_lossless_fold(text_content)
|
||||
|
||||
@@ -47,8 +47,8 @@
|
||||
},
|
||||
"overrides": {
|
||||
"esbuild": "^0.28.1",
|
||||
"zod": "^4.6.6",
|
||||
"toml": "^5.0.1"
|
||||
"zod": "4.6.7",
|
||||
"toml": "5.0.1"
|
||||
},
|
||||
"license": "Apache-2.0"
|
||||
}
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
import { tool } from "@opencode-ai/plugin";
|
||||
import { Effect, FileSystem, Path } from "effect";
|
||||
import { Primitive } from "effect/unstable/cli";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
function parseConfig(content: string): Promise<unknown> {
|
||||
const fs = FileSystem.makeNoop({
|
||||
exists: () => Effect.succeed(true),
|
||||
stat: () => Effect.succeed({ type: "File" } as FileSystem.File.Info),
|
||||
readFileString: () => Effect.succeed(content),
|
||||
});
|
||||
return Effect.runPromise(
|
||||
Primitive.fileParse({ format: "toml" }).parse("/headroom/config.toml").pipe(
|
||||
Effect.provideService(FileSystem.FileSystem, fs),
|
||||
Effect.provide(Path.layer),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
describe("patched OpenCode dependency compatibility", () => {
|
||||
it("builds and validates a tool with the upstream Zod schema interface", () => {
|
||||
const schema = tool.schema.object({
|
||||
hash: tool.schema.string().regex(/^[a-f0-9]{24}$/i),
|
||||
});
|
||||
const retrieve = tool({
|
||||
description: "Retrieve an original tool result",
|
||||
args: schema.shape,
|
||||
async execute(args) {
|
||||
return args.hash;
|
||||
},
|
||||
});
|
||||
expect(schema.parse({ hash: "a".repeat(24) })).toEqual({ hash: "a".repeat(24) });
|
||||
expect(schema.safeParse({ hash: "invalid" }).success).toBe(false);
|
||||
expect(retrieve.args.hash).toBe(schema.shape.hash);
|
||||
});
|
||||
|
||||
it("parses typed TOML through Effect's actual file parser", async () => {
|
||||
const content = `
|
||||
name = "headroom"
|
||||
enabled = true
|
||||
ports = [8787, 8788]
|
||||
[proxy]
|
||||
host = "127.0.0.1"
|
||||
ratio = 0.5
|
||||
[[models]]
|
||||
name = "coding"
|
||||
[[models]]
|
||||
name = "review"
|
||||
`;
|
||||
await expect(parseConfig(content)).resolves.toEqual({
|
||||
name: "headroom",
|
||||
enabled: true,
|
||||
ports: [8787, 8788],
|
||||
proxy: { host: "127.0.0.1", ratio: 0.5 },
|
||||
models: [{ name: "coding" }, { name: "review" }],
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps malformed TOML on Effect's parse-error channel", async () => {
|
||||
await expect(parseConfig("[unclosed")).rejects.toThrow("Failed to parse '.toml'");
|
||||
});
|
||||
});
|
||||
+8
-3
@@ -54,7 +54,7 @@ dependencies = [
|
||||
"litellm>=1.96.2,<2.0",
|
||||
"click>=8.3.3", # CLI framework; PYSEC-2026-2132 fix (command injection in click.edit())
|
||||
"rich>=13.0.0", # Rich terminal output
|
||||
"opentelemetry-api>=1.24.0", # Safe no-op OTEL API for instrumentation
|
||||
"opentelemetry-api>=1.45.1", # CVE-2026-114603: bounded, linear-time tracestate parsing
|
||||
# AST-aware code slicing (CodeCompressor); binary wheel. 0.44.1 is excluded:
|
||||
# that PyPI release was a compromised supply-chain build shipping an
|
||||
# info-stealer `sg.exe` (Trojan:Win64/Lazy!MTB) alongside the real binary
|
||||
@@ -414,6 +414,8 @@ Changelog = "https://github.com/headroomlabs-ai/headroom/blob/main/CHANGELOG.md"
|
||||
# transitively; the floor pins below ensure uv resolves to patched versions.
|
||||
[tool.uv]
|
||||
constraint-dependencies = [
|
||||
# PYSEC-2026-4320 — remove distrusted root certificates from the CA bundle.
|
||||
"certifi>=2026.10.19",
|
||||
# GHSA-5239-wwwm-4pmq (Low) — transitive via rich; fix at 2.20.0
|
||||
"pygments>=2.20.0",
|
||||
# GHSA-4xgf-cpjx-pc3j (Medium) — transitive via mcp; fix at 2.14.2
|
||||
@@ -429,8 +431,11 @@ constraint-dependencies = [
|
||||
"lxml-html-clean>=0.4.5",
|
||||
# CVE-2026-5241 (High) — direct optional dep for proxy/ml/voice; fix at 5.5.0
|
||||
"transformers>=5.5.0",
|
||||
# PYSEC-2026-3447 — transitive dependency; fix at 83.0.0
|
||||
"setuptools>=83.0.0",
|
||||
# GHSA-xmj3-9gf7-h52w — Windows wheel namespace path traversal.
|
||||
"setuptools>=84.1.0",
|
||||
# CVE-2026-112940/112913/112907. The patched series requires Python 3.11;
|
||||
# Python 3.10 retains the legacy locked version; it is not remediated by this marker.
|
||||
"filelock>=4.2.1; python_version >= '3.11'",
|
||||
# PYSEC-2026-3545/3546/3547 — transitive HTTP/WebSocket parser fixes;
|
||||
# CVE-2026-111063/111064, GHSA-2g87-pp6c-29x5; fix at 3.14.4
|
||||
"aiohttp>=3.14.4",
|
||||
|
||||
@@ -0,0 +1,483 @@
|
||||
"""Small shell results must stay verbatim through the lossy router path.
|
||||
|
||||
Issue #3590: ``headroom wrap claude`` with a low ``HEADROOM_MIN_TOKENS``
|
||||
compressed a short ``ls -la`` result into a numbers-only blob. The output
|
||||
still looked structured, so the model treated it as data instead of
|
||||
retrying. File reads and searches are already protected; shell output is
|
||||
deliberately compressible, but a small result is not worth the accuracy
|
||||
cost. Large build/test logs must keep going through the lossy path.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.config import TransformResult
|
||||
from headroom.providers import OpenAIProvider
|
||||
from headroom.tokenizer import Tokenizer
|
||||
from headroom.transforms.content_router import (
|
||||
CompressionStrategy,
|
||||
ContentRouter,
|
||||
ContentRouterConfig,
|
||||
ContentType,
|
||||
RouterCompressionResult,
|
||||
RoutingDecision,
|
||||
_is_small_shell_result,
|
||||
_shell_result_min_chars,
|
||||
)
|
||||
|
||||
|
||||
def _tokenizer() -> Tokenizer:
|
||||
provider = OpenAIProvider()
|
||||
return Tokenizer(provider.get_token_counter("gpt-4o"), "gpt-4o")
|
||||
|
||||
|
||||
def _lossy_result(original: str) -> RouterCompressionResult:
|
||||
"""Stand-in for the ML compressor, including its CCR recovery marker."""
|
||||
return RouterCompressionResult(
|
||||
compressed="128 448 09:26 1760 09:28 11980 16767 2394 1262\n<<ccr:deadbeef>>",
|
||||
original=original,
|
||||
strategy_used=CompressionStrategy.KOMPRESS,
|
||||
strategy_chain=["kompress"],
|
||||
routing_log=[
|
||||
RoutingDecision(
|
||||
ContentType.PLAIN_TEXT,
|
||||
CompressionStrategy.KOMPRESS,
|
||||
original_tokens=400,
|
||||
compressed_tokens=12,
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def _fake_compress(content: str, **kwargs: object) -> RouterCompressionResult:
|
||||
return _lossy_result(content)
|
||||
|
||||
|
||||
def _conversation(
|
||||
tool_name: str, tool_output: str, *, list_form: bool = False
|
||||
) -> list[dict[str, object]]:
|
||||
"""A conversation old enough that the tool result is compressible."""
|
||||
tool_content: object = [{"type": "text", "text": tool_output}] if list_form else tool_output
|
||||
messages: list[dict[str, object]] = [
|
||||
{"role": "system", "content": "You are a coding agent."},
|
||||
{"role": "user", "content": "run the command"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{
|
||||
"type": "tool_use",
|
||||
"id": "tool_1",
|
||||
"name": tool_name,
|
||||
"input": {"command": "ls -la"},
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": "tool_1",
|
||||
"content": tool_content,
|
||||
}
|
||||
],
|
||||
},
|
||||
]
|
||||
for i in range(10):
|
||||
messages.append({"role": "assistant", "content": f"step {i}"})
|
||||
messages.append({"role": "user", "content": f"continue {i}"})
|
||||
return messages
|
||||
|
||||
|
||||
def _route(tool_name: str, tool_output: str, *, list_form: bool = False) -> TransformResult:
|
||||
router = ContentRouter(ContentRouterConfig())
|
||||
router.compress = _fake_compress # type: ignore[assignment]
|
||||
return router.apply(
|
||||
_conversation(tool_name, tool_output, list_form=list_form),
|
||||
_tokenizer(),
|
||||
target_ratio=0.4,
|
||||
)
|
||||
|
||||
|
||||
def _openai_conversation(tool_name: str, tool_output: str) -> list[dict[str, object]]:
|
||||
messages: list[dict[str, object]] = [
|
||||
{"role": "system", "content": "You are a coding agent."},
|
||||
{"role": "user", "content": "run the command"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_1",
|
||||
"type": "function",
|
||||
"function": {"name": tool_name, "arguments": "{}"},
|
||||
}
|
||||
],
|
||||
},
|
||||
{"role": "tool", "tool_call_id": "call_1", "content": tool_output},
|
||||
]
|
||||
for i in range(10):
|
||||
messages.append({"role": "assistant", "content": f"step {i}"})
|
||||
messages.append({"role": "user", "content": f"continue {i}"})
|
||||
return messages
|
||||
|
||||
|
||||
def _route_openai(tool_name: str, tool_output: str) -> TransformResult:
|
||||
router = ContentRouter(ContentRouterConfig())
|
||||
router.compress = _fake_compress # type: ignore[assignment]
|
||||
return router.apply(
|
||||
_openai_conversation(tool_name, tool_output), _tokenizer(), target_ratio=0.4
|
||||
)
|
||||
|
||||
|
||||
def _tail(messages: list[dict[str, object]]) -> list[dict[str, object]]:
|
||||
for i in range(10):
|
||||
messages.append({"role": "assistant", "content": f"step {i}"})
|
||||
messages.append({"role": "user", "content": f"continue {i}"})
|
||||
return messages
|
||||
|
||||
|
||||
def _function_conversation(tool_name: str, tool_output: str) -> list[dict[str, object]]:
|
||||
"""Legacy OpenAI shape: role:function with the tool name on the message."""
|
||||
messages: list[dict[str, object]] = [
|
||||
{"role": "system", "content": "You are a coding agent."},
|
||||
{"role": "user", "content": "run the command"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_1",
|
||||
"type": "function",
|
||||
"function": {"name": tool_name, "arguments": "{}"},
|
||||
}
|
||||
],
|
||||
},
|
||||
{"role": "function", "name": tool_name, "content": tool_output},
|
||||
]
|
||||
return _tail(messages)
|
||||
|
||||
|
||||
def _route_function(tool_name: str, tool_output: str) -> TransformResult:
|
||||
router = ContentRouter(ContentRouterConfig())
|
||||
router.compress = _fake_compress # type: ignore[assignment]
|
||||
return router.apply(
|
||||
_function_conversation(tool_name, tool_output), _tokenizer(), target_ratio=0.4
|
||||
)
|
||||
|
||||
|
||||
def _text_block_conversation(tool_name: str, tool_output: str) -> list[dict[str, object]]:
|
||||
"""OpenAI shape: a tool message whose content is a list of text blocks."""
|
||||
messages: list[dict[str, object]] = [
|
||||
{"role": "system", "content": "You are a coding agent."},
|
||||
{"role": "user", "content": "run the command"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_1",
|
||||
"type": "function",
|
||||
"function": {"name": tool_name, "arguments": "{}"},
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "tool",
|
||||
"tool_call_id": "call_1",
|
||||
"content": [{"type": "text", "text": tool_output}],
|
||||
},
|
||||
]
|
||||
return _tail(messages)
|
||||
|
||||
|
||||
def _route_text_block(tool_name: str, tool_output: str) -> TransformResult:
|
||||
router = ContentRouter(ContentRouterConfig())
|
||||
router.compress = _fake_compress # type: ignore[assignment]
|
||||
return router.apply(
|
||||
_text_block_conversation(tool_name, tool_output), _tokenizer(), target_ratio=0.4
|
||||
)
|
||||
|
||||
|
||||
def _ls_output(rows: int = 18) -> str:
|
||||
files = "\n".join(
|
||||
f"-rw-r--r-- 1 user staff {1000 + i:>6} Oct 22 09:{i:02d} report_{i:02d}_final_v2.md"
|
||||
for i in range(rows)
|
||||
)
|
||||
return (
|
||||
"total 128\n"
|
||||
"drwxr-xr-x 12 user staff 384 Oct 22 09:26 .\n"
|
||||
"drwxr-xr-x 5 user staff 160 Oct 22 09:25 ..\n"
|
||||
f"{files}\n"
|
||||
)
|
||||
|
||||
|
||||
def _padded_output(length: int) -> str:
|
||||
"""A shell-like payload of exactly ``length`` characters."""
|
||||
base = "line: some structured shell output\n"
|
||||
return (base * (length // len(base) + 1))[:length]
|
||||
|
||||
|
||||
def test_small_bash_tool_result_stays_verbatim() -> None:
|
||||
"""A short shell result keeps its structure instead of becoming numbers."""
|
||||
output = _ls_output()
|
||||
assert len(output) < 2000
|
||||
|
||||
result = _route("Bash", output)
|
||||
|
||||
assert result.messages[3]["content"][0]["content"] == output
|
||||
assert "router:shell_small_result_verbatim" in result.transforms_applied
|
||||
|
||||
|
||||
def test_large_bash_tool_result_is_still_compressed() -> None:
|
||||
"""The guard must not disable compression for large build/test logs."""
|
||||
output = _ls_output(rows=80)
|
||||
assert len(output) > 2000
|
||||
|
||||
result = _route("Bash", output)
|
||||
|
||||
assert result.messages[3]["content"][0]["content"] != output
|
||||
assert "router:tool_result:kompress" in result.transforms_applied
|
||||
|
||||
|
||||
def test_small_non_shell_tool_result_is_still_compressed() -> None:
|
||||
"""The guard is scoped to shell tools, not every small tool result."""
|
||||
output = "a short reference payload\n" * 30
|
||||
|
||||
result = _route("SomeOtherTool", output)
|
||||
|
||||
assert result.messages[3]["content"][0]["content"] != output
|
||||
assert "router:tool_result:kompress" in result.transforms_applied
|
||||
|
||||
|
||||
def test_small_openai_tool_result_stays_verbatim() -> None:
|
||||
"""The OpenAI role:tool shape carries the same guard as Anthropic blocks."""
|
||||
output = _ls_output()
|
||||
|
||||
result = _route_openai("Bash", output)
|
||||
|
||||
assert result.messages[3]["content"] == output
|
||||
assert "router:shell_small_result_verbatim" in result.transforms_applied
|
||||
|
||||
|
||||
def test_shell_result_min_chars_can_disable_the_guard(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The escape hatch restores the previous behaviour for operators."""
|
||||
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "0")
|
||||
output = _ls_output()
|
||||
|
||||
result = _route("Bash", output)
|
||||
|
||||
assert result.messages[3]["content"][0]["content"] != output
|
||||
assert "router:tool_result:kompress" in result.transforms_applied
|
||||
|
||||
|
||||
def test_invalid_shell_result_min_chars_falls_back(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""A non-numeric override keeps the documented default instead of crashing."""
|
||||
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "junk")
|
||||
|
||||
assert _shell_result_min_chars() == 2000
|
||||
|
||||
|
||||
def test_zero_shell_result_min_chars_is_legitimate(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Zero is a real value ("no floor"), not an invalid one."""
|
||||
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "0")
|
||||
|
||||
assert _shell_result_min_chars() == 0
|
||||
|
||||
|
||||
def test_negative_shell_result_min_chars_disables_the_guard(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "-1")
|
||||
assert _shell_result_min_chars() == 0
|
||||
result = _route("Bash", _ls_output())
|
||||
assert "router:shell_small_result_verbatim" not in result.transforms_applied
|
||||
|
||||
|
||||
@pytest.mark.parametrize("text", [None, 123, ""])
|
||||
def test_empty_or_non_string_values_are_not_small_shell_results(text: object) -> None:
|
||||
assert not _is_small_shell_result("Bash", text, frozenset({"bash"}))
|
||||
|
||||
|
||||
def test_shell_result_floor_boundary_is_exclusive() -> None:
|
||||
"""The floor is "shorter than", so exactly 2000 chars stays compressible."""
|
||||
at_floor = _padded_output(2000)
|
||||
below_floor = _padded_output(1999)
|
||||
|
||||
at_result = _route("Bash", at_floor)
|
||||
below_result = _route("Bash", below_floor)
|
||||
|
||||
assert at_result.messages[3]["content"][0]["content"] != at_floor
|
||||
assert below_result.messages[3]["content"][0]["content"] == below_floor
|
||||
assert "router:shell_small_result_verbatim" in below_result.transforms_applied
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"tool_name",
|
||||
["Bash", "shell", "local_shell", "run_shell_command", "execute_command"],
|
||||
)
|
||||
def test_recognized_shell_aliases_stay_verbatim(tool_name: str) -> None:
|
||||
"""Harness aliases from the shared tool-name map get the same protection."""
|
||||
output = _ls_output()
|
||||
|
||||
result = _route(tool_name, output)
|
||||
|
||||
assert result.messages[3]["content"][0]["content"] == output
|
||||
assert "router:shell_small_result_verbatim" in result.transforms_applied
|
||||
|
||||
|
||||
def test_unrecognized_shell_like_name_is_still_compressed() -> None:
|
||||
"""A similar-looking name outside the configured set is not protected."""
|
||||
output = _ls_output()
|
||||
|
||||
result = _route("run_remote_shell", output)
|
||||
|
||||
assert result.messages[3]["content"][0]["content"] != output
|
||||
assert "router:tool_result:kompress" in result.transforms_applied
|
||||
|
||||
|
||||
def test_small_list_form_tool_result_stays_verbatim() -> None:
|
||||
"""The Anthropic list-form content wrapper is flattened and protected."""
|
||||
output = _ls_output()
|
||||
|
||||
result = _route("Bash", output, list_form=True)
|
||||
|
||||
assert result.messages[3]["content"][0]["content"] == [{"type": "text", "text": output}]
|
||||
assert "router:shell_small_result_verbatim" in result.transforms_applied
|
||||
|
||||
|
||||
def test_small_legacy_function_result_stays_verbatim() -> None:
|
||||
"""Legacy role:function resolves the shell name from message["name"]."""
|
||||
output = _ls_output()
|
||||
|
||||
result = _route_function("Bash", output)
|
||||
|
||||
assert result.messages[3]["content"] == output
|
||||
assert "router:shell_small_result_verbatim" in result.transforms_applied
|
||||
|
||||
|
||||
def test_small_top_level_text_block_stays_verbatim() -> None:
|
||||
"""A tool message normalized to top-level text blocks gets the guard."""
|
||||
output = _ls_output()
|
||||
|
||||
result = _route_text_block("Bash", output)
|
||||
|
||||
assert result.messages[3]["content"] == [{"type": "text", "text": output}]
|
||||
assert "router:shell_small_result_verbatim" in result.transforms_applied
|
||||
|
||||
|
||||
@pytest.mark.parametrize("role", ["tool", "function"])
|
||||
@pytest.mark.parametrize("length", [1999, 2000, 2400])
|
||||
def test_split_shell_result_uses_total_text_length(role: str, length: int) -> None:
|
||||
"""The floor applies to the whole result, not each provider text fragment."""
|
||||
messages = _text_block_conversation("Bash", "")
|
||||
if role == "function":
|
||||
messages[3] = {"role": "function", "name": "Bash"}
|
||||
blocks = [
|
||||
{"type": "text", "text": _padded_output(length // 2)},
|
||||
{"type": "text", "text": _padded_output(length - length // 2)},
|
||||
]
|
||||
messages[3]["content"] = blocks
|
||||
router = ContentRouter(ContentRouterConfig())
|
||||
router.compress = _fake_compress # type: ignore[assignment]
|
||||
|
||||
result = router.apply(messages, _tokenizer(), target_ratio=0.4)
|
||||
|
||||
assert (result.messages[3]["content"] == blocks) is (length < 2000)
|
||||
assert ("router:shell_small_result_verbatim" in result.transforms_applied) is (length < 2000)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("route", [_route_openai, _route_function, _route_text_block])
|
||||
def test_shell_floor_override_applies_to_wire_shapes(
|
||||
monkeypatch: pytest.MonkeyPatch, route: Callable[[str, str], TransformResult]
|
||||
) -> None:
|
||||
"""A positive override protects below the floor and compresses at it."""
|
||||
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "1000")
|
||||
below = route("Bash", _padded_output(999))
|
||||
at = route("Bash", _padded_output(1000))
|
||||
assert "router:shell_small_result_verbatim" in below.transforms_applied
|
||||
assert "router:shell_small_result_verbatim" not in at.transforms_applied
|
||||
|
||||
|
||||
def test_custom_shell_names_are_normalized_and_protected() -> None:
|
||||
"""An operator's custom mixed-case shell name gets the same floor."""
|
||||
router = ContentRouter(ContentRouterConfig(bash_tool_names=frozenset({"MyShell"})))
|
||||
router.compress = _fake_compress # type: ignore[assignment]
|
||||
output = _ls_output()
|
||||
result = router.apply(_openai_conversation("mYsHeLl", output), _tokenizer(), target_ratio=0.4)
|
||||
assert result.messages[3]["content"] == output
|
||||
assert "router:shell_small_result_verbatim" in result.transforms_applied
|
||||
|
||||
|
||||
@pytest.mark.parametrize("route", [_route_openai, _route_function, _route_text_block])
|
||||
def test_non_shell_results_remain_compressible_in_all_wire_shapes(
|
||||
route: Callable[[str, str], TransformResult],
|
||||
) -> None:
|
||||
output = _ls_output()
|
||||
result = route("SomeOtherTool", output)
|
||||
assert "router:shell_small_result_verbatim" not in result.transforms_applied
|
||||
assert result.messages[3]["content"] not in (output, [{"type": "text", "text": output}])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("route", [_route_openai, _route_function, _route_text_block])
|
||||
def test_shell_floor_can_be_disabled_in_all_wire_shapes(
|
||||
monkeypatch: pytest.MonkeyPatch, route: Callable[[str, str], TransformResult]
|
||||
) -> None:
|
||||
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "0")
|
||||
output = _ls_output()
|
||||
result = route("Bash", output)
|
||||
assert "router:shell_small_result_verbatim" not in result.transforms_applied
|
||||
assert result.messages[3]["content"] not in (output, [{"type": "text", "text": output}])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("conversation", [_openai_conversation, _function_conversation])
|
||||
def test_small_shell_result_diagnostics_explain_why_it_was_preserved(
|
||||
conversation: Callable[[str, str], list[dict[str, object]]],
|
||||
) -> None:
|
||||
output = _ls_output()
|
||||
router = ContentRouter(ContentRouterConfig())
|
||||
router.compress = _fake_compress # type: ignore[assignment]
|
||||
result = router.apply(
|
||||
conversation("Bash", output), _tokenizer(), target_ratio=0.4, collect_diagnostics=True
|
||||
)
|
||||
decision = next(item for item in result.message_decisions if item.message_index == 3)
|
||||
assert decision.action == "protected:shell_small_result"
|
||||
assert decision.tokens_before == decision.tokens_after
|
||||
assert result.messages[3]["content"] == output
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shape", ["anthropic", "tool", "function"])
|
||||
def test_small_shell_blocks_without_optional_route_counters(shape: str) -> None:
|
||||
"""Direct block callers can omit counters without losing the shell guard."""
|
||||
output = _ls_output()
|
||||
if shape == "anthropic":
|
||||
message = _conversation("Bash", output)[3]
|
||||
elif shape == "function":
|
||||
message = {
|
||||
"role": "function",
|
||||
"name": "Bash",
|
||||
"content": [{"type": "text", "text": output}],
|
||||
}
|
||||
else:
|
||||
message = _text_block_conversation("Bash", output)[3]
|
||||
transforms: list[str] = []
|
||||
router = ContentRouter(ContentRouterConfig())
|
||||
router.compress = _fake_compress # type: ignore[assignment]
|
||||
|
||||
result = router._process_content_blocks(
|
||||
message=message,
|
||||
content_blocks=message["content"],
|
||||
context="",
|
||||
transforms_applied=transforms,
|
||||
excluded_tool_ids=set(),
|
||||
ccr_retrieve_tool_ids=set(),
|
||||
tool_name_map={"tool_1": "Bash", "call_1": "Bash"},
|
||||
route_counts=None,
|
||||
messages_from_end=20,
|
||||
)
|
||||
|
||||
assert result is message
|
||||
assert transforms == ["router:shell_small_result_verbatim"]
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.config import DEFAULT_EXCLUDE_TOOLS
|
||||
from headroom.proxy.server import HeadroomProxy, ProxyConfig
|
||||
from headroom.transforms.content_detector import ContentType
|
||||
@@ -159,7 +161,11 @@ def test_web_tool_results_skip_cross_turn_dedup() -> None:
|
||||
assert second["content"] == payload # type: ignore[index]
|
||||
|
||||
|
||||
def test_bash_remains_compressible() -> None:
|
||||
@pytest.mark.parametrize("repetitions, expected_calls", [(100, 0), (200, 1)])
|
||||
def test_bash_compression_respects_small_result_floor(
|
||||
monkeypatch: pytest.MonkeyPatch, repetitions: int, expected_calls: int
|
||||
) -> None:
|
||||
monkeypatch.setenv("HEADROOM_SHELL_RESULT_MIN_CHARS", "2000")
|
||||
router = _router()
|
||||
calls = 0
|
||||
|
||||
@@ -177,8 +183,11 @@ def test_bash_remains_compressible() -> None:
|
||||
)
|
||||
|
||||
router.compress = fake_compress # type: ignore[method-assign]
|
||||
payload = "bash output " * 100
|
||||
payload = "bash output " * repetitions
|
||||
result = router.apply(_messages("Bash", payload), _Tokenizer())
|
||||
|
||||
assert calls == 1
|
||||
assert calls == expected_calls
|
||||
assert "router:excluded:tool" not in result.transforms_applied
|
||||
assert ("router:shell_small_result_verbatim" in result.transforms_applied) is (
|
||||
expected_calls == 0
|
||||
)
|
||||
|
||||
@@ -23,13 +23,15 @@ resolution-markers = [
|
||||
[manifest]
|
||||
constraints = [
|
||||
{ name = "aiohttp", specifier = ">=3.14.4" },
|
||||
{ name = "certifi", specifier = ">=2026.10.19" },
|
||||
{ name = "cryptography", specifier = ">=50.0.0" },
|
||||
{ name = "filelock", marker = "python_full_version >= '3.11'", specifier = ">=4.2.1" },
|
||||
{ name = "gitpython", specifier = ">=3.1.58" },
|
||||
{ name = "langsmith", specifier = ">=0.9.0" },
|
||||
{ name = "lxml-html-clean", specifier = ">=0.4.5" },
|
||||
{ name = "pydantic-settings", specifier = ">=2.14.2" },
|
||||
{ name = "pygments", specifier = ">=2.20.0" },
|
||||
{ name = "setuptools", specifier = ">=83.0.0" },
|
||||
{ name = "setuptools", specifier = ">=84.1.0" },
|
||||
{ name = "transformers", specifier = ">=5.5.0" },
|
||||
]
|
||||
overrides = [{ name = "pillow", specifier = ">=13.0.0" }]
|
||||
@@ -1765,7 +1767,7 @@ requires-dist = [
|
||||
{ name = "openai", marker = "extra == 'proxy'", specifier = ">=2.14.0" },
|
||||
{ name = "openpyxl", marker = "extra == 'dev'", specifier = ">=3.1.0" },
|
||||
{ name = "openpyxl", marker = "extra == 'spreadsheet'", specifier = ">=3.1.0" },
|
||||
{ name = "opentelemetry-api", specifier = ">=1.24.0" },
|
||||
{ name = "opentelemetry-api", specifier = ">=1.45.1" },
|
||||
{ name = "opentelemetry-exporter-otlp-proto-http", marker = "extra == 'dev'", specifier = ">=1.24.0" },
|
||||
{ name = "opentelemetry-exporter-otlp-proto-http", marker = "extra == 'otel'", specifier = ">=1.24.0" },
|
||||
{ name = "opentelemetry-sdk", marker = "extra == 'dev'", specifier = ">=1.24.0" },
|
||||
|
||||
Reference in New Issue
Block a user