fix(proxy): copy cached tool compaction results (hardening) (#4410)

## Description

> **Hardening, preventive.** No request sequence was found where this
changes what reaches the upstream or the client; the fix removes shared
mutable state before a future caller can depend on it. Please treat it
as low priority.

`compact_tools` and `compact_tool_descriptions` cache the compacted
`tools` list at module level and then put that same list into the
payload they return. Callers edit the tools in place afterwards
(`inject_tool_search_deferral` sets `cache_control` on a resident tool,
`helpers.py:4235`), so the edit stays in the cache and shows up in later
requests with the same digest. The returned payload now gets a deep copy
of the cached list.

Fixes #4403

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/tool_schema_compaction.py`: `updated["tools"] =
copy.deepcopy(compacted_tools)` at the four places in `compact_tools`
and `compact_tool_descriptions` (cache hit and cache miss in each). The
cache still stores the original list.
- `tests/test_tool_schema_compaction.py`:
`TestCompactedToolsAreNotShared`, one test per function: edit the
returned tool, then call again twice (a hit) and assert the edit is not
visible.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# new tests, source change reverted
FAILED tests/test_tool_schema_compaction.py::TestCompactedToolsAreNotShared::test_compact_tools_edit_does_not_leak_into_later_requests
FAILED tests/test_tool_schema_compaction.py::TestCompactedToolsAreNotShared::test_compact_tool_descriptions_edit_does_not_leak
2 failed, 27 passed

# with the fix
$ pytest tests/test_tool_schema_compaction.py
29 passed

$ pytest tests/test_tool_schema_compaction.py tests/test_openai_chat_tool_desc_compaction.py tests/test_anthropic_compaction_transforms.py
44 passed, 1 warning

$ ruff check headroom/proxy/tool_schema_compaction.py tests/test_tool_schema_compaction.py
All checks passed!
$ ruff format --check headroom/proxy/tool_schema_compaction.py tests/test_tool_schema_compaction.py
2 files already formatted
```

## Real Behavior Proof

- Environment: Ubuntu on WSL2, Python 3.12.15, upstream/main e995eb9f
plus this commit. `headroom/_core.abi3.so` copied (untracked, not
committed) from the official `headroom_ai-0.41.0` wheel because the
checkout has no built Rust core. "Before" = the source file with this
commit's change reverted (`git apply -R`).
- Exact command / steps: (1) start a stub upstream on 127.0.0.1:18870
and the real proxy `HEADROOM_TOOL_DESC_MAX_CHARS=60
ANTHROPIC_TARGET_API_URL=http://127.0.0.1:18870 python -P -m
headroom.proxy.server --host 127.0.0.1 --port 18871 --no-cache
--no-rate-limit`, send the same 14-tool request twice, record what the
stub receives; (2) in the same checkout, call the shipped functions in
the handler's order (`compact_tools`, then
`inject_tool_search_deferral`) and compact the same client request
again.
- Observed result: over HTTP the stub receives the same bodies before
and after (compaction still applies, nothing else changed), because the
proxy does not run tool-search deferral against a custom upstream URL.
In-process, before the fix the second identical request gets a
`cache_control` on `Bash` that the client never sent and shares tool
dicts with the first; after the fix it does not.
- Not tested: the leak through a real Anthropic upstream (tool-search
deferral is skipped for a non-Anthropic base URL, so the HTTP path
cannot show it); concurrent requests; the OpenAI and gateway callers;
`mypy`.

HTTP, before and after (stub log, identical):

```text
client request 1 HTTP 200
client request 2 HTTP 200
STUB tools=15 first_desc='Retrieve original uncompressed content that was compressed t…' cache_control_on=[]
STUB tools=15 first_desc='Retrieve original uncompressed content that was compressed t…' cache_control_on=[]
```

In-process, before (change reverted):

```text
request 1: compacted tools carry cache_control on: ['tool_3']
request 1: after tool-search deferral, tools carry cache_control on: ['Bash']
request 2 (fresh, identical client input): compacted tools carry cache_control on: ['tool_3', 'Bash']
request 1 and 2 share tool dicts: True
```

In-process, after (this commit):

```text
request 1: compacted tools carry cache_control on: ['tool_3']
request 1: after tool-search deferral, tools carry cache_control on: ['Bash']
request 2 (fresh, identical client input): compacted tools carry cache_control on: ['tool_3']
request 1 and 2 share tool dicts: False
```

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: no; returned tool content is equal,
only the object identity changes
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none known
- Rollback path: revert the commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Only the listed test files were run, not the whole suite. The deep copy
costs one extra copy of the tools list per call on both the hit and miss
path. I did not find a request sequence where the leak changes the body
sent upstream (same digest means the same client markers); the fix
removes the shared mutable state. Produced with AI assistance (Claude
Code) operated by me (HetCreep); the reproduction was run on my machine
on a clone of the repository.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4410"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
HetCreep
2026-10-24 08:44:16 -07:00
committed by GitHub
co-authored by Claude Opus 5.5 JerrettDavis
parent 3f941446bf
commit a25623109d
2 changed files with 47 additions and 4 deletions
+8 -4
View File
@@ -259,7 +259,8 @@ def compact_tools(
if after >= before:
return payload, False, before, after
updated = copy.deepcopy(payload)
updated["tools"] = compacted_tools
# deepcopy: the cached list is shared across requests; callers mutate tools in place
updated["tools"] = copy.deepcopy(compacted_tools)
return updated, True, before, after
compacted_tools = compact_tool_schema_value(tools)
@@ -271,7 +272,8 @@ def compact_tools(
return payload, False, before, after
updated = copy.deepcopy(payload)
updated["tools"] = compacted_tools
# deepcopy: the cached list is shared across requests; callers mutate tools in place
updated["tools"] = copy.deepcopy(compacted_tools)
return updated, True, before, after
@@ -420,7 +422,8 @@ def compact_tool_descriptions(
if after >= before:
return payload, False, before, after
updated = copy.deepcopy(payload)
updated["tools"] = compacted_tools
# deepcopy: the cached list is shared across requests; callers mutate tools in place
updated["tools"] = copy.deepcopy(compacted_tools)
return updated, True, before, after
compacted_tools = _truncate_descriptions_in_schema(tools, max_chars, strip_sem)
@@ -432,5 +435,6 @@ def compact_tool_descriptions(
return payload, False, before, after
updated = copy.deepcopy(payload)
updated["tools"] = compacted_tools
# deepcopy: the cached list is shared across requests; callers mutate tools in place
updated["tools"] = copy.deepcopy(compacted_tools)
return updated, True, before, after
+39
View File
@@ -469,3 +469,42 @@ class TestCompactToolDescriptions:
assert modified is True
savings_pct = (1 - after / before) * 100
assert savings_pct >= 10, f"Expected ≥10% savings, got {savings_pct:.1f}%"
class TestCompactedToolsAreNotShared:
"""Results must not alias the module-level cache (callers mutate tools in place)."""
@staticmethod
def _payload() -> dict:
return {
"tools": [
{
"name": "alias_probe",
"description": "Probe. " + "Extra words. " * 40,
"input_schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "Params",
"type": "object",
"properties": {"x": {"type": "integer", "title": "X"}},
},
}
]
}
def test_compact_tools_edit_does_not_leak_into_later_requests(self) -> None:
first, modified, _, _ = compact_tools(self._payload())
assert modified is True
first["tools"][-1]["cache_control"] = {"type": "ephemeral"}
for _ in range(2): # second call after the edit, and a cache-hit call
later, _, _, _ = compact_tools(self._payload())
assert "cache_control" not in later["tools"][-1]
def test_compact_tool_descriptions_edit_does_not_leak(self) -> None:
from headroom.proxy.tool_schema_compaction import compact_tool_descriptions
first, modified, _, _ = compact_tool_descriptions(self._payload(), max_chars=40)
assert modified is True
first["tools"][-1]["cache_control"] = {"type": "ephemeral"}
for _ in range(2):
later, _, _, _ = compact_tool_descriptions(self._payload(), max_chars=40)
assert "cache_control" not in later["tools"][-1]