fix(proxy): protect Codex exec file reads on the Responses path (#3621)

## Description

Codex CLI 0.153 sends every shell command as a Responses
`custom_tool_call` named `exec`. Its `input` is JavaScript that calls
`tools.exec_command({"cmd": ...})`, not JSON arguments. Read protection
(`HEADROOM_PROTECT_READS`, on in the default `coding` profile) only
parsed `function_call` and `local_shell_call`, so Codex's `sed -n`,
`nl`, and `cat` reads were lossy-compressed. Codex then re-ran searches
and re-read the same lines, which cancelled the per-request saving.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `content_router._custom_tool_call_commands` pulls each `cmd` out of an
`exec` script by JSON-decoding the object passed to `exec_command(`.
Input that is not that shape, or not strict JSON, yields nothing, so the
output stays compressible as before.
- The Responses read scan in `handlers/openai.py` handles
`custom_tool_call`. A script with several commands is treated as a read
when any command is a read, because it produces one output.
- Tests: helper parsing (including braces and quotes inside the
command), a Codex `exec` read kept verbatim with a content-part output,
a multi-command script with a read, and a control where test output
still compresses.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`) on the changed modules
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_openai_responses_read_protection.py
without the fix: 4 failed, 17 passed
with the fix:    21 passed

$ pytest tests/test_openai_responses_read_protection.py tests/test_openai_responses_compression_units.py \
    tests/test_responses_pyo3_compression.py tests/test_compression_units.py tests/test_compression_batches.py \
    tests/test_output_turn_policy.py tests/test_transforms_content_router.py tests/test_proxy
451 passed

$ uvx ruff@0.16.3 check <changed files>; mypy <changed modules>
All checks passed!
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: Codex CLI 0.153.2, gpt-5.6-terra, Headroom proxy with no
license and default settings, a recorder at the OpenAI edge logging
every request's usage.
- Exact command / steps: five SWE-bench-style bug-fix tasks, three runs
each, `codex exec` through the proxy. Compared against Codex direct to
OpenAI.
- Observed result:

| 15 runs each | Direct | Headroom before | Headroom with this fix |
|---|---|---|---|
| Solved | 15/15 | 15/15 | 15/15 |
| Requests per run | 6.1 | 7.3 | 6.1 |
| Cost per run | $0.0766 | $0.0803 | $0.0741 |
| Cost per request | $0.01245 | $0.01103 | $0.01226 |
| Prefix-cache breaks | 0 | 0 | 0 |

Before the fix, Codex re-read files it had just read (4.1 repeat file
reads per run against 2.1 direct), so the 11% per-request saving turned
into a 5% higher cost per run. With the fix, request count matches
direct and cost per run is 3% below direct (within noise at this sample
size).
- Not tested: interactive Codex, and Codex versions that still use
`function_call` shell tools (already covered by existing tests).

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: stable.
- Stable/default behavior changed: yes, Codex `exec` file reads are no
longer lossy-compressed when read protection is on.
- Kill switch / disable path: `HEADROOM_PROTECT_READS=0`.
- Unsafe override required: no.
- Qualification impact: less compression on Codex file reads, fewer
re-reads.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

No docs change: read protection is already documented, and this extends
it to one more wire shape. Codex also reads files with `awk 'NR>=a &&
NR<=b'`, which `_is_read_command` does not treat as a read. That is a
separate follow-up.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tejas Chopra
2026-09-16 22:00:31 -07:00
committed by GitHub
co-authored by Claude Opus 5
parent 35b11564c6
commit 2c56a1b3a7
3 changed files with 154 additions and 0 deletions
+14
View File
@@ -2157,10 +2157,13 @@ class OpenAIHandlerMixin:
# _tool_call_command_text helper:
# - function_call.arguments (Copilot bash, Codex exec_command, …)
# - local_shell_call.action (native Responses shell; argv or string)
# - custom_tool_call.input (Codex code-mode `exec`: JavaScript calling
# tools.exec_command({"cmd": …}))
# Content is gated per-output by _read_output_should_be_protected so
# confidently non-code DATA reads (lockfiles, JSON, logs, search) stay
# compressible, exactly like the chat path.
from headroom.transforms.content_router import (
_custom_tool_call_commands,
_is_read_command,
_read_output_should_be_protected,
_tool_call_command_text,
@@ -2177,6 +2180,17 @@ class OpenAIHandlerMixin:
command = _tool_call_command_text(item.get("arguments"))
elif item_type == "local_shell_call":
command = _tool_call_command_text(item.get("action"))
elif item_type == "custom_tool_call":
# One script can run several commands; its single output is a
# read when any of them is (over-protecting only costs savings).
command = next(
(
c
for c in _custom_tool_call_commands(item.get("input"))
if _is_read_command(c)
),
"",
)
else:
continue
call_id = item.get("call_id")
+34
View File
@@ -567,6 +567,40 @@ def _tool_call_command_text(raw: Any) -> str:
return cmd if isinstance(cmd, str) else ""
_EXEC_COMMAND_CALL_RE = re.compile(r"\bexec_command\s*\(")
def _custom_tool_call_commands(raw: Any) -> list[str]:
"""Extract shell commands from a Codex code-mode ``exec`` custom tool call.
Codex sends shell commands as a Responses ``custom_tool_call`` named ``exec``
whose ``input`` is a JavaScript snippet rather than JSON arguments::
const r = await tools.exec_command({"cmd": "sed -n '1,80p' f.py", "workdir": "…"});
text(r.output);
Returns every ``cmd`` passed to ``exec_command``, in order. Returns ``[]`` when
the input is not that shape; an argument object that is not strict JSON is
skipped, which leaves the output compressible exactly as before.
"""
if not isinstance(raw, str) or "exec_command" not in raw:
return []
decoder = json.JSONDecoder()
commands: list[str] = []
for match in _EXEC_COMMAND_CALL_RE.finditer(raw):
start = raw.find("{", match.end())
if start < 0 or raw[match.end() : start].strip():
continue
try:
args, _end = decoder.raw_decode(raw, start)
except ValueError:
continue
command = _tool_call_command_text(args)
if command:
commands.append(command)
return commands
def _fenced_shell_command(content: Any) -> str:
"""Extract the shell command from a TEXT-BASED agent's fenced code block.
@@ -529,3 +529,109 @@ def test_responses_read_scan_tolerates_non_dict_and_missing_call_id(monkeypatch)
assert modified is True
assert new_payload["input"][0] == "a bare string item"
assert new_payload["input"][3]["output"] == "kept words"
def _codex_exec_call(call_id: str, *commands: str) -> dict:
"""Codex code-mode `exec` custom tool call, as Codex 0.15x sends it."""
import json as _json
script = "".join(
"const r{i} = await tools.exec_command({args});\ntext(r{i}.output);\n".format(
i=i,
args=_json.dumps({"cmd": cmd, "workdir": "/repo", "yield_time_ms": 10000}),
)
for i, cmd in enumerate(commands)
)
return {"type": "custom_tool_call", "call_id": call_id, "name": "exec", "input": script}
def _codex_exec_output(call_id: str, text: str) -> dict:
return {
"type": "custom_tool_call_output",
"call_id": call_id,
"output": [
{"type": "input_text", "text": "Script completed\nWall time 0.1 seconds\nOutput:\n"},
{"type": "input_text", "text": text},
],
}
def test_custom_tool_call_commands_parses_codex_exec_input():
from headroom.transforms.content_router import _custom_tool_call_commands
call = _codex_exec_call("c", "sed -n '1,80p' tenacity/wait.py", "rg -n 'def f' src")
assert _custom_tool_call_commands(call["input"]) == [
"sed -n '1,80p' tenacity/wait.py",
"rg -n 'def f' src",
]
# Braces and quotes inside the command must not break the JSON scan.
awk = _codex_exec_call("c", "awk 'NR >= 3 {printf \"%4d: %s\\n\", NR, $0}' f.py")
assert _custom_tool_call_commands(awk["input"]) == [
"awk 'NR >= 3 {printf \"%4d: %s\\n\", NR, $0}' f.py"
]
def test_custom_tool_call_commands_ignores_other_shapes():
from headroom.transforms.content_router import _custom_tool_call_commands
assert _custom_tool_call_commands(None) == []
assert _custom_tool_call_commands({"cmd": "cat f"}) == []
assert _custom_tool_call_commands("*** Begin Patch\n*** Update File: f.py\n") == []
assert _custom_tool_call_commands("tools.exec_command(notJson)") == []
assert _custom_tool_call_commands("tools.exec_command({cmd: 'cat f'})") == []
def test_responses_codex_exec_read_stays_verbatim(monkeypatch):
"""Codex's `exec` custom tool: a sed/nl file read must reach the model verbatim."""
monkeypatch.setenv("HEADROOM_PROTECT_READS", "1")
handler = _handler_with_router(_lossy_router())
output = _codex_exec_output("call_exec", _NL_OUTPUT)
payload = {
"model": "gpt-5",
"input": [
_codex_exec_call("call_exec", "nl -ba tenacity/wait.py | sed -n '20,115p'"),
output,
],
}
new_payload, _modified, _s, _t, _u, _c, _a = _run(handler, payload)
assert new_payload["input"][1] == output
def test_responses_codex_exec_script_with_a_read_among_commands_stays_verbatim(monkeypatch):
"""One script, one output: protect it when any command in the script is a read."""
monkeypatch.setenv("HEADROOM_PROTECT_READS", "1")
handler = _handler_with_router(_lossy_router())
output = _codex_exec_output("call_multi", _NL_OUTPUT)
payload = {
"model": "gpt-5",
"input": [
_codex_exec_call(
"call_multi", "git status --short", "sed -n '1,130p' tenacity/wait.py"
),
output,
],
}
new_payload, _modified, _s, _t, _u, _c, _a = _run(handler, payload)
assert new_payload["input"][1] == output
def test_responses_codex_exec_test_output_still_compresses(monkeypatch):
"""Control: a Codex `exec` running tests is not a read and stays compressible."""
monkeypatch.setenv("HEADROOM_PROTECT_READS", "1")
handler = _handler_with_router(_lossy_router())
payload = {
"model": "gpt-5",
"input": [
_codex_exec_call("call_pytest", "python3 -m pytest -q tests/test_wait.py"),
_codex_exec_output("call_pytest", _NL_OUTPUT),
],
}
new_payload, modified, _s, _t, _u, _c, _a = _run(handler, payload)
assert modified is True
assert new_payload["input"][1] != _codex_exec_output("call_pytest", _NL_OUTPUT)