fix(proxy): protect Codex exec reads whose argument is a JavaScript object literal (#3737)

## Description

#3621 added read protection for Codex code-mode `exec` calls:
`_custom_tool_call_commands` pulls each `cmd` out of
`tools.exec_command(...)` so a `cat` / `sed -n` / `nl` read reaches the
model verbatim. It JSON-decodes the argument, and skips it when that
fails.

Codex usually writes that argument as a JavaScript object literal with a
bare key, not as JSON:

```js
const r = await tools.exec_command({cmd:"cat /private/tmp/hr-bundle.js","workdir":"/private/tmp"});
const r = await tools.exec_command({ cmd: "sed -n '1,80p' f.py", workdir: "/repo" });
```

Neither is valid JSON, so these reads were never protected and were
lossy-compressed as before #3621. Across 1,125 `exec` calls in real
Codex 0.15x session logs (`~/.codex/sessions`, September 2026), the
current parser returns commands for 129. With this change it returns
commands for 696, and the read output that read protection keeps
verbatim grows from about 50k to 425k tokens.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `content_router._custom_tool_call_commands`: when strict JSON decoding
of the `exec_command` argument fails, read the `cmd` property from the
object literal instead. The key may be bare or quoted; the value must be
a plain `"..."`, `'...'` or backtick string. Simple escapes are
unescaped.
- A template literal with `${...}` substitution, or a `cmd` that is not
a string literal (a variable, an expression), still yields nothing, so
that output stays compressible exactly as before.
- Strict JSON arguments take the existing path unchanged.
- Tests: the parser on five literal shapes (bare key, quoted key, single
quotes, backticks, key not first), the dynamic shapes that must still
yield nothing, and a Codex `exec` read with a bare-key argument kept
verbatim through
`_compress_openai_responses_live_text_units_with_router`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run --frozen --extra dev pytest tests/test_openai_responses_read_protection.py -q
23 passed in 1.50s

# With only the content_router.py change reverted:
FAILED tests/test_openai_responses_read_protection.py::test_custom_tool_call_commands_parses_javascript_object_literals
FAILED tests/test_openai_responses_read_protection.py::test_responses_codex_exec_javascript_literal_read_stays_verbatim
2 failed, 21 passed in 1.01s

$ uv run --frozen --extra dev pytest tests/ -q -k "read_protect or content_router or responses or codex"
1075 passed, 26 skipped, 12245 deselected

$ uvx ruff check headroom/transforms/content_router.py tests/test_openai_responses_read_protection.py
All checks passed!

$ uv run --frozen --extra dev mypy headroom/transforms/content_router.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS 15, Python 3.12, headroom-ai 0.38.0 as installed by
Headroom Desktop, Codex CLI / Codex Desktop 0.15x session logs.
- Exact command / steps: Parsed every `custom_tool_call` in all 102 real
`~/.codex/sessions/2026/09/**/*.jsonl` rollouts (86 contain `exec`
calls) with the current and the patched `_custom_tool_call_commands`,
then summed the output of calls whose command `_is_read_command` accepts
and whose output `_read_output_should_be_protected` keeps. Separately
ran a Codex `exec` read with a bare-key argument through the 0.38.0
wheel's Responses units path with `HEADROOM_PROTECT_READS=1`, with and
without the patched parser.
- Observed result: 129 of 1,125 `exec` calls parsed before, 696 after;
protected read output about 50k tokens before, 425k after. The bare-key
read was compressed without the patch and reached the model
byte-identical with it. A non-read `exec` output in the same payload
still compressed.
- Not tested: Codex versions before 0.15x, and whether fewer lossy reads
reduce Codex re-reads over a full session (the motivation #3621 gives).

## Runtime Rollout Safety

- Rollout-managed feature(s): None. This widens the existing
`HEADROOM_PROTECT_READS` read protection to an argument shape it was
meant to cover.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Yes, when `HEADROOM_PROTECT_READS` is
on: more Codex file-read outputs are kept verbatim instead of
compressed. With it off, nothing changes.
- Kill switch / disable path: `HEADROOM_PROTECT_READS=0` disables read
protection, including this path.
- Unsafe override required: No.
- Qualification impact: Codex requests with file reads compress somewhat
less. Replaying three real Codex coding sessions with read protection
on, the per-session savings rate moved +1.1, -0.4 and 0.0 percentage
points.
- Rollback path: Revert this commit; the parser returns to JSON-only.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not applicable.

## Additional Notes

No documentation change: this is internal to read protection. Follow-up
to #3621.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
gglucass
2026-09-24 10:48:44 -05:00
committed by GitHub
co-authored by Claude Opus 5.5 JerrettDavis
parent d971f7c398
commit ecb5e5afea
3 changed files with 141 additions and 14 deletions
+11 -8
View File
@@ -2195,19 +2195,22 @@ class OpenAIHandlerMixin:
command = _tool_call_command_text(item.get("action"))
elif item_type == "custom_tool_call":
# One script can run several commands; its single output is a
# read when any of them is (over-protecting only costs savings).
# read when any of them is, or when one's cmd is not a string
# literal and so might be (over-protecting only costs savings).
commands = _custom_tool_call_commands(item.get("input"))
command = next(
(
c
for c in _custom_tool_call_commands(item.get("input"))
if _is_read_command(c)
),
"",
(c for c in commands if c is not None and _is_read_command(c)),
"exec_command(<cmd not a string literal>)" if None in commands else "",
)
else:
continue
call_id = item.get("call_id")
if command and isinstance(call_id, str) and call_id and _is_read_command(command):
if (
command
and isinstance(call_id, str)
and call_id
and (item_type == "custom_tool_call" or _is_read_command(command))
):
read_command_by_call_id[call_id] = command
# Outputs protected by read-command detection. Also unioned into the
# cross-turn dedup protection set below: a [↑…] fold of a read would
+45 -4
View File
@@ -572,8 +572,36 @@ def _tool_call_command_text(raw: Any) -> str:
_EXEC_COMMAND_CALL_RE = re.compile(r"\bexec_command\s*\(")
# The `cmd` property of a JavaScript object literal, as Codex code-mode usually
# writes it: `{cmd: "sed -n '1,80p' f.py", workdir: "/repo"}` or
# `{cmd:"cat f","workdir":"/repo"}`. Matched from the opening brace; the key may
# be bare or quoted and the value a "...", '...' or `...` string that is the
# WHOLE property value (the next token ends it: `,` or `}`). Anything else -
# `"c" + "at f"`, `"cat f".trim()`, a template literal with `${...}` - is an
# expression whose value is not the literal and does not match.
_JS_CMD_PROPERTY_RE = re.compile(
r"""\{[^{}]*?(?<![\w$])(?:cmd|"cmd"|'cmd')\s*:\s*"""
r"""(?:"((?:[^"\\\n]|\\.)*)"|'((?:[^'\\\n]|\\.)*)'|`((?:[^`\\$]|\\.|\$(?!\{))*)`)"""
r"""(?=\s*[,}])"""
)
_JS_STRING_ESCAPES = {"n": "\n", "t": "\t", "r": "\r", "b": "\b", "f": "\f", "v": "\v"}
def _custom_tool_call_commands(raw: Any) -> list[str]:
def _js_string_value(body: str) -> str | None:
"""Unescape the body of a JavaScript string literal.
Returns None for an escape it does not decode (``\\x61``, ``\\u0061``, octal),
since guessing would misread ``c\\x61t f`` as something other than ``cat f``.
"""
escaped = re.findall(r"\\(.)", body, flags=re.S)
if any(c.isalnum() and c not in _JS_STRING_ESCAPES for c in escaped):
return None
return re.sub(
r"\\(.)", lambda m: _JS_STRING_ESCAPES.get(m.group(1), m.group(1)), body, flags=re.S
)
def _custom_tool_call_commands(raw: Any) -> list[str | None]:
"""Extract shell commands from a Codex code-mode ``exec`` custom tool call.
Codex sends shell commands as a Responses ``custom_tool_call`` named ``exec``
@@ -582,21 +610,34 @@ def _custom_tool_call_commands(raw: Any) -> list[str]:
const r = await tools.exec_command({"cmd": "sed -n '1,80p' f.py", "workdir": "…"});
text(r.output);
The argument is usually a JavaScript object literal rather than JSON
(``{cmd: "cat f.py"}``, unquoted key); when strict JSON decoding fails, the
``cmd`` property is read from the literal instead.
Returns every ``cmd`` passed to ``exec_command``, in order. Returns ``[]`` when
the input is not that shape; an argument object that is not strict JSON is
skipped, which leaves the output compressible exactly as before.
the input is not that shape. A call whose ``cmd`` is not statically known (a
variable, a concatenation, a template with substitution, an escape this does
not decode) yields ``None``: the command may be a read, and callers must treat
it as one rather than let an incomplete parse release the output.
"""
if not isinstance(raw, str) or "exec_command" not in raw:
return []
decoder = json.JSONDecoder()
commands: list[str] = []
commands: list[str | None] = []
for match in _EXEC_COMMAND_CALL_RE.finditer(raw):
start = raw.find("{", match.end())
if start < 0 or raw[match.end() : start].strip():
commands.append(None) # argument is not an object literal: dynamic
continue
try:
args, _end = decoder.raw_decode(raw, start)
except ValueError:
literal = _JS_CMD_PROPERTY_RE.match(raw, start)
if literal is None:
commands.append(None)
continue
body = next(group for group in literal.groups() if group is not None)
commands.append(_js_string_value(body) or None)
continue
command = _tool_call_command_text(args)
if command:
+85 -2
View File
@@ -577,8 +577,91 @@ def test_custom_tool_call_commands_ignores_other_shapes():
assert _custom_tool_call_commands(None) == []
assert _custom_tool_call_commands({"cmd": "cat f"}) == []
assert _custom_tool_call_commands("*** Begin Patch\n*** Update File: f.py\n") == []
assert _custom_tool_call_commands("tools.exec_command(notJson)") == []
assert _custom_tool_call_commands("tools.exec_command({cmd: 'cat f'})") == []
def test_custom_tool_call_commands_marks_non_literal_cmd_unknown():
"""A cmd that is not a whole string literal may still be a read: None, not skipped."""
from headroom.transforms.content_router import _custom_tool_call_commands
for script in (
"tools.exec_command(notJson)",
"tools.exec_command({cmd: someVariable})",
"tools.exec_command({cmd: `cat ${f}`})",
'tools.exec_command({cmd:"c"+"at f"})',
"tools.exec_command({cmd: 'c' + 'at f', workdir: '/repo'})",
'tools.exec_command({cmd: "cat f".trim()})',
'tools.exec_command({cmd: "c\\x61t f"})',
'tools.exec_command({cmd: "c\\u0061t f"})',
):
assert _custom_tool_call_commands(script) == [None], script
# The known commands around an unknown one are still returned, in order.
assert _custom_tool_call_commands(
'tools.exec_command({cmd: "ls"}); tools.exec_command({cmd: "c" + "at f"});'
) == ["ls", None]
def test_custom_tool_call_commands_parses_javascript_object_literals():
"""Codex usually writes the argument as a JS literal, not JSON (bare `cmd` key)."""
from headroom.transforms.content_router import _custom_tool_call_commands
script = (
'const a = await tools.exec_command({cmd:"cat /tmp/app.js","workdir":"/tmp"});\n'
"const b = await tools.exec_command({cmd: 'sed -n \\'1,80p\\' f.py', workdir: '/repo'});\n"
"const c = await tools.exec_command({ workdir: '/repo', cmd: `nl -ba f.py` });\n"
'const d = await tools.exec_command({cmd: "rg -n \\"def f\\" src", yield_time_ms: 1000});\n'
"const e = await tools.exec_command({'cmd': 'head -n 5 f.py'});\n"
)
assert _custom_tool_call_commands(script) == [
"cat /tmp/app.js",
"sed -n '1,80p' f.py",
"nl -ba f.py",
'rg -n "def f" src',
"head -n 5 f.py",
]
def test_responses_codex_exec_javascript_literal_read_stays_verbatim(monkeypatch):
"""The same read with Codex's usual bare-key argument must also stay verbatim."""
monkeypatch.setenv("HEADROOM_PROTECT_READS", "1")
handler = _handler_with_router(_lossy_router())
output = _codex_exec_output("call_exec", _NL_OUTPUT)
call = {
"type": "custom_tool_call",
"call_id": "call_exec",
"name": "exec",
"input": (
'const r = await tools.exec_command({cmd: "nl -ba tenacity/wait.py | '
'sed -n \'20,115p\'", workdir: "/repo", yield_time_ms: 10000});\n'
"text(r.output);\n"
),
}
payload = {"model": "gpt-5", "input": [call, output]}
new_payload, _modified, _s, _t, _u, _c, _a = _run(handler, payload)
assert new_payload["input"][1] == output
def test_responses_codex_exec_concatenated_cmd_stays_verbatim(monkeypatch):
"""`"c" + "at f"` runs `cat f`: an unparsed cmd must not release the read."""
monkeypatch.setenv("HEADROOM_PROTECT_READS", "1")
handler = _handler_with_router(_lossy_router())
output = _codex_exec_output("call_exec", _NL_OUTPUT)
call = {
"type": "custom_tool_call",
"call_id": "call_exec",
"name": "exec",
"input": (
'const r = await tools.exec_command({cmd: "c" + "at tenacity/wait.py", '
'workdir: "/repo"});\n'
"text(r.output);\n"
),
}
payload = {"model": "gpt-5", "input": [call, output]}
new_payload, _modified, _s, _t, _u, _c, _a = _run(handler, payload)
assert new_payload["input"][1] == output
def test_responses_codex_exec_read_stays_verbatim(monkeypatch):