fix(mcp): escape control characters when rendering TOML server blocks (#3964)

## Description

The Codex and Grok MCP registrars edit the user's `config.toml` in
place, rendering each string value (`command`, `args`, `env`) as a TOML
**basic string** via `_toml_str`. That helper escaped only `\` and `"`.

TOML basic strings forbid *literal control characters other than tab*,
so a value carrying a newline or carriage return — for example a
multi-line PEM private key passed through an env var — rendered as
unparseable TOML. The registrar's write guard (`_only_server_changed`)
then re-parses the proposed file, the parse fails, and the **entire
registration is rejected** with a misleading `does not parse ...
refusing to rewrite it` message that blames the user's config rather
than our serializer. A legitimate server spec simply cannot be
registered.

The fail-safe guard means this never corrupts the file, but it does
block a valid configuration.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `_toml_str` (in `headroom/mcp_registry/codex.py`) now escapes the
remaining control characters per the TOML spec: short escapes for `\b \t
\n \f \r`, and `\uXXXX` for any other C0 control / DEL. `\` and `"` are
still escaped. Tab remains valid either way.
- `grok.py` imports the single escaper from `codex.py` instead of
carrying its own byte-for-byte duplicate, so the two registrars can't
drift (grok already imports other shared helpers from codex).
- Tests: a regression test on each registrar registering an env value
with embedded newlines and asserting it both renders valid TOML and
round-trips through `get_server`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_mcp_registry/test_grok_registrar.py -k escapes_control_chars -q   # before fix
FAILED tests/test_mcp_registry/test_codex_registrar.py::test_register_escapes_control_chars_in_env_value
FAILED tests/test_mcp_registry/test_grok_registrar.py::test_register_escapes_control_chars_in_env_value
2 failed

$ pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_mcp_registry/test_grok_registrar.py -q   # after fix
44 passed

$ ruff check headroom/mcp_registry/codex.py headroom/mcp_registry/grok.py
All checks passed!
```

## Real Behavior Proof

- Environment: macOS (arm64), Python 3.13, local editable install
(`maturin develop --release`). Driven through the real
`CodexRegistrar.register_server` writing an actual
`~/.codex/config.toml` under a temp HOME.
- Exact command / steps: register a `headroom` server whose env `PEM`
value contains embedded newlines, then read it back with `get_server`,
before and after the patch.
- Observed result: The same multiline PEM env value changed from a
rejected registration to a registered server whose read-back value
exactly matched the original, as captured below.
  ```text
  === BEFORE fix ===
  register status: failed
detail: .../.codex/config.toml does not parse, or the Headroom block
holds
entries Headroom could not move out safely; refusing to rewrite it.
  read back env PEM: None
  === AFTER fix ===
  register status: registered
  detail: wrote to .../.codex/config.toml
  read back env PEM: '-----BEGIN KEY-----\nabc\n-----END KEY-----'
  ```
Before the patch a valid multi-line env value could not be registered;
after, it is written as valid TOML and round-trips verbatim.
- Not tested: launching the real Codex/Grok CLIs to consume the written
config (no such value is needed to demonstrate the TOML-validity fix;
round-trip through the registrar's own reader is shown instead).

## Runtime Rollout Safety

- Rollout-managed feature(s): none — local config serialization fix.
- Minimum rollout channel: n/a
- Stable/default behavior changed: only the escaping of control
characters that previously produced invalid TOML; normal specs render
identically.
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

`mypy` not run locally (not set up in this environment); `ruff
check`/`ruff format` are clean. No new dependencies. Grok's duplicate
`_toml_str` is removed in favor of importing codex's, which is a direct
consequence of the fix (one correct escaper instead of two copies), not
a standalone refactor.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3964"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Rajat Nagda
2026-10-05 09:58:51 -05:00
committed by GitHub
co-authored by JD Davis
parent f37ef59189
commit a96154f03b
4 changed files with 76 additions and 8 deletions
+29 -2
View File
@@ -347,10 +347,37 @@ def _render_block(spec: ServerSpec) -> str:
return "\n".join(lines)
# TOML basic strings forbid literal control characters other than tab, so a
# value carrying a newline/carriage-return (e.g. a multi-line PEM key in an
# env var) must be escaped or the rendered block is unparseable TOML. Only
# ``\`` and ``"`` were escaped before, so such a value produced invalid TOML
# and the write guard rejected the whole registration with a misleading
# "file does not parse" error.
_TOML_ESCAPES = {
"\\": "\\\\",
'"': '\\"',
"\b": "\\b",
"\t": "\\t",
"\n": "\\n",
"\f": "\\f",
"\r": "\\r",
}
def _toml_str(s: str) -> str:
"""Render a Python string as a TOML basic string literal."""
escaped = s.replace("\\", "\\\\").replace('"', '\\"')
return f'"{escaped}"'
out: list[str] = []
for ch in s:
escape = _TOML_ESCAPES.get(ch)
if escape is not None:
out.append(escape)
elif ch < "\x20" or ch == "\x7f":
# Remaining C0 controls (and DEL) have no short escape; TOML requires
# the \uXXXX form.
out.append(f"\\u{ord(ch):04X}")
else:
out.append(ch)
return '"' + "".join(out) + '"'
def _entry_to_spec(name: str, entry: dict[str, Any]) -> ServerSpec:
+1 -6
View File
@@ -18,7 +18,7 @@ from typing import Any
from headroom import fsutil
from .base import MCPRegistrar, RegisterResult, RegisterStatus, ServerSpec
from .codex import _evict_foreign_tables, _only_server_changed
from .codex import _evict_foreign_tables, _only_server_changed, _toml_str
if sys.version_info >= (3, 11):
import tomllib
@@ -201,11 +201,6 @@ def _render_block(spec: ServerSpec) -> str:
return "\n".join(lines)
def _toml_str(s: str) -> str:
escaped = s.replace("\\", "\\\\").replace('"', '\\"')
return f'"{escaped}"'
def _entry_to_spec(name: str, entry: dict[str, Any]) -> ServerSpec:
args_value = entry.get("args", [])
if isinstance(args_value, list):
@@ -524,3 +524,23 @@ def test_refuses_span_rewrite_that_would_change_other_entries(
assert reg.unregister_server("headroom") is False
assert reg.register_server(_spec(), force=True).status == RegisterStatus.FAILED
assert cfg.read_text() == before
def test_register_escapes_control_chars_in_env_value(tmp_path: Path) -> None:
"""A newline in an env value must be escaped so the block stays valid TOML.
TOML basic strings forbid literal control characters other than tab, so a
multi-line value (e.g. a PEM key in an env var) rendered with only ``\\``
and ``"`` escaping produced unparseable TOML. The write guard then rejected
the whole registration with a misleading "file does not parse" error.
"""
reg = _make_registrar(tmp_path)
pem = "-----BEGIN KEY-----\nabc\n-----END KEY-----"
result = reg.register_server(_spec(env={"PEM": pem}), force=True)
assert result.status == RegisterStatus.REGISTERED
# The rendered config parses and round-trips the value verbatim.
parsed = tomllib.loads(_config_path(tmp_path).read_text())
assert parsed["mcp_servers"]["headroom"]["env"]["PEM"] == pem
assert reg.get_server("headroom").env["PEM"] == pem
@@ -93,3 +93,29 @@ def test_refuses_span_rewrite_that_would_change_other_entries(
moved = ServerSpec(name="headroom", command="/opt/python", args=_spec().args)
assert reg.register_server(moved, force=True).status == RegisterStatus.FAILED
assert cfg.read_text() == before
def test_register_escapes_control_chars_in_env_value(tmp_path: Path) -> None:
"""Grok shares the codex TOML escaper: a newline env value must round-trip."""
import sys
if sys.version_info >= (3, 11):
import tomllib
else: # pragma: no cover
import tomli as tomllib
reg = _make_registrar(tmp_path)
pem = "-----BEGIN KEY-----\nabc\n-----END KEY-----"
spec = ServerSpec(
name="headroom",
command="/usr/bin/python",
args=("-m", "headroom.cli", "mcp", "serve"),
env={"PEM": pem},
)
result = reg.register_server(spec, force=True)
assert result.status == RegisterStatus.REGISTERED
parsed = tomllib.loads(reg._config_file.read_text())
assert parsed["mcp_servers"]["headroom"]["env"]["PEM"] == pem
assert reg.get_server("headroom").env["PEM"] == pem