mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(mcp): escape control characters when rendering TOML server blocks (#3964)
## Description The Codex and Grok MCP registrars edit the user's `config.toml` in place, rendering each string value (`command`, `args`, `env`) as a TOML **basic string** via `_toml_str`. That helper escaped only `\` and `"`. TOML basic strings forbid *literal control characters other than tab*, so a value carrying a newline or carriage return — for example a multi-line PEM private key passed through an env var — rendered as unparseable TOML. The registrar's write guard (`_only_server_changed`) then re-parses the proposed file, the parse fails, and the **entire registration is rejected** with a misleading `does not parse ... refusing to rewrite it` message that blames the user's config rather than our serializer. A legitimate server spec simply cannot be registered. The fail-safe guard means this never corrupts the file, but it does block a valid configuration. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `_toml_str` (in `headroom/mcp_registry/codex.py`) now escapes the remaining control characters per the TOML spec: short escapes for `\b \t \n \f \r`, and `\uXXXX` for any other C0 control / DEL. `\` and `"` are still escaped. Tab remains valid either way. - `grok.py` imports the single escaper from `codex.py` instead of carrying its own byte-for-byte duplicate, so the two registrars can't drift (grok already imports other shared helpers from codex). - Tests: a regression test on each registrar registering an env value with embedded newlines and asserting it both renders valid TOML and round-trips through `get_server`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_mcp_registry/test_grok_registrar.py -k escapes_control_chars -q # before fix FAILED tests/test_mcp_registry/test_codex_registrar.py::test_register_escapes_control_chars_in_env_value FAILED tests/test_mcp_registry/test_grok_registrar.py::test_register_escapes_control_chars_in_env_value 2 failed $ pytest tests/test_mcp_registry/test_codex_registrar.py tests/test_mcp_registry/test_grok_registrar.py -q # after fix 44 passed $ ruff check headroom/mcp_registry/codex.py headroom/mcp_registry/grok.py All checks passed! ``` ## Real Behavior Proof - Environment: macOS (arm64), Python 3.13, local editable install (`maturin develop --release`). Driven through the real `CodexRegistrar.register_server` writing an actual `~/.codex/config.toml` under a temp HOME. - Exact command / steps: register a `headroom` server whose env `PEM` value contains embedded newlines, then read it back with `get_server`, before and after the patch. - Observed result: The same multiline PEM env value changed from a rejected registration to a registered server whose read-back value exactly matched the original, as captured below. ```text === BEFORE fix === register status: failed detail: .../.codex/config.toml does not parse, or the Headroom block holds entries Headroom could not move out safely; refusing to rewrite it. read back env PEM: None === AFTER fix === register status: registered detail: wrote to .../.codex/config.toml read back env PEM: '-----BEGIN KEY-----\nabc\n-----END KEY-----' ``` Before the patch a valid multi-line env value could not be registered; after, it is written as valid TOML and round-trips verbatim. - Not tested: launching the real Codex/Grok CLIs to consume the written config (no such value is needed to demonstrate the TOML-validity fix; round-trip through the registrar's own reader is shown instead). ## Runtime Rollout Safety - Rollout-managed feature(s): none — local config serialization fix. - Minimum rollout channel: n/a - Stable/default behavior changed: only the escaping of control characters that previously produced invalid TOML; normal specs render identically. - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none - Rollback path: revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes `mypy` not run locally (not set up in this environment); `ruff check`/`ruff format` are clean. No new dependencies. Grok's duplicate `_toml_str` is removed in favor of importing codex's, which is a direct consequence of the fix (one correct escaper instead of two copies), not a standalone refactor. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3964" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
@@ -347,10 +347,37 @@ def _render_block(spec: ServerSpec) -> str:
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# TOML basic strings forbid literal control characters other than tab, so a
|
||||
# value carrying a newline/carriage-return (e.g. a multi-line PEM key in an
|
||||
# env var) must be escaped or the rendered block is unparseable TOML. Only
|
||||
# ``\`` and ``"`` were escaped before, so such a value produced invalid TOML
|
||||
# and the write guard rejected the whole registration with a misleading
|
||||
# "file does not parse" error.
|
||||
_TOML_ESCAPES = {
|
||||
"\\": "\\\\",
|
||||
'"': '\\"',
|
||||
"\b": "\\b",
|
||||
"\t": "\\t",
|
||||
"\n": "\\n",
|
||||
"\f": "\\f",
|
||||
"\r": "\\r",
|
||||
}
|
||||
|
||||
|
||||
def _toml_str(s: str) -> str:
|
||||
"""Render a Python string as a TOML basic string literal."""
|
||||
escaped = s.replace("\\", "\\\\").replace('"', '\\"')
|
||||
return f'"{escaped}"'
|
||||
out: list[str] = []
|
||||
for ch in s:
|
||||
escape = _TOML_ESCAPES.get(ch)
|
||||
if escape is not None:
|
||||
out.append(escape)
|
||||
elif ch < "\x20" or ch == "\x7f":
|
||||
# Remaining C0 controls (and DEL) have no short escape; TOML requires
|
||||
# the \uXXXX form.
|
||||
out.append(f"\\u{ord(ch):04X}")
|
||||
else:
|
||||
out.append(ch)
|
||||
return '"' + "".join(out) + '"'
|
||||
|
||||
|
||||
def _entry_to_spec(name: str, entry: dict[str, Any]) -> ServerSpec:
|
||||
|
||||
@@ -18,7 +18,7 @@ from typing import Any
|
||||
from headroom import fsutil
|
||||
|
||||
from .base import MCPRegistrar, RegisterResult, RegisterStatus, ServerSpec
|
||||
from .codex import _evict_foreign_tables, _only_server_changed
|
||||
from .codex import _evict_foreign_tables, _only_server_changed, _toml_str
|
||||
|
||||
if sys.version_info >= (3, 11):
|
||||
import tomllib
|
||||
@@ -201,11 +201,6 @@ def _render_block(spec: ServerSpec) -> str:
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _toml_str(s: str) -> str:
|
||||
escaped = s.replace("\\", "\\\\").replace('"', '\\"')
|
||||
return f'"{escaped}"'
|
||||
|
||||
|
||||
def _entry_to_spec(name: str, entry: dict[str, Any]) -> ServerSpec:
|
||||
args_value = entry.get("args", [])
|
||||
if isinstance(args_value, list):
|
||||
|
||||
@@ -524,3 +524,23 @@ def test_refuses_span_rewrite_that_would_change_other_entries(
|
||||
assert reg.unregister_server("headroom") is False
|
||||
assert reg.register_server(_spec(), force=True).status == RegisterStatus.FAILED
|
||||
assert cfg.read_text() == before
|
||||
|
||||
|
||||
def test_register_escapes_control_chars_in_env_value(tmp_path: Path) -> None:
|
||||
"""A newline in an env value must be escaped so the block stays valid TOML.
|
||||
|
||||
TOML basic strings forbid literal control characters other than tab, so a
|
||||
multi-line value (e.g. a PEM key in an env var) rendered with only ``\\``
|
||||
and ``"`` escaping produced unparseable TOML. The write guard then rejected
|
||||
the whole registration with a misleading "file does not parse" error.
|
||||
"""
|
||||
reg = _make_registrar(tmp_path)
|
||||
pem = "-----BEGIN KEY-----\nabc\n-----END KEY-----"
|
||||
|
||||
result = reg.register_server(_spec(env={"PEM": pem}), force=True)
|
||||
|
||||
assert result.status == RegisterStatus.REGISTERED
|
||||
# The rendered config parses and round-trips the value verbatim.
|
||||
parsed = tomllib.loads(_config_path(tmp_path).read_text())
|
||||
assert parsed["mcp_servers"]["headroom"]["env"]["PEM"] == pem
|
||||
assert reg.get_server("headroom").env["PEM"] == pem
|
||||
|
||||
@@ -93,3 +93,29 @@ def test_refuses_span_rewrite_that_would_change_other_entries(
|
||||
moved = ServerSpec(name="headroom", command="/opt/python", args=_spec().args)
|
||||
assert reg.register_server(moved, force=True).status == RegisterStatus.FAILED
|
||||
assert cfg.read_text() == before
|
||||
|
||||
|
||||
def test_register_escapes_control_chars_in_env_value(tmp_path: Path) -> None:
|
||||
"""Grok shares the codex TOML escaper: a newline env value must round-trip."""
|
||||
import sys
|
||||
|
||||
if sys.version_info >= (3, 11):
|
||||
import tomllib
|
||||
else: # pragma: no cover
|
||||
import tomli as tomllib
|
||||
|
||||
reg = _make_registrar(tmp_path)
|
||||
pem = "-----BEGIN KEY-----\nabc\n-----END KEY-----"
|
||||
spec = ServerSpec(
|
||||
name="headroom",
|
||||
command="/usr/bin/python",
|
||||
args=("-m", "headroom.cli", "mcp", "serve"),
|
||||
env={"PEM": pem},
|
||||
)
|
||||
|
||||
result = reg.register_server(spec, force=True)
|
||||
|
||||
assert result.status == RegisterStatus.REGISTERED
|
||||
parsed = tomllib.loads(reg._config_file.read_text())
|
||||
assert parsed["mcp_servers"]["headroom"]["env"]["PEM"] == pem
|
||||
assert reg.get_server("headroom").env["PEM"] == pem
|
||||
|
||||
Reference in New Issue
Block a user