Files
headroom/tests/test_proxy/test_log_model_id.py
b9f6255395 fix(proxy): log client-supplied model ids as one escaped token (#4854)
## Description

The Anthropic handler logged the request's `model` field raw in three
places (`RESPONSE-CACHE-HIT`, the `cache_mode_prefix_mismatch` skip
line, `UPSTREAM_ERROR`). `sanitize_anthropic_model_id` strips ANSI
styling but keeps an interior newline, so a model id with a newline
split one log record into two, and the second line could look like any
other record. The OpenAI handler had the same raw `model=` in its
cache-hit, upstream-error, `/v1/responses` passthrough (HTTP and WS) and
token-recount lines, and the Bedrock handler logged the model id from
the URL path the same way.

All of them now go through `headroom.log_safety.safe_id`, the helper
`providers/openai.py` already uses for model ids: the value is rendered
with `repr` (newlines escaped) and capped in length.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/anthropic.py`: `safe_id(model)` in the
cache-hit, prefix-mismatch and upstream-error log lines.
- `headroom/proxy/handlers/openai.py`: `safe_id(...)` in the cache-hit,
upstream-error, `/v1/responses` HTTP and WS passthrough, and raw-payload
recount log lines.
- `headroom/proxy/handlers/bedrock.py`: `safe_id(model_id)` in the
compression log line.
- `tests/test_proxy/test_log_model_id.py`: drives `/v1/messages` and
`/v1/chat/completions` through `create_app` with a model id that
contains a newline and checks the `UPSTREAM_ERROR` and
`RESPONSE-CACHE-HIT` records stay on one line.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# new test, with the fix
$ pytest -q tests/test_proxy/test_log_model_id.py
3 passed in 9.18s

# same test with the handler changes stashed (origin/main code)
FAILED tests/test_proxy/test_log_model_id.py::test_upstream_error_log_keeps_model_on_one_line[/v1/messages-headers0]
FAILED tests/test_proxy/test_log_model_id.py::test_upstream_error_log_keeps_model_on_one_line[/v1/chat/completions-headers1]
FAILED tests/test_proxy/test_log_model_id.py::test_anthropic_cache_hit_log_keeps_model_on_one_line
3 failed in 9.36s

# area tests
$ pytest -q tests/test_proxy tests/test_bedrock_*.py tests/test_log_safety.py
738 passed, 1 warning in 251.77s

$ ruff check / ruff format --check (0.17.3) on the 4 changed files: All checks passed, 4 files already formatted
$ mypy headroom/proxy/handlers/{anthropic,openai,bedrock}.py --ignore-missing-imports: Success: no issues found in 3 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.12, the real `headroom proxy` CLI
(`--stateless --no-telemetry`, isolated `HOME`) with
`--anthropic-api-url` pointed at a local HTTP server that answers every
request with an Anthropic-format 400. No mocks in the proxy process.
- Exact command / steps: `curl -X POST
http://127.0.0.1:<port>/v1/messages` with `"model": "claude-x\nINJECTED
fake log line"`; grep the proxy's stderr.
- Observed result:
- origin/main: `UPSTREAM_ERROR status=400 ... model=claude-x` followed
by a forged line starting `INJECTED fake log line compressed=no ...`;
the provider warning `Unknown Anthropic model 'claude-x` is also split,
with `INJECTED fake log line': using default Claude limit ...` on its
own line.
- this branch: `UPSTREAM_ERROR status=400 ... model='claude-x\nINJECTED
fake log line' compressed=no ...` and `Unknown Anthropic model
'claude-x\nINJECTED fake log line': using default Claude limit ...`,
each on one line; no line starts with the injected text.
- The live run found the provider warning, which the handler tests did
not cover; fixed in the second commit with its own regression test.
- Not tested: a live provider call. Separately observed on both trees
and not changed here: handlers echo the raw model into the
`x-headroom-model` response header, so a model id containing a newline
makes uvicorn/h11 drop the connection for that request.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: log format only: `model=` values are
now quoted (`model='claude-sonnet-4-6'`).
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none; anything that greps these lines for
`model=<id>` sees the id in quotes.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Fix adapted from a proposal by HetCreep.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4854"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-14 10:46:11 +05:30

152 lines
4.7 KiB
Python

"""A client-supplied model id is logged as one token, never as raw text.
``sanitize_anthropic_model_id`` keeps an interior newline, so the handlers must
render the model with ``safe_id`` wherever they log it.
"""
from __future__ import annotations
import json
import logging
from datetime import datetime
from unittest.mock import AsyncMock, patch
import pytest
fastapi = pytest.importorskip("fastapi")
httpx = pytest.importorskip("httpx")
from fastapi.testclient import TestClient # noqa: E402
from headroom.proxy.models import CacheEntry # noqa: E402
from headroom.proxy.server import ProxyConfig, create_app # noqa: E402
MULTILINE_MODEL = "claude-x\n2026-01-01 00:00:00 INFO headroom.other second-line"
class _Capture(logging.Handler):
def __init__(self) -> None:
super().__init__(level=logging.INFO)
self.messages: list[str] = []
def emit(self, record: logging.LogRecord) -> None:
self.messages.append(record.getMessage())
@pytest.fixture
def proxy_logs():
target = logging.getLogger("headroom.proxy")
handler = _Capture()
previous = target.level
target.addHandler(handler)
target.setLevel(logging.INFO)
try:
yield handler
finally:
target.removeHandler(handler)
target.setLevel(previous)
def _config(**overrides) -> ProxyConfig:
base = {
"optimize": False,
"cache_enabled": False,
"rate_limit_enabled": False,
"cost_tracking_enabled": False,
"log_requests": False,
"image_optimize": False,
}
base.update(overrides)
return ProxyConfig(**base)
def _error_response() -> httpx.Response:
return httpx.Response(
400,
content=json.dumps({"error": {"type": "invalid_request_error", "message": "bad"}}).encode(),
headers={"content-type": "application/json"},
)
def _assert_single_line(handler: _Capture, marker: str) -> None:
lines = [m for m in handler.messages if marker in m]
assert lines, f"{marker} was not logged"
for line in lines:
assert "\n" not in line
assert "claude-x\\n2026" in line
@pytest.mark.parametrize(
("path", "headers"),
[
("/v1/messages", {"x-api-key": "test-key", "anthropic-version": "2023-06-01"}),
("/v1/chat/completions", {"authorization": "Bearer test-key"}),
],
)
def test_upstream_error_log_keeps_model_on_one_line(proxy_logs, path, headers):
body = {
"model": MULTILINE_MODEL,
"max_tokens": 16,
"messages": [{"role": "user", "content": "hi"}],
}
with patch("headroom.proxy.server.AnyLLMBackend"):
app = create_app(_config())
with TestClient(app) as client:
client.app.state.proxy._retry_request = AsyncMock(return_value=_error_response())
client.post(path, headers=headers, json=body)
_assert_single_line(proxy_logs, "UPSTREAM_ERROR")
def test_anthropic_cache_hit_log_keeps_model_on_one_line(proxy_logs):
entry = CacheEntry(
response_body=json.dumps(
{
"id": "msg_cached",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-4-6",
"content": [{"type": "text", "text": "cached"}],
"stop_reason": "end_turn",
"usage": {"input_tokens": 1, "output_tokens": 1},
}
).encode(),
response_headers={},
created_at=datetime.now(),
ttl_seconds=3600,
)
body = {
"model": MULTILINE_MODEL,
"max_tokens": 16,
"messages": [{"role": "user", "content": "hi"}],
}
with patch("headroom.proxy.server.AnyLLMBackend"):
app = create_app(_config(cache_enabled=True))
with TestClient(app) as client:
proxy = client.app.state.proxy
proxy.cache.get = AsyncMock(return_value=entry)
proxy._retry_request = AsyncMock(side_effect=AssertionError("no upstream on a hit"))
client.post(
"/v1/messages",
headers={"x-api-key": "test-key", "anthropic-version": "2023-06-01"},
json=body,
)
_assert_single_line(proxy_logs, "RESPONSE-CACHE-HIT")
def test_unknown_anthropic_model_warning_logs_model_as_one_token() -> None:
from headroom.providers.anthropic import AnthropicProvider
target = logging.getLogger("headroom.providers.anthropic")
handler = _Capture()
target.addHandler(handler)
try:
AnthropicProvider().get_context_limit(MULTILINE_MODEL + " provider-warning")
finally:
target.removeHandler(handler)
[message] = [m for m in handler.messages if "Unknown Anthropic model" in m]
assert "\n" not in message
assert repr(MULTILINE_MODEL + " provider-warning") in message