mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 18:41:07 +08:00
fix/plugin-launcher-isolated-python
1
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
861e94d8c1 |
fix(proxy): keep exception text and the proxy token out of client output (#3851)
## Why
Two leaks of internal detail to clients and logs (findings 01-F4 and
10-F4).
- When an upstream connection failed, several response paths returned
`str(e)` from `httpx`, `ssl` or the OS in 500/502/503 bodies and SSE
`error` events. That text names the resolved upstream host, its address
and the failing library. A client that can steer the upstream
(`x-headroom-base-url`) could learn internal hostnames one error at a
time.
- `extract_tags` copied every `x-headroom-*` header into
`RequestOutcome.tags`, including `x-headroom-proxy-token`, which
gateways send on every turn. The token then appeared in the request log
(`--log-file`), dashboard facets and any export that carries tags.
## What changes
- New module `headroom/proxy/public_errors.py`. `classify_exception()`
maps transport and OS failures to fixed codes: `upstream_unreachable`,
`upstream_timeout`, `upstream_tls_error`, `upstream_protocol_error`;
anything else is `internal_error`. Every client- or model-visible path
fails closed: unrecognised exception text (`RuntimeError`, `ValueError`,
parser, database or library errors) never leaves the proxy. Provider
error text is forwarded only when the exception carries a provider HTTP
error response (`openai.APIStatusError`, which LiteLLM's provider errors
subclass, or `anthropic.APIStatusError`). Helpers build the existing
wire shapes with the same `error.type`, plus `code` and `request_id`.
The full exception stays in the server log under that request id.
- Every site that built a client-visible message from an exception now
uses it: `handlers/streaming.py` (direct, Bedrock and OpenAI-via-backend
SSE), `handlers/openai.py` (backend 500, `/v1/compress` 503, both
passthrough connect paths), `handlers/anthropic.py` (Bedrock backend
500), `handlers/bedrock.py` (connect 502), the memory tool handlers
(tool results returned to the model), and the `litellm` / `anyllm`
backends. The existing corporate-TLS hint is kept.
- `internal_header_policy.py` gains `is_credential_header()` and
`is_credential_tag_key()`. They match whole name components (`token`,
`secret`, `password`, `authorization`, `cookie`, `apikey`, `key`, and
similar), so `X-Headroom-Key` and `X-Headroom-Api-Key` are covered, so
numeric tags such as `tool-search-deferred-tokens` are not caught.
- `extract_tags` skips credential headers.
`savings_attribution.public_tags`, the last step before the request log,
also drops credential-shaped keys.
## Behaviour change
- Transport-layer and internal failures now return a fixed sentence plus
`code` and `request_id`. Clients that parsed the old free-text messages
should read `error.code`. Provider HTTP error responses (and their
status codes) are unchanged. In the LiteLLM / any-llm backends, an
exception that is not a provider response now returns `The proxy could
not complete the request.`; the HTTP status mapping is unchanged.
- Memory tool results for any failure return `{"status": "error",
"error": "<code>", "message": ...}` instead of the exception text.
- `x-headroom-*` headers whose name contains a credential component are
no longer recorded as tags. A benign label that happens to use such a
name should be renamed.
## Tests
- `tests/test_public_errors.py`: classification of transport exceptions
(including TLS wrapped in a connect error), non-transport exceptions
left alone, wire shapes carry `code` and `request_id`; the passthrough,
direct-streaming and `/v1/compress` error paths contain no upstream
detail; memory tool results hide transport and internal text;
`client_message` / `tool_result_error` / the LiteLLM backend return
`internal_error` for `RuntimeError`, `ValueError`, `KeyError` and
`JSONDecodeError` carrying a path, host and token (10 tests in this file
fail without the fail-closed change), while an
`openai.AuthenticationError` keeps its text; `extract_tags` and
`public_tags` never carry credentials. Six of these fail on `main`.
- Existing tests that asserted the old exception text in client output
now assert the new contract: the client gets the fixed `code` and
`request_id`, the exception text is absent, and the `headroom.proxy` log
has the full text under the same `request_id`
(`test_litellm_transport_errors.py`, `test_openai_backend_path.py`,
`test_proxy_compress_endpoint.py`,
`test_proxy_streaming_ratelimit_headers.py`).
- Manual run with every upstream connect failing: streaming
`/v1/messages`, streaming `/v1/chat/completions` and passthrough
`/v1/models` return `code: upstream_unreachable` with no address. A
request with `x-headroom-proxy-token` and `x-headroom-license-token`
leaves neither value in the `--log-file` output.
Not tested live: the Bedrock and LiteLLM / any-llm backend paths (unit
tests only) and OTEL/Langfuse export (tags are filtered before they
reach it).
## Open review points
- Resolved: unclassified exception text (JerrettDavis, 2026-09-29) now
fails closed in `client_message` and `tool_result_error`.
- Resolved: `X-Headroom-Key` is matched (`key` is a credential
component; the `api`+`key` pair rule is gone). No internal or test tag
key has a `key` component.
- `is_credential_header` special-cases `authorization`, `cookie`,
`api-key` and `x-api-key`, but its only caller passes only
`x-headroom-*` names, so those branches never run.
- `_is_tls_failure` also matches on message text. Provider HTTP error
responses are now checked first in the backends, so this only affects
non-provider exceptions.
- any-llm: only provider errors raised as `openai`/`anthropic`
`APIStatusError` keep their text; any other any-llm exception type
returns `internal_error`.
Revised 2026-10-01 after review: unrecognised exceptions now return a
fixed `internal_error` message and only provider HTTP error responses
keep their text.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|