fix(backends/litellm): match the caller-key Bearer scheme case-insensitively (RFC 7235) (#3965)

## Description

The LiteLLM backend forwards the inbound caller credential to the
upstream provider. It extracts that credential from the `Authorization`
header with a case-sensitive `startswith("Bearer ")`, duplicated across
four identical blocks (`send_message`, `stream_message`,
`send_openai_message`, `stream_openai_message`).

The HTTP auth-scheme token is **case-insensitive** per RFC 7235 §2.1, so
a client sending `Authorization: bearer <key>` (lowercase, or any other
casing) had its credential silently dropped. LiteLLM then fell back to
the target provider's own env key — which may not be configured —
producing a spurious upstream `401` that reads downstream like a model
failure rather than a dropped credential.

The proxy already classifies this same header case-insensitively
(`auth_policy.classify_auth_signals`, RFC 7235), so the backend was the
inconsistent half.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Add `_caller_key_from_headers(headers)` that parses the
`Authorization` header with `partition(" ")` and case-folds **only** the
scheme (`scheme.lower() == "bearer"`), returning the credential verbatim
and falling back to `x-api-key` exactly as before.
- Replace all four byte-identical caller-key extraction blocks with a
call to the helper, so they can no longer drift.
- Tests: helper-level unit tests (scheme casings, uppercase header name,
`x-api-key` fallback, verbatim credential) in
`test_litellm_caller_key.py`, plus an end-to-end regression in
`test_litellm_openai_passthrough.py` asserting a lowercase-`bearer` key
reaches `acompletion(api_key=...)`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_litellm_openai_passthrough.py -k lowercase_bearer -q   # before fix
FAILED tests/test_litellm_openai_passthrough.py::test_lowercase_bearer_caller_key_is_forwarded
1 failed

$ pytest tests/test_litellm_caller_key.py tests/test_litellm_openai_passthrough.py -q   # after fix
29 passed

$ ruff check headroom/backends/litellm.py tests/test_litellm_caller_key.py tests/test_litellm_openai_passthrough.py
All checks passed!
```

## Real Behavior Proof

- Environment: macOS (arm64), Python 3.13, local editable install
(`maturin develop --release`). Driven through the real
`LiteLLMBackend.send_openai_message` (provider `openrouter`) with the
upstream `acompletion` call captured.
- Exact command / steps: send a request carrying `Authorization: bearer
sk-caller-123` (lowercase scheme) and inspect the `api_key` LiteLLM is
actually called with, before and after the patch.
- Observed result: The lowercase Bearer caller credential changed from
absent to api_key=sk-caller-123 at the mocked upstream completion
boundary, as captured below.
  ```text
  === BEFORE fix ===
  api_key forwarded to upstream: None
  === AFTER fix ===
  api_key forwarded to upstream: 'sk-caller-123'
  ```
Before the patch the lowercase-scheme caller key was dropped (upstream
would fall back to a possibly-absent env key); after, it is forwarded.
- Not tested: a live round-trip to a real upstream provider (no provider
credentials in this environment); the forwarded `api_key` was captured
at the backend's `acompletion` boundary instead.

## Runtime Rollout Safety

- Rollout-managed feature(s): none — request-auth extraction fix.
- Minimum rollout channel: n/a
- Stable/default behavior changed: only that a differently-cased
`Bearer` scheme now forwards the caller key as an exact-cased `Bearer`
already did; `x-api-key` fallback unchanged.
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

`mypy` not run locally (not set up in this environment); `ruff
check`/`ruff format` are clean. No new dependencies. The four identical
caller-key blocks are collapsed into one helper as a direct consequence
of the fix (one correct parser instead of four copies), not a standalone
refactor.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3965"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
Rajat Nagda
2026-10-05 09:54:45 -05:00
committed by GitHub
co-authored by JD Davis
parent fe884612c0
commit 848d7a04b4
3 changed files with 78 additions and 25 deletions
+23 -24
View File
@@ -544,6 +544,25 @@ def _caller_key_travels_to(model: str, key: str) -> bool:
return provider not in _REJECTS_ANTHROPIC_KEY
def _caller_key_from_headers(headers: dict[str, str]) -> str:
"""Extract the caller's API key from inbound request headers.
The ``Authorization`` auth-scheme token is case-insensitive per RFC 7235
§2.1, so ``Authorization: bearer <key>`` must be read the same as
``Bearer <key>``. A case-sensitive ``startswith("Bearer ")`` dropped the
credential for a lowercase (or otherwise differently-cased) scheme, and the
request then fell back to the target provider's env key — which may not
exist, yielding a spurious upstream 401. Only the scheme is case-folded; the
credential itself is returned verbatim. Falls back to ``x-api-key`` when the
header carries no bearer credential (matching the prior behavior).
"""
auth_header = headers.get("authorization", headers.get("Authorization", ""))
scheme, sep, credentials = auth_header.partition(" ")
if sep and scheme.lower() == "bearer":
return credentials
return headers.get("x-api-key", "")
def get_provider_config(provider: str) -> ProviderConfig:
"""Get provider config, with fallback for unknown providers."""
if provider in PROVIDER_REGISTRY:
@@ -1290,12 +1309,7 @@ class LiteLLMBackend(Backend):
# Forwarding x-api-key (e.g. sk-ant-dummy) would override their credentials.
_env_auth_providers = ("bedrock", "vertex_ai", "vertex_ai_beta", "sagemaker")
if self.provider not in _env_auth_providers:
auth_header = headers.get("authorization", headers.get("Authorization", ""))
_caller_key = (
auth_header[7:]
if auth_header.startswith("Bearer ")
else headers.get("x-api-key", "")
)
_caller_key = _caller_key_from_headers(headers)
# Only forward it if it can actually authenticate the TARGET.
if _caller_key and _caller_key_travels_to(litellm_model, _caller_key):
kwargs["api_key"] = _caller_key
@@ -1416,12 +1430,7 @@ class LiteLLMBackend(Backend):
# Forwarding x-api-key (e.g. sk-ant-dummy) would override their credentials.
_env_auth_providers = ("bedrock", "vertex_ai", "vertex_ai_beta", "sagemaker")
if self.provider not in _env_auth_providers:
auth_header = headers.get("authorization", headers.get("Authorization", ""))
_caller_key = (
auth_header[7:]
if auth_header.startswith("Bearer ")
else headers.get("x-api-key", "")
)
_caller_key = _caller_key_from_headers(headers)
# Only forward it if it can actually authenticate the TARGET.
if _caller_key and _caller_key_travels_to(litellm_model, _caller_key):
kwargs["api_key"] = _caller_key
@@ -1824,12 +1833,7 @@ class LiteLLMBackend(Backend):
# Forwarding x-api-key (e.g. sk-ant-dummy) would override their credentials.
_env_auth_providers = ("bedrock", "vertex_ai", "vertex_ai_beta", "sagemaker")
if self.provider not in _env_auth_providers:
auth_header = headers.get("authorization", headers.get("Authorization", ""))
_caller_key = (
auth_header[7:]
if auth_header.startswith("Bearer ")
else headers.get("x-api-key", "")
)
_caller_key = _caller_key_from_headers(headers)
# Only forward it if it can actually authenticate the TARGET.
if _caller_key and _caller_key_travels_to(litellm_model, _caller_key):
kwargs["api_key"] = _caller_key
@@ -2039,12 +2043,7 @@ class LiteLLMBackend(Backend):
# Forwarding x-api-key (e.g. sk-ant-dummy) would override their credentials.
_env_auth_providers = ("bedrock", "vertex_ai", "vertex_ai_beta", "sagemaker")
if self.provider not in _env_auth_providers:
auth_header = headers.get("authorization", headers.get("Authorization", ""))
_caller_key = (
auth_header[7:]
if auth_header.startswith("Bearer ")
else headers.get("x-api-key", "")
)
_caller_key = _caller_key_from_headers(headers)
# Only forward it if it can actually authenticate the TARGET.
if _caller_key and _caller_key_travels_to(litellm_model, _caller_key):
kwargs["api_key"] = _caller_key
+34 -1
View File
@@ -15,7 +15,7 @@ from __future__ import annotations
import pytest
from headroom.backends.litellm import _caller_key_travels_to
from headroom.backends.litellm import _caller_key_from_headers, _caller_key_travels_to
ANTHROPIC_KEY = "sk-ant-api03-abc123"
@@ -65,3 +65,36 @@ def test_unknown_provider_keeps_the_pass_through() -> None:
key -- including when `get_llm_provider` raises on the model string."""
assert _caller_key_travels_to("some-self-hosted-thing", ANTHROPIC_KEY) is True
assert _caller_key_travels_to("", ANTHROPIC_KEY) is True
# ---------------------------------------------------------------------------
# _caller_key_from_headers — the Authorization scheme is case-insensitive
# (RFC 7235 §2.1), so `bearer <key>` must be read the same as `Bearer <key>`.
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"scheme",
["Bearer", "bearer", "BEARER", "BeArEr"],
)
def test_bearer_scheme_is_matched_case_insensitively(scheme: str) -> None:
"""A differently-cased Bearer scheme must still yield the caller key."""
headers = {"authorization": f"{scheme} {ANTHROPIC_KEY}"}
assert _caller_key_from_headers(headers) == ANTHROPIC_KEY
def test_uppercase_authorization_header_name_is_accepted() -> None:
headers = {"Authorization": f"bearer {ANTHROPIC_KEY}"}
assert _caller_key_from_headers(headers) == ANTHROPIC_KEY
def test_falls_back_to_x_api_key_without_a_bearer_credential() -> None:
assert _caller_key_from_headers({"x-api-key": ANTHROPIC_KEY}) == ANTHROPIC_KEY
# A bare scheme with no credential is not a bearer credential.
assert _caller_key_from_headers({"authorization": "Bearer"}) == ""
def test_credential_itself_is_returned_verbatim() -> None:
"""Only the scheme is case-folded; the credential keeps its exact case."""
mixed = "sk-AbC-123-XyZ"
assert _caller_key_from_headers({"authorization": f"bearer {mixed}"}) == mixed
+21
View File
@@ -259,3 +259,24 @@ async def test_reasoning_params_forwarded_top_level_streaming() -> None:
kwargs = mock_acomp.await_args.kwargs
assert kwargs["reasoning_effort"] == "high"
assert "reasoning_effort" not in kwargs.get("extra_body", {})
@pytest.mark.asyncio
async def test_lowercase_bearer_caller_key_is_forwarded() -> None:
"""A lowercase `authorization: bearer <key>` must still forward the key.
RFC 7235 §2.1 makes the auth-scheme token case-insensitive. A case-sensitive
`startswith("Bearer ")` dropped the caller credential for a lowercase scheme,
so litellm fell back to a (possibly absent) env key and the upstream 401'd.
"""
backend = make_backend() # provider=openrouter, not env-auth
with patch("headroom.backends.litellm.acompletion", new_callable=AsyncMock) as mock_acomp:
mock_acomp.return_value = make_response()
await backend.send_openai_message(
request_body(),
{"authorization": "bearer sk-caller-123"},
)
assert mock_acomp.await_args.kwargs.get("api_key") == "sk-caller-123"