mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): label plain OpenAI chat traffic openai, not custom (#3912)
## Description Three tests fail on `main`. Each of the PRs below passed CI on its own branch; the failures come from how they combine. - **Real bug (#3759 + #2693):** #3759 labels the chat outcome provider with `CUSTOM_BASE_PROVIDER if upstream_base_url else "openai"`. That was written against a handler where `upstream_base_url` held the `x-headroom-base-url` override. #2693 had already renamed the override to `custom_upstream_base_url` and made `upstream_base_url` the resolved upstream, which is never empty. Since both merged, **every chat request without a custom base is labelled `custom` instead of `openai`**. That affects `/stats`, `/metrics`, OTel and provider-dependent savings estimates. Not in any release yet. - **Stale test (#3365):** #3365 keys OpenAI rate limits by an HMAC of the credential. The flood test's stub limiter still compared against the raw `Authorization` value, so it never refused and the "limited" requests got 200 instead of 429. The code is right; the test is updated. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/openai.py`: the fallback label checks `custom_upstream_base_url`. - `tests/test_proxy_openai_cache_stability.py`: the stub limiter refuses `_openai_rate_limit_key({"authorization": limited_auth})`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) — one-identifier change - [ ] New tests added for new functionality — the existing tests already cover it and fail on `main` - [x] Manual testing performed ### Test Output ```text # on main59b8cefcFAILED tests/gateway/test_parity_proxy_vs_gateway.py::test_provider_bound_bytes_and_outcomes_match FAILED tests/test_proxy_openai_cache_stability.py::test_openai_chat_outcome_provider_uses_the_fixed_taxonomy FAILED tests/test_proxy_openai_cache_stability.py::test_openai_chat_custom_base_flood_cannot_grow_the_provider_set # this branch: those files plus rate-limit, passthrough and Grok call-site tests $ pytest tests/gateway/test_parity_proxy_vs_gateway.py tests/test_proxy_openai_cache_stability.py \ tests/test_proxy_openai_rate_limit_key.py tests/test_proxy_passthrough.py tests/test_grok_shared_proxy_call_sites.py 49 passed ``` ## Real Behavior Proof - Environment: macOS, Python 3.14, `main` at59b8cefc. - Exact command / steps: bisected with #3759's head merged onto each `main` commit. Taxonomy and parity tests first fail atef1c528e(#2693); the flood test first fails atb9e8462a(#3365). - Observed result: with this change all three pass. - Not tested: a live proxy's `/stats` output. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: restores the `openai` label for plain OpenAI chat traffic - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none - Rollback path: revert ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas — not needed - [ ] I have made corresponding changes to the documentation — not needed - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works — existing tests fail without it - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes Both PRs passed CI on their own branches because neither was retested against the other. Requiring branches to be up to date before merge, or a merge queue, would catch this class of break. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
ff1a0d69f4
commit
7df8bd87b5
@@ -3646,7 +3646,7 @@ class OpenAIHandlerMixin:
|
||||
# other custom base is the shared "custom" bucket. Never derive the
|
||||
# label from the request-controlled hostname — see the review on #3759.
|
||||
openai_chat_outcome_provider = custom_chat_provider or (
|
||||
CUSTOM_BASE_PROVIDER if upstream_base_url else "openai"
|
||||
CUSTOM_BASE_PROVIDER if custom_upstream_base_url else "openai"
|
||||
)
|
||||
|
||||
# Memory: Get user ID when memory is enabled. Reads `request.headers`
|
||||
|
||||
@@ -20,6 +20,7 @@ from opentelemetry.sdk.metrics.export import InMemoryMetricReader
|
||||
|
||||
from headroom.backends.base import BackendResponse
|
||||
from headroom.observability import HeadroomOtelMetrics, reset_otel_metrics, set_otel_metrics
|
||||
from headroom.proxy.handlers.openai import _openai_rate_limit_key
|
||||
from headroom.proxy.server import ProxyConfig, create_app
|
||||
from tests.test_observability_metrics import _collect_metrics
|
||||
|
||||
@@ -1297,7 +1298,7 @@ def test_openai_chat_custom_base_flood_cannot_grow_the_provider_set(
|
||||
"""Headroom's own limiter, refusing only ``limited_auth``."""
|
||||
|
||||
async def check_request(self, key: str = "default") -> tuple[bool, float]:
|
||||
return key != limited_auth, 1.0
|
||||
return key != _openai_rate_limit_key({"authorization": limited_auth}), 1.0
|
||||
|
||||
async def check_tokens(self, key: str, tokens: int) -> tuple[bool, float]:
|
||||
return True, 0.0
|
||||
|
||||
Reference in New Issue
Block a user