fix(proxy): label plain OpenAI chat traffic openai, not custom (#3912)

## Description

Three tests fail on `main`. Each of the PRs below passed CI on its own
branch; the failures come from how they combine.

- **Real bug (#3759 + #2693):** #3759 labels the chat outcome provider
with `CUSTOM_BASE_PROVIDER if upstream_base_url else "openai"`. That was
written against a handler where `upstream_base_url` held the
`x-headroom-base-url` override. #2693 had already renamed the override
to `custom_upstream_base_url` and made `upstream_base_url` the resolved
upstream, which is never empty. Since both merged, **every chat request
without a custom base is labelled `custom` instead of `openai`**. That
affects `/stats`, `/metrics`, OTel and provider-dependent savings
estimates. Not in any release yet.
- **Stale test (#3365):** #3365 keys OpenAI rate limits by an HMAC of
the credential. The flood test's stub limiter still compared against the
raw `Authorization` value, so it never refused and the "limited"
requests got 200 instead of 429. The code is right; the test is updated.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/openai.py`: the fallback label checks
`custom_upstream_base_url`.
- `tests/test_proxy_openai_cache_stability.py`: the stub limiter refuses
`_openai_rate_limit_key({"authorization": limited_auth})`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) — one-identifier change
- [ ] New tests added for new functionality — the existing tests already
cover it and fail on `main`
- [x] Manual testing performed

### Test Output

```text
# on main 59b8cefc
FAILED tests/gateway/test_parity_proxy_vs_gateway.py::test_provider_bound_bytes_and_outcomes_match
FAILED tests/test_proxy_openai_cache_stability.py::test_openai_chat_outcome_provider_uses_the_fixed_taxonomy
FAILED tests/test_proxy_openai_cache_stability.py::test_openai_chat_custom_base_flood_cannot_grow_the_provider_set

# this branch: those files plus rate-limit, passthrough and Grok call-site tests
$ pytest tests/gateway/test_parity_proxy_vs_gateway.py tests/test_proxy_openai_cache_stability.py \
    tests/test_proxy_openai_rate_limit_key.py tests/test_proxy_passthrough.py tests/test_grok_shared_proxy_call_sites.py
49 passed
```

## Real Behavior Proof

- Environment: macOS, Python 3.14, `main` at 59b8cefc.
- Exact command / steps: bisected with #3759's head merged onto each
`main` commit. Taxonomy and parity tests first fail at ef1c528e (#2693);
the flood test first fails at b9e8462a (#3365).
- Observed result: with this change all three pass.
- Not tested: a live proxy's `/stats` output.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: restores the `openai` label for plain
OpenAI chat traffic
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
— not needed
- [ ] I have made corresponding changes to the documentation — not
needed
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works — existing tests fail without it
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Both PRs passed CI on their own branches because neither was retested
against the other. Requiring branches to be up to date before merge, or
a merge queue, would catch this class of break.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Tejas Chopra
2026-10-01 14:25:56 -05:00
committed by GitHub
co-authored by Claude Opus 5.5
parent ff1a0d69f4
commit 7df8bd87b5
2 changed files with 3 additions and 2 deletions
+1 -1
View File
@@ -3646,7 +3646,7 @@ class OpenAIHandlerMixin:
# other custom base is the shared "custom" bucket. Never derive the
# label from the request-controlled hostname — see the review on #3759.
openai_chat_outcome_provider = custom_chat_provider or (
CUSTOM_BASE_PROVIDER if upstream_base_url else "openai"
CUSTOM_BASE_PROVIDER if custom_upstream_base_url else "openai"
)
# Memory: Get user ID when memory is enabled. Reads `request.headers`
+2 -1
View File
@@ -20,6 +20,7 @@ from opentelemetry.sdk.metrics.export import InMemoryMetricReader
from headroom.backends.base import BackendResponse
from headroom.observability import HeadroomOtelMetrics, reset_otel_metrics, set_otel_metrics
from headroom.proxy.handlers.openai import _openai_rate_limit_key
from headroom.proxy.server import ProxyConfig, create_app
from tests.test_observability_metrics import _collect_metrics
@@ -1297,7 +1298,7 @@ def test_openai_chat_custom_base_flood_cannot_grow_the_provider_set(
"""Headroom's own limiter, refusing only ``limited_auth``."""
async def check_request(self, key: str = "default") -> tuple[bool, float]:
return key != limited_auth, 1.0
return key != _openai_rate_limit_key({"authorization": limited_auth}), 1.0
async def check_tokens(self, key: str, tokens: int) -> tuple[bool, float]:
return True, 0.0