mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(telemetry): label custom-base chat upstreams from a fixed provider set (#3759)
## Description
Unrecognized custom-base chat upstreams report
`outcome_provider="openai"`.
Dashboards and per-provider stores merge every OpenAI-compatible vendor
except Zen into `openai`.
### Fix
Labels come from a fixed set in `custom_base_passthrough_telemetry()`:
| Custom base host | Label |
| --- | --- |
| `api.z.ai` | `zai` |
| `api.meta.ai` | `meta` |
| `api.openai.com` | `openai` |
| anything else | `custom` |
- **Match:** exact host, `chat/completions` path
- **Zen:** unchanged
- **No base header:** `openai`, as before
- **Bounded:** the hostname is never a label
Formatting and the SSRF guard are unchanged.
Pricing tables are unchanged; provider-dependent savings estimates can
change.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- **Fix:** `CUSTOM_BASE_CHAT_PROVIDERS`, exact host to label
- **Fix:** `CUSTOM_BASE_PROVIDER = "custom"` fallback
- **Fix:** chat handler reports the fixed-set label
- **Removed:** `_custom_base_provider_label()`
- **Test:** known hosts, subpath bases, host case
- **Test:** lookalike hosts, non-chat paths, bad URLs
- **Test:** `zai`, `meta`, `custom`, no-header control
- **Test:** 1,000-request flood across every sink
### Flood regression
Through the real chat handler:
1. 980 distinct hosts: ok, upstream 502, local 429, upstream 429
2. 20 `api.z.ai` requests: ok and local 429
3. Exact counts: in-memory, lifetime, OTel
4. `/metrics` labels: inside `{openai, zen, zai, meta, custom}`
The pre-seeded zero `unknown` failure bucket is the one extra label, on
`main` too.
## Testing
- [x] Unit tests pass (`pytest`): no new failures; see baseline below
- [x] Linting passes (`ruff check .`), plus `ruff format --check .`
- [x] Type checking passes (`mypy headroom`):
`--ignore-missing-imports`, mypy 1.20.2 as in CI
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
new tests against main c0292984 code:
FAILED tests/test_proxy_openai_cache_stability.py::test_openai_chat_custom_base_reports_zai_as_outcome_provider
FAILED tests/test_proxy_openai_cache_stability.py::test_openai_chat_outcome_provider_uses_the_fixed_taxonomy
FAILED tests/test_proxy_openai_cache_stability.py::test_openai_chat_custom_base_flood_cannot_grow_the_provider_set
FAILED tests/test_proxy_passthrough.py::test_custom_base_passthrough_telemetry_names_known_chat_hosts
FAILED tests/test_proxy_passthrough.py::test_custom_base_passthrough_telemetry_requires_a_chat_path_segment
E AssertionError: assert {'openai': 260} == {'zai': 10, 'custom': 250}
5 failed, 13 passed
branch: 18 passed
pytest, full suite:
main c0292984 11 failed, 12975 passed, 580 skipped
branch 11 failed, 12981 passed, 580 skipped
same 11 fail on both:
tests/test_proxy_health.py (4 readyz Kompress tests)
tests/test_read_maturation_handler_nobust.py (3)
tests/test_upstream_guard.py::test_no_route_forwards_to_a_loopback_upstream
tests/test_cli_proxy_improvements.py::TestMissingProxyDepsError (2)
tests/test_image_compression.py::TestOnnxRouter::test_full_classify_with_image
ruff check .: All checks passed!
ruff format --check .: 1634 files already formatted
mypy 1.20.2 headroom --ignore-missing-imports: Success: no issues found in 539 source files
```
## Real Behavior Proof
- Environment: Linux, Python 3.12.3, model `glm-5`
- Exact command / steps: see Commands below
- Observed result: see table
- Not tested: live OTel collector; `api.meta.ai` live
### Setup
1. **Trees:** `main` c0292984, this branch
2. **Proxy:** one per tree, own workspace
3. **Allowlist:** every host in `HEADROOM_ALLOWED_BASE_URLS`
The SSRF guard drops a base header whose host does not resolve and falls
back to `openai`.
The flood hosts do not resolve, so without the allowlist they would
never reach the label code.
### Commands
Per tree, from its checkout:
```bash
hosts="api.z.ai,$UPSTREAM_HOST"
for i in $(seq -f %04g 1 100); do hosts="$hosts,flood-$i.label-flood.test"; done
HEADROOM_ALLOWED_BASE_URLS="$hosts" HEADROOM_WORKSPACE_DIR=ws PYTHONPATH=. \
headroom proxy --port 8787 --log-file ws/requests.jsonl
```
Traffic:
```bash
chat() { # chat [base-url] [key] [content]
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8787/v1/chat/completions \
-H 'content-type: application/json' \
${1:+-H "x-headroom-base-url: $1"} ${2:+-H "authorization: Bearer $2"} \
-d '{"model":"glm-5","messages":[{"role":"user","content":"'"${3:-provider label check}"'"}]}'
}
chat https://api.z.ai/api/coding/paas/v4 # 401, no key
chat "$UPSTREAM_BASE_URL" "$UPSTREAM_KEY" # 200, OpenAI-compatible
chat "" sk-control # 200, Headroom response-cache hit
for i in $(seq -f %04g 1 100); do chat "https://flood-$i.label-flood.test/v1" sk-flood flood; done # 502
curl -s http://127.0.0.1:8787/stats | jq '.requests | {by_provider, failed_by_provider}'
curl -s http://127.0.0.1:8787/metrics | grep -o 'provider="[a-z]*"' | sort | uniq -c
```
### Result
| | `main` c0292984 | this branch |
| --- | --- | --- |
| `by_provider` | `openai` 2 | `custom` 1, `openai` 1 |
| `failed_by_provider` | `openai` 101, `unknown` 0 | `custom` 100, `zai`
1, `unknown` 0 |
| `/metrics` provider series | 13 `openai`, 1 `unknown` | 13 `custom`, 1
`zai`, 1 `openai`, 1 `unknown` |
On `main`, z.ai, the real upstream and all 100 flood hosts count as
`openai`.
On this branch, z.ai stays `zai` and the flood lands in one `custom`
bucket.
## Runtime Rollout Safety
- Rollout-managed feature(s): None.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: Yes, telemetry only. Custom-base chat
reports `zai`, `meta`, `openai` or `custom` instead of always `openai`.
Routing, auth, pricing tables and the SSRF guard are unchanged;
provider-dependent savings estimates can change.
- Kill switch / disable path: None dedicated.
- Unsafe override required: No.
- Qualification impact: None. Dashboards keyed on `openai` will see
`zai`, `meta` and `custom` series.
- Rollback path: Revert.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes (no new
failures)
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
**main**
https://github.com/user-attachments/assets/3a0790f0-36a2-4906-ab3b-049273547fa5
**this branch**
https://github.com/user-attachments/assets/c4bc678c-4c59-47d2-94b2-2e942a9e061c
## Additional Notes
- **Closes:** none; no issue tracks this
- **Related:** #2626, same convention for `xai`
- **Docs:** unchecked; no documented behavior
- **Trade-off:** proxied z.ai reports `custom`
- **Baseline:** 11 fail on `main` too; listed above
---------
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
@@ -93,10 +93,10 @@ from headroom.proxy.handlers._debug_dump import _debug_dump_mode, _redact_debug_
|
||||
from headroom.proxy.image_isolation import run_image_compression_isolated
|
||||
from headroom.proxy.outcome import RequestOutcome
|
||||
from headroom.proxy.output_shaper import shaper_enabled_for, steering_allowed_for
|
||||
from headroom.proxy.passthrough import CUSTOM_BASE_PROVIDER, is_opencode_zen_base
|
||||
from headroom.proxy.passthrough import (
|
||||
custom_base_passthrough_telemetry as _custom_base_passthrough_telemetry,
|
||||
)
|
||||
from headroom.proxy.passthrough import is_opencode_zen_base
|
||||
from headroom.proxy.project_context import (
|
||||
classify_project,
|
||||
get_current_project,
|
||||
@@ -3642,7 +3642,12 @@ class OpenAIHandlerMixin:
|
||||
handler_path,
|
||||
custom_upstream_base_url or "",
|
||||
)
|
||||
openai_chat_outcome_provider = custom_chat_provider or "openai"
|
||||
# Fixed taxonomy from the shared helper (zen, zai, meta, openai); any
|
||||
# other custom base is the shared "custom" bucket. Never derive the
|
||||
# label from the request-controlled hostname — see the review on #3759.
|
||||
openai_chat_outcome_provider = custom_chat_provider or (
|
||||
CUSTOM_BASE_PROVIDER if upstream_base_url else "openai"
|
||||
)
|
||||
|
||||
# Memory: Get user ID when memory is enabled. Reads `request.headers`
|
||||
# directly because `headers` was stripped of `x-headroom-*` for the
|
||||
|
||||
@@ -6,6 +6,19 @@ from urllib.parse import urlparse
|
||||
|
||||
OPENCODE_ZEN_HOSTS = {"opencode.ai", "www.opencode.ai"}
|
||||
|
||||
# Chat upstreams we can name, keyed by exact host. Bounded by design: the
|
||||
# request-controlled ``x-headroom-base-url`` must never become a telemetry
|
||||
# label, or a client sending a fresh hostname per request grows the
|
||||
# per-provider stores and exported series without limit (review on #3759).
|
||||
CUSTOM_BASE_CHAT_PROVIDERS = {
|
||||
"api.openai.com": "openai",
|
||||
"api.z.ai": "zai",
|
||||
"api.meta.ai": "meta",
|
||||
}
|
||||
|
||||
# Fixed label for chat traffic on any other custom base.
|
||||
CUSTOM_BASE_PROVIDER = "custom"
|
||||
|
||||
|
||||
def is_opencode_zen_base(base_url: str | None) -> bool:
|
||||
"""Return True when ``base_url`` targets the OpenCode Zen gateway.
|
||||
@@ -35,9 +48,19 @@ def custom_base_passthrough_telemetry(method: str, path: str, base_url: str) ->
|
||||
host = (urlparse(base_url.strip()).hostname or "").lower()
|
||||
except ValueError:
|
||||
return "", ""
|
||||
if host not in OPENCODE_ZEN_HOSTS:
|
||||
return "", ""
|
||||
normalized_path = path[1:] if path.startswith("/") else path
|
||||
if normalized_path == "zen/v1/chat/completions":
|
||||
return "chat/completions", "zen"
|
||||
if host in OPENCODE_ZEN_HOSTS:
|
||||
if normalized_path == "zen/v1/chat/completions":
|
||||
return "chat/completions", "zen"
|
||||
return "", ""
|
||||
# Known OpenAI-compatible chat hosts get a fixed name; chat-completions
|
||||
# traffic on any other custom base is the shared "custom" bucket and
|
||||
# everything else stays unnamed so tool traffic is not LLM telemetry.
|
||||
# Match whole path segments so ``/v1/notchat/completions`` stays unnamed.
|
||||
provider = CUSTOM_BASE_CHAT_PROVIDERS.get(host)
|
||||
is_chat_path = normalized_path == "chat/completions" or normalized_path.endswith(
|
||||
"/chat/completions"
|
||||
)
|
||||
if provider is not None and is_chat_path:
|
||||
return "chat/completions", provider
|
||||
return "", ""
|
||||
|
||||
@@ -4,6 +4,8 @@ from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import json
|
||||
import pathlib
|
||||
import re
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
@@ -13,9 +15,13 @@ import pytest
|
||||
pytest.importorskip("fastapi")
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
from opentelemetry.sdk.metrics import MeterProvider
|
||||
from opentelemetry.sdk.metrics.export import InMemoryMetricReader
|
||||
|
||||
from headroom.backends.base import BackendResponse
|
||||
from headroom.observability import HeadroomOtelMetrics, reset_otel_metrics, set_otel_metrics
|
||||
from headroom.proxy.server import ProxyConfig, create_app
|
||||
from tests.test_observability_metrics import _collect_metrics
|
||||
|
||||
|
||||
class _FakePrefixTracker:
|
||||
@@ -1177,3 +1183,221 @@ def test_openai_cache_mode_forwards_last_turns_bytes_unchanged(stream: bool) ->
|
||||
assert sent[0][2]["content"] == "[compressed call_1]"
|
||||
assert sent[1][: len(sent[0])] == sent[0]
|
||||
assert sent[2][: len(sent[1])] == sent[1]
|
||||
|
||||
|
||||
def test_openai_chat_custom_base_reports_zai_as_outcome_provider(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""An OpenAI-compatible ``x-headroom-base-url`` upstream is not ``openai``."""
|
||||
# Allowlist the host so the SSRF guard does not need live DNS.
|
||||
monkeypatch.setenv("HEADROOM_ALLOWED_BASE_URLS", "api.z.ai")
|
||||
captured = {}
|
||||
with _make_proxy_client() as client:
|
||||
proxy = client.app.state.proxy
|
||||
|
||||
async def _fake_stream_response(*args, **kwargs): # noqa: ANN002, ANN003
|
||||
captured.update(kwargs)
|
||||
return httpx.Response(200, text="data: [DONE]\n\n")
|
||||
|
||||
proxy._stream_response = _fake_stream_response
|
||||
client.post(
|
||||
"/v1/chat/completions",
|
||||
headers={
|
||||
"authorization": "Bearer test-key",
|
||||
"x-headroom-base-url": "https://api.z.ai/api/coding/paas/v4",
|
||||
},
|
||||
json={
|
||||
"model": "glm-5",
|
||||
"stream": True,
|
||||
"messages": [{"role": "user", "content": "hi"}],
|
||||
},
|
||||
)
|
||||
|
||||
assert captured.get("outcome_provider") == "zai"
|
||||
|
||||
|
||||
def test_openai_chat_outcome_provider_uses_the_fixed_taxonomy(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: pathlib.Path
|
||||
) -> None:
|
||||
"""Buffered chat labels each custom base from the fixed set, never the host."""
|
||||
# Keep the SSRF guard out of the way: these hosts are deliberately
|
||||
# unresolvable, and a DNS miss makes the handler ignore the override.
|
||||
monkeypatch.setattr("headroom.proxy.handlers.openai.is_safe_upstream_url", lambda url: True)
|
||||
# Keep the lifetime savings store out of the user's real on-disk state.
|
||||
monkeypatch.setenv("HEADROOM_SAVINGS_PATH", str(tmp_path / "savings.json"))
|
||||
with _make_proxy_client() as client:
|
||||
proxy = client.app.state.proxy
|
||||
|
||||
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001, ANN002, ANN003
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "chatcmpl_1",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {"role": "assistant", "content": "ok"},
|
||||
"finish_reason": "stop",
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 10, "completion_tokens": 1, "total_tokens": 11},
|
||||
},
|
||||
)
|
||||
|
||||
proxy._retry_request = _fake_retry
|
||||
|
||||
def _chat(base: str | None) -> None:
|
||||
headers = {"authorization": "Bearer test-key"}
|
||||
if base is not None:
|
||||
headers["x-headroom-base-url"] = base
|
||||
response = client.post(
|
||||
"/v1/chat/completions",
|
||||
headers=headers,
|
||||
json={"model": "glm-5", "messages": [{"role": "user", "content": "hi"}]},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
|
||||
_chat("https://api.meta.ai/v1")
|
||||
_chat("https://api.meta.ai/v1/sub")
|
||||
_chat("https://llm.example.internal/v1")
|
||||
# No header: the plain OpenAI path keeps its own label (control).
|
||||
_chat(None)
|
||||
|
||||
by_provider = proxy.metrics.requests_by_provider
|
||||
assert by_provider["meta"] == 2
|
||||
assert by_provider["custom"] == 1
|
||||
assert by_provider["openai"] == 1
|
||||
|
||||
|
||||
def test_openai_chat_custom_base_flood_cannot_grow_the_provider_set(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: pathlib.Path
|
||||
) -> None:
|
||||
"""Regression for the #3759 review: a host flood stays inside the fixed set.
|
||||
|
||||
The old code derived the outcome provider from the request-controlled
|
||||
``x-headroom-base-url`` hostname, so one fresh hostname per request grew
|
||||
the provider stores and exported series without bound. 980 distinct flood
|
||||
hosts plus 20 z.ai requests cover success, upstream failure, local 429 and
|
||||
upstream 429, and every sink must count exactly ``zai`` and ``custom``.
|
||||
"""
|
||||
fixed_providers = {"openai", "zen", "zai", "meta", "custom"}
|
||||
zai_base = "https://api.z.ai/api/coding/paas/v4"
|
||||
limited_auth = "Bearer rate-limited"
|
||||
# The flood hosts are unresolvable on purpose; without this stub the SSRF
|
||||
# guard would drop the override and relabel the flood as "openai".
|
||||
monkeypatch.setattr("headroom.proxy.handlers.openai.is_safe_upstream_url", lambda url: True)
|
||||
# Isolated sinks: a fresh lifetime store and workspace, so the exact
|
||||
# counts below are this test's traffic and nothing reaches ~/.headroom.
|
||||
monkeypatch.setenv("HEADROOM_SAVINGS_PATH", str(tmp_path / "savings.json"))
|
||||
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path / "workspace"))
|
||||
reader = InMemoryMetricReader()
|
||||
set_otel_metrics(HeadroomOtelMetrics(meter_provider=MeterProvider(metric_readers=[reader])))
|
||||
|
||||
class _LimitOneKey:
|
||||
"""Headroom's own limiter, refusing only ``limited_auth``."""
|
||||
|
||||
async def check_request(self, key: str = "default") -> tuple[bool, float]:
|
||||
return key != limited_auth, 1.0
|
||||
|
||||
async def check_tokens(self, key: str, tokens: int) -> tuple[bool, float]:
|
||||
return True, 0.0
|
||||
|
||||
async def _fake_retry(method, url, headers, body, stream=False, **kwargs): # noqa: ANN001, ANN002, ANN003
|
||||
if "flood-down-" in url:
|
||||
raise httpx.ConnectError("flood host down")
|
||||
if "flood-upstream429-" in url:
|
||||
return httpx.Response(429, json={"error": {"message": "slow down"}})
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "chatcmpl_flood",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"message": {"role": "assistant", "content": "ok"},
|
||||
"finish_reason": "stop",
|
||||
}
|
||||
],
|
||||
"usage": {"prompt_tokens": 5, "completion_tokens": 1, "total_tokens": 6},
|
||||
},
|
||||
)
|
||||
|
||||
expected_status = {"ok": 200, "down": 502, "limited": 429, "upstream429": 429}
|
||||
sent: dict[tuple[str, str], int] = {}
|
||||
flood_hosts: set[str] = set()
|
||||
try:
|
||||
with _make_proxy_client() as client:
|
||||
proxy = client.app.state.proxy
|
||||
proxy._retry_request = _fake_retry
|
||||
proxy.rate_limiter = _LimitOneKey()
|
||||
|
||||
for i in range(1000):
|
||||
if i % 50 == 0:
|
||||
provider, kind = "zai", ("ok" if i % 100 == 0 else "limited")
|
||||
base = zai_base
|
||||
else:
|
||||
provider, kind = "custom", ("limited", "upstream429", "down", "ok")[i % 4]
|
||||
host = f"flood-{kind}-{i}.example.invalid"
|
||||
flood_hosts.add(host)
|
||||
base = f"https://{host}/v1"
|
||||
sent[provider, kind] = sent.get((provider, kind), 0) + 1
|
||||
auth = limited_auth if kind == "limited" else "Bearer test-key"
|
||||
response = client.post(
|
||||
"/v1/chat/completions",
|
||||
headers={"authorization": auth, "x-headroom-base-url": base},
|
||||
json={"model": "glm-5", "messages": [{"role": "user", "content": f"turn {i}"}]},
|
||||
)
|
||||
assert response.status_code == expected_status[kind], (i, base)
|
||||
|
||||
# The traffic this test sends, pinned so the claim stays exact.
|
||||
assert len(flood_hosts) == 980
|
||||
assert sent == {
|
||||
("zai", "ok"): 10,
|
||||
("zai", "limited"): 10,
|
||||
("custom", "ok"): 250,
|
||||
("custom", "down"): 240,
|
||||
("custom", "limited"): 240,
|
||||
("custom", "upstream429"): 250,
|
||||
}
|
||||
|
||||
metrics = proxy.metrics
|
||||
# In-memory stores: exact counts; "unknown" is the seeded zero bucket.
|
||||
assert dict(metrics.requests_by_provider) == {"zai": 10, "custom": 250}
|
||||
assert dict(metrics.requests_failed_by_provider) == {"unknown": 0, "custom": 240}
|
||||
assert metrics.requests_rate_limited_by_source == {"headroom": 250, "upstream": 250}
|
||||
|
||||
# Lifetime store, isolated under tmp_path.
|
||||
lifetime = metrics.savings_tracker.lifetime_response()["requests"]
|
||||
assert lifetime["by_provider"] == {"zai": 10, "custom": 250}
|
||||
assert lifetime["failed_by_provider"] == {"custom": 240}
|
||||
assert lifetime["rate_limited_by_source"] == {"headroom": 250, "upstream": 250}
|
||||
|
||||
# Prometheus export: every provider series is from the fixed set.
|
||||
export = client.get("/metrics").text
|
||||
assert set(re.findall(r'provider="([^"]+)"', export)) <= fixed_providers | {"unknown"}
|
||||
assert 'headroom_requests_by_provider{provider="zai"} 10' in export
|
||||
assert 'headroom_requests_by_provider{provider="custom"} 250' in export
|
||||
|
||||
# OTel: every proxy data point carries a fixed provider attribute, and
|
||||
# z.ai stays its own series on the success and local-429 paths.
|
||||
otel = _collect_metrics(reader)
|
||||
otel_points: dict[tuple[str, str, str], float] = {}
|
||||
for name, metric in otel.items():
|
||||
if not name.startswith("headroom.proxy."):
|
||||
continue
|
||||
for point in metric.data.data_points:
|
||||
label = point.attributes.get("provider")
|
||||
assert label in fixed_providers, (name, label)
|
||||
if name.startswith("headroom.proxy.requests"):
|
||||
key = (name, label, point.attributes.get("source", ""))
|
||||
otel_points[key] = otel_points.get(key, 0) + point.value
|
||||
assert otel_points == {
|
||||
("headroom.proxy.requests", "zai", ""): 10,
|
||||
("headroom.proxy.requests", "custom", ""): 250,
|
||||
("headroom.proxy.requests.failed", "custom", ""): 240,
|
||||
("headroom.proxy.requests.rate_limited", "zai", "headroom"): 10,
|
||||
("headroom.proxy.requests.rate_limited", "custom", "headroom"): 240,
|
||||
("headroom.proxy.requests.rate_limited", "custom", "upstream"): 250,
|
||||
}
|
||||
finally:
|
||||
reset_otel_metrics()
|
||||
|
||||
@@ -49,3 +49,72 @@ def test_is_opencode_zen_base_rejects_other_or_missing_bases() -> None:
|
||||
assert not is_opencode_zen_base("https://custom.example")
|
||||
assert not is_opencode_zen_base("https://opencode.ai.evil.example")
|
||||
assert not is_opencode_zen_base("://bad-url")
|
||||
|
||||
|
||||
def test_custom_base_passthrough_telemetry_names_known_chat_hosts() -> None:
|
||||
# Exact hosts only, fixed labels: the taxonomy, not the request, decides.
|
||||
assert custom_base_passthrough_telemetry(
|
||||
"POST",
|
||||
"/v1/chat/completions",
|
||||
"https://api.z.ai/api/coding/paas/v4",
|
||||
) == ("chat/completions", "zai")
|
||||
assert custom_base_passthrough_telemetry(
|
||||
"POST",
|
||||
"v1/chat/completions",
|
||||
"https://api.meta.ai/v1",
|
||||
) == ("chat/completions", "meta")
|
||||
assert custom_base_passthrough_telemetry(
|
||||
"POST",
|
||||
"/v1/chat/completions",
|
||||
"https://api.openai.com/v1",
|
||||
) == ("chat/completions", "openai")
|
||||
# Host parsing is case-insensitive; the label stays fixed.
|
||||
assert custom_base_passthrough_telemetry(
|
||||
"POST",
|
||||
"/v1/chat/completions",
|
||||
"https://API.Z.AI/api/coding/paas/v4",
|
||||
) == ("chat/completions", "zai")
|
||||
|
||||
|
||||
def test_custom_base_passthrough_telemetry_keeps_everything_else_unnamed() -> None:
|
||||
# Lookalike hosts never match: an exact-host set cannot be talked into
|
||||
# naming an attacker-controlled upstream.
|
||||
assert custom_base_passthrough_telemetry(
|
||||
"POST",
|
||||
"/v1/chat/completions",
|
||||
"https://api.z.ai.evil.test/v1",
|
||||
) == ("", "")
|
||||
# Unknown hosts, non-chat paths, non-POST methods, and bad URLs all stay
|
||||
# unnamed so nothing request-controlled becomes a telemetry label.
|
||||
assert custom_base_passthrough_telemetry(
|
||||
"POST",
|
||||
"/v1/chat/completions",
|
||||
"https://llm.example.internal/v1",
|
||||
) == ("", "")
|
||||
assert custom_base_passthrough_telemetry(
|
||||
"POST",
|
||||
"/v1/embeddings",
|
||||
"https://api.z.ai/v1",
|
||||
) == ("", "")
|
||||
assert custom_base_passthrough_telemetry(
|
||||
"GET",
|
||||
"/v1/chat/completions",
|
||||
"https://api.z.ai/v1",
|
||||
) == ("", "")
|
||||
assert custom_base_passthrough_telemetry(
|
||||
"POST",
|
||||
"/v1/chat/completions",
|
||||
"://bad-url",
|
||||
) == ("", "")
|
||||
|
||||
|
||||
def test_custom_base_passthrough_telemetry_requires_a_chat_path_segment() -> None:
|
||||
# A bare suffix match would also name ``notchat/completions``.
|
||||
for path in ("/v1/notchat/completions", "/v1/xchat/completions", "notchat/completions"):
|
||||
assert custom_base_passthrough_telemetry("POST", path, "https://api.z.ai/v1") == ("", "")
|
||||
# The exact segment, bare or nested, still names the host.
|
||||
for path in ("chat/completions", "/chat/completions", "/api/paas/v4/chat/completions"):
|
||||
assert custom_base_passthrough_telemetry("POST", path, "https://api.z.ai/v1") == (
|
||||
"chat/completions",
|
||||
"zai",
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user