mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat(proxy): report DeepSeek as its own provider in dashboard (#2565)
## Description Fixes #1832. DeepSeek traffic through the OpenAI-compatible chat path (chat/completions, GitHub Copilot, custom bases) is labelled provider `openai` unless the upstream host is OpenCode Zen. So DeepSeek requests collapse into the generic `openai` bucket in the dashboard `by_provider` / Prometheus `requests_by_provider` breakdown — the reporter sees no separate DeepSeek line. Closes #1832 ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - Added `provider_label_from_model()` in `passthrough.py`: `deepseek*` model ids now report provider `deepseek`. - Wired it into the OpenAI chat handler's provider fallback: `custom_chat_provider or provider_label_from_model(model) or "openai"`. - Vendor routing prefixes (`deepseek/deepseek-chat`, `github/deepseek-v4`) resolve to the underlying model; the prefix table is ordered so adding more OpenAI-compatible providers later is a one-line change. - Display-only: pricing stays keyed on the model, not this label, so cost math is unchanged. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ pytest tests/test_proxy_passthrough.py -q tests/test_proxy_passthrough.py .... [100%] 4 passed in 0.30s $ ruff check headroom/proxy/handlers/openai.py headroom/proxy/passthrough.py tests/test_proxy_passthrough.py All checks passed! $ mypy headroom Success: no issues found in 509 source files ``` ## Real Behavior Proof - Environment: local checkout, Python venv, `pytest`/`ruff`/`mypy` as above. - Exact command / steps: classification is unit-tested in `tests/test_proxy_passthrough.py` — DeepSeek ids (incl. vendor prefixes and casing) map to provider `deepseek`; OpenAI/o-series/zen ids stay unlabelled. - Observed result: `deepseek*` requests now surface as a distinct `deepseek` provider in the `by_provider` / `requests_by_provider` breakdown instead of folding into `openai`. - Not tested: no live dashboard round against a real DeepSeek endpoint; verified at the classifier/handler boundary. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes Documentation checklist item is N/A — this is a display-only provider-label classification with no user-facing config or docs surface. Manual testing box left unchecked: validated at the unit boundary rather than a live DeepSeek dashboard round. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2565" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> ### Maintainer current-main requalification Exact head `ecc8dc149036ca65dd9828ab17e31e0a9f35c06c`, upstream main `0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`. Current-main integration preserves the contributor's model-based DeepSeek reporting and known-upstream/xAI precedence. One handler conflict was mechanically reconciled with main's _resolve_openai_chat_forward_path; the contributor attribution assignment AST is identical to its published head, and the chat-path calls match main's AST. All other413incoming patches (414paths total) match old-main to current-main after diff-location normalization. Main-relative scope remains only the intended three Python files. A local UTF-8 reconstruction corrected an initial encoding mistake before tests/final commit; no unintended text/comment changes remain. Fresh validation:384 passed,27 skipped,2 warnings across provider/passthrough/integration/routing/xAI and incoming-current-main regressions. Three contributor Python files pass Ruff lint/format; candidate offline uv lock check and main-relative whitespace pass. Complete tracked native inputset/everyrecordedinputhash/artifactchecksum verified before reuse of existing3982DLL; not a fresh build. Static assignment/patch proof is not live provider, dashboard or HTTP execution. Previous106attribution/cardinality probe results remain historical; no new106caseclaim. Fresh tests use synthetic/mocked upstreams, not a live DeepSeek credential/account. Author discussion and all current inline thread states reviewed; no outstanding inline findings or author replies. Recognized upstream labels and fixed unknown-custom bucket remain ahead of model fallback; no hostname-derived telemetry label introduced. Pricing unchanged. Source/test delta contains no workflow/setup/dependency/CLI/native path changes, and current-main publication security gate passes. Contributor correctness/intent unchanged. Exact published-head hosted checks and final fresh discussion/main/mergeability guard remain before approval. Maintainers handle CI; no PR merged. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JD Davis
JerrettDavis
parent
8388ff5efd
commit
9ad16a77f8
@@ -120,7 +120,11 @@ from headroom.proxy.handlers.streaming import _stream_outcome_status
|
||||
from headroom.proxy.image_isolation import run_image_compression_isolated
|
||||
from headroom.proxy.outcome import RequestOutcome
|
||||
from headroom.proxy.output_shaper import shaper_enabled_for, steering_allowed_for
|
||||
from headroom.proxy.passthrough import CUSTOM_BASE_PROVIDER, is_opencode_zen_base
|
||||
from headroom.proxy.passthrough import (
|
||||
CUSTOM_BASE_PROVIDER,
|
||||
is_opencode_zen_base,
|
||||
provider_label_from_model,
|
||||
)
|
||||
from headroom.proxy.passthrough import (
|
||||
custom_base_passthrough_telemetry as _custom_base_passthrough_telemetry,
|
||||
)
|
||||
@@ -3835,8 +3839,18 @@ class OpenAIHandlerMixin:
|
||||
_image_decision.apply_to_tags(tags)
|
||||
custom_upstream_base_url = _resolve_openai_upstream_base(request.headers)
|
||||
handler_path = _resolve_openai_chat_forward_path(request.headers, custom_upstream_base_url)
|
||||
openai_chat_outcome_provider = _openai_outcome_provider(
|
||||
request.method, handler_path, custom_upstream_base_url, upstream_base_url
|
||||
_, custom_chat_provider = _custom_base_passthrough_telemetry(
|
||||
request.method,
|
||||
handler_path,
|
||||
custom_upstream_base_url or "",
|
||||
)
|
||||
# Keep upstream labels and xAI routing ahead of model-based DeepSeek reporting.
|
||||
# Unknown custom hosts remain in the fixed custom bucket.
|
||||
openai_chat_outcome_provider = custom_chat_provider or (
|
||||
"xai"
|
||||
if _is_xai_upstream(upstream_base_url)
|
||||
else provider_label_from_model(model)
|
||||
or (CUSTOM_BASE_PROVIDER if custom_upstream_base_url else "openai")
|
||||
)
|
||||
|
||||
# Memory: Get user ID when memory is enabled. Reads `request.headers`
|
||||
|
||||
@@ -6,6 +6,33 @@ from urllib.parse import urlparse
|
||||
|
||||
OPENCODE_ZEN_HOSTS = {"opencode.ai", "www.opencode.ai"}
|
||||
|
||||
# OpenAI-compatible endpoints (chat/completions, GitHub Copilot, custom bases)
|
||||
# carry non-OpenAI models whose provider would otherwise collapse into the
|
||||
# generic ``openai`` dashboard bucket. Classify by model id so per-provider
|
||||
# reporting (dashboard ``by_provider`` / Prometheus ``requests_by_provider``)
|
||||
# attributes them correctly. Ordered longest-prefix-first so more specific ids
|
||||
# win. Display-only: pricing stays keyed on the model, not this label.
|
||||
_MODEL_PREFIX_PROVIDERS: tuple[tuple[str, str], ...] = (("deepseek", "deepseek"),)
|
||||
|
||||
|
||||
def provider_label_from_model(model: str | None) -> str:
|
||||
"""Infer a specific provider label from an OpenAI-compatible model id.
|
||||
|
||||
Returns ``""`` when the model does not map to a known non-OpenAI provider,
|
||||
so callers can keep their default label (``"openai"``).
|
||||
"""
|
||||
if not model:
|
||||
return ""
|
||||
name = model.strip().lower()
|
||||
# Strip a leading ``vendor/`` routing prefix (e.g. ``deepseek/deepseek-chat``).
|
||||
if "/" in name:
|
||||
name = name.rsplit("/", 1)[-1]
|
||||
for prefix, provider in _MODEL_PREFIX_PROVIDERS:
|
||||
if name.startswith(prefix):
|
||||
return provider
|
||||
return ""
|
||||
|
||||
|
||||
# Chat upstreams we can name, keyed by exact host. Bounded by design: the
|
||||
# request-controlled ``x-headroom-base-url`` must never become a telemetry
|
||||
# label, or a client sending a fresh hostname per request grows the
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
from headroom.proxy.passthrough import custom_base_passthrough_telemetry, is_opencode_zen_base
|
||||
from headroom.proxy.passthrough import (
|
||||
custom_base_passthrough_telemetry,
|
||||
is_opencode_zen_base,
|
||||
provider_label_from_model,
|
||||
)
|
||||
|
||||
|
||||
def test_custom_base_passthrough_telemetry_recognizes_opencode_zen_chat() -> None:
|
||||
@@ -37,6 +41,20 @@ def test_custom_base_passthrough_telemetry_ignores_non_matching_traffic() -> Non
|
||||
) == ("", "")
|
||||
|
||||
|
||||
def test_provider_label_from_model_classifies_deepseek() -> None:
|
||||
assert provider_label_from_model("deepseek-chat") == "deepseek"
|
||||
assert provider_label_from_model("deepseek-reasoner") == "deepseek"
|
||||
assert provider_label_from_model("DeepSeek-V4-Flash") == "deepseek"
|
||||
# Vendor-prefixed routing ids resolve to the underlying model.
|
||||
assert provider_label_from_model("deepseek/deepseek-chat") == "deepseek"
|
||||
assert provider_label_from_model("github/deepseek-v4") == "deepseek"
|
||||
|
||||
|
||||
def test_provider_label_from_model_leaves_openai_models_unlabeled() -> None:
|
||||
for model in ("gpt-5.4-mini", "openai/gpt-4o", "o3", "zen", "", None):
|
||||
assert provider_label_from_model(model) == ""
|
||||
|
||||
|
||||
def test_is_opencode_zen_base_recognizes_zen_origins() -> None:
|
||||
assert is_opencode_zen_base("https://opencode.ai")
|
||||
assert is_opencode_zen_base("https://www.opencode.ai")
|
||||
|
||||
Reference in New Issue
Block a user