mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
refactor(backends): reuse the LiteLLM tool converters in the any-llm backend (#4076)
## Description
`_convert_anthropic_tool` and `_convert_tool_choice` are written out in
both `headroom/backends/litellm.py` and `headroom/backends/anyllm.py`.
The two bodies are identical, including the `{"type": "none"}` bug fix
and its comment; only the docstrings differ. A future fix to one copy
could miss the other. This PR deletes the any-llm copy, and `anyllm.py`
imports the LiteLLM one instead.
Refs #4136
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)
## Changes Made
- `headroom/backends/anyllm.py`: remove the two duplicate functions and
add `from .litellm import _convert_anthropic_tool,
_convert_tool_choice`.
- `headroom/backends/litellm.py`: the docstrings now say the converters
are shared with the any-llm backend.
- No import cost is added. `headroom/backends/__init__.py` already
imports `.litellm`, so `headroom.backends.anyllm` could never be
imported without it.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ pytest -q tests/test_backend_anyllm.py tests/test_backend_bugs.py tests/test_litellm_transport_errors.py tests/test_litellm_openai_passthrough.py tests/test_provider_registry.py tests/test_package_init_lazy.py tests/test_proxy_response_cache_replay.py tests/test_proxy_openai_cache_stability.py tests/test_proxy/test_openai_backend_path.py tests/test_proxy/test_bedrock_sse_ping.py tests/test_proxy/test_anthropic_ccr_raise.py tests/test_proxy/test_openai_chat_savings_profile.py tests/test_proxy/test_anthropic_streaming_ccr_retrieve.py tests/test_backend_nonstreaming_cache_metrics.py tests/test_bedrock_prefix_tracker_wiring.py tests/test_backend_streaming_cache_metrics.py tests/test_openai_streaming_backend.py tests/test_bedrock_streaming_input_tokens.py
227 passed, 7 skipped in 67.13s
$ ruff check headroom/backends/
All checks passed!
$ mypy headroom/backends/anyllm.py --ignore-missing-imports
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS, repo venv, branch checkout at 25190e91, temp HOME
/ HEADROOM_WORKSPACE_DIR, offline.
- Exact command / steps: (1) shared smoke harness: start a real proxy
from this checkout against a local fake upstream and send one Anthropic
and one OpenAI request with a compressible tool result; (2) import both
backends and run the converters the any-llm backend now uses (command
below).
- Observed result: smoke requests return HTTP 200 with savings and no
new warnings; `headroom.backends.anyllm` resolves both converters to the
LiteLLM backend's functions, and each Anthropic `tool_choice` form maps
to the same OpenAI form as before (`any` -> `required`, `none` ->
`none`).
- Not tested: a live any-llm provider call through `--backend anyllm`;
it needs a real provider key and network.
```text
$ bash smoke.sh <worktree>
branch: cleanup/providers-evals-backend-tool-converters @ 25190e91
health: 200
POST /v1/messages -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines:
2 headroom.proxy - WARNING - event=proxy_offline_mode air-gap active — every connection Headroom initiates is refused (telemetry, update check, license reporter, model/tokenizer/binary/dataset downloads, remote Kompress, OTLP/Langfuse export, Copilot auth, subscription polling, OpenAI embedders, Headroom Cloud compression). Still allowed on purpose: forwarding your requests to the upstream you configured, operator-configured local endpoints, loopback health probes.
2 headroom.transforms.kompress_compressor - WARNING - Kompress: 'onnx/kompress-int8-wo.onnx' for chopratejas/kompress-v2-base is not cached and HEADROOM_OFFLINE forbids fetching it (HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.). Compression falls back to the non-ML path.
4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch found no usable file for chopratejas/kompress-v2-base
4 headroom.transforms.kompress_compressor - WARNING - Kompress: artifact prefetch refused for chopratejas/kompress-v2-base (HEADROOM_OFFLINE is set): HEADROOM_OFFLINE is set: refusing outbound network access for HuggingFace download of chopratejas/kompress-v2-base/onnx/kompress-int8-wo.onnx to huggingface.co. Unset HEADROOM_OFFLINE, or turn off the feature that needs this connection.
$ HOME=$(mktemp -d) python -c "from headroom.backends import anyllm, litellm; ..."
same function: True True
{'type': 'function', 'function': {'name': 'get_weather', 'description': 'Weather', 'parameters': {'type': 'object'}}}
{'type': 'auto'} -> auto
{'type': 'any'} -> required
{'type': 'none'} -> none
{'type': 'tool', 'name': 'get_weather'} -> {'type': 'function', 'function': {'name': 'get_weather'}}
```
## Runtime Rollout Safety
- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A, refactor only.
- Stable/default behavior changed: No.
- Kill switch / disable path: N/A.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A
## Additional Notes
Part of maintainer-requested cleanup #4136 (dead code, duplication,
debuggability), split into small PRs for review.
No new tests were added, because the existing tests in both backends
cover the shared converters. No documentation change is needed; the
functions are private.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
69a16cb59c
commit
03538f2f2f
@@ -16,6 +16,7 @@ from headroom.proxy.public_errors import client_message
|
||||
from headroom.utils import format_exception_message
|
||||
|
||||
from .base import Backend, BackendResponse, StreamEvent
|
||||
from .litellm import _convert_anthropic_tool, _convert_tool_choice
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -28,51 +29,6 @@ except ImportError:
|
||||
AnyLLM = None # type: ignore
|
||||
|
||||
|
||||
def _convert_anthropic_tool(tool: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Convert an Anthropic tool definition to the OpenAI function shape.
|
||||
|
||||
any-llm speaks OpenAI, so an Anthropic ``{name, description, input_schema}``
|
||||
tool must become ``{type: function, function: {name, description,
|
||||
parameters}}`` before it is forwarded, or the provider ignores/rejects the
|
||||
tools array and the model never calls a tool. Mirrors the LiteLLM backend's
|
||||
converter so both OpenAI-compatible backends send the same shape.
|
||||
"""
|
||||
func: dict[str, Any] = {"name": tool.get("name", "")}
|
||||
if "description" in tool:
|
||||
func["description"] = tool["description"]
|
||||
if "input_schema" in tool:
|
||||
func["parameters"] = tool["input_schema"]
|
||||
return {"type": "function", "function": func}
|
||||
|
||||
|
||||
def _convert_tool_choice(choice: Any) -> Any:
|
||||
"""Convert an Anthropic ``tool_choice`` to the OpenAI shape (mirrors LiteLLM).
|
||||
|
||||
Anthropic: ``{"type": "auto"}``, ``{"type": "any"}``, ``{"type": "none"}``,
|
||||
``{"type": "tool", "name": ...}``. OpenAI: ``"auto"``, ``"required"``,
|
||||
``"none"``, ``{"type": "function", "function": {"name": ...}}``. Passing the
|
||||
raw Anthropic dict through makes the provider reject or ignore it.
|
||||
"""
|
||||
if isinstance(choice, str):
|
||||
return choice
|
||||
if isinstance(choice, dict):
|
||||
choice_type = choice.get("type", "auto")
|
||||
if choice_type == "auto":
|
||||
return "auto"
|
||||
if choice_type == "any":
|
||||
return "required"
|
||||
if choice_type == "none":
|
||||
# Anthropic's {"type": "none"} means "do not use any tool this turn".
|
||||
# Without this branch it fell through to the "auto" default below,
|
||||
# inverting the instruction into "you may use tools" — the model
|
||||
# could then call a tool the client explicitly forbade. OpenAI's
|
||||
# equivalent is the string "none".
|
||||
return "none"
|
||||
if choice_type == "tool":
|
||||
return {"type": "function", "function": {"name": choice.get("name", "")}}
|
||||
return "auto"
|
||||
|
||||
|
||||
class AnyLLMBackend(Backend):
|
||||
"""Backend using any-llm for multi-provider support."""
|
||||
|
||||
|
||||
@@ -649,6 +649,8 @@ def _anthropic_usage_from_litellm(litellm_usage: Any) -> dict[str, Any]:
|
||||
def _convert_anthropic_tool(tool: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Convert Anthropic tool format to OpenAI function format.
|
||||
|
||||
Shared with the any-llm backend, so both OpenAI-compatible backends send the same shape.
|
||||
|
||||
Anthropic: {"name": "...", "description": "...", "input_schema": {...}}
|
||||
OpenAI: {"type": "function", "function": {"name": "...", "description": "...", "parameters": {...}}}
|
||||
"""
|
||||
@@ -661,7 +663,7 @@ def _convert_anthropic_tool(tool: dict[str, Any]) -> dict[str, Any]:
|
||||
|
||||
|
||||
def _convert_tool_choice(choice: Any) -> Any:
|
||||
"""Convert Anthropic tool_choice to OpenAI format.
|
||||
"""Convert Anthropic tool_choice to OpenAI format (shared with the any-llm backend).
|
||||
|
||||
Anthropic: {"type": "auto"}, {"type": "any"}, {"type": "none"}, {"type": "tool", "name": "..."}
|
||||
OpenAI: "auto", "required", "none", {"type": "function", "function": {"name": "..."}}
|
||||
|
||||
Reference in New Issue
Block a user