mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 18:41:07 +08:00
feat/https-compose-caddy
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
57a57088de |
fix(backends/litellm): map Anthropic tool_choice none to OpenAI none (#2689)
## Description
`_convert_tool_choice` in the LiteLLM backend converts an Anthropic
`tool_choice` into the OpenAI shape. It handled `{"type": "auto"}`,
`{"type": "any"}` and `{"type": "tool", "name": ...}`, but there was no
branch for Anthropic's `{"type": "none"}`, so it fell through to the
`return "auto"` default at the end of the function.
That inverts the instruction. A client that sends `tool_choice: {"type":
"none"}` is saying "do not use any tool this turn" (tools stay defined,
but the model must not call one). When such a request is routed through
the LiteLLM backend (Bedrock, Vertex, OpenRouter, and the other
providers this backend serves), the choice was rewritten to `"auto"`,
which tells the model it may call a tool. So the model could invoke a
tool the client explicitly forbade.
The string form already passed through unchanged
(`_convert_tool_choice("none") == "none"`, covered by an existing test);
only the Anthropic dict form, which is what Anthropic clients actually
send, was mishandled. OpenAI's equivalent is the string `"none"`, so
this maps `{"type": "none"}` to `"none"`.
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/backends/litellm.py` (`_convert_tool_choice`): added a
`{"type": "none"}` to `"none"` branch and updated the docstring shapes.
- `tests/test_backend_bugs.py`: added
`test_none_dict_not_inverted_to_auto` asserting the dict form maps to
`"none"` (it previously returned `"auto"`).
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
# Fail-before (source fix stashed, new test kept):
tests/test_backend_bugs.py::TestConvertToolChoice::test_none_dict_not_inverted_to_auto FAILED
assert _convert_tool_choice({"type": "none"}) == "none"
AssertionError: assert 'auto' == 'none'
# Pass-after (fix applied):
tests/test_backend_bugs.py ...........................................
43 passed in 4.83s
# uvx ruff@0.15.17 check -> All checks passed!
# uvx mypy@1.20.2 headroom/backends/litellm.py -> Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, litellm installed in the
project venv, pytest 9.1.1.
- Exact command / steps: fail-before with `git stash push
headroom/backends/litellm.py` and `python -m pytest
tests/test_backend_bugs.py -k TestConvertToolChoice` (the new assertion
fails, `_convert_tool_choice({"type": "none"})` returns `"auto"`), then
pass-after with `git stash pop` and rerunning the file (43 passed).
- Observed result: `_convert_tool_choice({"type": "none"})` now returns
`"none"` instead of `"auto"`, so an Anthropic client that forbids tool
use keeps that instruction when the request is routed through the
LiteLLM backend. The existing auto / any / tool / string cases are
unchanged.
- Not tested: a live end-to-end call to a provider with a real "none"
tool_choice (no provider credentials in this environment). The
conversion is a pure function verified directly, and the downstream
provider is responsible for honoring OpenAI's `"none"`.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
Docs and manual testing are N/A: this is a pure conversion-correctness
fix with no API surface change. The fallback `return "auto"` is kept for
genuinely unknown future types; only the known "none" type is now mapped
explicitly.
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
|
8ff46dc959 |
fix(backends/litellm): tolerate a bare string in tool_result content list (#3795)
## Description
`_convert_messages_for_litellm` flattens a list-shaped `tool_result`
content like this:
```python
if isinstance(tr_content, list):
tr_content = "\n".join(
b.get("text", "") for b in tr_content if b.get("type") == "text"
)
```
This assumes every element of the list is a dict. But a client may put a
**bare string** in the list (`content: ["result text"]` — a valid
Anthropic tool_result shape). `b.get(...)` on a `str` raises
`AttributeError: 'str' object has no attribute 'get'`, which 500s the
entire request via `--backend litellm` / bedrock.
Confirmed:
```python
_convert_messages_for_litellm([{"role":"user","content":[
{"type":"tool_result","tool_use_id":"tu_1","content":["a bare string result"]}]}])
# -> AttributeError: 'str' object has no attribute 'get'
```
Fix: accept bare strings (contribute their text), keep
`{"type":"text",...}` dict blocks, and skip other block types — the same
guard the sibling any-llm converter uses. Plain-string content and
text-block lists are unchanged.
(The neighboring missing-`tool_use_id` / missing-`id` `KeyError`s are
intentionally left as-is: defaulting `tool_call_id` to `""` would
produce an unroutable tool message and mask the error with a confusing
downstream failure, so a hard failure there is preferable to a
silent-wrong request. This PR is scoped to the unambiguous bare-string
crash.)
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/backends/litellm.py`: the `tool_result` list-content flatten
now handles bare-string elements and skips non-text blocks, using a
dedicated `text_pieces` local (avoids colliding with the function's
existing `list[dict]` `parts`).
- `tests/test_backend_bugs.py`: added
`test_tool_result_list_content_with_bare_string_block` — a list mixing a
bare string, a text block, and an image block flattens to the two text
pieces without raising.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
### Test Output
```text
tests/test_backend_bugs.py -k ConvertMessagesToolBlocks -> 8 passed
uvx ruff@0.16.8 check headroom/backends/litellm.py tests/test_backend_bugs.py -> All checks passed!
uvx mypy@1.20.2 headroom/backends/litellm.py -> clean (no error in this file)
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.16.8 and mypy 1.20.2 via uvx.
- Exact command / steps: called `_convert_messages_for_litellm` with a
`tool_result` whose content list contains a bare string — before the fix
it raised `AttributeError`; after, the tool message content is the
joined text. Verified plain-string content, text-block lists, and mixed
bare-string+text+image lists all produce the right joined text with no
crash. Red/green: the new test fails (AttributeError) with the fix
reverted; all 8 tool-block tests pass with it.
- Observed result: a tool continuation whose result content is a list
containing a bare string no longer 500s the request.
- Not tested: a live litellm/bedrock round-trip (the message conversion
is covered by unit tests).
## Runtime Rollout Safety
- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: no for conformant inputs (plain
string, text-block lists convert identically); a previously-crashing
bare-string list now converts instead of 500ing.
- Kill switch / disable path: N/A.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert this commit; a bare-string tool_result content
element again raises AttributeError.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A:
internal message conversion)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
Mirrors the `isinstance(b, dict)` guard used by the any-llm backend's
tool_result handling, so both backends tolerate the same client input
shape.
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
|
577336d2b6 |
fix(backends/litellm): stop dropping image blocks from /v1/messages requests (#3753)
## Description Thanks for the LiteLLM backend and the careful converter work in #3586. `_convert_messages_for_litellm` had no `image` branch: text+image turns lost the image and image-only turns were dropped. User-turn images now map to `image_url` parts, as `AnyLLMBackend._convert_content_blocks` already does. Fixes #3752 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring ## Changes Made - `headroom/backends/litellm.py`: `_anthropic_image_to_openai()`; user turns with an image send ordered text + image parts. - `tests/test_backend_bugs.py`: `TestConvertMessagesImageBlocks` (3 tests). | Case | Before | After | |---|---|---| | User text + image | ❌ image dropped | ✅ `[image_url, text]` → Converse `[image, text]` | | Image-only user turn | ❌ `""`, turn removed | ✅ `[image_url]` | | Text-only turns, tool_use / tool_result, thinking | ✅ | ✅ unchanged | | Assistant-turn image | dropped, 200 | ✅ unchanged (mapping it makes litellm's Bedrock transform raise, so it stays dropped) | | `/v1/chat/completions`, Rust Bedrock proxy | ✅ | ✅ not touched | ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest tests/test_backend_bugs.py -k Image # source reverted to main E AssertionError: assert 'What color?' == [{'type': 'image_url', 'image_url': {'url': 'data:image/png;base64,iVBORw0KGgo='}}, {'type': 'text', 'text': 'What color?'}] E AssertionError: assert '' == [{'type': 'image_url', 'image_url': {'url': 'https://x/a.png'}}] 2 failed, 1 passed $ pytest tests/test_backend_bugs.py -k Image # this branch 3 passed $ pytest <17 test files that import the LiteLLM backend> main: 231 passed, 3 skipped branch: 234 passed, 3 skipped $ ruff check . && ruff format --check . All checks passed! 1631 files already formatted $ mypy headroom --ignore-missing-imports # mypy 1.20.2, as in CI Success: no issues found in 538 source files ``` ## Real Behavior Proof - Environment: macOS, Python 3.12, litellm 1.102.1, `headroom proxy --backend bedrock --region us-east-1`, Bedrock API key. - Exact command / steps: the issue's request and the variants below, on main then this branch. - Observed result: every model below answers red 3/3 here and 0/3 on main; Converse now gets the `image` block. - Not tested: `tool_result` images (follow-up), Rust Bedrock proxy, non-Bedrock providers. ```text # POST /v1/messages, 64x64 red PNG + "What single color is this image? One word.", 3 tries per row (one main answer shown). # text+img = one user turn; img-only = image as the last user turn. Converse body captured at httpx. main this branch us.openai.gpt-6-sol text+img 0/3 red ('Unknown') user:[text] 3/3 red user:[image, text] us.openai.gpt-6-sol img-only 0/3 red ('Please upload the image.') turn gone 3/3 red user:[image] us.openai.gpt-6-luna text+img 0/3 red ('White') 3/3 red us.openai.gpt-6-luna img-only 0/3 red ('Unclear.') 3/3 red us.openai.gpt-6-astra text+img 0/3 red ('Unknown') 3/3 red us.openai.gpt-6-astra img-only 0/3 red ('Missing.') 3/3 red global.openai.gpt-6-astra text+img 0/3 red ('Image?') 3/3 red global.openai.gpt-6-astra img-only 0/3 red ('Please upload the image.') 3/3 red us.openai.gpt-5.6-sol text+img 0/3 red ('Unknown') 3/3 red us.openai.gpt-5.6-sol img-only 0/3 red ('Upload') 3/3 red claude-haiku-4-5 text+img 0/3 red ('Blue.') 3/3 red claude-haiku-4-5 img-only 0/3 red ('') 3/3 red stream=true, GPT-6 Sol/Luna/Astra 'Unknown' / 'White' / 'Unknown' 'Red' / 'Red' / 'Red' assistant-turn image, GPT-6 x3 200 'OK' 200 'OK' url-source image, GPT-6 Sol/Astra 'No image attached.' / 'No image is attached.' 'Community Savings' / 'Yes, COMMUNITY SAVINGS.' # url-source row: source.type=url pointing at this repo's headroom-savings.png, asked for the word in its title. ``` ## Runtime Rollout Safety <details><summary>No rollout gate</summary> - Rollout-managed feature(s): none. - Minimum rollout channel: n/a. - Stable/default behavior changed: user-turn images on the LiteLLM `/v1/messages` path are now forwarded. - Kill switch / disable path: none. - Unsafe override required: no. - Qualification impact: none. - Rollback path: revert this commit. </details> ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist <details><summary>7 of 8 checked (docs left open: no docs change)</summary> - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` </details> ## Screenshots (if applicable) N/A ## Additional Notes No docs change. Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
3c1a5cdb1c |
fix(backends/litellm): drop oversized tool names before Bedrock Converse (#2129)
## Description The Bedrock Converse API hard-rejects any request containing a tool name over 64 characters (`toolConfig.tools.N.member.toolSpec.name`). Claude Code includes every globally-added claude.ai MCP connector tool in every request it sends, even connectors the user hasn't enabled locally. One org-wide connector with a 65-char tool name is enough to fail every single request routed through this backend's Bedrock path, with no way to remove or disable the connector client-side. Direct Bedrock mode (`CLAUDE_CODE_USE_BEDROCK=1`, bypassing this proxy) is unaffected: it hits Bedrock's native Anthropic-compatible endpoint, which has no such length limit. Only the Converse API, which this LiteLLM-backed `bedrock` provider path uses, enforces it. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/backends/litellm.py`: `send_message` and `stream_message` both filter tools with names over 64 characters out of the payload before converting/forwarding, but only for `self.provider == "bedrock"`. Other providers are untouched. - `tests/test_backend_bugs.py`: new `TestBedrockOversizedToolNameFiltering` covering both `send_message` and `stream_message` — an oversized (65-char) name is dropped on `bedrock`, a name at exactly the 64-char boundary is kept, and non-`bedrock` providers forward oversized names unfiltered (the limit is a Bedrock Converse constraint, not a general one). - `CHANGELOG.md`: added a `### Fixed` entry under `Unreleased`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run pytest tests/test_backend_bugs.py tests/test_backend_anyllm.py -q ============================= test session starts ============================== platform darwin -- Python 3.13.13, pytest-9.0.3, pluggy-1.6.0 collected 57 items tests/test_backend_bugs.py .......................................... [ 73%] tests/test_backend_anyllm.py ............... [100%] ============================== 57 passed in 1.42s ============================== $ uv run ruff check headroom/backends/litellm.py tests/test_backend_bugs.py All checks passed! $ uv run mypy headroom/backends/litellm.py Success: no issues found in 1 source file ``` ## Real Behavior Proof - **Environment:** personal fork deployed as a real proxy (macOS launchd service, `headroom install apply`) with `--backend bedrock --mode token --code-aware --bedrock-profile sso-bedrock`, fronting a live Claude Code session with a globally-added-but-not-locally-enabled claude.ai MCP connector (`TopCounsel`) whose tool name is 65 characters. - **Exact command / steps:** run any Claude Code request through this deployment while the org-wide `TopCounsel` connector is present (it is included in the tool list on every request regardless of local enablement). - **Observed result:** before the fix, every request failed with a LiteLLM `BedrockException`: `1 validation error detected: Value 'mcp__claude_ai_TopCounsel_by_The_L_Suite__complete_authentication' at 'toolConfig.tools.N.member.toolSpec.name' failed to satisfy constraint: Member must have length less than or equal to 64`. After applying the fix (filtering the oversized tool out before the LiteLLM call), the same session proceeds normally with no validation error, confirmed live against this deployment. - **Not tested:** truncating the name instead of dropping it was tried and discarded during investigation — the model echoes the truncated name back in `tool_use` blocks, and Claude Code matches tool calls by the original full name, so truncation breaks routing on the return path. This PR drops the tool entirely rather than truncating, which is why it is not present as an alternative in the diff. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — backend logic change, no UI surface. ## Additional Notes - "I have made corresponding changes to the documentation" is unchecked: no file in `docs/`, `README.md`, or `CONTRIBUTING.md` documents backend-specific tool-list filtering behavior, so there is no existing section to update. - No linked issue number: this was found via independent investigation of a personal deployment (a live Bedrock validation failure), not filed as a `headroomlabs-ai/headroom` issue first. Checked `gh pr list`/`gh issue list` for existing coverage of "Bedrock Converse 64-char tool name" and found none open or merged. - A native Bedrock Anthropic-compatible endpoint backend (avoiding Converse's tool-name limit entirely) would be the more complete long-term fix, but is out of scope for this PR. --------- Co-authored-by: Ingmar Krusch <ingmar.krusch@gmail.com> Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
|
d5ca50cd03 |
fix(tests): stop module-level dotenv loaders from polluting os.environ during pytest collection
# The bug
Several test modules and two production modules loaded the project `.env`
at *import time*. During pytest collection (where every test module is
imported once), this populated `os.environ` with API keys from `.env`.
The skipif guards in `test_proxy_passthrough_integration.py` (and
others) evaluate at collection time:
@pytest.mark.skipif(not os.environ.get("OPENAI_API_KEY"), reason="...")
If the polluter module was collected *before* the guard, the guard saw
the leaked key, decided not to skip, and the integration tests ran
live against a fake key and failed. In a fresh local-dev venv with
`.env` + full `[dev]` extras, this manifested as ~16 spurious test
failures plus a misleading test runtime of 6+ minutes (live HTTP).
# Why now
CI does not see this (no `.env`). It only manifests when:
1. `litellm` (and friends) are installed — they run `dotenv.load_dotenv()`
on import, populating `os.environ` from `.env`.
2. A `.env` file with real API keys exists locally.
Until the venv was provisioned with the full `[dev]` extras during
recent test work, `pytest.importorskip("litellm")` and
`from headroom.pricing import litellm_pricing` both silently no-op'd
(via try/except ImportError → `LITELLM_AVAILABLE=False`), so the leak
never triggered. With litellm now installed, the latent bug surfaced.
# The fix — three patterns
1. **Production modules** (`headroom/pricing/litellm_pricing.py`,
`headroom/backends/litellm.py`): wrap the eager `import litellm` with
a snapshot/restore of `os.environ`. Any keys litellm's bundled
`python-dotenv` adds during import are deleted immediately. The
module is fully imported and cached in `sys.modules` so subsequent
imports hit the cache without re-running the side effect.
2. **Test modules using `pytest.importorskip("litellm")`**
(`test_backend_bugs.py`, `test_bedrock_region.py`,
`test_cost_tracker_counterfactual.py`): replace with
`tests._dotenv.importorskip_no_env_leak("litellm")`, which does the
same snapshot/restore around `importlib.import_module`.
3. **Test modules that intentionally need `.env` values for skipif
guards** (`test_compression_summary_*.py`, `test_query_echo.py`,
`test_cost_tracker_counterfactual.py`, `test_memory_usage_integration.py`,
`test_bundled_tools_savings.py`): replace module-level
`os.environ.setdefault(...)` / `dotenv.load_dotenv()` with
`tests._dotenv.load_env_overrides()` (returns a local dict — does
NOT mutate `os.environ`) plus `autouse_apply_env(...)` (function-
scoped fixture that applies via `monkeypatch.setenv`, auto-cleaned
at teardown). The skipif still works because
`ANTHROPIC_KEY = os.environ.get(...) or _env_overrides.get(...)`
reads from the local dict as fallback.
# Helper module
New `tests/_dotenv.py` exposes:
- `load_env_overrides() -> dict[str, str]` — read `.env` into a dict.
- `autouse_apply_env(overrides) -> fixture` — function-scoped autouse
fixture that applies via `monkeypatch.setenv`.
- `importorskip_no_env_leak(module) -> module` — drop-in
`pytest.importorskip` substitute that quarantines env mutations.
# Results
Local full-suite (excluding live-LLM and live-feed tests):
- Before: 46 failed, 4830 passed, 387s
- After: 2 failed, 4672 passed, 134s
The remaining 2 failures are unrelated environment-dependent tests
(missing `PIL` / Docker daemon).
|
||
|
|
feb163d8c8 |
Fix Bedrock/Vertex auth regression: don't forward api_key for env-auth providers (#105)
The 0.5.18 refactor added api_key forwarding from request headers to LiteLLM kwargs in all 4 handler methods. This breaks Bedrock (AWS SigV4) and Vertex AI (Google ADC) which authenticate via env vars, not API keys. Forwarding a dummy key like sk-ant-dummy overrides AWS credentials. Fix: skip api_key forwarding for bedrock, vertex_ai, vertex_ai_beta, and sagemaker providers. Applied to all 4 occurrences. |
||
|
|
7d25c87286 |
Fix tool_result ordering: no stray user message between tool_calls and tool
Bedrock requires role=tool messages immediately after assistant tool_calls. The previous fix inserted a user text message in between when the message contained both text and tool_result blocks, breaking the pairing. Drop text alongside tool_result (Claude Code never sends it in practice). Added ordering regression tests for the Bedrock constraint. |
||
|
|
8228f0edfb |
Fix streaming tool calls, compressed request bodies, beacon field names; bump to 0.5.13
- Fix LiteLLM stream_message: emit tool_use blocks from delta.tool_calls, set stop_reason from finish_reason (fixes silent MCP tool call failures) - Fix _convert_messages_for_litellm: convert Anthropic tool_result/tool_use to OpenAI role=tool/tool_calls format (fixes 500 on tool round-trips) - Fix proxy request body parsing: decompress zstd/gzip/deflate/brotli Content-Encoding before JSON decode (fixes Codex UnicodeDecodeError crash) - Fix telemetry beacon field names to match /stats endpoint (tokens.total_before_compression, not tokens.original) - Add dedupe_telemetry.py script for hourly Supabase row deduplication |
||
|
|
93d41b66b1 |
Commit message:
Fix OpenAI streaming with backends and /v1 double-path bug Add stream_openai_message() to LiteLLM and any-llm backends so /v1/chat/completions with stream:true returns SSE events instead of a JSON blob. Clients (Kilo Code, Cursor, etc.) were hanging because the proxy ignored the stream flag when routing through a backend. Also strip trailing /v1 from OPENAI_TARGET_API_URL to prevent double-path URLs like /v1/v1/models. |
||
|
|
4bea17ba8a |
Fix proxy backend bugs: env vars, tool forwarding, and provider support
- CLI now reads OPENAI_TARGET_API_URL, GEMINI_TARGET_API_URL, and HEADROOM_ANYLLM_PROVIDER environment variables - Add --openai-api-url and --gemini-api-url CLI flags - Remove --backend choices restriction so litellm-* backends work - Forward tools/tool_choice through LiteLLM and any-llm backends - Parse tool call arguments from JSON string to dict (Anthropic format) - Forward top_p, stop_sequences, tools in streaming paths - Update Vertex AI model map with Claude 3 through 4.6 (from official docs) - Bump version to 0.4.1 |