mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
feat/https-compose-caddy
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
4bf1bf55d1 |
fix(gemini): keep snake_case inline_data and function parts when compressing (#4290)
## Description Gemini's REST API accepts part fields under both their JSON (camelCase) names and their proto (snake_case) names, and Google's own `curl` examples use snake_case: `inline_data` / `mime_type`, `file_data`, `function_call`, `function_response`. Clients that build requests that way (hand-rolled `curl`/`requests` code, copies of Google's docs samples) send exactly those keys. `_has_non_text_parts()` in `headroom/proxy/handlers/gemini.py` only checks the camelCase keys. A snake_case image or tool part is not marked as preserved, so whenever compression rewrites the request (`optimized_messages != messages`), the content is rebuilt from text alone: - an entry with text plus `inline_data` loses the image, - `function_call` / `function_response` entries have no text at all, so they are dropped outright, and Gemini then receives a conversation with the image and tool turns missing. Through the real proxy, a 4-entry request reached the upstream as 2 entries (see the proof below). The same helper feeds the `cloudcode`, `countTokens` and batch paths. Closes: N/A (no matching issue found) ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/gemini.py`: `_has_non_text_parts()` also recognises `inline_data`, `file_data`, `function_call`, `function_response`, `executable_code`, `code_execution_result` and `thought_signature`, the snake_case forms of the keys it already checks; docstring notes both spellings. - `headroom/proxy/handlers/gemini.py`: the telemetry-only `include_function_responses` conversion (#819) reads `function_response` as well as `functionResponse`, so snake_case tool output also reaches waste-signal detection (added after review). - `tests/test_google_multimodal.py`: - the existing `test_each_non_text_key_detected` parametrization gains the seven snake_case keys; - `test_snake_case_parts_survive_compression_round_trip` runs the handler's own `_gemini_contents_to_messages` → modify text → `_messages_to_gemini_contents` → `_rebuild_gemini_contents` sequence on a snake_case image / `function_call` / `function_response` conversation and checks those three entries come back unchanged. - `tests/test_gemini_function_response_waste.py`: `test_flag_emits_tool_message_for_snake_case_function_response`. ## Testing - [x] Unit tests pass (`pytest`): `tests/test_google_multimodal.py`, `tests/test_proxy_handlers_batch.py`, `tests/test_gemini_*.py`, `tests/test_proxy_gemini_*.py`, not the full suite - [x] Linting passes (`ruff check .`): on the changed files - [ ] Type checking passes (`mypy headroom`): not run - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output New tests on `main` (fix reverted, tests kept): ```text $ python -m pytest tests/test_google_multimodal.py -q FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[inline_data] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[file_data] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[function_call] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[function_response] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[executable_code] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[code_execution_result] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[thought_signature] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_snake_case_parts_survive_compression_round_trip ========================= 8 failed, 61 passed in 8.25s ========================= ``` With the fix: ```text $ python -m pytest tests/test_google_multimodal.py tests/test_proxy_handlers_batch.py tests/test_gemini_*.py tests/test_proxy_gemini_*.py -q ======================= 148 passed, 27 skipped in 16.41s ======================= # (the skips are the live-API integration tests, which need a real GEMINI_API_KEY) $ ruff check headroom/proxy/handlers/gemini.py tests/test_google_multimodal.py tests/test_gemini_function_response_waste.py All checks passed! $ ruff format --check (same files) 3 files already formatted # waste-telemetry test on the previous commit (snake_case not read): E AssertionError: assert ['user'] == ['user', 'tool'] ``` ## Real Behavior Proof - Environment: macOS 27.0.1 (arm64), Python 3.12.11, source tree on this branch (native `headroom._core` from the 0.41.0 wheel), real `headroom proxy` in front of a local stub Gemini upstream (a small `ThreadingHTTPServer` that logs every request body and returns a canned `generateContent` reply), `HEADROOM_OFFLINE=1` - Exact command / steps: started the stub on :18931 and `HEADROOM_OFFLINE=1 python -m headroom.cli proxy --port 18932 --mode token --gemini-api-url http://127.0.0.1:18931 --no-rate-limit`, then `POST /v1beta/models/gemini-2.5-flash:generateContent` with 4 snake_case entries: user text + `inline_data` PNG, model `function_call`, user `function_response`, and a user turn with a 300-row JSON log (33 KB) so compression rewrites the request. Read back what the stub received. Same steps on `main` and this branch. - Observed result: both runs compress the log turn (33 KB → 10.8 KB; proxy log: `Tokens saved: 4,482`). On `main` the upstream gets 2 of 4 entries, with the image, `function_call` and `function_response` gone; on this branch it gets all 4, with only the log text compressed. ```text == main proxy status: 200 upstream received 2 of 4 contents entries user ['text'] len=60 user ['text'] len=10822 == this branch proxy status: 200 upstream received 4 of 4 contents entries user ['text', 'inline_data'] len=141 model ['function_call'] len=97 user ['function_response'] len=99 user ['text'] len=10822 ``` - Not tested: a real Gemini API key / live model (stub upstream only); the `cloudcode`, `countTokens` and batch endpoints, which call the same helper; streaming `streamGenerateContent`. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: N/A - Stable/default behavior changed: only for requests that use snake_case part names: those entries are now kept verbatim, like their camelCase equivalents already were. camelCase requests are unchanged. - Kill switch / disable path: N/A - Unsafe override required: no - Qualification impact: none - Rollback path: revert this commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - A request where **every** entry has a non-text part is forwarded unchanged by the existing early exit (`len(preserved_indices) == len(contents)`), before memory injection. That is already how camelCase `inlineData` requests behave on `main`; this PR only makes snake_case match it. - `headroom/image/compressor.py` also reads only `inlineData`. A snake_case image there is just not compressed, which is safe, so it isn't part of this PR. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
07cf547607 |
fix(proxy/gemini): tolerate malformed parts on the compression path (#2486)
## Description
Three helpers on the Gemini compression path read a content entry's
`parts` and iterate it without type guards:
```python
# _has_non_text_parts
parts = content.get("parts", [])
for part in parts: ...
# _rebuild_gemini_contents
had_text = any("text" in p for p in content.get("parts", []))
# _gemini_contents_to_messages
parts = content.get("parts", [])
text_parts = [p.get("text", "") for p in parts if "text" in p]
```
`parts` is request-controlled and `.get("parts", [])` only falls back
when the key is absent, so:
- a present-but-null `parts` returns `None`, and `for part in None` /
`any(... for p in None)` raises `TypeError`;
- a list carrying a bare string (a client that treats `parts` as a
string array) makes `p.get("text", "")` raise `AttributeError`, while
`"text" in p` silently does substring matching first;
- a null element in the list crashes the same way.
Any of these 500s the request on the compression path, on data that
parsed as valid JSON.
## Fix
Route all three helpers through a shared `_dict_parts(content)` that
returns the dict entries of `parts`, coercing a non-dict content or a
non-list `parts` to an empty list and dropping non-dict elements.
`_gemini_contents_to_messages` also reads `role` defensively for a
non-dict content entry. Conversion now degrades gracefully (the
malformed part contributes nothing) instead of raising. Well-formed
requests are unchanged.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/proxy/handlers/gemini.py`: add `_dict_parts`; use it in
`_has_non_text_parts`, `_rebuild_gemini_contents`, and
`_gemini_contents_to_messages`; read `role` defensively for a non-dict
content entry.
- `tests/test_gemini_function_response_waste.py`: regressions for null
`parts`, bare-string part elements, a null part element,
`_has_non_text_parts` on malformed parts, and a non-dict content entry.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_gemini_function_response_waste.py -q
16 passed
# with the fix reverted, the new malformed-parts tests fail with
# TypeError: 'NoneType' object is not iterable (and AttributeError on string parts)
$ uvx ruff@0.15.17 check headroom/proxy/handlers/gemini.py tests/test_gemini_function_response_waste.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/gemini.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: built a real `HeadroomProxy` and called
`_gemini_contents_to_messages` / `_has_non_text_parts` with contents
carrying `parts: null`, `parts: ["bare string", {text}]`, `parts: [null,
{text}]`, and a non-dict content entry; then reverted `gemini.py` and
re-ran.
- Observed result: with the fix each malformed shape converts without
raising and the valid text part is still emitted (`[{"role": "user",
"content": "kept"}]`); with the fix reverted the null-`parts` and
null-element cases raise `TypeError: 'NoneType' object is not iterable`
and the string-element case raises `AttributeError: 'str' object has no
attribute 'get'`. Ran against the actual module via
`tests/test_gemini_function_response_waste.py`.
- Not tested: a live Gemini request with malformed `parts` routed
through the full proxy compression pipeline end to end.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
|
||
|
|
9b0c840dd7 |
fix(gemini): surface functionResponse payloads to waste-signal detection (#897)
## Problem Fixes #819. Gemini `functionResponse` parts are preserved verbatim on the wire (by design — they are never compressed), but their payloads never reached `parse_messages`: `_gemini_contents_to_messages` only extracts `text` parts. Tool output — where most waste lives — contributed nothing to waste detection on either Gemini path, so `json_bloat`, `repetition`, and the new `reread` signal (#853/#854) were all blind to it. ## Fix (telemetry-only) 1. **`_gemini_contents_to_messages(..., include_function_responses=True)`** — new keyword-only flag. When set, each `functionResponse` payload is additionally emitted as a `role="tool"` message (dict payloads JSON-serialized, strings passed through, missing/`None` responses skipped). `preserved_indices` semantics are unchanged: the entries are still restored verbatim on the wire. 2. **`TransformPipeline.apply(..., waste_messages=...)`** — new optional kwarg (popped before transforms, like `record_metrics`). When provided, the waste-signal parse runs over this richer list instead of the transform input. Transforms, token accounting, and savings deltas are untouched — this is why the richer list is not simply fed to the pipeline: compressed copies of preserved entries are discarded on rebuild, which would corrupt savings reporting. 3. Both Gemini `generateContent` paths (native + Cloud Code Assist) build the enriched list and pass it through. The existing `role="tool"` parsing from #815 handles the rest: tool_result blocks, waste flags, and reread grouping all apply. ## Tests `tests/test_gemini_function_response_waste.py` — 11 new tests: - conversion: default unchanged (regression), dict/string payloads, missing response skipped, text-before-tool ordering, preserved_indices unchanged, circular-reference fallback - parsing: functionResponse payload produces tool_result blocks + `json_bloat`; identical payloads far apart count as `reread` - pipeline: `waste_messages` overrides the waste source, does not affect transform output/token counts, falls back to transform input when absent Full local sweep of touched suites: gemini multimodal, parser, safety rails, canonical pipeline — green. The 13 failures in `test_proxy_gemini_*_integration.py` are credential-dependent and identical on clean `main`. ## Live proof Mock Gemini upstream on a real port, proxy with `optimize=True`; conversation with a large functionResponse payload served twice (5 messages apart) plus compressible model text: ``` waste_signals: { "json_bloat": 35003, "reread": 11673, ... } PROOF OK: waste visible, wire verbatim ``` Upstream received both `functionResponse` entries byte-identical to the client request. ## Known limitations / follow-ups - The Cloud Code Assist path passes `waste_messages` but does not yet consume `result.waste_signals` into a recorded outcome (pre-existing gap; the native path records it). - Requests where **all** content entries are preserved (pure functionResponse/media conversations) early-exit before the pipeline and still produce no waste signals. - Codex/Responses-API counterpart is #820 (separate PR). Co-authored-by: integration-check <integration@local> |