mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(gemini): keep snake_case inline_data and function parts when compressing (#4290)
## Description Gemini's REST API accepts part fields under both their JSON (camelCase) names and their proto (snake_case) names, and Google's own `curl` examples use snake_case: `inline_data` / `mime_type`, `file_data`, `function_call`, `function_response`. Clients that build requests that way (hand-rolled `curl`/`requests` code, copies of Google's docs samples) send exactly those keys. `_has_non_text_parts()` in `headroom/proxy/handlers/gemini.py` only checks the camelCase keys. A snake_case image or tool part is not marked as preserved, so whenever compression rewrites the request (`optimized_messages != messages`), the content is rebuilt from text alone: - an entry with text plus `inline_data` loses the image, - `function_call` / `function_response` entries have no text at all, so they are dropped outright, and Gemini then receives a conversation with the image and tool turns missing. Through the real proxy, a 4-entry request reached the upstream as 2 entries (see the proof below). The same helper feeds the `cloudcode`, `countTokens` and batch paths. Closes: N/A (no matching issue found) ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/gemini.py`: `_has_non_text_parts()` also recognises `inline_data`, `file_data`, `function_call`, `function_response`, `executable_code`, `code_execution_result` and `thought_signature`, the snake_case forms of the keys it already checks; docstring notes both spellings. - `headroom/proxy/handlers/gemini.py`: the telemetry-only `include_function_responses` conversion (#819) reads `function_response` as well as `functionResponse`, so snake_case tool output also reaches waste-signal detection (added after review). - `tests/test_google_multimodal.py`: - the existing `test_each_non_text_key_detected` parametrization gains the seven snake_case keys; - `test_snake_case_parts_survive_compression_round_trip` runs the handler's own `_gemini_contents_to_messages` → modify text → `_messages_to_gemini_contents` → `_rebuild_gemini_contents` sequence on a snake_case image / `function_call` / `function_response` conversation and checks those three entries come back unchanged. - `tests/test_gemini_function_response_waste.py`: `test_flag_emits_tool_message_for_snake_case_function_response`. ## Testing - [x] Unit tests pass (`pytest`): `tests/test_google_multimodal.py`, `tests/test_proxy_handlers_batch.py`, `tests/test_gemini_*.py`, `tests/test_proxy_gemini_*.py`, not the full suite - [x] Linting passes (`ruff check .`): on the changed files - [ ] Type checking passes (`mypy headroom`): not run - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output New tests on `main` (fix reverted, tests kept): ```text $ python -m pytest tests/test_google_multimodal.py -q FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[inline_data] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[file_data] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[function_call] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[function_response] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[executable_code] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[code_execution_result] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[thought_signature] FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_snake_case_parts_survive_compression_round_trip ========================= 8 failed, 61 passed in 8.25s ========================= ``` With the fix: ```text $ python -m pytest tests/test_google_multimodal.py tests/test_proxy_handlers_batch.py tests/test_gemini_*.py tests/test_proxy_gemini_*.py -q ======================= 148 passed, 27 skipped in 16.41s ======================= # (the skips are the live-API integration tests, which need a real GEMINI_API_KEY) $ ruff check headroom/proxy/handlers/gemini.py tests/test_google_multimodal.py tests/test_gemini_function_response_waste.py All checks passed! $ ruff format --check (same files) 3 files already formatted # waste-telemetry test on the previous commit (snake_case not read): E AssertionError: assert ['user'] == ['user', 'tool'] ``` ## Real Behavior Proof - Environment: macOS 27.0.1 (arm64), Python 3.12.11, source tree on this branch (native `headroom._core` from the 0.41.0 wheel), real `headroom proxy` in front of a local stub Gemini upstream (a small `ThreadingHTTPServer` that logs every request body and returns a canned `generateContent` reply), `HEADROOM_OFFLINE=1` - Exact command / steps: started the stub on :18931 and `HEADROOM_OFFLINE=1 python -m headroom.cli proxy --port 18932 --mode token --gemini-api-url http://127.0.0.1:18931 --no-rate-limit`, then `POST /v1beta/models/gemini-2.5-flash:generateContent` with 4 snake_case entries: user text + `inline_data` PNG, model `function_call`, user `function_response`, and a user turn with a 300-row JSON log (33 KB) so compression rewrites the request. Read back what the stub received. Same steps on `main` and this branch. - Observed result: both runs compress the log turn (33 KB → 10.8 KB; proxy log: `Tokens saved: 4,482`). On `main` the upstream gets 2 of 4 entries, with the image, `function_call` and `function_response` gone; on this branch it gets all 4, with only the log text compressed. ```text == main proxy status: 200 upstream received 2 of 4 contents entries user ['text'] len=60 user ['text'] len=10822 == this branch proxy status: 200 upstream received 4 of 4 contents entries user ['text', 'inline_data'] len=141 model ['function_call'] len=97 user ['function_response'] len=99 user ['text'] len=10822 ``` - Not tested: a real Gemini API key / live model (stub upstream only); the `cloudcode`, `countTokens` and batch endpoints, which call the same helper; streaming `streamGenerateContent`. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: N/A - Stable/default behavior changed: only for requests that use snake_case part names: those entries are now kept verbatim, like their camelCase equivalents already were. camelCase requests are unchanged. - Kill switch / disable path: N/A - Unsafe override required: no - Qualification impact: none - Rollback path: revert this commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - A request where **every** entry has a non-text part is forwarded unchanged by the existing early exit (`len(preserved_indices) == len(contents)`), before memory injection. That is already how camelCase `inlineData` requests behave on `main`; this PR only makes snake_case match it. - `headroom/image/compressor.py` also reads only `inlineData`. A snake_case image there is just not compressed, which is safe, so it isn't part of this PR. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JerrettDavis
parent
2480100915
commit
4bf1bf55d1
@@ -156,6 +156,10 @@ class GeminiHandlerMixin:
|
||||
- thought / thoughtSignature: thinking parts, which must round-trip intact
|
||||
- non-string text, which the text round trip cannot rebuild
|
||||
|
||||
The REST API accepts both the camelCase JSON names and the snake_case
|
||||
proto field names (``inline_data``, ``function_call``, ...), so both
|
||||
spellings are checked.
|
||||
|
||||
Args:
|
||||
content: A single Gemini content entry with 'parts' list.
|
||||
|
||||
@@ -174,6 +178,13 @@ class GeminiHandlerMixin:
|
||||
"codeExecutionResult",
|
||||
"thought",
|
||||
"thoughtSignature",
|
||||
"inline_data",
|
||||
"file_data",
|
||||
"function_call",
|
||||
"function_response",
|
||||
"executable_code",
|
||||
"code_execution_result",
|
||||
"thought_signature",
|
||||
)
|
||||
):
|
||||
return True
|
||||
@@ -269,9 +280,10 @@ class GeminiHandlerMixin:
|
||||
|
||||
if include_function_responses:
|
||||
for part in parts:
|
||||
if "functionResponse" not in part:
|
||||
function_response = part.get("functionResponse", part.get("function_response"))
|
||||
if function_response is None:
|
||||
continue
|
||||
payload = self._function_response_text(part["functionResponse"])
|
||||
payload = self._function_response_text(function_response)
|
||||
if payload:
|
||||
messages.append({"role": "tool", "content": payload})
|
||||
|
||||
|
||||
@@ -89,6 +89,23 @@ class TestFunctionResponseConversion:
|
||||
# verbatim on the wire regardless of the telemetry conversion.
|
||||
assert preserved == {1}
|
||||
|
||||
def test_flag_emits_tool_message_for_snake_case_function_response(self, proxy):
|
||||
"""The REST API also accepts ``function_response``; its payload counts too."""
|
||||
payload = _big_payload()
|
||||
contents = [
|
||||
{"role": "user", "parts": [{"text": "fetch the data"}]},
|
||||
{
|
||||
"role": "user",
|
||||
"parts": [{"function_response": {"name": "fetch_data", "response": payload}}],
|
||||
},
|
||||
]
|
||||
messages, preserved = proxy._gemini_contents_to_messages(
|
||||
contents, include_function_responses=True
|
||||
)
|
||||
assert [m["role"] for m in messages] == ["user", "tool"]
|
||||
assert json.loads(messages[1]["content"]) == payload
|
||||
assert preserved == {1}
|
||||
|
||||
def test_flag_passes_string_response_through(self, proxy):
|
||||
contents = [_function_response_content("plain text tool output")]
|
||||
messages, _ = proxy._gemini_contents_to_messages(contents, include_function_responses=True)
|
||||
|
||||
@@ -182,6 +182,15 @@ class TestHasNonTextParts:
|
||||
# turns; previously not detected, so they were dropped on round-trip.
|
||||
"executableCode",
|
||||
"codeExecutionResult",
|
||||
# The REST API also accepts the proto field names, which Google's
|
||||
# own curl examples use; these were dropped on round-trip too.
|
||||
"inline_data",
|
||||
"file_data",
|
||||
"function_call",
|
||||
"function_response",
|
||||
"executable_code",
|
||||
"code_execution_result",
|
||||
"thought_signature",
|
||||
],
|
||||
)
|
||||
def test_each_non_text_key_detected(self, proxy, non_text_key):
|
||||
@@ -194,6 +203,36 @@ class TestHasNonTextParts:
|
||||
content = {"role": "user"}
|
||||
assert proxy._has_non_text_parts(content) is False
|
||||
|
||||
def test_snake_case_parts_survive_compression_round_trip(self, proxy):
|
||||
"""snake_case image and tool parts are restored after compression rewrites text."""
|
||||
contents = [
|
||||
{
|
||||
"role": "user",
|
||||
"parts": [
|
||||
{"text": "Describe this image"},
|
||||
{"inline_data": {"mime_type": "image/png", "data": "iVBORw0KGgo="}},
|
||||
],
|
||||
},
|
||||
{"role": "model", "parts": [{"function_call": {"name": "lookup", "args": {}}}]},
|
||||
{
|
||||
"role": "user",
|
||||
"parts": [{"function_response": {"name": "lookup", "response": {"r": "ok"}}}],
|
||||
},
|
||||
{"role": "user", "parts": [{"text": "thanks " * 100}]},
|
||||
]
|
||||
messages, preserved = proxy._gemini_contents_to_messages(contents)
|
||||
assert preserved == {0, 1, 2}
|
||||
|
||||
optimized = [dict(m) for m in messages]
|
||||
optimized[-1]["content"] = "thanks (compressed)"
|
||||
optimized_contents, _ = proxy._messages_to_gemini_contents(optimized)
|
||||
rebuilt = proxy._rebuild_gemini_contents(
|
||||
contents, preserved, {i: contents[i] for i in preserved}, optimized_contents
|
||||
)
|
||||
|
||||
assert rebuilt[:3] == contents[:3]
|
||||
assert rebuilt[3] == {"role": "user", "parts": [{"text": "thanks (compressed)"}]}
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Tests for _gemini_contents_to_messages preserved indices
|
||||
|
||||
Reference in New Issue
Block a user