Commit Graph
3 Commits
Author SHA1 Message Date
4bf1bf55d1 fix(gemini): keep snake_case inline_data and function parts when compressing (#4290)
## Description

Gemini's REST API accepts part fields under both their JSON (camelCase)
names and their proto (snake_case) names, and Google's own `curl`
examples use snake_case: `inline_data` / `mime_type`, `file_data`,
`function_call`, `function_response`. Clients that build requests that
way (hand-rolled `curl`/`requests` code, copies of Google's docs
samples) send exactly those keys.

`_has_non_text_parts()` in `headroom/proxy/handlers/gemini.py` only
checks the camelCase keys. A snake_case image or tool part is not marked
as preserved, so whenever compression rewrites the request
(`optimized_messages != messages`), the content is rebuilt from text
alone:

- an entry with text plus `inline_data` loses the image,
- `function_call` / `function_response` entries have no text at all, so
they are dropped outright,

and Gemini then receives a conversation with the image and tool turns
missing. Through the real proxy, a 4-entry request reached the upstream
as 2 entries (see the proof below). The same helper feeds the
`cloudcode`, `countTokens` and batch paths.

Closes: N/A (no matching issue found)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/gemini.py`: `_has_non_text_parts()` also
recognises `inline_data`, `file_data`, `function_call`,
`function_response`, `executable_code`, `code_execution_result` and
`thought_signature`, the snake_case forms of the keys it already checks;
docstring notes both spellings.
- `headroom/proxy/handlers/gemini.py`: the telemetry-only
`include_function_responses` conversion (#819) reads `function_response`
as well as `functionResponse`, so snake_case tool output also reaches
waste-signal detection (added after review).
- `tests/test_google_multimodal.py`:
- the existing `test_each_non_text_key_detected` parametrization gains
the seven snake_case keys;
- `test_snake_case_parts_survive_compression_round_trip` runs the
handler's own `_gemini_contents_to_messages` → modify text →
`_messages_to_gemini_contents` → `_rebuild_gemini_contents` sequence on
a snake_case image / `function_call` / `function_response` conversation
and checks those three entries come back unchanged.
- `tests/test_gemini_function_response_waste.py`:
`test_flag_emits_tool_message_for_snake_case_function_response`.

## Testing

- [x] Unit tests pass (`pytest`): `tests/test_google_multimodal.py`,
`tests/test_proxy_handlers_batch.py`, `tests/test_gemini_*.py`,
`tests/test_proxy_gemini_*.py`, not the full suite
- [x] Linting passes (`ruff check .`): on the changed files
- [ ] Type checking passes (`mypy headroom`): not run
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

New tests on `main` (fix reverted, tests kept):

```text
$ python -m pytest tests/test_google_multimodal.py -q
FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[inline_data]
FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[file_data]
FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[function_call]
FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[function_response]
FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[executable_code]
FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[code_execution_result]
FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_each_non_text_key_detected[thought_signature]
FAILED tests/test_google_multimodal.py::TestHasNonTextParts::test_snake_case_parts_survive_compression_round_trip
========================= 8 failed, 61 passed in 8.25s =========================
```

With the fix:

```text
$ python -m pytest tests/test_google_multimodal.py tests/test_proxy_handlers_batch.py tests/test_gemini_*.py tests/test_proxy_gemini_*.py -q
======================= 148 passed, 27 skipped in 16.41s =======================
# (the skips are the live-API integration tests, which need a real GEMINI_API_KEY)

$ ruff check headroom/proxy/handlers/gemini.py tests/test_google_multimodal.py tests/test_gemini_function_response_waste.py
All checks passed!
$ ruff format --check (same files)
3 files already formatted

# waste-telemetry test on the previous commit (snake_case not read):
E   AssertionError: assert ['user'] == ['user', 'tool']
```

## Real Behavior Proof

- Environment: macOS 27.0.1 (arm64), Python 3.12.11, source tree on this
branch (native `headroom._core` from the 0.41.0 wheel), real `headroom
proxy` in front of a local stub Gemini upstream (a small
`ThreadingHTTPServer` that logs every request body and returns a canned
`generateContent` reply), `HEADROOM_OFFLINE=1`
- Exact command / steps: started the stub on :18931 and
`HEADROOM_OFFLINE=1 python -m headroom.cli proxy --port 18932 --mode
token --gemini-api-url http://127.0.0.1:18931 --no-rate-limit`, then
`POST /v1beta/models/gemini-2.5-flash:generateContent` with 4 snake_case
entries: user text + `inline_data` PNG, model `function_call`, user
`function_response`, and a user turn with a 300-row JSON log (33 KB) so
compression rewrites the request. Read back what the stub received. Same
steps on `main` and this branch.
- Observed result: both runs compress the log turn (33 KB → 10.8 KB;
proxy log: `Tokens saved: 4,482`). On `main` the upstream gets 2 of 4
entries, with the image, `function_call` and `function_response` gone;
on this branch it gets all 4, with only the log text compressed.

  ```text
  == main
  proxy status: 200
  upstream received 2 of 4 contents entries
     user ['text'] len=60
     user ['text'] len=10822
  == this branch
  proxy status: 200
  upstream received 4 of 4 contents entries
     user ['text', 'inline_data'] len=141
     model ['function_call'] len=97
     user ['function_response'] len=99
     user ['text'] len=10822
  ```
- Not tested: a real Gemini API key / live model (stub upstream only);
the `cloudcode`, `countTokens` and batch endpoints, which call the same
helper; streaming `streamGenerateContent`.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: only for requests that use snake_case
part names: those entries are now kept verbatim, like their camelCase
equivalents already were. camelCase requests are unchanged.
- Kill switch / disable path: N/A
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- A request where **every** entry has a non-text part is forwarded
unchanged by the existing early exit (`len(preserved_indices) ==
len(contents)`), before memory injection. That is already how camelCase
`inlineData` requests behave on `main`; this PR only makes snake_case
match it.
- `headroom/image/compressor.py` also reads only `inlineData`. A
snake_case image there is just not compressed, which is safe, so it
isn't part of this PR.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-24 08:38:10 -07:00
Abhay Singh 07cf547607 fix(proxy/gemini): tolerate malformed parts on the compression path (#2486)
## Description

Three helpers on the Gemini compression path read a content entry's
`parts` and iterate it without type guards:

```python
# _has_non_text_parts
parts = content.get("parts", [])
for part in parts: ...
# _rebuild_gemini_contents
had_text = any("text" in p for p in content.get("parts", []))
# _gemini_contents_to_messages
parts = content.get("parts", [])
text_parts = [p.get("text", "") for p in parts if "text" in p]
```

`parts` is request-controlled and `.get("parts", [])` only falls back
when the key is absent, so:

- a present-but-null `parts` returns `None`, and `for part in None` /
`any(... for p in None)` raises `TypeError`;
- a list carrying a bare string (a client that treats `parts` as a
string array) makes `p.get("text", "")` raise `AttributeError`, while
`"text" in p` silently does substring matching first;
- a null element in the list crashes the same way.

Any of these 500s the request on the compression path, on data that
parsed as valid JSON.

## Fix

Route all three helpers through a shared `_dict_parts(content)` that
returns the dict entries of `parts`, coercing a non-dict content or a
non-list `parts` to an empty list and dropping non-dict elements.
`_gemini_contents_to_messages` also reads `role` defensively for a
non-dict content entry. Conversion now degrades gracefully (the
malformed part contributes nothing) instead of raising. Well-formed
requests are unchanged.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/handlers/gemini.py`: add `_dict_parts`; use it in
`_has_non_text_parts`, `_rebuild_gemini_contents`, and
`_gemini_contents_to_messages`; read `role` defensively for a non-dict
content entry.
- `tests/test_gemini_function_response_waste.py`: regressions for null
`parts`, bare-string part elements, a null part element,
`_has_non_text_parts` on malformed parts, and a non-dict content entry.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_gemini_function_response_waste.py -q
16 passed

# with the fix reverted, the new malformed-parts tests fail with
# TypeError: 'NoneType' object is not iterable  (and AttributeError on string parts)

$ uvx ruff@0.15.17 check headroom/proxy/handlers/gemini.py tests/test_gemini_function_response_waste.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/proxy/handlers/gemini.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: built a real `HeadroomProxy` and called
`_gemini_contents_to_messages` / `_has_non_text_parts` with contents
carrying `parts: null`, `parts: ["bare string", {text}]`, `parts: [null,
{text}]`, and a non-dict content entry; then reverted `gemini.py` and
re-ran.
- Observed result: with the fix each malformed shape converts without
raising and the valid text part is still emitted (`[{"role": "user",
"content": "kept"}]`); with the fix reverted the null-`parts` and
null-element cases raise `TypeError: 'NoneType' object is not iterable`
and the string-element case raises `AttributeError: 'str' object has no
attribute 'get'`. Ran against the actual module via
`tests/test_gemini_function_response_waste.py`.
- Not tested: a live Gemini request with malformed `parts` routed
through the full proxy compression pipeline end to end.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
2026-07-22 06:07:15 -07:00
Focused Instabilityandintegration-check 9b0c840dd7 fix(gemini): surface functionResponse payloads to waste-signal detection (#897)
## Problem

Fixes #819.

Gemini `functionResponse` parts are preserved verbatim on the wire (by
design — they are never compressed), but their payloads never reached
`parse_messages`: `_gemini_contents_to_messages` only extracts `text`
parts. Tool output — where most waste lives — contributed nothing to
waste detection on either Gemini path, so `json_bloat`, `repetition`,
and the new `reread` signal (#853/#854) were all blind to it.

## Fix (telemetry-only)

1. **`_gemini_contents_to_messages(...,
include_function_responses=True)`** — new keyword-only flag. When set,
each `functionResponse` payload is additionally emitted as a
`role="tool"` message (dict payloads JSON-serialized, strings passed
through, missing/`None` responses skipped). `preserved_indices`
semantics are unchanged: the entries are still restored verbatim on the
wire.
2. **`TransformPipeline.apply(..., waste_messages=...)`** — new optional
kwarg (popped before transforms, like `record_metrics`). When provided,
the waste-signal parse runs over this richer list instead of the
transform input. Transforms, token accounting, and savings deltas are
untouched — this is why the richer list is not simply fed to the
pipeline: compressed copies of preserved entries are discarded on
rebuild, which would corrupt savings reporting.
3. Both Gemini `generateContent` paths (native + Cloud Code Assist)
build the enriched list and pass it through.

The existing `role="tool"` parsing from #815 handles the rest:
tool_result blocks, waste flags, and reread grouping all apply.

## Tests

`tests/test_gemini_function_response_waste.py` — 11 new tests:
- conversion: default unchanged (regression), dict/string payloads,
missing response skipped, text-before-tool ordering, preserved_indices
unchanged, circular-reference fallback
- parsing: functionResponse payload produces tool_result blocks +
`json_bloat`; identical payloads far apart count as `reread`
- pipeline: `waste_messages` overrides the waste source, does not affect
transform output/token counts, falls back to transform input when absent

Full local sweep of touched suites: gemini multimodal, parser, safety
rails, canonical pipeline — green. The 13 failures in
`test_proxy_gemini_*_integration.py` are credential-dependent and
identical on clean `main`.

## Live proof

Mock Gemini upstream on a real port, proxy with `optimize=True`;
conversation with a large functionResponse payload served twice (5
messages apart) plus compressible model text:

```
waste_signals: { "json_bloat": 35003, "reread": 11673, ... }
PROOF OK: waste visible, wire verbatim
```

Upstream received both `functionResponse` entries byte-identical to the
client request.

## Known limitations / follow-ups

- The Cloud Code Assist path passes `waste_messages` but does not yet
consume `result.waste_signals` into a recorded outcome (pre-existing
gap; the native path records it).
- Requests where **all** content entries are preserved (pure
functionResponse/media conversations) early-exit before the pipeline and
still produce no waste signals.
- Codex/Responses-API counterpart is #820 (separate PR).

Co-authored-by: integration-check <integration@local>
2026-06-12 17:09:20 -05:00