mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat/https-compose-caddy
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
9f5cfcfd2b |
fix(image): flatten transparent images onto white before re-encoding as JPEG (#4296)
## Description
When Headroom resizes an image it re-encodes it as JPEG. That happens in
both `headroom/image/tile_optimizer.py:_resize_image_bytes`
(tile-boundary step) and
`headroom/image/compressor.py:ImageCompressor._resize_image`
(provider-format resize). Both prepare the image the same way:
```python
if resized.mode in ("RGBA", "P"):
resized = resized.convert("RGB")
```
`convert("RGB")` drops the alpha channel rather than compositing it, so
every fully transparent pixel shows whatever RGB value it was stored
with, which is usually `(0, 0, 0)`. A transparent-background PNG, such
as a diagram, chart export, logo or UI asset with dark text, comes out
as a **solid black image**. Through `ImageCompressor.compress(...,
provider="openai")` the model receives one colour and the text is gone
(proof below).
The same thing happens with the other kinds of PNG transparency: a
transparent palette entry (`P`), and a `tRNS` colour key on an `RGB` or
`L` image. Separately, `LA` (grayscale + alpha) raises `OSError: cannot
write mode LA as JPEG`, so `compress()` catches it and skips the
optimization.
Closes: N/A (no matching issue found)
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/image/tile_optimizer.py`: new `_jpeg_ready(img)` helper.
- Any image with transparency data (Pillow's `has_transparency_data`: an
alpha band, a palette `transparency` entry, or `tRNS` on RGB/L) is
composited onto a white RGB canvas using its alpha as the mask.
- `1`, `P`, `CMYK` and `YCbCr` images convert to RGB; RGB and L pass
through unchanged.
- High-bit-depth modes (`I;16`, `I`, `F`) are returned unconverted, so
the JPEG save fails as it does today and callers keep the original
image. A plain `convert("RGB")` would clip them to white.
- `_resize_image_bytes` uses the helper before `save(format="JPEG")`.
- `headroom/image/compressor.py`: `_resize_image` uses the same helper
in place of its copy of the `RGBA`/`P` conversion.
- `tests/test_image_compression.py` (runs without torch;
`tests/test_image_compressor.py` skips without it). The pixel checks
decode the JPEG that would be sent:
- `test_transparent_png_is_flattened_onto_white` (RGBA);
- `test_trns_and_palette_transparency_are_flattened_onto_white[RGB, L,
P]`;
- `test_grayscale_alpha_is_converted_not_raised` (LA);
- `test_compressor_resize_flattens_transparency_onto_white`
(`ImageCompressor._resize_image`);
- `test_high_bit_depth_is_not_squashed_to_8_bit`, which guards that
`I;16` is left unconverted.
## Testing
- [x] Unit tests pass (`pytest`): `tests/test_image_compression*.py`,
`tests/test_image_memo.py`, `tests/test_image_log_redaction.py`,
`tests/test_provider_openai_images.py`,
`tests/test_provider_codex_images.py`, not the full suite
- [x] Linting passes (`ruff check .`): on the changed files
- [ ] Type checking passes (`mypy headroom`): not run
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
New tests on `main` (fix reverted, tests kept):
```text
$ python -m pytest tests/test_image_compression.py -q -k "transparen or trns or grayscale_alpha or high_bit or flattens"
E OSError: cannot write mode LA as JPEG
E ImportError: cannot import name '_jpeg_ready' from 'headroom.image.tile_optimizer'
FAILED tests/test_image_compression.py::TestMessageOptimization::test_transparent_png_is_flattened_onto_white
FAILED tests/test_image_compression.py::TestMessageOptimization::test_trns_and_palette_transparency_are_flattened_onto_white[RGB]
FAILED tests/test_image_compression.py::TestMessageOptimization::test_trns_and_palette_transparency_are_flattened_onto_white[L]
FAILED tests/test_image_compression.py::TestMessageOptimization::test_trns_and_palette_transparency_are_flattened_onto_white[P]
FAILED tests/test_image_compression.py::TestMessageOptimization::test_grayscale_alpha_is_converted_not_raised
FAILED tests/test_image_compression.py::TestMessageOptimization::test_high_bit_depth_is_not_squashed_to_8_bit
FAILED tests/test_image_compression.py::TestMessageOptimization::test_compressor_resize_flattens_transparency_onto_white
======================= 7 failed, 29 deselected in 2.89s =======================
```
(The `I;16` guard fails on `main` only because the helper doesn't exist
yet. It's there to keep high-bit-depth images from being clipped.)
With the fix:
```text
$ python -m pytest tests/test_image_compression*.py tests/test_image_memo.py tests/test_image_log_redaction.py \
tests/test_provider_openai_images.py tests/test_provider_codex_images.py -q
======================== 95 passed, 4 skipped in 11.18s ========================
$ ruff check headroom/image tests/test_image_compression.py
All checks passed!
$ ruff format --check headroom/image/tile_optimizer.py tests/test_image_compression.py
2 files already formatted
```
## Real Behavior Proof
- Environment: macOS 27.0.1 (arm64), Python 3.12.11, source tree on this
branch (native `headroom._core` from the 0.41.0 wheel), Pillow 13.0.0,
`HEADROOM_OFFLINE=1` (no torch, so ML technique routing falls back as it
does in a default proxy install)
- Exact command / steps: drew a 1920×1080 RGBA PNG with a transparent
background and a black label bar, put it in an OpenAI-format user
message (`image_url` data URL) with the text "Describe this diagram",
ran `ImageCompressor().compress(messages, provider="openai")` (what the
proxy's image step calls), decoded the image it would send, and printed
the background pixel, a label pixel and the number of distinct colours.
Same script on `main` and on this branch.
- Observed result: both runs resize to a 910×512 JPEG. On `main` the
image sent is a single colour: background `(0, 0, 0)`, the same as the
label, so the label is invisible. On this branch the background is white
`(255, 255, 255)` and the label stays black.
```text
== main
sent: data:image/jpeg (910, 512) | background pixel: (0, 0, 0) | label
pixel: (0, 0, 0)
distinct colours: 1
== this branch
sent: data:image/jpeg (910, 512) | background pixel: (255, 255, 255) |
label pixel: (0, 0, 0)
distinct colours: 43
```
- Not tested: a live model call; Anthropic and Google payloads through
`ImageCompressor` (they use the same `_resize_image`); semi-transparent
pixels are composited onto white, which matches how most viewers show a
PNG on a light page but isn't the only possible choice.
## Runtime Rollout Safety
- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: only for images with transparency
(now flattened onto white instead of turning black) and LA images (now
resized instead of skipped). Opaque RGB / L images are encoded exactly
as before, and high-bit-depth images still fall back to the original as
before.
- Kill switch / disable path: N/A
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
`_jpeg_ready` lives in `tile_optimizer.py`, which has no heavy imports,
and `compressor.py` imports it lazily inside `_resize_image`, the same
way it already imports PIL there.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
|
||
|
|
13081f4647 |
Skip image tests when Pillow not installed (CI fix)
Tests that create PIL images now have @needs_pillow decorator. Token estimation and tile math tests still run without Pillow. |
||
|
|
65e4a35546 |
Add OCR routing via RapidOCR + measure actual tokens after compression
Transcode technique now runs RapidOCR to extract text from images. Falls back to full_low if OCR confidence < 70% or no text detected. Token counting is now done AFTER compression by measuring the actual output — no hardcoded estimates. OCR text counted by char length, resized images re-estimated from new dimensions. RapidOCR added to headroom-ai[image] extra (~15MB ONNX models, ~180ms CPU). 4 new OCR tests (extraction, blank image, confidence threshold, full pipeline). 29 total image compression tests passing. |
||
|
|
7984bd2389 |
Re-enable image compression: tile optimizer + ONNX router, no PyTorch
Image compression pipeline: 1. Tile-boundary alignment (pure math, 0ms) — saves 340-680 tokens/image on OpenAI by snapping to 512px tile boundaries 2. ONNX technique router (32MB, 6ms) — classifies query intent 3. ONNX SigLIP image encoder (95MB, 49ms) — analyzes image properties Total: ~127MB models, ~55ms latency, no PyTorch dependency. Re-enabled in both Anthropic and OpenAI proxy handlers. 25 tests. |