Files
headroom/wiki/image-compression.md
Tejas ChopraandClaude Opus 5 27bdc2a899 docs(wiki): correct the unpublished tree, and remove invented telemetry (#3390)
`wiki/` **is not published anywhere.** `.github/workflows/docs.yml` says
so in its own header, no mkdocs or sphinx config exists, and `wiki/**`
sits in `paths-ignore` for CI — changes there don't even run a build. It
survives as pending-migration reference material for contributors.

That's why this is separate from the published-site work in #3388:
nothing here reaches a user through `docs.headroomlabs.ai`. It still
matters, because a contributor migrating a page would carry its errors
across, and because the repo is public.

## The finding that justifies this PR on its own

`wiki/benchmarks.md` carried an entire **invented "Production Telemetry"
section**, presented as real-world fleet data:

> real-world production telemetry from 250+ active proxy instances …
**52ms median overhead** … **1.4 billion tokens saved** across 249
instances

Plus P90/P99 timings, ~$4,000 saved, an OS split, a version-share
breakdown, and a dated collection window of March 30 – April 2, 2026.

A repo-wide grep for those figures **matches only that one file.** No
script, test or fixture produces them. The published benchmarks page
never carried the section, so the site owner had already excised it.

Removed rather than re-estimated. The per-category ranges hanging off it
(4.8%, 40–80%, 83–94%, 85–94%, 60–76%) went with it and now point at the
concrete compression table on the same page.

The same page also described `HEADROOM_TELEMETRY` as the opt-in switch
governing fleet-wide numbers. Fleet data would come from
`HEADROOM_BEACON` — a **separate switch that defaults ON**. That note is
gone with the section it footnoted; the accurate description of both
switches shipped to the live site in #3388.

## Other corrections, each verified against source

- **`wiki/agno.md`** constructed `HeadroomAgnoModel(config=...)`; the
field is `headroom_config` (`integrations/agno/model.py:135`). The
sibling `HeadroomPreHook` genuinely does take `config`, which is how
they got conflated.
- **`wiki/ccr.md`** documented CLI flags that don't exist, and a dead
demo script.
- **`wiki/image-compression.md`** had two broken CLI examples.
- **`wiki/getting-started.md`** used `plan.transforms_applied`;
`SimulationResult` exposes `transforms`. (`CompressResult` really does
have `transforms_applied` — two different result types, easy to cross.)
- **`wiki/cli.md`** documents 15 of ~27 top-level commands; an audit
note now says so rather than implying completeness.

## Disposition, for whoever picks up the migration

`wiki/cli.md` is the highest-value page to port — no site page is a full
CLI reference, and `docs.yml` names it as the reason `wiki/` still
exists. `wiki/integration-guide.md` and `wiki/configuration.md` also
cover ground the site doesn't.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01HRaEPjQcCSAwD2xbQMNENu

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-02 15:53:02 +05:30

325 lines
8.0 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Image Compression
Headroom automatically compresses images in your LLM requests, reducing token usage by **40-90%** while maintaining answer accuracy.
## Overview
Vision models charge by the token, and images are expensive:
- A 1024x1024 image costs ~765 tokens (OpenAI)
- A 2048x2048 image costs ~2,900 tokens
Headroom's image compression uses a **trained ML router** to analyze your query and automatically select the optimal compression technique:
| Technique | Savings | When Used |
|-----------|---------|-----------|
| `full_low` | ~87% | General questions ("What is this?") |
| `preserve` | 0% | Fine details needed ("Count the whiskers") |
| `crop` | 50-90% | Region-specific ("What's in the corner?") |
| `transcode` | ~99% | Text extraction ("Read the sign") |
## How It Works
```
User uploads image + asks question
↓
[Query Analysis]
TrainedRouter (MiniLM from HuggingFace)
Classifies: "What animal is this?" → full_low
↓
[Image Analysis]
SigLIP analyzes image properties
(has text? complex? fine details?)
↓
[Apply Compression]
OpenAI: detail="low"
Anthropic: Resize to 512px
Google: Resize to 768px
↓
Compressed request to LLM
```
## Quick Start
### With Headroom Proxy (Zero Code Changes)
```bash
# Start the proxy
headroom proxy --port 8787
# Connect your client
ANTHROPIC_BASE_URL=http://localhost:8787 claude
```
Images are automatically compressed based on your queries.
### With HeadroomClient
```python
from headroom import HeadroomClient, OpenAIProvider
from openai import OpenAI
client = HeadroomClient(original_client=OpenAI(), provider=OpenAIProvider())
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "What animal is this?"},
{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}},
],
}
],
)
# Image automatically compressed with detail="low" (87% savings)
```
### Direct API
```python
from headroom.image import ImageCompressor
compressor = ImageCompressor()
# Compress images in messages
compressed_messages = compressor.compress(messages, provider="openai")
# Check savings
print(f"Saved {compressor.last_savings:.0f}% tokens")
print(f"Technique: {compressor.last_result.technique.value}")
```
## Configuration
### Proxy Configuration
```bash
# Image compression runs as part of the `image` built-in compressor and is
# enabled by default. There is no dedicated --image-optimize toggle; select
# compressors explicitly to disable it (flag is singular: --compressor):
headroom proxy --compressor smart_crusher,kompress,code_aware,search,log,tabular,config,html
```
### Programmatic Configuration
```python
from headroom.image import ImageCompressor
compressor = ImageCompressor(
model_id="chopratejas/technique-router", # HuggingFace model
use_siglip=True, # Enable image analysis
device="cuda", # Use GPU if available
)
```
## Provider Support
| Provider | Detection | Compression Method |
|----------|-----------|-------------------|
| **OpenAI** | `image_url` | Sets `detail="low"` |
| **Anthropic** | `image` with `source` | Resizes to 512px |
| **Google** | `inlineData` | Resizes to 768px (tile-optimized) |
### OpenAI
Uses the native `detail` parameter:
```python
# Before
{"type": "image_url", "image_url": {"url": "data:..."}}
# After (full_low technique)
{"type": "image_url", "image_url": {"url": "data:...", "detail": "low"}}
```
### Anthropic
Resizes the image using PIL:
```python
# Before: 1024x1024 image (~1,398 tokens)
# After: 512x512 image (~349 tokens) - 75% savings
```
### Google Gemini
Resizes to 768px (optimal for Gemini's 768x768 tile system):
```python
# Before: 1536x1536 image (4 tiles × 258 = 1,032 tokens)
# After: 768x768 image (1 tile × 258 = 258 tokens) - 75% savings
```
## Techniques Explained
### `full_low` (87% savings)
Best for general understanding questions:
- "What is this?"
- "Describe the scene"
- "Is this indoors or outdoors?"
The model doesn't need fine details to answer these questions.
### `preserve` (0% savings)
Required when fine details matter:
- "Count the whiskers"
- "What brand is shown?"
- "Read the serial number"
- "What time does the clock show?"
### `crop` (50-90% savings)
For region-specific queries:
- "What's in the top-right corner?"
- "Focus on the background"
- "Zoom into the left side"
*Note: Currently implemented as resize. True cropping coming soon.*
### `transcode` (99% savings)
For text extraction (converts image to text):
- "Read the sign"
- "What does it say?"
- "Transcribe the document"
*Note: Runs OCR and replaces the image with the extracted text. If OCR fails or returns low confidence, it falls back to `full_low` (not `preserve`).*
## The Trained Router
The routing decision is made by a fine-tuned **MiniLM** classifier:
- **Model**: `chopratejas/technique-router` on HuggingFace
- **Size**: ~128MB
- **Accuracy**: 93.7% on validation set
- **Training data**: 1,157 examples across 4 techniques
The model is downloaded automatically on first use and cached locally.
### Training Data Examples
| Query | Technique |
|-------|-----------|
| "What animal is this?" | `full_low` |
| "Count the spots" | `preserve` |
| "Read the text on the sign" | `transcode` |
| "What's in the corner?" | `crop` |
## Performance
### Token Savings by Query Type
| Query Type | Before | After | Savings |
|------------|--------|-------|---------|
| General ("What is this?") | 765 | 85 | 89% |
| Detail ("Count items") | 765 | 765 | 0% |
| Region ("Top corner?") | 765 | 85 | 89% |
| Text ("Read the sign") | 765 | 85 | 89% |
### Latency
- Router inference: ~10ms (CPU), ~2ms (GPU)
- Image resize: ~5-20ms depending on size
- First request: +2-3s (model download, cached after)
## Troubleshooting
### Model Download Issues
The HuggingFace model downloads on first use:
```python
# Force a specific cache directory
import os
os.environ["HF_HOME"] = "/path/to/cache"
from headroom.image import ImageCompressor
compressor = ImageCompressor()
```
### GPU Memory
SigLIP requires ~400MB GPU memory. To use CPU only:
```python
compressor = ImageCompressor(device="cpu")
```
### Disable Image Compression
```bash
# Proxy (flag is singular: --compressor)
headroom proxy --compressor smart_crusher,kompress,code_aware,search,log,tabular,config,html
```
```python
# Direct
# Simply don't call compress()
```
## API Reference
### `ImageCompressor`
```python
class ImageCompressor:
def __init__(
self,
model_id: str | None = None, # resolves to "chopratejas/technique-router" if unset
use_siglip: bool = True,
device: str | None = None,
): ...
def has_images(self, messages: list[dict]) -> bool:
"""Check if messages contain images."""
def compress(
self,
messages: list[dict],
provider: str = "openai",
) -> list[dict]:
"""Compress images in messages."""
@property
def last_result(self) -> CompressionResult | None:
"""Result of last compression."""
@property
def last_savings(self) -> float:
"""Savings percentage from last compression."""
```
### `CompressionResult`
```python
@dataclass
class CompressionResult:
technique: Technique # full_low, preserve, crop, transcode
original_tokens: int # Estimated tokens before
compressed_tokens: int # Estimated tokens after
confidence: float # Router confidence (0-1)
@property
def savings_percent(self) -> float:
"""Percentage of tokens saved."""
```
### `Technique`
```python
class Technique(Enum):
FULL_LOW = "full_low" # 87% savings
PRESERVE = "preserve" # 0% savings
CROP = "crop" # 50-90% savings
TRANSCODE = "transcode" # 99% savings
```
## See Also
- [Compression Guide](compression.md) - Text compression techniques
- [CCR Guide](ccr.md) - Reversible compression with retrieval
- [Proxy Guide](proxy.md) - Zero-code deployment
- [Architecture](ARCHITECTURE.md) - System design