mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-23 02:49:17 +08:00
`wiki/` **is not published anywhere.** `.github/workflows/docs.yml` says so in its own header, no mkdocs or sphinx config exists, and `wiki/**` sits in `paths-ignore` for CI — changes there don't even run a build. It survives as pending-migration reference material for contributors. That's why this is separate from the published-site work in #3388: nothing here reaches a user through `docs.headroomlabs.ai`. It still matters, because a contributor migrating a page would carry its errors across, and because the repo is public. ## The finding that justifies this PR on its own `wiki/benchmarks.md` carried an entire **invented "Production Telemetry" section**, presented as real-world fleet data: > real-world production telemetry from 250+ active proxy instances … **52ms median overhead** … **1.4 billion tokens saved** across 249 instances Plus P90/P99 timings, ~$4,000 saved, an OS split, a version-share breakdown, and a dated collection window of March 30 – April 2, 2026. A repo-wide grep for those figures **matches only that one file.** No script, test or fixture produces them. The published benchmarks page never carried the section, so the site owner had already excised it. Removed rather than re-estimated. The per-category ranges hanging off it (4.8%, 40–80%, 83–94%, 85–94%, 60–76%) went with it and now point at the concrete compression table on the same page. The same page also described `HEADROOM_TELEMETRY` as the opt-in switch governing fleet-wide numbers. Fleet data would come from `HEADROOM_BEACON` — a **separate switch that defaults ON**. That note is gone with the section it footnoted; the accurate description of both switches shipped to the live site in #3388. ## Other corrections, each verified against source - **`wiki/agno.md`** constructed `HeadroomAgnoModel(config=...)`; the field is `headroom_config` (`integrations/agno/model.py:135`). The sibling `HeadroomPreHook` genuinely does take `config`, which is how they got conflated. - **`wiki/ccr.md`** documented CLI flags that don't exist, and a dead demo script. - **`wiki/image-compression.md`** had two broken CLI examples. - **`wiki/getting-started.md`** used `plan.transforms_applied`; `SimulationResult` exposes `transforms`. (`CompressResult` really does have `transforms_applied` — two different result types, easy to cross.) - **`wiki/cli.md`** documents 15 of ~27 top-level commands; an audit note now says so rather than implying completeness. ## Disposition, for whoever picks up the migration `wiki/cli.md` is the highest-value page to port — no site page is a full CLI reference, and `docs.yml` names it as the reason `wiki/` still exists. `wiki/integration-guide.md` and `wiki/configuration.md` also cover ground the site doesn't. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01HRaEPjQcCSAwD2xbQMNENu Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
325 lines
8.0 KiB
Markdown
325 lines
8.0 KiB
Markdown
# Image Compression
|
||
|
||
Headroom automatically compresses images in your LLM requests, reducing token usage by **40-90%** while maintaining answer accuracy.
|
||
|
||
## Overview
|
||
|
||
Vision models charge by the token, and images are expensive:
|
||
- A 1024x1024 image costs ~765 tokens (OpenAI)
|
||
- A 2048x2048 image costs ~2,900 tokens
|
||
|
||
Headroom's image compression uses a **trained ML router** to analyze your query and automatically select the optimal compression technique:
|
||
|
||
| Technique | Savings | When Used |
|
||
|-----------|---------|-----------|
|
||
| `full_low` | ~87% | General questions ("What is this?") |
|
||
| `preserve` | 0% | Fine details needed ("Count the whiskers") |
|
||
| `crop` | 50-90% | Region-specific ("What's in the corner?") |
|
||
| `transcode` | ~99% | Text extraction ("Read the sign") |
|
||
|
||
## How It Works
|
||
|
||
```
|
||
User uploads image + asks question
|
||
↓
|
||
[Query Analysis]
|
||
TrainedRouter (MiniLM from HuggingFace)
|
||
Classifies: "What animal is this?" → full_low
|
||
↓
|
||
[Image Analysis]
|
||
SigLIP analyzes image properties
|
||
(has text? complex? fine details?)
|
||
↓
|
||
[Apply Compression]
|
||
OpenAI: detail="low"
|
||
Anthropic: Resize to 512px
|
||
Google: Resize to 768px
|
||
↓
|
||
Compressed request to LLM
|
||
```
|
||
|
||
## Quick Start
|
||
|
||
### With Headroom Proxy (Zero Code Changes)
|
||
|
||
```bash
|
||
# Start the proxy
|
||
headroom proxy --port 8787
|
||
|
||
# Connect your client
|
||
ANTHROPIC_BASE_URL=http://localhost:8787 claude
|
||
```
|
||
|
||
Images are automatically compressed based on your queries.
|
||
|
||
### With HeadroomClient
|
||
|
||
```python
|
||
from headroom import HeadroomClient, OpenAIProvider
|
||
from openai import OpenAI
|
||
|
||
client = HeadroomClient(original_client=OpenAI(), provider=OpenAIProvider())
|
||
|
||
response = client.chat.completions.create(
|
||
model="gpt-4o",
|
||
messages=[
|
||
{
|
||
"role": "user",
|
||
"content": [
|
||
{"type": "text", "text": "What animal is this?"},
|
||
{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}},
|
||
],
|
||
}
|
||
],
|
||
)
|
||
# Image automatically compressed with detail="low" (87% savings)
|
||
```
|
||
|
||
### Direct API
|
||
|
||
```python
|
||
from headroom.image import ImageCompressor
|
||
|
||
compressor = ImageCompressor()
|
||
|
||
# Compress images in messages
|
||
compressed_messages = compressor.compress(messages, provider="openai")
|
||
|
||
# Check savings
|
||
print(f"Saved {compressor.last_savings:.0f}% tokens")
|
||
print(f"Technique: {compressor.last_result.technique.value}")
|
||
```
|
||
|
||
## Configuration
|
||
|
||
### Proxy Configuration
|
||
|
||
```bash
|
||
# Image compression runs as part of the `image` built-in compressor and is
|
||
# enabled by default. There is no dedicated --image-optimize toggle; select
|
||
# compressors explicitly to disable it (flag is singular: --compressor):
|
||
headroom proxy --compressor smart_crusher,kompress,code_aware,search,log,tabular,config,html
|
||
```
|
||
|
||
### Programmatic Configuration
|
||
|
||
```python
|
||
from headroom.image import ImageCompressor
|
||
|
||
compressor = ImageCompressor(
|
||
model_id="chopratejas/technique-router", # HuggingFace model
|
||
use_siglip=True, # Enable image analysis
|
||
device="cuda", # Use GPU if available
|
||
)
|
||
```
|
||
|
||
## Provider Support
|
||
|
||
| Provider | Detection | Compression Method |
|
||
|----------|-----------|-------------------|
|
||
| **OpenAI** | `image_url` | Sets `detail="low"` |
|
||
| **Anthropic** | `image` with `source` | Resizes to 512px |
|
||
| **Google** | `inlineData` | Resizes to 768px (tile-optimized) |
|
||
|
||
### OpenAI
|
||
|
||
Uses the native `detail` parameter:
|
||
```python
|
||
# Before
|
||
{"type": "image_url", "image_url": {"url": "data:..."}}
|
||
|
||
# After (full_low technique)
|
||
{"type": "image_url", "image_url": {"url": "data:...", "detail": "low"}}
|
||
```
|
||
|
||
### Anthropic
|
||
|
||
Resizes the image using PIL:
|
||
```python
|
||
# Before: 1024x1024 image (~1,398 tokens)
|
||
# After: 512x512 image (~349 tokens) - 75% savings
|
||
```
|
||
|
||
### Google Gemini
|
||
|
||
Resizes to 768px (optimal for Gemini's 768x768 tile system):
|
||
```python
|
||
# Before: 1536x1536 image (4 tiles × 258 = 1,032 tokens)
|
||
# After: 768x768 image (1 tile × 258 = 258 tokens) - 75% savings
|
||
```
|
||
|
||
## Techniques Explained
|
||
|
||
### `full_low` (87% savings)
|
||
|
||
Best for general understanding questions:
|
||
- "What is this?"
|
||
- "Describe the scene"
|
||
- "Is this indoors or outdoors?"
|
||
|
||
The model doesn't need fine details to answer these questions.
|
||
|
||
### `preserve` (0% savings)
|
||
|
||
Required when fine details matter:
|
||
- "Count the whiskers"
|
||
- "What brand is shown?"
|
||
- "Read the serial number"
|
||
- "What time does the clock show?"
|
||
|
||
### `crop` (50-90% savings)
|
||
|
||
For region-specific queries:
|
||
- "What's in the top-right corner?"
|
||
- "Focus on the background"
|
||
- "Zoom into the left side"
|
||
|
||
*Note: Currently implemented as resize. True cropping coming soon.*
|
||
|
||
### `transcode` (99% savings)
|
||
|
||
For text extraction (converts image to text):
|
||
- "Read the sign"
|
||
- "What does it say?"
|
||
- "Transcribe the document"
|
||
|
||
*Note: Runs OCR and replaces the image with the extracted text. If OCR fails or returns low confidence, it falls back to `full_low` (not `preserve`).*
|
||
|
||
## The Trained Router
|
||
|
||
The routing decision is made by a fine-tuned **MiniLM** classifier:
|
||
|
||
- **Model**: `chopratejas/technique-router` on HuggingFace
|
||
- **Size**: ~128MB
|
||
- **Accuracy**: 93.7% on validation set
|
||
- **Training data**: 1,157 examples across 4 techniques
|
||
|
||
The model is downloaded automatically on first use and cached locally.
|
||
|
||
### Training Data Examples
|
||
|
||
| Query | Technique |
|
||
|-------|-----------|
|
||
| "What animal is this?" | `full_low` |
|
||
| "Count the spots" | `preserve` |
|
||
| "Read the text on the sign" | `transcode` |
|
||
| "What's in the corner?" | `crop` |
|
||
|
||
## Performance
|
||
|
||
### Token Savings by Query Type
|
||
|
||
| Query Type | Before | After | Savings |
|
||
|------------|--------|-------|---------|
|
||
| General ("What is this?") | 765 | 85 | 89% |
|
||
| Detail ("Count items") | 765 | 765 | 0% |
|
||
| Region ("Top corner?") | 765 | 85 | 89% |
|
||
| Text ("Read the sign") | 765 | 85 | 89% |
|
||
|
||
### Latency
|
||
|
||
- Router inference: ~10ms (CPU), ~2ms (GPU)
|
||
- Image resize: ~5-20ms depending on size
|
||
- First request: +2-3s (model download, cached after)
|
||
|
||
## Troubleshooting
|
||
|
||
### Model Download Issues
|
||
|
||
The HuggingFace model downloads on first use:
|
||
|
||
```python
|
||
# Force a specific cache directory
|
||
import os
|
||
|
||
os.environ["HF_HOME"] = "/path/to/cache"
|
||
|
||
from headroom.image import ImageCompressor
|
||
|
||
compressor = ImageCompressor()
|
||
```
|
||
|
||
### GPU Memory
|
||
|
||
SigLIP requires ~400MB GPU memory. To use CPU only:
|
||
|
||
```python
|
||
compressor = ImageCompressor(device="cpu")
|
||
```
|
||
|
||
### Disable Image Compression
|
||
|
||
```bash
|
||
# Proxy (flag is singular: --compressor)
|
||
headroom proxy --compressor smart_crusher,kompress,code_aware,search,log,tabular,config,html
|
||
```
|
||
|
||
```python
|
||
# Direct
|
||
# Simply don't call compress()
|
||
```
|
||
|
||
## API Reference
|
||
|
||
### `ImageCompressor`
|
||
|
||
```python
|
||
class ImageCompressor:
|
||
def __init__(
|
||
self,
|
||
model_id: str | None = None, # resolves to "chopratejas/technique-router" if unset
|
||
use_siglip: bool = True,
|
||
device: str | None = None,
|
||
): ...
|
||
|
||
def has_images(self, messages: list[dict]) -> bool:
|
||
"""Check if messages contain images."""
|
||
|
||
def compress(
|
||
self,
|
||
messages: list[dict],
|
||
provider: str = "openai",
|
||
) -> list[dict]:
|
||
"""Compress images in messages."""
|
||
|
||
@property
|
||
def last_result(self) -> CompressionResult | None:
|
||
"""Result of last compression."""
|
||
|
||
@property
|
||
def last_savings(self) -> float:
|
||
"""Savings percentage from last compression."""
|
||
```
|
||
|
||
### `CompressionResult`
|
||
|
||
```python
|
||
@dataclass
|
||
class CompressionResult:
|
||
technique: Technique # full_low, preserve, crop, transcode
|
||
original_tokens: int # Estimated tokens before
|
||
compressed_tokens: int # Estimated tokens after
|
||
confidence: float # Router confidence (0-1)
|
||
|
||
@property
|
||
def savings_percent(self) -> float:
|
||
"""Percentage of tokens saved."""
|
||
```
|
||
|
||
### `Technique`
|
||
|
||
```python
|
||
class Technique(Enum):
|
||
FULL_LOW = "full_low" # 87% savings
|
||
PRESERVE = "preserve" # 0% savings
|
||
CROP = "crop" # 50-90% savings
|
||
TRANSCODE = "transcode" # 99% savings
|
||
```
|
||
|
||
## See Also
|
||
|
||
- [Compression Guide](compression.md) - Text compression techniques
|
||
- [CCR Guide](ccr.md) - Reversible compression with retrieval
|
||
- [Proxy Guide](proxy.md) - Zero-code deployment
|
||
- [Architecture](ARCHITECTURE.md) - System design
|