mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
Audited all 52 pages of docs.headroomlabs.ai/docs against source, executing code rather than reading it. **36 pages changed.** Every figure that could not be traced to a benchmark, a test, or a committed artifact was measured or removed — none was replaced with a different guess. ## The site was telling users things that were not true **Telemetry.** `proxy.mdx` stated *"nothing is sent to Headroom Labs"* and that the anonymous beacon *"has been removed from the code."* Neither is true — `BEACON_DEFAULT_ON = True` (`headroom/telemetry/beacon.py:56`), `is_beacon_enabled()` is fail-open, and there is a live POST at `headroom/telemetry/session.py:75`. `proxy.mdx` and `limitations.mdx` now describe both switches accurately, say why the beacon exists (compression quality is the product; it shows when a release regresses a ratio across real workloads), and list the 12 fields the collector allowlists instead of asserting "content-free". The install id is disclosed for what it is: a random UUID, not a machine fingerprint, resettable by deleting the file. **Numbers.** The landing page's "Proof" table claimed 92/92/47/73% savings from a harness that seeded nothing, so it could not reproduce a figure even in principle. | Scenario | Claimed | Measured | |---|---|---| | Code search (100 results) | 92% | **21%** | | SRE incident debugging | 92% | **57%** | | Codebase exploration | 47% | **42%** | | GitHub issue triage | 73% | **30%** | `real_world_agent_benchmark.py` now seeds, and `benchmarks/index_proof_table.py` measures the same four scenarios through the real tokenizer and the real `compress()` path with no network call. Stable across five seeds; the page cites the command. Latency was wrong in the *other* direction — 189 ms and 943 ms published against **0.21 ms and 0.80 ms** measured. The docs made the product look ~1000× slower than it is. **Examples.** Every `agno` example was broken at line one, including Quick Start: `HeadroomAgnoModel`'s first dataclass field is `id`, so a positional wrapped model bound to the wrong field and raised `ValueError`. `api-reference.mdx` — the canonical signature reference — had 13 broken examples, including three `AttributeError`s in the first three lines of its `simulate()` block. Four integration pages omitted the `[proxy]` extra and then told the reader to run `headroom proxy`, which fails immediately. `ccr.mdx` showed SDK-level retrieval that only exists in the proxy handlers. Also: six Prometheus metric names that are never emitted, a Grafana dashboard filtering on a label no metric carries, a documented deploy workflow that does not exist, a 365-day retention default the code caps at 30, and CCR TTL stated as 5 minutes against an actual 1800s. ## Canonical URL bug `robots.ts`, `sitemap.ts` and `layout.tsx` fell back to a hardcoded `headroom-docs.vercel.app` when `NEXT_PUBLIC_SITE_URL` was unset — so the live site served a sitemap and `Host:` on the wrong domain and pointed crawlers at the wrong canonical. Verified in the build output: robots and all 53 sitemap URLs now carry `docs.headroomlabs.ai`. Setting `NEXT_PUBLIC_SITE_URL=https://docs.headroomlabs.ai` in Vercel is still worth doing so previews match, but the code no longer depends on it. ## Verification Four adversarial judges re-checked this work. They confirmed the landing-page table reproduces digit-for-digit across five seeds, found **zero cross-page contradictions**, independently regenerated all nine surviving compression percentages, and proved `headroom_keep_turns` dead at runtime with a mocked-transport test. They also caught two links this audit had reversed to the deprecated org, and 18 examples it had missed — both fixed here. `next build` passes with all 52 pages prerendered, matching the pre-change baseline. ## Follow-up, not in this PR - GitHub org migration (`chopratejas` → `headroomlabs-ai`) across source, manifests and packaging, plus stale-domain fixes in `llms.txt`, the dashboard UI and `pyproject.toml`. - `wiki/` disposition: it is unpublished and unbuilt, and `wiki/benchmarks.md` carried an invented production-telemetry section (250+ instances, 1.4 billion tokens saved) that exists nowhere in the repo. - `langchain.mdx` imports are dead against LangChain 1.x (`langchain.memory`, `langchain.retrievers`, `create_openai_tools_agent`). Needs a pin-vs-rewrite decision. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01HRaEPjQcCSAwD2xbQMNENu --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
141 lines
3.7 KiB
Plaintext
141 lines
3.7 KiB
Plaintext
---
|
|
title: Strands
|
|
description: Context compression for Strands Agents via model wrapping and hook-based tool output compression.
|
|
---
|
|
|
|
Headroom integrates with [Strands Agents](https://github.com/strands-agents/sdk-python) through two patterns: wrap the model for full conversation compression, or hook into tool calls for targeted tool output compression.
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
pip install headroom-ai strands-agents
|
|
```
|
|
|
|
## Quick start
|
|
|
|
```python
|
|
from strands import Agent
|
|
from strands.models.bedrock import BedrockModel
|
|
from headroom.integrations.strands import HeadroomStrandsModel
|
|
|
|
model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
|
|
optimized = HeadroomStrandsModel(wrapped_model=model)
|
|
|
|
agent = Agent(model=optimized)
|
|
response = agent("Investigate the production incident")
|
|
|
|
print(f"Tokens saved: {optimized.total_tokens_saved}")
|
|
```
|
|
|
|
## Model wrapping
|
|
|
|
Wraps the Strands `Model` interface. Every call to `stream()` compresses messages before they reach the provider:
|
|
|
|
```python
|
|
from headroom import HeadroomConfig
|
|
from headroom.integrations.strands import HeadroomStrandsModel
|
|
|
|
optimized = HeadroomStrandsModel(
|
|
wrapped_model=model,
|
|
config=HeadroomConfig(),
|
|
)
|
|
|
|
agent = Agent(model=optimized)
|
|
response = agent("Analyze these logs")
|
|
```
|
|
|
|
## Hook provider (tool output compression)
|
|
|
|
Compresses tool call results via Strands' hook system. Uses SmartCrusher on JSON arrays returned by tools:
|
|
|
|
```python
|
|
from strands import Agent
|
|
from strands.models.bedrock import BedrockModel
|
|
from headroom.integrations.strands import HeadroomHookProvider
|
|
|
|
model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-20250514-v1:0")
|
|
hooks = HeadroomHookProvider(
|
|
compress_tool_outputs=True,
|
|
min_tokens_to_compress=200,
|
|
preserve_errors=True,
|
|
)
|
|
|
|
agent = Agent(model=model, hooks=[hooks])
|
|
response = agent("Search the database for recent failures")
|
|
|
|
print(f"Tokens saved by hooks: {hooks.total_tokens_saved}")
|
|
```
|
|
|
|
The hook preserves error items, anomalous values (statistical outliers), items matching the query context, and boundary items (first/last).
|
|
|
|
## Both together
|
|
|
|
Model wrapping compresses conversation history. Hooks compress individual tool results. Use both for maximum savings:
|
|
|
|
```python
|
|
from headroom.integrations.strands import HeadroomStrandsModel, HeadroomHookProvider
|
|
|
|
optimized = HeadroomStrandsModel(wrapped_model=model)
|
|
hooks = HeadroomHookProvider(compress_tool_outputs=True)
|
|
|
|
agent = Agent(model=optimized, hooks=[hooks])
|
|
```
|
|
|
|
## How it works
|
|
|
|
```
|
|
Agent decides to call tool
|
|
|
|
|
v
|
|
Tool executes, returns result
|
|
|
|
|
v
|
|
HeadroomHookProvider (optional)
|
|
compresses tool result JSON
|
|
|
|
|
v
|
|
Agent builds next API request
|
|
|
|
|
v
|
|
HeadroomStrandsModel.stream()
|
|
compresses full message list
|
|
|
|
|
v
|
|
Provider API (Bedrock, etc.)
|
|
```
|
|
|
|
The model wrapper uses the full Headroom pipeline (CacheAligner, ContentRouter). The hook provider uses SmartCrusher directly for fast JSON compression.
|
|
|
|
## Structured output
|
|
|
|
`structured_output()` is an async generator (it delegates to the wrapped model's own `structured_output()`), so it must be consumed with `async for`:
|
|
|
|
```python
|
|
from pydantic import BaseModel
|
|
|
|
class Analysis(BaseModel):
|
|
severity: str
|
|
root_cause: str
|
|
recommendation: str
|
|
|
|
async for event in optimized.structured_output(Analysis, messages):
|
|
result = event
|
|
```
|
|
|
|
## Metrics
|
|
|
|
```python
|
|
for m in optimized.metrics_history:
|
|
print(f" {m.tokens_before} -> {m.tokens_after} ({m.tokens_saved} saved)")
|
|
|
|
print(f"Total saved: {optimized.total_tokens_saved}")
|
|
```
|
|
|
|
## Supported providers
|
|
|
|
| Strands Model | Provider Detected |
|
|
|--------------|-------------------|
|
|
| `BedrockModel` | Anthropic (via Bedrock) |
|
|
| `OllamaModel` | OpenAI-compatible |
|
|
| Custom `Model` | Falls back to estimation |
|