Files
headroom/examples/07-context-compression.ipynb
Tejas ChopraandClaude Opus 5 96dfc7005f chore: finish the org and docs-domain migration (#3389)
Two stale identities left over from moves that were only half applied.

## GitHub org

`install/state.py` already migrated Docker images to
`ghcr.io/headroomlabs-ai/headroom` and carries a
`_DEPRECATED_IMAGE_REPO` constant that rewrites the old path on load —
so this move was decided long ago. Everything else still said
`chopratejas/headroom`.

Rewritten across packaging (`pyproject.toml`, `Cargo.toml`), both plugin
marketplaces, the agent-hooks manifests, install scripts, CI templates,
crate docs and tests.

**Two deliberate exclusions:**
- **`ghcr.io/chopratejas/headroom` is untouched everywhere.** It is the
*input* to `_migrate_deprecated_image()` and to the tests covering it.
Rewriting it would silently disable the migration.
- **`CHANGELOG.md` keeps its 238 historical references.** GitHub's
rename redirect resolves them, and rewriting would bury this diff in
noise.

**HuggingFace model ids are NOT affected and were never in scope.**
`chopratejas/kompress-base`, `kompress-v2-base`, `kompress-finance`,
`technique-router`, `technique-router-onnx` and
`siglip-image-encoder-onnx` are account-scoped identifiers on a
different service. All six were validated against the HF API before this
change:

| Model | HTTP |
|---|---|
| `kompress-base` | 200 |
| `kompress-v2-base` | 200 |
| `technique-router` | 200 |
| `technique-router-onnx` | 200 |
| `siglip-image-encoder-onnx` | 200 |
| `kompress-finance` | **401 (private)** |

The 401 is unrelated to this migration, but worth knowing:
`kompress_compressor.py:1255` references it, so that path is not
reachable by an OSS user without access.

**One functional fix, not cosmetic:** `_marketplace_source()`
(`headroom/cli/init.py`) returned the old org as its fallback, so plugin
installs were pointed at the retired repo.

## Docs domain

The site moved to `docs.headroomlabs.ai`, but eleven references still
named the vercel.app host — including **`llms.txt` (27 links)**, which
is served at `docs.headroomlabs.ai/llms.txt` and read by LLM crawlers,
the **dashboard's own Documentation link**, and a `headroom wrap` help
message. The docs app's canonical-URL fallbacks were fixed separately in
#3388.

## Also folded in

`RUST_DEV.md` corrections found while verifying the above:
- `make build-wheel` points at `Cargo.toml`, not `pyproject.toml`
- the Rust toolchain is pinned, not tracking `stable`
- `headroom-simulators` was missing from the workspace listing
- the Python `CompressionStore` defaults to **SQLite**, not in-memory,
when `HEADROOM_CCR_BACKEND` is unset (verified by execution) — this one
matters for multi-worker reasoning

And the install scripts pointed at `docs/docker-install.md`, a path that
no longer exists; they now point at the live docs page.

## Verification

297 tests pass, including the plugin-manifest, install-state and
release-workflow suites that assert on these exact strings. `ruff check`
clean. Unrelated lockfile churn (`uv.lock`, both `package-lock.json`) is
deliberately excluded.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01HRaEPjQcCSAwD2xbQMNENu

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-02 15:53:35 +05:30

11 KiB

Context Compression

What is it

Context Compression is the act of statistically reducing tool output size while preserving the information the LLM needs to answer the user's question.

Why it helps

  • Avoids Context Distraction: Verbose tool outputs dilute the signal. Compression removes filler words and redundant phrasing while keeping key facts, errors, and anomalies.
  • No extra LLM call required: Unlike pruning (notebook 04) and summarization (notebook 05) which call GPT-4o-mini per tool result, compression runs locally using statistical and ML-based token analysis. Zero additional cost, lower latency.

Context Compression in Practice

Headroom is an open-source context optimization library that provides multi-algorithm compression. It auto-detects content type (JSON, code, logs, text) and routes to the optimal compressor:

  • SmartCrusher: Statistically analyzes JSON arrays — keeps errors, anomalies, and query-relevant items
  • Kompress: ModernBERT token classifier — removes redundant tokens from text while preserving meaning
  • CodeCompressor: AST-aware compression for source code

When items are highly diverse (like RAG retriever chunks), Headroom keeps all items and compresses the text within each one — no information is dropped.

Context Compression in LangGraph

We'll replace the LLM-based pruning/summarization step with a local compression call. The agent structure is identical to notebooks 04 and 05 — only the tool processing node changes.

In [ ]:
# Install headroom (one-time)
# !pip install "headroom-ai[all]"
In [ ]:
from langchain_community.document_loaders import WebBaseLoader

urls = [
    "https://lilianweng.github.io/posts/2025-05-01-thinking/",
    "https://lilianweng.github.io/posts/2024-11-28-reward-hacking/",
    "https://lilianweng.github.io/posts/2024-07-07-hallucination/",
    "https://lilianweng.github.io/posts/2024-04-12-diffusion-video/",
]

docs = [WebBaseLoader(url).load() for url in urls]
In [ ]:
from langchain_text_splitters import RecursiveCharacterTextSplitter

docs_list = [item for sublist in docs for item in sublist]

text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
    chunk_size=3000, chunk_overlap=50
)
doc_splits = text_splitter.split_documents(docs_list)
In [ ]:
from langchain.embeddings import init_embeddings
from langchain_core.vectorstores import InMemoryVectorStore

embeddings = init_embeddings("openai:text-embedding-3-small")
vectorstore = InMemoryVectorStore.from_documents(documents=doc_splits, embedding=embeddings)
retriever = vectorstore.as_retriever()
In [ ]:
from langchain.tools.retriever import create_retriever_tool
from rich.console import Console
from rich.pretty import pprint

console = Console()

retriever_tool = create_retriever_tool(
    retriever,
    "retrieve_blog_posts",
    "Search and return information about Lilian Weng blog posts.",
)

result = retriever_tool.invoke({"query": "types of reward hacking"})
console.print("[bold green]Retriever Tool Results:[/bold green]")
pprint(result)
In [ ]:
from langchain.chat_models import init_chat_model

llm = init_chat_model("anthropic:claude-sonnet-4-20250514", temperature=0)

tools = [retriever_tool]
tools_by_name = {tool.name: tool for tool in tools}

llm_with_tools = llm.bind_tools(tools)
In [ ]:
from typing import Literal

from IPython.display import Image, display
from langchain_core.messages import SystemMessage, ToolMessage
from langgraph.graph import END, START, MessagesState, StateGraph

from headroom import compress


class State(MessagesState):
    """Extended state that includes a summary field for context compression."""

    summary: str


rag_prompt = """You are a helpful assistant tasked with retrieving information from a series of technical blog posts by Lilian Weng.
Clarify the scope of research with the user before using your retrieval tool to gather context. Reflect on any context you fetch, and
proceed until you have sufficient context to answer the user's research request."""


def llm_call(state: State) -> dict:
    """Execute LLM call with system prompt and message history."""
    messages = [SystemMessage(content=rag_prompt)] + state["messages"]
    response = llm_with_tools.invoke(messages)
    return {"messages": [response]}


def should_continue(state: State) -> Literal["tool_node_with_compression", "__end__"]:
    """Decide if we should continue the loop or stop."""
    messages = state["messages"]
    last_message = messages[-1]
    if last_message.tool_calls:
        return "tool_node_with_compression"
    return END


def tool_node_with_compression(state: State):
    """Execute tool calls and compress results with Headroom.

    Instead of calling GPT-4o-mini to prune or summarize (notebooks 04, 05),
    we use Headroom's compress() — no LLM call, no extra cost.

    Headroom auto-detects content type and applies the right compressor:
    - JSON arrays → SmartCrusher (statistical, keeps anomalies + query-relevant items)
    - Plain text → Kompress (ModernBERT token compression)
    - Code → CodeCompressor (AST-aware)

    For diverse retriever results (each chunk is unique), Headroom keeps ALL
    items and compresses the text within each one.
    """
    result = []
    for tool_call in state["messages"][-1].tool_calls:
        tool = tools_by_name[tool_call["name"]]
        observation = tool.invoke(tool_call["args"])

        # Build a minimal message list so Headroom can extract the user query
        # for relevance-aware compression (keeps chunks matching the question).
        user_query = state["messages"][0].content if state["messages"] else ""
        temp_messages = [
            {"role": "user", "content": user_query},
            {"role": "tool", "content": observation, "tool_call_id": tool_call["id"]},
        ]

        compressed = compress(temp_messages, model="claude-sonnet-4-20250514")
        compressed_content = compressed.messages[-1]["content"]

        result.append(ToolMessage(content=compressed_content, tool_call_id=tool_call["id"]))

    return {"messages": result}


# Build workflow
agent_builder = StateGraph(State)

agent_builder.add_node("llm_call", llm_call)
agent_builder.add_node("tool_node_with_compression", tool_node_with_compression)

agent_builder.add_edge(START, "llm_call")
agent_builder.add_conditional_edges(
    "llm_call",
    should_continue,
    {
        "tool_node_with_compression": "tool_node_with_compression",
        END: END,
    },
)
agent_builder.add_edge("tool_node_with_compression", "llm_call")

agent = agent_builder.compile()

display(Image(agent.get_graph(xray=True).draw_mermaid_png()))
In [ ]:
from utils import format_messages

query = "What are the types of reward hacking discussed in the blogs?"
result = agent.invoke({"messages": [{"role": "user", "content": query}]})
format_messages(result["messages"])

How it compares

Technique Notebook Token Reduction Extra LLM Call Extra Cost
RAG Baseline 01 — No $0
Context Pruning 04 ~56% Yes (GPT-4o-mini) ~$0.003/call
Context Summarization 05 ~68% Yes (GPT-4o-mini) ~$0.003/call
Context Compression 07 ~30-40% No $0

Key differences:

  • No LLM call: Pruning and summarization call GPT-4o-mini per tool result. Compression runs locally.
  • No information loss: For diverse retriever results (each chunk is unique), Headroom keeps ALL items and compresses text within each one. Pruning removes entire chunks; summarization rewrites them.
  • Reversible: Headroom's CCR (Compress-Cache-Retrieve) stores originals. The LLM can call headroom_retrieve to get full uncompressed content if it needs more detail.
  • Content-aware: Different content types get different treatment. JSON arrays → statistical analysis. Plain text → ML token compression. Code → AST-aware compression.

The trade-off: pruning and summarization can achieve higher compression (56-68%) because they use an LLM to judge relevance. Compression achieves 30-40% without any LLM call — making it faster and free.