mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat/https-compose-caddy
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
9624d40d5e |
docs(claude): keep Remote Control behind Headroom via an HTTPS_PROXY route (#1779) (#4575)
## Description Claude Code v2.1.196+ disables Remote Control (`/remote-control`, `/rc`: a local session mirrored to claude.ai/code and the Claude mobile apps) whenever `ANTHROPIC_BASE_URL` names a host other than `api.anthropic.com`. The check is client-side and compares the host only, so every standard Headroom setup (`ANTHROPIC_BASE_URL=http://127.0.0.1:8787`) loses Remote Control, and the only documented fix so far is "run Claude without Headroom for those sessions". The troubleshooting page already named the route that remains: leave `ANTHROPIC_BASE_URL` unset and reach Headroom through an `HTTPS_PROXY` front end with a local CA trusted through `NODE_EXTRA_CA_CERTS` (the enterprise proxy path Claude Code supports). It was left as an open item on #1779. This PR ships that route, tested end to end with a real Claude Code session steered from the Claude iOS app while every request went through Headroom. How it works: ```text Claude Code --HTTPS_PROXY--> mitmdump 127.0.0.1:8790 api.anthropic.com /v1/messages* -> Headroom 127.0.0.1:8787 -> api.anthropic.com api.anthropic.com other paths -> api.anthropic.com (unchanged) any other host (claude.ai, OAuth) -> raw CONNECT tunnel, never decrypted ``` - `ANTHROPIC_BASE_URL` is unset, so the gate passes and subscription auth is untouched. - mitmproxy decrypts `api.anthropic.com` only (`--allow-hosts`). The Remote Control bridge (`claude.ai`) is never decrypted. - Only the Messages API goes to Headroom. Feature flags, telemetry and the rest of the API keep going straight to Anthropic. One trap found while setting it up, and handled here: the Headroom proxy is often started by the `headroom init hook ensure` hook, so it inherits Claude Code's environment (the running proxy had `CLAUDECODE=1`). Its httpx client honours `HTTPS_PROXY`, so its upstream calls would come back through mitmproxy and loop. The docs tell users to prefix the hook with `env -u HTTPS_PROXY -u HTTP_PROXY -u NODE_EXTRA_CA_CERTS`. As a second guard, the addon marks the requests it redirects (`x-rc-proxy-hop`) and sends a marked request straight to Anthropic. The marker deliberately avoids the `x-headroom-` prefix: Headroom strips those headers before forwarding upstream (`_strip_internal_headers`), which would erase the marker. My first local version used `x-headroom-rc` and the new test catches exactly that. Refs #1779 ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `examples/claude_code_remote_control/route_to_headroom.py`: the mitmproxy addon (redirect `/v1/messages*` to Headroom, loop guard, SSE streaming, `HEADROOM_RC_UPSTREAM` override). mitmproxy is imported only for type checking, so it is not a Headroom dependency. - `examples/claude_code_remote_control/README.md`: setup (macOS / Linux), Claude Code settings, starting Headroom without the proxy variables, verification, caveats, rollback. - `docs/content/docs/troubleshooting.mdx`: the Remote Control section offers this route next to "launch Claude without Headroom", and the "What remains" paragraph now points to it. - `examples/README.md`: index entry. - `tests/test_issue_1779_https_proxy_route.py`: routing rules pinned with stand-in flow objects. No Headroom runtime code changes, no new dependency. ## Testing - [x] Unit tests pass (`pytest`) — the new file plus the existing #1779 gate tests - [x] Linting passes (`ruff check .`) — on the changed Python files, ruff 0.17.0 (the pinned version) - [ ] Type checking passes (`mypy headroom`) — N/A, no change under `headroom/` - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest -q tests/test_issue_1779_https_proxy_route.py tests/test_issue_1779_remote_control_gate.py 64 passed in 0.77s # Loop-guard test with the marker renamed to x-headroom-rc (proves it catches the trap): FAILED tests/test_issue_1779_https_proxy_route.py::test_hop_marker_survives_headroom_header_stripping 1 failed, 9 passed $ ruff check <changed files> && ruff format --check <changed files> All checks passed! ``` Smoke test of the addon in a real `mitmdump` (12.3.1) with Headroom running on 8787: ```text POST https://api.anthropic.com/v1/messages -> "server connect 127.0.0.1:8787", "Streaming response from 127.0.0.1" GET https://api.anthropic.com/v1/models -> "server connect api.anthropic.com:443" (not redirected) GET https://claude.ai/ (no custom CA given) -> TLS handshake succeeds, i.e. tunnelled, not intercepted ``` ## Real Behavior Proof - Environment: macOS (Darwin 27), Claude Code 2.1.312 (native install, claude.ai Max subscription), Headroom 0.37.0 (`headroom proxy --mode token --backend anthropic`, started by the `headroom init hook ensure` hooks), mitmproxy 12.3.1 run as a LaunchAgent. - Exact command / steps: `~/.claude/settings.json` `env` = `HTTPS_PROXY=http://127.0.0.1:8790`, `NO_PROXY=localhost,127.0.0.1,::1`, `NODE_EXTRA_CA_CERTS=~/.mitmproxy/mitmproxy-ca-cert.pem`, `ANTHROPIC_BASE_URL` removed, `remoteControlAtStartup: true`, both headroom hook commands prefixed with `env -u HTTPS_PROXY -u HTTP_PROXY -u NODE_EXTRA_CA_CERTS`. Started a new `claude` session. - Observed result: `/status` shows the proxy; answers stream normally; the mitmdump log shows one `127.0.0.1:8787` request per turn and no repeats; the session shows up in the Claude iOS app (Code tab) and answers messages sent from the phone; Headroom's savings keep counting those requests. - Not tested: Linux and Windows (the README gives the Linux equivalent but it was not run), the VS Code extension and Claude Desktop sessions, `headroom wrap claude` (which sets `ANTHROPIC_BASE_URL` itself, so it is not compatible with this route). ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: N/A - Stable/default behavior changed: no, opt-in recipe; nothing changes for existing users - Kill switch / disable path: restore `ANTHROPIC_BASE_URL`, remove the three variables, stop mitmdump - Unsafe override required: no - Qualification impact: none - Rollback path: revert the commit (docs, example and test only) ## Review Readiness - [x] I performed a self-review - [ ] PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes - Possible follow-up, not in this PR: `remote_control_gate_message` / `remote_control_gate_short_message` still only suggest running Claude without Headroom. They could link to this section. I left them untouched to keep this PR docs-only and avoid changing strings the #1779 / #3426 tests pin. - Security trade-off, stated in the README: the user trusts a locally generated CA for `api.anthropic.com` only; the CA key stays on the machine (`~/.mitmproxy`). Everything else is tunnelled. ### Maintainer current-main requalification Exact head `b9d7a8a9e72a87ebdc69e1cbcb9e00df3265a22a`, upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. Mechanically updated from current upstream main. The addon, host/path selection, hop-loop guard, header handling and streaming behavior are unchanged. The maintainer documentation fix addresses the new Devin note: remove ANTHROPIC_BASE_URL from all active settings scopes, explicitly including project .claude/settings.json and .claude/settings.local.json, while retaining other entries/hooks. Source inspection confirms project-scoped headroom init claude writes the variable to settings.local.json, so removing only the user/shell setting is insufficient. The setup and caveat now cover that case. Both issue-1779 suites pass (64 tests), plus ten shared Gemini fixture cases after the test-only repair for independently verified main failures. Changed Python Ruff/format and whitespace checks pass. No new runtime dependency or contributor routing logic was introduced. The author's existing macOS/native Claude/Remote Control/mitmproxy end-to-end evidence and exclusions were read; these independent Windows checks cover routing contracts and documentation accuracy, not a live macOS/Linux/Windows Remote Control session or a fresh native build. Author readiness attestations remain unchanged; nothing merged. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. ### Maintainer current-main requalification Exact head `c5e1ea2afcbd7e4c91a75869706c6f68f08f7e3a`, upstream main `c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`. Final main synchronization with upstream |
||
|
|
e67b3c8a29 |
feat(vertex): add Gemini 3.8 Flash agent benchmark suite (#3371)
## Summary Adds a reproducible, end-to-end multi-turn agent benchmark suite for **Gemini 3.8 Flash** on Google Cloud Vertex AI (Gemini Enterprise Agent Platform) under `examples/vertex_gemini_benchmark/`. Compares **Direct Vertex AI** vs **Headroom-proxied Vertex AI** across 4 realistic production agent scenarios: 1. **SRE Incident Root Cause Analysis**: 350+ Kubernetes & microservice logs + Prometheus metrics (identifying connection pool exhaustion, cluster endpoint, port, and cascading circuit-breaker trips). 2. **Codebase Security Audit & PR Review**: Multi-file code search results + git diff (identifying insecure `alg: none` and `bypass_auth` JWT backdoors). 3. **Enterprise BigQuery Analytics**: 500 tabular transaction rows (computing summaries and isolating fraudulent accounts and high-spend outliers). 4. **Multi-Turn RAG & System Architecture Synthesis**: 25 dense technical chunks regarding agent context management contracts. ### Live Benchmark Results (Gemini 3.8 Flash on Vertex AI `global`) ```text ================================================================================== 📊 FINAL BENCHMARK SUMMARY: GEMINI-3.8-FLASH ON VERTEX AI ================================================================================== Scenario | Baseline Prompt | Headroom Prompt | Reduction ---------------------------------------------------------------------------------- SRE Incident Root Cause Analysis | 51,775 | 13,842 | 73.3% Codebase Security Audit & PR Review | 6,821 | 4,632 | 32.1% Enterprise BigQuery Analytics & Outlier Detection | 49,020 | 6,120 | 87.5% Multi-Turn RAG & Architecture Synthesis | 4,071 | 1,437 | 64.7% ---------------------------------------------------------------------------------- TOTAL / AGGREGATE | 111,687 | 26,031 | 76.7% Metric | Baseline | Headroom | Delta / Impact ---------------------------------------------------------------------------------- Total Prompt Tokens | 111,687 | 26,031 | - 76.7% Total All Tokens | 116,639 | 29,168 | - 75.0% Total Cost ($ USD) | $ 0.09410 | $ 0.02849 | - 69.7% Avg Latency (ms) | 8921ms | 5748ms | +3173ms Ground Truth Accuracy (Absolute) | 100.0% | 100.0% | 100.0% Retained ================================================================================== ``` ## Real Behavior Proof • **Environment**: Linux, Python 3.11, live Google Cloud project (`alanblount-demo`) with Vertex AI enabled, Application Default Credentials (`gcloud auth application-default login`). • **Exact command / steps**: `GCP_PROJECT_ID=alanblount-demo python examples/vertex_gemini_benchmark/benchmark.py --model gemini-3.8-flash` • **Observed result**: Headroom proxy started cleanly, executed all 4 multi-turn scenarios on `gemini-3.8-flash`, verified ground truth error causes and anomalies, recorded a **76.7% prompt token reduction** and **69.7% cost savings**, exported `results.json`, and shut down cleanly. • **Not tested**: Windows. ## Runtime Rollout Safety • **Rollout-managed feature(s)**: None; self-contained benchmark suite under `examples/`. • **Minimum rollout channel**: standard release. • **Stable/default behavior changed**: None; purely adds benchmark tooling and documentation under `examples/`. • **Kill switch / disable path**: N/A. • **Unsafe override required**: None. • **Qualification impact**: None. • **Rollback path**: Revert PR. ## Review Readiness [✓] I have performed a self-review [✓] This PR is ready for human review ## Checklist [✓] My code follows the project's style guidelines [✓] I have performed a self-review of my code [✓] I have commented my code, particularly in hard-to-understand areas [✓] I have made corresponding changes to the documentation [✓] My changes generate no new warnings [✓] I did not edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes • Companion to PR #2640 (which adds in-band Vertex diagnostic hints and fixes). |
||
|
|
d789a7c528 |
feat(transforms): tabular + spreadsheet (.xlsx/.xls) compression (#1128)
## Description Adds a content-type-aware path for **tabular data** — CSV/TSV, markdown tables, fixed-width text, and binary `.xlsx`/`.xls` spreadsheets — by routing them through the existing, battle-tested `SmartCrusher` instead of letting them fall through to `PLAIN_TEXT → Kompress`. The pipeline already compressed tables losslessly when handed a JSON array of records. This wires up the missing front door: detect tabular text (and ingest binary spreadsheets), convert to JSON records, and reuse `SmartCrusher.crush()`. No new compression algorithm. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Detection** (`content_detector.py`): new `ContentType.TABULAR` + `_try_detect_tabular()` for CSV/TSV, markdown tables, and fixed-width columns. Ordered after search/log (which also look "delimited") and before code, with a prose-rejection guard so it never steals `file:line:content` search output, `key: value` logs, or sentences with incidental commas. Rust backend returns `plain_text` for unknown types and the router already falls back to the Python detector, so **no Rust change**. - **Bridge** (`tabular_ingest.py`): stdlib parsers + `to_records()` + a `TabularCompressor` that parses → JSON records → `SmartCrusher` (lossless `csv-schema` first; lossy row-drop with reversible `<<ccr:HASH>>` markers stays SmartCrusher's built-in fallback). Only adopts a result when it actually saves bytes. - **Spreadsheets** (`spreadsheet_ingest.py`): `.xlsx`/`.xls` → per-sheet CSV text at the SDK boundary. Optional deps (`pip install headroom-ai[spreadsheet]`) fail loudly with an install hint, never silently degrade. - **Routing** (`content_router.py`): `CompressionStrategy.TABULAR`, `enable_tabular_compressor` flag, lazy getter, apply branch, strategy maps, Kompress fallback eligibility. - **SDK** (`compress.py`): `compress_spreadsheet(path, ...)` helper (one message per sheet). - **Packaging** (`pyproject.toml`): new `[spreadsheet]` extra; `openpyxl` added to `[dev]` so the xlsx path is exercised in CI. - **Docs/demo**: `examples/tabular_compression_demo.py` + README entry. ### Design note: lossless-only Compact, all-unique tables with no query yield ~0 savings — this is correct, not a bug. SmartCrusher returns `skip:unique_entities_no_signal` and won't drop unique rows without a duplicate/relevance signal. Real wins come from verbose/redundant tables and query-driven selection. A pressure-driven lossy row sampler was considered and intentionally not added. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_transforms_tabular.py -q collected 20 items tests/test_transforms_tabular.py .................... [100%] ============================== 20 passed in 7.15s ============================== $ ruff check headroom/transforms/tabular_ingest.py headroom/transforms/spreadsheet_ingest.py All checks passed! $ mypy headroom/transforms/tabular_ingest.py headroom/transforms/spreadsheet_ingest.py Success: no issues found in 2 source files ``` `tests/test_transforms_tabular.py` (20 tests): detection true positives + no-misroute negatives (search/log/JSON/prose), parser units (incl. fixed-width), the CSV→SmartCrusher bridge, router routing + disable flag, and `.xlsx` ingestion (skipif openpyxl missing) + error paths. `spreadsheet_ingest` 100% / `tabular_ingest` 90% line coverage. ## Real Behavior Proof - **Environment:** local checkout of `feat/tabular-compression`, Python 3.x, `pip install -e ".[dev]"`. - **Exact command / steps:** `python examples/tabular_compression_demo.py` (no API key required). - **Observed result:** ```text === Raw tabular text (ContentRouter, char-level) === compact unique CSV strat=tabular chars 1306 -> 1072 ( 17.9% saved) redundant CSV strat=tabular chars 2661 -> 1350 ( 49.3% saved) verbose markdown strat=tabular chars 2019 -> 1580 ( 21.7% saved) === Full pipeline (real tokenizer) === redundant CSV tokens 768 -> 394 ( 48.7% saved) === Binary spreadsheet (.xlsx) === 2-sheet workbook tokens 1092 -> 683 ( 37.5% saved) ``` - **Not tested:** legacy `.xls` binary path (needs optional `xlrd` + binary fixture; `# pragma: no cover`); base64-embedded `.xlsx` inside multimodal blocks (out of scope, noted as a follow-up). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - CHANGELOG/version are intentionally untouched: this repo uses **release-please**, which bumps the version and CHANGELOG via automated `chore: release main` PRs, not per-feature PRs. - The `.xls` path is `# pragma: no cover` (legacy, needs optional `xlrd` + a binary fixture). - Follow-up (out of scope): base64-embedded `.xlsx` inside tool-result/multimodal blocks; porting tabular parsers into the Rust core for parity. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
95e9b39b6d |
feat: Add AWS Strands Agents SDK integration
## Description Add Headroom integration with AWS Strands Agents SDK, enabling automatic context optimization and tool output compression for Strands-based agents. Fixes #14 ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Documentation update ## Changes Made ### Core Integration (`headroom/integrations/strands/`) - **HeadroomHookProvider** - Implements Strands `HookProvider` interface for automatic tool output compression via `AfterToolCallEvent`. Compresses verbose tool outputs before they enter conversation context. - **HeadroomStrandsModel** - Model wrapper that extends Strands `Model` base class for message-level optimization. Implements all required abstract methods: `stream()`, `get_config()`, `update_config()`, `structured_output()`. - **Provider auto-detection** - Automatically detects appropriate Headroom provider (Anthropic, OpenAI, Google) based on wrapped Strands model type. - **`strands-agents` as optional dependency** - Install with `pip install headroom-ai[strands]` ### Testing (`tests/integrations/test_strands/`) - **Real integration tests (25 tests)** - Use actual AWS Bedrock API calls with Claude 3 Haiku. Skip automatically when credentials unavailable. - **Unit tests (57 tests)** - Mock-based tests for internal logic, edge cases, and error handling. No credentials required. ### Demo (`examples/strands_bedrock_demo.py`) - Interactive demo showcasing both integration patterns - Visual before/after compression comparison with token savings - 4 verbose tools (search, logs, database, metrics) demonstrating real savings - Supports `--hook` and `--model` flags for individual demos ## Testing All tests verified: - [x] Unit tests pass (57 tests) - [x] Integration tests pass (25 tests with real Bedrock API) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom/integrations/strands/`) - [x] Formatting passes (`ruff format --check`) - [x] Demo runs successfully with ~50% token savings ## Test Output ``` $ pytest tests/integrations/test_strands/ -v =================== 82 passed in 90.09s =================== $ ruff check headroom/integrations/strands/ --ignore E402 All checks passed! $ mypy headroom/integrations/strands/ --ignore-missing-imports Success: no issues found ``` ## Demo Results ``` ╭────────────────────────────────────────────────────────────╮ │ HeadroomHookProvider Results │ │────────────────────────────────────────────────────────────│ │ Tokens BEFORE compression: 51,961 │ │ Tokens AFTER compression: 25,658 │ │ Tokens SAVED: 26,303 (50.6%) │ ╰────────────────────────────────────────────────────────────╯ ``` |
||
|
|
175746cc26 |
Prepare for OSS release v0.2.0
This commit prepares Headroom for public open source release with comprehensive documentation, licensing, and community infrastructure. License & Legal: - Add Apache 2.0 LICENSE file - Add NOTICE file with third-party attributions - Add SECURITY.md for vulnerability reporting Community: - Add CONTRIBUTING.md with contribution guidelines - Add CODE_OF_CONDUCT.md (Contributor Covenant) - Add GitHub issue templates (bug report, feature request) - Add pull request template Documentation: - Update README.md with compelling value proposition - Add docs/getting-started.md - Add docs/proxy.md for proxy server documentation - Add docs/transforms.md for transform reference - Add docs/api.md for API reference - Add examples/README.md Package Infrastructure: - Add headroom/py.typed for PEP 561 compliance - Add headroom/cli.py for CLI entry point - Add .github/workflows/ci.yml for CI pipeline - Add .github/workflows/publish.yml for PyPI publishing - Update pyproject.toml with proper metadata New Features: - Add multi-provider support (Google, Cohere, LiteLLM, OpenAI-compatible) - Add universal tokenizer registry with multiple backends - Add model registry with pricing and context limits - Add production proxy server with caching and rate limiting Code Quality: - Fix 83 lint issues via ruff auto-fix - Fix version consistency (benchmarks 0.1.0 → 0.2.0) - Add skip decorators for optional dependency tests |