mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix/plugin-launcher-isolated-python
37
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
02a2b568d4 |
fix(examples): keep the macos launchagent running when reinstall is declined (#4734)
## Description In `examples/deployment/macos-launchagent/install.sh` (lines 96-113 on `1da8930a`), when the plist already exists the script runs `launchctl bootout` on the running service first and only then asks `Reinstall? [y/N]`. Answering `N` prints `Installation cancelled.` and exits 0, but the service is already stopped, so a cancelled install leaves the proxy down until the next login or a manual `launchctl bootstrap`. This PR moves the stop block below the prompt, unchanged, so `bootout` runs only after the user confirmed, or in `--unattended` mode (which never prompts). Nothing else changes. Part of #4730 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `examples/deployment/macos-launchagent/install.sh`: the `launchctl print` / `launchctl bootout` block moves from before the `Reinstall?` prompt to after it, with a one-line comment. 6 lines moved, 1 comment added. ## Testing - [x] Unit tests pass (`pytest`) (the new test file only; I did not run the full suite) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed This is a shell example script, so the Python lint and type checks do not apply to it. The new test file is checked with ruff (below). The project's bug-fix policy asks for a test that fails before and passes after, so a committed test runs the real script against stubs (below), and the earlier manual driver is kept under it. ### Test Output #### Committed regression test `tests/test_macos_launchagent_install_cancel.py` runs the real `install.sh` under `bash` with stub `uname`, `launchctl`, `headroom` and `lsof` on `PATH` and a fake `HOME`. It is skipped when `bash` is not on `PATH`. Environment: Ubuntu on WSL2, bash, Python venv `/var/tmp/cb-fix-8mh5/repo/.venv`, pytest `-p no:cacheprovider`, test file run from a scratch copy (no `headroom` import, no repo conftest). Same test file, `install.sh` taken from `origin/main` (`1da8930a`): ```text FAILED test_declining_reinstall_keeps_the_service_running 1 failed, 1 passed in 0.14s (test_confirming_reinstall_stops_the_service passes: bootout is still called after y) ``` Same test file, `install.sh` from this branch: ```text 2 passed in 0.06s ``` `ruff format --check` and `ruff check` on the new file: clean. #### Earlier manual driver Driver: copies `install.sh` and the plist template into a scratch directory, puts stubs first on `PATH` (`uname` prints `Darwin`; `headroom` exits 0; `lsof` exits 1; `launchctl` appends each call to a log and exits 0, so `launchctl print` reports a running service), points `HOME` at a fake home that already contains `Library/LaunchAgents/com.headroom.proxy.plist`, and feeds stdin. Git Bash on Windows 11. Paths shortened to `<run>`; output filtered to the lines that matter. `origin/main` (`1da8930a`), `echo N | bash install.sh`: ```text ⚠ LaunchAgent already installed at: <run>/fakehome/Library/LaunchAgents/com.headroom.proxy.plist ==> Stopping existing service... Installation cancelled. rc=0 --- launchctl call log: launchctl print gui/197609/com.headroom.proxy launchctl bootout gui/197609/com.headroom.proxy ``` This branch, `echo N | bash install.sh`: ```text ⚠ LaunchAgent already installed at: <run>/fakehome/Library/LaunchAgents/com.headroom.proxy.plist Installation cancelled. rc=0 --- launchctl call log: (no calls) ``` This branch, `printf 'y\n\n' | bash install.sh` (answer `y`, then the default port). Reinstall still boots out and bootstraps; `origin/main` produces the same four calls: ```text ⚠ LaunchAgent already installed at: <run>/fakehome/Library/LaunchAgents/com.headroom.proxy.plist ==> Stopping existing service... ✓ Created: <run>/fakehome/Library/LaunchAgents/com.headroom.proxy.plist ✓ LaunchAgent loaded successfully ✓ Service is running ⚠ Service is running but port 8787 is not listening yet ✓ Headroom proxy installed successfully! rc=0 --- launchctl call log: launchctl print gui/197609/com.headroom.proxy launchctl bootout gui/197609/com.headroom.proxy launchctl bootstrap gui/197609 <run>/fakehome/Library/LaunchAgents/com.headroom.proxy.plist launchctl print gui/197609/com.headroom.proxy ``` (The "port 8787 is not listening" warning comes from the `lsof` stub exiting 1; it is not a finding.) ## Real Behavior Proof - Environment: Git Bash on Windows 11 with stubbed `uname`, `launchctl`, `headroom`, `lsof`; no macOS, no real LaunchAgent. - Exact command / steps: the driver above, `echo N | bash install.sh` on `origin/main`'s script and on this branch's, then `printf 'y\n\n' | bash install.sh` on both. - Observed result: on `origin/main`, answering `N` made the stub log `launchctl bootout` before `Installation cancelled.`; on this branch the log is empty. Answering `y` makes the same four `launchctl` calls on both. - Not tested: real macOS launchd (the proof uses stubs on Windows and WSL, not a live LaunchAgent); real `launchctl` on macOS; the `--unattended` path (the block there runs unchanged, only its place in the file differs, but I did not run it). ## Runtime Rollout Safety - Rollout-managed feature(s): none; example script only. - Minimum rollout channel: n/a - Stable/default behavior changed: only the cancel path: the running service is no longer stopped when the user declines. - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none - Rollback path: revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes (new test file only; the stubbed runs above are the check) - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - Scope is an example script; the effect of the bug is a stopped proxy on the user's own machine. - Other defects in the same directory are in #4730 and separate PRs. Produced with AI assistance (Claude Code) operated by me (HetCreep); the driver was run on my machine against a clone of the repository. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
9624d40d5e |
docs(claude): keep Remote Control behind Headroom via an HTTPS_PROXY route (#1779) (#4575)
## Description Claude Code v2.1.196+ disables Remote Control (`/remote-control`, `/rc`: a local session mirrored to claude.ai/code and the Claude mobile apps) whenever `ANTHROPIC_BASE_URL` names a host other than `api.anthropic.com`. The check is client-side and compares the host only, so every standard Headroom setup (`ANTHROPIC_BASE_URL=http://127.0.0.1:8787`) loses Remote Control, and the only documented fix so far is "run Claude without Headroom for those sessions". The troubleshooting page already named the route that remains: leave `ANTHROPIC_BASE_URL` unset and reach Headroom through an `HTTPS_PROXY` front end with a local CA trusted through `NODE_EXTRA_CA_CERTS` (the enterprise proxy path Claude Code supports). It was left as an open item on #1779. This PR ships that route, tested end to end with a real Claude Code session steered from the Claude iOS app while every request went through Headroom. How it works: ```text Claude Code --HTTPS_PROXY--> mitmdump 127.0.0.1:8790 api.anthropic.com /v1/messages* -> Headroom 127.0.0.1:8787 -> api.anthropic.com api.anthropic.com other paths -> api.anthropic.com (unchanged) any other host (claude.ai, OAuth) -> raw CONNECT tunnel, never decrypted ``` - `ANTHROPIC_BASE_URL` is unset, so the gate passes and subscription auth is untouched. - mitmproxy decrypts `api.anthropic.com` only (`--allow-hosts`). The Remote Control bridge (`claude.ai`) is never decrypted. - Only the Messages API goes to Headroom. Feature flags, telemetry and the rest of the API keep going straight to Anthropic. One trap found while setting it up, and handled here: the Headroom proxy is often started by the `headroom init hook ensure` hook, so it inherits Claude Code's environment (the running proxy had `CLAUDECODE=1`). Its httpx client honours `HTTPS_PROXY`, so its upstream calls would come back through mitmproxy and loop. The docs tell users to prefix the hook with `env -u HTTPS_PROXY -u HTTP_PROXY -u NODE_EXTRA_CA_CERTS`. As a second guard, the addon marks the requests it redirects (`x-rc-proxy-hop`) and sends a marked request straight to Anthropic. The marker deliberately avoids the `x-headroom-` prefix: Headroom strips those headers before forwarding upstream (`_strip_internal_headers`), which would erase the marker. My first local version used `x-headroom-rc` and the new test catches exactly that. Refs #1779 ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `examples/claude_code_remote_control/route_to_headroom.py`: the mitmproxy addon (redirect `/v1/messages*` to Headroom, loop guard, SSE streaming, `HEADROOM_RC_UPSTREAM` override). mitmproxy is imported only for type checking, so it is not a Headroom dependency. - `examples/claude_code_remote_control/README.md`: setup (macOS / Linux), Claude Code settings, starting Headroom without the proxy variables, verification, caveats, rollback. - `docs/content/docs/troubleshooting.mdx`: the Remote Control section offers this route next to "launch Claude without Headroom", and the "What remains" paragraph now points to it. - `examples/README.md`: index entry. - `tests/test_issue_1779_https_proxy_route.py`: routing rules pinned with stand-in flow objects. No Headroom runtime code changes, no new dependency. ## Testing - [x] Unit tests pass (`pytest`) — the new file plus the existing #1779 gate tests - [x] Linting passes (`ruff check .`) — on the changed Python files, ruff 0.17.0 (the pinned version) - [ ] Type checking passes (`mypy headroom`) — N/A, no change under `headroom/` - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest -q tests/test_issue_1779_https_proxy_route.py tests/test_issue_1779_remote_control_gate.py 64 passed in 0.77s # Loop-guard test with the marker renamed to x-headroom-rc (proves it catches the trap): FAILED tests/test_issue_1779_https_proxy_route.py::test_hop_marker_survives_headroom_header_stripping 1 failed, 9 passed $ ruff check <changed files> && ruff format --check <changed files> All checks passed! ``` Smoke test of the addon in a real `mitmdump` (12.3.1) with Headroom running on 8787: ```text POST https://api.anthropic.com/v1/messages -> "server connect 127.0.0.1:8787", "Streaming response from 127.0.0.1" GET https://api.anthropic.com/v1/models -> "server connect api.anthropic.com:443" (not redirected) GET https://claude.ai/ (no custom CA given) -> TLS handshake succeeds, i.e. tunnelled, not intercepted ``` ## Real Behavior Proof - Environment: macOS (Darwin 27), Claude Code 2.1.312 (native install, claude.ai Max subscription), Headroom 0.37.0 (`headroom proxy --mode token --backend anthropic`, started by the `headroom init hook ensure` hooks), mitmproxy 12.3.1 run as a LaunchAgent. - Exact command / steps: `~/.claude/settings.json` `env` = `HTTPS_PROXY=http://127.0.0.1:8790`, `NO_PROXY=localhost,127.0.0.1,::1`, `NODE_EXTRA_CA_CERTS=~/.mitmproxy/mitmproxy-ca-cert.pem`, `ANTHROPIC_BASE_URL` removed, `remoteControlAtStartup: true`, both headroom hook commands prefixed with `env -u HTTPS_PROXY -u HTTP_PROXY -u NODE_EXTRA_CA_CERTS`. Started a new `claude` session. - Observed result: `/status` shows the proxy; answers stream normally; the mitmdump log shows one `127.0.0.1:8787` request per turn and no repeats; the session shows up in the Claude iOS app (Code tab) and answers messages sent from the phone; Headroom's savings keep counting those requests. - Not tested: Linux and Windows (the README gives the Linux equivalent but it was not run), the VS Code extension and Claude Desktop sessions, `headroom wrap claude` (which sets `ANTHROPIC_BASE_URL` itself, so it is not compatible with this route). ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: N/A - Stable/default behavior changed: no, opt-in recipe; nothing changes for existing users - Kill switch / disable path: restore `ANTHROPIC_BASE_URL`, remove the three variables, stop mitmdump - Unsafe override required: no - Qualification impact: none - Rollback path: revert the commit (docs, example and test only) ## Review Readiness - [x] I performed a self-review - [ ] PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes - Possible follow-up, not in this PR: `remote_control_gate_message` / `remote_control_gate_short_message` still only suggest running Claude without Headroom. They could link to this section. I left them untouched to keep this PR docs-only and avoid changing strings the #1779 / #3426 tests pin. - Security trade-off, stated in the README: the user trusts a locally generated CA for `api.anthropic.com` only; the CA key stays on the machine (`~/.mitmproxy`). Everything else is tunnelled. ### Maintainer current-main requalification Exact head `b9d7a8a9e72a87ebdc69e1cbcb9e00df3265a22a`, upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. Mechanically updated from current upstream main. The addon, host/path selection, hop-loop guard, header handling and streaming behavior are unchanged. The maintainer documentation fix addresses the new Devin note: remove ANTHROPIC_BASE_URL from all active settings scopes, explicitly including project .claude/settings.json and .claude/settings.local.json, while retaining other entries/hooks. Source inspection confirms project-scoped headroom init claude writes the variable to settings.local.json, so removing only the user/shell setting is insufficient. The setup and caveat now cover that case. Both issue-1779 suites pass (64 tests), plus ten shared Gemini fixture cases after the test-only repair for independently verified main failures. Changed Python Ruff/format and whitespace checks pass. No new runtime dependency or contributor routing logic was introduced. The author's existing macOS/native Claude/Remote Control/mitmproxy end-to-end evidence and exclusions were read; these independent Windows checks cover routing contracts and documentation accuracy, not a live macOS/Linux/Windows Remote Control session or a fresh native build. Author readiness attestations remain unchanged; nothing merged. Exact-head hosted checks are being handled by maintainers. No new finding in the reviewed scope; formal approval awaits exact-head hosted qualification; nothing merged. ### Maintainer current-main requalification Exact head `c5e1ea2afcbd7e4c91a75869706c6f68f08f7e3a`, upstream main `c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`. Final main synchronization with upstream |
||
|
|
a91c48a3a4 |
fix(examples): validate the macos launchagent port as plain decimal (#4733)
## Description
`examples/deployment/macos-launchagent/install.sh` checks the port with
`! [[ "${PORT}" =~ ^[0-9]+$ ]] || [[ "${PORT}" -lt 1024 ]] || [[
"${PORT}" -gt 65535 ]]` (line 126 on `1da8930a`). The regex lets any
digit string through, and `-lt` / `-gt` inside `[[ ]]` do bash
arithmetic, so:
- a leading `0` is read as octal: `0100000` is 32768 and passes; `08080`
is not valid octal, bash prints an arithmetic error, and the failed test
counts as false, so it is accepted;
- a number past 64 bits wraps: `18446744073709553616` passes the range
test;
- `01024` is rejected although the decimal value 1024 is in range.
The real command is `headroom proxy --port`, a `click.IntRange(1,
65535)` (`headroom/cli/proxy.py:298-301`, read, not run), so an accepted
bad value is written into the plist and the LaunchAgent fails when it
starts.
This PR requires 4 or 5 decimal digits with no leading zero
(`^[1-9][0-9]{3,4}$`) before the existing range test. Every accepted
value is then plain decimal 1000-99999, which cannot overflow, and the
`-lt 1024` / `-gt 65535` checks do the rest. A port written with a
leading zero (`01024`) is rejected; that is deliberate.
Part of #4730
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `examples/deployment/macos-launchagent/install.sh`: the port regex is
`^[1-9][0-9]{3,4}$` instead of `^[0-9]+$`. One line.
## Testing
- [x] Unit tests pass (`pytest`) (the new test file only; I did not run
the full suite)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
This is a shell example script, so the Python lint and type checks do
not apply to it. The new test file is checked with ruff (below). The
project's bug-fix policy asks for a test that fails before and passes
after, so a committed test runs the real script (below), and the earlier
probe is kept under it.
### Test Output
#### Committed regression test
`tests/test_macos_launchagent_install_port.py` runs the real
`install.sh` under `bash` with stub `uname`, `launchctl`, `headroom` and
`lsof` on `PATH` and a fake `HOME`. It is skipped when `bash` is not on
`PATH`. Environment: Ubuntu on WSL2, bash, Python venv
`/var/tmp/cb-fix-8mh5/repo/.venv`, pytest `-p no:cacheprovider`, test
file run from a scratch copy (no `headroom` import, no repo conftest).
Same test file, `install.sh` taken from `origin/main` (`1da8930a`):
```text
FAILED test_invalid_port_is_rejected[0100000]
FAILED test_invalid_port_is_rejected[08080]
FAILED test_invalid_port_is_rejected[18446744073709553616]
3 failed, 7 passed in 0.30s
```
Same test file, `install.sh` from this branch:
```text
10 passed in 0.33s
```
`ruff format --check` and `ruff check` on the new file: clean.
#### Earlier manual driver
A small probe takes the condition line that follows `# Validate port
number` out of the given `install.sh` and evaluates it for each value
(Git Bash on Windows 11, bash 5.3.15; `<probe>` stands for the local
path of my probe script).
`origin/main` (`1da8930a`):
```text
condition: ! [[ "${PORT}" =~ ^[0-9]+$ ]] || [[ "${PORT}" -lt 1024 ]] || [[ "${PORT}" -gt 65535 ]]
'0100000' ACCEPT
'08080' <probe>: line 8: [[: 08080: value too great for base (error token is "08080") <probe>: line 8: [[: 08080: value too great for base (error token is "08080") ACCEPT
'18446744073709553616' ACCEPT
'01024' REJECT
'1023' REJECT
'1024' ACCEPT
'8787' ACCEPT
'65535' ACCEPT
'65536' REJECT
'abc' REJECT
'' REJECT
```
This branch:
```text
condition: ! [[ "${PORT}" =~ ^[1-9][0-9]{3,4}$ ]] || [[ "${PORT}" -lt 1024 ]] || [[ "${PORT}" -gt 65535 ]]
'0100000' REJECT
'08080' REJECT
'18446744073709553616' REJECT
'01024' REJECT
'1023' REJECT
'1024' ACCEPT
'8787' ACCEPT
'65535' ACCEPT
'65536' REJECT
'abc' REJECT
'' REJECT
```
`ACCEPT` means the condition is false, so `install.sh` goes on; `REJECT`
means it calls `fatal`. The three bad values and nothing else change
from ACCEPT to REJECT, plus `08080` loses its bash error. `01024` was
already rejected on main and stays rejected.
## Real Behavior Proof
- Environment: Git Bash on Windows 11, bash 5.3.15. No macOS available.
- Exact command / steps: the probe above, run once on `origin/main`'s
`install.sh` and once on this branch's, for the values `0100000 08080
18446744073709553616 01024 1023 1024 8787 65535 65536 abc ''`.
- Observed result: on `origin/main` `0100000`, `08080` (with a bash
arithmetic error) and `18446744073709553616` are accepted; on this
branch all three are rejected, and `1024`, `8787`, `65535` are still
accepted. Tables above.
- Not tested: real macOS launchd; macOS bash 3.2 (the system bash) and
`launchctl`; I did not run `install.sh` end to end on a Mac. That
`headroom proxy --port` rejects such values is read from
`headroom/cli/proxy.py:298-301`, not run.
## Runtime Rollout Safety
- Rollout-managed feature(s): none; example script only.
- Minimum rollout channel: n/a
- Stable/default behavior changed: no for valid ports. Invalid ports
that were accepted are now rejected, and `--port 01024`-style input with
a leading zero is rejected.
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes (new test
file only; the probe above is the check)
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
- Scope is an example script. The wrong port only breaks that user's own
LaunchAgent; nothing here is a security issue.
- Other defects in the same directory are in #4730 and separate PRs.
Produced with AI assistance (Claude Code) operated by me (HetCreep); the
probe was run on my machine against a clone of the repository.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
|
46a3d910c9 |
docs(examples): name the port variable shell-integration.sh reads (#4735)
## Description `examples/deployment/macos-launchagent/README.md` (line 12 on `1da8930a`) tells users to add `export HEADROOM_PORT=8787` before sourcing `shell-integration.sh`. That script reads `HEADROOM_PROXY_PORT` (`shell-integration.sh:22`), and `install.sh:214` prints the same name in its own instructions. A user who follows the README with a non-default port exports a variable the script ignores, so it checks port 8787 and sets `ANTHROPIC_BASE_URL` to the wrong port or reports the proxy as not running. This PR changes the README line to `HEADROOM_PROXY_PORT`. Docs only. Part of #4730 ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `examples/deployment/macos-launchagent/README.md`: `export HEADROOM_PORT=8787` becomes `export HEADROOM_PROXY_PORT=8787`. One line. ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed Docs-only change; no code to run. Manual check: a grep of the directory before and after (below). ### Test Output `grep -rn "HEADROOM_PORT\|HEADROOM_PROXY_PORT" examples/deployment/macos-launchagent` on `origin/main` (`1da8930a`): ```text ./com.headroom.proxy.plist.template:24: <key>HEADROOM_PROXY_PORT</key> ./install.sh:214:echo " export HEADROOM_PROXY_PORT=${PORT}" ./README.md:12:export HEADROOM_PORT=8787 ./shell-integration.sh:11:# export HEADROOM_PROXY_PORT=8787 # Optional: customize port (default: 8787) ./shell-integration.sh:22:HEADROOM_PROXY_PORT="${HEADROOM_PROXY_PORT:-8787}" ./shell-integration.sh:34: lsof -iTCP:"${HEADROOM_PROXY_PORT}" -sTCP:LISTEN -t >/dev/null 2>&1 ./shell-integration.sh:59: export ANTHROPIC_BASE_URL="http://localhost:${HEADROOM_PROXY_PORT}" ./shell-integration.sh:64: export ANTHROPIC_BASE_URL="http://localhost:${HEADROOM_PROXY_PORT}" ./shell-integration.sh:65: echo "✓ Headroom proxy started on port ${HEADROOM_PROXY_PORT}" ./shell-integration.sh:68: echo "⚠ Headroom proxy not running on port ${HEADROOM_PROXY_PORT}" ``` This branch (same command): identical, except line 12 is now ```text ./README.md:12:export HEADROOM_PROXY_PORT=8787 ``` so `HEADROOM_PORT` no longer appears in the directory. ## Real Behavior Proof - Environment: docs-only; read in a clone of the repository at `1da8930a`. - Exact command / steps: the grep above, before and after. - Observed result: `README.md:12` was the only place in the directory that used `HEADROOM_PORT`; after the change every mention uses the name `shell-integration.sh` reads. - Not tested: the script itself was not run (docs-only). `wiki/macos-deployment.md` (lines 185, 249, 532) uses `HEADROOM_PORT` for the same shell-integration step; I left it out to keep this PR to the example directory. ## Runtime Rollout Safety - Rollout-managed feature(s): none; docs only. - Minimum rollout channel: n/a - Stable/default behavior changed: no - Kill switch / disable path: n/a - Unsafe override required: no - Qualification impact: none - Rollback path: revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes (docs only, none affected) - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes - Other defects in the same directory are in #4730 and separate PRs. Produced with AI assistance (Claude Code) operated by me (HetCreep). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
|
0ddaa56c21 |
fix(vertex): explain Vertex failures in-band, correct the model/region matrix (#2640)
## Description Vertex rejects onboarding requests for a handful of recurring reasons — expired ADC, a partner model never enabled in Model Garden, a model not served in the requested location, a missing optional SDK — and its own error text names none of the fixes. Users see a bare `404` or an opaque `500` through their SDK and cannot tell a proxy bug from a project-config gap. This makes the proxy explain those failures where it happens, adds a `[vertex]` extra so the LiteLLM Vertex backend can actually start, and corrects an example, a test matrix, and three documentation claims that could not have worked as written. Closes #2632 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made **Explain Vertex failures in-band** (`headroom/providers/vertex/diagnostics.py`, new) - On `401/403/404/429`, attach an actionable hint three ways: a WARNING in the proxy log, an `x-headroom-hint` response header, and a note appended to `error.message`. The last is load-bearing — SDKs raise with the message string and discard headers. - Successful responses are never modified; annotation is idempotent; statuses we cannot explain get no hint rather than a plausible-sounding wrong one. - Routes opt in via a one-line `@with_vertex_diagnostics` decorator reading path params FastAPI already injects (**+8 lines** in `proxy_routes.py`, no re-indentation). - Same treatment for backend setup failures, where `--backend vertex` surfaced LiteLLM's raw `No module named 'vertexai'` as an opaque 500. **Make `--backend vertex` able to start** (`pyproject.toml`, `registry.py`, `cli/proxy.py`) - New `[vertex]` extra carrying `google-cloud-aiplatform>=1.38`. Deliberately **not** folded into `[proxy]`: measured at ~175 MB installed, and the native passthrough routes never needed it. Follows the existing `proxy-prod` split. - Startup preflight keyed on the **resolved provider**, so every alias normalized at `registry.py:229` (`vertex`, `vertex_ai`, `litellm-vertex`, `google-vertex`, `googlevertex`) is covered. Skipped when a backend class is injected, so existing fake-based tests are unaffected. - New `BackendUnavailableError` so the CLI prints the remedy and exits `2` instead of dumping a traceback through startup internals. **Correct things that could not have worked** | | was | is | |---|---|---| | SDK client | `http_options={"api_endpoint": ...}` | `{"base_url": "http://..."}` — `api_endpoint` is not a field on `HttpOptions`; pydantic rejects it before any request is sent | | Thinking config | `ThinkingConfig(thinking_budget_tokens=)` | `thinking_budget=` | | Request payload | `contents: [{parts: [...]}]` | adds `role: "user"` — the `global` endpoint enforces it, regional was silently lenient | | Auth guidance | `gcloud auth print-access-token` | `application-default` — the plain user token gets `401 ACCESS_TOKEN_TYPE_UNSUPPORTED` | | Test matrix | model × location cross-product | explicit servable pairs | - Test matrix is now four pairs that can actually serve: `global`/`gemini-flash-latest`, `global`/`claude-sonnet-4-6`, `europe-west2`/`gemini-3.5-flash`, `us-east5`/`claude-sonnet-4-6`. Gemini 3.x has no US regional endpoint and `-latest` aliases are global-only, so a pinned `us-central1` Gemini case can never pass. No Gemini 2.5 — it retires 2026-10-16. - `test_thinking_levels` asserted only `status_code < 500` while sending an ignored `thinkingBudgetTokens` key: it passed on 400s and proved nothing. Now asserts 200 plus scored usage metadata with the correct `thinkingBudget` spelling. - Cases the caller's project genuinely cannot serve now **skip** with an actionable reason instead of reporting a red proxy. **Docs** (`proxy.mdx`, `installation.mdx`, `wiki/vertex.md`) - `proxy.mdx` claimed `publisher=anthropic` passthrough "routes through the same LiteLLM-Vertex path as `--backend vertex_ai`". It does not — without the flag it is a direct forward, which is why it works on a bare `[proxy]` install where the LiteLLM path cannot even import. - Documented the trap of combining `--backend vertex` with native Vertex URLs: LiteLLM takes project/region/model from its own config and ignores the URL. - `installation.mdx` and `wiki/vertex.md` said the SDK is "not included in any extra". - Cleared 15 pre-existing `ruff` findings in the two files this PR rewrites (4 remain in `plugins/headroom-oauth2`, untouched by this work). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ mypy headroom Success: no issues found in 511 source files $ ruff check . Found 4 errors. # all in plugins/headroom-oauth2, pre-existing, untouched by this PR $ pytest tests/test_proxy_vertex_native_integration.py tests/test_vertex_diagnostics.py -q 44 passed, 1 warning in 69.86s # 8 live integration + 36 offline diagnostics, 0 skipped $ pytest tests/ -k "anthropic_messages or handler_anthropic or backend or registry or cli_proxy or vertex or proxy_routes" 853 passed, 49 skipped, 9345 deselected, 2 warnings in 28.41s ``` New tests in `tests/test_vertex_diagnostics.py` (36, no credentials required): status coverage, per-publisher remedy divergence, idempotency, `content-length` consistency after body rewrite, non-JSON and body-less/streaming responses, every backend alias preflighted, other providers unaffected, injected backend bypasses the preflight. ## Real Behavior Proof - Environment: macOS 14, Python 3.11.15, live GCP project (`alanblount-sandbox`) with `aiplatform.googleapis.com` enabled and Claude enabled in Model Garden; ADC via `gcloud auth application-default login` - Exact command / steps: `export GCP_PROJECT_ID=alanblount-sandbox && export GCP_ACCESS_TOKEN=$(gcloud auth application-default print-access-token) && uv run --extra proxy --with google-genai python3 examples/vertex_genai_sdk_demo.py && uv run --extra proxy --with pytest-asyncio pytest tests/test_proxy_vertex_native_integration.py tests/test_vertex_diagnostics.py -q && headroom proxy --backend vertex` (exits 2, then retried with `[proxy,vertex]` extra installed) - Observed result: demo printed `✓ Standard response received successfully! > 1, 2, 3, 4, 5` and `✓ Thinking response received successfully!`; 44 tests passed (8 live integration, 36 diagnostics), 0 skipped; `--backend vertex` exited 2 with the remedy message; same request with the extra installed returned 200; invalid region produced `[headroom] hint:` in the response body and `x-headroom-hint` header; success responses were byte-clean - Not tested: Windows; `us`/`eu` multi-region endpoints and Claude 4.7+ (project cannot serve them); Provisioned Throughput; Rust proxy path; `--backend vertex` dropping URL project/model on native Vertex routes (documented as known gap in Additional Notes) ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A — proxy and CLI behavior; terminal output is quoted above. ## Additional Notes - `uv.lock` intentionally not included: the new extra is opt-in and does not change default resolution. - Error-body rewriting is confined to already-failing responses and wrapped so a diagnostic can never turn a clean upstream error into a proxy 500. - The preflight raises where the provider is resolved (`registry.py`) rather than at the CLI, so programmatic embedders get the same guarantee; the CLI only formats it. - Three test files fail to collect on this branch (`test_hermes_passthrough_compression`, `test_observability_metrics`, `test_observability_tracing`). They predate this work and are unrelated, so they were excluded rather than fixed — happy to open a separate issue. - `[vertex]` was chosen over adding the dependency to `[proxy]` after measuring the footprint at ~175 MB. If maintainers prefer it in `[proxy]`, that is a one-line change. ## Runtime Rollout Safety - Rollout-managed feature(s): none; diagnostics annotate already-failing Vertex responses and the backend dependency is opt-in through the vertex extra. - Minimum rollout channel: normal stable release after Vertex diagnostics, route, and backend-path tests pass. - Stable/default behavior changed: successful responses and native passthrough remain unchanged; supported Vertex failures gain an actionable header/message, and selecting the LiteLLM Vertex backend without its SDK now fails once at startup. - Kill switch / disable path: use native Vertex passthrough without the LiteLLM backend, or human-revert the diagnostic/preflight change. - Unsafe override required: none. - Qualification impact: offline diagnostics, native integration, and mocked LiteLLM Vertex request-shaping tests must remain green. - Rollback path: human-revert the PR; no persisted state or migration is involved. <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2640" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> ### Maintainer current-main requalification Exact head `5d7071f348d2462f666fd7ac1ceb8ac18050169a`, upstream main `2d3ee51846435ef5b320eea4806af14e602b0d8c`. Re-reviewed latest integration head |
||
|
|
d4c0810ea3 |
fix(client): honor configured cache optimizer enablement (#4501)
## Description `HeadroomConfig(cache_optimizer=...enabled=False)` does not disable the SDK's provider cache optimizer. The constructor gates initialization on its separate `enable_cache_optimizer=True` default, so an explicitly disabled configuration still modifies outgoing requests. The constructor documents that `config` takes precedence over individual settings. Use the effective configuration flag for optimizer initialization and the setup event's `cache_optimizer_enabled` metadata. Without a supplied config, the existing constructor flag still populates that configuration and preserves its current behavior. With a supplied config, its flag is authoritative, including when the separate constructor argument disagrees. No issue is linked: this was reproduced during a public SDK configuration audit. #4270 concerns storage URL precedence; #4500 concerns Anthropic stream dispatch; neither fixes this cache enablement gate. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Read the effective `CacheOptimizerConfig.enabled` flag when choosing whether to initialize an optimizer, including an explicitly supplied optimizer. - Report that same configured flag in setup lifecycle metadata. - Cover all four config/constructor boolean combinations and the disabled custom-optimizer case. - Include a runnable actual-SDK localhost HTTP example with optional raw request capture. ## Testing - [ ] Unit tests pass (`pytest`): complete required suite pending; focused results below. - [x] Linting passes (`ruff check .`): pinned Ruff0.17.0, whole tree. - [x] Type checking passes (`mypy headroom`): normal whole-package commit hook passed. - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text New regressions on unchanged main17d38: 3 failed, 2 passed Patched regressions + existing client/provider/pipeline suites: 57 passed Normal commit hooks: passed, including pinned Ruff0.17.0, formatting, whole-package mypy, plugin/version checks and commitlint ``` Exact focused command: ```sh HEADROOM_OFFLINE=1 HEADROOM_BEACON=off DO_NOT_TRACK=1 \ PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 PYTHONPATH=. python -m pytest \ -p pytest_asyncio.plugin tests/test_client_cache_optimizer_config.py \ tests/test_cache/test_client_integration.py tests/test_provider_registry.py \ tests/test_provider_registry_extended.py tests/test_canonical_pipeline.py -q ``` Global plugin autoload is disabled to isolate an unrelated installed plugin's interaction with an existing registry test's process-wide import monkeypatch. The initial broader pipeline run encountered an unchanged test requiring the then-unavailable native extension. After the extension was built from the unchanged base Rust sources, the exact frozen head passed all 57 selected tests, including that case. This is a focused pass; complete required pre-push validation remains separate. ## Real Behavior Proof - Environment: macOS arm64, Python3.13.5, actual OpenAI SDK2.41.0 and HTTPX0.28.1. Audit fixture runs Headroom in optimize mode, with cache alignment and SmartCrusher disabled, offline/beacon-off/DNT, and temporary workspace/config/cache directories. A loopback HTTP server returns synthetic completions; no remote model endpoint or paid API is contacted. - Exact command / steps: `PYTHONPATH=. python examples/cache_optimizer_config_local.py`. Optionally set `HEADROOM_PROOF_WIRE_DIR=/tmp/headroom-cache-wire` to retain the actual HTTP request body bytes. - Observed result: before the fix, `enabled=False` still selected `openai-prefix-stabilizer` and changed the outgoing system text from `"Keep exact spaces.\n"` to `"Keep exact spaces."`. Afterward, the disabled configuration reports no optimizer and preserves the exact original bytes. The enabled positive control still selects the optimizer and sends exactly the same HTTP JSON as before the patch. Both cases receive `local fixture` from the actual SDK/HTTP round trip. - Not tested: live model endpoints, model quality, billed savings, semantic cache enablement, or unrelated native runtime features. The fixture's token usage is synthetic, not a billing measurement. ## Runtime Rollout Safety - Rollout-managed feature(s): N/A; existing Python SDK configuration gate. - Minimum rollout channel: N/A. - Stable/default behavior changed: the supplied config's cache optimizer flag is honored. If both a config and a contradictory constructor flag are supplied, config wins as documented. Constructor-only defaults remain unchanged. - Kill switch / disable path: set `config.cache_optimizer.enabled=False`, or use `enable_cache_optimizer=False` without a supplied config. - Unsafe override required: no. - Qualification impact: no rollout-channel change. - Rollback path: revert this two-line production change. ### Additional native validation and upload limits The native extension now builds from unchanged source with Rust1.95.0 and a matching Xcode26.4 SDK. The exact-head Makefile Python test-file list passes: **177 passed, 4 skipped**, with one existing Anthropic token-count approximation warning. The previously blocked broader client/pipeline selection now passes **57 tests**. Whole-tree pinned Ruff passes. Full pytest, the complete pre-push workflow and required hosted CI are not claimed as passing. The installed `scripts/install-git-hooks.sh` documents an occasional `git push --no-verify` upload exception. It is used for this independently reviewed Python-only correction after bounded native validation; full Rust workspace tests remain limited by the local disk reserve. No Rust sources or Cargo dependencies change. Prepared and independently reviewed with AI assistance. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes Only two production lines change. Cache optimizer implementation, semantic caching, network destinations, dependencies and native code are unchanged. The example demonstrates existing documented configuration behavior; it introduces no new runtime capability. --------- Signed-off-by: Rudy Celekli <rudy@gradiahq.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
3910caabd6 |
fix(client): preserve Anthropic create stream flags (#4500)
## Description `HeadroomClient.messages.create(..., stream=False)` raises `TypeError: ... got multiple values for keyword argument 'stream'` before reaching Anthropic. The wrapper hardcodes `stream=False` and forwards the caller's same keyword through `**kwargs`. `stream=True` fails the same way. Consume the flag explicitly and preserve the native entrypoint distinction: `messages.create(stream=True)` returns an iterator of raw events; `messages.stream()` continues to return Anthropic's stream context manager. A private transport selector keeps their dispatch separate while lifecycle extension metadata continues to report the existing `anthropic` API style. No issue is linked: this was reproduced while auditing the existing SDK compatibility contract. Related open #4482 owns existing wrapped-stream cleanup; this change does not replace that cleanup implementation. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Accept and forward an explicit `stream` flag from Anthropic `Messages.create`. - Dispatch streaming create calls through native `messages.create(stream=True)` and reuse the existing stream wrapper. - Preserve non-streaming response metrics and native `messages.stream()` behavior. - Add regression tests and a runnable loopback example using the actual Anthropic SDK and HTTPX. ## Testing - [ ] Unit tests pass (`pytest`): full required suite pending; focused results below. - [x] Linting passes (`ruff check .`): full pinned Ruff0.17.0 passes. - [x] Type checking passes (`mypy headroom`): normal full-package pre-commit hook passed - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text Baseline regression: 3 failed, 1 passed Patched focused client/provider suites: 48 passed Changed-file Ruff lint/format and git diff --check: passed ``` The focused runs used existing Python 3.13.5; changed-file lint/format also passed the pinned Ruff 0.17.0. Normal repository pre-commit and commit-message hooks passed, including full-package mypy, pinned Ruff, plugin-version checks and commitlint. The complete required pre-push checks remain separate pending gates. Global pytest plugin discovery was disabled because an unrelated installed plugin interferes with an existing test's process-wide import monkeypatch; pytest_asyncio.plugin was explicitly enabled. Final focused results: 48 passed in 9.11s without warnings; this is not a full-suite pass. ## Real Behavior Proof - Environment: macOS 27.0.1 arm64, Python 3.13.5, actual Anthropic SDK 0.107.1, HTTPX 0.28.1. Audit mode; cache optimization disabled; offline mode, beacon off and DNT; temporary task-owned workspace/config/cache directories. The example starts a loopback HTTP fixture, with a non-secret fixture API key. - Exact command / steps: `PYTHONPATH=. python examples/anthropic_create_stream_local.py`. - Observed result: native and wrapped non-streaming calls both returned `local fixture`. Native and wrapped streaming create calls yielded the same six event types (`message_start`, `content_block_start`, `content_block_delta`, `content_block_stop`, `message_delta`, `message_stop`). Existing `messages.stream()` produced `local fixture` through its context manager. All five actual HTTP responses were closed after full consumption. Lifecycle events kept `api_style=anthropic`. Before the fix, the native call succeeded while the wrapped call raised the duplicate-keyword TypeError without making another HTTP request. - Not tested: actual Anthropic endpoints, paid models, model quality, production deployment, early-close resource cleanup (owned by #4482), or stream usage accounting. ## Runtime Rollout Safety - Rollout-managed feature(s): N/A; existing Python SDK argument/transport compatibility. - Minimum rollout channel: N/A. - Stable/default behavior changed: explicit `stream=False` is accepted; explicit `stream=True` uses the native create iterator. Omitted flag retains non-streaming behavior. - Kill switch / disable path: use the original SDK client. - Unsafe override required: no. - Qualification impact: no rollout-channel changes. - Rollback path: revert this change. ### Broader check limits and upload exception Rust1.95.0 `cargo fmt --all -- --check` and `cargo clippy --workspace -- -D warnings` passed on the unchanged native sources. The cold workspace-test build was stopped twice by a disk-reserve guard; no full Rust-test or native-dependent Python-suite pass is claimed. Normal commit hooks passed, including pinned Ruff0.17.0, whole-package mypy and commitlint. These patches do not change Rust sources or Cargo dependencies. The [installed pre-push hook](https://github.com/headroomlabs-ai/headroom/blob/17d38e5699b7d7c69ac6addcd762f39e42dd7f00/scripts/install-git-hooks.sh) documents an occasional `git push --no-verify` upload exception. It was used for this independently reviewed Python-only fix after the bounded native-check attempts. Required hosted CI and maintainer review remain pending; the complete pre-push gate has not passed locally. Prepared and independently reviewed with AI assistance; actual runtime proof and its limits are recorded above. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes No new dependency or provider capability is introduced. The local HTTP fixture is protocol evidence, not a claim of live provider verification. Required full Rust/Python pre-push gates must be reported before publication readiness; normal commit hooks passed. Streaming create returns Headroom’s existing event iterator. Native Stream-specific methods/attributes and early-close behavior are not added or qualified by this change. ### Follow-up native validation The actual Python extension now builds and imports from the unchanged native source using Rust1.95.0 and the matching Xcode26.4 SDK. At this exact PR head, the Makefile pre-push Python test-file selection passes **177 tests with 4 skips**; one existing Anthropic approximation warning is reported. Whole-tree pinned Ruff0.17.0 passes. The complete Rust workspace test build was again stopped by the local disk-reserve guard before completion; this is not a Rust-test pass. The incomplete task-owned build cache was removed after all build processes exited, retaining the source-built extension and logs. Full pytest, complete pre-push success and required hosted CI remain unclaimed. <!-- security-sync-20261028 --> ### Security baseline sync — 2026-10-28 Current exact head: `f85946f309c0c26308b1462bf7006f0b078ac7be`. Normal history merge of upstream `3c6f82b06091206379106f8fe1a54f2400ed7a93` brings in the maintainer-owned dependency remediation [#4517](https://github.com/headroomlabs-ai/headroom/pull/4517), following the prior npm baseline sync. The authored fix and its test/example files are unchanged by this sync; no additional dependency policy or runtime logic was introduced. This imports the maintainer remediation requested by review; it does not establish audit success or approval. Validation on this head (macOS arm64, Python3.13.5, offline mode/beacon off/DNT, existing source-built native extension): `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -p pytest_asyncio.plugin tests/test_anthropic_client_create_stream.py tests/test_providers/test_anthropic.py tests/test_provider_registry_extended.py -q` (58 passed; one existing token-count approximation warning). `PYTHONPATH=. python examples/anthropic_create_stream_local.py` passed against actual Anthropic0.107.1 and HTTPX0.28.1: native/wrapped six-event streaming agreement, native stream context manager, all five HTTP responses closed. `PRE_COMMIT_HOME=<existing task cache> pre-commit run --all-files` passed, including pinned Ruff, whole-package mypy, version and merge-conflict checks. Normal commit hooks and commitlint passed. Existing native sources/Cargo dependencies are unchanged. The documented occasional `git push --no-verify` exception was used for a normal fast-forward upload; full Rust workspace tests, full pytest and complete pre-push qualification were not rerun or claimed. Hosted security checks and maintainer review remain separate gates. Existing human-readiness declarations are preserved; this status update is evidence of local validation and a baseline sync, not human approval or a claim that hosted CI is green. As of `2026-10-28T17:40:32.107305+00:00`, hosted pip-audit and all five npm audits passed on this exact head ([security workflow](https://github.com/headroomlabs-ai/headroom/actions/runs/40737711996/job/122990639723)); cargo audit also passed. Other hosted jobs remain in progress, and the human review state remains `CHANGES_REQUESTED`. This is not complete CI qualification or human approval. Signed-off-by: Rudy Celekli <rudy@gradiahq.com> Co-authored-by: JD Davis <mxjerrett@gmail.com> |
||
|
|
02ce467d2d |
chore(integrations): fix the broken LangChain example and stale docstrings (#4134)
## Description
This PR fixes a broken example and two stale docstrings in the
integrations.
- `examples/langchain_demo/run_comparison.py` could not run its Headroom
arm at all. It failed in three places, in this order:
1. `HeadroomConfig(smart_crusher_threshold=...,
smart_crusher_max_items=..., cache_alignment=..., rolling_window=...)`:
none of these keyword arguments exist, so it raised `TypeError` before
anything ran. (Rolling-window was removed from the pipeline; see
`docs/content/docs/architecture.mdx`.)
2. `HeadroomChatModel(..., headroom_config=config)`: the constructor
parameter is `config`, so this raised `TypeError: ... got multiple
values for keyword argument 'headroom_config'`.
3. `headroom_model.get_total_tokens_saved()`: `HeadroomChatModel` has no
such method; it exposes the `total_tokens_saved` property, so this would
have raised `AttributeError` at the end of each scenario.
- The `headroom/integrations/mcp/server.py` module docstring showed a
`HeadroomMCPMiddleware` class that does not exist.
- The `headroom/integrations/__init__.py` docstring showed the same Agno
import line twice.
Refs #4136
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `examples/langchain_demo/run_comparison.py`:
- the config becomes `HeadroomConfig()`, with a comment saying why. The
old keyword arguments do not exist, and their nearest real equivalent
(`smart_crusher=SmartCrusherConfig(...)`) would have no effect:
`TransformPipeline` builds its `ContentRouter` from the router's own
defaults and never reads `HeadroomConfig.smart_crusher`. That wiring gap
is a separate product change; it is recorded in the docs ledger because
`api-reference.mdx` shows such a setting;
- `headroom_config=config` becomes `config=config`;
- `headroom_model.get_total_tokens_saved()` becomes
`headroom_model.total_tokens_saved`.
- the printed narrative no longer claims a 20-item cap or a CacheAligner
gain (CacheAligner is off by default, and the 15-item setting applies
only to lossy row selection); the simulation figure is labelled a rough
5x estimate.
- `headroom/integrations/mcp/server.py`: the "Middleware" docstring
example now uses the real `HeadroomMCPClientWrapper(base_client,
config=config)` and its `call_tool`.
- `headroom/integrations/__init__.py`: the first Agno example line now
shows the package-level import `from headroom.integrations import
HeadroomAgnoModel`, which is re-exported there. The "or explicitly" line
keeps the subpackage import.
## Testing
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ python -c "from headroom.integrations.langchain import HeadroomChatModel as H; print(type(H.__dict__.get('total_tokens_saved')).__name__, hasattr(H, 'get_total_tokens_saved'))"
property False
$ python -c "from headroom.integrations import HeadroomAgnoModel; print('ok')"
ok
$ python -m py_compile examples/langchain_demo/run_comparison.py && echo compiled
compiled
# run_agent_headroom(SCENARIOS[0]) with ChatOpenAI replaced by a fake LangChain chat model (no API key)
# that first asks for a search_logs tool call, then answers "done"
origin/main: TypeError: HeadroomConfig.__init__() got an unexpected keyword argument 'smart_crusher_threshold'
this branch: model calls: 2 | 2nd call message types: ['SystemMessage', 'HumanMessage', 'AIMessage', 'ToolMessage']
OK scenario='User Account Investigation' tool_calls=1 tool_output_tokens=25643 final='done' messages=5
$ pytest tests/test_integrations/mcp tests/test_integrations/langchain/test_chat_model.py
70 passed, 9 skipped
$ uvx ruff@0.16.8 check / format --check <3 touched files> -> All checks passed! / already formatted
```
## Real Behavior Proof
- Environment: macOS, Python 3.12, repo venv.
- Exact command / steps: the commands above. The example run imports
`examples.langchain_demo.run_comparison`, swaps `ChatOpenAI` for a
`FakeMessagesListChatModel` that first returns a `search_logs` tool call
and then answers "done", and calls `run_agent_headroom` on the first
scenario. So construction, a tool-result turn through
`HeadroomChatModel`, and the metrics read all run.
- Observed result: on origin/main the Headroom arm raises `TypeError` at
config construction; on this branch it returns an `AgentRun`. The
docstring imports resolve.
- Smoke harness (real proxy started from this checkout against a local
fake upstream; temp HOME and workspace, offline):
```text
branch: cleanup/cli-integrations-integrations-unused-imports @
|
||
|
|
14a4b54302 |
docs: add README for examples/mcp_demo (#3045)
## Description Adds `examples/mcp_demo/README.md` in the same style as `examples/langchain_demo/README.md`. `examples/mcp_demo` contains three runnable demo scripts plus mock-server generators, but no README — the demo is undiscoverable and its entry points (module invocation with `PYTHONPATH=.`, the `OPENAI_API_KEY` requirement for the eval) are only documented inside script docstrings. ## Type of Change - [x] Documentation update ## Changes Made - Added `examples/mcp_demo/README.md`: overview, prerequisites, how to run each demo script (including `PYTHONPATH=.` module invocation), mock-server generation, and expected output for the eval demo (`OPENAI_API_KEY` requirement noted). ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # Docs-only change: no Python surface touched, so pytest/ruff/mypy are unaffected. # Manual checks performed: # - every command in the README cross-checked against the scripts' argparse/docstrings # - structure and tone match examples/langchain_demo/README.md # - referenced file paths all exist in examples/mcp_demo/ ``` ## Real Behavior Proof - Environment: local checkout of `main` + branch - Exact command / steps: walked each documented command against the actual scripts in `examples/mcp_demo/`; verified every referenced path exists - Observed result: documented invocations match the scripts' `__main__` entry points and flags; env-var requirements match the code - Not tested: live OpenAI-backed eval run (needs `OPENAI_API_KEY`); mock-server demos were verified by reading the code paths ## Runtime Rollout Safety - Rollout-managed feature(s): none — example documentation only - Minimum rollout channel: N/A - Stable/default behavior changed: none - Kill switch / disable path: N/A - Unsafe override required: none - Qualification impact: none - Rollback path: revert the single-file commit ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes Docs-only PR: pytest/ruff/mypy items intentionally unchecked (no Python code changed). Signed-off-by: LeonSGP43 <LeonSGP43@users.noreply.github.com> Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
e67b3c8a29 |
feat(vertex): add Gemini 3.8 Flash agent benchmark suite (#3371)
## Summary Adds a reproducible, end-to-end multi-turn agent benchmark suite for **Gemini 3.8 Flash** on Google Cloud Vertex AI (Gemini Enterprise Agent Platform) under `examples/vertex_gemini_benchmark/`. Compares **Direct Vertex AI** vs **Headroom-proxied Vertex AI** across 4 realistic production agent scenarios: 1. **SRE Incident Root Cause Analysis**: 350+ Kubernetes & microservice logs + Prometheus metrics (identifying connection pool exhaustion, cluster endpoint, port, and cascading circuit-breaker trips). 2. **Codebase Security Audit & PR Review**: Multi-file code search results + git diff (identifying insecure `alg: none` and `bypass_auth` JWT backdoors). 3. **Enterprise BigQuery Analytics**: 500 tabular transaction rows (computing summaries and isolating fraudulent accounts and high-spend outliers). 4. **Multi-Turn RAG & System Architecture Synthesis**: 25 dense technical chunks regarding agent context management contracts. ### Live Benchmark Results (Gemini 3.8 Flash on Vertex AI `global`) ```text ================================================================================== 📊 FINAL BENCHMARK SUMMARY: GEMINI-3.8-FLASH ON VERTEX AI ================================================================================== Scenario | Baseline Prompt | Headroom Prompt | Reduction ---------------------------------------------------------------------------------- SRE Incident Root Cause Analysis | 51,775 | 13,842 | 73.3% Codebase Security Audit & PR Review | 6,821 | 4,632 | 32.1% Enterprise BigQuery Analytics & Outlier Detection | 49,020 | 6,120 | 87.5% Multi-Turn RAG & Architecture Synthesis | 4,071 | 1,437 | 64.7% ---------------------------------------------------------------------------------- TOTAL / AGGREGATE | 111,687 | 26,031 | 76.7% Metric | Baseline | Headroom | Delta / Impact ---------------------------------------------------------------------------------- Total Prompt Tokens | 111,687 | 26,031 | - 76.7% Total All Tokens | 116,639 | 29,168 | - 75.0% Total Cost ($ USD) | $ 0.09410 | $ 0.02849 | - 69.7% Avg Latency (ms) | 8921ms | 5748ms | +3173ms Ground Truth Accuracy (Absolute) | 100.0% | 100.0% | 100.0% Retained ================================================================================== ``` ## Real Behavior Proof • **Environment**: Linux, Python 3.11, live Google Cloud project (`alanblount-demo`) with Vertex AI enabled, Application Default Credentials (`gcloud auth application-default login`). • **Exact command / steps**: `GCP_PROJECT_ID=alanblount-demo python examples/vertex_gemini_benchmark/benchmark.py --model gemini-3.8-flash` • **Observed result**: Headroom proxy started cleanly, executed all 4 multi-turn scenarios on `gemini-3.8-flash`, verified ground truth error causes and anomalies, recorded a **76.7% prompt token reduction** and **69.7% cost savings**, exported `results.json`, and shut down cleanly. • **Not tested**: Windows. ## Runtime Rollout Safety • **Rollout-managed feature(s)**: None; self-contained benchmark suite under `examples/`. • **Minimum rollout channel**: standard release. • **Stable/default behavior changed**: None; purely adds benchmark tooling and documentation under `examples/`. • **Kill switch / disable path**: N/A. • **Unsafe override required**: None. • **Qualification impact**: None. • **Rollback path**: Revert PR. ## Review Readiness [✓] I have performed a self-review [✓] This PR is ready for human review ## Checklist [✓] My code follows the project's style guidelines [✓] I have performed a self-review of my code [✓] I have commented my code, particularly in hard-to-understand areas [✓] I have made corresponding changes to the documentation [✓] My changes generate no new warnings [✓] I did not edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes • Companion to PR #2640 (which adds in-band Vertex diagnostic hints and fixes). |
||
|
|
96dfc7005f |
chore: finish the org and docs-domain migration (#3389)
Two stale identities left over from moves that were only half applied. ## GitHub org `install/state.py` already migrated Docker images to `ghcr.io/headroomlabs-ai/headroom` and carries a `_DEPRECATED_IMAGE_REPO` constant that rewrites the old path on load — so this move was decided long ago. Everything else still said `chopratejas/headroom`. Rewritten across packaging (`pyproject.toml`, `Cargo.toml`), both plugin marketplaces, the agent-hooks manifests, install scripts, CI templates, crate docs and tests. **Two deliberate exclusions:** - **`ghcr.io/chopratejas/headroom` is untouched everywhere.** It is the *input* to `_migrate_deprecated_image()` and to the tests covering it. Rewriting it would silently disable the migration. - **`CHANGELOG.md` keeps its 238 historical references.** GitHub's rename redirect resolves them, and rewriting would bury this diff in noise. **HuggingFace model ids are NOT affected and were never in scope.** `chopratejas/kompress-base`, `kompress-v2-base`, `kompress-finance`, `technique-router`, `technique-router-onnx` and `siglip-image-encoder-onnx` are account-scoped identifiers on a different service. All six were validated against the HF API before this change: | Model | HTTP | |---|---| | `kompress-base` | 200 | | `kompress-v2-base` | 200 | | `technique-router` | 200 | | `technique-router-onnx` | 200 | | `siglip-image-encoder-onnx` | 200 | | `kompress-finance` | **401 (private)** | The 401 is unrelated to this migration, but worth knowing: `kompress_compressor.py:1255` references it, so that path is not reachable by an OSS user without access. **One functional fix, not cosmetic:** `_marketplace_source()` (`headroom/cli/init.py`) returned the old org as its fallback, so plugin installs were pointed at the retired repo. ## Docs domain The site moved to `docs.headroomlabs.ai`, but eleven references still named the vercel.app host — including **`llms.txt` (27 links)**, which is served at `docs.headroomlabs.ai/llms.txt` and read by LLM crawlers, the **dashboard's own Documentation link**, and a `headroom wrap` help message. The docs app's canonical-URL fallbacks were fixed separately in #3388. ## Also folded in `RUST_DEV.md` corrections found while verifying the above: - `make build-wheel` points at `Cargo.toml`, not `pyproject.toml` - the Rust toolchain is pinned, not tracking `stable` - `headroom-simulators` was missing from the workspace listing - the Python `CompressionStore` defaults to **SQLite**, not in-memory, when `HEADROOM_CCR_BACKEND` is unset (verified by execution) — this one matters for multi-worker reasoning And the install scripts pointed at `docs/docker-install.md`, a path that no longer exists; they now point at the live docs page. ## Verification 297 tests pass, including the plugin-manifest, install-state and release-workflow suites that assert on these exact strings. `ruff check` clean. Unrelated lockfile churn (`uv.lock`, both `package-lock.json`) is deliberately excluded. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01HRaEPjQcCSAwD2xbQMNENu --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
6bdc8c44a3 |
docs(metrics): ship an importable Grafana dashboard (#2168)
## Description
<!-- Briefly explain the change and why it is needed. -->
The metrics docs describe the `headroom_*` Prometheus metric family and
suggest example Grafana panels, but ship no importable dashboard — users
have to build one by hand. This adds a ready-to-import Grafana dashboard
built **only** on documented metric names (`headroom_requests_total`,
`headroom_tokens_saved_total`, `headroom_tokens_input_total`, and the
`headroom_overhead_ms_*` millisecond summary), and links it from the
**Grafana Dashboard** section of `docs/content/docs/metrics.mdx`.
This is a docs/examples-only addition — no source code changes.
Closes #
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Added `examples/grafana/headroom-dashboard.json` — a ready-to-import
Grafana dashboard (7 panels, uid `headroom-compression`) built entirely
on Headroom's documented `/metrics` names. Panels cover tokens saved,
input tokens, request rate, average processing overhead
(`headroom_overhead_ms_sum` / `headroom_overhead_ms_count` with
min/max), tokens-saved/sec, and request rate by pool. It uses **no
histograms** (the proxy emits none). The `pool`/`source` template
variables use regex matchers (`=~`) so they are optional and match
series without those labels.
- Updated `docs/content/docs/metrics.mdx` — linked the new dashboard
from the **Grafana Dashboard** section with import instructions, keeping
the existing ad-hoc PromQL query table alongside it.
## Testing
<!-- Check what you actually ran, then paste the real command output
below. -->
- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed
Docs/examples-only change, manually verified: the dashboard JSON is
well-formed and every PromQL query references only the documented
`headroom_*` metric names from `docs/content/docs/metrics.mdx`.
### Test Output
```text
$ python3 -c "import json; d=json.load(open('examples/grafana/headroom-dashboard.json')); print('valid JSON,', len(d['panels']), 'panels, uid', d['uid'])"
valid JSON, 7 panels, uid headroom-compression
```
PromQL queries used by the panels (all against documented `headroom_*`
metrics):
```text
sum(headroom_tokens_saved_total{pool=~"$pool", hook=~"$hook"})
sum(headroom_tokens_input_total{pool=~"$pool", hook=~"$hook"})
sum(rate(headroom_requests_total{pool=~"$pool", hook=~"$hook"}[$__rate_interval]))
sum(rate(headroom_overhead_ms_sum{pool=~"$pool", hook=~"$hook"}[$__rate_interval])) / clamp_min(sum(rate(headroom_overhead_ms_count{pool=~"$pool", hook=~"$hook"}[$__rate_interval])), 1)
sum(rate(headroom_tokens_saved_total{pool=~"$pool", hook=~"$hook"}[$__rate_interval])) by (pool)
max(headroom_overhead_ms_max{pool=~"$pool", hook=~"$hook"})
min(headroom_overhead_ms_min{pool=~"$pool", hook=~"$hook"})
sum(rate(headroom_requests_total{pool=~"$pool", hook=~"$hook"}[$__rate_interval])) by (pool)
```
## Real Behavior Proof
- Environment: local checkout of the PR branch; Python 3 for JSON
validation.
- Exact command / steps: ran the JSON-validation command above (see Test
Output) — parses cleanly, reports 7 panels and uid
`headroom-compression`; then read every panel target and confirmed each
PromQL query references only metric names documented in
`docs/content/docs/metrics.mdx` (`headroom_requests_total`,
`headroom_tokens_saved_total`, `headroom_tokens_input_total`,
`headroom_overhead_ms_{sum,count,min,max}`). No histogram metrics are
referenced.
- Observed result: JSON is valid and importable via Grafana's
**Dashboards → New → Import → Upload**; no datasource UID is hard-coded,
so the importer prompts for a Prometheus datasource. Queries match the
documented metric family.
- Not tested: a full live Grafana import against a running proxy
scraping real `/metrics` was not performed in CI. Verification was
limited to JSON validity and query/metric-name correctness against the
documented metrics.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
Additive docs/examples only — no source code, tests, or runtime behavior
changed.
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
## Screenshots (if applicable)
N/A — dashboard is imported from JSON; see the PromQL and panel list
above.
## Additional Notes
<!-- Mention any N/A checklist items, tradeoffs, follow-ups, or
maintainer context. -->
Test-related checklist items are N/A: this is an additive docs/examples
change with no application code, so `pytest`/`mypy`/`ruff` and new unit
tests do not apply. The dashboard JSON was validated and its queries
checked against the documented metric names instead.
---------
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
|
||
|
|
10251b65ca |
docs: sync README + benchmarks with code (drop retired IntelligentContext/RollingWindow) (#1545)
## Description Sync the docs with the code after the live-zone realignment. The `IntelligentContextManager` (ICM), `RollingWindow`, and scoring modules were deleted in PR #350 (May 2026), but the README and benchmark docstrings still advertised them as live, and an example still imported the deleted module (broken on run). This fixes the README + benchmarks and removes the dead example. I validated the README against the code with three parallel static-analysis sub-agents (features/architecture, CLI/extras/wrap-matrix, public API/integrations). Most of the README checked out accurate; only the items below were stale/wrong. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - README: removed the `IntelligentContext` bullet and `IntelligentContext / RollingWindow` from the transforms list (both deleted in PR #350). - README: standardized `Kompress-base` -> `Kompress-v2-base` to match the HF model id `chopratejas/kompress-v2-base` and the existing badges (diagram re-aligned). - README: corrected the CodeCompressor language list to match the `CodeLanguage` enum (added TS, C, Perl). - README: softened the unanchored "6 algorithms" tagline to "content-aware compressors". - README: Cortex Code is library-mode only — there is no `headroom wrap cortex`, so the compatibility-matrix row no longer shows a wrap checkmark. - Deleted `examples/test_intelligent_context_toin_ccr.py` — it imported the deleted `IntelligentContextManager` (ImportError on run) and is unreferenced. - Removed stale `RollingWindow` mentions from benchmark docstrings/comments (`benchmarks/__init__.py`, `bench_transforms.py`, `bench_latency.py`, `scenarios/conversations.py`); the accurate PR-B1 retirement comment is kept. ## Testing - [ ] Unit tests pass (`pytest`) — N/A, docs/docstring + example deletion only - [x] Linting passes — `ruff check` clean on all changed benchmark files - [ ] Type checking passes — N/A (no type-relevant changes) - [ ] New tests added — N/A - [x] Manual testing performed — see Real Behavior Proof ### Test Output ```text $ ruff check benchmarks/__init__.py benchmarks/bench_transforms.py benchmarks/bench_latency.py benchmarks/scenarios/conversations.py All checks passed! # stale refs remaining in README/benchmarks (excluding accurate retirement notes): $ grep -rn "IntelligentContext|RollingWindow|Kompress-base" README.md benchmarks/ | grep -v retire (only benchmarks/bench_transforms.py:362 — the accurate PR-B1 retirement comment) # deleted example is unreferenced anywhere: $ grep -rn "test_intelligent_context_toin_ccr" --include=*.md --include=*.yml --include=*.py . (no hits) ``` ## Real Behavior Proof - Environment: macOS (darwin, arm64), Python 3.12 `.venv`, ruff 0.14.x, repo at branch `docs/sync-readme-with-code` off latest `main`. - Exact command / steps: (1) three parallel sub-agents grep/Read-validated README claims vs `headroom/`, `pyproject.toml`, `sdk/typescript/`; (2) directly verified each flagged mismatch (`CodeLanguage` enum, `HF_MODEL_ID`, absence of `IntelligentContext`/`RollingWindow` classes); (3) confirmed the example imports a deleted module and is unreferenced; (4) `ruff check` on changed benchmark files; (5) re-grepped README + benchmarks for any remaining stale refs. - Observed result: README and benchmark docstrings now match the code; the only surviving `RollingWindow` string is the accurate retirement comment; the broken example is removed; ruff passes; the ASCII architecture diagram still aligns after the `Kompress-v2-base` rename. - Not tested: rendering of the README on GitHub/PyPI (text-only change); the separate `docs/content/` and `wiki/` doc sets (see Additional Notes — out of scope for this PR). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective — N/A (docs/example cleanup) - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md — N/A (Release Please auto-generates from the conventional commit) ## Additional Notes **Larger related finding (NOT in this PR):** the published docs site (`docs/content/docs/*.mdx`) and the `wiki/*.md` set still document `IntelligentContextManager`, `RollingWindow`, `RollingWindowConfig`, `IntelligentContextConfig`, and `ScoringWeights` as live API — with `from headroom import RollingWindow` / `from headroom.transforms import IntelligentContextManager` code examples that would `ImportError`. It is half-migrated (a couple of `.mdx` files already note "removed in 0.9.x" while neighbors still teach it as current). This is ~15 files and the fixes require rewriting examples to the live-zone model, not just deletions — recommended as a focused follow-up PR rather than bundling it here. |
||
|
|
a639540959 |
chore: remove committed node_modules + stray/internal markdown (repo hygiene) (#1528)
## Description Repo hygiene for a public OSS project: removes committed `node_modules`, stray/internal/draft markdown, and commercial-surface references — keeping every real doc (the published docs site, the wiki guides, and all component READMEs) intact. Every file was content-audited before removal, and load-bearing files were verified against the code/CI and kept. Net: **1,695 files changed, +23 / −266,409** (the deletions are dominated by a committed `node_modules` tree). Closes # (no tracking issue) ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [x] Code refactoring (no functional changes) ## Changes Made **Removed (verified to have no code/CI dependencies):** - `examples/vercel-ai-sdk-pr/` — 1,649 committed `node_modules` files (zero example source); `node_modules/` added to `.gitignore`. - `docs/spec/` (23 draft "Living Specification" files — orphaned, `1.0.0-draft`, drifted from the code), `docs/superpowers/` (2 agent plans), `docs/proposals/` (2 internal/commercial memos). - 6 orphan `docs/*.md` (auth-modes, bedrock, claude-code-vertex-headroom, cortex-code, output-token-reduction-guide, rtk-loop-weighting). - `PR.md` (committed PR draft), `ENTERPRISE.md`, `.github/FUNDING.yml`. **Content scrubs:** - Removed unreleased "Headroom Cloud" / `api.headroom.ai` / `hr_` references from `configuration.mdx`, `wiki/configuration.md`, `wiki/typescript-sdk.md`, `sdk/typescript/README.md` (reworded to neutral, accurate phrasing). - Dropped a stale "awaiting maintainer before merge" line from `plugins/headroom-oauth2/SPEC.md`; tidied `.gitignore` comments (kept the protective `headroom-managed/` ignore rule). - Fixed the now-dangling links into removed files (README nav/`output-token-reduction` link, `scripts/README`, `wiki/vertex`). **Explicitly KEPT (load-bearing — would orphan in-code citations if removed):** - `.changelog.md` — consumed by `.github/workflows/release.yml` (read as the release-notes file). - `REALIGNMENT/`, `docs/observability.md`, `docs/rtk-architecture.md`, `wiki/plans/`, `TESTING-copilot-subscription.md` — referenced by the Rust core / Python / tests as design docs. ## Testing - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [ ] Type checking passes (`mypy`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # Docs/markdown + .gitignore only — no Python/Rust source changed, so the # behavioral test suite is unaffected. Verified the cleanup did not orphan # references or break the published docs site: $ git ls-files 'docs/content/docs/*.mdx' | wc -l # published site intact 42 $ # meta.json nav unchanged; no published page removed. $ grep -rnI "Headroom Cloud|api.headroom.ai|'hr_" $(git ls-files '*.md' '*.mdx') >>> none $ # dangling refs to removed files (excl pre-existing P0/P2 spec stubs that $ # never existed in git): none remaining. ``` ## Real Behavior Proof - Environment: macOS, local git clone of the repo (markdown/.gitignore changes only — no runtime). - Exact command / steps: 4 read-only content-audit agents classified every `.md`/`.mdx` file; each removal candidate was cross-checked against the codebase (`grep` for citations in `.rs`/`.py`/tests, workflows, and configs); only files with no dependents were removed; the tree was re-grepped after removal to confirm no new dangling references; verified the published docs site page count (`git ls-files 'docs/content/docs/*.mdx' | wc -l` = 42, unchanged). - Observed result: the 42-page published docs site and all wiki guides are untouched; no source or workflow references a removed file; `.changelog.md` (consumed by release.yml) and the code-cited design docs were detected as dependencies and kept; the committed `node_modules` tree is removed and `node_modules/` is gitignored so it can't be re-committed; zero "Headroom Cloud"/`headroom.dev` references remain. - Not tested: N/A — no executable code changed (only markdown, `.mdx`, and `.gitignore`), so the behavioral test suite is unaffected. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - This branch deletes `.github/FUNDING.yml` while PR #1526 edits it — the two will be sequenced at merge (delete wins). - A follow-up option (not in this PR): also remove the internal design docs that are currently cited by the code (`REALIGNMENT/`, `docs/observability.md`, `docs/rtk-architecture.md`, `wiki/plans/`) — that requires scrubbing ~15–20 in-code citations so nothing dangles, so it's deliberately deferred. - Untracked local working files (`benchmarks/hf_pilot/`, `tools/copilot-test/`) are intentionally left out of git (not committed). |
||
|
|
d789a7c528 |
feat(transforms): tabular + spreadsheet (.xlsx/.xls) compression (#1128)
## Description Adds a content-type-aware path for **tabular data** — CSV/TSV, markdown tables, fixed-width text, and binary `.xlsx`/`.xls` spreadsheets — by routing them through the existing, battle-tested `SmartCrusher` instead of letting them fall through to `PLAIN_TEXT → Kompress`. The pipeline already compressed tables losslessly when handed a JSON array of records. This wires up the missing front door: detect tabular text (and ingest binary spreadsheets), convert to JSON records, and reuse `SmartCrusher.crush()`. No new compression algorithm. Closes # ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Detection** (`content_detector.py`): new `ContentType.TABULAR` + `_try_detect_tabular()` for CSV/TSV, markdown tables, and fixed-width columns. Ordered after search/log (which also look "delimited") and before code, with a prose-rejection guard so it never steals `file:line:content` search output, `key: value` logs, or sentences with incidental commas. Rust backend returns `plain_text` for unknown types and the router already falls back to the Python detector, so **no Rust change**. - **Bridge** (`tabular_ingest.py`): stdlib parsers + `to_records()` + a `TabularCompressor` that parses → JSON records → `SmartCrusher` (lossless `csv-schema` first; lossy row-drop with reversible `<<ccr:HASH>>` markers stays SmartCrusher's built-in fallback). Only adopts a result when it actually saves bytes. - **Spreadsheets** (`spreadsheet_ingest.py`): `.xlsx`/`.xls` → per-sheet CSV text at the SDK boundary. Optional deps (`pip install headroom-ai[spreadsheet]`) fail loudly with an install hint, never silently degrade. - **Routing** (`content_router.py`): `CompressionStrategy.TABULAR`, `enable_tabular_compressor` flag, lazy getter, apply branch, strategy maps, Kompress fallback eligibility. - **SDK** (`compress.py`): `compress_spreadsheet(path, ...)` helper (one message per sheet). - **Packaging** (`pyproject.toml`): new `[spreadsheet]` extra; `openpyxl` added to `[dev]` so the xlsx path is exercised in CI. - **Docs/demo**: `examples/tabular_compression_demo.py` + README entry. ### Design note: lossless-only Compact, all-unique tables with no query yield ~0 savings — this is correct, not a bug. SmartCrusher returns `skip:unique_entities_no_signal` and won't drop unique rows without a duplicate/relevance signal. Real wins come from verbose/redundant tables and query-driven selection. A pressure-driven lossy row sampler was considered and intentionally not added. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_transforms_tabular.py -q collected 20 items tests/test_transforms_tabular.py .................... [100%] ============================== 20 passed in 7.15s ============================== $ ruff check headroom/transforms/tabular_ingest.py headroom/transforms/spreadsheet_ingest.py All checks passed! $ mypy headroom/transforms/tabular_ingest.py headroom/transforms/spreadsheet_ingest.py Success: no issues found in 2 source files ``` `tests/test_transforms_tabular.py` (20 tests): detection true positives + no-misroute negatives (search/log/JSON/prose), parser units (incl. fixed-width), the CSV→SmartCrusher bridge, router routing + disable flag, and `.xlsx` ingestion (skipif openpyxl missing) + error paths. `spreadsheet_ingest` 100% / `tabular_ingest` 90% line coverage. ## Real Behavior Proof - **Environment:** local checkout of `feat/tabular-compression`, Python 3.x, `pip install -e ".[dev]"`. - **Exact command / steps:** `python examples/tabular_compression_demo.py` (no API key required). - **Observed result:** ```text === Raw tabular text (ContentRouter, char-level) === compact unique CSV strat=tabular chars 1306 -> 1072 ( 17.9% saved) redundant CSV strat=tabular chars 2661 -> 1350 ( 49.3% saved) verbose markdown strat=tabular chars 2019 -> 1580 ( 21.7% saved) === Full pipeline (real tokenizer) === redundant CSV tokens 768 -> 394 ( 48.7% saved) === Binary spreadsheet (.xlsx) === 2-sheet workbook tokens 1092 -> 683 ( 37.5% saved) ``` - **Not tested:** legacy `.xls` binary path (needs optional `xlrd` + binary fixture; `# pragma: no cover`); base64-embedded `.xlsx` inside multimodal blocks (out of scope, noted as a follow-up). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes - CHANGELOG/version are intentionally untouched: this repo uses **release-please**, which bumps the version and CHANGELOG via automated `chore: release main` PRs, not per-feature PRs. - The `.xls` path is `# pragma: no cover` (legacy, needs optional `xlrd` + a binary fixture). - Follow-up (out of scope): base64-embedded `.xlsx` inside tool-result/multimodal blocks; porting tabular parsers into the Rust core for parity. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
0d4571f72f |
docs: fix broken macos-deployment.md link in launchagent example (#985)
## Description The macOS LaunchAgent example README links to `../../../docs/macos-deployment.md`, but that file does not exist — the guide lives at `wiki/macos-deployment.md`. This fixes the broken link (path and text) so "complete documentation" resolves. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `examples/deployment/macos-launchagent/README.md`: link target `../../../docs/macos-deployment.md` → `../../../wiki/macos-deployment.md`, and the link text `docs/macos-deployment.md` → `wiki/macos-deployment.md`. ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # The old target does not exist; the real guide is under wiki/: $ git ls-files '*macos-deployment.md' wiki/macos-deployment.md $ ls docs/content/docs | grep -i macos # nothing — no docs/macos-deployment.md $ test -f examples/deployment/macos-launchagent/../../../wiki/macos-deployment.md && echo "new link resolves" new link resolves # This was the only stale reference to docs/macos-deployment.md in the repo: $ grep -rn 'docs/macos-deployment' --include='*.md' --include='*.mdx' . (only the line fixed by this PR, now pointing at wiki/) ``` ## Real Behavior Proof - Environment: local clone at `origin/main`; documentation-only change. - Exact command / steps: ran a relative-link checker across all Markdown/MDX, which flagged `examples/deployment/macos-launchagent/README.md:168` as the only broken internal link; confirmed the guide is at `wiki/macos-deployment.md`; repointed the link there. - Observed result: the new relative path `../../../wiki/macos-deployment.md` resolves to the existing macOS Deployment Guide (which itself documents this exact LaunchAgent setup). - Not tested: N/A — single-line Markdown link fix; no code, build, or runtime behavior involved. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes Documentation-only change, so `pytest`/`ruff`/`mypy` over the `headroom` package are N/A — the diff contains no Python source. The target guide (`wiki/macos-deployment.md`) covers the same LaunchAgent deployment this example sets up, so it is the correct destination for "complete documentation". |
||
|
|
42b1cd24de |
docs: fix env var discrepancies across markdown files
Audit all .md files against codebase; fix wrong names, remove phantom
variables, and correct outdated values:
- HEADROOM_PROXY_PORT → HEADROOM_PORT (proxy.py envvar="HEADROOM_PORT")
- HEADROOM_BIND → HEADROOM_HOST + HEADROOM_PORT (RUST_DEV.md)
- HEADROOM_LEARN_{CLAUDE,CODEX,GEMINI}_ENABLED → HEADROOM_LEARN_CLI
(only HEADROOM_LEARN_CLI exists in learn/analyzer.py)
- HEADROOM_TRACING_ENABLED → HEADROOM_LANGFUSE_ENABLED=1 with correct
LANGFUSE_PUBLIC_KEY/SECRET_KEY vars (tracing.py)
- HEADROOM_LOG_LEVEL/LOG_FORMAT → --log-level CLI flag / RUST_LOG
(no HEADROOM_LOG_LEVEL var exists in code)
- HEADROOM_LOG_LEVEL/HEADROOM_STORE_URL/HEADROOM_DEFAULT_MODE rows
removed from wiki/configuration.md (all phantom)
- HEADROOM_SUMMARY_{ENABLED,THRESHOLD,RATIO} noted as not yet
implemented (no code exists)
- HEADROOM_DB_URL/HEADROOM_CACHE_BACKEND → explanatory notes pointing
to HEADROOM_WORKSPACE_DIR (no external DB support in code)
- HEADROOM_DB_PATH/HEADROOM_CACHE_PATH table rows replaced with actual
HEADROOM_WORKSPACE_DIR/CONFIG_DIR (paths.py)
|
||
|
|
20dc1f28f3 |
fix(proxy): Strands MCP bundle + backend path fixes + Codex fail-closed protection
Three logically-related sets of proxy changes ship in this branch:
1. Strands integration on the Bedrock path (HeadroomBundle + 4 OpenAI
handler fixes + LiteLLM cache stats + dep pin)
2. /stats MCP aggregation (cross-process events log → proxy summary)
3. Codex compression-failure fail-closed (WS + HTTP /v1/responses)
== 1. Strands integration on the Bedrock path ==
* HeadroomBundle (headroom/integrations/strands/bundle.py): single-helper
MCP wiring for a Strands Agent — Headroom MCP server (headroom_compress
/ headroom_retrieve / headroom_stats) plus optional Serena MCP and
optional in-process compression hook. Constructor builds unstarted
MCPClient instances per server; Strands' Agent owns the subprocess
lifecycle. Default config: MCP enabled, Serena enabled, hook OFF
(proxy is the single source of truth for compression). User-side
integration is two lines in any Strands app.
* headroom/proxy/handlers/openai.py — backend path now:
- calls PrefixCacheTracker.update_from_response (was direct-OpenAI only)
- intercepts CCR headroom_retrieve tool_calls server-side, mirroring
the Anthropic handler pattern; NO silent fallback, re-raises on
CCR errors (per feedback_no_silent_fallbacks)
- works for both non-streaming and streaming paths
* headroom/proxy/handlers/streaming.py: _stream_openai_via_backend now
accepts prefix_tracker + optimized_messages, parses cache stats from
the SSE final-usage frame (cache_creation_input_tokens added to the
state machine), records CCR retrieve feedback via a new
_record_ccr_feedback_from_openai_sse helper. Streaming CCR intercept
is intentionally out of scope (mirrors Anthropic streaming behaviour).
* headroom/backends/litellm.py: send_openai_message response usage block
now carries cache_read_input_tokens / cache_creation_input_tokens
(Anthropic/Bedrock dialect) and prompt_tokens_details.cached_tokens
(OpenAI dialect). Backwards-compatible — cold-start callers see the
same 3-key shape; cache keys appear only when the underlying provider
returns them. Pinned by test_no_cache_fields_means_no_cache_keys.
* headroom/proxy/auth_mode.py: ("strands-agents/", "strands") added to
CLIENT_UA_MAP. Production callers should also set X-Client: strands
since the default openai-python UA carries no Strands signal.
* pyproject.toml: huggingface-hub>=1.5.0,<2.0 pinned in [ml] so a sibling
install (e.g. strands-agents) can't drag the version below the floor
transformers 5.x requires (otherwise Kompress silently goes
"unavailable").
== 2. /stats MCP aggregation ==
* headroom/proxy/cost.py: _aggregate_mcp_events() reads the cross-process
shared events file the Headroom MCP server already writes to and
surfaces summary.mcp with three new keys:
- compressions (count of headroom_compress invocations)
- tokens_removed (sum of input - output across those)
- retrievals (count of headroom_retrieve — the load-bearing
over-compression alarm; if it grows linearly
with turn count, lossy compressors are
dropping info the model actually needs)
Defensive on every axis — missing MCP SDK, missing file, malformed
events, read errors — never blocks /stats.
* examples/strands_bundle_demo.py: stats panel prints the new fields so
the demo shows the full proxy-HTTP + MCP-tool story in one view.
== 3. Codex compression-failure fail-closed protection ==
Reported by Camille (2026-05-21): Codex threads were locking with
"ran out of room in the model's context window" after Headroom's
compression timed out on an oversized response.create frame and
forwarded the original ~1.7 MB frame to the upstream, which then
rejected it. Codex's auto-compact heuristic gates on the upstream-
reported total_usage_tokens (which Headroom had been shrinking on
earlier turns), so its compaction never fired and the thread locked.
Validated against open Codex issues (CLI + Desktop share codex-rs/core):
* #16068 — confirms compaction gates on total_usage_tokens,
estimated_token_count is computed but only logged
* #19806 — confirms image token estimator unbounded, contributes to
the same ContextManager.get_total_token_usage → auto-compaction chain
* headroom/proxy/helpers.py: decide_compression_failure_action() with a
unit-tested decision matrix:
- asyncio.TimeoutError → refuse, always
- non-timeout failure + frame > 256 KiB (configurable) → refuse
- non-timeout failure + small frame → forward (legacy)
Operator escape hatches:
- HEADROOM_WS_FAIL_OPEN_ON_COMPRESSION_FAILURE=1 restores legacy
- HEADROOM_WS_COMPRESSION_FAIL_THRESHOLD_BYTES tunes the threshold
* headroom/proxy/handlers/openai.py (WS /v1/responses): consults the
helper after compression failure. On refuse: close client websocket
code 1009 with "headroom: compression <reason> — please compact
context and retry" reason; set termination_cause for the outer
lifecycle finally; return.
* headroom/proxy/handlers/openai.py (HTTP /v1/responses): same helper.
On refuse: raise HTTPException(413) with a structured error body so
FastAPI's HTTPException handler emits a clean 413. The existing
`except HTTPException: raise` guard in this handler already ensures
the 413 propagates without being swallowed by the 502 catch-all.
Anthropic /v1/messages NOT changed in this branch: no equivalent bug
report on Anthropic-protocol clients, Claude Code (Anthropic-owned)
handles context overflow via its own cache_control/ephemeral
primitives, and Cursor/Aider don't maintain the local-Y estimate the
Codex bug requires. Deferred until a real report lands; the patch is
a one-liner reusing the same helper.
== Tests + verification ==
* tests/test_backends/test_litellm_cache_stats.py — 3 tests pinning
cache-stat surfacing across Anthropic/OpenAI dialects + backwards-
compat for no-cache responses.
* tests/test_proxy/test_openai_backend_path.py — 5 tests (Bedrock cache
fields, OpenAI fallback shape, CCR intercept with provider="openai",
CCR re-raise on exception, streaming signature contract).
* tests/test_proxy/test_mcp_stats_aggregation.py — 5 tests pinning the
aggregator across compress+retrieve mixes, empty events, unknown event
types, missing token fields, and read failures.
* tests/test_proxy/test_compression_failure_action.py — 12 tests pinning
the fail-closed decision matrix (timeout always refuses, small
transient passes through, oversize refuses, env override variants,
custom threshold, invalid threshold falls back, 0/negative ignored).
* examples/strands_bedrock_demo.py — model_id bumped from deprecated
Claude 3 Haiku to Sonnet 4.5 (the deprecated model now errors on
account access).
* examples/strands_via_proxy_demo.py — proxy + Bedrock cache + streaming
smoke test.
* examples/strands_mcp_dispatch_test.py — pure MCP round-trip probe.
* examples/strands_bundle_demo.py — full Strands + HeadroomBundle E2E
demo (this is the shape a real Strands user copies into their app).
Full pytest: 5327 passed, 178 skipped. The previously-failing
test_core_operations.py::TestAddBatch::test_add_batch_basic passes now
that the huggingface-hub pin in pyproject.toml unblocks transformers
imports.
E2E verified live against AWS Bedrock (Sonnet 4.5):
* cache_write=10,438 on turn A → cache_read=10,438 on turn B
* streaming SSE final usage frame carries cache_read_input_tokens
* 78.7% reduction on a 50 KB JSON tool_result via SmartCrusher (
dispatched per-content-type by ContentRouter)
* Strands Agent + HeadroomBundle: model autonomously called
headroom_compress + headroom_retrieve via MCP; CompressionStore
round-trip succeeded; final answer correct.
|
||
|
|
14d00c6c74 |
Token-level cache hit rate, compression-vs-cache tracking, dashboard SQL, security plan
Cache stats: - hit_rate is now token-level (cache_read / total_input) not request-level - Track uncached_input_tokens per provider in metrics - Preserve request_hit_rate as secondary metric Compression-vs-cache: - Detect when compression busts the prefix cache (expected_cached - actual_read) - Two simple session-level numbers: tokens_saved vs cache_bust_tokens - Log CACHE-BUST per request, aggregate in /stats and telemetry beacon - Single new column in proxy_telemetry_v2: cache_bust_tokens Dashboard infra: - SQL for dashboard_summary table + pg_cron hourly refresh - Hourly + daily aggregation from proxy_telemetry_v2 - Upgrade scripts for adding hourly_stats and cache bust columns |
||
|
|
de1a7e2ddc |
Add notebook for langchain-ai/how_to_fix_your_context PR
07-context-compression.ipynb: Context Compression technique using Headroom. Same RAG setup as notebooks 01-06 (Lilian Weng blog posts, Claude Sonnet, OpenAI embeddings). Replaces GPT-4o-mini pruning/summarization with local Headroom compress() — zero extra LLM calls, zero cost. Ready to fork and PR to https://github.com/langchain-ai/how_to_fix_your_context |
||
|
|
39af65a259 |
Use realistic verbose RAG chunks in demo — triggers Kompress within-item
Previous chunks were too dense (400 chars, no filler). Real blog post retrieval returns verbose explanatory text (~1000 chars per chunk). Kompress now compresses 31-47% within each item. Demo results: 4424 → 2756 tokens (38% savings), all 12 items kept, 6/6 key concepts preserved, zero extra LLM calls. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
f121138c8a |
Fix context-blind compression: pass user query to SmartCrusher relevance scorer
ROOT CAUSE: compress() did not extract the user's question from messages. The pipeline received empty context, so SmartCrusher selected items by statistics only (position, anomaly, boundary) — keeping irrelevant chunks and dropping relevant ones. FIX: _extract_user_query() in compress.py finds the most recent user message and passes it as `context` kwarg through the pipeline. SmartCrusher's RelevanceScorer now receives the actual query and scores items by relevance. Before: 12 RAG chunks → kept hallucination/video (0/6 key terms) After: 12 RAG chunks → kept reward hacking content (3/4 key terms) Also adds: - examples/context_compression_demo.py — real compression demo for OSS PR - examples/test_ccr.py — content preservation verification - OSS_PR_STRATEGY.md — PR target list for LangChain ecosystem Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
95e9b39b6d |
feat: Add AWS Strands Agents SDK integration
## Description Add Headroom integration with AWS Strands Agents SDK, enabling automatic context optimization and tool output compression for Strands-based agents. Fixes #14 ## Type of Change - [x] New feature (non-breaking change that adds functionality) - [x] Documentation update ## Changes Made ### Core Integration (`headroom/integrations/strands/`) - **HeadroomHookProvider** - Implements Strands `HookProvider` interface for automatic tool output compression via `AfterToolCallEvent`. Compresses verbose tool outputs before they enter conversation context. - **HeadroomStrandsModel** - Model wrapper that extends Strands `Model` base class for message-level optimization. Implements all required abstract methods: `stream()`, `get_config()`, `update_config()`, `structured_output()`. - **Provider auto-detection** - Automatically detects appropriate Headroom provider (Anthropic, OpenAI, Google) based on wrapped Strands model type. - **`strands-agents` as optional dependency** - Install with `pip install headroom-ai[strands]` ### Testing (`tests/integrations/test_strands/`) - **Real integration tests (25 tests)** - Use actual AWS Bedrock API calls with Claude 3 Haiku. Skip automatically when credentials unavailable. - **Unit tests (57 tests)** - Mock-based tests for internal logic, edge cases, and error handling. No credentials required. ### Demo (`examples/strands_bedrock_demo.py`) - Interactive demo showcasing both integration patterns - Visual before/after compression comparison with token savings - 4 verbose tools (search, logs, database, metrics) demonstrating real savings - Supports `--hook` and `--model` flags for individual demos ## Testing All tests verified: - [x] Unit tests pass (57 tests) - [x] Integration tests pass (25 tests with real Bedrock API) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom/integrations/strands/`) - [x] Formatting passes (`ruff format --check`) - [x] Demo runs successfully with ~50% token savings ## Test Output ``` $ pytest tests/integrations/test_strands/ -v =================== 82 passed in 90.09s =================== $ ruff check headroom/integrations/strands/ --ignore E402 All checks passed! $ mypy headroom/integrations/strands/ --ignore-missing-imports Success: no issues found ``` ## Demo Results ``` ╭────────────────────────────────────────────────────────────╮ │ HeadroomHookProvider Results │ │────────────────────────────────────────────────────────────│ │ Tokens BEFORE compression: 51,961 │ │ Tokens AFTER compression: 25,658 │ │ Tokens SAVED: 26,303 (50.6%) │ ╰────────────────────────────────────────────────────────────╯ ``` |
||
|
|
83c0334ccd |
docs: update documentation for IntelligentContext TOIN + CCR integration
Updates across multiple docs to reflect the new message-level compression with TOIN + CCR integration: - docs/ccr.md: Add CCR-enabled components table, message-level CCR section - docs/ARCHITECTURE.md: Expand Transform 6 with TOIN + CCR integration details - docs/configuration.md: Add CCR integration config and marker format - docs/proxy.md: Add CCR integration note for context management - docs/README.md: Update to reference IntelligentContextManager as default Also adds examples/test_intelligent_context_toin_ccr.py for scale testing the TOIN + CCR integration with real API calls. |
||
|
|
da74341858 |
Add hierarchical memory system with graph + vector storage
Implement comprehensive memory system supporting: - Local backend (SQLite + FTS5 + HNSW) for zero-dependency operation - Mem0 backends (Neo4j + Qdrant) for production graph memory - DirectMem0Adapter for optimized pre-extracted data (bypasses LLM) - Memory extraction with facts, entities, and relationships - Proxy integration with --memory flag for automatic memory injection Key components: - headroom/memory/backends/: LocalBackend, Mem0Backend, DirectMem0Adapter - headroom/memory/system.py: MemorySystem with tool-based interface - headroom/memory/extraction.py: Entity and relationship extraction - headroom/proxy/memory_handler.py: Proxy integration layer - headroom/prediction/feature_extractor.py: Content analysis features Testing: - 217 new memory system tests covering all backends - LoCoMo evaluation framework for memory quality assessment - Integration tests for proxy memory functionality Also removes deprecated example files in favor of focused test coverage. |
||
|
|
bd2d447c26 |
Add quality retention eval and fix linting for Python 3.12
- Add quality_retention_eval.py for needle-in-haystack testing to verify intelligent compression retains critical information (100% retention achieved) - Add intelligent_context_integration_test.py for comprehensive pipeline testing - Add test_progressive_summarizer.py with 36 tests for ProgressiveSummarizer - Add HeadroomConfig parameter to HeadroomClient for direct config injection - Update pipeline.py with IntelligentContextManager wiring and logging - Fix all ruff linting issues and format for Python 3.12 compatibility - Add comprehensive_eval.py benchmark for multi-scenario evaluation - Add real_data_demo.py for production-scale volume testing - Add reasoning agent test examples (groq, debug) |
||
|
|
18e01a0480 |
Merge pull request #6 from smartwatermelon/claude/docs-macos-deployment-20260119
Add macOS LaunchAgent deployment guide and templates |
||
|
|
063b6e85f4 |
fix(deployment): correct port placeholder in LaunchAgent plist template
The plist template was using ${HEADROOM_PROXY_PORT} in ProgramArguments,
but LaunchAgent doesn't expand environment variables in that context.
Changed to use __PORT__ placeholder which install.sh replaces via sed.
This fixes the "invalid int value" error when starting the proxy service.
AI review: Clean (via pre-commit hook)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
||
|
|
3ac26207b8 |
docs(deployment): add macOS LaunchAgent deployment guide and templates
Add comprehensive macOS deployment support for running headroom proxy as a persistent background service using LaunchAgent. This enables automatic startup, crash recovery, and proper lifecycle management for local development environments. Files added: - examples/deployment/macos-launchagent/com.headroom.proxy.plist.template - examples/deployment/macos-launchagent/install.sh (shellcheck-clean) - examples/deployment/macos-launchagent/uninstall.sh (shellcheck-clean) - examples/deployment/macos-launchagent/shell-integration.sh (bash + zsh) - examples/deployment/macos-launchagent/README.md - docs/macos-deployment.md Key features: - Configurable port via HEADROOM_PROXY_PORT environment variable (default: 8787) - Automated installation and uninstallation scripts - Shell integration supporting both bash and zsh - Comprehensive documentation with troubleshooting guide - All shell scripts are shellcheck-clean (zero errors, warnings, or info messages) Files modified: - .gitignore: Added CLAUDE.md to prevent committing local config - docs/README.md: Added Deployment & Operations section with navigation entry AI review: Pending (will be run by pre-commit hook) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|
|
39a55b4810 |
Fix HeadroomAgnoModel to optimize tool outputs at invoke level
Previously, HeadroomAgnoModel called wrapped_model.response() which ran the tool execution loop internally. This meant tool outputs (often 60k+ chars) were never optimized - only the initial messages were compressed. The fix delegates response() to the inherited Model.response(), which calls self.invoke() for each API call. Our invoke() override optimizes messages before delegating to wrapped_model.invoke(), ensuring tool outputs are compressed on every API request. Results from multi_tool_agent_test.py with Claude Sonnet: - Tokens before optimization: 25,713 - Tokens after optimization: 6,100 - Tokens saved: 19,613 (76.3%) - Both baseline and optimized found all critical information Also adds: - multi_tool_agent_test.py: Real function calling test with 4 tools - multi_tool_compression_test.py: Direct compression test - README update with multi-tool agent test results |
||
|
|
8766d83f68 |
Rewrite README with verified before/after examples
- Replace marketing claims with actual API test results - Add needle-in-haystack test: critical error at position 67/100 - Show real JSON compression: 100 entries → 6 entries (93.9% reduction) - Verified with Claude Sonnet: 87.6% fewer tokens, 4/4 correct answers - Add example scripts for reproducing the tests |
||
|
|
9c9bb30ded |
Add persistent memory system with zero-latency inline extraction
Features: - with_fast_memory(): Zero-latency inline extraction (Letta-style) - Memory extracted as part of LLM response, no extra API calls - Semantic retrieval with local embeddings (sub-50ms) - with_memory(): Background extraction for non-blocking memory - SQLite + FTS5 storage with vector similarity search - Multi-user isolation by user_id Memory enables temporal compression - extract key facts instead of carrying full conversation history (4000 tokens → 50 tokens). Includes: - Comprehensive test suite (71 new tests) - Documentation (docs/memory.md) - Benchmark examples comparing approaches - E2E test with LLM-as-judge evaluation |
||
|
|
d724f14022 |
v0.2.2: Add CCR Response Handler, Context Tracker, and restructure docs
Features: - CCR Response Handler: Automatically intercepts and handles headroom_retrieve tool calls - CCR Context Tracker: Multi-turn awareness with proactive expansion of relevant compressed content - New CCR demo script showing before/after flow Documentation: - Restructured README from 885 lines to 190 lines for better DevEx - Split detailed docs into focused guides: ccr.md, sdk.md, configuration.md, text-compression.md, llmlingua.md, metrics.md, errors.md - Updated docs/README.md index with all new documentation Tests: - Added comprehensive tests for Response Handler (32 tests) - Added comprehensive tests for Context Tracker (32 tests) - All 977 tests passing |
||
|
|
e4a41faa33 |
Fix all ruff lint and format errors for CI
- Fix E402: Move module-level imports to top of file - Fix F401: Add noqa for availability check imports - Fix F402: Rename loop variables shadowing imports - Fix E722: Replace bare except with except Exception - Fix B904: Add exception chaining (from e) - Fix F811: Remove duplicate imports - Fix B027: Add noqa for empty close() method - Fix E741: Rename ambiguous variable l -> label - Fix I001: Import sorting issues - Apply ruff format to all 106 files All 902 tests pass. |
||
|
|
90d3aea44c |
Publish headroom-ai v0.2.0 to PyPI with DevEx fixes
- Renamed package from 'headroom' to 'headroom-ai' (PyPI name conflict)
- Fixed numpy/jinja2 imports to be lazy (core install no longer crashes)
- Fixed SQLite default path (now uses temp directory)
- Fixed f-string {tool} crash in proxy server
- Updated README with correct package name and examples
- Added quickstart and troubleshooting docs
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
||
|
|
175746cc26 |
Prepare for OSS release v0.2.0
This commit prepares Headroom for public open source release with comprehensive documentation, licensing, and community infrastructure. License & Legal: - Add Apache 2.0 LICENSE file - Add NOTICE file with third-party attributions - Add SECURITY.md for vulnerability reporting Community: - Add CONTRIBUTING.md with contribution guidelines - Add CODE_OF_CONDUCT.md (Contributor Covenant) - Add GitHub issue templates (bug report, feature request) - Add pull request template Documentation: - Update README.md with compelling value proposition - Add docs/getting-started.md - Add docs/proxy.md for proxy server documentation - Add docs/transforms.md for transform reference - Add docs/api.md for API reference - Add examples/README.md Package Infrastructure: - Add headroom/py.typed for PEP 561 compliance - Add headroom/cli.py for CLI entry point - Add .github/workflows/ci.yml for CI pipeline - Add .github/workflows/publish.yml for PyPI publishing - Update pyproject.toml with proper metadata New Features: - Add multi-provider support (Google, Cohere, LiteLLM, OpenAI-compatible) - Add universal tokenizer registry with multiple backends - Add model registry with pricing and context limits - Add production proxy server with caching and rate limiting Code Quality: - Fix 83 lint issues via ruff auto-fix - Fix version consistency (benchmarks 0.1.0 → 0.2.0) - Add skip decorators for optional dependency tests |
||
|
|
9c7d4512d6 |
Initial commit: Headroom SDK - LLM context optimization toolkit
A comprehensive SDK for optimizing LLM context windows, reducing token usage while preserving critical information for AI agents. Core Features: - SmartCrusher: Statistical compression of tool outputs (70-85% reduction) - CacheAligner: Prefix optimization for prompt cache hits - RollingWindow: Intelligent context window management - BM25/Hybrid relevance scoring for smart item selection Integrations: - OpenAI and Anthropic provider support - LangChain integration (ChatModel, Callbacks, Runnable) - MCP (Model Context Protocol) integration for tool compression Test Coverage: - 372 tests passing across all modules - 35 performance benchmarks - Real-world agent evaluations with 88% token savings Key Components: - headroom/transforms/: Core compression transforms - headroom/providers/: OpenAI and Anthropic support - headroom/integrations/: LangChain and MCP integrations - headroom/relevance/: BM25 and hybrid scoring - headroom/pricing/: Model pricing registry - benchmarks/: Performance benchmark suite - examples/: Usage examples and demos |