Commit Graph
240 Commits
Author SHA1 Message Date
3aafdf2dcc fix(router): keep source code out of the prose (Kompress) fallbacks (#4787)
## Description

When the code compressor declined a block of source code, the router
passed the block to Kompress. Kompress is a prose model. It drops
low-salience words, and in code those words are keywords such as `for`,
`in`, `if` and `import`. The result no longer parses. #4214 shows a
fenced Python script that comes back with `for` and `in` removed.

This PR stops that in the Python router and in the Rust live-zone
dispatcher. When the code compressor declines, the code is forwarded
unchanged.

Closes #4214

Credit: the Python change is oxura's work from
https://github.com/oxura/headroom/tree/fix/fenced-code-routing-4214-20261018
(277ca1f). Their commit is the first commit here, rebased onto `main`,
and oxura is its author. The test, docs and Rust commits were added on
top. oxura asked us to open this PR from our fork
([comment](https://github.com/headroomlabs-ai/headroom/issues/4214#issuecomment-6797572655)),
and said they have not run this branch, so the model and native results
below are ours, not theirs.

## Behaviour change to review

With `enable_code_aware=False` (the default), source code now stays
unchanged. Before this PR, it was sent to Kompress and lost keywords.
With `enable_code_aware=True`, code that the code compressor declines
also stays unchanged. Before, Kompress ran on it.
A code block the code compressor does compress is unaffected. Prose,
logs, JSON and tables are unaffected.

Side effect: a code block that Kompress used to shrink (and break) now
keeps its full size. This is the intended trade, but it means fewer
tokens saved on code when `enable_code_aware=False`.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update
- [ ] New feature
- [ ] Breaking change (the default-config behaviour change above is
intended, and it only stops damage to code)

## Changes Made

Five commits:

- `596764c3` `fix(router): keep source code out of prose fallbacks`.
Author: oxura. Removes the CODE_AWARE to Kompress fallbacks in
`headroom/transforms/content_router.py`. This is 277ca1f rebased onto
`main`, and it differs from 277ca1f in two ways only: the two test-file
conflicts are resolved, and its `wiki/proxy.md` hunk is dropped, because
`AGENTS.md` says not to add content under `wiki/`.
- `7705a4e6` `test(router): repin the code-to-Kompress tests and add the
#4214 fence regression`. Author: Tao Hansen, with a `Co-authored-by:
oxura` trailer. Five tests asserted the old fallback. They are rewritten
to assert the new behaviour (code unchanged, Kompress never sees it)
instead of being deleted. Adds a regression test for the script from
#4214, with `enable_code_aware` both off and on. Updates
`docs/content/docs/code-compression.mdx` and
`docs/content/docs/how-compression-works.mdx`, which said that detected
code goes to Kompress.
- `6631668a` `fix(live-zone): leave source code unchanged when the code
compressor declines`. Author: Tao Hansen. Same fix in
`crates/headroom-core/src/transforms/live_zone.rs`: a declined
`SourceCode` block is now a NoOp, not a Kompress input. Tests:
`tests/live_zone_source_code_decline.rs` (single test, needs no model),
and new cases in `tests/live_zone_dispatch.rs`.
- `d8776283` `fix(router): turn off the code compressor's internal
Kompress fallback`. Author: Tao Hansen. Follows review: the router's
code compressor was still built with
`fallback_to_kompress=self.config.enable_kompress` (#4719), so unknown
languages, Perl and AST failures could build a `KompressCompressor`
inside `CodeAwareCompressor`. The router now always sets
`fallback_to_kompress=False`. The #4719 test that pinned `True`
(`test_enabled_kompress_keeps_code_aware_fallback`) is repinned as
`test_enabled_kompress_still_turns_off_code_aware_fallback`, and the
unknown-language, Perl (by language hint, since `detect_language()`
reports Perl-dominant code as unknown) and AST-failure paths are tested
with spies on `is_kompress_available` and `KompressCompressor`, with
`enable_kompress` both off and on.
`docs/content/docs/code-compression.mdx` notes that the router always
turns the option off.
- `c1f8464e` `fix(router): keep passthrough code from external
compressors and stale caches`. Author: Tao Hansen. Follows review:
`_try_external_compressor` skips `PASSTHROUGH` blocks (code with
`prefer_code_aware_for_code=False`, whole or as a mixed section), which
would otherwise be offered to a prose-only external compressor as
`text/plain`. The disk cache namespace now also hashes
`_ROUTING_POLICY_REVISION`, so a same-version restart does not replay
code that Kompress damaged before this change. Tests: two
external-dispatch cases with a `text/plain` external, and one namespace
test. All three fail on the previous head.

I did not touch `CHANGELOG.md`.

## Testing

- [x] Unit tests pass (`pytest`): see below, the failures are
environment failures that also fail on `main`
- [x] Linting passes (`ruff check .`, `ruff format --check .`)
- [x] Type checking passes (`mypy
headroom/transforms/content_router.py`)
- [x] New tests added for new behaviour
- [x] Manual testing performed (real model replay, below)

### Test Output

macOS, Python 3.14. Head is now `c1f8464e`.

```text
After the second review fix (c1f8464e): router, code-aware, code-compressor, cache, external-dispatch and compressor-selection tests (107 files): 2106 passed, 13 skipped. ruff clean.
After the rebase onto main c05fbddd and the review fix (d8776283):
  router, code-aware and code-compressor tests (37 files): 743 passed, 2 skipped
  tests/test_code_aware_kompress_fallback.py: 8 passed; with the router change reverted, 4 fail
    (the enable_kompress=True cases: config pin, unknown language, Perl, AST failure)
  ruff check / ruff format --check headroom tests: clean
  cargo test -p headroom-core: 1151 passed, 0 failed
Earlier run on 9df60c7c (before the rebase):
router tests (28 files incl. both changed files): 557 passed, 2 skipped, 1 failed:
  test_content_router_ml_stage_deadline.py::test_non_ml_prework_does_not_consume_the_ml_budget,
  a timing test the branch does not touch; it passes 3 of 3 times when rerun alone
ruff check .:           All checks passed!
ruff format --check headroom tests: 1714 files already formatted
mypy headroom/transforms/content_router.py: Success: no issues found in 1 source file
cargo test -p headroom-core:   22 test binaries, 1151 passed, 0 failed
cargo fmt --check / cargo clippy -p headroom-core --tests -D warnings: clean
pytest tests (full):     31 failed, 19208 passed, 513 skipped
```

Rust bug-fix tests, fail before and pass after (the router-test commit's
source with the new Rust tests, then the live-zone commit; measured
before the rebase as `9df60c7c`, and the Rust code is unchanged since):

```text
before: declined_source_code_never_starts_kompress ... FAILED (Kompress init ran once, expected 0; rerun on 7705a4e6)
        source_code_declined_by_code_compressor_is_never_kompressed ... FAILED (model loaded: Go source was Kompressed)
after:  all 11 live_zone_dispatch tests ok; live_zone_source_code_decline ok
```

The dispatch fail-before was measured on the earlier, code-identical
branch (only documentation files differ). The dispatch test only fails
before the fix when the Kompress model is loaded (`HF_HOME` with the
model, `ORT_DYLIB_PATH` set). The single-test binary fails before the
fix on any host.

Python regression: with the new tests run against `origin/main`'s router
(no fix), 9 test cases fail (the #4214 fence test in both configs, the
repinned tests, and the fenced-code keyword test). With the fix they
pass.

### Failures in the full `pytest tests` run

The full run on the branch had 31 failures, none in router or
compression code. I compared them with pristine `origin/main`
(200c838c):

- 11 `tests/test_proxy_uds.py` tests fail on `main` too. The socket path
is too long under this checkout directory (`OSError`). They fail the
same way in isolation on `main`.
- 17 other failures are in `test_proxy_health.py`,
`test_read_maturation_handler_nobust.py`, `test_stateless_no_disk.py`,
`test_memory_bridge.py`, `test_cli/*`, `test_image_compression*.py`,
`test_offline_egress_chokepoint.py` and `test_proxy_scalability.py`. A
full run on `main` fails 29 tests, and the set differs from run to run
(the two sets overlap by 17 of 31). I re-ran the 14 failures that were
not in the `main` list in isolation on the branch: 12 pass, the other 2
are the UDS tests above.
- Known flaky test:
`tests/test_content_router_ml_stage_deadline.py::test_saturated_remote_calls_fail_open_without_queueing_their_payloads`
failed once in a loaded run and passes on rerun. File repeated 3 times
on `596764c3` and on `origin/main`: 16 passed each time. It is timing
based and not caused by this change. It passed in the 558-test router
run above.

## Real Behavior Proof

- Environment: macOS arm64, Python 3.14 in the checkout's own `.venv`,
real `kompress-v2-base` int8 ONNX model from an offline copy of the
public HF cache (`HF_HOME`, `HF_HUB_OFFLINE=1`), Rust `ort` via the
venv's onnxruntime.
- Fence: a 1324-character valid Python script. The issue quotes only
four of its lines, so the rest is a reconstruction in the same style,
not the reporter's bytes.
- Exact command / steps: `ContentRouter.compress` on the fence followed
by about 40 lines of prose, for four configs (default;
`enable_code_aware=True`; `prefer_code_aware_for_code=False`;
`enable_code_aware=True` with lossless-then-lossy). Also
`headroom.compress` on a full Anthropic message list with the fence in a
`tool_result`.
- Observed result: Before (v0.44.0): in the default and code-aware
configs the fence comes back as 1144 characters and does not parse
(`import json, collections` becomes `json, collections`; `[s for s in
...]` becomes `[s s in ...]`). After (`9df60c7c`): in all four configs
and in `headroom.compress`, the fence is byte-identical, 1324
characters, and parses. Prose control: the same router on prose still
compresses. A prose-only block went from 5709 to 5357 characters
(strategy TEXT). A prose / fence / prose block went from 12760 to 12056
characters (strategy MIXED), and the fence in it is byte-identical and
parses.
- Not tested: the reporter's exact 1324-character fence (only four lines
are public, so the tests use the reconstructed fence described above);
the Rust live-zone path end to end through the proxy with a real request
(covered by `cargo test` only); platforms other than macOS; the full
proxy on a live deployment.

## Other places where code can still reach a prose model

I checked every path from code to a prose or ML compressor, in Python
and Rust.

| Path | State |
| --- | --- |
| Python `CODE_AWARE` branch, code compressor declines or is disabled |
Fixed here: the block stays unchanged |
| Python post-strategy fallback after "no savings" (`content_router.py`,
`fallback_eligible_strategy`) | Never applied to `CODE_AWARE`. It only
applies to `SMART_CRUSHER` and `CONFIG` (JSON and config files), so
unchanged |
| Python lossless-then-lossy (opt-in) after the lossless fold |
`CODE_AWARE`, `DIFF` and `PASSTHROUGH` are excluded, so unchanged |
| Python lossless-then-lossy layer for `SEARCH`, `LOG`, `HTML` (opt-in)
| Prose-like output, not code. Unchanged |
| Python relevance split for `LOG` and `SEARCH` | Not code. Unchanged |
| Python mixed content with fenced code (`_compress_mixed`) | Each fence
goes through the `CODE_AWARE` branch, so it is fixed by this change.
Covered by the new fence tests |
| Python `prefer_code_aware_for_code=False` | Already PASSTHROUGH before
this PR |
| Python `_experimental_compress_read`
(`HEADROOM_EXPERIMENTAL_READ_KEEP_RATIO`) | Deliberate opt-in experiment
that runs Kompress on file reads, which can be code. Off by default.
Left as is |
| Python size gate and deadline fallbacks in `_try_ml_compressor` | Run
only after a caller chose Kompress, so code no longer reaches them |
| Python `CodeAwareCompressor` internal fallback
(`fallback_to_kompress`: unknown language, Perl, no tree-sitter, AST
failure) | Fixed in commit 4: the router always builds it with
`fallback_to_kompress=False`. Covered by Kompress-spy tests |
| Python external compressor dispatch for `PASSTHROUGH` code | Fixed in
commit 5: never offered to an external compressor |
| Disk result cache made before this change | Fixed in commit 5: a new
namespace through `_ROUTING_POLICY_REVISION` |
| Rust `live_zone.rs` `SourceCode` arm | Fixed here (commit 3) |
| Rust `live_zone.rs` `PlainText` arm to Kompress | Only plain text. It
is the single remaining call to `kompress_or_noop` |
| Rust: prose text with an embedded fenced code block | **Not
reproduced.** I built synthetic tool results of prose around the fenced
Python script, at 6 KB, 20 KB and 71 KB. On this branch the dispatcher
classified each one as `source_code`, so commit 3 leaves it unchanged,
and Kompress was never consulted (`kompress_init_runs() == 0`). Rust has
no fence-aware splitting, but I found no input where that sends code to
Kompress, so this PR files no follow-up. |
| Python `config_compressor.py` docstring says YAML/TOML/INI "fall
through to the lossy prose path" | Documentation only: a stale comment,
not a code path. Config content has its own `CONFIG` strategy. Can be
refreshed in a later change |

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: yes, in the default config code is no
longer sent to Kompress (see "Behaviour change to review"). Tokens saved
on code drop in return for code that stays valid.
- Kill switch / disable path: none added. Revert the PR to restore the
old fallback.
- Unsafe override required: no.
- Qualification impact: none expected. Compression of code by the code
compressor is unchanged.
- Rollback path: revert the three commits. No data, schema or config
migration.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines (`ruff`, `cargo
fmt`, `clippy` clean)
- [x] I have made corresponding changes to the documentation
(`docs/content/docs/code-compression.mdx`,
`docs/content/docs/how-compression-works.mdx`)
- [x] I have added tests that prove my fix is effective
- [x] I did **not** edit `CHANGELOG.md`
- [ ] New and existing unit tests pass locally: the only failures are
environment failures that also fail on `main`, listed above

## Additional Notes

- Rust has no fence-aware splitting like the Python router has. A
synthetic prose-plus-fence repro did not reach Kompress (see the table
above), so no follow-up issue is filed.
- One stale comment remains in `content_router.py`
(`_try_ml_compressor`, the Phase 0 size-gate note lists
`CODE_AWARE->KOMPRESS` as an entry point). It is comment-only; not
changed here to keep the review commits as they are. Say if you want it
fixed in this PR.
- Reviewers: the Python change is easiest to review commit by commit.
Commit 1 is oxura's change, and commits 2 and 3 are the tests, docs and
the Rust fix.

---------

Co-authored-by: oxura <grozafroza44@gmail.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-16 23:25:06 -06:00
0d203db6b8 fix(copilot): keep new VS Code chats on the Local harness so they reach the proxy (#4899)
## Description

VS Code can run a Copilot chat on its "Copilot harness" (Agent Host, the
bundled Copilot SDK runtime) instead of the Local harness. That runtime
ignores `github.copilot.advanced.debug.overrideCapiUrl` /
`overrideProxyUrl`, so the chat goes straight to GitHub. Copilot's small
background calls (title, intent) still reach the proxy, so the proxy
looks connected while the main conversation never arrives. The harness
default (`chat.defaultToCopilotHarness`,
`chat.editor.preferCopilotHarness`) is tagged `experiment: startup` in
VS Code, so it can be flipped per user by rollout.

`headroom wrap vscode` now pins new chats to the Local harness.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/providers/copilot/vscode.py`: the managed block also writes
`"chat.defaultToCopilotHarness": false` and
`"chat.editor.preferCopilotHarness": false`. Because they sit inside the
marker block, `headroom unwrap vscode` removes them. A harness setting
the user already set outside the block is left alone (no duplicate key).
New helper `copilot_harness_settings_enabled(path)` reports a user value
of `true`.
- `headroom/cli/wrap.py`: one line saying new chats stay on the Local
harness and existing chats keep their engine; a warning instead when the
user's own setting is `true`. `--no-configure` prints the two extra
settings.
- `docs/content/docs/vscode-copilot.mdx`: documents the harness settings
and the troubleshooting case.
- Blocks written by older versions (including ones with
`overrideAuthType`) are replaced by marker as before.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
tests/test_provider_copilot_vscode_config.py + tests/test_cli/test_wrap_vscode.py: 32 passed.
Against origin/main source: 7 of those fail (block contents, old-block upgrade, user-set key [True]/[False], wrap output, --no-configure output).
ruff 0.17.4 check + format --check on the 5 changed Python files: clean.
mypy on the changed source files: 8 errors, all in headroom/cache/prefix_tracker.py and headroom/ccr/mcp_server.py, identical on origin/main; none in changed files.
```

## Real Behavior Proof

- Environment: macOS 27.2.1, Python 3.12.13, VS Code 1.145.2 with GitHub
Copilot signed in, core run from this branch, real GitHub Copilot
upstream. Real user `settings.json` (backed up and restored afterwards).
- Exact command / steps:
  1. `headroom wrap vscode -p 8795` from this branch.
2. `code chat -n -m agent "Reply with the single word OK and nothing
else."` (opens a new VS Code window and sends one Agent-mode chat).
3. Read the proxy's `/stats`, VS Code's `agenthost.log`, then `headroom
unwrap vscode`.
- Observed result:
  ```text
VS Code Copilot proxy settings updated: ~/Library/Application
Support/Code/User/settings.json
Keep using Copilot's normal model picker; the selected model is
preserved.
New chats stay on VS Code's Local harness; a chat keeps the engine it
started with, so start a new chat.

  settings.json after wrap:
  {
  	// --- Headroom Copilot proxy ---
"github.copilot.advanced.debug.overrideProxyUrl":
"http://127.0.0.1:8795/p/win-local",
"github.copilot.advanced.debug.overrideCapiUrl":
"http://127.0.0.1:8795/p/win-local",
  	"chat.defaultToCopilotHarness": false,
  	"chat.editor.preferCopilotHarness": false
  	// --- end Headroom Copilot proxy ---
  }

  proxy requests before chat: 0 {}
proxy requests after one Agent-mode chat in a new VS Code window: 4
{'gpt-4o-mini-2024-07-18': 3, 'claude-opus-5.5': 1}
  agent host turns during the chat: 0

Removed Headroom Copilot proxy settings from ~/Library/Application
Support/Code/User/settings.json
  settings.json after unwrap:
  {
  }
  ```
Separately, on the same machine with the settings edited by hand (same
VS Code, same proxy, new window each time):
`"chat.defaultToCopilotHarness": true` -> the Agent-mode chat ran in the
Agent Host (provider `copilotcli`) and 0 requests reached the proxy;
`"chat.defaultToCopilotHarness": false` +
`"chat.editor.preferCopilotHarness": false` -> Local harness, 4 requests
reached the proxy including the main `/v1/messages` call.
`"chat.temporaryDefaultToLocalHarness": true` as a user setting had no
effect, so it is not written.
- Not tested: a VS Code where Microsoft's rollout has flipped the
default (the explicit user value is expected to win, but that could not
be reproduced); an organization that enforces managed sandbox settings,
where VS Code can require the Copilot harness regardless; VS Code
profiles, workspace-level settings, Insiders/VSCodium; Windows and
Linux; the persistent `headroom start --persistent --vscode` path in the
Enterprise client.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: `headroom wrap vscode` writes two
more VS Code user settings inside its managed block. No proxy behaviour
changes.
- Kill switch / disable path: `headroom unwrap vscode` removes them; a
user who sets either key themselves keeps their value.
- Unsafe override required: no.
- Qualification impact: none expected.
- Rollback path: revert this commit; `unwrap` removes blocks of either
format.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- This keeps users off the Copilot harness; it does not route that
harness through Headroom. Supporting it is separate work.
- Only the configured settings file is checked for a user value. A
`true` in workspace settings, another profile, or policy is not
detected.
- After `unwrap` the keys return to VS Code's default, which a rollout
may move to the Copilot harness.
- Existing chats keep the harness they started with; the wrap output
says to start a new chat.
- `e2e/copilot_live.py` and `e2e/wrap/run.py` were not run.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4899"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-16 23:19:43 -06:00
93331f2874 fix(proxy): retain final backend streaming usage (#4814)
## Description

Translated OpenAI streams (`/v1/chat/completions` through a backend such
as `anyllm` or LiteLLM) recorded the **first nonzero** usage snapshot
instead of the last one.

`_absorb` in `_stream_openai_via_backend` wrote a counter only when the
accumulator was still falsy:

```python
if key in usage and not stream_state.get(key):
    stream_state[key] = usage[key]
```

A provider that reports climbing cumulative snapshots on successive
content chunks therefore had every later frame thrown away. Replaying
the two frames from the issue through the real proxy path, `/stats`
recorded 10 input / 1 output tokens and `/metrics` recorded 2 cache
reads — while the client received both frames intact. The bug is in
accounting after delivery, not a dropped upstream event.

Closes #4263

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `_parse_sse_usage_from_buffer` (openai branch): absent counters no
longer default to `0`. A frame that omits a field was previously
indistinguishable from one reporting an explicit zero, which is a
prerequisite for "absent keeps the last value".
- `_absorb`: assigns unconditionally, so the latest cumulative snapshot
wins. An absent field keeps its earlier value; an explicit zero still
replaces it.
- Removed `_parse_completion_tokens_from_sse_chunk`. It existed to catch
completion-only frames, but the buffered SSE parser already handles
those, and its `not stream_state["output_tokens"]` guard latched the
output count a second time.
- Replaced that helper's unit tests with eight regressions driving the
real `_stream_openai_via_backend` path through stream finalization, in
Chat and Responses shapes.
- `docs/content/docs/proxy.mdx`: one paragraph on the `POST
/v1/chat/completions` page documenting the accounting rule, in the
published docs site.

Net: -100/+113 lines, of which most deletions are the removed helper and
its implementation-only tests.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_streaming_usage_parser.py -q
........                                                                 [100%]
8 passed in 1.01s

$ ruff check headroom/proxy/handlers/streaming.py tests/test_streaming_usage_parser.py
All checks passed!

$ ruff format --check headroom/proxy/handlers/streaming.py tests/test_streaming_usage_parser.py
2 files already formatted

$ mypy headroom/proxy/handlers/streaming.py
Success: no issues found in 1 source file
```

Regression set — the new tests against the unfixed handler (`git stash`
of `streaming.py` only):

```text
$ python -m pytest tests/test_streaming_usage_parser.py -q
FAILED ...[latest-snapshot-chat]      assert (10, 1, 2, 3) == (13, 5, 9, 1)
FAILED ...[latest-snapshot-responses]
FAILED ...[partial-chat]              assert (10, 1, 2, 3) == (10, 5, 2, 3)
FAILED ...[partial-responses]
FAILED ...[explicit-zero-chat]        assert (10, 1, 2, 3) == (0, 0, 0, 0)
FAILED ...[explicit-zero-responses]
6 failed, 2 passed in 1.07s
```

Six fail before the fix, all eight pass after — matching the issue
exactly.

Wider streaming and parser suites:

```text
$ python -m pytest tests/test_streaming_usage_parser.py tests/test_streaming_sse_malformed_events.py \
    tests/test_provider_billed_input.py tests/test_proxy_cache_ttl_metrics.py tests/test_proxy_handler_helpers.py \
    tests/test_proxy_streaming_request_logger.py tests/test_route_advice.py tests/test_h2_stream_reset_retry.py \
    tests/test_proxy_streaming_resilience.py tests/test_proxy_streaming_ratelimit_headers.py -q
3 failed, 607 passed in 10.21s
```

The three failures are
`tests/test_proxy_cache_ttl_metrics.py::test_stats_endpoint_reports_*`,
which raise `SystemExit: 78` from `_check_rust_core()` in `server.py`.
They are environmental (unbuilt Rust core in this checkout) and fail
identically on unmodified `upstream/main` with this branch stashed —
unrelated to this change.

## Real Behavior Proof

- Environment: Windows, Python 3.13.11, this branch, `PYTHONPATH` set to
the checkout.
- Exact command / steps: replayed the issue's two usage snapshots as a
Chat stream through `_stream_openai_via_backend` with a stub backend,
then read the resulting `RequestOutcome`.
- Observed result: the corrected counts from the issue; previously 10 /
1 / 2.

```text
provider_input_tokens=13 (expect 13)
output_tokens        =5 (expect 5)
cache_read_tokens    =9 (expect 9)
```

- Not tested: a live `anyllm`/LiteLLM upstream over real HTTP (no
credentials in this environment). The replay drives the identical
handler and parser path the proxy uses; the frames are byte-for-byte the
issue's.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a.
- Stable/default behavior changed: token accounting for translated
OpenAI streams now records the final snapshot. This is the fix; counts
move up from truncated to complete.
- Kill switch / disable path: none beyond reverting this commit.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: `git revert`.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

Authorship is credited to @oxura, who diagnosed this in #4263 and wrote
the original patch on their fork; they were holding it back on the
ten-open-PR limit. This PR carries their change rebased onto current
`upstream/main` (their branch was cut from `fbf7bc32`, and
`streaming.py` has since drifted ~200 lines), with the commit message
and tests adapted to the current tree. Their `Authored-by` trailer is on
the commit.

One hunk from their patch was redirected: they documented the rule in
`wiki/proxy.md`, which `AGENTS.md:38` says is unpublished and must not
receive new content, so the paragraph now lives on the `POST
/v1/chat/completions` page in `docs/content/docs/proxy.mdx`. Thanks to
Devin Review for flagging it.

Co-authored-by: oxura <grozafroza44@gmail.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-16 19:51:50 -06:00
NAZIYAandJerrettDavis 6b41b6e451 feat(proxy): add cli and settings knob for ccr_inject_system_instructions (#4642)
<!-- maintainer-current-main -->
## Maintainer latest-main checkpoint
Published `d13ba17dff65a5b31253ef3cf0775ffc2504d8da` contains current
upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 150
CLI/settings/Gemini tests passed. Earlier live behavior proofs below
retain their commit/platform scope; contributor runtime logic is
unchanged.

The common shard-2 failure was independently confirmed on main's hosted
job 127069149772 and locally (four failures, six passes): the Gemini
memory fixture mocked an unused body reader and provided no request
stream. The maintainer test-only follow-up now constructs a real
FastAPI/ASGI Request carrying JSON bytes and exercises the actual body
reader. All six test-function ASTs/assertions are unchanged, and all 10
parameterized Gemini cases pass. Touched-file Ruff/format and whitespace
checks pass. This fixes the inherited fixture instead of asking
contributors to run CI or weakening the memory/content assertions. Fresh
exact-head hosted qualification is requested. Nothing merged.
<!-- /maintainer-current-main -->

## Description



Closes #4636.



`ccr_inject_system_instructions` (which instructs models how to retrieve
compressed context when CCR markers are present) previously existed as a
field on `ProxyConfig` but could only be toggled programmatically via
the Python API. Operators deploying `headroom proxy` as a container or
standalone service had no CLI flag, environment variable, or settings
store entry to enable it without wrapping the process in custom Python
code.



This PR exposes `ccr_inject_system_instructions` as:

1. CLI flag: `--ccr-inject-system-instructions`

2. Environment variable: `HEADROOM_CCR_INJECT_SYSTEM_INSTRUCTIONS`

3. Curated settings store field: `ccr_inject_system_instructions` in
`settings_store.SETTINGS`

4. Server environment configuration: loaded in
`_proxy_config_from_env()`



The default remains `False` preserving exact existing behavior.



## Type of Change



- [x] New feature (non-breaking change that adds functionality)



## Changes Made



- `headroom/cli/proxy.py`: Add
`@click.option("--ccr-inject-system-instructions", is_flag=True,
envvar="HEADROOM_CCR_INJECT_SYSTEM_INSTRUCTIONS", ...)`, add parameter
to `proxy()`, and thread it into `ProxyConfig`.

- `headroom/settings_store.py`: Add
`SettingField("HEADROOM_CCR_INJECT_SYSTEM_INSTRUCTIONS",
"ccr_inject_system_instructions", ...)` to `SETTINGS` registry so it is
validated, saved to `settings.json`, and surfaced on `/settings`.

- `headroom/proxy/server.py`: Read
`HEADROOM_CCR_INJECT_SYSTEM_INSTRUCTIONS` in `_proxy_config_from_env()`
for multi-worker and container deployments.

- `docs/content/docs/proxy.mdx`: Document
`--ccr-inject-system-instructions` and
`HEADROOM_CCR_INJECT_SYSTEM_INSTRUCTIONS` in the CCR configuration
table.

- `tests/test_cli_proxy_env.py`: Add test cases verifying
`--ccr-inject-system-instructions`,
`HEADROOM_CCR_INJECT_SYSTEM_INSTRUCTIONS`, `settings.json`, and
`_proxy_config_from_env()` correctly populate
`ProxyConfig.ccr_inject_system_instructions`.



## Testing



- [x] Unit tests pass (`pytest tests/test_proxy/test_settings_store.py
tests/test_cli_proxy_env.py`)

- [x] Linting passes (`ruff check`)

- [x] Formatting passes (`ruff format --check`)

- [x] New tests added for new functionality

- [x] Manual testing performed



### Test Output



```text

$ pytest tests/test_proxy/test_settings_store.py -q

40 passed, 1 warning in 2.18s



$ pytest tests/test_cli_proxy_env.py -k "test_ccr_inject_system_instructions or test_settings_file_reaches_ccr_inject_system_instructions or test_proxy_config_from_env_reads_ccr_inject_system_instructions" -q

5 passed, 91 deselected in 3.42s



$ ruff check headroom/cli/proxy.py headroom/settings_store.py headroom/proxy/server.py tests/test_cli_proxy_env.py

All checks passed!



$ ruff format --check headroom/cli/proxy.py headroom/settings_store.py headroom/proxy/server.py tests/test_cli_proxy_env.py

4 files already formatted

```



## Real Behavior Proof



- Environment: Windows 11 x86_64, Python 3.14.5, Click CliRunner test
harness and settings store test harness.

- Exact command / steps: Executed `runner.invoke(main, ["proxy",
"--ccr-inject-system-instructions"])`, verified captured ProxyConfig;
tested env var `HEADROOM_CCR_INJECT_SYSTEM_INSTRUCTIONS=1`, verified
ProxyConfig; tested `settings.json` with
`{"ccr_inject_system_instructions": true}`, verified ProxyConfig; tested
`_proxy_config_from_env()`.

- Observed result: In all test cases,
ProxyConfig.ccr_inject_system_instructions resolved to True when enabled
and False by default, matching the behavior of sibling knobs like
HEADROOM_CCR_INLINE_RESOLVE and HEADROOM_CCR_BUFFERED_CHAT_STREAMING.

- Not tested: Live Kubernetes deployment or multi-node container
cluster.



## Runtime Rollout Safety



- Rollout-managed feature(s): none, opt-in configuration knob

- Minimum rollout channel: n/a

- Stable/default behavior changed: no, default remains False

- Kill switch / disable path: omit the flag or unset
HEADROOM_CCR_INJECT_SYSTEM_INSTRUCTIONS

- Unsafe override required: no

- Qualification impact: none while disabled

- Rollback path: git revert this commit



## Review Readiness



- [x] I have performed a self-review

- [x] This PR is ready for human review



## Checklist



- [x] My code follows the project's style guidelines

- [x] I have performed a self-review of my code

- [x] I have commented my code, particularly in hard-to-understand areas

- [x] I have made corresponding changes to the documentation

- [x] My changes generate no new warnings

- [x] I have added tests that prove my fix is effective or that my
feature works

- [x] New and existing unit tests pass locally with my changes

- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)



## Additional Notes



Follows the exact pattern established by `--ccr-inline-resolve` and
`--ccr-buffered-chat-streaming` (#4631).



<!-- devin-review-badge-begin -->



---



<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4642"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>

<!-- devin-review-badge-end -->



## Maintainer runtime and integration qualification

Published `9c0b0a10d63227554350a3ba2cb936e09cea1286` includes upstream
main `ee975d49c0b742e34b09df50a01c7b5680e85cdc` with a clean merge and
no contributor logic changes. Main is actively advancing, so final
qualification must check the then-current main again.



Windows/Python 3.12.13: `python -m pytest tests/test_cli_proxy_env.py
tests/test_proxy/test_settings_store.py -q` reports **140 passed** with
the existing development native extension. Touched-file Ruff, format and
whitespace checks pass; no fresh native build claimed.



Real CLI proof (no replaced run_server or provider transport): launch
the actual Click entry point `from headroom.cli import main; main()` in
six isolated subprocesses with `proxy --mode token --no-optimize
--no-cache --no-rate-limit --no-ccr-proactive-expansion
--no-memory-tools --no-memory-context --disable-kompress
--anthropic-api-url http://127.0.0.1:<fixture-port>`. Each process has
an isolated workspace/settings file, offline/beacon opt-out and a real
CompressionStore entry created before launch; GET `/v1/retrieve/{hash}`
confirms ownership. Send `/v1/messages` with that stored hash in a
historical tool-result marker, through the actual TCP proxy to a local
HTTP Anthropic-shaped provider fixture, and inspect the received
upstream JSON.



Observed: all six calls return HTTP 200. Default: no `Compressed Context
Available` instructions. CLI flag, environment
`HEADROOM_CCR_INJECT_SYSTEM_INSTRUCTIONS=1`, and a settings file with
`{"ccr_inject_system_instructions":true}` each produce the instructions
upstream. `--no-ccr` and `--lossless`, both requested alongside the
flag, produce neither instructions nor a retrieval tool despite the
owned historical marker. This verifies the disabled-CCR finding is
addressed on the real request path. The documented
warm-prefix/cache-mode skip remains an intentional limitation; this run
uses token mode.



Not tested: live cloud models, Kubernetes/multiple nodes, every provider
wire shape, warm-cache replay or a fresh native build. The local HTTP
fixture is synthetic; request/configuration/ownership paths and TCP
transport are real. Hosted CI must finish on the final head; nothing
merged.



### Maintainer current-main requalification

Exact head `b38e5a5937cf09c3b5127f3f066e0384db235955`, upstream main
`ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`.

Clean integration with upstream/main
ea59ac3eef. Resolved only the
independently introduced Gemini test-fixture conflict by retaining
main's stricter two-chunk ASGI receive fixture; existing assertions and
contributor production behavior remain intact.114 CLI-env/Gemini-memory
tests pass with actual isolated Click8.5.0; touched-source Ruff and
whitespace pass. Re-reviewed new port/Hermes/settings and
no-response-handling overlap: injection still defaults false and
no-CCR/lossless disable it. Prior six actual CLI/TCP
proxy/upstream-fixture request proofs remain scoped to unchanged
injection behavior; no fresh cloud/native build claim. No new source
finding; exact-head hosted qualification pending. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `5addea3958b4a1bd4eec05ae23a58c9732f516c6`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 114 passed in 14.10s. Selected real
dependency overlays and native DLL paths are recorded in
final4642-fresh-native-tests.log; the DLL provenance/input hashes are in
final-native-artifact-proof.json. No redundant rerun is implied after
this Cargo-only merge. Whitespace passes. Prior source review and scoped
real-behavior evidence remain applicable to unchanged contributor
inputs.

No new source finding in the reviewed scope. Final formal approval
awaits exact-head hosted qualification. The author rebased the branch
after the preceding maintainer publication; independently compared old
and current trees differ only by one blank line in the Gemini fixture.
Current-head guarded synchronization preserves that change and the false
default/no-CCR/lossless behavior.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `7b0112872216941c25aa0f88e45701c39df3dd5c`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Current-main requalification
against2b0315b2d4b0d54947050bbdad2de9e89df06378. All71 integration
patches exactly match c01cf07-to-current-main delta after diff-location
normalization; contributor flag/default/no-CCR/lossless policy
unchanged.277focusedCLI/Gemini/Kompress/must-keep/doctor tests
pass,1deprecationwarning. Six fresh actual CLI subprocess/TCP
proxy/local upstream cases pass with an owned marker:default
instructionsfalse/retrievaltrue;flag,environment,settings
instructionstrue/retrievaltrue;no-CCR andlossless
instructionsfalse/retrievalfalse. This addresses configuration/wire
behavior,not cloud processing;cache-mode limitation remains
documented.33changedPythonfiles pass Ruff lint/format;offline
lock/whitespacepass. Existingpr3982nativeDLLverified
fullinputhashes/artifactchecksum;no freshbuild claimed.
Currentauthor/Devin discussion checked,previouscode/docs/live-proof
findings remain addressed. No new sourcefinding in reviewed scope.
Exact-head hostedqualification pending beforeapproval;maintainers
handlechecks,nothingmerged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `60a7ec28a53c5cb55e64e39a9182191b17fe9090`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 17 integration patches
exactly match the prior-reviewed-main delta after diff-location
normalization; contributor flag/default/no-CCR/lossless policy
unchanged. Fresh CLI/doctor and incoming Vertex/backend tests: 298
passed, 11 skipped, 1 warning. Twelve changed Python files pass Ruff
lint/format; offline lock and whitespace pass. An initial test command
selected a nonexistent settings test path and collected nothing; the
corrected complete suite above passed.

Fresh actual CLI subprocess/TCP proxy/local-upstream cases pass with an
owned marker: default instructions false/retrieval true; flag,
environment and settings instructions true/retrieval true; no-CCR and
lossless instructions false/retrieval false. This establishes local
configuration/wire behavior, not cloud processing; documented cache-mode
limits remain. Author/Devin discussion checked; prior findings addressed
by source guards, documentation and scoped real TCP evidence. Static
production/test review found no unrelated executable, workflow or
dependency changes; flagged inherited workflow-validation job is
excluded by unchanged workflow paths. Full native
input-set/hash/artifact checksum verified before DLL reuse, no fresh
build claimed. Formal approval awaits exact published-head hosted
qualification. Maintainers handle checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-16 19:43:14 -06:00
JD DavisandCopilot e9301f34c1 feat(tray): opt-in system tray icon with savings counter, dashboard and update alert (#4726)
## Description

Adds an **opt-in system tray icon** for Headroom while it runs (Windows
first; macOS/Linux planned). The icon is the Headroom logomark and its
right-click menu shows a compact savings counter (tokens and estimated $
saved), opens the dashboard, alerts when an update is available, and
lets you quit.

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- New `headroom/tray/` package: `core.py` (GUI-free stats
polling/formatting), `icon.py` (logomark drawn with Pillow, no binary
assets in the wheel), `app.py` (pystray app), `launcher.py` (spawns the
tray as a child of the proxy).
- `headroom proxy --tray` starts the tray with the proxy; `headroom tray
--port N` attaches to a running proxy.
- Menu: live `▼ tokens · ~$ saved` counter (from `/stats` →
`persistent_savings.lifetime`, polled every 5s), Open Dashboard, "Update
available" (only when newer; reuses `headroom.update_check`, honours
`HEADROOM_UPDATE_CHECK=off`; one-time notification), Quit.
- Icon states: normal, orange dot when an update exists, grey when the
proxy is offline.
- New optional extra `headroom-ai[tray]` (`pystray`, `pillow`).
Unsupported platform / missing extra: `--tray` prints why and the proxy
starts normally.
- Docs: `docs/content/docs/system-tray.mdx` + nav entry; screenshots in
`docs/public/assets/tray/`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
> python -m ruff check headroom\tray headroom\cli tests\test_tray_core.py
All checks passed!
> python -m mypy headroom\tray headroom\cli\tray.py
Success: no issues found in 7 source files
> python -m pytest tests\test_tray_core.py tests\cli tests\test_offline_egress_chokepoint.py tests\test_proxy\test_settings_store.py -q
293 passed, 2 skipped
```

(Scoped runs; I did not run the full suite or `mypy headroom` over the
whole package.)

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.13, 3840x2160 display, `pystray`
+ `pillow` 12.3.0, branch based on `upstream/main` @ 1da8930ab.
- Exact command / steps: run the 4 steps below.
1. `HEADROOM_UPDATE_CHECK=off`, then `python -m headroom.cli proxy
--tray --port 8799`
2. Confirmed `GET /health` → 200 and a `python -m headroom.tray --host
127.0.0.1 --port 8799 --owns-proxy` child process.
3. Opened the taskbar overflow, right-clicked the Headroom icon,
captured the menu (below).
  4. Clicked **Quit Headroom**.
- Observed result: icon appeared with the logomark; menu showed `▼ 256M
tokens · ~$765.96 saved` (my real lifetime savings store), **Open
Dashboard**, **Quit Headroom**. After Quit, `/health` no longer
responded (proxy shut down gracefully).
- Not tested: the real update toast and "Update available" menu item
(covered only by a unit test of `available_update()` and a rendered
icon); clicking Open Dashboard / double-click default action; macOS and
Linux (intentionally unsupported for now); HiDPI/light-theme variations;
`headroom tray` standalone Exit in the UI.

## Runtime Rollout Safety

- Rollout-managed feature(s): None (not rollout-channel gated).
- Minimum rollout channel: N/A
- Stable/default behavior changed: No. Off by default; only active with
`--tray`.
- Kill switch / disable path: omit `--tray` (or pass `--no-tray`).
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: revert this PR; the new extra and command are additive.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes (only
`tests/test_tray_core.py` and `tests/cli` were run)
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

**Tray icon** (left: real icon in the Windows taskbar overflow, zoomed;
then rendered states: live, update available, proxy offline)

![Tray icon
states](https://raw.githubusercontent.com/JerrettDavis/headroom/3e824653772642c95fded190b454fff82f2ce86a/docs/public/assets/tray/tray-icon.png)

**Right-click menu** (real capture)

![Tray right-click
menu](https://raw.githubusercontent.com/JerrettDavis/headroom/3e824653772642c95fded190b454fff82f2ce86a/docs/public/assets/tray/tray-menu.png)

**How to use**

```bash
pip install "headroom-ai[tray]"
headroom proxy --tray        
headroom tray --port 8787    # attach to an already-running proxy
```

Windows hides new tray icons in the `^` overflow next to the clock; drag
the icon onto the taskbar to pin it, then right-click. Full guide:
`docs/content/docs/system-tray.mdx`.

## Feature spec

- **API surface**: `headroom proxy --tray/--no-tray`, `headroom tray
--port N`, extra `headroom-ai[tray]`. No change to existing behavior
when the flag is absent.
- **User stories**: Given the proxy runs with `--tray`, when I
right-click the icon, then I see tokens/$ saved, Open Dashboard and
Quit. Edge: given the proxy is down, the icon turns grey and the tooltip
says offline.
- **Failure modes / recovery**: unsupported platform or missing extra
prints a reason and the proxy starts normally; the tray child exits when
the proxy dies (stdin EOF); a stats fetch failure shows the offline
state and retries every 5s. Quit sends SIGINT for graceful shutdown
without a force-exit deadline.
- **Security**: tray only polls the local proxy `/stats`; no network
calls of its own (update info read from the existing cache; honours
`HEADROOM_UPDATE_CHECK=off`); child is spawned with a fixed argv.

## Dependency justification

- **Why**: `pystray` is the standard cross-platform tray library
(Win/macOS/Linux); `pillow` is required by it for icons and lets us draw
the logomark with no binary assets. No existing dep provides a tray.
- **Maintainers**: pystray (moses-palmer), BSD/LGPL, stable ~0.19.5
releases; Pillow (python-pillow org), very active, frequent security
releases.
- **Install surface**: optional extra only (not default). pystray pulls
`six`, plus pywin32-free ctypes on Windows; Pillow ships prebuilt
wheels. No install-time network access.
- **Why these versions**: `pystray>=0.19.5` is the current release
(required new functionality); `pillow>=12.3.0` matches the repo pin
level already used elsewhere to avoid known CVEs.

## Additional Notes

- The menu screenshot shows my real local savings numbers.
- The Quit item stops the proxy only when the tray was started via
`headroom proxy --tray`; the standalone `headroom tray` shows "Exit" and
only closes the tray.
- Follow-ups: macOS/Linux backends (pystray needs the main thread on
macOS, hence the child-process design).

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-11-16 19:40:55 -06:00
JD DavisandCopilot ba26fd7d92 docs(install): add Windows auto-start and warm-join happy path (#4725)
## Description

Documents the Windows happy path for persistent auto-start and
warm-join. Behavior already exists upstream (boot + 5-minute health
tasks, S4U-to-session fallback from #4627, per-port `wrap`
reuse/recovery); the docs had no end-to-end Windows walkthrough.

Closes #

## Type of Change

- [x] Documentation update

## Changes Made

- Add "Windows happy path: auto-start and warm-join" to
`docs/content/docs/persistent-installs.mdx`.

## Testing

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
Docs-only; no automated tests apply.
```

## Real Behavior Proof

- Environment: Windows 11, non-elevated user, headroom-ai 0.43.1
runtime; persistent-task profiles `local-agents` (9010), `local-copilot`
(9012).
- Exact command / steps: `headroom install status --profile
local-agents`; `headroom wrap claude --port 9010 -- --version` (joined
the running proxy, no second proxy); 3 concurrent `headroom wrap codex
--port 9555 -- --version` on a cold port (one "Starting Headroom proxy",
others joined); confirmed S4U `schtasks /Create` returns "Access is
denied" for this user, matching the documented fallback.
- Observed result: proxy healthy and reused; Codex traffic observed in
proxy log; one proxy per port.
- Not tested: S4U/logged-off recovery (denied on this account);
`headroom install apply` against 0.44 from a clean machine.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: no
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert docs commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4725"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-11-16 19:40:21 -06:00
66e0086952 feat(opencode): add repeatable --extra-model flag for custom wrap-time models (#2364)
## Description

`headroom wrap opencode` injects a hardcoded model list
(`HEADROOM_OPENCODE_MODELS`) into the `headroom` provider block of
`opencode.json`. There's no way to add custom models (for example a
free-tier DeepSeek) at wrap time. Hand-patching `opencode.json` is the
only workaround, and the next wrap overwrites it.

This PR adds a repeatable `--extra-model` flag and a
`HEADROOM_OPENCODE_EXTRA_MODELS` env var. Both take
`id[=name[=context_window[=output_limit]]]` specs. Extra models are
merged into the injected provider block alongside the built-ins, and
flag specs win over env specs on id collision. `name` defaults to the
id, `context_window` to 200000, and `output_limit` to `min(16384,
context_window)`.

Fields are separated by `=` rather than `:` because model ids routinely
contain `:`. Ollama ids are `name:tag` (`qwen2.5-coder:7b`), and
OpenRouter free-tier ids end in `:free`. With a `:` separator those were
silently cut down to a different id (Devin review, 2026-10-03).

The PR also fixes the wrap/unwrap lifecycle the flag depends on. A wrap
that starts without an `opencode.json` used to write the file with no
backup and no record of what Headroom put in it, so `headroom unwrap
opencode` left the injected models and the proxy base URL behind while
reporting that OpenCode was no longer routed through Headroom, and a
re-wrap under `--openai-api-url` kept an extra model Headroom had
injected but no longer advertised. Wrap now records what it owns in
`opencode.json.headroom-managed.json`, and unwrap uses that record: it
removes the model ids and the base URL Headroom wrote, keeps everything
else in the file, drops an extra model that is no longer requested, and
deletes the config file when Headroom created it and nothing else is
left in it.

Closes #2359

## Type of Change

- [x] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [x] Documentation update
- [ ] Refactoring (no functional changes)

## Changes Made

- `headroom/providers/opencode/config.py`:
- New `parse_extra_model_spec()`, which splits on `=` with an optional
fourth `output_limit` field and raises `ValueError` on an empty id or a
non-integer / non-positive limit.
- New `extra_models_from_env()`, which reads comma-separated specs from
`HEADROOM_OPENCODE_EXTRA_MODELS`.
- An `extra_models` parameter threaded through
`headroom_provider_entry()`, `_render_provider_block()` and
`inject_opencode_provider_config()`.
- With `keep_user_entries` (#3125's `--openai-api-url` gate),
`--extra-model` entries win over a kept user entry with the same id.
- New managed-state sidecar (`opencode.json.headroom-managed.json`)
recording the model ids and base URL Headroom injected and whether
Headroom created the config file. `inject_opencode_provider_config()`
writes it on every injection; `record_opencode_config_origin()` marks a
config that Headroom creates because none existed.
- `remove_managed_opencode_provider()` drops the recorded model ids, the
recorded base URL, and the provider block once nothing else is left in
it. Entries the user added or edited are preserved.
- `_drop_stale_managed_models()` prunes an extra model Headroom injected
but no longer advertises, including an extra that overrode a built-in id
and is dropped again, so a re-wrap without the flag restores the
built-in defaults rather than keeping the old override.
- `snapshot_opencode_config_if_unwrapped()` no longer snapshots a config
Headroom wrote itself as pre-wrap content.
- `headroom/providers/opencode/runtime.py`: extras also reach the
launch-time `OPENCODE_CONFIG_CONTENT`.
- `headroom/cli/wrap.py`:
- A repeatable `--extra-model` option on `wrap opencode`. Env and flag
specs are parsed up front, and a malformed spec raises
`click.BadParameter`. The merged dict goes to both
`inject_opencode_provider_config` call sites (prepare-only and launch)
and to the launch env. Also adds a help text and docstring example.
- `wrap opencode` records the config origin before the first config
mutation.
- `unwrap opencode` undoes the recorded provider entries when there is
no pre-wrap backup, keeps the rest of the file, and removes a config
Headroom created once nothing Headroom-owned is left in it.
- `docs/content/docs/opencode.mdx`: new "Adding models" subsection
(format, defaults, Ollama/OpenRouter example, built-in id override, the
env var's no-comma limitation) and an env-var table row. The subsection
and the "How It Works Under The Hood" unwrap step now describe the
managed-state record and the automatic cleanup it enables.
- Tests:
- Spec parsing: full spec with output, defaults, the output cap at small
contexts, colon-bearing ids (`qwen2.5-coder:7b`,
`deepseek/deepseek-chat:free`, a name containing `:`), and invalid specs
including a bad output limit.
- Env parsing, provider-entry merge, and end-to-end inject. Also
@JerrettDavis's collision test across both `keep_user_entries` values.
- CLI: the flag reaches `opencode.json`, the env var persists, both
reach `OPENCODE_CONFIG_CONTENT` on launch, the flag wins over a kept
user entry under `--openai-api-url`, and a malformed spec exits with a
usage error.
- Managed state: the sidecar records the injected ids and base URL, a
config the user already had is not marked as Headroom-created, a re-wrap
does not snapshot Headroom's own config, a dropped extra model is pruned
(including an override of a built-in id, which returns to the built-in
defaults), a user model and a user-edited model survive, and the
provider removal keeps unrelated user entries.
- Lifecycle: a fresh wrap followed by an unwrap removes the config file
again (with and without the MCP entry), user content added after a fresh
wrap survives the unwrap, a re-wrap without the extra model drops the
stale one, and a directly injected provider entry is undone by the
unwrap callback as well.
- A user setting that is false, zero or an empty string is content, not
an empty section: such a setting survives the unwrap instead of costing
the user the whole file.

## Testing

- [x] Unit tests pass locally
- [x] Lint/format/type checks pass locally
- [x] New tests added for new functionality

The new parsing tests fail against the previous `:`-based parser:

```text
FAILED tests/test_providers_opencode_config.py::test_parse_extra_model_spec_keeps_colons_in_ids[qwen2.5-coder:7b-...]
FAILED tests/test_providers_opencode_config.py::test_parse_extra_model_spec_keeps_colons_in_ids[deepseek/deepseek-chat:free=DeepSeek Free-...]
FAILED tests/test_providers_opencode_config.py::test_parse_extra_model_spec_full
...
================ 21 failed, 2 passed, 103 deselected in 0.86s =================
```

With this branch (head 6be848942):

```text
$ pytest tests/test_providers_opencode_config.py tests/test_cli/test_wrap_opencode.py -q
179 passed, 2 skipped in 3.31s

$ pytest tests -q -k "opencode"
322 passed, 26 skipped

$ ruff check headroom/providers/opencode/config.py headroom/cli/wrap.py tests/test_providers_opencode_config.py tests/test_cli/test_wrap_opencode.py
All checks passed!
$ ruff format --check (same files)
4 files already formatted
$ mypy headroom --ignore-missing-imports --no-site-packages
headroom\fileperms.py:92: error: Module has no attribute "fchmod"; maybe "chmod"?  [attr-defined]
headroom\proxy\interceptors\astgrep.py:220: error: Module has no attribute "O_DIRECTORY"  [attr-defined]
...
Found 7 errors in 2 files (checked 561 source files)
```

The mypy errors are the Windows-only `os.fchmod` / `os.O_DIRECTORY`
false positives in files this PR does not touch. The same Windows
path-escaping failures that `test_build_launch_env_with_project` used to
hit (`tests/test_cli/test_unwrap_claude.py`,
`tests/test_wrap_code_memory.py`, `tests/test_install/test_planner.py`,
`tests/test_cli/test_wrap_stale_marker.py`) reproduce on unchanged
`main` and are unrelated to this change. `Dependency audit (pip-audit)`
passes on this head; only `Rust advisories (cargo audit)` fails, on
advisories published after upstream `main`'s last green run (2026-11-13
04:47) — `quinn-udp` `RUSTSEC-2026-0538` and `rustix`
`RUSTSEC-2026-0486`. `Cargo.lock` on this branch is byte-identical to
upstream `main` (`81818d49`), so the PR does not introduce them.

## Real Behavior Proof

- Environment: Windows 11, Python 3.13, this branch vs. the previous
`:`-based parser and vs. the previous unwrap lifecycle,
`HOME`/`USERPROFILE` pointed at a temp dir.
- Exact command / steps: `headroom wrap opencode --prepare-only
--no-serena --extra-model "qwen2.5-coder:7b" --extra-model
"deepseek/deepseek-chat:free=DeepSeek Free=65536"`, then read
`provider.headroom.models` from `~/.config/opencode/opencode.json`; then
`headroom unwrap opencode --no-stop-proxy` against a config that did not
exist before the wrap.
- Observed result: with the old parser the entries were `qwen2.5-coder`
(name `7b`) and `deepseek/deepseek-chat` (name `free=DeepSeek
Free=65536`, context 200000): wrong ids, and the paid DeepSeek model
instead of the free one. With this branch they're `qwen2.5-coder:7b`
(name defaults to the id, context 200000, output 16384) and
`deepseek/deepseek-chat:free` (name `DeepSeek Free`, context 65536,
output 16384), next to the built-in `gpt-4o` / `gpt-4.1`. After the
unwrap of a fresh wrap, `opencode.json` and its managed-state sidecar
are both gone and no `provider.headroom` block is left. After the unwrap
of a fresh wrap the user had added to, the file keeps the user's own
model id, key and `theme`, and only the injected ids and the
`http://127.0.0.1:8787/v1` base URL are gone.
- Not tested: a full (non-prepare-only) wrap launching the real OpenCode
binary against a live upstream; the OpenCode TS plugin side (extras are
user-supplied and intentionally not synced into
`plugins/opencode/src/provider.ts` `DEFAULT_MODELS`).

## Runtime Rollout Safety

- Rollout-managed feature(s): none; CLI-only opt-in flag and env var on
`headroom wrap opencode`, plus a `opencode.json.headroom-managed.json`
sidecar next to the OpenCode config
- Minimum rollout channel: stable
- Stable/default behavior changed: no. Without `--extra-model` or
`HEADROOM_OPENCODE_EXTRA_MODELS` the injected provider block is
unchanged. `unwrap opencode` now also removes the provider entries
Headroom recorded, which is what it already reported doing
- Kill switch / disable path: drop the flag and unset
`HEADROOM_OPENCODE_EXTRA_MODELS`, then re-wrap; delete the sidecar to
stop Headroom from tracking the config
- Unsafe override required: no
- Qualification impact: none; no proxy, compression or request-path code
changed
- Rollback path: revert the PR; `headroom unwrap opencode` restores the
pre-wrap `opencode.json` when a backup exists

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2364"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-17 02:38:10 +05:30
d6df127d57 feat(compress): accept Gemini and Bedrock Converse bodies on the gateway turn (#4679)
## Description

`/v1/compress` in gateway mode (a request with a `gateway` block) now
accepts two more native request bodies:
- Gemini `generateContent`
- Bedrock Converse

It compresses their text slots and returns `body` in the same native
shape. This lets a gateway like LiteLLM compress native Gemini and
Bedrock passthrough traffic, which it couldn't before.

`config.mode: "ccr"` is rejected for these shapes with a 400. Every
other mode is marker-free for them.

(Was stacked on #4617, which has merged; the branch now carries current
`main`.)

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

### How

This works the same way `gateway_responses` does for Codex: it builds a
view instead of translating the body.
- Every compressible text slot becomes one chat-shaped pseudo message.
- Compressed text is written back by its path in the body.
- Everything else stays byte-identical: `thoughtSignature`, thought
parts, `functionCall`, `inlineData`, `toolUse`, `cachePoint`,
`reasoningContent`, `tools`/`toolConfig`.
- Tool calls appear in the view, so tool-name rules still apply. `Read`
stays verbatim, for example.
- A JSON slot is only written back if it still parses to the same JSON
type.
- If the compressed list doesn't line up with the slots, the original
transcript is forwarded and a warning is logged.

### CCR on native bodies (review fix)

A native body can't carry `headroom_retrieve`. Its `tools`/`toolConfig`
are forwarded untouched, and `create_ccr_tool_definition` only builds
chat, Anthropic and Responses shapes. It also never gets a re-drive. So
a CCR marker in a native body points at content the model can never
fetch. Before this fix, `config.mode: "ccr"` on a Gemini or Converse
gateway turn did exactly that: log output came back as a marker with
`ccr_hashes: [<hash>]` and `obligations: []`.

- `handle_compress` now answers `ccr` on a native view with `400
invalid_request`. The message names the shape, explains why, and lists
the supported modes.
- The check runs where the mode is chosen, right after `attach_view` has
recorded the shape. That is before any compression, session work or CCR
store write.
- **The rejection does not depend on `gateway.can_redrive`.** Even a
gateway that can re-drive has no way to put the retrieval tool into a
native request, so there is no capability under which native `ccr`
works.
- **Without a `gateway` block nothing changes.** A Gemini body is
already a 400 (no `messages`). A Converse body goes through the plain
messages path, where its untyped blocks are opaque: 0 tokens saved, no
markers in any mode, `ccr` included (checked). So the rule is
consistent: reject `ccr` wherever Headroom reads a native shape.
- The bypass header still returns the body untouched with no
compression, in any mode.
- I checked every other mode on the same marker-producing input
(default, `lossless`, `lossy_inline`, `lossless_then_lossy`): no marker
and no hash in any of them. These modes build pipelines with
`ccr_inject_marker=False, ccr_enabled=False`. Stale-Read notices are
hashless in marker-free modes since #4678.
- **Second defect, same family:** with `HEADROOM_DEDUPE=1`, cross-turn
dedup wrote `[↑NL same as msg 4]` into a Converse tool result. That `4`
counts view messages (system blocks, tool calls, result slots), not the
caller's `messages`, so it named the wrong message. Native views now set
`cross_turn_dedup_recoverable=False`, as `lossless` mode already did.
- The `not self.native_shape` guards on CCR arming and re-drive in
`GatewayTurn.finish` stay as defence in depth.

### Caller contract

**Gemini**
- Send top-level `contents` as a list, with no `messages` or `input`.
- `systemInstruction`/`system_instruction`, `tools` and
`generationConfig` are optional and pass through.

**Converse**
- Send `messages` where every message has role `user` or `assistant`,
and every content block has no `type` key.
- At least one block must use a Converse key: `text`, `toolUse`,
`toolResult`, `cachePoint`, and so on.
- `system` is optional, as a list of blocks.

**Both shapes**
- `config.mode`: omit it (marker-free default), or use `lossless`,
`lossy_inline` or `lossless_then_lossy`. `ccr` is a 400.
- Only gateway turns get a view. Without a `gateway` block the endpoint
behaves exactly as before. A client-sent `_headroom_native_view` key is
dropped, and `BODY_CONTROL_KEYS` strips the real one before the body is
returned.
- `config.frozen_message_count` and `config.protect_recent` count native
messages: Gemini `contents` entries and Converse `messages`.
`session.frozen_message_count` comes back in native units. The
`/v1/usage` reply's `frozen_message_count` still counts view messages,
and the docs say so.
- No re-drive is ever armed, so `obligations` never contains `redrive`,
and request hooks run stream-safe-only. Tool compaction and output
steering are skipped.
- Relayed usage can be:
- Gemini `usageMetadata`, where `promptTokenCount` already includes
`cachedContentTokenCount`.
- Converse `usage`, where `inputTokens` excludes `cacheReadInputTokens`
and `cacheWriteInputTokens`.
  - The whole provider response that carries either.
- Every gateway answer, fail-open ones included, carries `wire_shape`:
`gemini`, `converse`, `openai_responses` or `messages`. A caller that
sent a native body and gets anything else, or no field at all, is
talking to an older Headroom and should forward its original body.
- Top-level `messages` in the answer is the Converse list for Converse
and the chat-shaped view for Gemini.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`): no new errors; the 8
reported are in `ccr/mcp_server.py` and `cache/prefix_tracker.py` and
are identical on `main`
- [x] New tests added for new functionality
- [x] Manual testing performed (local replay, below)

New in `tests/gateway/test_native_shapes.py`:
- `test_ccr_mode_is_rejected_for_native_bodies[gemini|converse]`: 400,
`invalid_request`, message asserted.
- `test_other_modes_forward_native_bodies_without_ccr_markers`: 4 modes
x 2 shapes, on log output large enough to produce a marker. Checks no
hash, no marker, the shape echoed, non-message keys byte-identical and
the native list mapped back.
- `test_ccr_still_works_on_a_chat_gateway_turn`: the same log output in
a chat body with `ccr` still carries a marker. This proves the input
above is marker-producing, and that chat `ccr` didn't regress.
- `test_native_bodies_get_no_dedup_pointer_into_view_messages`.
- `test_native_turns_are_never_asked_to_redrive` now runs in the default
mode, because `ccr` is a 400.

### Test Output

```text
# before the fix (new tests only, rest of the file passing)
FAILED tests/gateway/test_native_shapes.py::test_ccr_mode_is_rejected_for_native_bodies[gemini]
FAILED tests/gateway/test_native_shapes.py::test_ccr_mode_is_rejected_for_native_bodies[converse]
FAILED tests/gateway/test_native_shapes.py::test_native_bodies_get_no_dedup_pointer_into_view_messages
3 failed, 25 passed

# after
tests/gateway/test_native_shapes.py: 28 passed

# tests/gateway tests/test_gateway_turn_unit.py tests/test_compress_api.py
# tests/test_proxy_compress_endpoint.py tests/test_compress_session_mode.py
# tests/test_compress_session_shared.py tests/test_compress_passthrough.py tests/test_compress_failure.py
1 failed, 507 passed, 4 skipped
# the one failure, test_lossy_inline_emits_no_ccr_markers[1-json_minifier], fails identically on main fda75de5

uvx ruff@0.17.3 check / format --check (changed files): All checks passed! / 3 files already formatted
```

## Real Behavior Proof

These are local synthetic replays. They are not provider qualification.

- Environment: macOS, Python venv, branch head `530c20aa`. Headroom runs
in-process via `create_app(ProxyConfig(optimize=True,
cache_enabled=False, rate_limit_enabled=False,
cost_tracking_enabled=False, log_requests=False, image_optimize=False))`
behind `fastapi.testclient.TestClient` from a loopback client. The
"provider" is an `httpx.MockTransport` that records each body it
receives and answers with a Gemini `generateContent` or Bedrock Converse
response, including usage. `LITELLM_LOCAL_MODEL_COST_MAP=True`. No real
provider calls.
- Exact command / steps: `LITELLM_LOCAL_MODEL_COST_MAP=True python -I -c
"import sys; sys.path.insert(0,'<worktree>');
exec(open('native_replay.py').read())"`. For each shape the script does
three things:
1. POSTs the native body plus `gateway: {can_redrive, session_affinity,
can_relay_response: true}` to `/v1/compress`.
2. Sends the returned `body` through the mock upstream at the real
Gemini or Bedrock URL.
3. POSTs the provider's whole response to `/v1/compress/response` with
the `turn_id`.

It then sends both native bodies again with `config.mode: "ccr"`. The
tool output is the 399-line grep listing used in the tests. The `ccr`
requests use 3000 log lines, which produce a marker in `ccr` mode.
- Observed result:

```text
[gemini] request half: wire_shape=gemini obligations=['relay_usage'] tokens 7246->3863 ccr_hashes=[] marker_in_body=False
[gemini] all 399 grep lines still readable in the forwarded body=True
[gemini] non-message keys unchanged=True first two contents unchanged=True
[gemini] provider got 16125 bytes (raw 29694), control keys forwarded=[], host=generativelanguage.googleapis.com
[gemini] response half: 200 {'action': 'done', ..., 'billed_usage': {'input_tokens': 900, 'output_tokens': 12, 'cache_read_input_tokens': 512}}
[converse] request half: wire_shape=converse obligations=['relay_usage'] tokens 8823->4843 ccr_hashes=[] marker_in_body=False
[converse] all 399 grep lines still readable in the forwarded body=True
[converse] non-message keys unchanged=True first two messages unchanged=True
[converse] provider got 16184 bytes (raw 29761), control keys forwarded=[], host=bedrock-runtime.us-west-2.amazonaws.com
[converse] response half: 200 {'action': 'done', ..., 'billed_usage': {'input_tokens': 300, 'output_tokens': 12, 'cache_read_input_tokens': 600, 'cache_creation_input_tokens': 0}}
[gemini ccr] 400 {'error': {'type': 'invalid_request', 'message': "config.mode='ccr' is not supported for gemini bodies: Headroom cannot add its retrieval tool to a native Gemini or Bedrock Converse request, so the model could not resolve a CCR marker. Use lossless, lossy_inline or lossless_then_lossy, or omit config.mode for the default marker-free mode."}}
[converse ccr] 400 {'error': {'type': 'invalid_request', 'message': "config.mode='ccr' is not supported for converse bodies: ..."}}
provider calls: 2
```

Before the fix, the same `ccr` requests returned 200 with `ccr_hashes`
of length 1, a `hash=` marker in the forwarded body, and `obligations:
[]`.
- Not tested:
- live Gemini (AI Studio or Vertex) and live Bedrock Converse: the
provider was mocked, so provider acceptance of the rewritten bodies is
unverified;
  - real Kong `ai-proxy` or a real LiteLLM gateway in front of Headroom;
- streaming callers (`streamGenerateContent`, `ConverseStream`): the
request half is identical, but no streamed response was relayed;
  - Gemini `thoughtsTokenCount` in output accounting;
  - session mode with a Redis store for native bodies;
- multimodal parts (`inlineData`, `image`, `document`) beyond their
byte-identical pass-through in unit tests.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. Native shapes only activate for
gateway-mode `/v1/compress` callers that send a `gateway` block and a
native body.
- Minimum rollout channel: n/a
- Stable/default behavior changed: no for existing callers. Chat,
Anthropic and Responses bodies are unchanged. A Converse body without a
`gateway` block goes through the plain path as before. Before this PR, a
native body with a `gateway` block was misread as chat (Converse) or
rejected (Gemini).
- Kill switch / disable path: the gateway omits the `gateway` block, or
forwards its original body when `wire_shape` doesn't match.
- Unsafe override required: no
- Qualification impact: none for existing paths
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

Known follow-ups, not in this PR:
- A `functionResponse.response` with any string field never compresses
its nested dict values. The JSON slot is only used when there are no
string fields. This is a missed saving, not a correctness bug.
- `handlers/gemini.py` (the proxy path) and `gateway_native.py` map
Gemini parts with different rules, and their part-key lists are kept in
step by hand. They should share one constant.
- `protect_recent` translation and stream-safe hook gating for native
turns have no dedicated test.
- Gemini `thoughtsTokenCount` is not added to output tokens when usage
is relayed, so output savings attribution undercounts for thinking
models.
- This is not specific to native shapes: the marker-free default folds
very repetitive logs to a single line (for example `[3000 lines omitted:
3000 INFO]`) on chat bodies too, with no retrieval path.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-15 01:52:50 +05:30
491afe98b1 fix(proxy): reject a private x-headroom-base-url on chat and responses with 400 (#4857)
## Description

When the upstream guard rejected an `x-headroom-base-url` override (an
address on a private, loopback, link-local or otherwise non-public
network, or a host outside `HEADROOM_ALLOWED_BASE_URLS` in allowlist
mode), `_resolve_openai_upstream_base` logged a warning and returned
`None`. `/v1/chat/completions` and `/v1/responses` then sent the request
— the prompt and the client's `Authorization`, meant for the gateway the
client named — to the configured `OPENAI_API_URL` instead. Every other
route that reads the header (`headroom/providers/proxy_routes.py`:
Anthropic messages, Gemini, models, embeddings, audio, passthrough)
refuses with `400 Rejected unsafe upstream base URL`.

The OpenAI chat and responses handlers now refuse the same way.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update

## Changes Made

- `headroom/proxy/handlers/openai.py`: `_resolve_openai_upstream_base`
raises `HTTPException(400, "Rejected unsafe upstream base URL")` instead
of returning `None` when the guard rejects the override (malformed /
non-http(s) values are unchanged: still ignored).
- `docs/content/docs/security-model.mdx`: one behaviour for all routes
(the page previously documented the two different outcomes).
-
`tests/test_upstream_guard.py::test_openai_routes_refuse_a_private_override_instead_of_using_the_default[chat|responses]`:
real app, configured OpenAI target is a local recording server; asserts
400 and that the configured target is never contacted.

**Sibling PRs that touch the same code** (merge order note):
`fix/upstream-guard-off-loop` adds `_resolve_openai_upstream_base_async`
next to this function; whichever merges second must make the async
resolver raise the same 400 (that PR's rejection branch is the one place
to change). `fix/upstream-idna-pin-once` touches the call sites only.
Suggested order: idna-pin-once, off-loop, then this PR rebased.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# New test on origin/main (c21b0813)
E   AssertionError: {"secret":"internal-only"}
E   assert 200 == 400                (chat: answered by the configured target)
E   assert 200 == 400                (responses: answered by the configured target)
======================= 2 failed, 61 deselected in 5.64s =======================

# This branch
======================= 2 passed, 61 deselected in 5.81s =======================

# Area suites: test_upstream_guard, test_grok_shared_proxy_call_sites, test_proxy/test_openai_upstream_header,
# test_proxy_openai_cache_key_integration, test_upstream_credential_scoping, test_header_isolation,
# test_proxy_hardening, test_openai_codex_routing, test_proxy_loopback_gating
================= 429 passed, 10 warnings in 87.64s (0:01:27) ==================

$ ruff check / ruff format --check (0.17.3) on changed files
All checks passed!
2 files already formatted
```

## Real Behavior Proof

- Environment: macOS, Python 3.12, `python -m headroom.cli proxy
--no-optimize` from this branch and from origin/main,
`OPENAI_TARGET_API_URL` pointing at a local server that records every
request it receives.
- Exact command / steps:
`curl http://127.0.0.1:$PORT/v1/chat/completions -H 'authorization:
Bearer client-gateway-key' -H 'x-headroom-base-url:
http://192.168.1.50:8000/v1' -d '{"model":"gpt-4o","messages":[...]}'`,
and the same for `/v1/responses`.
- Observed result:
  ```text
  == origin/main
  POST /v1/chat/completions -> HTTP 200 {"...":"from-configured-target"}
  POST /v1/responses        -> HTTP 200 {"...":"from-configured-target"}
  configured target received:
    /v1/chat/completions auth= Bearer client-gateway-key
    /v1/responses        auth= Bearer client-gateway-key
  == this branch
POST /v1/chat/completions -> HTTP 400 {"detail":"Rejected unsafe
upstream base URL"}
POST /v1/responses -> HTTP 400 {"detail":"Rejected unsafe upstream base
URL"}
  configured target received: (nothing)
  ```
- Not tested: Codex WebSocket responses path (it does not read this
header).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: **yes.** A client that sends
`x-headroom-base-url` pointing at a private/LAN/loopback address (or, in
allowlist mode, an unlisted host) to `/v1/chat/completions` or
`/v1/responses` now gets a 400 instead of having its request answered by
the configured OpenAI target. Such a request never reached the host the
client named, so the only working setups affected are ones that relied
on the silent fallback. A DNS lookup that fails or times out for a
public gateway is also a rejection, so it now surfaces as a 400 instead
of quietly sending the request to the default target.
- Kill switch / disable path: operators who route to an internal gateway
add it to `HEADROOM_ALLOWED_BASE_URLS` (existing setting); clients can
drop the header to use the configured target.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Fix adapted from a proposal by HetCreep.

Follow-up, not in this PR: the OpenCode plugin
(`plugins/opencode/src/transport.ts`, `isRoutableUpstream`) only keeps
loopback upstreams out of the proxy, so an OpenCode provider on a LAN
address is still routed through Headroom, which now answers 400 for it
(previously the request went to the default OpenAI target). Sending
private-network upstreams direct needs a source change plus regenerating
the committed bundles (`headroom/providers/opencode/_dist/index.js`,
`hook-shim/handler.js`), and hostnames that resolve to LAN addresses
cannot be recognised without DNS, so it is left for a separate PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4857"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Reported by @JoaoMarcos44. Thank you for your contribution! 🩵

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-15 01:52:42 +05:30
5d44855c6f fix(proxy): withhold per-project savings and quota stats from /stats network callers (#4859)
## Description

`GET /stats` removes `recent_requests` / `request_logs` for a caller
that is not loopback, not a trusted dashboard CIDR peer and has not
presented the proxy token. The same payload still carried
`savings.per_project`, `persistent_savings.projects` (project names and
per-project spend) and the provider quota sections
(`get_quota_registry().get_all_stats()`, e.g. account login and plan).
For that same caller `/stats-lifetime` already drops `projects` and
`/quota` is refused, so `/stats` was the one route that still handed
them out.

`/stats` and `/stats?cached=1` now drop those fields under the same rule
(`_request_can_view_stats_metadata`). The nested `savings` /
`persistent_savings` dicts are rebuilt rather than edited in place, so
the shared cached snapshot still serves loopback callers in full.
Token-authenticated callers keep seeing everything, as documented in
"What the proxy token unlocks"; that docs table now lists the extra
fields.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/server.py` (`/stats`): for callers without
stats-metadata access, also pop every quota tracker key and drop
`savings.per_project` and `persistent_savings.projects`; docstring
updated.
- `docs/content/docs/security-model.mdx`: the `/stats` row of "What the
proxy token unlocks" lists per-project savings and the quota sections.
- `tests/test_proxy_stats_network_gating.py`: network caller on `/stats`
and `/stats?cached=1` gets no project name / quota data; loopback still
gets them after a network caller primed the cached snapshot; a token
holder still gets them.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest -q tests/test_proxy_stats_network_gating.py
5 passed in 18.60s

# with the server.py change stashed (origin/main code)
FAILED tests/test_proxy_stats_network_gating.py::test_stats_withholds_project_names_and_quota_stats_from_network_callers[/stats]
FAILED tests/test_proxy_stats_network_gating.py::test_stats_withholds_project_names_and_quota_stats_from_network_callers[/stats?cached=1]
2 failed, 3 passed in 7.10s

$ pytest -q tests/test_proxy_loopback_gating.py tests/test_proxy_stats_recent_requests.py tests/test_stats_output_reduction_gate.py tests/test_proxy_stats_network_gating.py
174 passed, 5 warnings in 44.87s

ruff check / format --check (0.17.3): All checks passed, 2 files already formatted
mypy headroom/proxy/server.py: Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS, Python 3.12,
`create_app(ProxyConfig(stateless=True, ...))` under FastAPI
`TestClient`, one recorded request for a project.
- Exact command / steps: `GET /stats` once as a network peer
(`203.0.113.9`, Host `203.0.113.5`) and once as loopback; print whether
`savings.per_project` and `persistent_savings.projects` are present.
- Observed result:
- origin/main: network caller gets `savings.per_project` and
`persistent_savings.projects` with every project name.
- this branch: network caller gets `savings.per_project: None` and no
`persistent_savings.projects`; loopback still gets both in full.
- Not tested: a real Copilot/Codex quota tracker (the test registers a
stand-in tracker).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: yes, for network callers without
dashboard-CIDR or token access only: `/stats` no longer includes
per-project savings or quota sections. Loopback dashboards, CIDR peers
and token holders are unchanged.
- Kill switch / disable path: configure
`HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` or present the proxy
token, as for `/stats-lifetime` and `/quota`.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Fix adapted from a proposal by HetCreep.
`persistent_savings.storage_path` (a local filesystem path) is still in
the network payload; left as is to keep this change to the fields the
sibling routes already withhold.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4859"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-14 10:46:33 +05:30
6d8121cfd4 fix(proxy): send operator extra headers only to a designated origin (#4858)
## Description

`upstream_trust.is_trusted_upstream` decides whether operator-configured
extra headers (`OPENAI_TARGET_API_HEADERS` /
`ANTHROPIC_TARGET_API_HEADERS`, stored as secrets) may be merged for a
client-chosen `x-headroom-base-url`. It compared hostnames only, so
`http://<designated host>` or `https://<designated host>:<other port>`
also received those headers, over cleartext in the http case.

It also used one trusted set for every provider, so a chat/responses
request with `x-headroom-base-url: https://api.anthropic.com` received
the OpenAI extras, and `/v1/messages` pointed at
`https://api.openai.com` or the configured OpenAI gateway received the
Anthropic extras.

Trust is now decided per provider, on the origin (scheme, host, port).
The OpenAI extras go only to the OpenAI target / `api.openai.com`; the
Anthropic extras only to the Anthropic target, the oMLX target or
`api.anthropic.com`; `HEADROOM_UPSTREAM_ALLOWED_HOSTS` entries count for
both.

Origin rules:
- a bare designation (`api.openai.com`, or a scheme-less configured URL)
means `https` on port 443;
- configured provider API URLs (`OPENAI_TARGET_API_URL`,
`ANTHROPIC_TARGET_API_URL`, oMLX, Gemini/Vertex/Cloud Code/Bedrock)
designate their own origin, e.g. `http://omlx.internal:8888`;
- `HEADROOM_UPSTREAM_ALLOWED_HOSTS` entries may be a bare host,
`host:port` (https) or a full URL (`http://gateway.internal:4000`).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update

## Changes Made

- `headroom/proxy/upstream_trust.py`: `_origin()`,
`trusted_upstream_origins()`, `is_trusted_upstream` matches on origin;
malformed ports and bracketed hosts are refused instead of raising; the
withheld-headers warning is keyed and logged by origin
(`host=http://gateway.internal:4000`) so a listed bare host reached over
http still warns; the now-unused host-only helpers
`trusted_upstream_hosts` / `_env_allowed_hosts` are removed (only one
test used them).
- `merge_extra_headers` / `is_trusted_upstream` take `provider`; the
OpenAI chat and responses handlers pass `"openai"`, the Anthropic
handler (`_build_outbound_anthropic_headers`) passes `"anthropic"`.
Without `provider` the union of all targets is used, as before.
- Docs: `configuration.mdx` (designation rules, example, upgrade note),
`security-model.mdx`, `pipeline-extensions.mdx`.
- Tests: new `tests/test_upstream_trust_origin.py` (8 cases, through
`merge_extra_headers` and `is_trusted_upstream`);
`tests/test_proxy/test_omlx_routing.py` asserts through
`is_trusted_upstream` instead of the removed helper.

**Sibling PRs:** `fix/upstream-idna-pin-once`,
`fix/upstream-guard-off-loop` and `fix/openai-base-url-reject` change
how `handlers/openai.py` resolves the URL passed to
`merge_extra_headers`; this PR only changes `upstream_trust.py`, so
there is no textual overlap and it can merge in any order.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# New tests on origin/main (c21b0813)
FAILED tests/test_upstream_trust_origin.py::test_secret_extra_headers_are_not_merged_for_another_scheme_or_port
FAILED tests/test_upstream_trust_origin.py::test_a_configured_api_url_designates_its_own_origin
FAILED tests/test_upstream_trust_origin.py::test_a_scheme_less_configured_api_url_means_https
FAILED tests/test_upstream_trust_origin.py::test_allowed_hosts_entries_designate_one_origin[llm.corp-...]
FAILED tests/test_upstream_trust_origin.py::test_allowed_hosts_entries_designate_one_origin[llm.corp:8443-...]
FAILED tests/test_upstream_trust_origin.py::test_allowed_hosts_entries_designate_one_origin[http://llm.corp:8080-...]
FAILED tests/test_upstream_trust_origin.py::test_malformed_destinations_are_not_trusted[https://[api.openai.com]/v1]
FAILED tests/test_upstream_trust_origin.py::test_malformed_destinations_are_not_trusted[https://api.openai.com:notaport/v1]
============================== 8 failed in 1.13s ===============================

# This branch
============================== 8 passed in 1.67s ===============================

# Every test file that mentions extra_headers / UPSTREAM_ALLOWED_HOSTS / upstream_trust (24 files)
=========== 571 passed, 12 skipped, 8 warnings in 166.82s (0:02:46) ============
# after review follow-ups (origin-keyed warning, helper removal)
tests/test_upstream_trust_origin.py tests/test_proxy/test_omlx_routing.py tests/test_upstream_credential_scoping.py
tests/test_proxy/test_factory_passthrough.py tests/test_grok_shared_proxy_call_sites.py tests/test_header_isolation.py
======================= 145 passed, 1 warning in 37.22s ========================

$ ruff check / ruff format --check (0.17.3) on changed files
All checks passed!
$ mypy headroom/proxy/upstream_trust.py --ignore-missing-imports
Success: no issues found in 1 source file
```

### Per-provider scoping (follow-up commit 37ff98d0)

```text
# 3 new tests in tests/test_upstream_trust_origin.py, before the commit
FAILED tests/test_upstream_trust_origin.py::test_each_provider_extras_stay_with_that_provider
FAILED tests/test_upstream_trust_origin.py::test_an_allowed_host_is_designated_for_both_providers
FAILED tests/test_upstream_trust_origin.py::test_the_handlers_scope_extras_to_their_own_provider
# after
============================== 11 passed in 0.71s ==============================
# every test file mentioning extra_headers / UPSTREAM_ALLOWED_HOSTS / upstream_trust
============ 574 passed, 12 skipped, 8 warnings in 86.12s (0:01:26) ============
```

Full app (`create_app`, upstream replaced by an httpx MockTransport that
records headers; `openai_api_url=https://gateway.example.invalid`,
OpenAI extra `x-operator-oai`, Anthropic extra `x-operator-ant`):

```text
before:
chat-xprov base=https://api.anthropic.com        -> 200 oai=OAI-MARKER ant=-
resp-xprov base=https://api.anthropic.com        -> 200 oai=OAI-MARKER ant=-
msgs-xprov base=https://api.openai.com           -> 200 oai=-          ant=ANT-MARKER
msgs-xprov base=https://gateway.example.invalid  -> 200 oai=-          ant=ANT-MARKER
after:
chat-xprov base=https://api.anthropic.com        -> 200 oai=-          ant=-
resp-xprov base=https://api.anthropic.com        -> 200 oai=-          ant=-
msgs-xprov base=https://api.openai.com           -> 200 oai=-          ant=-
msgs-xprov base=https://gateway.example.invalid  -> 200 oai=-          ant=-
```
Same-provider cases are unchanged (chat to the OpenAI gateway and
messages to `https://api.anthropic.com` still carry their own extras).

## Real Behavior Proof

- Environment: macOS, Python 3.12, `python -m headroom.cli proxy
--no-optimize` from origin/main and from this branch with
`HEADROOM_UPSTREAM_ALLOWED_HOSTS=localtest.me`,
`HEADROOM_ALLOWED_BASE_URLS=localtest.me`,
`OPENAI_TARGET_API_HEADERS='{"X-Operator-Key": "operator-secret"}'`;
`localtest.me` resolves to 127.0.0.1 where a local server records
request headers.
- Exact command / steps: `curl
http://127.0.0.1:$PORT/v1/chat/completions -H 'x-headroom-base-url:
http://localtest.me:$REC/v1' ...` (plain http, non-443 port of the
designated bare host).
- Observed result:
  ```text
  == origin/main
  HTTP 200
recorder: /v1/v1/chat/completions auth= Bearer client-key
x-operator-key= operator-secret
  == this branch
  HTTP 200
recorder: /v1/v1/chat/completions auth= Bearer client-key
x-operator-key= None
  log: upstream_extra_headers_withheld host=http://localtest.me:21154
  ```
  The request is still proxied; only the operator's header is withheld.
- Not tested: Copilot / Bedrock live upstreams (they pass
`upstream_url=None` or do not merge extras, so they are unaffected).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: **yes, for one setup.** An operator
who listed a bare host in `HEADROOM_UPSTREAM_ALLOWED_HOSTS` and has
clients reach it over plain `http://` or a non-443 port (via
`x-headroom-base-url`) stops sending the configured extra headers there
after upgrading. The request is still proxied, and the proxy logs
`upstream_extra_headers_withheld host=<scheme>://<host>:<port>` once per
origin. Also, a deployment that relied on one provider's extras reaching
the other provider's configured target (for example the Anthropic extras
sent to `OPENAI_TARGET_API_URL` through `/v1/messages`) stops sending
them; list that origin in `HEADROOM_UPSTREAM_ALLOWED_HOSTS` if that is
intended. Remedy for the scheme/port case: list the exact origin, e.g.
`HEADROOM_UPSTREAM_ALLOWED_HOSTS=http://gateway.internal:4000` (or
`gateway.internal:8443` for https on another port). Configured provider
targets keep working because they designate their own origin.
- Kill switch / disable path: the env entry above.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Fix adapted from a proposal by HetCreep.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4858"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-14 10:46:28 +05:30
ab1d0292df fix(proxy): refuse HTTP requests from untrusted browser origins (#4853)
## Description

The provider routes (`/v1/messages`, `/anthropic/v1/messages`,
`/v1/chat/completions`, `/v1/responses`, Bedrock/Vertex routes and the
catch-all passthrough) add operator-held credentials to what they
forward: `*_extra_headers`, the Copilot token
(`apply_copilot_api_auth`), and the provider env key on the
litellm/anyllm backends when the request carries none. Loopback callers
skip `HEADROOM_PROXY_TOKEN`, and a page open in the user's browser
reaches the proxy as a loopback caller. It can send a JSON body as
`Content-Type: text/plain`, which needs no CORS preflight, and the proxy
parses and forwards it. CORS only stops the page from reading the
answer, not the proxy from sending the request. Only the WebSocket
routes checked `Origin`.

The security gate now applies the WebSocket Origin rule to every HTTP
request, before routing. A request is refused with 403 when it carries
an `Origin` that is not:

- a loopback page (`http(s)://127.0.0.1|localhost|[::1]` on any port;
this is the dashboard),
- an origin listed in `HEADROOM_WS_ORIGINS` / `HEADROOM_CORS_ORIGINS`
(the same list `_is_allowed_websocket_origin` honours),
- for a non-loopback caller only, the proxy's own address (a remote
dashboard behind `HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS`). A
loopback caller gets no same-origin pass, since a non-loopback `Host`
with a loopback peer means a page's hostname was pointed at 127.0.0.1.

A cross-site subresource load (`<img>`, `<script>`) sends no `Origin` on
GET, so `Sec-Fetch-Site: cross-site` is refused too, except for a
top-level navigation. Requests without `Origin` (Claude Code, Codex,
Copilot's extension host, SDKs, curl) are unaffected.

Because the check sits in the gate, it covers every backend, including
the litellm/anyllm env-key fallback, without changing how a keyless
request from a local tool is authenticated.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/server.py`: `_is_refused_browser_request(request)` and
a step 0 in `_security_gate` that answers 403 before the token check and
routing. The 403 comes with the usual security headers and a hint naming
`HEADROOM_WS_ORIGINS`, and an `event=proxy_browser_origin_rejected` log
line records the escaped Origin. An origin listed in either
`HEADROOM_WS_ORIGINS` or `HEADROOM_CORS_ORIGINS` is accepted, even when
both are set.
- `tests/test_proxy_browser_origin.py`: text/plain POSTs to
`/v1/messages` and `/v1/chat/completions` from a loopback peer with a
foreign / `null` / look-alike Origin never reach the upstream; without
Origin, from a loopback page, and from a configured origin they still
forward with the operator header; cross-site subresource GET refused;
cross-site navigation still served; loopback peer with a re-pointed
hostname refused.
- `tests/test_proxy_loopback_gating.py`: two existing tests that sent a
foreign `Origin` to `/stats` and `/metrics` now expect the gate's 403
(they already asserted refusal / no metadata; the gate now refuses
earlier).
- `docs/content/docs/security-model.mdx`: describes the rule, and lists
the setups that need an explicit origin: a reverse proxy on the same
host, and app-scheme desktop shells.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# new test file on origin/main (before the fix)
$ pytest tests/test_proxy_browser_origin.py
FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers0-/v1/chat/completions]
FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers0-/v1/messages]
FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers1-/v1/chat/completions]
FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers1-/v1/messages]
FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers2-/v1/chat/completions]
FAILED ...test_foreign_page_request_is_refused_before_the_upstream[browser_headers2-/v1/messages]
FAILED ...test_configured_origin_still_forwards
FAILED ...test_cross_site_subresource_without_origin_is_refused
========================= 8 failed, 5 passed in 11.06s =========================

# with the fix
$ pytest tests/test_proxy_browser_origin.py tests/test_proxy_loopback_gating.py tests/test_proxy_cors.py
======================= 175 passed, 5 warnings in 34.36s =======================

# wider sweep: tests/test_proxy*.py tests/test_proxy/ tests/test_header_isolation.py
#   tests/test_upstream_credential_scoping.py tests/test_openai_codex_routing.py
#   tests/test_dashboard_* tests/test_copilot* (-k "not playwright")
= 12 failed, 2368 passed, 114 skipped, 30 deselected in 483.68s =
# the 12 failures (tests/test_proxy_uds.py x11, one test_proxy_compress_endpoint lossy-inline case)
# fail identically on origin/main in this environment

$ ruff check headroom/proxy/server.py tests/test_proxy_browser_origin.py tests/test_proxy_loopback_gating.py
All checks passed!
$ mypy headroom/proxy/server.py --ignore-missing-imports
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS, Python 3.12. `headroom proxy --host 127.0.0.1
--no-optimize --anthropic-api-url http://127.0.0.1:<L>
--anthropic-extra-headers '{"x-api-key":"OPERATOR_KEY_SENTINEL"}'`, no
proxy token, with a small HTTP listener on `<L>` that logs each
request's `x-api-key` and prompt. One run on origin/main, one on this
branch.
- Exact command / steps: three `curl -X POST
http://127.0.0.1:<port>/v1/messages -H 'Content-Type: text/plain' --data
'<messages JSON>'`, with `Origin: http://page.example`, with `Origin:
http://127.0.0.1:<port>`, and with no Origin.
- Observed result:
  ```text
  == origin/main
  Origin: http://page.example -> HTTP 200
  Origin: http://127.0.0.1:18961 -> HTTP 200
  (no Origin) -> HTTP 200
  listener saw:
{"path": "/v1/messages", "x-api-key": "OPERATOR_KEY_SENTINEL", "prompt":
"request-1 (Origin: http://page.example)"}
{"path": "/v1/messages", "x-api-key": "OPERATOR_KEY_SENTINEL", "prompt":
"request-2 (Origin: http://127.0.0.1:18961)"}
{"path": "/v1/messages", "x-api-key": "OPERATOR_KEY_SENTINEL", "prompt":
"request-3 (X-No-Origin: 1)"}
  == this branch
  Origin: http://page.example -> HTTP 403
  Origin: http://127.0.0.1:18962 -> HTTP 200
  (no Origin) -> HTTP 200
  listener saw:
{"path": "/v1/messages", "x-api-key": "OPERATOR_KEY_SENTINEL", "prompt":
"request-2 (Origin: http://127.0.0.1:18962)"}
{"path": "/v1/messages", "x-api-key": "OPERATOR_KEY_SENTINEL", "prompt":
"request-3 (X-No-Origin: 1)"}
  ```
Dashboard on the same branch: `GET /dashboard` as a cross-site
navigation -> 200; same-origin `POST /settings` -> 200; same-origin `GET
/stats` -> 200.
- Not tested: a real Copilot upstream (the refusal happens before any
handler, so it is the same for every route).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: HTTP requests carrying an `Origin`
that is neither loopback, configured, nor (for a remote caller) the
proxy's own address now get 403. A browser app on another origin that
calls the proxy directly must be listed in `HEADROOM_WS_ORIGINS` (or
`HEADROOM_CORS_ORIGINS`, which also governs CORS). Electron/webview
renderers that send `Origin: null` or an app scheme are refused unless
listed.
- Kill switch / disable path: list the origin in `HEADROOM_WS_ORIGINS`.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

A companion PR (#4846) stops a `*` entry in `HEADROOM_WS_ORIGINS` /
`HEADROOM_CORS_ORIGINS` from accepting every Origin. Both PRs share
`_is_allowed_websocket_origin`, so once both land `*` is ignored on HTTP
too; until then, `HEADROOM_CORS_ORIGINS=*` keeps HTTP open to every
Origin, as it keeps the WebSocket routes today.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-14 10:46:06 +05:30
70ba72138d fix(mcp): check Host and Origin on the Streamable HTTP transport (#4852)
## Description

`headroom mcp serve --transport http` built the MCP SDK's
`StreamableHTTPSessionManager` without `security_settings`. With no
settings, the SDK's transport security middleware turns its Host/Origin
validation off, so the server answered whatever `Host` and `Origin` a
request carried. A page whose hostname is pointed at 127.0.0.1 could
then reach the loopback MCP server from the user's browser and call its
tools (retrieve, compress, stats).

The session manager now gets `TransportSecuritySettings` derived from
the bind the operator asked for:

- loopback bind (`127.0.0.1`, `localhost`, `::1`) or wildcard bind
(`0.0.0.0`, `::`): `Host` must be `127.0.0.1:<port>`, `localhost:<port>`
or `[::1]:<port>`;
- any other bind address: `Host` must be that address and port;
- `HEADROOM_MCP_ALLOWED_HOSTS` (comma-separated `name:port`) adds names
for clients that reach the server through a hostname or a reverse proxy.
A wildcard bind without it logs a warning saying so.

`Origin`, when present, must be `http(s)://` one of the allowed hosts.
MCP clients that send no `Origin` are unaffected.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/ccr/mcp_http.py`: new `transport_security_settings(host,
port)`; `create_streamable_http_session_manager` takes `host`/`port`
(defaulting to the CLI defaults `127.0.0.1:8788`) and passes the
settings; `serve_streamable_http` threads its bind through.
- `tests/test_ccr_mcp_http.py`: the existing round-trip test now
addresses the server by its bind; new tests for a foreign `Host` (421)
and a foreign `Origin` (403), and for `HEADROOM_MCP_ALLOWED_HOSTS` on a
wildcard bind.
- `docs/content/docs/mcp.mdx`: one paragraph on the Host rule and
`HEADROOM_MCP_ALLOWED_HOSTS`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# mcp 2.3.0 (the pinned range is >=2.2.0,<3.0.0)
$ pytest tests/test_ccr_mcp_http.py
============================== 4 passed in 1.29s ===============================

# same request on origin/main vs this branch (in-process ASGI, mcp 2.3.0)
== origin/main
Host=page.example:8788 Origin=http://page.example:8788 -> HTTP 200 session=yes
Host=127.0.0.1:8788 Origin=https://page.example -> HTTP 200 session=yes
Host=127.0.0.1:8788 Origin=- -> HTTP 200 session=yes
== this branch
Host=page.example:8788 Origin=http://page.example:8788 -> HTTP 421 session=no
Host=127.0.0.1:8788 Origin=https://page.example -> HTTP 403 session=no
Host=127.0.0.1:8788 Origin=- -> HTTP 200 session=yes

$ ruff check headroom/ccr/mcp_http.py tests/test_ccr_mcp_http.py
All checks passed!
$ mypy headroom/ccr/mcp_http.py --ignore-missing-imports
Success: no issues found in 1 source file
```

The new tests call `create_streamable_http_session_manager(..., host=,
port=)`, which does not exist on main, so on main they fail with a
`TypeError`; the probe above is the behavioural before/after.

## Real Behavior Proof

- Environment: macOS, Python 3.12, mcp 2.3.0, `headroom mcp serve
--transport http --host 127.0.0.1 --port 18931`.
- Exact command / steps: `curl -X POST http://127.0.0.1:18931/mcp` with
an `initialize` body, `Content-Type: application/json`, `Accept:
application/json, text/event-stream`, and (a) `Host: page.example:18931`
+ `Origin: http://page.example:18931`, (b) `Origin:
https://page.example`, (c) no Origin.
- Observed result:
  ```text
  Host: page.example:18931 / Origin: http://page.example:18931 -> 421
  Host: 127.0.0.1:18931 / Origin: https://page.example -> 403
  Host: 127.0.0.1:18931 / (no Origin) -> 200
  server log: WARNING: Invalid Host header: page.example:18931
              WARNING: Invalid Origin header: https://page.example
  ```
- Not tested: a real remote client through `HEADROOM_MCP_ALLOWED_HOSTS`
(covered by the unit test).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: the HTTP transport now refuses a
`Host` that is not the bind address (421) and a foreign browser `Origin`
(403). The stdio transport (the default) is untouched. Someone serving
on `0.0.0.0` and connecting remotely by hostname or IP must list that
name in `HEADROOM_MCP_ALLOWED_HOSTS`; the server logs a warning on such
a bind.
- Kill switch / disable path: `HEADROOM_MCP_ALLOWED_HOSTS=name:port,...`
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

Fix adapted from a proposal by Anton Dzyatkovskiy.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4852"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-14 10:46:00 +05:30
c21b0813bd fix(compress): stop adding markers the caller did not ask for (#4678)
`/v1/compress` returns a body that someone else's client forwards
unchanged, so anything Headroom adds there reaches the provider. Two
things got added that the caller never asked for.

## Stale-Read retrieval hashes in marker-free modes

Read lifecycle replaces a Read that a later edit made stale with a short
notice. That notice always ended in `Retrieve original: hash=…` and
wrote the original to the CCR store, even in the default and
`lossy_inline` modes. The docs call those modes marker-free, and their
callers have no retrieval tool, so the hash pointed nowhere.

- Marker-free modes keep the notice ("re-read the file for current
content") but drop the hash and skip the store write.
- Lossless routers (proxy `--lossless`,
`ContentRouterConfig(lossless=True)`) skip read lifecycle entirely,
because the notice drops the file's bytes.
- Read maturation now recognises the hashless notice. Otherwise it would
replace the notice with a marker whose hash points at the notice itself.

## Cache breakpoints the client didn't send

On the session path, `normalize_message_cache_control` fell back to
placing `{"type": "ephemeral"}` on the last block message when the
client marked nothing. A caller that chose not to cache got billed cache
writes. `/v1/compress` now passes `place_unmarked=False`: if the client
marked no message, markers are stripped and none are placed. Proxy
handlers keep today's behaviour, and client markers in a misaligned list
still go through the old path.

## Tests

Each new test fails on main.

- `test_marker_free_modes_mark_a_stale_read_without_a_retrieval_hash`,
for the default mode and `lossy_inline`
- `test_lossless_router_keeps_a_stale_read`
- `test_session_turns_add_no_marker_the_caller_did_not_send`
- `test_respects_lifecycle_markers`, extended to the hashless notice

The related suites (gateway, transforms, read maturation, prefix
tracker, cache_control, compress endpoint and session) pass: 1371 passed
and 9 skipped before the review fixes, then 222 of 222 on the touched
files after them. Lint is clean on CI's ruff 0.17.0, and mypy is clean
on the touched modules.

## Not in this PR

- Read maturation (`handlers/anthropic.py`) is not gated on `--lossless`
or `--no-ccr` either. It's the same kind of bug on the proxy path, so it
belongs in a separate fix.
- The "lossless densify only" pipeline in `compress.py` sets
`ccr_enabled=False` without `lossless=True`, so read lifecycle still
runs there.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4678"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Real Behavior Proof

Maintainer qualification on Windows, Python 3.12.13, using an actual
Uvicorn server bound to an owned loopback socket and HTTPX over real
HTTP. The normal `/v1/compress` endpoint and compression pipeline ran
with offline mode and Kompress disabled; no handler, compression stage,
HTTP response or store was mocked. Workspace, configuration, savings and
TOIN paths were isolated. All 243 native inputs and the reused artifact
SHA-256
`5be63a4f31d3e38e7427d28380ecb977616e3f6a3dd6a1c59af5ff6361a05978` were
verified; this was not a fresh native build.

- Exact command / steps: the committed probe is `python -m pytest -q -s
tests/test_compress_runtime_http.py`. The qualifying run used the
maintainer loopback-only launcher with an explicit verified native path
and SHA-256, followed by the four focused
session/API/maturation/lifecycle suites. All 138 tests passed. The probe
starts the actual server, GETs `/health`, and sends nine POSTs to
`/v1/compress`: a Read followed by Edit in default, lossy-inline and
lossless modes, then three session turns without caller cache markers
and three turns with a caller marker. It stops and joins the server in
`finally`.
- Observed result: health and all nine compression requests returned
HTTP 200. Default and lossy-inline emitted stale notices with no
retrieval hash or CCR hashes, saving 3,576 tokens per request. Lossless
preserved the original Read bytes. All three unmarked session turns
returned zero cache markers; all three marked controls retained exactly
one. Session compression saved 7,350 / 15,901 / 25,958 tokens. Server
shutdown joined successfully.
- Before-fix control: the same real-server requests with all six
affected production modules replaced by current main
`200c838c7393d20ae56a0c338f832ed27133c398` failed five assertions:
default and lossy-inline included retrieval hashes, and each unmarked
session turn added a cache marker. Lossless and the three marked
controls passed on main and are not claimed as reproduced regressions.
Candidate source was restored byte-for-byte.
- Not tested: provider forwarding or billing, a live model, deployment
behavior, or the separate proxy read-maturation scope already listed
above. External startup connection attempts were blocked by the
qualifying launcher; this is not a zero-attempt network claim. No author
readiness or self-review declaration is added by this maintainer
evidence.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-13 09:39:10 -06:00
ParideboyandJD Davis c05fbdddc8 fix(proxy): let an extension declare its install() fail-closed (#4737) (#4773)
## Description

`plugins/headroom-oauth2/SPEC.md:67` promised that a missing or invalid
config at
startup makes `install()` raise and the proxy **fail closed** ("won't
start
mis-auth'd"), and its `CHANGELOG.md:25` said "Fails closed on
misconfiguration".

`install()` does raise. But the loader that calls it, `install_all()` in
`headroom/proxy/extensions.py`, wrapped every `install(app, config)` in
`except Exception`, logged a warning, and continued. So with `oauth2`
enabled and
a bad config the proxy started **without** `OAuth2Middleware` — and
since that
middleware is what replaces the client's `Authorization` with the minted
bearer,
the proxy relayed the client's own `Authorization` header upstream
unchanged.
The only signals were a log warning and one stderr line.

Skip-and-continue is the right default and stays the default: an
optional plugin
must not brick the proxy, and a licence-gate failure should leave the
feature
absent rather than fatal. It is the wrong rule for an extension that
*is* the
upstream credential. That is a per-extension property, so the extension
declares
it:

```python
def install(app, config): ...
install.fail_closed = True
```

`install_all` re-raises instead of swallowing when the failing `install`
carries
the marker, so `create_app` raises and the process never serves.
Everything
unmarked keeps today's behaviour exactly, including extensions installed
before
the failing one (none after it run). Reading the marker off the loaded
callable
costs the extension one line and core no per-name allowlist; an older
core that
does not know the marker ignores it and behaves as before — the plugin
documents
that.

Closes #4737

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/extensions.py`: new `fail_closed` marker on the
`install`
callable; `install_all` re-raises a marked extension's failure (chained
from
the original exception) instead of logging and skipping. Module
docstring
gains a "Failing closed" section covering when the marker is
appropriate.
- `headroom/proxy/server.py`: the comment at the `install_all` call site
said the
process never fails on an extension install; it now names the exception.
- `plugins/headroom-oauth2/src/headroom_oauth2/__init__.py`: set
`install.fail_closed = True`, with a comment on why skipping would be
worse
  than not starting.
- `plugins/headroom-oauth2`: `SPEC.md` failure-mode row now matches
reality (plus
a paragraph on the marker and old cores), `README.md` gains a
**Startup** note,
  `CHANGELOG.md` 0.2.0 entry, version bumped in `pyproject.toml` +
  `__version__`.
- `docs/content/docs/oauth2-upstream.mdx`: one line under **Notes**.
- `CHANGELOG.md` is **not** touched — release-please owns it.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```
$ pytest tests/test_proxy_extensions.py -q
6 passed

$ PYTHONPATH=.:plugins/headroom-oauth2/src pytest plugins/headroom-oauth2/tests/test_oauth2.py -q
41 passed in 10.81s

$ ruff check headroom/proxy/extensions.py headroom/proxy/server.py \
    tests/test_proxy_extensions.py plugins/headroom-oauth2/
All checks passed!

$ ruff format --check <same files>
11 files already formatted

$ mypy headroom   # 19 pre-existing errors in unrelated modules (astgrep, mcp_server,
                  # model_metadata) from this Windows/mypy version; none in extensions.py
```

Both new tests fail without the fix, for the right reason:

```
$ git stash push -- headroom/proxy/extensions.py
$ pytest tests/test_proxy_extensions.py plugins/headroom-oauth2/tests/test_oauth2.py -q
FAILED tests/test_proxy_extensions.py::test_install_all_aborts_startup_when_fail_closed_extension_fails
FAILED plugins/headroom-oauth2/tests/test_oauth2.py::test_install_all_aborts_startup_on_bad_config
2 failed, 45 passed
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.11, this PR's branch checked out,
`PYTHONPATH` pointing at the checkout and `plugins/headroom-oauth2/src`
(no
package install needed). No proxy request was sent; the extension's own
`OAuth2Middleware` behaviour is already covered by its existing ASGI
tests.
- Exact command / steps: the reporter's own script shape — real
`install_all`,
real `headroom_oauth2.install`, config from `os.environ`, `discover()`
replaced
  by a one-entry list — plus a second script that goes through the real
  `create_app(ProxyConfig(proxy_extensions=["oauth2"]))`.

  ```python
os.environ["HEADROOM_OAUTH2_TOKEN_URL"] = "http://idp.example.com/token"
  os.environ["HEADROOM_OAUTH2_CLIENT_ID"] = "c"
  os.environ["HEADROOM_OAUTH2_CLIENT_SECRET"] = "s"
  ext.discover = lambda: iter([("oauth2", headroom_oauth2.install)])
  create_app(ProxyConfig(optimize=False, cache_enabled=False,
rate_limit_enabled=False, cost_tracking_enabled=False,
                         proxy_extensions=["oauth2"]))
  ```

- Observed result: `create_app` raises, and with a valid config installs
the
  middleware. After the patch:

  ```
ok misconfigured: create_app raised RuntimeError: proxy extension
'oauth2' failed
      to install and is marked fail_closed, so the proxy will no...
ok valid config: app.user_middleware = ['ActivityMiddleware',
'WebSocketAuthMiddleware',
'BaseHTTPMiddleware', 'RequestBodyLimitMiddleware', 'OAuth2Middleware',
'BaseHTTPMiddleware', 'CORSMiddleware',
'WebSocketProjectPrefixMiddleware']
  ```

  With the plugin's own message preserved as `__cause__`:

  ```
install_all raised: RuntimeError: proxy extension 'oauth2' failed to
install and is
marked fail_closed, so the proxy will not start: headroom-oauth2
misconfigured:
token_url must be https (loopback http allowed for tests; set
allow_insecure=True /
  HEADROOM_OAUTH2_ALLOW_INSECURE=1 to override)
__cause__: RuntimeError headroom-oauth2 misconfigured: token_url must be
https (...)
  proxy would NOT start (create_app raises before uvicorn serves)
  ```

  Before the patch (same script, `extensions.py` + plugin stashed):

  ```
[headroom] proxy extensions SKIPPED: oauth2 (install failed - running
without them; see logs)
  FAIL: create_app returned; proxy would start without the middleware
  ```

  which is exactly the issue's reported behaviour.

- Not tested: a real `headroom proxy` process start through uvicorn with
a
live IdP, and the request path at the wire level (the client's
`Authorization`
reaching an upstream). A multi-worker start (`--workers N`) was not
exercised
either; `create_app_from_env` goes through the same `create_app`, so
each
  worker that cannot install the extension should exit.
  `tests/test_proxy_extension_ordering.py` and two
`test_scope_and_proxy_token.py` cases fail on this machine with or
without the
patch (they need the compiled Rust core, which is not built here) —
verified
  pre-existing by stashing the patch.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. No flag, no staged rollout.
- Minimum rollout channel: any; the marker is inert unless an extension
sets it.
- Stable/default behavior changed: **no** for core — an unmarked
extension that
  raises is still skipped with the same warning and stderr line. For
`headroom-oauth2` 0.2.0, yes and deliberately: a bad config now stops
startup.
- Kill switch / disable path: drop `--proxy-extension oauth2` /
`HEADROOM_PROXY_EXTENSIONS`, or fix the config. Both need no code
change.
- Unsafe override required: no. `HEADROOM_OAUTH2_ALLOW_INSECURE=1` still
lets an
  operator opt into a non-loopback `http` token URL explicitly.
- Qualification impact: none beyond `headroom-oauth2` 0.2.0's own notes.
- Rollback path: revert the commit, or pin `headroom-oauth2<0.2.0` (its
marker is
ignored by any core, so downgrading the plugin alone restores old
behaviour).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

Two decisions a reviewer may want to weigh in on:

1. **Marker vs. core allowlist.** Reading `fail_closed` off the entry
point means
core guarantees nothing about *which* extensions are security-critical,
and a
malicious/broken plugin can also opt into aborting startup. The
alternative was
   a hard-coded list of vetted names in core, which makes every new
security-critical extension a core change. Happy to switch if
maintainers
   prefer that trade.
2. **Not covered here:** a `fail_closed` extension that is *not
discovered at
all* (typo in the name, package not installed) is still only a warning,
and
that has the same fail-open effect. Closing it needs a way to know the
intended
marker for a name that cannot be loaded — the allowlist in (1), or an
env
opt-in — so I left it out of scope and mention it here rather than
inventing a
   mechanism.

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-11 23:05:55 -06:00
brcebnandJD Davis 9624d40d5e docs(claude): keep Remote Control behind Headroom via an HTTPS_PROXY route (#1779) (#4575)
## Description

Claude Code v2.1.196+ disables Remote Control (`/remote-control`, `/rc`:
a local session mirrored to claude.ai/code and the Claude mobile apps)
whenever `ANTHROPIC_BASE_URL` names a host other than
`api.anthropic.com`. The check is client-side and compares the host
only, so every standard Headroom setup
(`ANTHROPIC_BASE_URL=http://127.0.0.1:8787`) loses Remote Control, and
the only documented fix so far is "run Claude without Headroom for those
sessions".

The troubleshooting page already named the route that remains: leave
`ANTHROPIC_BASE_URL` unset and reach Headroom through an `HTTPS_PROXY`
front end with a local CA trusted through `NODE_EXTRA_CA_CERTS` (the
enterprise proxy path Claude Code supports). It was left as an open item
on #1779. This PR ships that route, tested end to end with a real Claude
Code session steered from the Claude iOS app while every request went
through Headroom.

How it works:

```text
Claude Code --HTTPS_PROXY--> mitmdump 127.0.0.1:8790
    api.anthropic.com  /v1/messages*  -> Headroom 127.0.0.1:8787 -> api.anthropic.com
    api.anthropic.com  other paths    -> api.anthropic.com (unchanged)
    any other host (claude.ai, OAuth) -> raw CONNECT tunnel, never decrypted
```

- `ANTHROPIC_BASE_URL` is unset, so the gate passes and subscription
auth is untouched.
- mitmproxy decrypts `api.anthropic.com` only (`--allow-hosts`). The
Remote Control bridge (`claude.ai`) is never decrypted.
- Only the Messages API goes to Headroom. Feature flags, telemetry and
the rest of the API keep going straight to Anthropic.

One trap found while setting it up, and handled here: the Headroom proxy
is often started by the `headroom init hook ensure` hook, so it inherits
Claude Code's environment (the running proxy had `CLAUDECODE=1`). Its
httpx client honours `HTTPS_PROXY`, so its upstream calls would come
back through mitmproxy and loop. The docs tell users to prefix the hook
with `env -u HTTPS_PROXY -u HTTP_PROXY -u NODE_EXTRA_CA_CERTS`. As a
second guard, the addon marks the requests it redirects
(`x-rc-proxy-hop`) and sends a marked request straight to Anthropic. The
marker deliberately avoids the `x-headroom-` prefix: Headroom strips
those headers before forwarding upstream (`_strip_internal_headers`),
which would erase the marker. My first local version used
`x-headroom-rc` and the new test catches exactly that.

Refs #1779

## Type of Change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `examples/claude_code_remote_control/route_to_headroom.py`: the
mitmproxy addon (redirect `/v1/messages*` to Headroom, loop guard, SSE
streaming, `HEADROOM_RC_UPSTREAM` override). mitmproxy is imported only
for type checking, so it is not a Headroom dependency.
- `examples/claude_code_remote_control/README.md`: setup (macOS /
Linux), Claude Code settings, starting Headroom without the proxy
variables, verification, caveats, rollback.
- `docs/content/docs/troubleshooting.mdx`: the Remote Control section
offers this route next to "launch Claude without Headroom", and the
"What remains" paragraph now points to it.
- `examples/README.md`: index entry.
- `tests/test_issue_1779_https_proxy_route.py`: routing rules pinned
with stand-in flow objects.

No Headroom runtime code changes, no new dependency.

## Testing

- [x] Unit tests pass (`pytest`) — the new file plus the existing #1779
gate tests
- [x] Linting passes (`ruff check .`) — on the changed Python files,
ruff 0.17.0 (the pinned version)
- [ ] Type checking passes (`mypy headroom`) — N/A, no change under
`headroom/`
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest -q tests/test_issue_1779_https_proxy_route.py tests/test_issue_1779_remote_control_gate.py
64 passed in 0.77s

# Loop-guard test with the marker renamed to x-headroom-rc (proves it catches the trap):
FAILED tests/test_issue_1779_https_proxy_route.py::test_hop_marker_survives_headroom_header_stripping
1 failed, 9 passed

$ ruff check <changed files> && ruff format --check <changed files>
All checks passed!
```

Smoke test of the addon in a real `mitmdump` (12.3.1) with Headroom
running on 8787:

```text
POST https://api.anthropic.com/v1/messages  -> "server connect 127.0.0.1:8787", "Streaming response from 127.0.0.1"
GET  https://api.anthropic.com/v1/models    -> "server connect api.anthropic.com:443" (not redirected)
GET  https://claude.ai/ (no custom CA given) -> TLS handshake succeeds, i.e. tunnelled, not intercepted
```

## Real Behavior Proof

- Environment: macOS (Darwin 27), Claude Code 2.1.312 (native install,
claude.ai Max subscription), Headroom 0.37.0 (`headroom proxy --mode
token --backend anthropic`, started by the `headroom init hook ensure`
hooks), mitmproxy 12.3.1 run as a LaunchAgent.
- Exact command / steps: `~/.claude/settings.json` `env` =
`HTTPS_PROXY=http://127.0.0.1:8790`, `NO_PROXY=localhost,127.0.0.1,::1`,
`NODE_EXTRA_CA_CERTS=~/.mitmproxy/mitmproxy-ca-cert.pem`,
`ANTHROPIC_BASE_URL` removed, `remoteControlAtStartup: true`, both
headroom hook commands prefixed with `env -u HTTPS_PROXY -u HTTP_PROXY
-u NODE_EXTRA_CA_CERTS`. Started a new `claude` session.
- Observed result: `/status` shows the proxy; answers stream normally;
the mitmdump log shows one `127.0.0.1:8787` request per turn and no
repeats; the session shows up in the Claude iOS app (Code tab) and
answers messages sent from the phone; Headroom's savings keep counting
those requests.
- Not tested: Linux and Windows (the README gives the Linux equivalent
but it was not run), the VS Code extension and Claude Desktop sessions,
`headroom wrap claude` (which sets `ANTHROPIC_BASE_URL` itself, so it is
not compatible with this route).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: no, opt-in recipe; nothing changes
for existing users
- Kill switch / disable path: restore `ANTHROPIC_BASE_URL`, remove the
three variables, stop mitmdump
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the commit (docs, example and test only)

## Review Readiness

- [x] I performed a self-review
- [ ] PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

- Possible follow-up, not in this PR: `remote_control_gate_message` /
`remote_control_gate_short_message` still only suggest running Claude
without Headroom. They could link to this section. I left them untouched
to keep this PR docs-only and avoid changing strings the #1779 / #3426
tests pin.
- Security trade-off, stated in the README: the user trusts a locally
generated CA for `api.anthropic.com` only; the CA key stays on the
machine (`~/.mitmproxy`). Everything else is tunnelled.


### Maintainer current-main requalification

Exact head `b9d7a8a9e72a87ebdc69e1cbcb9e00df3265a22a`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Mechanically updated from current upstream main. The addon, host/path
selection, hop-loop guard, header handling and streaming behavior are
unchanged. The maintainer documentation fix addresses the new Devin
note: remove ANTHROPIC_BASE_URL from all active settings scopes,
explicitly including project .claude/settings.json and
.claude/settings.local.json, while retaining other entries/hooks. Source
inspection confirms project-scoped headroom init claude writes the
variable to settings.local.json, so removing only the user/shell setting
is insufficient. The setup and caveat now cover that case.

Both issue-1779 suites pass (64 tests), plus ten shared Gemini fixture
cases after the test-only repair for independently verified main
failures. Changed Python Ruff/format and whitespace checks pass. No new
runtime dependency or contributor routing logic was introduced. The
author's existing macOS/native Claude/Remote Control/mitmproxy
end-to-end evidence and exclusions were read; these independent Windows
checks cover routing contracts and documentation accuracy, not a live
macOS/Linux/Windows Remote Control session or a fresh native build.
Author readiness attestations remain unchanged; nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `c5e1ea2afcbd7e4c91a75869706c6f68f08f7e3a`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final main synchronization with upstream
c01cf07d94 after all six Dependabot
merges. The example addon and routing/gate test inputs are unchanged
from the previously approved branch; reviewed the additive
troubleshooting documentation overlap. Actual isolated Click8.5.0 and
the fresh combined native DLL:20 HTTPS-proxy-route/Gemini tests plus57
Remote Control gate tests pass; Ruff and whitespace pass. Initial gate
setup errors were caused by C: temp-drive exhaustion; isolated workspace
temp on G: passes with no source workaround or weakened assertion.
Existing scoped macOS/native Claude Remote Control proof remains
applicable to the unchanged addon; no fresh Windows
live-Remote-Control/provider claim. No contributor settings policy
rewritten or readiness attestations changed. No new source finding;
final approval awaits exact-head hosted qualification. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `5400c2c4fd86922f8e2f76d2752c9a5e1bdbe6f3`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Requalified against upstream main
2b0315b2d4. All 71 integration patches
match the main delta after diff-location normalization. The HTTPS-proxy
addon and Remote Control route/gate inputs are byte-identical to the
preceding reviewed head. The docs explicitly cover user, project and
project-local settings, preserving other environment entries and hooks;
this addresses the prior Devin project-scope finding.

All 208 HTTPS-route, Remote Control gate, Gemini and doctor tests pass
using isolated temporary storage on G:. Ruff lint/format passes for all
33 changed Python files; offline lock and whitespace checks pass. The
existing native DLL matches its recorded build inputs and checksum; no
fresh build is claimed. Earlier scoped macOS/native Claude Remote
Control evidence remains applicable to the unchanged addon. No fresh
Windows live Remote Control, cloud/provider or mobile-client
qualification is claimed. Current author and Devin discussion
considered; no new blocker found. Exact-head hosted qualification is
pending before approval. Maintainers handle CI; nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-11 09:54:43 -06:00
Mansurshakh JaparovandJD Davis 0f3b0d8a21 fix(memory): resolve pi cwd before injecting project tools (#4618)
## Description

Closes #4442.

pi's system `<cwd>` block now resolves the memory project. If project
scope is still unresolved with the `empty` fallback, the Anthropic
handler no longer advertises unusable memory tools, including tools
recorded earlier in the session.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Parse pi cwd only in system text. Keep main's line parser, header/CLI
precedence and limited user-message fallback.
- Reuse `is_project_unresolved` for context and sticky-tool replay.
Client tools and global/user storage behavior stay intact.
- Add initialized-handler scope regressions, replace
source-inspection/forwarding-only checks, and document project scope in
the published memory page.

## Testing

- [x] Unit tests pass (`pytest`) — related suite and the normal critical
gate, not the entire Python suite locally.
- [x] Linting passes (`ruff check .`) — hosted lint passed on 5ace63e2;
local filesystem limitation below.
- [x] Type checking passes (`mypy headroom`) — normal full hook with its
existing `--ignore-missing-imports` setting.
- [x] New tests added for the scope regression
- [x] Manual testing performed

### Test Output

```text
# 16 related test files
329 passed, 12 skipped, 1 warning in 43.07s

# ordinary pre-push: make ci-precheck
178 passed, 5 skipped, 1 warning in 10.87s
✅ ci-precheck PASSED — safe to push.

uv run ruff format --check .
1860 files already formatted
```

Normal commit hooks (Ruff, formatter, full mypy, commitlint) and the
complete pre-push path passed: Rust fmt/clippy/workspace tests, native
build/import verification, critical Python tests and commitlint. The
existing Starlette test-client warning was not filtered.

Local whole-tree Ruff reports EXE002 on five **unchanged** OAuth plugin
files: my `fuseblk` checkout exposes them as executable despite Git
modes `100644`; `chmod` returns zero without changing the mode. The
uncached check agrees. I did not change those files, add shebangs or
ignore the rule. Hosted Ruff, formatting and full mypy passed on
5ace63e2; the local filesystem limitation remains.

The first full CI caught a missing scope predicate on the existing
lifecycle test double. Fixed in
[5ace63e2](https://github.com/headroomlabs-ai/headroom/commit/5ace63e2cbd2295f9594d1e0442a679aad99979c);
its complete file passes (`5 passed, 1 warning in 13.01s`). This
follow-up changes no production code or assertions and adds no runtime
fallback. The behavior proof below ran on 7aeb83cf with the same
production source; that qualification is historical; current-head checks
are recorded below.

### Author follow-up: 80453a03

Moved the project-scope guidance out of the unpublished wiki and into
`docs/content/docs/memory.mdx`; this commit changes no production code
or tests. Viewed the actual Next.js `/docs/memory#proxy-project-scope`
page and its table-of-contents entry; no browser errors.

The existing router/scope tests pass: 65 passed, with four SQLite
ResourceWarnings. All seven pi-parser lines and all six branch edges are
covered, so I did not add duplicate tests for the old Codecov
percentage.

Normal commit and pre-push hooks passed, including Rust checks, native
build/import, 180 critical Python tests (5 skipped, 1
tokenizer-approximation warning), and commitlint. My first push timed
out with the wrong local ONNX path; the next found missing pip in the
fresh uv environment. Both setup issues were fixed without bypassing
checks. Current-head hosted CI is not yet qualified.

## Real Behavior Proof

- Environment: Linux x86-64, Python 3.13.14, freshly built native core,
real TCP/Uvicorn proxy and SQLite/ONNX memory backend. Only the upstream
is a local Anthropic-shaped HTTP server; no provider charges.
- Exact command / steps: after `uv sync --locked --extra dev --extra
proxy --extra memory` and `make verify-rust-core`, run this from the
checkout. It prints the actual loaded native module/hash and leaves a
disposable `.headroom-4442-*` directory for inspecting SQLite. Both HTTP
listeners shut down before exit.

<details>
<summary>Standalone HTTP/SQLite reproduction</summary>

```bash
./.venv/bin/python - <<'PY'
import hashlib,http.client,json,os,socket,tempfile,threading,time
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from pathlib import Path
import uvicorn
import headroom._core as core
assert Path(core.__file__).resolve().parent == Path.cwd().resolve()/"headroom"
native_sha=hashlib.sha256(Path(core.__file__).read_bytes()).hexdigest()
from headroom.memory.storage_router import ProjectResolver,RequestContext,extract_system_prompt
from headroom.proxy.server import ProxyConfig,create_app
from headroom.proxy.memory_handler import MEMORY_TOOL_NAMES
from headroom.proxy.helpers import get_session_tool_tracker
root=Path(tempfile.mkdtemp(prefix=".headroom-4442-",dir=Path.cwd()))
project_a=root/"pi%2Fproject-a";project_b=root/"project-b"
project_a.mkdir(exist_ok=True);project_b.mkdir(exist_ok=True)
os.environ["HEADROOM_WORKSPACE_DIR"]=str(root/"workspace")
os.environ["HEADROOM_BEACON"]="off"
os.environ["HEADROOM_TELEMETRY"]="off"
os.environ["NO_PROXY"]="127.0.0.1,localhost"
forwarded=[]
class Gateway(BaseHTTPRequestHandler):
    def do_POST(self):
        body=json.loads(self.rfile.read(int(self.headers["Content-Length"])))
        forwarded.append({"path":self.path,"body":body})
        payload=json.dumps({"id":"msg_local_scope","type":"message","role":"assistant","model":"claude-haiku-4-5","content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn","stop_sequence":None,"usage":{"input_tokens":10,"output_tokens":1}}).encode()
        self.send_response(200);self.send_header("Content-Type","application/json");self.send_header("Content-Length",str(len(payload)));self.end_headers();self.wfile.write(payload)
    def log_message(self,*args):pass
upstream=ThreadingHTTPServer(("127.0.0.1",0),Gateway)
ut=threading.Thread(target=upstream.serve_forever,daemon=True);ut.start()
config=ProxyConfig(mode="cache",anthropic_api_url=f"http://127.0.0.1:{upstream.server_port}",memory_enabled=True,memory_db_path=str(root/"legacy.db"),traffic_learning_enabled=True,subscription_tracking_enabled=False,cost_tracking_enabled=False,cache_enabled=False,rate_limit_enabled=False,log_requests=False,image_optimize=False,ccr_inject_tool=False,ccr_handle_responses=False,ccr_context_tracking=False,anthropic_pre_upstream_memory_context_timeout_seconds=30.0)
app=create_app(config)
sock=socket.socket();sock.bind(("127.0.0.1",0));sock.listen(128)
port=sock.getsockname()[1]
server=uvicorn.Server(uvicorn.Config(app,log_level="warning",lifespan="on",timeout_graceful_shutdown=3))
st=threading.Thread(target=server.run,kwargs={"sockets":[sock]},daemon=True);st.start()
try:
    deadline=time.monotonic()+180
    while not server.started:
        if not st.is_alive() or time.monotonic()>deadline:raise RuntimeError("real proxy startup failed")
        time.sleep(.02)
    session="headroom4442-native-project-scope"
    common={"Content-Type":"application/json","x-api-key":"local-unbilled-key","anthropic-version":"2023-06-01","x-headroom-user-id":"native-scope-user","x-headroom-session-id":session}
    client_tools=[{"name":"get_weather","description":"Weather","input_schema":{"type":"object","properties":{}}}]
    rows=[]
    cases=[("fresh_unresolved","You are helpful.",{},False),("pi_cwd",f"<cwd>\n{project_a}\n</cwd>",{},True),("warm_unresolved","You are helpful.",{},False),("pi_cwd_restored",f"<cwd>\n{project_a}\n</cwd>",{},True),("explicit_header_wins",f"<cwd>\n{project_b}\n</cwd>",{"x-headroom-project-id":"explicit-project"},True),("ordinary_user_cwd_ignored","You are helpful.",{},False)]
    for label,system,extra,expected in cases:
        body={"model":"claude-haiku-4-5","max_tokens":16,"system":system,"messages":[{"role":"user","content":f"cwd:\n<cwd>\n{project_b}\n</cwd>" if label=="ordinary_user_cwd_ignored" else "hello scope"}],"tools":client_tools}
        before=len(forwarded)
        conn=http.client.HTTPConnection("127.0.0.1",port,timeout=90)
        conn.request("POST","/v1/messages",body=json.dumps(body).encode(),headers={**common,**extra})
        response=conn.getresponse();payload=response.read();conn.close()
        assert response.status==200,(label,response.status,payload.decode())
        assert len(forwarded)==before+1,(label,"unexpected upstream request count")
        actual=forwarded[-1]["body"]
        names={tool.get("name") for tool in actual.get("tools",[])}
        assert names==({"get_weather"}|MEMORY_TOOL_NAMES if expected else {"get_weather"}),(label,names)
        scope=ProjectResolver().resolve(RequestContext(headers={**common,**extra},system_prompt=extract_system_prompt(body),base_user_id="native-scope-user"))
        rows.append({"case":label,"status":response.status,"memoryTools":sorted(names & MEMORY_TOOL_NAMES),"project":scope,"gatewayPath":forwarded[-1]["path"]})
    assert forwarded[1]["body"]["tools"]==forwarded[3]["body"]["tools"],"restored definitions changed"
    tracker=get_session_tool_tracker()
    assert tracker.should_inject("anthropic",session),"probe did not exercise an actual recorded sticky session"
    from urllib.parse import quote
    key_a=ProjectResolver().resolve(RequestContext(headers={"x-headroom-cwd":quote(str(project_a),safe="/")},system_prompt="",base_user_id="native-scope-user"))[0]
    assert rows[1]["project"][0]==key_a
    assert rows[4]["project"][1]=="explicit-project"
    router=app.state.proxy.memory_handler._router
    assert router is not None,"real memory backend never initialized"
    opened=router.open_backends()
    project_db=root/"memories"/"projects"/key_a/"memory.db"
    assert project_db in opened and project_db.is_file(),("real pi project database not opened",[str(x) for x in opened])
    print(json.dumps({"mode":config.mode,"realHttpListener":f"http://127.0.0.1:{port}","localGateway":f"http://127.0.0.1:{upstream.server_port}","rows":rows,"realStickyGoldenSession":True,"realPiProjectDatabase":str(project_db),"openedDatabases":[str(x) for x in opened],"nativeModule":core.__file__,"nativeSha256":native_sha,"noLivePiOrPaidProvider":True},indent=2))
finally:
    server.should_exit=True;st.join(20)
    upstream.shutdown();upstream.server_close();ut.join(5);sock.close()
    assert not st.is_alive() and not ut.is_alive(),"native proof leaked a server"


PY
```

</details>

- Observed result: unchanged `main` fc1ba891 fails at the first
unresolved request: five proxy memory tools reach upstream. With the
fix, all six requests return HTTP 200:

| Same-session request | Proxy memory tools |
| --- | ---: |
| Fresh, no cwd | 0 |
| pi system cwd | 5 |
| Cwd absent again | 0 |
| Same pi cwd restored | 5 |
| Explicit project header, different system cwd | 5 |
| Cwd block only in ordinary user text | 0 |

The client weather tool stays present. Restored definitions are
structurally equal to the first resolved turn; the explicit header wins.
The literal-percent cwd agrees with its encoded-header form, and the
actual project SQLite database opens at the expected key.

- Not tested: live pi/Caveman, paid-provider cache accounting/billing,
Windows, or other provider protocols. This does not claim to fix Caveman
header forwarding.

## Runtime Rollout Safety

- Rollout-managed feature(s): none added; existing Anthropic memory
injection.
- Minimum rollout channel: unchanged.
- Stable/default behavior changed: pi cwd resolves; unresolved
project/empty scope no longer adds unusable proxy memory tools.
- Kill switch / disable path: existing `--no-memory`; explicit
project/cwd headers still work.
- Unsafe override required: no.
- Qualification impact: no storage migration or new dependency; scoped
HTTP/SQLite behavior checked.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes — related
suite and critical gate, not a whole-suite claim
- [x] I did **not** edit `CHANGELOG.md` — release-please owns it

## Screenshots (if applicable)

N/A; the observable surface is the HTTP payload and project database.

## Additional Notes

AI-assisted; no subagents. OpenAI injection policy is unchanged. Keeping
this draft until the current hosted checks are qualified.

One earlier post-build probe completed the requests and closed both
listeners but hit its whole-process shutdown deadline. The same program
and native artifact later exited normally with faulthandler enabled; its
diagnostic abort timer did not fire. I have not identified the
intermittent cause and am not claiming a shutdown fix. No production
timeout or feature flag was changed to get that result.


The two npm audits now pass on 5ace63e2, along with the other Security
jobs. The earlier [unchanged-base
failure](https://github.com/headroomlabs-ai/headroom/actions/runs/41399323584)
is historical evidence, not a current blocker. This PR changes no
dependency manifests, lockfiles, workflow or audit exceptions.


### Maintainer current-main requalification

Exact head `60597623e00967153d947a46bf197950250f310e`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Clean mechanical update from current upstream main, preserving the pi
system-cwd resolver, request-scoped project guard, main's cache-mode
search skip/logging, and client-tool preservation. Ordinary user cwd
blocks still do not enable the pi fallback; unresolved project scope
still blocks both context injection and replayed memory tools. No
contributor production logic was rewritten and no new production finding
remains in this reviewed scope.

The initial combined tree dropped main's memory-invariant file and two
stream/timeout regression cases. Maintainer test-only repairs restore
those tests rather than reducing coverage: the invariant file is
byte-identical to main, every main assertion in the four
restored/updated test files is preserved by AST comparison, and three
fixtures now supply the handler protocol methods needed for resolved
scope and beta headers. The lifecycle main control passes without that
fixture extension; no claim is made that this particular fixture failure
independently fails main. All 222 affected
scope/invariant/stream/timeout/lifecycle/backpressure/cache-stability/system-prompt
tests pass with one skip. Ten shared Gemini request-fixture cases also
pass after the separate test-only repair for independently verified main
failures. All 12 main-relative changed Python files pass Ruff/format;
whitespace checks pass.

The author's existing real TCP/SQLite/ONNX reproduction and explicit
limits were read and remain in the body. These fresh Windows checks
qualify the combined request-scope regressions; they are not a claim of
rerunning that Linux proof, live pi/client/provider behavior, or a fresh
native build. An existing binary was supplied solely for test startup.
Draft status is preserved; readiness remains the author's decision and
nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `60e01cf7c25e90abb9f5cb66fa221da83e7c5177`, upstream main
`ea59ac3eef3a3dc02076e9fe4bb30d572ca9c1a9`.

Clean integration with upstream/main
ea59ac3eef. Re-reviewed project-scoping
delta against intervening memory/backend/stream changes: pi cwd remains
limited to system context; ordinary user cwd blocks do not enable
fallback; unresolved projects skip context/tool injection and sticky
tool replay while retaining client tools and main cache-mode no-search
behavior.232
scope/invariant/stream/timeout/lifecycle/backpressure/cache/system-prompt/Gemini
tests pass,1optional skip; touched-source Ruff and whitespace pass. No
contributor production policy rewritten. Earlier author TCP/SQLite/ONNX
evidence remains scoped, with no fresh native build or live-pi/provider
claim. Existing draft preserved; final approval awaits exact-head hosted
qualification. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `e93ceb35df7f78bfa2dcc0611609131cf2b08b0a`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 232 passed, 1 skipped, 1 warning in
47.62s. Selected real dependency overlays and native DLL paths are
recorded in final4618-fresh-native-tests.log; the DLL provenance/input
hashes are in final-native-artifact-proof.json. No redundant rerun is
implied after this Cargo-only merge. Whitespace passes. Prior source
review and scoped real-behavior evidence remain applicable to unchanged
contributor inputs.

No new source finding in the reviewed scope. Final formal approval
awaits exact-head hosted qualification. Project resolution and
unresolved-scope injection/replay gates remain unchanged. Existing draft
status is preserved.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `d693dd2b6e33fd47ff853f7e28d938fc1dbc30ac`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Current-main requalification
against2b0315b2d4b0d54947050bbdad2de9e89df06378. Clean source
integration preserves pi systemcwd resolution and unresolved-scope
context/tool/sticky-replay gates.70integration patches exactly matchmain
delta;Anthropic exception patch differs only byclosing-parenthesis diff
alignment and its exception AST equalsmain. Broad
memory/router/projectscope/continuation/backpressure/cache/lifecycle/doctor/nativeops
run:405passed,1skipped,one newmain fake-handler fixture failed
missingis_project_unresolved. Maintainer adds only that fake method
returningFalse,respecting resolvedfixture scope;all39backpressure tests
nowpass,including unchanged
injectedSQLiteexception,HTTP200andsemaphorerelease assertions. No
production logic changed.33changedPythonlint/format,offline
lock/whitespacepass;fixedtestlint/formatrechecked.
Existingpr3982nativeDLLverified fullinputhashes/checksum;no
freshbuildclaimed. Currentauthor/Devin discussion checked,inline
endpoint empty;prior scoped realbehavior evidence remains applicable to
unchanged contributor inputs. No new sourcefinding;no
realprovider/agentlaunch claim. Draftpreserved.
Exactnewheadhostedqualification pending;maintainers
handlechecks,nothingmerged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `0da0b5a01958eaaac1d625227cb4bdb9477c5d3b`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 17 integration patches
exactly match the prior-reviewed-main delta after diff-location
normalization. Contributor pi system-cwd resolution and
unresolved-project memory context/tool/sticky-replay gates are
unchanged. Fresh
memory-router/project-scope/streaming/continuation/backpressure/cache/lifecycle
and incoming Vertex/backend/CLI tests: 381 passed, 12 skipped, 1
warning. Twelve changed Python files pass Ruff lint/format; offline lock
and whitespace pass.

Static production/test scope and security review considered user versus
system cwd inputs, malformed/empty cwd, missing scope and stale tool
replay: tests keep pi cwd tags in ordinary user content from resolving
project scope, and unresolved scope blocks replay. No unrelated
executable, workflow, dependency or setup changes. Flagged inherited
workflow-validation job is excluded by unchanged workflow paths. Author
discussion checked; collected inline endpoint has no findings. Prior
scoped behavior evidence applies to unchanged contributor inputs; no new
real agent/provider launch claimed. Native DLL reused only after
complete tracked input-set/hash and artifact checksum verification, no
fresh build claimed. Draft preserved. Formal approval awaits exact
published-head hosted qualification. Maintainers handle checks; nothing
merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-10 22:28:01 -06:00
Parideboy a385ea8e60 fix(benchmarks): drop the duplicate protect_recent=0 table from the proof harness (#4754)
## Description

`benchmarks/index_proof_table.py` printed two tables, "default
(protect_recent=4)" and "full corpus (protect_recent=0)". The committed
`benchmarks/results/index_proof_table.txt` showed them identical in
every row and in the total, so the second run and its label carried no
information — and the comment above them claimed the default "shields
almost every tool result", which the output contradicts.

They cannot ever differ. `ContentRouter` applies `protect_recent` only
to content detected as `SOURCE_CODE`:

```python
is_code = detection.content_type == ContentType.SOURCE_CODE
if protect_recent > 0 and messages_from_end <= protect_recent and is_code:
```

Every scenario in this harness is a JSON tool payload. All ten payloads
detect as `JSON_ARRAY` under **both** backends (pure-Python and
`HEADROOM_DETECT_BACKEND=rust`), so the guard never fires at any
`protect_recent` value. This is not a stale committed artifact that a
rerun would fix.

Of the three options on the issue, this takes the first: one table with
the limitation stated, rather than a duplicate or a behavior change to
the compressor.

Closes #4732

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update

## Changes Made

- `benchmarks/index_proof_table.py`: print one table (the default
config) and replace the misleading comment with the actual reason the
`protect_recent` comparison is a no-op for JSON payloads.
- `benchmarks/results/index_proof_table.txt`: regenerated. The table
body is unchanged; the duplicate second table is gone.
- `tests/test_index_proof_table.py`: new output-level tests asserting
the harness prints exactly one table and that the docs Markdown block
states the config it was measured under. Both fail on the base revision
and pass with this change.
- `tests/test_transforms_content_router.py`: new
`test_protect_recent_does_not_shield_non_code_tool_payloads` pins the
router reason the two tables could never differ.
- `headroom/compress.py`: `CompressConfig.protect_recent` docstring said
"Don't compress the last N messages"; it guards recent **code** only.
- `docs/content/docs/context-management.mdx`,
`docs/content/docs/proxy.mdx`, `docs/content/docs/index.mdx`,
`README.md`: same overstatement corrected. The published table now names
the configuration it was measured under.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_index_proof_table.py tests/test_transforms_content_router.py -q
109 passed, 1 warning in 14.83s

$ uv run ruff check benchmarks/index_proof_table.py headroom/compress.py \
    tests/test_index_proof_table.py tests/test_transforms_content_router.py
All checks passed!

$ uv run ruff format --check tests/test_index_proof_table.py
1 file already formatted
```

Base-revision check for the new output tests (harness restored to
`318e8e08`):

```text
$ uv run pytest tests/test_index_proof_table.py -q
FAILED tests/test_index_proof_table.py::test_proof_table_prints_exactly_one_table
FAILED tests/test_index_proof_table.py::test_proof_markdown_block_is_the_default_config
2 failed in 5.15s
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.11, `uv` venv at the branch head
`9b7e2f95` (rebased onto `acb58a0c`); content detection ran on the
pure-Python backend (no ONNX Runtime 1.24+), and every payload was
re-checked under `HEADROOM_DETECT_BACKEND=rust`.
- Exact command / steps: `uv run python benchmarks/index_proof_table.py`
on the base revision, then the same command on this branch, then `git
diff` against the committed `benchmarks/results/index_proof_table.txt`.
- Observed result: base revision printed two tables identical in all
four rows and in the total (178,024 → 104,261, 41%). This branch prints
one table with the same numbers, and its output matches the committed
result file byte for byte once the first line is excluded (that line
names whichever detector backend the host has).
- Not tested: whether the committed numbers would change under the
native detector — also left open on the issue. They do not: both
backends classify all ten payloads as `JSON_ARRAY`, so detection routes
them identically.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. Only a benchmark script, its
committed output, tests, and docs change.
- Minimum rollout channel: stable, no gating.
- Stable/default behavior changed: no. `ContentRouter`,
`CompressConfig`, and `compress()` behavior are untouched; only a
docstring and documentation prose changed. The published table's numbers
are identical to before.
- Kill switch / disable path: not applicable.
`benchmarks/index_proof_table.py` is a manual, offline script with no
runtime surface.
- Unsafe override required: no.
- Qualification impact: none. Serialization, caching, and compression
output are unchanged; the new tests are offline and stubbed.
- Rollback path: revert this commit. The harness returns to printing the
duplicate table; no state or migration is involved.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

`mypy headroom` left unchecked: the touched Python files add no type
surface beyond a benchmark script and a test module, and mypy was not
run locally.

Review feedback addressed: the initial version of this PR added only a
router-level test, which passes on the base revision and so could not
catch the duplicate table returning. The two output-level tests in
`tests/test_index_proof_table.py` replace that gap, and the
base-revision failure above is their proof. The `context-management.mdx`
file-read claim was also too broad — read protection releases
confidently non-code data (JSON, CSV/tabular, build/test logs, git
diffs, HTML, search output), so the wording now says so.

`CHANGELOG.md` is untouched, per CONTRIBUTING.


The branch was rebased onto `acb58a0c` (4 upstream commits: #4727,
#4728, #4735, #4743) with no file overlap. Re-verified after the rebase:
109 focused tests pass, `ruff check`/`format --check` clean.
`benchmarks/index_proof_table.py` is byte-identical between `318e8e08`
and `acb58a0c`, so the base-revision failure above still applies
unchanged.
2026-11-10 12:35:38 -06:00
SD 8f04221b2d fix(proxy): turn off the image stage when --compressor leaves out image (#4723)
## Description

`headroom proxy --compressor` / `HEADROOM_COMPRESSORS` lists `image` and
says unselected built-ins are disabled. The selection only sets
`ContentRouterConfig.enable_image_optimizer`, which the proxy does not
consult. The Anthropic and OpenAI handlers gate their image stage on
`ProxyConfig.image_optimize`, and nothing set it from a selection. So
`--compressor smart_crusher` still sent every screenshot through the
image stage.

This PR makes the image gate honor the selection.
`ImageCompressionDecision`, which both handlers already call, reads
`config.compressors` on each request and skips the image stage with a
new reason, `image_not_selected`, when a selection leaves out `image`.
The operator's `image_optimize` is never written, and
`image_optimize=False` still wins. This is Gap 1 from #4613, agreed with
the reporter there. Gap 2 (Windows self-update while a proxy is running)
will be a separate PR.

Refs #4613

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/image_compression_policy.py`:
- `image_compressor_selected(compressors)` follows the same rules as
`_apply_compressor_selection`: no selection, an empty one or `*` keeps
images; otherwise `image` must be named.
- `decide_image_compression` takes `image_selected` (default `True`) and
adds `image_not_selected`, which comes after `image_optimize_disabled`
and before `no_messages`.
- `headroom/proxy/image_compression_decision.py`: `decide` passes the
selection from `config.compressors` and exposes `image_selected` next to
the other inputs it saw. The new reason appears in `image_skip_reason`.
- `tests/test_compressor_selection_image_stage.py`: sends a screenshot
request through the real proxy app with only the upstream faked. It runs
on two routes: Anthropic in token mode, and OpenAI chat in cache mode,
which still compresses images in cache mode. The cases are:
  - a selection without `image` never enters the image stage;
  - a selection with `image`, `*`, and no selection each enter it once;
  - a config reused across two proxies follows the current selection;
  - `image_optimize=False` with `image` selected still skips the stage.
- `tests/test_image_compression_decision.py`: covers the new reason, its
precedence, the wildcard and blank selections, and a config with no
`compressors` attribute.
- `docs/content/docs/proxy.mdx`: the `--compressor` row says that
leaving out `image` also turns off the image stage.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`): see Additional Notes
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

The new tests against `main`'s source (b21479e0):

```text
FAILED tests/test_compressor_selection_image_stage.py::TestImageStageFollowsSelection::test_selection_without_image_skips_the_image_stage[anthropic-token]
FAILED tests/test_compressor_selection_image_stage.py::TestImageStageFollowsSelection::test_selection_without_image_skips_the_image_stage[openai-cache]
FAILED tests/test_compressor_selection_image_stage.py::TestImageStageFollowsSelection::test_reused_config_follows_the_current_selection[anthropic-token]
FAILED tests/test_compressor_selection_image_stage.py::TestImageStageFollowsSelection::test_reused_config_follows_the_current_selection[openai-cache]
4 failed, 8 passed
```

At this head:

```text
$ pytest tests/test_compressor_selection.py tests/test_compressor_selection_image_stage.py tests/test_image_compression_decision.py tests/test_image_compression_policy.py tests/test_router_external_dispatch.py tests/test_proxy_eager_preload_bind.py -q
99 passed

$ pytest <every test file that mentions image_skip_reason, ImageCompressionDecision, image_optimize or compressors> -m "not live" -q
12 failed, 1994 passed, 26 skipped

$ ruff check <changed files> && ruff format --check <changed files>
All checks passed!
```

The 12 failures are in `tests/test_image_compressor.py`,
`tests/test_ccr_batch_processor.py`,
`tests/test_garbled_compression_fixes.py` and
`tests/test_transforms/test_content_router.py`. The same 12 fail on
`main` in my environment (the image ones because the venv has Pillow
12), so they are not from this change.

## Real Behavior Proof

- Environment: macOS (Darwin 24.6.0, arm64), Python 3.12.3, Pillow
13.0.0. Older Pillow makes the proxy forward images unchanged.
- Exact command / steps: the real proxy app in token mode on the
Anthropic route, with the real image compressor and image worker; only
the upstream model is faked, and it answers instantly. Each setting
sends the same 2560x1600 PNG screenshot ("what is the exact error text
in this screenshot?") 5 times. Per request, the script records
image-stage entries, what reached the upstream, and the time the proxy
took. Run once on `main` (b21479e0) and once on this branch.
- Observed result: on `main`, `--compressor smart_crusher` still ran the
image stage on 5 of 5 requests and downscaled every screenshot
(`full_low`, 1,870 → 255 tokens) at about 45 ms each. With this PR it
ran on 0 of 5, the original image went upstream unchanged, and each
request took 4.6 ms. With no selection or with `image` selected, both
builds behave the same. `image_optimize` stays `True` in every run.

| Build | Setting | Image-stage runs | Image sent upstream | Proxy time
p50 |
|---|---|---|---|---|
| `main` | no `--compressor` | 5 of 5 | downscaled (`full_low`, 1,870 →
255 tokens) | 45.4 ms |
| `main` | `--compressor smart_crusher` | **5 of 5** | **downscaled** |
**45.2 ms** |
| `main` | `--compressor smart_crusher,image` | 5 of 5 | downscaled |
46.9 ms |
| this PR | no `--compressor` | 5 of 5 | downscaled | 48.2 ms |
| this PR | `--compressor smart_crusher` | **0 of 5** | **original,
unchanged** | **4.6 ms** |
| this PR | `--compressor smart_crusher,image` | 5 of 5 | downscaled |
47.6 ms |

The first image request of each process also loads the image models
(about 1.0–1.3 s here). With `image` left out, that load doesn't happen.
- Not tested: a live upstream, the OCR (`transcode`) technique (the
router picked `full_low` for this screenshot), and Windows.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: only for a `--compressor` /
`HEADROOM_COMPRESSORS` selection that leaves out `image`. Those proxies
now forward images unchanged, as the help text already says. With no
selection, a selection that includes `image`, or `*`, nothing changes.
- Kill switch / disable path: add `image` to the selection, or remove
the selection
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the PR's commits

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

n/a

## Additional Notes

- Review follow-up: the first revision (619ea4ad) wrote
`image_optimize=False` into the caller's `ProxyConfig`. Now the decision
reads the selection on each request, so a reused config follows its
current selection, and an explicit `image_optimize=False` is kept.
- #1327 proposes a separate `HEADROOM_DISABLE_IMAGE_OPTIMIZER` variable.
This PR doesn't add one; it makes the existing option behave as its help
text says.
- `mypy headroom/proxy/server.py` stops in my local environment on a
`numpy` stub (`Type statement is only supported in Python 3.12 and
greater`) before reaching project code. CI runs mypy.
2026-11-10 10:39:18 -06:00
gglucassandClaude Opus 5.5 3ba422361e fix(cache): save prefix-tracker state when the exit signal arrives (#4715)
## Description

Follow-up to #4299. With `HEADROOM_PREFIX_STATE_FILE` set, the proxy
saves its prefix trackers in `HeadroomProxy.shutdown()`, the last
lifespan step. uvicorn runs the lifespan shutdown only after it has
waited up to `timeout_graceful_shutdown=10` for in-flight requests. So a
supervisor that SIGKILLs sooner loses the save whenever a request is
open at the stop, and the next start re-writes every warm conversation's
cache. Extra Headroom stops the proxy with SIGTERM and SIGKILLs 2s
later; a systemd `TimeoutStopSec` or a container stop timeout under 10s
does the same.

This saves the trackers as the signal arrives. Measured with one request
in flight: main wrote the state file 10.2s after SIGTERM, this branch
0.02s after it.

No existing issue; found while turning the state file on in Extra
Headroom.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `_save_prefix_state_on_exit_signal` (`headroom/proxy/server.py`),
called from `HeadroomProxy.startup()` right after `load_state`, only
when the state file is set. It wraps the SIGINT, SIGTERM and SIGBREAK
handlers already installed (uvicorn's: the lifespan runs inside its
`capture_signals`). The wrapper schedules one `save_state` on the event
loop with `call_soon_threadsafe`, then calls the wrapped handler, so
uvicorn's shutdown is unchanged.
- The save never runs inside the signal handler, so it cannot read a
tracker that a paused coroutine was midway through updating. A signal
with no Python handler (`SIG_DFL`) is left alone, and off the main
thread (TestClient's lifespan) nothing is installed.
- `shutdown()` hands the signals back right after its own save: the
helper returns an undo that restores each handler that is still its
wrapper (one another owner installed since is left alone) and turns the
wrapper into a pass-through, so a lifespan run without uvicorn (an
embedding app, tests) never leaves a wrapper bound to a closed loop.
- The shutdown save stays. It runs later and overwrites the snapshot
with the newer state when the process lives that long; `write_text` is
atomic, so a SIGKILL during that second write leaves the first snapshot.
- Docs: the `HEADROOM_PREFIX_STATE_FILE` row in
`docs/content/docs/configuration.mdx` says when it is saved.
- Tests (`tests/test_prefix_tracker_restart.py`): the signal-time save
lands on the loop after uvicorn's handler got the signal, and only once
for two signals; `SIG_DFL` is left alone; no handler is installed off
the main thread; after two `HeadroomProxy.startup()`/`shutdown()`
lifespans on separate loops SIGTERM reaches the app's own handler; a
wrapper left behind by overlapping lifespans passes through after its
loop closed; undo leaves a handler installed later alone; undo off the
main thread only disarms the wrapper. Those four fail on 2fd4854e.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run --frozen --extra dev pytest tests/test_prefix_tracker_restart.py -q
28 passed in 5.62s
$ uv run --frozen --extra dev pytest tests -q -k "shutdown or lifespan or signal or startup or restart"
292 passed, 14 skipped, 19230 deselected, 2 warnings in 28.63s
$ uv run --frozen --extra dev ruff check .
All checks passed!
$ uv run --frozen --extra dev ruff format --check .
1921 files already formatted
$ uv run --frozen --extra dev mypy headroom/proxy/server.py --ignore-missing-imports
headroom/memory/adapters/embedders.py:389: error: Unsupported operand types for - ("None" and "int")  [operator]
(no error in server.py; embedders.py is untouched)
$ uv run --frozen --extra dev mypy headroom --ignore-missing-imports
Found 3 errors in 2 files (checked 583 source files)
  headroom/memory/adapters/embedders.py:389, headroom/cache/prefix_tracker.py:1397-1398 (files this PR does not touch)
```

## Real Behavior Proof

- Environment: macOS 27 (arm64), this branch at 4783339b against main at
bb8d18a5, the real proxy process (`python -m headroom.cli proxy --port
6799 --no-http2 --no-rate-limit`, `HEADROOM_MODE=token`,
`HEADROOM_ROLLOUT_CHANNEL=beta`, clean `HOME`,
`HEADROOM_PREFIX_STATE_FILE` set) in front of a local fake Anthropic
upstream that reports a cache read.
- Exact command / steps: one `/v1/messages` turn so the proxy holds a
tracker, then a second request that the fake upstream holds for 60s,
then SIGTERM to the proxy's process group 2s later; record when the
state file appears and when the process exits. Repeated with no request
in flight.
- Observed result: with a request in flight, main wrote the state file
10.2s after SIGTERM and this branch 0.02s after it; both exited after
uvicorn's 10s drain, which is unchanged.

  | Stop | State file written | Process exited |
  |---|---|---|
  | main, request in flight | 10.22s | 10.24s |
  | branch, request in flight | 0.02s | 10.24s |
  | branch, idle | 0.03s | 0.14s |

The branch logged `SessionTrackerStore: saved 2 of 2 prefix trackers`
twice in the in-flight run: once on the signal, once from `shutdown()`.
- Not tested: Windows (SIGBREAK, and SIGTERM delivery there), Linux,
uvicorn before 0.29 (handlers installed with `loop.add_signal_handler`;
the lock pins 0.40.0), and a real Claude Code session across the restart
(#4299's proof covers the restore itself).

## Runtime Rollout Safety

- Rollout-managed feature(s): None; this only changes when the opt-in
`HEADROOM_PREFIX_STATE_FILE` save happens.
- Minimum rollout channel: Stable.
- Stable/default behavior changed: No; with the variable unset no
handler is installed and nothing is written.
- Kill switch / disable path: Unset `HEADROOM_PREFIX_STATE_FILE`.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this PR; the shutdown save, the file format and
the restore path are unchanged.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A, no UI change.

## Additional Notes

- `mypy headroom` is unchecked because of the 3 errors above, which are
in files this PR does not touch; `mypy` on `headroom/proxy/server.py`
reports no error in that file.
- Windows: a supervisor that force-kills the process tree (no console
signal) still loses the save; that needs a periodic save, which this PR
does not attempt.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 12:34:42 -06:00
2649835a51 feat(wrap): make the Serena code-memory MCP opt-in (#4509)
## What changes

`headroom wrap claude|codex|grok|opencode` no longer registers Serena by
default.

| | Before | After |
|---|---|---|
| Default | Serena registered | No code-memory MCP |
| Opt in | (default) | `--code-memory serena` or
`HEADROOM_CODE_MEMORY=serena` (deprecated `--serena` still works) |
| Opt out | `--code-memory none` / `--no-serena` | Still accepted |
| Upgrading user, Serena installed by Headroom | Kept | Removed on next
wrap, with the message "Serena is now off by default; pass --code-memory
serena to keep it" |
| Upgrading user, Serena added by hand | Kept | Kept, no message (shown
only with `--verbose` or an explicit opt-out) |

OpenCode now accepts `--code-memory` and uses the same setup path as the
other agents. Its `--no-serena` flag is hidden and deprecated.

## Why

Measured on 2026-10-27, Serena started exactly as `wrap` registers it:

- It adds 22 tools, about 5.9K tokens of tool definitions on every
request where tools aren't deferred. Its server instruction tells the
model to call `initial_instructions`, which costs another 2K tokens.
- Each launch takes about 6 s cold via `uvx`, plus a background `serena
project index` that starts language servers. It also writes a `.serena/`
folder into the user's repo.
- Serena was registered in 6 of our own Claude Code sessions and called
0 times. Separately, an offline replay of about 108K tool calls found
that symbol-level reads have little to recover for Claude Code.

## Not in this PR

- `headroom.integrations.strands` still defaults
`enable_serena_mcp=True`. It's public integration API, so it needs a
deprecation step instead of a silent flip.
- The opt-in `--serena-instructions` block in CLAUDE.md/AGENTS.md is
never removed, by unwrap or anything else. Users who opted into it keep
instructions that point at tools no longer registered. This predates
this PR and will be fixed separately.
- For Claude Code, Serena entries are per project, so removal happens
the next time `wrap` or `unwrap` runs in each project. The docs say so.

## Tests

- `pytest tests/test_wrap_code_memory.py tests/test_cli
tests/test_mcp_registry`: 1368 passed. 5 failed, and those 5 fail the
same way on `main` in my local venv:
`test_mcp_reconcile.py::test_ordinary_install_does_not_adopt_serena` and
4 in `test_wrap_persistent.py`, all from an environment/install
mismatch.
- New tests:
- The default wrap removes a Serena entry Headroom installed and prints
how to keep it.
- The default wrap leaves a hand-added Serena alone and prints nothing.
  - `--serena --no-serena` resolves to none.
- `wrap opencode --code-memory serena` registers Serena with the `agent`
context.
- `ruff check`, `ruff format --check` and `mypy` are clean on the
changed files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

## Real Behavior Proof

- **Environment:** macOS (Darwin 27.2), Python 3.12.13, Claude Code
2.1.318, real `claude` CLI doing every `claude mcp add`/`remove`. A
temporary `HOME`, so `~/.claude.json` starts empty, and a fresh git repo
per case.
- **Builds:** "before" = `main` at `12864fc8`; "after" = this PR at
`56557dc9`.
- **Command:** `headroom wrap claude --no-proxy [flags] -- --version`.
Wrap does its full MCP setup, then launches Claude Code, which prints
its version and exits, so no model call is made. `state` is read from
`~/.claude.json` (user scope and this project's local scope) and the
project's `.mcp.json` after each run.

```
== 1. Upgrade: Serena installed by Headroom (project/local scope)
    $ headroom wrap claude --no-proxy -- --version    (cwd=projA, build=12864fc8)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     scoped to this project — pass --code-memory-scope user to register it for every Claude Code session
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projA): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projA, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projA): - | .mcp.json: -

== 2. Upgrade: Serena added by hand (claude mcp add)
    state| user: ['headroom'] | local(projB): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projB, build=56557dc9)
    exit=0
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projB): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy --code-memory none -- --version    (cwd=projB, build=56557dc9)
    exit=0
    wrap|   Serena MCP is present but user-managed — leaving it in place (Headroom only removes entries it installed)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projB): ['serena'] | .mcp.json: -

== 3. Explicit opt-in, then rollback to the default
    $ headroom wrap claude --no-proxy --code-memory serena -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     scoped to this project — pass --code-memory-scope user to register it for every Claude Code session
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): - | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     scoped to this project — pass --code-memory-scope user to register it for every Claude Code session
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): ['serena'] | .mcp.json: -
    $ headroom wrap claude --no-proxy --serena -- --version    (cwd=projC, build=56557dc9)
    exit=0
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projC): ['serena'] | .mcp.json: -

== 4. User scope: installed by Headroom with --code-memory-scope user
    $ headroom wrap claude --no-proxy --code-memory-scope user -- --version    (cwd=projD, build=12864fc8)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     registered for every Claude Code session on this machine (--code-memory-scope user)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom', 'serena'] | local(projD): - | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projD, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projD): - | .mcp.json: -

== 5. User scope opt-in on the PR build
    $ headroom wrap claude --no-proxy --code-memory serena --code-memory-scope user -- --version    (cwd=projE, build=56557dc9)
    exit=0
    wrap|   Serena MCP: registered (restart Claude Code if it was already running)
    wrap|     registered for every Claude Code session on this machine (--code-memory-scope user)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom', 'serena'] | local(projE): - | .mcp.json: -
    $ headroom wrap claude --no-proxy -- --version    (cwd=projE, build=56557dc9)
    exit=0
    wrap|   Removed previously-installed Serena MCP (Serena is now off by default; pass --code-memory serena to keep it)
    wrap| 2.1.318 (Claude Code)
    state| user: ['headroom'] | local(projE): - | .mcp.json: -
```

What this shows:
1. A Serena entry Headroom installed on `main` is removed on the first
wrap after upgrading, with the "off by default" message. This holds for
both scopes (cases 1 and 4).
2. A hand-added Serena entry is kept, and the default wrap prints
nothing. `--code-memory none` keeps it and says it's user-managed (case
2).
3. Opting in works with `--code-memory serena`,
`HEADROOM_CODE_MEMORY=serena` and the deprecated `--serena`, at project
scope and user scope (cases 3 and 5). Running the default afterwards
rolls it back.
4. Project scope here means Claude Code's `local` scope, which is where
wrap registers. Wrap never writes `.mcp.json`.

Not tested live: `wrap codex|grok|opencode` (unit tests only), Windows,
and an interactive Claude session that calls Serena tools.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4509"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Maintainer post-merge qualification
Published `983b6bb15119a3ecb8f6e20bf014b8bed123e95d` includes upstream
main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 19 selector/Serena
tests pass,1optional skip; Ruff/format/diff pass. Clean main sync;
contributor selector/ownership code unchanged. Existing real
upgrade/configuration/launch proof requirement remains; jCodeMunch
proposal is unrelated to this source delta. Contributor correctness
logic was not rewritten. Existing Changes Requested decision retained
without duplicate findings. Scoped Windows validation uses the existing
development native extension, not a fresh build. Hosted qualification is
handled by maintainers; nothing merged.


### Maintainer current-main requalification

Exact head `e8e3011187a2ffdfbb6c9932db26b22ec911ab39`, upstream main
`ecbaf86d78cf52d185fbffa306157b9a4cc1b906`.

Mechanically updated from upstream/main
ecbaf86d78 without changing contributor
selector/ownership logic. Reviewed the current discussion and
integration delta. All 29 focused code-memory/Serena-disable/Gemini
tests pass with one optional skip; Ruff0.17.3 lint/format and whitespace
checks pass. Existing Windows HOME/USERPROFILE fixture isolation remains
intact.

The prior upgrade/runtime qualification requirement remains unresolved:
post-patch real configuration/launch evidence must cover an existing
Headroom-owned Serena entry, a hand-managed entry, explicit opt-in, and
Claude project/user scopes, with commands/results and untested-agent
limits. No new author evidence was found to close it. This pass does not
claim those actual launch/upgrade flows were executed; passing unit
tests and the main sync do not establish that deployment behavior. The
hold is not an inherited-main CI defect. Maintainers handle hosted
checks; no contributor CI execution requested. Nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `0b49a79cb192b294dde7ecb1c72b75fffa2733d5`, upstream main
`8d3e9e5ca7cd645eb73084b848edbea5fc796e1a`.

Clean mechanical sync with upstream
main8d3e9e5ca7cd645eb73084b848edbea5fc796e1a without changing
contributor selector/ownership logic. Current source, author discussion
and Devin findings reviewed.29focused code-memory/Serena-disable/Gemini
tests pass with1optional skip; changed Python Ruff0.17.3check/format and
whitespace pass. Windows HOME/USERPROFILE fixture isolation preserved.

Existing real upgrade/runtime qualification remains missing: existing
Headroom-owned Serena versus hand-managed entry, explicit opt-in, and
Claude project/user scopes need observed configuration/launch results
with explicit untested-agent limits. No new evidence found to close it;
these launch/upgrade paths were not executed in this pass. Passing unit
tests and clean main integration do not establish deployment behavior.
No new provider/native-build claim or inherited-main pipeline
attribution. Maintainers handle hosted checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `30dfb386fa7284c9817bf5d7db0648fb10830055`, upstream main
`7592e681459f14a2041eb5f524a658cb6ad1dd92`.

Clean current-main
sync7592e681459f14a2041eb5f524a658cb6ad1dd92,including HNSW re-index and
memory relation-key fixes after917release-recovery merge. Contributor
selector/ownership logic unchanged; fresh discussion
considered.37wrapper/Serena/memory-regression/Gemini tests pass
with1optional skip;whitespace passes. PriorRuff0.17.3validation applies
to unchanged PR files.

Existing real upgrade/launch evidence hold remains: Headroom-owned
versus hand-managed Serena
entries,explicitoptin,andClaudeproject/userscopes need
observedpostpatchresults. No newevidence closesit;actualupgrade/launch
pathsnotrerun. No freshnativebuild/provider claim.
MaintainershandleCI;nothingmerged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `e1778a5368626547ad958926d5570edf153182f6`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Clean integration with upstream/main
2d3ee51846 after the external 0.43.1
release merge. Exact headroom/crates/tests comparison against previously
reviewed 30dfb386f shows no changes. Re-reviewed generated metadata; 78
release/version/MCP tests pass on this tree and whitespace passes.
Earlier 37 wrapper/Serena/memory/Gemini passes and one optional skip
remain scoped to unchanged inputs.

Existing actual upgrade/launch proof hold remains: Headroom-owned versus
hand-managed Serena entries, explicit opt-in, and Claude project/user
scopes need observed postpatch results. Those runtime paths were not
rerun and no new evidence closes the hold. No contributor ownership
policy changed, no fresh native-build/provider claim. Maintainers handle
checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `173c5739d7357ca9398faa41b4d1ce3b72d6c550`, upstream main
`7877241d84c93b3ec487e07324245838ddae85cc`.

Cleanly integrated upstream/main7877241d84c93b3ec487e07324245838ddae85cc
and re-reviewed the wrapper overlap with Hermes registration and port
discovery.35 selector/Serena-disable/Hermes provider and wrapper tests
pass,1optional skip; Ruff and whitespace pass. Contributor
ownership/default policy is unchanged. Existing actual
upgrade/configuration/launch proof requirement remains open for
Headroom-owned versus user-managed entries, explicit opt-in, and Claude
project/user scopes. No runtime proof is inferred from mocks or
native-build claim made. Maintainers handle hosted checks; nothing
merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `eb1c3d66669cf71f881f1ba06dc049d36b20da7b`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 35 passed, 1 skipped in 2.93s.
Selected real dependency overlays and native DLL paths are recorded in
final4509-fresh-native-tests.log; the DLL provenance/input hashes are in
final-native-artifact-proof.json. No redundant rerun is implied after
this Cargo-only merge. Whitespace passes. Prior source review and scoped
real-behavior evidence remain applicable to unchanged contributor
inputs.

Existing real upgrade/configuration/launch qualification remains open
for Headroom-owned versus user-managed Serena entries, explicit
opt-in/rollback, and Claude project/user scopes. No new author evidence
closes it; contributor ownership/default policy is unchanged.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `43bfea8a164c21ab69793b3905fbf1457fef7bc2`, upstream main
`da6d9736cd82074988c7d0a2360b49d9d8b6c0a0`.

Current-main requalification against
da6d9736cd. All69 integration patches
exactly match c01cf07-to-current-main delta after diff-location
normalization; contributor selector/default/Serena ownership logic
unchanged.162 focused code-memory,Serena disable,Hermes,Windows
supervisor,search-compressor tests pass;3 platform/optional skips. All31
changed Python files pass Ruff lint/format; offline lock and whitespace
pass. Native pr3982 DLL verified against complete tracked native-input
hash proof and artifact checksum; no fresh native-build claim.
Author/Devin discussion checked. Existing real
upgrade/configuration/launch qualification remains open: prior
Headroom-owned Serena entry versus user-managed entry, explicit
opt-in/rollback, Claude project/user scopes. No new author evidence
closes it; this run does not assert actual agent launch or deployment
proof. Maintainers handle hosted checks; no contributor policy rewrite
or merge.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `56557dc91725422257f90c3cb5c5f04a665e6043`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 19 integration patches
exactly match the prior-reviewed-main da6d9736 to current-main delta
after diff-location normalization; contributor selector/default/Serena
ownership logic is unchanged. Fresh code-memory/Serena
disable/scope/reconciliation/migration, doctor and incoming
Vertex/backend/CLI tests: 330 passed, 12 skipped, 1 warning. Fourteen
changed Python files pass Ruff lint/format; offline lock and whitespace
checks pass.

Author/Devin discussion read. Existing real upgrade/configuration/launch
qualification remains open: prior Headroom-owned entry versus
user-managed entry, explicit opt-in/rollback, Claude project/user
scopes. This run establishes no actual agent launch or deployment proof.
Static source/test scope and security inspection found no new workflow,
dependency or setup path; the inherited flagged workflow-validation job
is excluded by the exact PR's unchanged workflow paths. Native DLL
reused after complete tracked input-set/hash and artifact-checksum
verification; no fresh build claimed. Maintainers handle exact-head
hosted checks; no PR merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-09 12:23:08 -06:00
bb8d18a56a feat(wrap): add headroom wrap droid for Factory Droid (#4143)
## Description

Add `headroom wrap droid` so Factory subscription traffic, including
Droid Core, can be compressed on the existing Factory plan.

Droid ignores `ANTHROPIC_BASE_URL` / `OPENAI_BASE_URL` for
Factory-served models. It honors `FACTORY_API_BASE_URL`. This wrapper
points that at the local proxy and gives the proxy a Factory upstream
(`--factory-api-url` / `FACTORY_TARGET_API_URL`, default
`https://api.factory.ai`).

With a Factory upstream configured the proxy registers:

- `POST /api/llm/a/v1/messages` — compressed with the existing Anthropic
pipeline and forwarded to `{factory}/api/llm/a/v1/messages`. A
client-sent `x-headroom-base-url` is ignored. `x-headroom-*` headers are
stripped. Operator Anthropic extra headers are withheld from the
untrusted Factory host.
- Every other `/api/*` path — forwarded verbatim to Factory, except
Hermes Studio's scoped prefixes and requests that name their own
`x-headroom-base-url`.

Nothing outside `/api/` is sent to Factory.
`select_passthrough_base_url` and `upstream_trust` are untouched, so a
leftover Codex, Claude, or Copilot config pointed at a Factory-mode
proxy keeps its own upstream and credentials. That is the deliberate
difference from https://github.com/headroomlabs-ai/headroom/pull/2352,
which is still open and which this does not close. #2352 sends every
unrouted request on a Factory-mode proxy to Factory.

`wrap droid` writes no Droid config. It threads `factory_api_url`
through the existing proxy-reuse logic. With `--no-proxy` it refuses
unless `/health` shows a Headroom proxy serving the same Factory
upstream. OpenAI-shaped `/api/llm/o/*` traffic is forwarded
uncompressed. Those routes are deferred.

This commit is `74c889fb` rebased onto current `main` (`0dd8e0b5`). The
only conflicts were `docs/content/docs/meta.json` (kept both the new
`hermes` page and `droid`) and the `--no-proxy` branch in
`headroom/cli/wrap.py` (kept upstream's OpenAI upstream check and the
Factory check).

Refs #2351

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom wrap droid` sets `FACTORY_API_BASE_URL` to the local proxy
and resolves the Factory upstream from `--factory-api-url`, then
`FACTORY_API_BASE_URL`, then `https://api.factory.ai`. Loopback,
credential, query, and fragment URLs are rejected.
- Factory inference and `/api/*` passthrough routes are registered only
when a Factory upstream is configured. The default proxy is unchanged.
- `/health` reports `factory_api_url`.
- Docs: `docs/content/docs/droid.mdx`.

## Testing

- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

Rebased tree `/home/hermes/workspace/headroom-rebase` at `0dd8e0b5`,
importing that tree via `PYTHONPATH` and the existing venv at
`/home/hermes/workspace/headroom/.venv` (Python 3.11.16).

```text
tests/test_cli/test_wrap_droid.py ............................           [ 31%]
tests/test_provider_droid_runtime.py ................................    [ 66%]
```

60 passed. Those are the wrap launch-env tests and the Factory runtime
tests.

`tests/test_proxy/test_factory_decoy_proof.py` did not complete in this
worktree. The proxy refuses to start because `headroom._core` is not
built here (`ModuleNotFoundError`, exit 78). Setting
`HEADROOM_REQUIRE_RUST_CORE=false` still did not finish within 180s. The
decoy proof (decoy upstream stays silent, Factory stand-in receives the
inference request, spoofed `x-headroom-*` headers, REST passthrough,
`--no-proxy` refusals) was written for this change and passed in the
pre-rebase review session. It was not re-run to a green result after
this rebase.

`ruff` and `mypy` were not re-run after the rebase.

## Real Behavior Proof

- Environment: Linux, Python 3.11.16, rebased commit `0dd8e0b5` on top
of `upstream/main` `8c1beb20`. No Factory token. No call to
`api.factory.ai`.
- Exact command / steps:
`PYTHONPATH=/home/hermes/workspace/headroom-rebase
/home/hermes/workspace/headroom/.venv/bin/python -m pytest
tests/test_cli/test_wrap_droid.py tests/test_provider_droid_runtime.py`
- Observed result: 60 passed.
- Not tested: a live Factory subscription, `droid` itself, the socket
decoy proof on this rebased commit (blocked locally by the missing Rust
extension), `ruff`, `mypy`, and CI on GitHub.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. Factory routes register only when
`factory_api_url` / `--factory-api-url` is set.
- Minimum rollout channel: stable. Default proxy behavior is unchanged.
- Stable/default behavior changed: no, unless a Factory upstream is
configured.
- Kill switch / disable path: do not pass `--factory-api-url` and do not
set `FACTORY_TARGET_API_URL`. `wrap droid` is opt-in.
- Unsafe override required: no.
- Qualification impact: none on the default proxy.
- Rollback path: revert this commit. No config is written into Droid.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

The pre-rebase commit `74c889fb` was approved by a fresh Claude Opus 5.5
high review with an empty blocking list. This commit is that change
rebased onto current main. The two conflict resolutions are described
above and have not had a second Opus pass.

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Does not close #2352. That PR is the prior attempt. This one is a
current-main rebuild with the catch-all leak removed: only `/api/*` is
forwarded to Factory.

akbarkanso is not a Headroom maintainer. Please do not expect this
account to merge.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4143"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `e28dfe6422dfcb9b2cbd5ce0fe673196d3b6ad40`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Updated against upstream/main e4a60ed7. Resolved the OpenAI passthrough
overlap by retaining main's prepared-request/model/timing parameters and
soft-passthrough guard alongside the contributor's verbatim flag;
generic compression requires both guards to permit it. Re-reviewed
Factory direct-routing/verbatim semantics and existing discussion. All
93 wrapper/runtime/Factory passthrough/socket-decoy tests pass with
existing native extension; changed-file Ruff lint/format and whitespace
checks pass. No contributor routing policy was rewritten.

Independently reran the official Droid1.3.0 CLI with isolated home,
dummy API key, actual Headroom/Uvicorn proxy and controlled LAN HTTP
upstream on this exact combined head. The CLI consumed the controlled
Anthropic SSE reply and exited zero with HEADROOM_QUALIFICATION_OK. The
message request preserved dummy Authorization and forwarded no
x-headroom headers. This is controlled transport/runtime qualification,
not a live Factory subscription or savings benchmark, and not a fresh
native build. Applied shared test-only Gemini fixture repair for
independently established main failures; all ten cases pass. Formal
approval awaits exact-head hosted qualification.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `7dd0725a9ac4dd8103dedf78552ceca886fc3620`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Clean mechanical merge with upstream main
c01cf07d94. Re-reviewed current
Factory/Droid routing and author/Devin discussion. Factory inference
retains force_direct to bypass translating backends; Factory passthrough
retains verbatim alongside main's soft-passthrough/prepared-request
guards. Hermes scoped proxy prefixes remain excluded from Factory
dispatch; the existing standalone environment precedence is
intentionally consistent with sibling targets, while Click/wrap retain
flag precedence. No contributor routing policy rewritten.

Fresh wrapper/runtime/Factory passthrough/socket-decoy/Hermes
verification: 100 passed, with test-client/runpy warnings only. All 14
changed Python files pass Ruff lint/format; main-relative whitespace
clean and owned tree clean. Tests use the current-main native DLL for
local startup. Earlier official Droid 1.3.0 controlled-upstream runtime
proof remains scoped to its original run; no fresh official CLI replay,
Factory subscription inference, savings benchmark or branch-native build
is claimed. No new blocker found; approval awaits exact-head hosted
checks. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-07 20:48:53 -06:00
9b2dec56f9 feat(proxy): show per-request /stats metadata to token-authenticated callers (#4168)
## Description

`/stats` and `/stats-lifetime` withhold the per-request tail
(`recent_requests`, `request_logs`) and project names from every
non-loopback caller, unless the request's `Host` is an IP literal
**and** the peer is inside
`HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS`.

That rule cannot be met by a common deployment: the proxy runs on a
container platform with `HEADROOM_PROXY_TOKEN` set, and the dashboard
sits on its own host behind an authenticating reverse proxy (here Entra
ID sign-in in front of a small Caddy that adds `x-headroom-proxy-token`
after the user signed in). That reverse proxy reaches Headroom with a
domain `Host` (the platform routes internal calls by Host, so it cannot
be an IP) and from a peer address that is not stable enough to list. The
dashboard loads, the aggregate counters work, and the "Recent Requests"
table stays empty with no indication why.

This PR also admits callers whose proxy token the security gate has
already verified (`request.state.proxy_authenticated`, which the gate
sets only after the constant-time token comparison).

**This is a nice-to-have, not a bug fix.** The current behaviour is
documented and deliberate, and the existing trusted-CIDR path keeps
working unchanged. If you would rather not widen who sees this metadata,
closing the PR is fine; we carry the change on our fork either way.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/server.py`: add `_request_can_view_stats_metadata`:
true when the gate verified the proxy token, otherwise the existing
`_request_can_view_dashboard_metadata` result. `/stats` and
`/stats-lifetime` use it. The helper is deliberately not used by
`/settings*` or `/transformations/feed`, which keep their loopback /
trusted-client guards.
- `tests/test_proxy_stats_recent_requests.py`: add a regression test
(domain `Host`, non-loopback peer, token sent) for `/stats` and
`/stats?cached=1`, asserting the tail is served with the token and that
the gate still answers 401 without it or with a wrong one.
- `docs/content/docs/security-model.mdx`: new section **What the proxy
token unlocks**. It lists exactly which fields a token holder now also
sees: `recent_requests`, `request_logs` and `config` on `/stats` (and
`?cached=1`), and `projects` and `persistence.error` on
`/stats-lifetime`. It says the settings routes and the
`require_loopback` routes stay closed to a token. It also warns that the
token is one shared secret, not a per-caller identity: every holder sees
every caller's request metadata, so a deployment that hands one token to
several clients that must not see each other's metadata should run one
proxy per client. The dashboard-CIDR table row now points to the
section.
- `docs/content/docs/proxy.mdx` (the `HEADROOM_PROXY_TOKEN` row) and
`SECURITY.md` (the local-control-surfaces bullet): one line each
pointing to the same consequence.
- `headroom/proxy/server.py` docstrings: the `/stats` handler and
`_request_can_view_stats_metadata` now describe the actual access rule.
The helper no longer claims the change widens nothing.

Who gains access: holders of `HEADROOM_PROXY_TOKEN`. This does widen
what the token grants. Before, it granted inference and the read-only
operator routes. Now it also grants the request metadata of every caller
on the proxy. With no token configured, nothing changes (the flag is
never set).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom/proxy/server.py`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

At `1c6666d` merged with current `main` (`4abbf07`):

```text
$ python -m pytest -q tests/test_proxy_stats_recent_requests.py tests/test_proxy_loopback_gating.py \
    tests/test_dashboard_accessibility.py tests/test_proxy_hardening.py
214 passed, 1 skipped in 44.60s

$ python -m mypy headroom/proxy/server.py
Success: no issues found in 1 source file

$ ruff check headroom/proxy/server.py tests/test_proxy_stats_recent_requests.py
All checks passed!
$ ruff format --check headroom/proxy/server.py tests/test_proxy_stats_recent_requests.py
2 files already formatted
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13, this branch at `4abbf07`
(`1c6666d` merged with current `main`). A real `headroom proxy` with a
throwaway `HEADROOM_PROXY_TOKEN`, bound to a non-loopback LAN address,
with `HEADROOM_OFFLINE=1` and `--anthropic-api-url` pointed at a local
stub that answers `/v1/messages` with a fixed Anthropic message:
  ```text
headroom proxy --host 192.168.0.48 --port 18788 --no-optimize
--anthropic-api-url http://127.0.0.1:18999
  ```
- Exact command / steps: a client on `192.168.0.48` (not loopback, in no
trusted CIDR) sends what the dashboard's reverse proxy sends. That is
`Host: headroom-dashboard.example.test` (a domain, as the platform
routes by Host) plus `x-headroom-proxy-token`. It first makes one
token-authenticated `POST /v1/messages` so there is a request to list,
then reads the stats endpoints with the token, without it, and with a
wrong one. It also tries the settings and loopback-only routes with the
token. Same script against the same build with only
`headroom/proxy/server.py` swapped back to `main`'s, for the before
column.
- Observed result: verified token holders receive shared stats metadata,
while missing or incorrect tokens are rejected; exact before/after
responses follow.

  ```text
request (domain Host, non-loopback peer) before (main) after (this PR)
POST /v1/messages token 200 200
GET /stats token 200, recent_requests/request_logs/ 200,
recent_requests=1 request_logs=1
config absent config present, first={"provider":
"anthropic","model":"claude-sonnet-5-5"}
GET /stats?cached=1 token 200, same fields absent 200, same as /stats
GET /stats-lifetime token 200, projects absent 200, projects present
GET /stats no token 401 401
GET /stats wrong token 401 401
GET /stats-lifetime no token 401 401
GET /settings token 404
GET /settings/schema token 404
GET /dashboard/settings token 404
GET /transformations/feed token 404
  ```

The proxy log shows `event=proxy_auth_rejected path=/stats …
reason=missing_token`, `… reason=bad_token` and `path=/stats-lifetime …
reason=missing_token` for the three rejections.
- Not tested: the full suite (only the four files above). The deployed
dashboard end to end: the run above reproduces its request shape against
a real proxy, without the Entra/Caddy hop in front.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: only when `HEADROOM_PROXY_TOKEN` is
set. Token-authenticated remote callers now receive `recent_requests`,
`request_logs` and `config` from `/stats`, and `projects` and
`persistence.error` from `/stats-lifetime`, for every caller of the
proxy. Loopback and trusted-CIDR behaviour is unchanged, and the
settings and loopback-only routes are not opened to the token.
- Kill switch / disable path: unset the token (then nothing changes), or
revert the commit
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- Documentation: done; see the security-model section above.
- A possible alternative that avoids widening the rule: an opt-in env
var (e.g. `HEADROOM_STATS_METADATA_FOR_TOKEN=1`). Happy to switch to
that if you prefer the default to stay strict.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_013MpfLG8sVzqPhD34teqdLD

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4168"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `09689b73ddeb8879a2f2c68726f9e885cd9bfdee`, upstream main
`e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`.

Mechanically merged upstream/main e4a60ed7 and re-reviewed
stats-metadata authorization plus the existing security/runtime
discussion. Only callers whose proxy token authentication succeeded gain
the additional per-request stats metadata; settings and loopback-only
routes retain their separate guards. Documentation explicitly states
that token holders share every caller's traffic metadata. No contributor
production logic was rewritten.

All 260 focused stats/recent-request, loopback-gating and hardening
tests pass; Ruff lint/format and whitespace checks pass. Existing
before/after real HTTP evidence remains explicitly scoped; local tests
use the existing native extension, not a fresh native build. Applied the
shared test-only Gemini streamed-request fixture repair for
independently established main failures; all ten cases pass. Formal
approval awaits exact-head hosted qualification.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `fa07187e666ca7895d1cc5708a9d930f38399fe1`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Clean mechanical merge with upstream main
c01cf07d94. Reviewed current author
discussion and Devin runtime/documentation findings. Additional stats
metadata is allowed only after successful proxy-token authentication or
the existing dashboard authorization; settings and loopback-only routes
retain separate guards. The security documentation explicitly states
that one shared token exposes every caller's traffic metadata to every
holder. No contributor authorization policy rewritten.

Fresh focused stats/recent-request, loopback-gating and hardening
verification: 260 tests passed with six existing deprecation/OpenAPI
warnings. Changed-file Ruff lint/format and main-relative whitespace
checks pass; owned tree clean. Existing non-loopback before/after HTTP
evidence remains scoped; no fresh live replay or branch-native build is
claimed. Python tests use the current-main native DLL. No new source
blocker found; approval awaits exact-head hosted checks. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `20b62cc35f7a005b91a6c9054c41cfd51ac29c37`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean main integration against 0ddaa56c21.
315 stats/recent-request, loopback/security gating and incoming-main
tests passed, 11 optional skips, 6 warnings. Exact incoming-main
patches, lint/format, offline lock and whitespace checks pass. Complete
recorded native input-set/hash and artifact checksum verified before
existing DLL reuse; no fresh native build claimed.

Current author discussion and both resolved Devin threads/replies
reviewed. Verified token-authenticated state expands only stats metadata
authorization; settings and require_loopback guards remain separate.
Documentation explicitly lists fields and warns that the shared token
exposes every caller's request metadata to every holder, with separate
proxies required for isolation. This is an intentional documented
shared-trust policy, not a per-user identity. Prior author's
non-loopback real-proxy before/after evidence is retained with its
scope; this refresh adds local tests, not a fresh live replay or
browser/reverse-proxy proof. No new source blocker found.

Source/test/prose delta has no
workflow/setup/dependency/CLI/install/provider/OpenClaw/SDK path
changes. Inherited flagged workflow-validation and Init/Wrap bootstrap
paths are excluded. Docs build uses unchanged-main
workflow/package/lock/config and this PR changes only prose there.
Exact-head Actions and rollup remain pending before approval. Nothing
merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-07 20:42:27 -06:00
33f067a243 feat(proxy): add lossless /v1/compress mode and detect provider by wire shape (#4617)
## Description

Two gaps stop a gateway from using `/v1/compress` well. Both showed up
while fixing LiteLLM's `headroom` guardrail, which is the follow-up PR
that needs this one.

1. **No mode compresses messages losslessly without markers.** A gateway
with no retrieval loop can't resolve a marker or call
`headroom_retrieve`, so it needs a mode with nothing to retrieve. The
existing marker-free modes still run lossy compressors.
2. **Hooks guess the provider from the model name.** A gateway can send
any client's shape to any model. Claude Code's Anthropic body routed to
Gemini looked like OpenAI, so skill condensing found nothing to change.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)

## Changes Made

**`config.mode="lossless"`**
- Runs format-native folds only, with no markers and no CCR store
writes. A registered lossless provider still applies.
- Turns off stale-Read pointers, which drop the file body and need the
retrieval tool.
- Turns off cross-turn dedup pointers, which a client that never numbers
its messages can't resolve.
- Scope: the guarantee covers message compression (lossless codecs, no
CCR). On a gateway turn, tool schema compaction and registered turn
hooks still run as configured, as in every other mode; description
truncation only happens when an operator sets
`HEADROOM_TOOL_DESC_MAX_CHARS`.

**`provider_for_body`**
- Hooks and the session tracker now use the body's wire shape, and fall
back to the model name when the shape proves nothing, such as plain text
with no system prompt or tools.
- Anthropic's own tool types (server tools such as
`web_search_20250305`, client tools such as `bash_20250124`) end in an
eight-digit date and count as the Anthropic shape, so a first-turn body
whose only tool is a server tool is read correctly.
- `route.provider` still follows the model.
- The cache breakpoint budget runs when either the shape or the model is
Anthropic, and never on a Responses body.
- `gateway_response_fields` loses its unused `provider` parameter, and
`RESPONSE_RECEIVED` keeps reporting the model's provider.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`, `ruff format --check .`, on the
pinned 0.17.0)
- [ ] Type checking passes (`mypy headroom`): one error, in
`headroom/integrations/langchain/chat_model.py`, which this PR doesn't
touch (local mypy 1.19.1)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
pytest tests/test_proxy_compress_endpoint.py tests/gateway tests/test_gateway_turn_unit.py tests/test_compress_route_tokenizer_by_model.py
425 passed, 4 skipped

Regression tests with the fixes removed: 8 failed, 7 passed
```

## Real Behavior Proof

- **Environment:** LiteLLM main with the follow-up guardrail change,
this branch as the sidecar, and a fake upstream that simulates prompt
caching. No real provider API was called.
- **Exact command / steps:** replayed a scripted 17-call agent session
with exact-duplicate outputs removed, once direct to the fake upstream
and once through LiteLLM and `/v1/compress`.
- **Observed result:** 0 errors, 0 rewrites of already-sent results,
`Read` and `Edit` results unchanged, about 1.3s added in total.

| Client | Prompt cost vs direct |
|---|---|
| Anthropic Messages, Claude Code cache markers | −3.9% |
| Chat Completions | −3.8% |
| Responses | −4.1% |

The script's outputs are synthetic, so treat these as a comparison
between setups, not a forecast for real traffic.

## Known limits

- The default mode and `lossy_inline` can still return a stale-Read
`Retrieve original: hash=` marker, although the docs call them
marker-free. This predates the PR and needs a separate fix. Proxy
`--lossless` has no read-lifecycle guard either, from reading the code.
- With no `ContentRouter`, `_derived_compress_pipeline` falls back to
the default pipeline, so `lossless` wouldn't hold. Production always has
a router, so only test stubs reach this.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4617"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Maintainer post-merge revisit: main e9331b6b
Published head `e50b001bd05c9bc85405accca49cbef0e7786e51` incorporates
upstream main `e9331b6b963b5b6554f45943abed2366aaa25ea6` after #4142,
#4637 and #4641 merged. 164 endpoint/gateway/tokenizer/admission tests
pass; one json_minifier strategy assertion fails identically on actual
main e9331b6b. Ruff/format/diff pass. Import-only conflict resolved by
retaining both provider_for_body and RequestBodyTooLarge. Both existing
holds reconfirmed: lossless HTTP output removes unique description
suffix via tool_desc_compaction; Anthropic server tool with Gemini model
is classified OpenAI.

Contributor correctness logic was not rewritten. The existing Changes
Requested decision remains applicable; unchanged findings are not
reposted as duplicate reviews. Scoped Windows qualification uses the
existing development native extension, not a fresh build. Hosted checks
must finish independently; no contributor CI action requested and
nothing merged by this review.


## Maintainer post-merge revisit: main cc1eb540
Published `38b871120e318f6eaa1cb2ce16f96d4f6cc2b1e8` includes upstream
main `cc1eb5402aade58204b96b6b4b4d280cdd5456d2`. 90 gateway/tokenizer
tests pass; both existing HTTP/provider counterexamples reconfirmed.
Clean merge; Ruff/format/diff pass. Contributor correctness logic was
not rewritten. The existing Changes Requested decision remains
applicable; unchanged findings are not reposted. Scoped Windows
qualification uses the existing development native extension, not a
fresh build. Hosted checks must finish independently; nothing merged by
this review.


### Maintainer current-main requalification

Exact head `33395b5cdd7ee9facce9f97f2a3250f5e6e49cb2`, upstream main
`ecbaf86d78cf52d185fbffa306157b9a4cc1b906`.

Mechanically merged upstream/main
ecbaf86d78 without changing contributor
policy logic. Re-reviewed the combined gateway/provider delta and
existing comments. All 90 focused gateway/tokenizer tests and ten
inherited Gemini fixture cases pass; Ruff0.17.3 lint/format and
whitespace checks pass. The shared Gemini repair was already present, so
no duplicate fixture commit was added.

Both existing correctness findings independently remain on this combined
tree. An HTTP200 /v1/compress request with config.mode=lossless still
drops UNIQUE_REQUIREMENT_AT_END from a long tool description, reporting
tool_desc_compaction. An Anthropic web_search_20250305 tool body with a
Gemini-named model still returns provider openai. Existing requested
semantic fixes/regressions remain necessary; maintainers are handling CI
and no contributor CI execution is requested. No contributor logic was
rewritten and nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `4c40adb53a68f382e32f00620adcfaf1cff477f5`, upstream main
`8d3e9e5ca7cd645eb73084b848edbea5fc796e1a`.

Clean mechanical sync with upstream
main8d3e9e5ca7cd645eb73084b848edbea5fc796e1a including intervening
release/runtime/cache/CCR/SSE/HTTP1.1 merges. No contributor
gateway/provider policy rewritten. Fresh author/Devin discussion
considered.100focused gateway-contract/tokenizer/Gemini tests
pass;7changed Python files pass Ruff0.17.3check/format and whitespace
checks pass. Last main delta after test start changes only
Rust/OpenClaw, leaving tested Python inputs unchanged.

Both existing blockers remain freshly reproduced from this worktree
(source import path verified): real app POST/v1/compress lossless
returns200 but drops UNIQUE_REQUIREMENT_AT_END from tool description via
tool_desc_compaction; Anthropic server-tool wire shape with Gemini model
returns provider openai. Preserve promised lossless content and classify
server-tool wire forms correctly, with regressions. No external provider
session/nativebuild claimed. Copied extension supports Python startup;
maintainers handle hosted checks. Nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `8b7b2b0d02daf1a031828ecb13095ab315431f7e`, upstream main
`917edc62230c3c4a8a415e2053da68dab1f003dd`.

Current-main integration against917edc62230c3c4a8a415e2053da68dab1f003dd
is clean. Compared entire headroom/crates/Python manifests+lock/tests
inputs to prior qualified4c40adb53: byte-identical. Intervening changes
are reviewed npm maintenance and release-envelope recovery workflow; no
contributor gateway policy changed.75new-base
release-workflow/normalizer tests pass,whitespace passes.
Prior100gateway/tokenizer/Gemini results remain scoped to those
identical inputs rather than relabeled fresh.

Both gateway blockers freshly rechecked from worktree: Anthropic
server-tool+Gemini body classifiesopenai; actual lossless/v1/compress
returns200 but drops unique description suffix via tool_desc_compaction.
Retain existing losslessness/classification corrections. No fresh
nativebuild/provider claim. Maintainers handle exactheadCI;nothing
merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `490ff6defdebddda9106e9e143d42d1e463b6324`, upstream main
`7592e681459f14a2041eb5f524a658cb6ad1dd92`.

Clean integration with upstream/main
7592e68145. Reviewed both intervening
memory fixes and fresh PR discussion. Gateway/runtime/Rust inputs are
unchanged from the previously reviewed head; only the two main memory
fixes and their regressions enter this merge. Nine memory/Gemini
regression tests pass; main-relative whitespace passes.

Fresh actual-app probes retain both existing blockers: Anthropic
server-tool content with a Gemini model classifies as OpenAI; lossless
/v1/compress removes the unique tool-description suffix through
tool_desc_compaction. No contributor policy was rewritten. Existing
native extension used for Python startup, not a fresh native-build
claim. Maintainers handle hosted checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `d3cb3a02af03991a696461160ee0db322dd8bf48`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Clean integration with upstream/main
2d3ee51846 after the external 0.43.1
release merge. Exact headroom/crates/tests comparison against prior
reviewed 490ff6def shows no changes. Re-reviewed generated release
metadata; 78 release/version/MCP tests pass on this composed tree, and
whitespace passes. Previous nine memory/Gemini regressions and fresh
gateway probes remain scoped to identical runtime inputs.

Both established blockers remain: lossless /v1/compress drops the unique
tool-description suffix, and Anthropic server-tool content with a Gemini
model classifies as OpenAI. Runtime probes were not relabeled as rerun
after this metadata-only merge. No contributor policy rewritten; no
fresh native-build claim. Maintainers handle exact-head checks; nothing
merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `e0c873630e9ffc7bc0d0d2fdd22c483b731e49a3`, upstream main
`910588e6b15ab54c4099d03703e5fa20c8abb757`.

Clean mechanical synchronization with upstream
main910588e6b15ab54c4099d03703e5fa20c8abb757, including the externally
merged runtime fixes, docs dependency update and tree-sitter lock
repair. No contributor gateway/provider policy rewritten. Current
author/Devin discussion considered. All90 gateway
contract/tokenizer-by-model tests pass on the preceding composed runtime
tree; the final dependency-only merge changes only pyproject/uv.lock,
leaving those gateway/tokenizer sources unchanged. No redundant rerun on
the final merge is claimed. Changed Python Ruff lint/format and
main-relative whitespace pass.

Both existing blockers freshly reproduce on the composed runtime paths:
lossless /v1/compress returns200 but drops UNIQUE_REQUIREMENT_AT_END
from the tool description and reports tool_desc_compaction; Anthropic
server-tool content with a Gemini model is classified as OpenAI.
Preserve the lossless original payload and classify the full request
protocol consistently, with endpoint regressions for these inputs. The
main fixes do not close either defect.

Tests use the existing native core for startup, not a fresh native
build; no live provider traffic was made. Maintainers handle hosted
checks; no inherited-main pipeline failure attributed and nothing
merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `507433606f698249e546e1418928caee99d0c901`, upstream main
`7877241d84c93b3ec487e07324245838ddae85cc`.

Clean mechanical integration with
upstream/main7877241d84c93b3ec487e07324245838ddae85cc. Gateway
production/contracts remain byte-identical to the preceding reviewed
head; re-reviewed the new CLI/Hermes integration.127 gateway
request/response contract and model-tokenizer tests pass; whitespace
passes. Both existing holds freshly reproduce on this exact tree:
lossless /v1/compress returns200 while dropping
UNIQUE_REQUIREMENT_AT_END and reporting tool_desc_compaction; Anthropic
server-tool content with a Gemini model is classified as OpenAI.
Preserve the lossless original payload and classify the full request
protocol consistently, with endpoint regressions. No contributor
gateway/provider policy rewritten. Existing native core used for
startup; no fresh native build or live provider claim. Maintainers
handle hosted checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `b0daece6b10f623a6e7b0dc3eb01c73e05fa3209`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 127 passed, 3 warnings in 104.78s
(0:01:44). Selected real dependency overlays and native DLL paths are
recorded in final4617-fresh-native-tests.log; the DLL provenance/input
hashes are in final-native-artifact-proof.json. No redundant rerun is
implied after this Cargo-only merge. Whitespace passes. Prior source
review and scoped real-behavior evidence remain applicable to unchanged
contributor inputs.

Both existing blockers freshly reproduce using the combined native DLL:
lossless /v1/compress returns200 but removes UNIQUE_REQUIREMENT_AT_END
and reports tool_desc_compaction; Anthropic server-tool content with a
Gemini model is classified as OpenAI. Preserve the original lossless
payload and classify the complete wire protocol, with endpoint
regressions.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `ec6abc0228a93c5eab1881938041dfee344b1e3a`, upstream main
`2973ca593d2fe65c3259941fdff8009175eff456`.

Requalified after human merges #4667/#4669/#4670 against upstream main
ce42754912. All 11 integration patches
exactly match the prior-main-to-current-main delta after ignoring diff
locations; no contributor gateway/provider policy changed. Fresh gateway
request/response contracts, model-tokenizer, gateway unit,
Kompress-disable and must-keep suites: 207 passed, 3 warnings. Six
changed Python files pass Ruff lint/format; offline lock validation and
whitespace pass. Existing matching-core DLL used; native
source/lock/toolchain unchanged by these main merges, with no fresh
native build or live-provider claim. Both existing blockers freshly
reproduce: lossless /v1/compress returns 200 while removing
UNIQUE_REQUIREMENT_AT_END and reporting tool_desc_compaction; Anthropic
server-tool content with a Gemini model is classified as OpenAI.
Preserve original lossless payload and recognize complete wire protocol
with endpoint regressions. Author and Devin discussions checked;
existing findings remain actionable. Maintainers handle hosted checks;
nothing merged.
Subsequent human merge #4671 advanced main to
2973ca593d before publication; guard
stopped the old candidate. Cleanly integrated the new workflow-only
delta and re-reviewed action pinning with 62 workflow/release tests
passing. Runtime/native sources are byte-identical to the immediately
preceding 207-test and real-behavior qualification; those results apply
to unchanged inputs. All prior-main hosted results are historical for
readiness.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `9f857218486fa451ecacc675863a64bc95f45bb0`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Current-main requalification
against2b0315b2d4b0d54947050bbdad2de9e89df06378 after later human
merges. All37 integration patches exactly match2973ca59-to-current-main
delta after diff-location normalization; contributor gateway/provider
policy unchanged.393gateway request/response
contracts,model-tokenizer,gateway
unit,Kompress-disable,must-keep,doctor,search tests
pass;3warnings.27changedPythonfiles pass Ruff lint/format;offline
lock/whitespacepass. Existing pr3982 native DLL verified every tracked
native-input hash/artifactchecksum;no fresh build claimed. Author/Devin
discussion checked. Both correctness blockers freshly reproduce:lossless
/v1/compress returns200while droppingUNIQUE_REQUIREMENT_AT_END and
reportingtool_desc_compaction;Anthropicserver-toolcontent
withGeminimodel classifiedOpenAI. Preserve original lossless payload and
classify full request wire protocol consistently,with endpoint
regressions. No live provider qualification,contributor logic rewrite,or
inherited-main CI fault attributed. Maintainers handle hosted
checks;nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `41b2145418db7f5980cd52e6235ab330fcd31ac0`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 17 integration patches
exactly match the prior-reviewed-main to current-main delta after
diff-location normalization; contributor gateway/provider policy is
unchanged. Fresh gateway/tokenizer/search/doctor and incoming
Vertex/backend/CLI qualification: 455 passed, 11 skipped, 3 warnings.
Additional gateway request/response contracts and compress-endpoint
suite: 158 passed, 1 warning. Twelve changed Python files pass Ruff
lint/format; offline lock and whitespace checks pass.

Author discussion and outstanding Devin findings read. Both existing
correctness findings freshly reproduce on this exact tree: lossless
/v1/compress returns 200 but drops UNIQUE_REQUIREMENT_AT_END through
tool_desc_compaction; Anthropic server-tool content with a Gemini model
classifies as OpenAI. Existing requested corrections remain; no new
duplicate finding. Local app probes are not live provider proof. Native
DLL reused only after verifying the complete tracked input set, every
recorded input SHA256 and artifact checksum against
pr3982-c01-native-proof.json; no fresh build claimed. Maintainers handle
exact-head hosted checks. No PR merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-07 20:36:11 -06:00
alan blountandJD Davis 0ddaa56c21 fix(vertex): explain Vertex failures in-band, correct the model/region matrix (#2640)
## Description

Vertex rejects onboarding requests for a handful of recurring reasons —
expired ADC, a partner model never enabled in Model Garden, a model not
served in the requested location, a missing optional SDK — and its own
error text names none of the fixes. Users see a bare `404` or an opaque
`500` through their SDK and cannot tell a proxy bug from a
project-config gap.

This makes the proxy explain those failures where it happens, adds a
`[vertex]` extra so the LiteLLM Vertex backend can actually start, and
corrects an example, a test matrix, and three documentation claims that
could not have worked as written.

Closes #2632

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

**Explain Vertex failures in-band**
(`headroom/providers/vertex/diagnostics.py`, new)

- On `401/403/404/429`, attach an actionable hint three ways: a WARNING
in the proxy log, an `x-headroom-hint` response header, and a note
appended to `error.message`. The last is load-bearing — SDKs raise with
the message string and discard headers.
- Successful responses are never modified; annotation is idempotent;
statuses we cannot explain get no hint rather than a plausible-sounding
wrong one.
- Routes opt in via a one-line `@with_vertex_diagnostics` decorator
reading path params FastAPI already injects (**+8 lines** in
`proxy_routes.py`, no re-indentation).
- Same treatment for backend setup failures, where `--backend vertex`
surfaced LiteLLM's raw `No module named 'vertexai'` as an opaque 500.

**Make `--backend vertex` able to start** (`pyproject.toml`,
`registry.py`, `cli/proxy.py`)

- New `[vertex]` extra carrying `google-cloud-aiplatform>=1.38`.
Deliberately **not** folded into `[proxy]`: measured at ~175 MB
installed, and the native passthrough routes never needed it. Follows
the existing `proxy-prod` split.
- Startup preflight keyed on the **resolved provider**, so every alias
normalized at `registry.py:229` (`vertex`, `vertex_ai`,
`litellm-vertex`, `google-vertex`, `googlevertex`) is covered. Skipped
when a backend class is injected, so existing fake-based tests are
unaffected.
- New `BackendUnavailableError` so the CLI prints the remedy and exits
`2` instead of dumping a traceback through startup internals.

**Correct things that could not have worked**

| | was | is |
|---|---|---|
| SDK client | `http_options={"api_endpoint": ...}` | `{"base_url":
"http://..."}` — `api_endpoint` is not a field on `HttpOptions`;
pydantic rejects it before any request is sent |
| Thinking config | `ThinkingConfig(thinking_budget_tokens=)` |
`thinking_budget=` |
| Request payload | `contents: [{parts: [...]}]` | adds `role: "user"` —
the `global` endpoint enforces it, regional was silently lenient |
| Auth guidance | `gcloud auth print-access-token` |
`application-default` — the plain user token gets `401
ACCESS_TOKEN_TYPE_UNSUPPORTED` |
| Test matrix | model × location cross-product | explicit servable pairs
|

- Test matrix is now four pairs that can actually serve:
`global`/`gemini-flash-latest`, `global`/`claude-sonnet-4-6`,
`europe-west2`/`gemini-3.5-flash`, `us-east5`/`claude-sonnet-4-6`.
Gemini 3.x has no US regional endpoint and `-latest` aliases are
global-only, so a pinned `us-central1` Gemini case can never pass. No
Gemini 2.5 — it retires 2026-10-16.
- `test_thinking_levels` asserted only `status_code < 500` while sending
an ignored `thinkingBudgetTokens` key: it passed on 400s and proved
nothing. Now asserts 200 plus scored usage metadata with the correct
`thinkingBudget` spelling.
- Cases the caller's project genuinely cannot serve now **skip** with an
actionable reason instead of reporting a red proxy.

**Docs** (`proxy.mdx`, `installation.mdx`, `wiki/vertex.md`)

- `proxy.mdx` claimed `publisher=anthropic` passthrough "routes through
the same LiteLLM-Vertex path as `--backend vertex_ai`". It does not —
without the flag it is a direct forward, which is why it works on a bare
`[proxy]` install where the LiteLLM path cannot even import.
- Documented the trap of combining `--backend vertex` with native Vertex
URLs: LiteLLM takes project/region/model from its own config and ignores
the URL.
- `installation.mdx` and `wiki/vertex.md` said the SDK is "not included
in any extra".
- Cleared 15 pre-existing `ruff` findings in the two files this PR
rewrites (4 remain in `plugins/headroom-oauth2`, untouched by this
work).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ mypy headroom
Success: no issues found in 511 source files

$ ruff check .
Found 4 errors.        # all in plugins/headroom-oauth2, pre-existing, untouched by this PR

$ pytest tests/test_proxy_vertex_native_integration.py tests/test_vertex_diagnostics.py -q
44 passed, 1 warning in 69.86s          # 8 live integration + 36 offline diagnostics, 0 skipped

$ pytest tests/ -k "anthropic_messages or handler_anthropic or backend or registry or cli_proxy or vertex or proxy_routes"
853 passed, 49 skipped, 9345 deselected, 2 warnings in 28.41s
```

New tests in `tests/test_vertex_diagnostics.py` (36, no credentials
required): status coverage, per-publisher remedy divergence,
idempotency, `content-length` consistency after body rewrite, non-JSON
and body-less/streaming responses, every backend alias preflighted,
other providers unaffected, injected backend bypasses the preflight.

## Real Behavior Proof

- Environment: macOS 14, Python 3.11.15, live GCP project
(`alanblount-sandbox`) with `aiplatform.googleapis.com` enabled and
Claude enabled in Model Garden; ADC via `gcloud auth application-default
login`
- Exact command / steps: `export GCP_PROJECT_ID=alanblount-sandbox &&
export GCP_ACCESS_TOKEN=$(gcloud auth application-default
print-access-token) && uv run --extra proxy --with google-genai python3
examples/vertex_genai_sdk_demo.py && uv run --extra proxy --with
pytest-asyncio pytest tests/test_proxy_vertex_native_integration.py
tests/test_vertex_diagnostics.py -q && headroom proxy --backend vertex`
(exits 2, then retried with `[proxy,vertex]` extra installed)
- Observed result: demo printed `✓ Standard response received
successfully! > 1, 2, 3, 4, 5` and `✓ Thinking response received
successfully!`; 44 tests passed (8 live integration, 36 diagnostics), 0
skipped; `--backend vertex` exited 2 with the remedy message; same
request with the extra installed returned 200; invalid region produced
`[headroom] hint:` in the response body and `x-headroom-hint` header;
success responses were byte-clean
- Not tested: Windows; `us`/`eu` multi-region endpoints and Claude 4.7+
(project cannot serve them); Provisioned Throughput; Rust proxy path;
`--backend vertex` dropping URL project/model on native Vertex routes
(documented as known gap in Additional Notes)


## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — proxy and CLI behavior; terminal output is quoted above.

## Additional Notes

- `uv.lock` intentionally not included: the new extra is opt-in and does
not change default resolution.
- Error-body rewriting is confined to already-failing responses and
wrapped so a diagnostic can never turn a clean upstream error into a
proxy 500.
- The preflight raises where the provider is resolved (`registry.py`)
rather than at the CLI, so programmatic embedders get the same
guarantee; the CLI only formats it.
- Three test files fail to collect on this branch
(`test_hermes_passthrough_compression`, `test_observability_metrics`,
`test_observability_tracing`). They predate this work and are unrelated,
so they were excluded rather than fixed — happy to open a separate
issue.
- `[vertex]` was chosen over adding the dependency to `[proxy]` after
measuring the footprint at ~175 MB. If maintainers prefer it in
`[proxy]`, that is a one-line change.
## Runtime Rollout Safety

- Rollout-managed feature(s): none; diagnostics annotate already-failing
Vertex responses and the backend dependency is opt-in through the vertex
extra.
- Minimum rollout channel: normal stable release after Vertex
diagnostics, route, and backend-path tests pass.
- Stable/default behavior changed: successful responses and native
passthrough remain unchanged; supported Vertex failures gain an
actionable header/message, and selecting the LiteLLM Vertex backend
without its SDK now fails once at startup.
- Kill switch / disable path: use native Vertex passthrough without the
LiteLLM backend, or human-revert the diagnostic/preflight change.
- Unsafe override required: none.
- Qualification impact: offline diagnostics, native integration, and
mocked LiteLLM Vertex request-shaping tests must remain green.
- Rollback path: human-revert the PR; no persisted state or migration is
involved.
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2640"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `5d7071f348d2462f666fd7ac1ceb8ac18050169a`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Re-reviewed latest integration head c70c1a497 and current author
responses, including fixed public error hints, log escaping, body/header
consistency, all four backend error sites, and native-route versus
LiteLLM SDK dependency behavior. Clean integration with upstream/main
2d3ee51846. No new PR-specific source
blocker found in reviewed scope; no contributor diagnostic/routing logic
rewritten.

232 Vertex diagnostics/native-route/backend/CLI/release/version/MCP
tests pass with11 skips. Offline lock validation passes (268 packages);
all12 changed Python files pass Ruff check/format, and whitespace
passes. These checks cover synthetic error/transport paths and do not
establish live cloud regional availability or billing behavior. Earlier
author live evidence remains scoped to its stated environment; no fresh
cloud request or native build claimed here. Historical shared
cached-read fixture failure was main-reproduced and is not a
contributor-code blocker. Maintainers handle current exact-head hosted
qualification before approval; nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `486a716adbf642ae5e43276df4c2dc1e63b901d7`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Clean mechanical sync with upstream main
c01cf07d94. Re-reviewed current
author/Devin discussion and public diagnostic/error boundaries,
native-versus-SDK route separation and startup preflight. This sync does
not alter the PR Vertex diagnostics, backend or route-registry
modules.158 focused diagnostics/native-route/backend/streaming/CLI tests
pass,11live-environment skips,1deprecation warning. Offline uv lock
--check passes;all12 changed Python files pass Ruff lint/format and
main-relative whitespace passes. Fixed hints remain separate from raw
provider error text; multi-worker preflight and non-Google partner
guidance retain resolved regressions. No contributor diagnostic/routing
correctness logic rewritten. Tests use existing current-main DLL and
synthetic errors/transports; no fresh native build, live regional
availability/provider inference or billing qualification claimed. Formal
approval awaits exact-head hosted qualification; maintainers handle CI.
Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `0b75c609f24562c8b6ed2aaeaf58239984b4810c`, upstream main
`ce4275491299231bc9d2907833f72f72ea22b8a9`.

Requalified after human merges #4667,#4669,#4670 against upstream main
ce42754912. Earlier approval and
publication guards stopped stale candidates before either was posted.
Current clean merge retains Vertex diagnostics/native-versus-SDK routing
and main enable_kompress checks, lossless cold-prefix disable, and
Python token-window boundary behavior.190 combined
Vertex/backend/CLI/Kompress-disable/must-keep tests
pass,11live-environment skips,1warning. Offline lock validation,12-file
Ruff lint/format and main-relative whitespace pass. No contributor
diagnostic/routing logic rewritten. Native Rust/Cargo sources are
identical to c01 across these merges; existing matching-core DLL
supports tests, not a fresh native build. No new live Vertex
regional/provider/billing qualification claimed. All prior-main hosted
results are superseded for readiness; approval awaits exact checks on
this published merge head. Maintainers handle CI; no agent merge.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `1fe62702ce614315679d68d9d126e31c57208c67`, upstream main
`00d4f441d3f1d48c463fea57e6c574c2f3671302`.

Requalified after human merges #4667,#4669,#4670 against upstream main
ce42754912. Earlier approval and
publication guards stopped stale candidates before either was posted.
Current clean merge retains Vertex diagnostics/native-versus-SDK routing
and main enable_kompress checks, lossless cold-prefix disable, and
Python token-window boundary behavior.190 combined
Vertex/backend/CLI/Kompress-disable/must-keep tests
pass,11live-environment skips,1warning. Offline lock validation,12-file
Ruff lint/format and main-relative whitespace pass. No contributor
diagnostic/routing logic rewritten. Native Rust/Cargo sources are
identical to c01 across these merges; existing matching-core DLL
supports tests, not a fresh native build. No new live Vertex
regional/provider/billing qualification claimed. All prior-main hosted
results are superseded for readiness; approval awaits exact checks on
this published merge head. Maintainers handle CI; no agent merge.

Final current-main requalification:
07d5add5ba, including human merges #4671
action pinning and #4648 multimodal memory preservation. Clean
mechanical merge; Vertex/backend/CLI inputs retain the preceding
190-test qualification. Fresh workflow/release suite: 62 passed. Fresh
memory wrapper suite: 9 passed. New wrapper retains list payload parts
and extracts textual search inputs; no contributor Vertex logic
rewritten. Existing native input provenance remains unchanged. Hosted
qualification must run on this final head before approval.

Newer human merge #4646 advanced authoritative main to
8cdce172ea before publication. Cleanly
integrated quote-aware read detection; reviewed new masking/segmentation
overlap. Fresh read-command/content-router suite: 188 passed, 1 optional
SOCKS warning. Vertex code remains unchanged by this merge.

Final additional human merge #4067 packaging changes advanced main to
a8905214d: cleanly integrated explicit torch-extras split, retaining
Vertex optional dependencies. Fresh ONNX/optional-dependency contracts:
12 passed. Offline lock check and whitespace pass. No live clean-install
or Torch backend claim. Earlier publication candidates were stopped
before push by fresh-main guard.

Latest human merge #4042 advances authoritative main to
cde584d7fe. Clean merge preserves optional
LangChain availability handling and CI dependency setup. Fresh
binding/compatibility/startup-log/xAI regression suite: 29 passed. New
main integration re-reviewed; no contributor Vertex behavior rewritten.

Subsequent #4049 main merge 00d4f441d adds graceful context lookup/init
failure handling. Cleanly integrated with Vertex handler diagnostics
intact. Fresh Anthropic pre-upstream/memory native ops/OpenAI Codex
routing/WebSocket lifecycle suite: 192 passed, 1 deprecation warning.
Reviewed exception handling and transport overlap; no live provider
qualification claim.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `6c63811d6de7491ebce9a16f89e197f5f5dd86dd`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Requalified against authoritative
main2b0315b2d4b0d54947050bbdad2de9e89df06378 after further human merges.
All10 integration patches exactly match00d4f441-to-current-main changes
after diff-location normalization; contributor Vertex
diagnostics/routing policy unchanged.539combined
Vertex/backend/CLI/Kompress-disable/must-keep/binary/doctor/Windows-supervisor/provider/search/text
tests pass;13platform/live-environment skips,2warnings. Nine changed
Python files pass Ruff lint/format; offline lock and whitespace pass.
Existing pr3982 native DLL verified against every tracked native
build-input hash and artifact checksum; no fresh native build claimed.
Current author/Devin discussion checked; prior fixes remain intact. No
live Vertex regional/provider/billing,IPv6 listener or unsupported cloud
qualification claimed. No new finding in reviewed scope. Exact-head
hosted checks must qualify this published head before formal approval;
maintainers handle them. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-06 12:27:47 -06:00
lorenzozaneandJerrettDavis a8905214d9 fix(packaging)!: keep PyTorch extras out of all (#4067)
<!-- maintainer-current-main -->
## Maintainer latest-main checkpoint
Published `fb146c0663b30488da0a49cfb33624ee82862074` contains current
upstream main `e4a60ed72f6ef5b28569f8bfbe2163cfda32c948`. 22
dependency/Gemini tests and lock check passed. Earlier live behavior
proofs below retain their commit/platform scope; contributor runtime
logic is unchanged.

The common shard-2 failure was independently confirmed on main's hosted
job 127069149772 and locally (four failures, six passes): the Gemini
memory fixture mocked an unused body reader and provided no request
stream. The maintainer test-only follow-up now constructs a real
FastAPI/ASGI Request carrying JSON bytes and exercises the actual body
reader. All six test-function ASTs/assertions are unchanged, and all 10
parameterized Gemini cases pass. Touched-file Ruff/format and whitespace
checks pass. This fixes the inherited fixture instead of asking
contributors to run CI or weakening the memory/content assertions. Fresh
exact-head hosted qualification is requested. Nothing merged.
<!-- /maintainer-current-main -->

## Description



On Linux, installing `headroom-ai[all]` resolved PyTorch and NVIDIA CUDA
packages even for users who only needed the CLI/proxy profile. The
locked dependency graph included those packages through `ml`, `memory`,
`evals`, and `voice`.



This keeps those PyTorch-backed extras available for explicit
installation while removing them from `[all]`. Image support and the
other CLI/proxy dependencies remain included; dependency metadata cannot
detect installed GPU hardware.



Closes #3339



## Type of Change



- [ ] Bug fix (non-breaking change that fixes an issue)

- [ ] New feature (non-breaking change that adds functionality)

- [x] Breaking change (fix or feature that would cause existing
functionality to change)

- [ ] Documentation update

- [ ] Performance improvement

- [ ] Code refactoring (no functional changes)



## Changes Made



- Remove `ml`, `memory`, `evals`, and `voice` from the `[all]` extra
while keeping each separately installable.

- Retain `image` and the other default CLI/proxy capabilities.

- Update README and installation docs to describe the opt-in extras and
CUDA package implications.

- Add a dependency contract assertion for the `[all]` extra.



## Testing



- [ ] Unit tests pass (`pytest`)

- [ ] Linting passes (`ruff check .`)

- [ ] Type checking passes (`mypy headroom`)

- [ ] New tests added for new functionality

- [x] Manual testing performed



### Test Output



```text

Focused dependency contract tests: 3 passed, 5 deselected.

Python 3.13 locked `uv export` exited 0; the fixed export omits torch, CUDA, NVIDIA, sentence-transformers, and datasets. The baseline export contained them.

`uv lock --check`, scoped Ruff check/format, and `git diff --check` exited 0.

The broader dependency test module had 3 passed and 5 import failures because `opentelemetry` is absent from the local environment.

```



## Real Behavior Proof



- Environment: Linux, Python 3.13, locked uv dependency resolution.

- Exact command / steps: `UV_PYTHON_DOWNLOADS=never uv export --python
3.13 --extra all --no-dev --locked --no-hashes --no-emit-project`.

- Observed result: exit 0; the resolved export no longer contains torch,
CUDA, NVIDIA, sentence-transformers, or datasets, all of which were
present in the baseline export.

- Not tested: a full environment install and GPU runtime behavior.



## Runtime Rollout Safety



- Rollout-managed feature(s): None.

- Minimum rollout channel: Stable package release.

- Stable/default behavior changed: Yes; `[all]` no longer installs the
four PyTorch-backed extras.

- Kill switch / disable path: Not applicable; install the desired extras
explicitly.

- Unsafe override required: No.

- Qualification impact: Users of those capabilities must add the
corresponding extras.

- Rollback path: Restore the extras to `[all]` and regenerate `uv.lock`.



## Review Readiness



- [x] I have performed a self-review

- [x] This PR is ready for human review



## Checklist



- [x] My code follows the project's style guidelines

- [x] I have performed a self-review of my code

- [ ] I have commented my code, particularly in hard-to-understand areas

- [x] I have made corresponding changes to the documentation

- [ ] My changes generate no new warnings

- [x] I have added tests that prove my fix is effective or that my
feature works

- [ ] New and existing unit tests pass locally with my changes

- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)



## Screenshots (if applicable)



Not applicable to this dependency-resolution change.



## Additional Notes



The `[all]` install profile intentionally no longer includes
PyTorch-backed optional capabilities; each remains available through its
named extra. The focused regression and locked resolver proof passed,
but the full dependency test module and a full install were not verified
locally.





## Maintainer integration qualification

Published head `3229690b613810d496f015cedd086ca1d7421843` includes
current upstream main `31dcaf41`. The merge preserves main's `lean`
dependency profile and deprecated `sandbox` alias alongside the
contributor's opt-in `[all]` policy. No optional-feature runtime logic
was changed.



- Reconciled the existing macOS dependency test with the declared
policy, retaining the transitive exclusion check and independently
verifying explicit `ml`/`voice` opt-ins on Intel macOS and Linux. All 12
tests across `tests/test_onnx_dependency_contract.py` and
`tests/test_optional_dependencies.py` pass on Windows/Python 3.12.13.

- Evaluation workflow installs now explicitly request `evals`, and the
model-allowed weekly job also requests `ml`, preserving its previous
intended dependencies. The evaluation guide and AI install index
identify those opt-ins. These address the existing Devin workflow/docs
findings.

- `uv lock --check`, touched Python Ruff/format and whitespace checks
pass. All 253 locked package versions are unchanged; only the root
`[all]` dependency metadata/flattened list changes.

- Real resolver commands from this checkout: `uv export --extra all
--no-dev --locked --no-hashes --no-emit-project` and `uv export --extra
all --extra ml --extra voice --extra evals --no-dev --locked --no-hashes
--no-emit-project`. Evaluating their actual requirement markers for
Python 3.12/3.13/3.14 on Linux x86_64, Windows AMD64, macOS arm64 and
macOS x86_64 shows `[all]` excludes torch, sentence-transformers,
datasets and CUDA/NVIDIA packages in all 12 environments. Explicit
opt-ins retain datasets and torch where supported; Intel macOS still
excludes torch.

- Not tested: a fresh full environment install, GPU runtime, live paid
evaluations or a fresh native build. Hosted CI must qualify the
published final head separately.



### Maintainer current-main requalification

Exact head `f1ed1e0019037e7b53ff9ef548843449c3c7a16f`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Current-main integration against upstream
c01cf07d94. The sole conflict was the
inherited Gemini fixture: adopted main's streamed-body helper after
verifying all six test-function ASTs unchanged. No contributor
dependency policy rewritten. The declared breaking change keeps
Torch-backed ml/memory/evals/voice opt-in while preserving the default
CLI/proxy bundle, image capability and main's lean profile. Evaluation
workflows and guides explicitly request removed extras; current
author/Devin discussion considered.

Fresh verification: 25 dependency/Gemini tests pass; offline uv lock
check resolves 253 packages; changed Python lint/format and
main-relative whitespace pass. Actual locked exports evaluated over 12
OS/Python marker environments (Linux/Windows/macOS, Python 3.10–3.13):
all excludes Torch/NVIDIA dependencies, explicit
all+ml+memory+evals+voice retains Torch. This is resolver/export proof,
not a fresh full installation or GPU runtime claim. No new source
blocker found; formal approval awaits exact-head hosted checks. Nothing
merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-06 09:23:03 -06:00
Shubham SrivastavaandJerrettDavis ce42754912 fix(sdk): keep Vercel AI SDK tool-result output kinds through compression (#4670)
## Description

`headroomMiddleware` / `withHeadroom` convert the Vercel AI SDK prompt
to OpenAI chat shape for `/v1/compress` and back. A `tool-result` part's
`output` is one of six kinds (`@ai-sdk/provider`, LanguageModelV3 and V4
alike): `text`, `json`, `error-text`, `error-json`, `execution-denied`,
and `content` (a list of text, file and custom members). The converters
flattened every kind to one string and the reverse side guessed the kind
back by trying `JSON.parse`. So after a round trip:

- **`error-text` / `error-json` came back as `text` / `json`.** The AI
SDK produces the error kinds for every tool that throws
(`createToolModelOutput`, `errorMode` text/json). `@ai-sdk/anthropic`
sets `is_error: true` only for those two kinds and `@ai-sdk/openai`
wraps them as `{"error": ...}`; both were gone, so a failed tool read as
a successful one.
- **`execution-denied` came back as a `json` value holding the denial
object** (`{ type: "json", value: { type: "execution-denied", reason }
}`), because it has no `value` and the whole output was stringified. The
providers send its `reason` (or "Tool call execution denied."), and
`@ai-sdk/openai` reads `output.providerOptions.openai.approvalId` from
it.
- **A `content` output** — what a tool's `toModelOutput` returns for a
screenshot, a PDF or an MCP result — **was JSON-stringified, compressed
as text (base64 included) and restored as `json`**, so the provider sent
the base64 as text instead of the `image` / `document` blocks
(`@ai-sdk/anthropic`) or `input_image` / `input_file` items
(`@ai-sdk/openai`, Responses) it builds from the members.
- Output-level `providerOptions` were dropped, and the guess flipped
kinds on its own: `text` `"42"` became `json` `42`, `json` `"ok"` became
`text` `"ok"`.

Captured on `main` (`ecbaf86d`) by the new tests before the change:

```text
error-text "ECONNREFUSED 10.0.0.7:443"  -> { type: "text", value: "ECONNREFUSED 10.0.0.7:443" }
error-json { code: 7 }                  -> { type: "json", value: { code: 7 } }
execution-denied "Not from CI."         -> { type: "json", value: { type: "execution-denied", reason: "Not from CI." } }
content [text, file, custom]            -> { type: "json", value: [ ...the members as JSON... ] }
text "42"                               -> { type: "json", value: 42 }
json "ok"                               -> { type: "text", value: "ok" }
output providerOptions                  -> dropped
```

Same carrier pattern as #4564. The output's `value` still travels as the
OpenAI tool message `content`, so the proxy compresses it exactly as
before; the rest of the output object (`type`, `reason`,
`providerOptions`) rides on the message under `output` and rebuilds the
same kind on the way back. A `content` output goes out as the tool
message's content array: `/v1/compress` compresses the top-level `text`
blocks of a tool message as fragments of one tool result and passes
every other block through untouched (the same block walk that already
carries the `reasoning` parts), so text members shrink and file / custom
members come back byte for byte, with no per-version mapping (V3
`image-data` / `file-url` / `file-id` and V4 `file` / `custom` members
alike). A JSON body the proxy no longer returned as JSON would come back
as the kind's text twin (`json` → `text`, `error-json` → `error-text`);
in practice the proxy keeps JSON tool bodies parseable (see the proof).

Closes # (none — follow-up to #4564, which listed this under "Not in
this PR")

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **`sdk/typescript/src/utils/format.ts`**
- `vercelToOpenAI`: a `tool-result` whose `output` is an object sends
its `value` as `content` — `text` / `error-text` as the string, `json` /
`error-json` JSON-encoded, `execution-denied` as `""`, `content` as the
member array — and the rest of the output under `output`. A bare
`output` value or the pre-v6 `result` field behaves as before.
- `openAIToVercel`: with the carrier present, rebuilds an output of that
kind from the returned content (`execution-denied` without a value;
`content` from the array; `json` / `error-json` parsed, text twin if the
body no longer parses; any other kind as the string). Without it, the
old JSON.parse guess stays for OpenAI-native tool messages.
- `openAIToGemini`: reads the tool body as text whichever shape it has
(follows the type widening; Gemini input never produces the array).
- **`sdk/typescript/src/types.ts`** — `ToolMessage.content` is `string |
Array<TextContentPart | ToolResultMediaPart>`, `ToolMessage.output?` is
the carrier, `ToolResultMediaPart` is the verbatim member type.
- **Tests** — `test/utils/format.test.ts` (13): a round trip per kind
including output-level `providerOptions` and the V4 and V3 member
shapes; the wire shape for `error-text`, `content` and
`execution-denied`; the `error-json` → `error-text` twin when a body
stops parsing, and a guard for the `json` → `text` twin.
`test/adapters/vercel-ai.test.ts` (1): a typed `LanguageModelV4Prompt`
with a denied, a multi-part and an error result through
`transformParams`, asserting on the restored results (one OpenAI tool
message is one result, so a multi-result tool message comes back as one
message per result, as before this PR).
- **Docs** — `docs/content/docs/vercel-ai-sdk.mdx`: one sentence on what
the output kinds keep.

**Not in this PR.** Regrouping the split tool messages into the AI SDK's
one-message-per-step shape (pre-existing; the providers group
consecutive tool messages themselves). The Gemini-native converter's
`thought` parts and `thoughtSignature` (separate PR).

## Testing

- [x] Unit tests pass (`npm test` — vitest, incl. typecheck)
- [x] Type checking passes (`npm run typecheck` — tsc --noEmit)
- [x] Build passes (`npm run build` — tsup)
- [x] New tests added for new functionality
- [x] Manual testing performed (below)

### Test Output

```text
$ cd sdk/typescript && npm test
 Test Files  19 passed | 5 skipped (24)
      Tests  449 passed | 33 skipped (482)
Type Errors  no errors

$ npm run typecheck      # tsc --noEmit — clean
$ npm run build          # tsup — ESM and DTS built
```

## Real Behavior Proof

- Environment: published proxy `headroom-ai 0.43.1` (`uvx --from
'headroom-ai[proxy]==0.43.1' headroom proxy --host 127.0.0.1 --port
8799`), this branch built with `npm run build`, `ai` 7.0.137 /
`@ai-sdk/provider` 4.0.26 types, Node v22.16.0, macOS.
- Exact command / steps: a LanguageModelV4 prompt whose one tool message
carries five results — `execution-denied` with a reason and
`openai.approvalId`; a `content` result with a 56 KB text member, a 24
KB base64 `file` member and a `custom` member; an `error-text` and an
`error-json` with 56 KB bodies; a 56 KB `json` — passed through
`headroomMiddleware().transformParams` against the live proxy, with the
request the proxy received and its answer captured.
- Observed result: every kind came back as itself, the text bodies
shrank by about 44%, the file and custom members and the denial came
back byte-identical. Full output:

```text
== wire shape the proxy received (tool messages) ==
  call_1: content=string(0) output={"type":"execution-denied","reason":"Not from CI.","providerOptions":{"openai":{"approvalId":"apr_1"}}}
  call_2: content=[text, file, custom] output={"type":"content"}
  call_3: content=string(56064) output={"type":"error-text"}
  call_4: content=string(56043) output={"type":"error-json"}
  call_5: content=string(56002) output={"type":"json"} messageProviderOptions=yes
== proxy answer: tokens 81695 -> 57772, transforms=["router:protected:system_message","router:text_block:mixed","router:mixed:0.45","router:smart_crusher:0.57","router:smart_crusher:0.45"]
== what the AI SDK gets back ==
system/user/assistant byte-identical: true
  results back: call_1=execution-denied, call_2=content, call_3=error-text, call_4=error-json, call_5=json; toolNames kept: true
  call_1 execution-denied byte-identical (reason + output providerOptions): true
  call_2 content: members back [text, file, custom]; text member 56022 -> 31308 chars; file member byte-identical: true; custom member byte-identical: true
  call_3 error-text: type=error-text, 56064 -> 31350 chars, still starts with the error line: true
  call_4 error-json: type=error-json, 56043 -> 31731 chars, value is an object with keys error/last_rows, last_rows now a string
  call_5 json: type=json, 56002 -> 31690 chars, value is a JSON string (the proxy returned the compact table as a JSON string literal): "[400]{id:int,level:string,msg:string,path:string,service:str"...
  message-level providerOptions on the last tool message only: -,-,-,-,yes
```

The JSON kinds: the proxy keeps a JSON tool body parseable — the array
body came back as a JSON string holding the compact table and the object
body kept its shape with the nested array compacted — so both parse and
keep their kind; the text-twin fallback did not trigger. The same prompt
through the converters on `main` loses every item listed in the
Description (the new tests, run before the change).
- Not tested: an end-to-end multi-step `generateText` with a denied
approval or an image-returning tool against a provider (no provider keys
in this environment); the provider behaviour is taken from the installed
`@ai-sdk/anthropic` 4.0.78 and `@ai-sdk/openai` 4.0.91 sources and the
`ai` 7.0.137 core cited above.

## Runtime Rollout Safety

- Rollout-managed feature(s): none (TypeScript SDK only, no proxy
change)
- Minimum rollout channel: n/a
- Stable/default behavior changed: for SDK users, a Vercel tool result's
OpenAI-shaped message now carries an `output` key, and a `content`
result sends its member array instead of a JSON string; `/v1/compress`
already passes both through. `ToolMessage.content` widens to `string |
array` (additive).
- Kill switch / disable path: unchanged — `fallback: true` returns the
original prompt when the proxy is unreachable; unwrap the model to
disable compression.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- The `output` carrier never reaches an OpenAI endpoint through the
SDK's own paths, like the #4564 carriers: the universal `compress()`
returns messages in the input format, and the OpenAI adapter never sees
Vercel input.
- Enumerated before writing: `LanguageModelV3ToolResultOutput` and
`LanguageModelV4ToolResultOutput` in `@ai-sdk/provider` 4.0.26. `ai` 7's
`LanguageModelMiddleware` is V4-based while the SDK's peer range (`ai >=
6.0.0`) still includes V3 prompts; both member shapes are carried the
same way, which is why the members are not mapped.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4670"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `e26525d382a135f1fe2a7f0b8671bfbae1f2d64a`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Reviewed the five-file SDK delta, adjacent conversion/middleware paths,
existing discussion and Devin no-issues result. The output carrier
retains denial/error/content kinds and output-level providerOptions;
JSON parse failures retain the error flag via error-text. Mixed content
members stay in the tool-message array. Current proxy route accepts
message dictionaries and returns pipeline messages; no new schema
stripping is introduced. Author runtime proof and its provider-inference
limitations were considered. Fresh verification on the clean mechanical
main merge: 449 tests passed, 33 skipped; typecheck and SDK build
passed; main-relative whitespace passed. No contributor logic changed.
Hosted CI must qualify the published merge head; no new provider-runtime
test or inference claim.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-06 08:52:20 -06:00
02c4e922f7 fix(cli): resolve active proxy port for read commands (#3522)
## Description

`headroom wrap copilot` can intentionally start a dedicated proxy on
port 8788
(or fall forward when 8787 is busy), but read-only CLI commands
previously
defaulted directly to 8787. As a result, `headroom dashboard` could open
a dead
or unrelated dashboard even though the wrapped session reported its
proxy on
8788.

This change extends the shared `headroom/cli/port_discovery.py` resolver
(introduced by #3855) to include live wrap-client markers, preferring
the
newest healthy session while preserving explicit `--port` and
`HEADROOM_PORT`
overrides.

Closes: N/A (no matching open issue found)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Extended the shared port discovery module from #3855 with live
wrap-client
marker validation (PID liveness and reuse checks), newest-session
ordering,
  and `/health` verification.
- Updated `dashboard`, `inspect`, and the runtime probe used by `learn`
to
  follow the newest healthy live wrap session after explicit and
  `HEADROOM_PORT` overrides.
- Kept `doctor`'s requested-port behavior; it now reports a healthy
proxy
  discovered from wrap-client markers elsewhere, with a `--port` hint.
- Added a warning when `dashboard` cannot reach the resolved port.
- Exported the actual proxy port as `HEADROOM_PORT` into every wrapped
agent
  environment.
- Reused #3855's port-discovery module rather than introducing a
parallel
  resolver.
- Added regression coverage for stale/recycled markers, multiple
sessions,
  explicit/environment overrides, dashboard output, doctor diagnostics,
  inspect/learn routing, and wrapped child environments.
- Documented automatic discovery and wrong-port troubleshooting.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_cli/test_proxy_port_discovery.py \
    tests/test_cli/test_dashboard_port_discovery.py \
    tests/test_cli/test_port_alignment.py \
    tests/test_cli/test_wrap_helpers.py \
    tests/test_cli_doctor.py tests/test_cli_inspect.py tests/test_cli_learn.py -q
267 passed in 5.49s

$ uv run ruff check .
All checks passed!

$ uv run ruff format --check .
1677 files already formatted

$ uv run mypy headroom/cli/port_discovery.py headroom/cli/proxy.py \
    headroom/cli/doctor.py headroom/cli/inspect.py headroom/cli/learn.py
Success: no issues found in 5 source files

$ uv run pre-commit run --all-files
Sync plugin versions.....................................................Passed
Verify Ruff version alignment............................................Passed
check for merge conflicts................................................Passed
ruff (legacy alias)......................................................Passed
ruff format..............................................................Passed
mypy.....................................................................Passed
```

After the first rebase, `make ci-precheck` completed `cargo fmt --check`
and
`cargo clippy --workspace -- -D warnings`, then stalled in the existing
Rust
Magika/detection tests (multiple tests exceeded 60 seconds). The same
stall
recurred during the push hook; it was stopped after confirming the
affected
suite. The final rebase onto #3883 was followed by the Python/CLI
regression
suite and all repository pre-commit gates above, but not another Rust
test run.

## Real Behavior Proof

- Environment: Ubuntu 22.04.5 LTS x86_64; Python 3.10.12; uv 0.12.10;
Headroom proxy 0.40.0-dev, started on isolated port 18788 with an
isolated live-client marker.
- Exact command / steps: Started an isolated process that wrote its live
PID marker and launched `headroom proxy --port 18788` under a temporary
workspace, with `HEADROOM_PORT` unset; then ran `env -u HEADROOM_PORT -u
HEADROOM_PORT_DISCOVERY
HEADROOM_WORKSPACE_DIR=/tmp/headroom-port-discovery-proof-20260930/workspace
./.venv/bin/headroom dashboard --no-open` and curled
`http://127.0.0.1:18788/dashboard`.

```text
$ env -u HEADROOM_PORT -u HEADROOM_PORT_DISCOVERY \
    HEADROOM_WORKSPACE_DIR=/tmp/headroom-port-discovery-proof-20260930/workspace \
    ./.venv/bin/headroom dashboard --no-open
  Detected live proxy on port 18788
  Dashboard: http://127.0.0.1:18788/dashboard

$ curl --max-time 5 -sS -o /dev/null -w '%{http_code}\n' \
    http://127.0.0.1:18788/dashboard
200

# Proxy health before discovery returned:
service=headroom-proxy status=healthy ready=true
```

- Observed result: With `HEADROOM_PORT` unset, the CLI selected the
marker-associated healthy proxy on 18788, printed the matching dashboard
URL, and the endpoint returned HTTP 200. The earlier manual run with a
real wrapped Copilot session also confirmed discovery on 8788 before
this rebase.
- Not tested: Starting a new interactive `wrap copilot` session after
this rebase (avoided launching an agent); Windows/macOS PID-identity
behavior; multiple simultaneous real proxies; browser UI launch. Unit
tests cover marker ordering, PID reuse, and explicit/environment
overrides.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: `dashboard`, `inspect`, and `learn`
now follow
the newest healthy live wrap proxy when no port override is set;
`doctor`
remains on the requested/default port and reports a live proxy
elsewhere.
- Kill switch / disable path: Pass `--port`, set `HEADROOM_PORT`, or set
  `HEADROOM_PORT_DISCOVERY=0` to bypass automatic discovery.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit; commands return to the fixed 8787
default.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from
  my Conventional Commit PR title

## Screenshots (if applicable)

N/A — terminal output above captures the behavior.

## Additional Notes

Port discovery is integrated into #3855's shared `port_discovery.py`;
candidates
are accepted only when `/health` identifies `service: headroom-proxy`,
so an
unrelated local JSON service cannot be mistaken for a Headroom proxy.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3522"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `e66a6393f83278aa07e9965f5b87fed820603677`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Re-reviewed latest integration head 420c2d636 and current author
response, including isolated workspace/port fixtures and live-marker
ordering/pruning regressions. Clean integration with upstream/main
2d3ee51846. Explicit CLI ports and
HEADROOM_PORT retain precedence; discovery bounds candidate probes,
verifies Headroom health identity, prefers newest live wrap markers, and
preserves configured-agent discovery. No new PR-specific correctness
blocker found in scope.

All 436
dashboard/port/discovery/wrap/doctor/inspect/learn/release/version/MCP
tests pass. All 14 changed Python files pass Ruff check/format;
whitespace passes. Previous cached-read freeze-boundary failure was
independently reproduced on main and is not a contributor-code blocker.
Maintainers handle fresh exact-head qualification and distinguish
inherited failures; no contributor CI execution requested. Existing
native extension supported startup, not a fresh native build. No
contributor discovery logic rewritten; nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-05 21:10:06 -06:00
HetCreepandClaude Opus 5.5 ecbaf86d78 docs(security): describe the per-route outcome of an unsafe base-URL override (#4660)
## Description

Documentation for #4433. `docs/content/docs/security-model.mdx` says a
rejected `x-headroom-base-url` override is "**ignored rather than
fatal** — the request proceeds to the configured upstream and a warning
is logged". On `e4a60ed7` that is true only for the OpenAI Chat
Completions and Responses handlers. Every other route that reads the
header answers `400 {"detail": "Rejected unsafe upstream base URL"}`
without contacting an upstream (`raise HTTPException` at
`headroom/providers/proxy_routes.py:129`, `:167`, `:327`, `:533`,
`:582`, `:600`).

This PR changes the paragraph to describe what the code does today, per
route group. It does not change behaviour and does not pick a direction:
#4433 lists the two ways to make the routes consistent, and that choice
stays with the maintainers. If the code is unified later, this paragraph
can go back to one sentence.

Refs #4433

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `docs/content/docs/security-model.mdx`, "Where it may point": the
"ignored rather than fatal" sentence is replaced. The OpenAI Chat
Completions and Responses handlers ignore an unsafe override and use the
configured upstream. The other routes that read the header reject the
request with HTTP 400 before contacting any upstream. The routes in each
group are named. The rest of the section, including
`HEADROOM_ALLOWED_BASE_URLS`, is unchanged.

## Testing

- [ ] Unit tests pass (`pytest`) (docs-only change; no code touched)
- [ ] Linting passes (`ruff check .`) (no Python changed)
- [ ] Type checking passes (`mypy headroom`) (no Python changed)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# every route of create_app(...) on e4a60ed7, requested three ways (see Real Behavior Proof)
# C = configured upstream contacted, O = override upstream contacted, - = no upstream
POST /v1/chat/completions        control=200/C  allowed=200/O  unsafe=200/C
POST /v1/responses               control=200/C  allowed=200/O  unsafe=200/C
POST /v1/messages                control=200/C  allowed=200/O  unsafe=400/-  Rejected unsafe upstream base URL
POST /v1/embeddings              control=200/C  allowed=200/O  unsafe=400/-  Rejected unsafe upstream base URL
GET  /v1/models                  control=200/C  allowed=200/O  unsafe=400/-  Rejected unsafe upstream base URL
POST /v1beta/models/x:generateContent  control=200/C  allowed=200/O  unsafe=400/-  Rejected unsafe upstream base URL
POST /v1/some/passthrough        control=200/C  allowed=200/O  unsafe=400/-  Rejected unsafe upstream base URL
```

## Real Behavior Proof

- Environment: Ubuntu (WSL2) on Windows 11, Python 3.12.15, `headroom`
Python source at `e4a60ed7`, offline. Every upstream is a local HTTP
server on `127.0.0.1`.
- Exact command / steps: built the real app with
`create_app(ProxyConfig(optimize=False, cache_enabled=False,
rate_limit_enabled=False, ...))`, with every provider API URL pointing
at a recording server C, and `HEADROOM_ALLOWED_BASE_URLS` set to a
second recording server O. Under `TestClient` with the app lifespan
running, every registered route was requested three times: without the
header (control); with `x-headroom-base-url` set to server O, which is
allowed; and with `x-headroom-base-url: http://127.0.0.1:1/v1`, which is
unsafe. Each run was repeated with an `api-key` header (the Azure
branch) and gave the same results. The script is below.
- Observed result: three groups.
- Ignore and use the configured upstream (allowed override reaches O,
unsafe override reaches C with status 200): `/v1/chat/completions`,
`/chat/completions`, `/inference/v1/chat/completions`, `/v1/responses`,
`/responses`, `/v1/codex/responses`, `/backend-api/responses`,
`/backend-api/codex/responses`.
- Reject with 400 and contact nothing (allowed override reaches O,
unsafe override returns `400 Rejected unsafe upstream base URL`):
`/v1/messages`, `/v1/messages/count_tokens`, `/v1/models`,
`/v1/models/{id}`, `/v1/embeddings`, `/v1/moderations`,
`/v1/audio/speech`, `/v1/audio/transcriptions`, the native Gemini routes
(`:generateContent`, `:streamGenerateContent`, `:countTokens`,
`:embedContent`, `:batchEmbedContents`, `/v1beta/models`,
`/v1beta/cachedContents`), `/api/hello`, and the catch-all passthrough.
`/v1/alpha/search` also returns 400, although the allowed override did
not move it off the configured upstream.
- The header has no effect (all three runs reach C): the batch routes,
Vertex, `/anthropic/v1/messages`, images, live, and the Responses
subpaths. These were left out of the doc text because the paragraph is
about routes that read the header.
- Not tested: real provider upstreams (the recorders return a canned
200), the Rust proxy, and the WebSocket Responses route.

<details><summary>Probe script</summary>

Run from a checkout of `e4a60ed7` with the proxy's Python dependencies
installed: `PYTHONPATH=$PWD python probe.py`.

```python
"""Probe #4433: what each proxy route does with an unsafe x-headroom-base-url.

Every configured upstream points at a local recording server. The unsafe
override names 127.0.0.1:1 (loopback, nothing listens). For each route we send
the same request twice: without the header (control) and with it, and record
the HTTP status, the error detail and whether the recorder was hit.
"""

import json
import os
import re
import sys
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

os.environ.update(HEADROOM_OFFLINE="1", DO_NOT_TRACK="1", HEADROOM_BEACON_ENABLED="0")
HITS = []


class H(BaseHTTPRequestHandler):
    def log_message(self, *a):
        pass

    def _any(self):
        n = int(self.headers.get("Content-Length") or 0)
        self.rfile.read(n)
        HITS.append(self.server.server_port)
        body = json.dumps({
            "id": "x", "object": "chat.completion", "model": "gpt-4o-mini", "created": 0,
            "choices": [{"index": 0, "finish_reason": "stop",
                         "message": {"role": "assistant", "content": "ok"}}],
            "usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2},
            "type": "message", "role": "assistant", "content": [{"type": "text", "text": "ok"}],
            "data": [], "output": [],
        }).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    do_POST = do_GET = do_PUT = do_DELETE = _any


srv = ThreadingHTTPServer(("127.0.0.1", 0), H)
threading.Thread(target=srv.serve_forever, daemon=True).start()
REC = f"http://127.0.0.1:{srv.server_port}"
srv2 = ThreadingHTTPServer(("127.0.0.1", 0), H)
threading.Thread(target=srv2.serve_forever, daemon=True).start()
REC2 = f"http://127.0.0.1:{srv2.server_port}"
os.environ["HEADROOM_ALLOWED_BASE_URLS"] = REC2

from fastapi.testclient import TestClient  # noqa: E402
from headroom.proxy.server import ProxyConfig, create_app  # noqa: E402

app = create_app(ProxyConfig(
    optimize=False, cache_enabled=False, rate_limit_enabled=False,
    anthropic_api_url=REC, openai_api_url=REC, gemini_api_url=REC,
    cloudcode_api_url=REC, vertex_api_url=REC,
))
client = TestClient(app, raise_server_exceptions=False)
client.__enter__()

UNSAFE = "http://127.0.0.1:1/v1"
BODY = {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "hi"}],
        "input": "hi", "max_tokens": 5, "prompt": "hi", "query": "hi"}

routes = []
for r in app.routes:
    methods = getattr(r, "methods", None)
    if not methods or not hasattr(r, "path"):
        continue
    path = r.path
    if path.startswith(("/health", "/stats", "/metrics", "/admin", "/dashboard", "/docs",
                        "/openapi", "/redoc", "/debug", "/livez", "/readyz", "/v1/compress",
                        "/cache", "/subscription", "/quota", "/transformations", "/memory",
                        "/telemetry", "/config", "/session", "/prefs")):
        continue
    concrete = re.sub(r"\{model_id(:path)?\}", "gpt-4o-mini", path)
    concrete = re.sub(r"\{path:path\}", "v1/some/passthrough", concrete)
    concrete = re.sub(r"\{[^}]+\}", "x", concrete)
    m = "POST" if "POST" in methods else sorted(methods)[0]
    routes.append((m, concrete, r.endpoint.__name__))


def call(m, p, extra):
    HITS.clear()
    h = {"authorization": "Bearer sk-local", "content-type": "application/json", **extra}
    try:
        resp = client.request(m, p, headers=h, json=BODY if m != "GET" else None)
        detail = resp.text[:90].replace("\n", " ")
        return resp.status_code, detail, "".join("C" if x == srv.server_port else "O" for x in HITS) or "-"
    except Exception as e:  # noqa: BLE001
        return "EXC", repr(e)[:90], "-"


rows = []
for m, p, name in sorted(set(routes)):
    for label, extra in (("plain", {}), ("azure", {"api-key": "azure-local"})):
        c = call(m, p, extra)
        u = call(m, p, {**extra, "x-headroom-base-url": UNSAFE})
        a = call(m, p, {**extra, "x-headroom-base-url": REC2 + "/v1"})
        rows.append((m, p, name, label, c, u, a))

for m, p, name, label, c, u, a in rows:
    print(f"{m}|{p}|{name}|{label}|control={c[0]}/{c[2]}|allowed={a[0]}/{a[2]}|unsafe={u[0]}/{u[2]}|{u[1] if u[0] != c[0] else ''}")
srv.shutdown()
srv2.shutdown()
```

</details>

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A
- Stable/default behavior changed: No (documentation only).
- Kill switch / disable path: N/A
- Unsafe override required: No
- Qualification impact: None
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Picked up after Ehsan-hash withdrew the offer to write this
(https://github.com/headroomlabs-ai/headroom/issues/4433#issuecomment-6654267536).
The line numbers in my 2026-10-24 comment on #4433 predate the two route
guards added since; the ones above are from `e4a60ed7`.

Produced with AI assistance (Claude Code) operated by me (HetCreep); the
probe was run on my machine against a clone of the repository.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4660"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 07:30:31 -06:00
HetCreepandJerrettDavis ee975d49c0 docs(security): sync security model with egress pollers, counts and beacon payload (#4539)
## Description

Documentation only. Brings `security-model.mdx`, `SECURITY.md`,
`architecture.mdx` and the `telemetry/session.py` docstrings back in
line with the code on main. Every item below was re-checked against
upstream main `a2d4fe72`; line numbers are that commit's.

Fixed:

1. Egress table: added `api.anthropic.com`
(`headroom/subscription/client.py:30` `_USAGE_URL`; default on,
`proxy/models.py:488-489`, 300 s; only polls when
`read_cached_oauth_token()` finds a token, `client.py:52-81`; switch
`HEADROOM_NO_SUBSCRIPTION_TRACKING`, `cli/proxy.py:568`; `guard_egress`
at `client.py:113` so `HEADROOM_OFFLINE` blocks it).
2. Egress table: added `api.github.com` Copilot quota poll
(`subscription/copilot_quota.py:6-14` token list, `:218-222` default 60
s, `:318` `guard_egress`; registered unconditionally at
`proxy/server.py:2359`). I found no dedicated disable switch, so the row
says so and names the two ways to stop it (unset the token variables, or
`HEADROOM_OFFLINE`).
3. "seven runtime knobs" is now "ten". `RUNTIME_ENV_KNOBS`
(`proxy/runtime_env.py:60`) has 10 `Knob(` entries, counted with `ast`.
The table on the same page already lists ten rows. (The issue said
eight; that was stale.)
4. "59" is now "63" in three places. `SETTINGS` (`settings_store.py:64`)
has 63 `SettingField(` entries, counted with `ast`.
5. Request log row: "Inherits umask" is now "`0600`, owner-only".
`proxy/request_logger.py:163` opens the file with
`fileperms.open_owner_only`.
6. Beacon payload: `security-model.mdx` and `SECURITY.md` now mention
the schema v2 sections and the 15-variable `config` allow-list
(`telemetry/session.py:134` `SCHEMA_VERSION = 2`; snapshot keys at
`:823-1175`; `_CONFIG_ENV_KEYS` at `:359-373`).
7. `architecture.mdx` TOIN sentence: it said nothing is "sent off the
box", but `toin.py:683` calls `record_tool_shape` before TOIN's own
enabled check, and with the beacon on (default, `beacon.py:59`) a
bucketed structural descriptor goes into `shapes.by_tool`
(`session.py:1051-1070`, `:2031-2100`). The sentence now says so and
links the beacon section.
8. `telemetry/session.py` docstrings (module docstring and the entry
points at `record_compression`, `record_content_shape`,
`record_outcome`) said the beacon is off unless `HEADROOM_TELEMETRY` is
on. `beacon.py:1-12` and `is_beacon_enabled()` say on by default.
Comment text only.

Also changed: one sentence in the beacon section that called the beacon
"the one egress path that is enabled without any action from you". The
two pollers are also on by default (token-gated), so it now says
"enabled by default whatever credentials you have".

Dropped: nothing. All seven items from the issue and its comment still
held on main. Item 3's number was corrected from eight to ten.

Not covered, noticed while reading: `docs/content/docs/proxy.mdx:33`
still describes the older, smaller beacon payload and "about 2KB";
`headroom/subscription/codex_rate_limits.py:454` is a third poller
(Codex rate limits) that is not in the egress table. I left both for a
separate change.

Refs #4422

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `docs/content/docs/security-model.mdx`: two egress rows, three counts,
request-log permissions, beacon v2 payload bullet, one wording fix.
- `SECURITY.md`: beacon payload sentence.
- `docs/content/docs/architecture.mdx`: TOIN sentence.
- `headroom/telemetry/session.py`: docstring text only, no code change.

## Testing

- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ ruff check headroom/telemetry/session.py
All checks passed!
$ ruff format --check headroom/telemetry/session.py
1 file already formatted
$ PYTHONPATH=<worktree> python -c "import headroom.telemetry.session"
import ok (SCHEMA_VERSION 2)
$ pytest -q tests/test_packaging_extras_and_security_docs.py
6 passed in 2.54s
```

Ruff was run on the one changed Python file, not the whole tree. mypy
and the full test suite were not run (docstring-only change).

## Real Behavior Proof

- Environment: WSL Ubuntu, Python venv, worktree of upstream main
a2d4fe72 plus this one commit.
- Exact command / steps: counted `Knob(` in `RUNTIME_ENV_KNOBS` and
`SettingField(` in `SETTINGS` with Python `ast`; read each cited code
path; ran `ruff check`, `ruff format --check`, the module import and
`tests/test_packaging_extras_and_security_docs.py`.
- Observed result: 10 knobs, 63 settings fields; ruff clean; module
imports; 6 tests passed.
- Not tested: the pollers were not run against the live Anthropic or
GitHub APIs, the docs site was not built, and the mdx table rendering
was not previewed.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: no, documentation and docstring text
only
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

n/a

## Additional Notes

Unticked checklist items are n/a for a documentation change. The counts
in the docs are still hand-maintained; a check that derives them from
`RUNTIME_ENV_KNOBS` and `SETTINGS` (option 2 in the issue, related to
#3181) would stop this drifting again, but that is a separate change.

Produced with AI assistance (Claude Code) operated by me (HetCreep);
every count and line above was checked against a clone of upstream main
a2d4fe72.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4539"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-04 22:56:20 -06:00
662a7f6fd8 fix(memory): keep learned evidence until persistence succeeds (#4229)
## Description

Closes #4224 and #4226.

A full queue or failed save no longer makes the learner forget a pattern
permanently. Observations received while the save is queued or running
are included too. `min_evidence=1` saves the first sighting.

The review caught another case: SQLite can commit before vector indexing
fails or the task is cancelled. Retrying that save now reuses the
committed row rather than inserting a duplicate. If index repair fails
again, the pattern and its save token stay pending until repair
succeeds.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update

## Changes Made

- Keep queued patterns as live evidence aggregates until persistence
completes.
- Retain the complete aggregate and its save-attempt token after
failure, cancellation or restart.
- Reconcile only the missing evidence. A delayed SQLite update from a
cancelled task cannot count the same observations twice or move
`last_seen_at` backwards.
- Add real SQLite regressions and document recovery behavior.

## Testing

- [ ] Unit tests pass (pytest) — complete changed learner file verified;
not the entire Python suite locally.
- [ ] Linting passes (ruff check .) — changed-file checks and normal
commit hooks passed.
- [ ] Type checking passes (mypy headroom) — normal mypy commit hook
passed.
- [x] Manual testing performed

### Test Output

```text
Complete tests/test_memory/test_traffic_learner.py:
230 passed in 16.40s
```

The complete learner file ran with the normal installed pytest plugins;
no test selection or warning filters were changed.

The normal full `make ci-precheck` pre-push gate also passed: Rust
formatting, clippy and workspace tests; native extension build; all 181
critical Python tests; commitlint. Rust tests used the supported
`RUST_TEST_THREADS=1` setting; no tests, timeouts or gates were changed
or bypassed.

## Real Behavior Proof

- Environment: Linux x86-64, Python 3.13.14, locally rebuilt native
extension, actual `LocalBackend` with `embedder_backend="onnx"`, SQLite,
and default cache/model settings. No live LLM provider requests.
- Exact command / steps: pass `No, use uv instead of pip for Python
packages.` through the public `TrafficLearner.on_messages` at threshold
three. Make the first three actual vector-index calls raise, then let
the fourth call the original index method. Add observations between
attempts; inspect SQLite, pending ownership and actual search after each
attempt. Persistence, embeddings, cache and retrieval are real; only the
failure boundary is controlled.

The learner tests also cover a rejected SQLite UPDATE, shutdown
continuing after one queued save fails, and index-refresh failure on
already-confirmed evidence.

- Observed result: all phase assertions passed and the standalone
process exited 0.

| Attempt | Rows | SQLite evidence | Pending | Search evidence |
| --- | ---: | ---: | --- | --- |
| Initial index failure | 1 | 3 | yes | not found |
| First repair failure | 1 | 5 | yes | not found |
| Second repair failure | 1 | 6 | yes | not found |
| Successful repair | 1 | 7 | no | 7 |

The original memory ID and save token survive failed repairs. Only the
successful repair marks the pattern saved. Normal evidence-only update
handling is unchanged.

- Not tested: live providers, model quality, Windows/macOS behavior, or
the entire Python suite. Hosted checks for this revision are not claimed
here.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; existing live learning.
- Minimum rollout channel: unchanged.
- Stable/default behavior changed: failed writes retain evidence;
explicitly configured threshold one works; observations during saves are
no longer lost.
- Kill switch / disable path: `--no-learn` or existing memory
configuration.
- Unsafe override required: no.
- Qualification impact: actual persistence and retrieval verified
locally.
- Rollback path: revert the fix; no schema migration.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have made corresponding changes to the documentation
- [x] I did not edit CHANGELOG.md — release-please owns it

## Additional Notes

AI-assisted; no subagents.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4229"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <tejas@headroomlabs.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 22:39:53 -06:00
Mansurshakh JaparovandJerrettDavis 50bf775e8c fix(proxy): bypass compression for large admitted histories (#4142)
## Description

Closes #4139.

A history that is too large to compress should still reach the provider.
This keeps the 100 MiB hard admission limit and bypasses request-side
transforms above 16 MiB decoded size. The hard limit still applies
before forwarding.

Current head: `62ffe3defef1684f70d96f1640818d99445a43bb`, merging
current main `a2d4fe72` into `1902d8ef`. The
42-test/native/manual/pre-push results below qualify `1902d8ef`, not
this new merge. New-head shard 4 has a cached-prefix fixture failure
under investigation; coverage uploads are still incomplete.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement
- [x] Documentation update

## Changes Made

- Keep the startup override and both wire/decompressed hard bounds.
Large admitted histories skip normalization, image processing, memory
injection, tokenization and compression on Messages, Chat/Responses,
Gemini and Bedrock HTTP routes.
- Keep provider routing and backend translation, including Copilot Chat,
Codex `store:false` and Vertex route-forced streaming. Native forwarding
retains decoded bytes unless the provider's wire contract requires a
rewrite; translation retains the full message text.
- Report actual provider usage, with zero compression savings or
attempted tokens for the bypass. Keep `/v1/compress` and WebSocket
contracts unchanged.
- Add real-provider coverage for large Gemini JSON/SSE requests with
query-key auth, and OpenAI JSON/SSE through the installed LiteLLM
adapter. No admission-policy or CI changes in this follow-up.

## Testing

- [x] Unit tests pass (focused suite and normal critical pre-push suite)
- [x] Linting passes (normal hooks)
- [x] New tests added for admission, recovery and passthrough accounting
- [x] Manual testing performed against the real native build

### Test Output

```sh
python -m pytest -q tests/test_proxy_compression_admission.py tests/test_proxy/test_gemini_plugin_routed_streaming.py --cov=headroom --cov-branch
# 42 passed, 41 warnings
```

Normal pre-push `make ci-precheck` passed: 1668 Rust tests (3 ignored),
181 Python tests, lint/type checks and commitlint. No hook bypass. The
initial deep-module coverage command hit a Pydantic import-isolation
error; using the repo's `--cov=headroom` convention passed both complete
modules without disabling plugins or tests.

The earlier `96c5fe7e` qualification covered the other native protocols,
configured/per-request backends, hard-ceiling recovery and complete
local shards 2/3. Those runs are historical, not a new whole-suite
claim. New-head hosted success is not claimed.

## Real Behavior Proof

- Environment: Linux x86_64, Python 3.13.14, freshly built real
`headroom._core`, public Headroom ASGI routes, a real localhost HTTP
provider and installed LiteLLM. Synthetic credentials, offline mode and
`DO_NOT_TRACK=1`; no live provider calls.
- Exact command / steps: Start
`create_app(ProxyConfig(backend="litellm-openai", ...))` with all
upstream URLs set to the localhost peer and
`litellm.api_base=peer+"/v1"`. Leave the production 16 MiB soft/100 MiB
hard limits unchanged. POST `"history " * (16777216 // 8 + 1024) + "LAST
TURN"` as a user message to Chat Completions, then as a Gemini content
part to generateContent/streamGenerateContent, with streaming off/on.
The peer rejects a changed history, requires Gemini query-key auth and
one `alt=sse`, and returns protocol-valid JSON/SSE with 101 input/1
output tokens. The maintained four small-ceiling cases are in
`tests/test_proxy_compression_admission.py` and run with the command
above.
- Observed result: All four >16 MiB requests returned 200 with the full
history intact. JSON/SSE and Gemini auth survived; streamGenerateContent
forced SSE even with incoming `alt=json`. Each request recorded exactly
101 provider input/1 output tokens, zero compression attempts/savings
and no failures; `/livez` stayed 200. Earlier recorded qualification
also covered all five native HTTP protocols, real translating backends
and 413 recovery.
- Not tested: Live cloud accounts, the reporter's deployed 4 GB VPS,
screenshot/image quality or a zero-JSON-parse path. One envelope/model
JSON parse remains; no proxy-only RSS guarantee is claimed.

## Runtime Rollout Safety

- Rollout-managed feature(s): None; this is the admission/compatibility
fix.
- Minimum rollout channel: Stable/default.
- Stable/default behavior changed: Keep the 100 MiB hard ceiling; bypass
request-side transforms above 16 MiB decoded size on the named HTTP
routes.
- Kill switch / disable path: Set
`HEADROOM_REQUEST_BODY_MAX_BYTES=16777216` (or lower) and restart,
accepting local 413s instead of the large-body path.
- Unsafe override required: No.
- Qualification impact: Local native protocols, backend translation,
hard bounds and recovery exercised; live-provider/VPS qualification
remains external.
- Rollback path: Revert the admission revision and restart; no storage
migration.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] I have made corresponding changes to the documentation
- [x] I did not edit `CHANGELOG.md`

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-04 22:37:21 -06:00
31dcaf41f7 feat(ccr): answer headroom_retrieve on streaming chat completions (#4631)
## Description

Streaming `/v1/chat/completions` compresses tool output into retrieval
markers but cannot answer `headroom_retrieve`, so the content is lost to
the model on this path. Sessions that mix streaming and non-streaming
turns also flip the tool in and out of `tools` and bust the prefix each
way.

Behind a new default-off flag, a streaming turn that carries a marker
now runs as `stream: false` upstream through the same code as a
non-streaming turn, then goes back as `chat.completion.chunk` frames.
Builds on the groundwork in `feat/buffered-ccr-chat-streaming` and the
#3816 continuation loop.

Part of #3815. Flipping the default stays a separate decision, as the
issue asks.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `--ccr-buffered-chat-streaming` /
`HEADROOM_CCR_BUFFERED_CHAT_STREAMING`, on the CLI too because `headroom
proxy` builds its own config.
- The non-streaming branch became `_direct_chat_turn()` at the same
indent, so usage, the prefix tracker and savings are shared rather than
copied. `git diff` shows almost no moved lines there.
- Two gates. Eligibility is decided before injection and lets the tool
stay resident. Buffering is decided at dispatch from the final body, so
a turn with no marker keeps its real stream (the #3071 lesson).
- `_buffered_chat_reply_as_sse` wraps the JSON reply: non-200 passes
through with its status, a lone leftover retrieval fails closed with the
continuation's status. A retrieval next to a client tool follows
`handle_response`: a successful continuation replaces the turn, and only
a failed one goes back to the client whole.
- `stream_options` is dropped upstream since OpenAI answers 400 to it
without a stream. The usage chunk follows the same rule as a plain
stream.
- Frames and the OpenAI continuation now carry other non-empty message
fields, and frames keep opaque tool call fields, so `reasoning_content`,
`reasoning_details` and Gemini thought signatures survive.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`): see Test Output, no errors
in changed files
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest <every file touching the chat route, handle_response, ccr, buffered or settings_store>
3067 passed, 73 skipped
$ ruff check . && ruff format --check .
All checks passed! 1881 files already formatted
$ mypy headroom
2 errors, both in files this PR does not touch (memory/adapters/embedders.py, integrations/langchain/chat_model.py)
```

The full suite did not finish locally (stopped at a time limit about 40%
through, no failures to that point), so CI is the full run. `cargo`
checks were not run since no Rust changed.
`tests/test_openai_chat_buffered_ccr.py` covers the list in the issue:
frames ending in `[DONE]`, `finish_reason: tool_calls`, status fidelity
inside the grace window, heartbeat past it, identical accounting against
a non-streaming turn, byte-stable tools across a mixed session, flag off
stays on the plain stream.

## Real Behavior Proof

- Environment: macOS 27 arm64, Python 3.13, real `headroom proxy --mode
token` in front of a local stub upstream, real `openai` 2.41 SDK client
with `stream=True`. No provider keys.
- Exact command / steps: `headroom proxy --port 18932 --mode token
--openai-api-url http://127.0.0.1:18931
[--ccr-buffered-chat-streaming]`, then one turn with a 1,500 line log as
the tool result. It compresses to a `Retrieve more: hash=` marker. The
stub calls `headroom_retrieve` when it is offered and a marker is
present.
- Observed result: flag off, the model only gets the summary. Flag on,
the retrieval is answered inside the proxy and the SDK reads a normal
stream with usage.

```text
flag off
  upstream: stream=true  tools=[list_rows]
  client:   "I can only see a compressed summary."

flag on
  upstream: stream=false tools=[list_rows, headroom_retrieve] reply=tool_call
  upstream: (continuation)                                    reply=text
  log:      CCR: buffering streaming chat turn (stream:false upstream, resynthesized as chat.completion.chunk)
            CCR: retrieval tool call on the direct chat path, resolving before replying
  client:   4 chunks, "Row row-0007 is in the original output.", finish_reason=stop, usage present
```

- Not tested: a real provider, Copilot, xAI, or a gateway with its own
strict validation. Windows.

## Runtime Rollout Safety

- Rollout-managed feature(s): none, plain opt-in flag
- Minimum rollout channel: n/a
- Stable/default behavior changed: no
- Kill switch / disable path: flag is off by default
- Unsafe override required: no
- Qualification impact: none while off
- Rollback path: unset the flag

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- Memory tools still only go in on non-streaming turns, so with memory
on a mixed session keeps that part of the toggle. A turn with no marker
cannot run the memory loop, so I left it.
- Turn hooks treat a buffered turn like a stream: stream-safe
`on_request` only, no `on_response`. Buffering is decided after hooks
run, so anything else would split a hook's two halves.
- The issue mentions a ChatGPT-auth exclusion. The chat handler has no
such routing, so there was nothing to exclude.
- Past the grace window `x-headroom-*` headers are lost, same as the
shared wrapper on the other paths.
- #3600 picks a marker-free pipeline for streaming chat. If it lands, an
eligible turn needs to keep the marker pipeline.

---------

Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-04 14:41:30 -06:00
92c4407e0e docs: add Kong AI Gateway integration page (#4625)
## Description

Adds a Kong AI Gateway page to the Integrations section. Headroom can
run as the compression service behind Kong AI Gateway's AI Prompt
Compressor policy (AI Gateway 2.2 and later): Kong sends each request to
Headroom and forwards the compressed request to the LLM provider, so
applications that call the gateway get compression without installing
Headroom. The page follows the layout of the other integration pages:
How it works, Installation (running Headroom so a remote data plane can
reach it), Quick start (the policy file, applied with kongctl, picking
up where Kong's getting started guide ends), and Options.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `docs/content/docs/kong-ai-gateway.mdx`: new integration page with How
it works, Installation, Quick start, and Options sections.
- `docs/content/docs/meta.json`: lists the page under Integrations,
before LiteLLM.

## Testing

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ cd docs && npm ci && npm run build
✓ Compiled successfully in 3.8s
✓ Generating static pages using 12 workers (191/191) in 1175ms
exit=0
(the build's only warnings are the two existing CSS notices for ::highlight(fd-search) from fumadocs-ui)

Links on the rendered page, checked against the target pages:
  OK  /docs/proxy#post-v1compress
  OK  /docs/proxy#session-mode
  OK  /docs/docker-install
  200 https://developer.konghq.com/ai-gateway/
  200 https://developer.konghq.com/ai-gateway/policies/ai-prompt-compressor/
  200 https://developer.konghq.com/ai-gateway/get-started/
  200 https://developer.konghq.com/operator/konnect/how-to/ai-gateway-headroom/
Sidebar order: AutoGen, Kong AI Gateway, LiteLLM
```

## Real Behavior Proof

- Environment: macOS (Darwin 25.6.0), Python 3.14, headroom-ai 0.42.0
installed with `pip install "headroom-ai[proxy]==0.42.0"`, kongctl
1.15.0, Kong AI Gateway 2.2.0 data plane (`kong/kong-ai-gateway:2.2.0`,
Kong Gateway 3.15.0.6) in Docker Desktop connected to Kong Konnect (US),
OpenAI `gpt-4.1-mini` as the provider. The gateway has two AI Models:
one that lists the policy and one with `policies: []`.
- Exact command / steps: started Headroom with the page's command, found
the gateway ID with `kongctl get ai-gateways -o yaml`, applied the
page's `headroom-policy.yaml` unchanged with `kongctl apply -f
headroom-policy.yaml`, sent a 200-record JSON tool result to both
models, then reproduced each failure the page mentions. Output below.
- Observed result: with `global: true`, the model with `policies: []`
went through Headroom (one `/v1/compress` call, 10,250 input tokens down
to 5,122), and the model that also lists the policy still made exactly
one call. In the conversation test, OpenAI counted 10,272 input tokens
without compression and 5,145 with it (50% fewer), the answer still
listed the three error records, and the follow-up turn on the same
`x-session-id` was 49% smaller with 4,992 of its input tokens served
from OpenAI's prompt cache.
- Not tested: the Kubernetes path through Kong Operator, a data plane on
a separate host (the token check was exercised in process instead, see
below), kongctl 1.24 or later (Kong's getting started guide now asks for
1.24.0+), AI Gateway versions other than 2.2.0, providers other than
OpenAI, and streaming requests.

```text
$ HEADROOM_COMPRESS_ALLOW_REMOTE=1 HEADROOM_PROXY_TOKEN=<token> headroom proxy --host 0.0.0.0 --port 8787
$ curl -s localhost:8787/livez
{"service":"headroom-proxy","status":"healthy","alive":true,"version":"0.42.0",...}

$ kongctl get ai-gateways -o yaml        # prints display_name and the full id of each gateway
$ kongctl apply -f headroom-policy.yaml
...
Do you want to continue? Type 'yes' to confirm:
Executing changes:
[1/1] [namespace: default] Updating ai_gateway_policy: headroom-compressor... ✓
Complete.

With the policy global (input tokens = OpenAI usage.prompt_tokens):
  model with policies: []         input tokens 5,122   Headroom calls for this request: 1
  model that lists the policy     input tokens 5,122   Headroom calls for this request: 1
With the policy attached to one model only (global: false):
  model with policies: []         input tokens 10,250  Headroom calls for this request: 0
  model that lists the policy     input tokens 5,122   Headroom calls for this request: 1

Headroom log for a compressed request:
  Pipeline complete: 10275 -> 5148 tokens (saved 5127, 49.9% reduction)
$ curl -s localhost:8787/stats   # summary.compression
  {'requests_compressed': 2, 'avg_compression_pct': 49.6, 'best_detail': '10,275 → 5,148 tokens', 'total_tokens_removed': 10254}

Failures, through Kong (the client sees 500 "failed to compress prompts: Failure when calling the Headroom compression service"):
  Headroom without HEADROOM_COMPRESS_ALLOW_REMOTE=1  -> data plane log: "404 from /v1/compress: this is NOT a missing route — ..."
  Headroom stopped                                    -> data plane log: "request failed: connection refused"
  POST / (a compressor_url without /v1/compress)      -> HTTP 421 {"message": "Welcome to the OpenAI API! ..."}
  headroom proxy --host 0.0.0.0 without a token       -> Error: Refusing to bind '0.0.0.0': ...
  token check, in-process TestClient (non-loopback)   -> no token 401, wrong token 401, Kong's headers 200
```

## Runtime Rollout Safety

- Rollout-managed feature(s): None, documentation only.
- Minimum rollout channel: N/A, documentation only.
- Stable/default behavior changed: No.
- Kill switch / disable path: N/A, documentation only.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert the commits.

## Review Readiness

- [x] I have performed a self-review
- [ ] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not included. The page uses the existing docs components only
(`Callout`, a table, and code blocks) and was checked locally with `npm
run dev` and `npm run start`.

## Additional Notes

- Unchecked items are N/A for a docs-only change: no code, tests, or
type changes.
- Docker Desktop presents the data plane to Headroom as `127.0.0.1`, and
Headroom checks `HEADROOM_PROXY_TOKEN` only for non-loopback callers, so
the end-to-end run couldn't exercise the token check. I verified it in
process against 0.42.0's app with a non-loopback client (results above);
the source is `_security_gate` in `headroom/proxy/server.py`.
- Kong-side behavior on the page comes from Kong's [policy
reference](https://developer.konghq.com/ai-gateway/policies/ai-prompt-compressor/#headroom-compressor-service),
Kong's getting started guide, and the policy's Lua source in the
`kong/kong-ai-gateway:2.2.0` image
(`kong/llm/compression/headroom/init.lua`).
- This page covers the policy built into Kong AI Gateway. The proxy
page's existing "Kong plugin" section covers `kong-plugin-headroom` for
Kong Gateway 3.9; the repository it links to
(https://github.com/headroomlabs-ai/kong-plugin-headroom) returns 404
for anonymous visitors.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-04 09:15:30 -06:00
d9cee45268 fix: [BUG] DirectMem0Adapter.search_memories calls removed mem0 search API (#4518)
<!-- oss-harness:v2 -->
Fixes #4246.

Focused Python fix for: [BUG] DirectMem0Adapter.search_memories calls
removed mem0 search API

Changed implementation: headroom/memory/backends/direct_mem0.py.
Regression tests: tests/test_memory/test_direct_mem0.py.

Verification (working directory `.`): `.venv/bin/python -m pytest -q -x
-p no:cacheprovider -- /workspace/tests/test_memory/test_direct_mem0.py`
- Patched code: 19 tests, zero failures/errors.
- Original code with the same regression tests: 1 failures, zero
collection/setup errors.
- Base commit: `f10725e6b9940d8bc2fcde950c86f8ac97543cb6`.

AI-assisted contribution: OpenCode using Autoloops wrote this patch. The
automated harness independently ran the tests listed above. No human
review is claimed.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4518"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: nandanadileep <116407139+nandanadileep@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Nandana Dileep <nandanadileep@users.noreply.github.com>
2026-11-04 08:33:51 -06:00
Shubham SrivastavaandJerrettDavis a26faaeee3 fix(sdk): carry Vercel AI SDK reasoning parts and provider options through compression (#4564)
## Description

`headroomMiddleware` / `withHeadroom` for the Vercel AI SDK convert the
prompt to OpenAI chat shape before `/v1/compress` and back afterwards.
The converters only model `text`, `file`/`image`, `tool-call` and
`tool-result` parts, so everything else the AI SDK puts on a prompt is
dropped: **`reasoning` parts**, and **`providerOptions`** on every
message and part (`providerExecuted` on tool calls too).

That is the state the providers need back on the next step of a tool
loop:

- `@ai-sdk/anthropic` rebuilds `thinking` / `redacted_thinking` blocks
from `reasoning` parts' `providerOptions.anthropic.signature` /
`redactedData`; without them the request reaches Anthropic with
`tool_use` and no thinking block, which its docs say "triggers a 400
error". Message- and part-level `anthropic.cacheControl` breakpoints are
read from `providerOptions` as well.
- `@ai-sdk/google` carries Gemini thought signatures as
`providerOptions.google.thoughtSignature` on text, tool-call, file and
reasoning parts. Its current release injects a sentinel for Gemini 3 "to
keep the request from failing with HTTP 400" and names the cause:
"application code that drops `providerOptions.google.thoughtSignature`
when persisting or serializing assistant tool-call messages" — which is
what this middleware did.
- `@ai-sdk/openai` keeps Responses `itemId` /
`reasoningEncryptedContent` the same way.

Captured on `main` (`a2d4fe72`) by running a LanguageModelV3 step-2
prompt through the public converters, which is the middleware's exact
path — every message changed:

```text
[0] system   : providerOptions (anthropic.cacheControl) dropped
[1] user     : text part providerOptions (anthropic.cacheControl) dropped
[2] assistant: both reasoning parts dropped (signature + redactedData); text and tool-call
               providerOptions (google.thoughtSignature, openai.itemId) dropped
[3] tool     : tool-result providerOptions (anthropic.cacheControl) dropped
```

The proxy does not need any of this removed. `/v1/compress` copies
messages with their extra keys, keeps extra keys on text parts even when
it rewrites the text, and passes unknown content parts through (verified
live, below). So the converters now carry the AI SDK state as-is and the
reverse conversion hands it back verbatim.

Closes # (none — follow-up to #4489; same converter sweep)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **`sdk/typescript/src/utils/format.ts`**
- `vercelToOpenAI` / `openAIToVercel`: `providerOptions` on a
system/user/assistant message stays on the OpenAI message under the same
key; on text parts it stays on the part; on `image`/`file` parts it
rides inside the existing `image_url` carrier object. Assistant parts
the converter does not model — `reasoning`, `reasoning-file`, `custom`,
provider-executed `tool-result` — keep their place in the OpenAI
`content[]` verbatim (text-only turns still collapse to a string unless
a text part carries options; same for user turns). `tool-call` →
`tool_calls[i]` plus `providerOptions` / `providerExecuted`, and
`position` (how many content parts preceded the call), so
`openAIToVercel` puts each call back in its place among reasoning / text
/ provider-executed parts instead of appending it; a turn whose text
follows a tool call keeps the content array for the same reason. One
OpenAI tool message is one `tool-result` part, so the part's options
take `providerOptions` and the Vercel message's go under
`messageProviderOptions` on the last split message only (a copy on each
would multiply an Anthropic cache breakpoint); a
`tool-approval-response` part (no OpenAI equivalent) is carried on
`part` and restored as-is. All of it restored on the way back.
- `detectFormat`: `reasoning` / `reasoning-file` / `custom` /
`tool-approval-*` parts are Vercel markers (Anthropic spells its block
`thinking`); `image_url` / `input_audio` / `refusal` / `file`-object
parts are OpenAI markers. A message-level `providerOptions` key is
Vercel evidence only once no message carries a structural marker,
because the public OpenAI message types allow the key too.
  - `toOpenAI(messages, format?)` accepts a declared format.
- **`sdk/typescript/src/types.ts`, `src/index.ts`** — `ProviderOptions`;
`providerOptions?` on `TextContentPart`, `ToolCall` (plus
`providerExecuted?`) and the four message types; `ReasoningContentPart`
and a literal-typed `PassthroughContentPart` join `ContentPart`, so
`part.type === "text"` narrowing stays exact; `CompressOptions.format`.
- **`sdk/typescript/src/compress.ts`** — `format` (declared input
format) skips autodetection and is passed to `toOpenAI`.
- **`sdk/typescript/src/adapters/vercel-ai.ts`** — the middleware and
`compressVercelMessages` call `compress(prompt, { format: "vercel" })`
and use `result.messages`, instead of converting themselves and then
handing OpenAI-shaped messages to `compress()`, which re-detects the
format. That re-detection is why `format` exists: reasoning parts are
not OpenAI-shaped, so a converted conversation whose first marker is a
reasoning part would have been read as Vercel input again and lost its
tool calls; and a text-only prompt is indistinguishable from OpenAI
messages, which the middleware could never declare before.
- **Tests** — `test/utils/format.test.ts`: the original "skips reasoning
parts" expectation becomes a carry-verbatim one; a step-2 prompt
(system/user/assistant/tool with options everywhere, two reasoning
parts, tool-call options) round-trips exactly; tool-call
`providerOptions`/`providerExecuted`; provider-executed `tool-result`
and `tool-approval-response` parts verbatim; user text with options kept
as a part; text-only assistant turns still a string; file-part options
through the media carrier. `test/adapters/vercel-ai.test.ts`: a typed
`LanguageModelV3Prompt` through `transformParams` with the proxy
rewriting only the tool message. `test/compress.test.ts`: `format:
"vercel"` on a text-only prompt. `test/utils/detect-format.test.ts`: the
new markers, and OpenAI `image_url` / `file` / `input_audio` parts
outranking `providerOptions` (the review repro included).
`test/types.test-d.ts`: the public type changes. Review round 1 added:
`compress()` with no `format` keeping an OpenAI user message with
`providerOptions`, `image_url` and `file`; round trips for `[tool-call,
reasoning, tool-call]`, a provider-executed search turn and `[tool-call,
text]`; the `text, tool-call` wire shape unchanged; message-level
options once per split tool message; the search turn through
`transformParams`.
- **Docs** — `docs/content/docs/vercel-ai-sdk.mdx` (what is carried),
`docs/content/docs/api-reference.mdx` and `sdk/typescript/README.md`
(the `format` option).

**Not in this PR.** Tool-result `output` kinds: every output is still
flattened to a string and restored as `json`/`text`, so `error-text` /
`error-json` lose their error-ness and `content` outputs with media
become JSON text. Same converter, different defect — next PR. The
Gemini-native converter's `thought` parts and `thoughtSignature` are
also a separate PR.

## Testing

- [x] Unit tests pass (`npm test` — vitest, incl. typecheck)
- [x] Type checking passes (`npm run typecheck` — tsc --noEmit)
- [x] Build passes (`npm run build` — tsup)
- [x] New tests added for new functionality
- [x] Manual testing performed (below)

### Test Output

```text
$ cd sdk/typescript && npm test
 Test Files  19 passed | 5 skipped (24)
      Tests  435 passed | 33 skipped (468)
Type Errors  no errors

$ npm run typecheck      # tsc --noEmit — clean
$ npm run build          # tsup — ESM, CJS and DTS built
```

## Real Behavior Proof

- Environment: published proxy `headroom-ai 0.42.0` (`uvx --from
'headroom-ai[proxy]' headroom proxy --port 8799`), this branch built
with `npm run build`, Node v22.16.0, macOS.
- Exact command / steps: a LanguageModelV3 step-2 prompt — system and
user with `anthropic.cacheControl`, assistant with two `reasoning` parts
(signature, redactedData), a text part and a tool call carrying
`google.thoughtSignature` / `openai.itemId`, message-level options, and
a `tool-result` carrying a 72 KB JSON log plus `cacheControl` — passed
through `headroomMiddleware().transformParams` against the live proxy;
the returned prompt compared message by message with the input.
- Observed result: every message other than the tool result came back
byte-identical, the tool result kept its options and shrank from 72 432
to 34 916 chars. Full output:

```text
system byte-identical (cacheControl kept): true
user byte-identical (text providerOptions kept): true
assistant byte-identical (2 reasoning parts, text + tool-call providerOptions, message providerOptions): true
tool-result output: 72432 -> 34916 chars, type=text, toolName=fetch_gateway_metrics, providerOptions kept: true
tool-result output head: "[400]{id:int,level:string,msg:string,path:string,service:string,status:int,ts:string}\n0,WA"
```

Review round 1, same proxy: an interleaved provider-executed turn
(`reasoning, tool-call, tool-result, reasoning, text, tool-call`)
through `headroomMiddleware`, then an OpenAI user message with
`providerOptions` + `image_url` + `file` (and a 56 KB tool result)
through `compress()` with no `format`:

```text
assistant byte-identical (order back: reasoning, tool-call, tool-result, reasoning, text, tool-call): true
tool-result output: 56002 -> 31287 chars, type=text, toolName=fetch_gateway_metrics; message providerOptions kept: true
detectFormat: openai
user message byte-identical (image_url and file kept): true
tool message: 56002 -> 31690 chars; compressed=true
```

The same payload through the converters on `main` loses every item
listed in the Description (captured before the change with the same
script).
- Not tested: an end-to-end multi-step `generateText` against Anthropic
/ Gemini with thinking enabled (no provider keys in this environment);
the provider behaviour is taken from the `@ai-sdk/anthropic` and
`@ai-sdk/google` sources and Anthropic's docs quoted above.

## Runtime Rollout Safety

- Rollout-managed feature(s): none (TypeScript SDK only, no proxy
change)
- Minimum rollout channel: n/a
- Stable/default behavior changed: for SDK users, OpenAI-shaped messages
sent to `/v1/compress` now carry the AI SDK's `providerOptions` keys and
reasoning parts, which the proxy already passes through; the public
`ContentPart` union grows (additive).
- Kill switch / disable path: unchanged — `fallback: true` returns the
original prompt when the proxy is unreachable; unwrap the model to
disable compression.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- The carried keys never reach an OpenAI endpoint through the SDK's own
paths: the universal `compress()` returns messages in the input format,
and the OpenAI adapter never sees Vercel input.
- Enumerated before writing: the V3 and V4 prompt unions
(`@ai-sdk/provider` 4.0.26) and the `ModelMessage` unions
(`@ai-sdk/provider-utils` 5.0.58); every assistant and tool part type in
them is either modelled or carried verbatim.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4564"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-03 23:41:35 -06:00
6cc0801219 docs: fix generate_report example arguments (#4435)
## Description

The `generate_report()` example in `docs/content/docs/api-reference.mdx`
and `wiki/api.md` passes `format="html"` and `period="day"`, and says it
produces "HTML/Markdown" reports. Neither keyword exists. Running the
documented snippet verbatim on 0.41.x:

```
TypeError: generate_report() got an unexpected keyword argument 'format'
```

Actual signature (`headroom/reporting/generator.py`):
`generate_report(store_url, output_path="report.html", start_time=None,
end_time=None) -> str`. It writes HTML only and returns the output file
path.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Replace the example with the real arguments (`output_path`,
`start_time`), say it returns the path, drop "Markdown" from the
description. Both files, docs only.
- `store_url` uses the `sqlite:////absolute/path/...` form that
`configuration.mdx` already documents: `sqlite:///headroom.db` resolves
to `/headroom.db` at the filesystem root, so the old example also fails
with `OperationalError: unable to open database file` on a normal
machine.
- `start_time` is timezone-aware (`datetime.now(timezone.utc)`), since
`format_timestamp` treats a naive datetime as already UTC (fixed in
0338644 after review).

## Testing

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

No regression test: the change is documentation text only, no code under
`headroom/` is modified, and there is no docs-snippet test harness in
the repo to extend.

### Test Output

```text
naive (old doc) cutoff: 2026-10-22T10:00:00Z
aware (new doc) cutoff: 2026-10-22T15:00:00Z
```

(`format_timestamp` from this branch, simulating a UTC-5 host at
2026-10-23 10:00 local = 15:00Z.)

## Real Behavior Proof

- Environment: headroom 0.41.x, macOS, Python 3.12.
- Exact command / steps: run the old snippet;
`inspect.signature(headroom.generate_report)`; run the new snippet with
an absolute `sqlite:////...` path.
- Observed result: old snippet raises `TypeError`; signature matches the
one above; new snippet writes `report.html` and returns its path.
- Not tested: Windows; the date filter was checked through
`format_timestamp` only, not an end-to-end report against a seeded
store.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a (docs only)
- Stable/default behavior changed: no
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

---------

Signed-off-by: Anton Dziatkovskii <dzyatkovskiy.a@gmail.com>
Co-authored-by: Anton Dziatkovskii <dzyatkovskiy.a@gmail.com>
Co-authored-by: tonydzi <194927794+tonydzi@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-03 22:51:09 -06:00
Mansurshakh JaparovandJerrettDavis 93279f3208 fix(smart-crusher): preserve unchanged JSON bytes (#4044)
## Description

Closes #3697.

An unchanged SmartCrusher result now keeps the original bytes and
reports no structural compression. Formatting-only savings come from the
existing Rust JsonMinifier, not from pretending the JSON value changed.

The minifier preserves duplicate keys, order, escapes and number
literals. A mixed result cannot use one section's CCR marker to hide
unmarked loss in another.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update

## Changes Made

- Keep SmartCrusher's exact-byte no-op behavior; route actual whitespace
savings through JsonMinifier in Python and the eligible native live
zone.
- Keep all records and JSON literals; do not store originals or inject
CCR markers for lossless minification.
- Preserve mixed routing and reject uncovered lossy sections in both
tool-result formats. Explicit no-CCR behavior is unchanged.
- Replace incidental forwarding tests with native minification and
complete-record regressions; document the distinction.

## Testing

- [ ] Entire Python suite — not claimed.
- [x] Four consumer/native modules: `python -m pytest -q
tests/test_recursive_json.py tests/test_transforms_content_router.py
tests/test_transforms/test_smart_crusher_rust_parity.py
tests/test_transforms/test_smart_crusher_lossless_default.py`
- [x] Changed-file Ruff checks; normal commit hooks, including mypy and
commitlint.
- [x] Manual public-API runs, with and without the HTML extra.

### Test Output

Consumer/native modules:

```text
collected 136 items
============================= 136 passed in 36.43s =============================
```

Normal pre-push (fmt, clippy, 1,668 Rust tests with 3 ignored, native
build, Python checks, commitlint):

```text
======================== 180 passed, 1 warning in 8.35s ========================
✅ ci-precheck PASSED — safe to push.
To https://github.com/oxura/headroom.git
   64688217..0002af3e  HEAD -> fix/3697-unchanged-json-passthrough
```

The first push stopped on an inherited commit footer because
`origin/main` was stale. Fetching the actual upstream ref fixed the lint
range; the normal push then passed. No source merge, lint-rule change or
hook bypass.

Hosted checks on `0002af3e` now pass: all four Python shards, Windows
permissions, smoke tests, native wrappers, parity and Docker e2e.
[Checks](https://github.com/headroomlabs-ai/headroom/pull/4044/checks).

## Real Behavior Proof

- Environment: Linux x86-64, Python 3.13.14, Rust 1.95 and the actual
extension built from this checkout. No provider billing calls.
- Exact command / steps: the earlier localhost `/v1/compress` and
TransformPipeline runs exercised the original fix. For this follow-up,
call `ContentRouter.compress` on a data-only HTML page containing 60
complete records, with `smart_crusher_lossless_only=True`,
`smart_crusher_with_compaction=False`, and `enable_kompress=False`. Run
it once without trafilatura, then with the locked 2.0.0 extra. The new
public-API test contains the full input/config. Separately call
`route_embedded_json` on a compact array followed by a padded array,
with Unicode and CRLF surroundings.
- Observed result: both HTML modes keep all 60 records and the
surrounding markup, with no unrecoverable section. The unchanged first
JSON span does not stop the second span's compaction; surrounding text
stays exact. Final Codecov is 96%; the two remaining lines are the
deferred HTML fallback return. The real no-HTML-extra API run executed
both, and local coverage also exercises the unchanged-span branch.
- Not tested: live provider acceptance, invoices or production cache hit
rates. No native eligibility or admission gates changed.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; existing JSON routing.
- Minimum rollout channel: unchanged.
- Stable/default behavior changed: eligible formatting savings return as
json_minifier; standalone SmartCrusher remains byte-exact on no-ops.
- Kill switch / disable path: existing routing configuration and
HEADROOM_LIVE_ZONE_DISABLE_ARMS.
- Unsafe override required: no.
- Qualification impact: no-op, formatting, structural and CCR paths
exercised separately.
- Rollback path: revert the minifier integration; no persistence
migration.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have made corresponding changes to the documentation
- [x] I did not edit CHANGELOG.md — release-please owns it

## Additional Notes

Addresses the [minifier
review](https://github.com/headroomlabs-ai/headroom/pull/4044#issuecomment-6157583285).
The maintainer’s main merge is retained. The latest follow-up only
changes consumer tests.

AI-assisted; no subagents.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-03 21:32:42 -06:00
fe69f38116 fix(privacy): turn off Mem0's PostHog telemetry by default (#4619)
## Description

Turns off Mem0's PostHog usage telemetry by default for Headroom's
memory-stack backends.

Mem0 sends `mem0.init`, `mem0.add`, `mem0.search` and
`mem0.notice_displayed` events to `https://us.i.posthog.com` unless
`MEM0_TELEMETRY` is false. The events carry config, OS and version
fields plus hashed entity ids, but no memory text or queries. It is on
by default in every Mem0 release we lock (2.0.5 and 2.2.1). Found while
collecting runtime evidence for #4614.

Same approach as #4602 (ONNX Runtime): `setdefault`, so an explicit
caller value wins.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/memory/backends/mem0.py`:
`disable_mem0_telemetry_by_default()` sets `MEM0_TELEMETRY=False` unless
already set. `Mem0Backend._ensure_client` calls it before `from mem0
import Memory`.
- `headroom/memory/backends/direct_mem0.py`: `DirectMem0Adapter` calls
it before its `from mem0 import Memory`. These are the only two mem0
imports in `headroom/`.
- `docs/content/docs/installation.mdx`: the `memory-stack` row says
telemetry is off and how to allow it.

Mem0 reads the variable once, when `mem0.memory.telemetry` is imported.
A host process that imported mem0 before Headroom must set it itself;
the docstring says so.

## Testing

- [x] Unit tests pass (`pytest`):
`tests/test_memory/test_mem0_telemetry.py
tests/test_memory/test_direct_mem0.py
tests/test_memory/test_mem0_system_adapter_deprecation.py` → 28 passed
- [x] Linting passes (`ruff check`, `ruff format --check` on
`headroom/memory tests/test_memory`)
- [x] Type checking: `mypy headroom/memory/backends/mem0.py` reports 9
errors, identical on `main` (`direct_mem0.py:203`, `embedders.py:389`)
- [x] New tests added: backend sets the default before importing mem0,
caller value wins, real mem0 reads the default as disabled
- [x] Manual testing performed

### Test Output

```text
28 passed in 1.62s
# on main (new test file only):
FAILED test_mem0_telemetry.py::test_backend_turns_mem0_telemetry_off_before_importing_mem0
FAILED test_mem0_telemetry.py::test_real_mem0_reads_the_default_as_disabled
2 failed, 1 passed
```

## Real Behavior Proof

- Environment: macOS arm64, Python 3.12.13, mem0ai 2.2.1, posthog
7.81.2, Qdrant 1.15.5 in Docker, OpenAI replaced by a local stub.
- Steps: patched `posthog.Posthog.capture` to record instead of send,
then ran a `Mem0Backend` save and search.
- Observed before: `mem0.init`, `mem0.add`, `mem0.search`,
`mem0.notice_displayed` captured, with properties like `client_version`,
`collection`, `embedding_model`, `encoded_ids`, `os`, `python_version`,
`vector_store`.
- Observed after: `test_real_mem0_reads_the_default_as_disabled` imports
the real `mem0.memory.telemetry` in a clean subprocess and reads
`MEM0_TELEMETRY == False`. Mem0's capture path returns early when it is
false.
- Not tested: Mem0 cloud mode (`Mem0Config(mode="cloud")`), which talks
to Mem0's hosted API regardless.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: yes, Mem0 telemetry is no longer sent
by default. Memory behavior is unchanged.
- Kill switch / disable path: `MEM0_TELEMETRY=True` restores Mem0's
default
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] I did **not** edit `CHANGELOG.md`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4619"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-03 18:25:49 -06:00
Kenneth WongandKenneth Wong 31f1304026 fix(privacy): turn off ONNX Runtime's Microsoft telemetry by default (#4602)
## Description

Since 1.31.0, ONNX Runtime's Linux and macOS builds include Microsoft's
1DS telemetry client
(`onnxruntime/core/platform/posix/telemetry_1ds.cc`, absent through
1.30.0). It is **on by default**. `OneDsTelemetry::Initialize` turns it
off only when a CI variable (`GITHUB_ACTIONS`, `TF_BUILD`, ...),
`ORT_RUNNING_UNIT_TESTS` or `ORT_DISABLE_TELEMETRY` is set. Otherwise
ORT:

- writes a device id under `~/Library/Application
Support/Microsoft/DeveloperTools/.onnxruntime/` on macOS, and sends a
SHA-256 of it with each event;
- queues events in an `onnxruntime.db` next to it and uploads them to
`https://mobile.events.data.microsoft.com/OneCollector/1.0`;
- sends a `SessionCreation` event for each session, with
`modelFileName`, `modelGraphName`, `modelProducerName`,
`modelGraphHash`, `modelWeightHash`, execution provider ids and
versions, and hardware vendor and device types.

Headroom requires `onnxruntime>=1.24.0` on Python 3.11+ and currently
resolves 1.31.1. The proxy loads ORT twice: the Python package
(Kompress, image router) and the dylib the Rust core loads through
`ORT_DYLIB_PATH` (Magika, embeddings). Each copy initializes the
uploader. In my runs the proxy opened a TLS connection to Microsoft
about 11 s after start, while the models loaded, and sent about 4 KB.
`HEADROOM_OFFLINE` doesn't touch it, and
`docs/content/docs/security-model.mdx` doesn't list this destination. CI
doesn't see it because GitHub Actions sets `GITHUB_ACTIONS`, which ORT
treats as CI. The same client is in the manylinux wheels (checked in
`onnxruntime-1.31.1-cp311-cp311-manylinux_2_28_aarch64.whl`), so Docker
deployments are affected too.

ORT reads `ORT_DISABLE_TELEMETRY` once per copy, when that copy
initializes, and then never creates the uploader or the device id. So
`import headroom` now sets it with
`os.environ.setdefault("ORT_DISABLE_TELEMETRY", "1")`, in the same place
as the existing `ensure_ort_dylib_pinned()` hook, before anything in the
package can load ORT. An explicit caller value wins, including `0` for
anyone who wants ORT's telemetry on.

Design choice: this is off always, not only under `HEADROOM_OFFLINE` (as
`pin_crewai_telemetry_off` does for CrewAI). The events describe the
user's models and hardware and go to a third party the user never chose.
Headroom's own beacon is opt-out and documented, but this upload was
neither documented nor tied to any Headroom setting. If maintainers
prefer offline-only, `_OfflineEnvPin("ORT_DISABLE_TELEMETRY", "1")` in
`headroom/offline.py` is a small change to this PR.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/_ort.py`: `disable_ort_telemetry_by_default()` sets
`ORT_DISABLE_TELEMETRY=1` unless the variable is already set.
- `headroom/__init__.py`: calls it at package import, next to
`ensure_ort_dylib_pinned()`.
- `crates/headroom-core/src/transforms/magika_detector.rs`:
`init_ort_from_path` commits the environment with
`.with_telemetry(ort_telemetry_opted_in(ORT_DISABLE_TELEMETRY))`. The
`ort` crate's default is `true`, and it calls `EnableTelemetryEvents`.
With this change, telemetry is off when the variable is unset or set to
a value ORT treats as disabled, and on for any other explicit value, so
`ORT_DISABLE_TELEMETRY=0` re-enables it for Rust-loaded models too. This
only stops later events, after the uploader has started, so it is a
backup for Rust-only callers; the environment variable is the real fix.
A unit test covers each case.
- `tests/test_transforms/test_ort_dylib.py`: the default is set; a
caller's value (`0`) is kept; a subprocess `import headroom` leaves
`ORT_DISABLE_TELEMETRY=1` with `onnxruntime` not yet imported.
- `docs/content/docs/security-model.mdx`: a row for
`mobile.events.data.microsoft.com` in the network egress table.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest -q tests/test_transforms/test_ort_dylib.py -k importing          # without the headroom/__init__.py change
E   AssertionError: assert {'ort_loaded'...'value': None} == {'ort_loaded'... 'value': '1'}
1 failed, 12 deselected
$ pytest -q tests/test_transforms/test_ort_dylib.py tests/test_package_init_lazy.py tests/test_offline_egress_chokepoint.py \
    tests/test_transforms/test_detect_fallback_1123.py tests/test_router_external_dispatch.py
223 passed, 2 skipped in 57.30s
$ ruff check / ruff format --check on the changed .py files (ruff 0.16.8)
All checks passed!
$ mypy headroom/_ort.py
headroom/memory/adapters/embedders.py:389: error: Unsupported operand types for - ("None" and "int")   # same single error on main
$ cargo clippy --all-features --tests -D warnings / cargo fmt --check -p headroom-core
clean
$ cargo test -p headroom-core --all-features --lib ort_telemetry
1 passed
```

## Real Behavior Proof

- Environment: macOS 27 (arm64), Python 3.11.17, onnxruntime 1.31.1
(CoreML, Azure and CPU providers), no CI variables set; `origin/main` at
`fc1ba891` and this branch, each in its own worktree with the same
`headroom-ai[all]` environment.
- Exact command / steps: (1) a bare script that creates one CPU
`InferenceSession` for `chopratejas/technique-router-onnx` and idles 60
s, with `ORT_DISABLE_TELEMETRY` unset, then set to `1`. (2)
`PYTHONPATH=<worktree> python -m headroom.cli proxy --port 8799
--no-telemetry` with `HEADROOM_UPDATE_CHECK=off` and
`ORT_DISABLE_TELEMETRY` unset, running until Kompress reports healthy.
Both record every TCP flow of the process with `nettop -m tcp -p <pid>
-L 0 -s 1`, and remote owners come from `whois`.
- Observed result: (1) unset: one TLS flow to a Microsoft Corporation
address (4.150.223.107:443, about 9.5 KB in and 4 KB out), with
`onnxruntime.db` touched; set to `1`: no TCP flows, and the database is
untouched. (2) `main`: 2 of 2 runs open a flow to Microsoft
(74.151.238.96, 40.74.98.197) about 11 s after start, with about 9.5 KB
in and 4 KB out; the collector rotates across Azure addresses. This
branch: no flow to Microsoft in 6 runs; only the Anthropic upstream,
plus litellm's GitHub pricing fetch in some runs.
- Not tested: Linux and Windows at runtime. The Linux wheel contains the
same client (`OneDsTelemetry` and the collector URL appear in
`libonnxruntime.so.1.31.1`), but I only measured macOS. Windows uses
ORT's separate ETW provider, which doesn't read this variable.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a, process environment default
- Stable/default behavior changed: yes; ONNX Runtime no longer uploads
usage telemetry to Microsoft from Headroom processes unless
`ORT_DISABLE_TELEMETRY` is set to a false value
- Kill switch / disable path: `ORT_DISABLE_TELEMETRY=0` restores ORT's
own default
- Unsafe override required: no
- Qualification impact: none; inference paths, providers and models are
unchanged
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

A host that imports `onnxruntime` before `headroom` has already
initialized ORT, so it has to set the variable itself. This is the same
limitation `pin_litellm_cost_map_local` documents.

In one early run, the fixed proxy did open the Microsoft flow. I
couldn't reproduce it in six later runs, and its log reports a different
computed `__version__` (`0.43.0-dev` against `0.42.1-dev` for the same
tree), so the import state of that run is uncertain. Setting the
variable before the process starts was clean in every run, for both the
bare script and the proxy.

---------

Co-authored-by: Kenneth Wong <17061293+KennethWKZ@users.noreply.github.com>
2026-11-03 17:01:04 -06:00
0ae116d46c docs(site): Python-first quickstart and a FastAPI page built on compress() (#4529)
## Description

Both named sandbox runs (Codex, Claude Code: "install headroom-ai and
add POST /chat that compresses messages") built exactly this endpoint
from the installed source, because no docs page described it. The
unbranded run and the web query "fastapi reduce openai token cost
middleware" found nothing of ours. The quickstart led with TypeScript,
which needs the proxy anyway, and installed `[all]`, which pulls torch,
the weight the chat engines objected to when they chose to hand-roll a
trimmer instead.

Part of the agent-readiness work: README (#4530), llms.txt (#4531),
AGENTS.md (#4532), skill (#4533), PyPI README (#4534) link to
`/docs/fastapi`, so this PR should merge first or alongside them.

## Type of Change

- [x] Documentation update

## Changes Made

- `docs/content/docs/fastapi.mdx` (new, first under Integrations): `POST
/chat` on `compress()`, an offline test request with expected output,
the one-line forward to OpenAI/Anthropic, what is kept, and a note that
`CompressionMiddleware` is for proxy-shaped apps, not apps that call a
provider.
- `quickstart.mdx`: Python tab first in all four Tabs blocks; base
install with an `[all]` note; a "no provider key needed" callout with
the zero-network one-liner.
- `index.mdx`: description carries the task phrases (reduce input
tokens, token cost, context window) so the generated `/llms.txt` does
too.

## Testing

- [x] Manual testing performed
- [x] `npm run types:check` in `docs/` (fumadocs-mdx + next typegen +
tsc): exit 0

### Test Output

```text
types:check (docs/): ✓ Types generated successfully, exit 0
Page test request, published 0.42.0 wheel, 3 runs: 20031 -> 95 FATAL kept: True ['router:protected:user_message', 'router:log:0.00']
```

## Real Behavior Proof

- Environment: macOS arm64, Python 3.12 venv with the **published**
`headroom-ai==0.42.0` wheel plus `fastapi`, `uvicorn[standard]`; run
from outside the checkout (inside the checkout, the source `headroom/`
dir shadows the wheel and has no compiled `_core`).
- Exact command / steps: `HEADROOM_OFFLINE=1 uvicorn main:app --port
8767` with the page's `main.py`, then the page's test request, three
times.
- Observed result: as pasted below.
  ```text
20031 -> 95 FATAL kept: True ['router:protected:user_message',
'router:log:0.00']
  ```
- Not tested: the `/answer` snippet against a live provider (needs a
key); rendering on the deployed Vercel site (Vercel preview will show
it).

## Runtime Rollout Safety

- Rollout-managed feature(s): none; documentation and metadata only, no
runtime code path changes
- Minimum rollout channel: n/a
- Stable/default behavior changed: no
- Kill switch / disable path: n/a (revert the PR)
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert the commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Additional Notes

The example file was dropped from this PR on request: as a
network-facing module under `examples/` it became a CodeQL taint source
and surfaced seven library sinks as new alerts. The page carries the
full code.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-29 20:26:56 -05:00
Shubham SrivastavaandJD Davis cd5e2b9e73 fix(sdk): send Anthropic messages to /v1/compress natively so thinking blocks survive (#4489)
## Description

The TypeScript SDK's Anthropic adapter converts every request to OpenAI
chat shape before `/v1/compress` and back afterwards. The converters
only model `text`, `tool_use`, `tool_result`, `image` and `document`, so
an assistant turn's **`thinking` and `redacted_thinking` blocks are
dropped**, `cache_control` markers on text blocks are lost, server-tool
blocks vanish, and a `tool_result` whose content is a block array is
flattened to text (an image inside it becomes a JSON string).
`compress()` does the same for Anthropic-format input.

Anthropic requires the thinking blocks back during a tool-use turn. From
the Thinking docs: "**Required:** within a tool-use turn, pass thinking
blocks back" and "Within the latest assistant message, the sequence of
consecutive `thinking` blocks must match what the model generated in the
original request: you can't rearrange, edit, or partially drop them.
This includes `redacted_thinking` blocks." The tool-workflows page:
"rebuilding the message or filtering out `redacted_thinking` blocks
triggers a 400 error." So a thinking-enabled tool-use turn through
`withHeadroom(anthropic)` reaches Anthropic with its `tool_use` block
but no thinking block.

Verified on `main` through the public converters (the adapter's exact
path):

```text
assistant blocks in : thinking, redacted_thinking, text, tool_use
assistant blocks out: text, tool_use
user cache_control kept: false
```

The conversion is unnecessary. Since #2743 (0.34.0) `/v1/compress` "does
no format conversion — callers send whichever wire shape they already
use"; the proxy docs list "content-block lists with `tool_use` /
`tool_result` / `thinking` blocks" among the natively compressed shapes
and say "an Anthropic-native caller does not need to convert to OpenAI
format first". The adapter page itself called the conversion "an SDK
choice, not an API requirement" while also calling it lossless. This PR
stops converting: the adapter and `compress()` send Anthropic messages
to the proxy as they are and forward what comes back.

Closes # (none — found while sweeping the converters after #4048)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **`sdk/typescript/src/adapters/anthropic.ts`** — no conversion:
`compress(params.messages, …)` and forward `result.messages` (or the
original messages on fallback, as before). The converter imports go
away.
- **`sdk/typescript/src/compress.ts`** — when the detected format is
`anthropic`, the messages go to the proxy unconverted and the proxy's
messages are returned unconverted; every other format still goes through
`toOpenAI` / `fromOpenAI`. `hooks.computeBiases` receives the messages
exactly as they are sent to the proxy.
- **`sdk/typescript/src/utils/format.ts`** — `detectFormat` treats
`thinking` / `redacted_thinking` blocks as Anthropic markers (a
thinking-only turn used to fall through to the OpenAI default).
`anthropicToOpenAI` / `openAIToAnthropic` are unchanged and still
exported.
- **Tests** — `test/adapters/anthropic.test.ts` rewritten for native
pass-through: a thinking-enabled tool-use turn (user text with
`cache_control`, assistant `thinking` + `redacted_thinking` + `text` +
`tool_use`, array `tool_result` with an image) is posted unchanged and
the proxy's returned messages are forwarded unchanged;
tool_use/tool_result and image turns likewise; fallback unchanged.
`test/compress.test.ts`: native round trip for Anthropic input, and the
#4048 document-only case now asserts the native document block.
`test/utils/detect-format.test.ts`: thinking / redacted_thinking
detection.
- **`docs/content/docs/anthropic-sdk.mdx`** — "How it works" describes
the native path; the conversion table, the "This conversion is lossless"
claim and the "SDK choice" callout are replaced by what is actually
preserved.

**Not in this PR.** `simulate()` still converts Anthropic input (a dry
run; nothing reaches Anthropic). The Vercel middleware drops `reasoning`
parts and `providerOptions` the same way, and the Gemini converter
merges `thought` parts into text; both need carriers because the proxy
cannot read those shapes, so they are separate PRs.

## Testing

- [x] Unit tests pass (`npm test` — vitest, incl. typecheck)
- [x] Type checking passes (`npm run typecheck` — tsc --noEmit)
- [x] Build passes (`npm run build` — tsup)
- [x] New tests added for new functionality
- [x] Manual testing performed (below)

### Test Output

```text
$ cd sdk/typescript && npm test
 Test Files  19 passed | 5 skipped (24)
      Tests  406 passed | 33 skipped (439)
Type Errors  no errors

$ npm run typecheck      # tsc --noEmit — clean
$ npm run build          # tsup — ESM, CJS and DTS built
```

## Real Behavior Proof

- Environment: published proxy `headroom-ai 0.41.0` (`uvx --from
'headroom-ai[proxy]' headroom proxy --port 8799`), this branch built
with `npm run build`, Node v22.16.0, macOS.
- Exact command / steps: one three-message conversation — a user text
block with `cache_control`, an assistant turn `[thinking,
redacted_thinking, text, tool_use]`, and a `tool_result` carrying a 72
KB JSON log — sent (a) through the public converters, which is what the
adapter did before, (b) through `withHeadroom()` wrapping a fake client
that captures the forwarded params, (c) through `compress()`.
- Observed result: before the change the conversion dropped both
thinking blocks and the `cache_control` marker; after it, the assistant
and user turns came back from the live proxy byte-identical while the
`tool_result` shrank from 72 432 to 34 916 chars (14 781 tokens saved).
Full output:

```text
== before: toOpenAI -> fromOpenAI (what the adapter used to do) ==
assistant blocks in : thinking, redacted_thinking, text, tool_use
assistant blocks out: text, tool_use
user cache_control kept: false

== after: withHeadroom(anthropic) against the live proxy ==
assistant turn byte-identical (thinking, redacted_thinking, text, tool_use): true
user turn byte-identical (cache_control kept): true
tool_result content: 72432 -> 34916 chars, still a tool_result block: true
compress(): tokensBefore=30471 tokensAfter=15690 tokensSaved=14781 transforms=["router:tool_result:mixed"]
compress() result thinking block byte-identical: true
```

The same tool result sent in OpenAI shape (`role: "tool"`) saves the
same 14 781 tokens (`router:mixed:0.39`), so the native path compresses
no less than the converted one did.
- Not tested: an end-to-end request to the Anthropic API with thinking
enabled (no key in this environment); the 400 is taken from Anthropic's
documentation quoted above.

## Runtime Rollout Safety

- Rollout-managed feature(s): none (TypeScript SDK only, no proxy
change)
- Minimum rollout channel: n/a
- Stable/default behavior changed: for SDK users, Anthropic messages now
reach `/v1/compress` in Anthropic shape; the proxy has compressed that
shape natively since 0.34.0.
- Kill switch / disable path: unchanged — `fallback: true` forwards the
original messages when the proxy is unreachable; unwrap the client to
disable compression.
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- Token counts for Claude models now come from the proxy's per-model
tokenizer registry instead of being counted on an OpenAI-shaped copy, so
`tokensBefore` / `tokensAfter` for the Anthropic adapter may differ
slightly from previous versions.
- Follow-ups, each its own PR: Vercel `reasoning` parts and
`providerOptions` (Anthropic signatures, Gemini `thoughtSignature`,
OpenAI `itemId`), and Gemini `thought` parts / `thoughtSignature`.

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-28 23:37:52 -05:00
b0d62f1d6d docs: clarify Target savings metric is % of new input tokens (#4485)
## Description

Closes #4444

The `Target savings` column in the savings-profile tables
(`docs/content/docs/proxy.mdx`) said `~50%` / `~70%` / `~90%` / `~60%`
without saying what the percentage is measured against. This PR spells
out per issue #4444 that the metric is **% of new input tokens**, and
adds one sentence to the `coding` rows noting that whole-request and
lifetime ratios are much lower in cache mode (where only the newest
delta is compressed).

## Type of Change

- [x] Documentation update (docs only; no code, config, or tests
changed)

## Changes Made

- `docs/content/docs/proxy.mdx`:
- Savings-profile quick reference table (~line 112): `Target savings`
cells now read `~N% of new input tokens`, and the `coding` row's Notes
column notes whole-request/lifetime ratios are much lower in cache mode.
- Profile detail table (~line 194): same wording for `coding` /
`agent-90` / `balanced` / `general`.
  - No other files touched (1 file, 2 table hunks).

## Testing

- [x] Linting/build validation via repository CI
- [ ] Unit tests pass (`pytest`) — n/a, docs-only change, no code paths
touched

### Test Output

```text
$ gh pr checks 4485
Validate Next.js build   pass   run 40264197799
template                 pass
```

## Real Behavior Proof

- Environment: Linux (CI ubuntu-latest), docs Next.js/Fumadocs app
- Exact steps: `Validate Next.js build` workflow (run 40264197799) fully
builds the docs site with `proxy.mdx` included, so the two revised
tables compile and render in the built page
- Observed result: build job passed on this head (`03039a6`); on-disk
text verified to read `~50% of new input tokens (emergent)` etc. in both
tables
- Untested scope: Vercel production deploy itself (Vercel's Git
integration runs after merge); wording rendering differences in other
MDX environments

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: none — documentation copy only
- Kill switch / disable path: n/a
- Unsafe override required: none
- Qualification impact: none
- Rollback path: revert the PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings (docs build + template check
green)
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — table text is visible in the diff.

## Additional Notes

Sent by an AI coding agent (OpenCode) on behalf of Yi-111-a.
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4485"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Yi-111-a <yi-111-a@example.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-28 23:35:15 -05:00
ShiYuan XiongandJerrettDavis 4ee616453f fix(router): keep small shell results verbatim (#3590) (#4426)
## Description

Fixes #3590. Small shell tool results (default: shorter than 2000
characters) now bypass lossy compression. Byte-lossless folds such as
search grouping may still apply. File reads and searches were already
protected; shell output is deliberately compressible, but a short `ls
-la`, `git status`, or JSON dump is not worth the accuracy loss
described in #3590.

The real proxy-level before/after evidence is in Real Behavior Proof. A
live Claude Code session is still untested.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added `HEADROOM_SHELL_RESULT_MIN_CHARS` (default `2000`) and
`_is_small_shell_result()`, applied to OpenAI `role:"tool"`, legacy
`role:"function"`, top-level text-block tool messages, and Anthropic
`tool_result` string/list-form paths after the lossless search fold and
before lossy compression.
- Normalized `ContentRouterConfig.bash_tool_names` once and extended it
with the shell aliases already in `headroom.learn._shared` (`shell`,
`local_shell`, `run_shell_command`, `execute_command`, `exec_command`,
`terminal`, `run_command`, `run_terminal_command`).
- Measure a top-level text-block shell result using the combined text
length, so a large result cannot bypass compression merely by splitting
into small blocks.
- Added boundary coverage for both tool/function block shapes, positive
threshold overrides and custom case-insensitive shell names; documented
the combined-length contract in `docs/content/docs/proxy.mdx`.
- Updated the existing proxy-level Bash regression to assert both sides
of the 2000-character floor, fixing the hosted shard-4 failure. Added
kill-switch, malformed/empty-value and diagnostic coverage.
- Carried dependency security remediation requested in review: certifi
and the OpenTelemetry family, three patched Rust crates, and patched npm
locks for docs, SDK, OpenCode and E2E tooling. These use existing
packages, explicit security floors and targeted dependency overrides;
libc platform constraints remain intact.

## Testing

- [x] Hosted CI passes on the current head, including all four Python
test shards
- [x] Changed Python tests passed Ruff in the preceding regression
revision; no Python runtime source changed in this dependency follow-up
- [ ] Full local Python type check passes (Windows/test-double baseline
limitations)
- [x] Regression and dependency-interface coverage added
- [x] Real proxy proof and isolated CLI smoke evidence recorded

### Test Output

Latest follow-up: Windows, Python 3.11 / Node 22.20.0 for the OpenCode
suite, and portable official Node 24.22.1 for E2E CLI qualification. The
portable Node archive's SHA-256 matched nodejs.org's published checksum.

```text
Python shell-floor/web-tool/router/alias/tracing + OpenClaw wrapper/bridge selection:
119 passed, 2 warnings in 10.37s

uv lock --check
Resolved 262 packages

uv tree --locked --python-version 3.10 --package filelock
filelock v4.1.0
uv tree --locked --python-version 3.12 --package filelock
filelock v4.2.4

FileLock 4.2.4 process smoke:
contention times out, release permits next process

cd plugins/opencode
npm test
Test Files  4 passed (4)
Tests       60 passed (60)
npm run typecheck
> node node_modules/@typescript/native/bin/tsc --noEmit
npm run build
DTS Build success
npm audit --package-lock-only --audit-level=low --registry=https://registry.npmjs.org
found 0 vulnerabilities

After merging current upstream main (including #4453):

cd e2e/wrap/tools
node node_modules/openclaw/openclaw.mjs --version
OpenClaw 2026.10.6
npm audit --package-lock-only --audit-level=low --registry=https://registry.npmjs.org
2 vulnerabilities (1 moderate, 1 high)
# limited to npm's bundled copies:
# GHSA-rxp7-6hxx-x8q5 — node_modules/npm/node_modules/ip-address 10.7.2
# GHSA-4qv8-2p9j-2685 and GHSA-xjj6-f33q-929c — node_modules/npm/node_modules/undici 6.29.0
# exact scoped exceptions expire 2026-11-15; no OpenClaw path is excepted

production-lock pip-audit:
No known vulnerabilities found
```

- The Python native extension remains the existing 0.41.0 Windows
wheel's `_core.pyd`; all Python source under test comes from this PR
checkout. No native rebuild or Torch/model inference qualification is
claimed.
- The 3 new OpenCode compatibility tests exercise the actual upstream
`tool.schema` interface and Effect `Primitive.fileParse` TOML
success/error channels, mocking only filesystem I/O. They accompany the
explicit Zod/TOML overrides.
- E2E CLI checks used isolated `OPENCLAW_STATE_DIR` /
`OPENCLAW_CONFIG_PATH`, with no model request or modification of the
user's normal OpenClaw configuration. E2E dependencies were installed
with lifecycle scripts disabled. The full Docker/model E2E suite was not
run locally.
- Earlier unchanged docs/SDK lock remediation passed: SDK **351 passed,
33 skipped** plus typecheck/build; docs typecheck/build with 185 static
pages; both npm audits were zero. The previous head also passed the
hosted Rust security audit.
- Full local Python suite was not rerun. The earlier remote-timeout
float-boundary failure remains outside this change.

## Real Behavior Proof

- Environment: Windows 10 x64, Python 3.11.0, Headroom source `main` at
`e995eb9` (0.41.1-dev), native `_core` from the 0.41.0 wheel because no
Rust toolchain is available locally. `HEADROOM_DISABLE_KOMPRESS=1` keeps
the strategy deterministic; the guard sits before strategy selection, so
the same path applies when Kompress is enabled.
- Exact command / steps: local harness (not committed) `python
proof-3590/run_proof.py` starts the source-tree proxy in `--mode token`
with `HEADROOM_MIN_TOKENS=0` and a local OpenAI-compatible upstream, and
sends a chat request containing a Bash `ps aux` tool result (1719 chars)
followed by later turns. The upstream records the transformed request
body.
- Observed result: `HEADROOM_SHELL_RESULT_MIN_CHARS=0` produced
`compressed=1`, `router:tabular:0.85`, 802 -> 697 tokens (saved 105),
and upstream received a 1455-char schema representation. Default `2000`
produced `shell_small_result: 1`, `compressed=0`, 802 -> 802 tokens, and
upstream received the original 1719-char output byte-for-byte.
- Review follow-up proof: `python proof-3590/run_block_review.py` ran
the real proxy against the same local upstream without replacing its
compressor. A 1719-character shell result split into two text blocks
stayed unchanged; a 3438-character result split into two 1719-character
blocks remained compressible. The previous PR revision incorrectly
protected both large-result fragments. This harness is local, not part
of the PR.
- Not tested: a live `headroom wrap claude` session; the ML Kompress
path; Linux/macOS; the full suite (one pre-existing baseline failure).

## Runtime Rollout Safety

- Rollout-managed feature(s): none; ContentRouter default path.
- Minimum rollout channel: default proxy.
- Stable/default behavior changed: yes. Shell results between
`min_chars_for_block_compression` (500) and
`HEADROOM_SHELL_RESULT_MIN_CHARS` (2000) now stay verbatim instead of
taking lossy compression. At ~1500 chars (~375 tokens), this keeps
roughly 325 extra tokens per result for the life of the context.
- Kill switch / disable path: `HEADROOM_SHELL_RESULT_MIN_CHARS=0`.
- Unsafe override required: no.
- Qualification impact: compression ratio may decrease for small shell
outputs; no API or correctness changes.
- Rollback path: revert this commit or set
`HEADROOM_SHELL_RESULT_MIN_CHARS=0`.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — no UI changes.

## Additional Notes

This follows the [dependency-remediation
review](https://github.com/headroomlabs-ai/headroom/pull/4426#pullrequestreview-5636487191).
It does not disable audits, ignore advisories, reduce audit severity, or
use `npm audit fix --force`.

- **Python:** certifi/OpenTelemetry/Rust fixes from the prior revision
remain. This follow-up raises setuptools to **84.1.0** for the Windows
wheel namespace traversal advisory and uses FileLock **4.2.4** on Python
≥3.11. The universal lock remains resolvable with the published Python
≥3.10 contract. **Python 3.10 now selects FileLock 4.1.0, but it remains
below the 4.2.1 security floor** because fixed releases require ≥3.11;
this is not presented as all-interpreter remediation.
- **OpenCode:** pin security overrides **Zod 4.6.7** and **TOML 5.0.1**
without downgrading the plugin or replacing its pinned Effect version.
TOML crosses a major version (4.3.0 → 5.0.1); its Node ≥20 floor and
consumed `parse(content)` API are unchanged. The added tests validate
the exercised schema and Effect parsing contracts, not every possible
TOML consumer. This override needs normal maintainer dependency review.
- **E2E:** the original remediation moved OpenClaw to 2026.10.5; the
upstream sync now brings **OpenClaw 2026.10.6** with **Undici 8.12.3**
and a Node floor of **≥24.22.1 <25 or ≥26.12.1**. The only remaining raw
npm findings are npm's published bundled copies, handled by the exact,
expiring E2E-only exception policy described below.
- Dependency packages and runtime floors were checked against the
upstream registry metadata. Locks use public registries and retain
existing libc platform constraints.

### Security qualification after upstream sync

Merge commit `e09d37c9274c812f26e994d01a623542a4a7ec31` brings this
branch to upstream `main` through `a616232`, including
[#4453](https://github.com/headroomlabs-ai/headroom/pull/4453). That
upstream change establishes the repository's current security policy for
the dependency cases that previously blocked this PR:

- **Production Python lock:** Python ≥3.11 resolves Pillow 13.0.0 and
FileLock 4.2.4. The current lock passes `uv lock --check`; an equivalent
export of the production `all` set passes `pip-audit 2.10.1 --no-deps
--disable-pip` with **no known vulnerabilities**.
- **Published-package resolution:** FastEmbed still caps Pillow below 13
when ordinary pip resolves package metadata. Upstream now disables image
decoding on Pillow below 13 and separately audits the pip-resolved set,
accepting only Pillow findings covered by that tested decoder guard.
This is a bounded runtime mitigation, not a claim that the older Pillow
release has no advisories.
- **E2E npm lock:** OpenClaw 2026.10.6 uses fixed Undici 8.12.3. Raw npm
audit now reports only npm 12.2.1's bundled Undici 6.29.0 and ip-address
10.7.2 copies. Because npm does not allow overrides to replace its
published bundled contents and no newer release is available, upstream
accepts only the three exact advisory/path/version pairs in
`e2e/wrap/tools/npm-audit-exceptions.json`, only for the non-shipping
E2E harness, and only until **2026-11-15**. The workflow fails on any
extra finding, version/path mismatch, stale exception or expiry.
- **Python 3.10:** the lock now selects FileLock 4.1.0, but it remains
below the 4.2.1 security floor because fixed releases require Python
≥3.11. This supported-interpreter limitation is documented by the
upstream policy and is not represented as universal remediation.

The earlier OpenCode standalone-wheel mismatch remains fixed by
`dee73be75cf81bb67a73573861f1b3ed4067a2bf`: typecheck, build, all 60
plugin tests and the standalone build passed locally, both generated
artifacts matched their Python-shipped paths byte-for-byte, and the
shipped bundle imported with the expected `headroom` id and both
`server`/`setup` entry points.

### Current merge status

- Current head `ab93d7917a6660c704069bb87b6c02f9c744cebc` is mergeable;
all router/documentation inline review threads are resolved.
- JerrettDavis approved this head on 2026-10-24, superseding the earlier
security-related change requests.
- All 12 pull-request workflows on this head completed successfully,
including
[CI](https://github.com/headroomlabs-ai/headroom/actions/runs/40122163824)
(all four Python test shards),
[Security](https://github.com/headroomlabs-ai/headroom/actions/runs/40122163777),
wheel builds/import smoke tests, OpenCode, docs, and Docker E2E checks.
The workflows are no longer awaiting approval.
- The PR remains open, awaiting maintainer merge.
- A live Claude Code/model session, the ML Kompress path, a native
rebuild and the full Docker/model E2E suite remain unverified locally;
the hosted results above are separate from the recorded local
qualification.

Implemented and verified with AI assistance.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-28 23:31:00 -05:00
18a588b3f0 fix(proxy): keep deferred tool loads out of the prefix-tracker lineage key (#4204)
## Description

The Anthropic handler keys a conversation's prefix-tracker lineage on
`segment_fingerprint({model, tools, tool_choice, thinking,
output_config})` (#2917, for #2671), with the whole `tools` array in it.
Claude Code appends every tool its ToolSearch loads to `tools`, still
marked `defer_loading: true`, and Anthropic keeps deferred tools out of
the prompt it caches. So the provider's cache key stays the same on a
load while Headroom's lineage key changes: the request after a load
starts a new lineage at frozen=0, the pipeline re-runs the whole
history, and the history comes out with different bytes (token mode
compresses Read results it had forwarded verbatim while they were
recent; cache mode forwards history raw that it had compressed). The
provider then re-writes the whole message tier. On one machine over 5
days, the request right after a deferred load had a large cache miss
21.9% of the time against 0.4% for every other request: 53 misses, 9.2M
tokens, 12% of all cache writes.

This PR leaves `defer_loading` tools out of the lineage key, the same
way Anthropic leaves them out of its cache key, but only when they reach
the provider still deferred. On a third-party Anthropic upstream the
handler's tool-search downgrade strips `defer_loading`, and a
translating backend sends every tool as a plain function; there a load
does change the provider's key, so the lineage key keeps counting them,
as on main. The decision reuses the downgrade's own predicate, now
computed once before the lineage key. Any tool the prompt renders still
starts a new lineage, as before.

### This is a trade

The reset was not pure waste: re-running the history at frozen=0 also
compressed old history that the freeze policy otherwise keeps verbatim.
Keeping the lineage gives that up. Both sides, measured 2026-10-13 on
one scripted 16-turn Claude Code session with two deferred loads, run
through a scratch proxy with the change on and off (same filter applied
downstream to the 0.39.0 wheel, not this branch):

| | main (lineage resets on a load) | this change |
|---|---|---|
| cache writes | 345k tokens | 121k tokens |
| cache reads | 1.36M tokens | 1.72M tokens |
| `tok_saved/tok_before` | 12.1% | 5.7% |
| net input cost | baseline | -43% (5-minute writes) to -50% (1-hour
writes) |

So for sessions that load deferred tools, tokens saved and the
list-priced `compression_savings_usd` go DOWN while the provider bill
goes down more. Anyone judging this change on `tok_saved/tok_before`
alone will read it as a regression; check `cache_read` per request (or
`cache_read/forwarded`) next to it.

The repro below shows the mechanics on one request (token mode,
tracker-estimated tokens): with the reset, turn 10 forwards 33,133
tokens, all of them written; with this change it forwards 55,456, of
which 55,441 are read and 15 written. At 0.1x for reads and 1.25x for
5-minute writes that is 41,416 against 5,563 base-input tokens (66,266
against 5,574 with 2x 1-hour writes). The reset's smaller history saves
2,232 base-input tokens on each later turn, so it pays back after 16
more turns (5-minute writes) or 27 (1-hour writes), and only if no other
deferred load resets it again first.

Closes #4203

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/cache/prefix_tracker.py`: new `cache_key_tools(tools)`
returns the tools without the entries whose `defer_loading` is set. A
non-list passes through unchanged and the input is never mutated. It
reads no environment: the handler decides when to apply it.
`segment_fingerprint` itself is unchanged; its docstring now says it
hashes what it is given and that callers pass only what the provider
keys its cache on.
- `headroom/proxy/runtime_env.py`: `HEADROOM_DEFERRED_TOOL_AFFINITY` is
a registered knob in `RUNTIME_ENV_KNOBS`, read per request through
`runtime_env.getenv` in the Anthropic handler, so a reusing `headroom
wrap` pushes it to the running proxy and `POST /admin/runtime-env` sets
it with no restart. Listed in the `security-model.mdx` allowlist.
- `headroom/proxy/handlers/anthropic.py`:
- The tool-search downgrade predicate, `_third_party_anthropic_upstream
= provider_name == "anthropic" and not
anthropic_first_party_tool_search_supported(upstream_base_url or
self.ANTHROPIC_API_URL)`, moved from the downgrade (main L3199-3202) to
just before the lineage key, and the downgrade reuses it. Neither input
changes between the two points.
- `affinity_tools` goes through `cache_key_tools` only when that
predicate is false and `self.anthropic_backend` is None, the backend
condition Headroom's own tool-search injector already uses (main L3239).
Otherwise every tool counts, as on main.
- The forwarded `tools` are untouched; the change is affinity metadata
only. `segment_fingerprint` has no other caller.
- A tool whose `defer_loading` is cleared (the client un-defers it) is
rendered into the prompt, so it counts again and starts a new lineage,
as Anthropic's key does.
- Tests: `tests/test_anthropic_deferred_tool_lineage.py` drives the real
handler (`create_app` + `TestClient`; the upstream is replaced by a
model of Anthropic's prompt cache over the forwarded bytes, deferred
tools unkeyed) through 9 turns of Read results and a 10th that adds a
tool. The requests are in the shape Claude Code 2.1.296 sends while it
defers tools: its client-side `ToolSearch` tool, a
`DeferredToolPlaceholder` that is deferred on every request, and on the
load turn ToolSearch's `tool_result` with a `tool_reference` plus the
loaded tool appended to `tools`, still deferred.
- `test_deferred_tool_load_keeps_the_lineage_and_its_cached_prefix`:
same tracker, frozen count kept (19), forwarded history byte-identical,
the provider reads the whole previous turn back, the loaded tool and its
`tool_reference` are forwarded intact, and no
`tool_search_deferred_orphan` warning is logged.
-
`test_deferred_tool_load_sent_eagerly_starts_a_new_lineage[third-party-upstream]`
(`anthropic_api_url` set to a gateway) and `[translating-backend]` (an
`anthropic_backend` that sends tools as plain functions): the forwarded
tools carry no `defer_loading`, the load gets a new tracker at frozen=0,
and the provider reads 0 tokens back either way.
- `test_kill_switch_keys_deferred_tools_again`: with
`HEADROOM_DEFERRED_TOOL_AFFINITY=0` the deferred load resets the lineage
as on main, and the provider reads 0 tokens back (the cost this fixes,
so the cache model is not vacuous).
- `test_eager_tool_still_starts_a_new_lineage`: a non-deferred tool
still gets a new tracker at frozen=0.
- `test_cache_key_tools_drops_only_deferred_tools`: `defer_loading:
false` is kept, the input is not mutated, `None` passes through, the
kill switch returns the list as given.
- The file points `HEADROOM_SAVINGS_EVENTS_PATH` at `tmp_path`, so its
proxied requests do not append to the developer's real savings ledger.
It also sets `HEADROOM_CLIENT_TOOL_SEARCH_NAMES=ToolSearch`: Headroom
does not recognize Claude Code's client-side `ToolSearch` by default, so
this request shape logs `tool_search_deferred_orphan` on main too (see
Additional Notes). Naming it is the documented remedy and changes
nothing on the lineage path.
- Commits: 426a45ec (the fix), af23a94e (Claude Code's tool-search shape
in the test), 2cca8abe (deferred tools still count where they reach the
provider eagerly).

### Interactions checked (line numbers on main f1bc2609)

- **Third-party upstreams (tool-search downgrade).**
`strip_first_party_tool_search_tools_for_third_party_upstream` (L3205)
removes `tool_search_tool_*` entries and strips `defer_loading` from
every tool when `_third_party_anthropic_upstream` holds (provider
`anthropic`, target base URL whose host is not `api.anthropic.com`, from
`x-headroom-base-url` or `--anthropic-api-url`). Those requests reach
the provider with every tool eager, so the lineage key now counts
deferred tools for them, exactly as on main. The predicate is the
downgrade's own variable, computed earlier, not a copy.
- **Translating backends.** Every non-None `anthropic_backend` (litellm,
any-llm) converts tools with `_convert_anthropic_tool`, which drops
`defer_loading`. Deferred tools count in the key there too, as on main.
- **Per-request route advice (not covered).** An extension can pick a
translating backend per request through `request.state.headroom_route`;
the handler resolves it at dispatch (L3666), well after the key, and
resolving it has side effects (it rewrites `body["model"]`). Such a
request with the default first-party backend keeps deferred tools out of
the key although the chosen backend sends them eagerly. Same outcome as
main's behaviour for a tool change on that path would need the advice
resolved before the key; left out.
- **Vertex.** Its routes pass a Vertex provider name, so the downgrade
does not run and the tools are forwarded with `defer_loading` as the
client sent them. They stay out of the key, which matches what is
forwarded.
- **Headroom's own tool-search deferral.** `inject_tool_search_deferral`
(L3254) runs after the fingerprint (L1663) on the same request, so the
fingerprint always sees the client's tools: tools Headroom defers still
count in the key, exactly as on main, and this change only drops tools
the CLIENT sent deferred. Once any client tool carries `defer_loading`,
`request_already_defers_tools` stands Headroom's injector down, so on
every load after the first one Headroom defers nothing on either side of
the load.
- Edge case, the first load: if Headroom's injector was deferring on the
turns before it (12+ tools and no deferral signal; checked with Claude
Code's 14 built-in tools, where it defers WebSearch, NotebookEdit,
ExitPlanMode and KillShell and adds `tool_search_tool_regex_20251119`),
the client's first deferred tool makes it stand down, so the forwarded
`tools` change and Anthropic's key changes with them. This change keeps
the lineage on that turn too. The provider re-writes the history once
either way; with this change it re-writes the same bytes instead of a
recompressed history, the same trade as above on one turn. Matching this
exactly would need the fingerprint taken after the injector, which runs
well after the tracker is needed.
- Not covered: a client that adds a tool which Headroom's injector then
defers still starts a new lineage although Anthropic's key may not
change. Whether Headroom defers depends on the whole array
(`_TOOL_SEARCH_MIN_TOOLS = 12`, the resident set); left for a follow-up
if it shows up in data.
- **#4056** (turn hooks ignore `defer_loading` when measuring a hook's
tool delta). Same view of the flag, no interaction: turn hooks run at
L3337, after the fingerprint. A hook that un-defers a tool
(headroom-tool-search's hot tools) changes the forwarded tools but never
reached the lineage key. `without_deferral_flags` is not reusable here:
it keeps the tool and drops the flag, this needs the tool dropped.
- **#4172** (tool-search cache-marker relocation mutates the client's
tools). No interaction with the lineage key: the mutation happens after
the fingerprint, each request parses a fresh body, and
`segment_fingerprint` drops `cache_control` anyway. `cache_key_tools`
builds a new list.
- **Other lineage keys.** The OpenAI chat and Gemini handlers call
`resolve_tracker` without `cache_affinity`, the Responses handler
resolves no lineage at all (it only computes a session id), and
`compute_session_id` does not hash tools, so none of them has this bug.
- **INPUT_RECEIVED extensions** can replace `tools` before the
fingerprint (L1197). A tool an extension marks `defer_loading` there is
now left out of the key too, which matches what is forwarded unless a
later stage changes it.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

Unit tests: the new file plus every test file that touches
`segment_fingerprint`, `cache_affinity`, `resolve_tracker`,
`defer_loading` or #2671, not the whole suite. Lint and types: `ruff
check`, `ruff format --check` and `mypy` on the changed files, not the
whole tree. Every run used a throwaway `HOME` (`HOME=$(mktemp -d)
HEADROOM_BEACON=off`), since `tests/conftest.py` clears `HEADROOM_*` and
does not redirect the savings ledger.

### Test Output

The new tests with only `headroom/proxy/handlers/anthropic.py` swapped
back, first to main f1bc2609, then to the first commit of this PR
(426a45ec, before the downgrade gate):

```text
$ git show upstream/main:headroom/proxy/handlers/anthropic.py > headroom/proxy/handlers/anthropic.py
$ python -m pytest tests/test_anthropic_deferred_tool_lineage.py -q -p no:cacheprovider
E   assert <headroom.cache.prefix_tracker.PrefixCacheTracker object at 0x...> is <headroom.cache.prefix_tracker.PrefixCacheTracker object at 0x...>
FAILED tests/test_anthropic_deferred_tool_lineage.py::test_deferred_tool_load_keeps_the_lineage_and_its_cached_prefix
========================= 1 failed, 5 passed in 3.94s ==========================

$ git show 426a45ec:headroom/proxy/handlers/anthropic.py > headroom/proxy/handlers/anthropic.py
$ python -m pytest tests/test_anthropic_deferred_tool_lineage.py -q -p no:cacheprovider
E   assert <headroom.cache.prefix_tracker.PrefixCacheTracker object at 0x...> is not <headroom.cache.prefix_tracker.PrefixCacheTracker object at 0x...>
FAILED tests/test_anthropic_deferred_tool_lineage.py::test_deferred_tool_load_sent_eagerly_starts_a_new_lineage[third-party-upstream]
FAILED tests/test_anthropic_deferred_tool_lineage.py::test_deferred_tool_load_sent_eagerly_starts_a_new_lineage[translating-backend]
========================= 2 failed, 4 passed in 3.76s ==========================
```

With the fix (2cca8abe):

```text
$ HOME=$(mktemp -d) HEADROOM_BEACON=off python -m pytest tests/test_anthropic_deferred_tool_lineage.py tests/test_anthropic_fork_lineage_cache.py tests/test_issue_2671_block_growth_cache.py tests/test_cache/test_prefix_tracker.py tests/test_proxy_anthropic_cache_stability.py tests/test_tool_search_client_deferral.py tests/test_issue_746_tool_search.py tests/test_hook_tool_deferral_accounting.py tests/test_tool_schema_savings_policy.py tests/test_openai_tool_search_deferral.py tests/test_proxy_openai.py tests/test_proxy_openai_cache_stability.py tests/test_cache_mode_cold_start.py tests/test_cold_start_fast_pass.py tests/test_anthropic_pre_upstream_backpressure.py tests/test_anthropic_stage_timings.py tests/gateway/test_native_deferral.py tests/gateway/test_kong_integration.py -q -p no:cacheprovider
======================= 459 passed, 5 skipped in 31.33s ========================

$ ruff check headroom/cache/prefix_tracker.py headroom/proxy/handlers/anthropic.py tests/test_anthropic_deferred_tool_lineage.py && ruff format --check <same 3 files>   # ruff 0.16.8
All checks passed!
3 files already formatted

$ mypy headroom/cache/prefix_tracker.py headroom/proxy/handlers/anthropic.py
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: macOS 27.0.1 arm64, Python 3.11.13, main f1bc2609 versus
this branch (2cca8abe) in the same venv. The main run was repeated on
the released 0.40.0 wheel (Python 3.12.13) with the same outcome (new
lineage, 0 read / 27,457 written on turn 10).
- Exact command / steps: the script below builds the real proxy with
`create_app(ProxyConfig(optimize=True, mode=...,
anthropic_api_url=...))` and posts 10 `/v1/messages` requests through
`TestClient`, in the shape Claude Code 2.1.296 sends while it defers
tools (client-side `ToolSearch`, a deferred `DeferredToolPlaceholder`):
9 turns of Read results, then the turn that carries ToolSearch's
`tool_reference` with the loaded MCP tool appended as `defer_loading:
true`. A control run appends a normal tool on a plain 10th turn instead.
Only `_retry_request` is replaced, by a model of Anthropic's prompt
cache that keys on the forwarded non-deferred tools, the system prompt
and the messages (cache markers ignored) and returns
`cache_read_input_tokens`/`cache_creation_input_tokens` so the tracker
freezes as it would live. A spy on `resolve_tracker` records which
tracker each request got and its frozen count. All state goes to a temp
dir, the beacon is off, nothing leaves the machine.
`PYTHONPATH=<checkout> LITELLM_LOCAL_MODEL_COST_MAP=True python
repro_deferred_tool_lineage.py`, again with `REPRO_MODE=cache
REPRO_TOOL=Bash`, and again with
`REPRO_UPSTREAM=https://gateway.example.com/anthropic` (third-party
upstream, never dialled).
- Observed result: first-party, main resets the lineage on the deferred
load in both modes and the provider reads nothing back; this branch
keeps the lineage, its frozen count and the forwarded bytes, and the
provider reads the whole previous turn. Third-party, both reset, because
the provider gets the loaded tool eagerly and its key changes anyway.
The control resets everywhere, as it should.

Token mode, Read results, first-party (`lineage` numbers trackers in
order of first use; `prefix_bytes` compares the forwarded history with
the previous turn's; token counts are the tracker's estimate):

```text
=== main f1bc2609
defer_loading tool added (ToolSearch load)
  turn  tools                      lineage  frozen  prefix_bytes  cache_read  cache_write
     1  Read                             0       0             -           0         6169
     2  Read                             0       3          same        6169         6159
   ...
     9  Read                             0      17          same       49282         6159
    10  Read+defer_loading               1       0       CHANGED           0        33133
  -> turn 10: NEW lineage, frozen=0, provider read 0 / wrote 33133 tokens
eager tool added (control)
    10  Read+eager                       1       0       CHANGED           0        39277
RESULT: deferred load resets lineage=True; eager tool resets lineage=True

=== this branch
defer_loading tool added (ToolSearch load)
     9  Read                             0      17          same       49282         6159
    10  Read+defer_loading               0      19          same       55441           15
  -> turn 10: same lineage, frozen=19, provider read 55441 / wrote 15 tokens
eager tool added (control)
    10  Read+eager                       1       0       CHANGED           0        39277
RESULT: deferred load resets lineage=False; eager tool resets lineage=True
```

Turns 1-9 are identical on both. On main, turn 10's reset compresses the
7 Read results that had aged out of the read-protection window (24,786
-> 10,828 bytes each); those bytes were cached verbatim, so the provider
re-writes everything. That compression is the side of the trade this
change gives up.

Token mode, third-party upstream
(`REPRO_UPSTREAM=https://gateway.example.com/anthropic`), identical on
main and this branch:

```text
    10  Read+defer_loading               1       0       CHANGED           0        33133
  -> turn 10: NEW lineage, frozen=0, provider read 0 / wrote 33133 tokens
RESULT: deferred load resets lineage=True; eager tool resets lineage=True
```

Cache mode, Bash results (`REPRO_MODE=cache REPRO_TOOL=Bash`),
first-party:

```text
=== main f1bc2609
    10  Bash+defer_loading               1       0       CHANGED           0        55456
  -> turn 10: NEW lineage, frozen=0, provider read 0 / wrote 55456 tokens
=== this branch
    10  Bash+defer_loading               0      19          same       52252           15
  -> turn 10: same lineage, frozen=19, provider read 52252 / wrote 15 tokens
```

In cache mode the reset gives no compression in exchange in this run:
the new lineage forwards the first tool result raw again (10,828 ->
24,786 bytes) and compresses nothing in its place, so main forwards and
writes 55,456 tokens where this branch reads 52,252 and writes 15.

- Not tested: no live Anthropic traffic from this branch. The 5-day miss
rates and the 16-turn A/B above were measured with the same filter
applied to the 0.39.0 wheel downstream, without the third-party/backend
gate (that machine talks to api.anthropic.com directly), not with this
branch. Streaming requests, a real Bedrock/litellm backend, Vertex, and
Linux/Windows were not run; the backend case is covered only by the
handler test's stand-in backend. Headroom's own tool-search injector
does not run in the repro (it sends fewer than 12 tools); its stand-down
on a client-deferred tool was checked by calling
`inject_tool_search_deferral` directly on Claude Code's 14 built-in
tools with and without one deferred MCP tool.

<details>
<summary>repro_deferred_tool_lineage.py</summary>

```python
"""Repro: a deferred tool load resets the Anthropic prefix-tracker lineage.

Drives the real proxy app (create_app -> /v1/messages -> AnthropicHandlerMixin)
through FastAPI's TestClient. Only the upstream call is replaced, by a model of
Anthropic's prompt cache that leaves defer_loading tools out of its key.

Session, in the shape Claude Code 2.1.296 sends while it defers tools: tools =
[Read, ToolSearch (client-side), DeferredToolPlaceholder (defer_loading)].
Turns 1-9 are Read results; turn 10 carries ToolSearch's tool_result with a
tool_reference to one MCP tool, which is appended to `tools` with
defer_loading: true. A control run adds a normal tool on a plain turn 10.

Usage: PYTHONPATH=<checkout> LITELLM_LOCAL_MODEL_COST_MAP=True python repro_deferred_tool_lineage.py
       REPRO_MODE=token|cache (default token), REPRO_TOOL=Read|Bash (default Read),
       REPRO_UPSTREAM=<base url> (default first-party api.anthropic.com).
       Token mode keeps recent Read results verbatim and compresses older ones.
"""

from __future__ import annotations

import copy
import json
import os
import sys
import tempfile

# Keep every file and network side effect away from the real ~/.headroom.
_tmp = tempfile.mkdtemp(prefix="dta-repro-")
os.environ.update(
    {
        "HOME": _tmp,
        "HEADROOM_WORKSPACE_DIR": os.path.join(_tmp, "ws"),
        "HEADROOM_CONFIG_DIR": os.path.join(_tmp, "cfg"),
        "HEADROOM_SAVINGS_EVENTS_PATH": os.path.join(_tmp, "savings_events.jsonl"),
        "HEADROOM_SAVINGS_PATH": os.path.join(_tmp, "proxy_savings.json"),
        "HEADROOM_SETTINGS_PATH": os.path.join(_tmp, "settings.json"),
        "HEADROOM_TOIN_PATH": os.path.join(_tmp, "toin.json"),
        "HEADROOM_CCR_BACKEND": "memory",
        "HEADROOM_BEACON": "off",
        "DO_NOT_TRACK": "1",
        "HEADROOM_HARD_WATCHDOG_SECS": "0",
        "HEADROOM_PORT_DISCOVERY": "0",
    }
)

import logging  # noqa: E402

logging.disable(logging.WARNING)

import httpx  # noqa: E402
from fastapi.testclient import TestClient  # noqa: E402

import headroom  # noqa: E402
from headroom.cache.prefix_tracker import PrefixCacheTracker  # noqa: E402
from headroom.proxy.server import ProxyConfig, create_app  # noqa: E402

MODEL = "claude-sonnet-4-5-20250929"
SYSTEM = [{"type": "text", "text": "You are a coding agent. " * 40}]
TOOL_NAME = os.environ.get("REPRO_TOOL", "Read")
READ = {"name": TOOL_NAME, "input_schema": {"type": "object"}}
TOOL_SEARCH = {
    "name": "ToolSearch",
    "description": "Search for tools by name or description.",
    "input_schema": {"type": "object", "properties": {"query": {"type": "string"}}},
}
PLACEHOLDER = {
    "name": "DeferredToolPlaceholder",
    "description": "Reserved placeholder that keeps deferred tool loading active.",
    "input_schema": {"type": "object"},
    "defer_loading": True,
}
TOOLS = [READ, TOOL_SEARCH, PLACEHOLDER]
DEFERRED = {
    "name": "mcp__docs__search",
    "input_schema": {"type": "object"},
    "defer_loading": True,
}
UPSTREAM = os.environ.get("REPRO_UPSTREAM") or None
EAGER = {"name": "WebFetch", "input_schema": {"type": "object"}}


def strip(obj):
    if isinstance(obj, dict):
        return {k: strip(v) for k, v in obj.items() if k != "cache_control"}
    if isinstance(obj, list):
        return [strip(v) for v in obj]
    return obj


def history(turns):
    messages = [{"role": "user", "content": [{"type": "text", "text": "Audit the repo."}]}]
    for i in range(turns):
        tid = f"toolu_{i:02d}"
        rows = [{"path": f"src/m{i}/f{j}.py", "size": 1000 + j, "ok": True} for j in range(400)]
        messages += [
            {
                "role": "assistant",
                "content": [
                    {
                        "type": "tool_use",
                        "id": tid,
                        "name": TOOL_NAME,
                        "input": {"file_path": f"/repo/m{i}.json"},
                    }
                ],
            },
            {
                "role": "user",
                "content": [
                    {"type": "tool_result", "tool_use_id": tid, "content": json.dumps(rows)}
                ],
            },
        ]
    messages[-1]["content"][-1]["cache_control"] = {"type": "ephemeral"}
    return messages


def after_tool_search(turns):
    """Turn 10 of the deferred run: ToolSearch's result loads DEFERRED."""
    messages = history(turns)
    del messages[-1]["content"][-1]["cache_control"]
    messages += [
        {
            "role": "assistant",
            "content": [
                {"type": "tool_use", "id": "toolu_ts", "name": "ToolSearch", "input": {"query": "docs"}}
            ],
        },
        {
            "role": "user",
            "content": [
                {
                    "type": "tool_result",
                    "tool_use_id": "toolu_ts",
                    "content": [{"type": "tool_reference", "tool_name": DEFERRED["name"]}],
                    "cache_control": {"type": "ephemeral"},
                }
            ],
        },
    ]
    return messages


class PromptCache:
    """Anthropic's prompt cache at message granularity; deferred tools unkeyed."""

    def __init__(self):
        self.written = set()

    def request(self, body):
        messages = body["messages"]
        tools = [t for t in body.get("tools") or [] if not t.get("defer_loading")]
        head = json.dumps(strip([tools, body.get("system")]), sort_keys=True)
        keys = [
            head + json.dumps(strip(messages[: i + 1]), sort_keys=True)
            for i in range(len(messages))
        ]
        hit = max((i for i in range(len(messages)) if keys[i] in self.written), default=-1)
        self.written.add(keys[-1])
        tokens = PrefixCacheTracker._estimate_message_tokens(messages)
        return sum(tokens[: hit + 1]), sum(tokens[hit + 1 :])


def run(added):
    app = create_app(
        ProxyConfig(
            optimize=True,
            mode=os.environ.get("REPRO_MODE", "token"),
            disable_kompress=True,
            rate_limit_enabled=False,
            cost_tracking_enabled=False,
            log_requests=False,
            anthropic_api_url=UPSTREAM,
        )
    )
    cache = PromptCache()
    seen = []
    with TestClient(app) as client:
        proxy = client.app.state.proxy
        store = proxy.session_tracker_store
        resolve = store.resolve_tracker

        def spy(*args, **kwargs):
            tracker = resolve(*args, **kwargs)
            seen.append({"tracker": tracker, "frozen": tracker.get_frozen_message_count()})
            return tracker

        store.resolve_tracker = spy

        async def upstream(method, url, headers, body, stream=False, **kwargs):
            read, write = cache.request(body)
            seen[-1].update(forwarded=copy.deepcopy(body["messages"]), read=read, write=write)
            usage = {
                "input_tokens": 10,
                "output_tokens": 2,
                "cache_read_input_tokens": read,
                "cache_creation_input_tokens": write,
            }
            return httpx.Response(
                200,
                json={
                    "id": "msg_x",
                    "type": "message",
                    "role": "assistant",
                    "model": MODEL,
                    "content": [{"type": "text", "text": "ok"}],
                    "stop_reason": "end_turn",
                    "usage": usage,
                },
            )

        proxy._retry_request = upstream
        last = after_tool_search(9) if added is DEFERRED else history(10)
        requests = [(TOOLS, history(n)) for n in range(1, 10)] + [([*TOOLS, added], last)]
        for tools, messages in requests:
            r = client.post(
                "/v1/messages",
                headers={"x-api-key": "k", "anthropic-version": "2023-06-01"},
                json={
                    "model": MODEL,
                    "max_tokens": 64,
                    "system": SYSTEM,
                    "tools": tools,
                    "messages": messages,
                },
            )
            assert r.status_code == 200, r.text[:300]
    return seen


def report(label, seen):
    ids = {}
    print(f"\n{label}")
    print("  turn  tools                      lineage  frozen  prefix_bytes  cache_read  cache_write")
    for i, s in enumerate(seen):
        lineage = ids.setdefault(id(s["tracker"]), len(ids))
        same = "-"
        if i:
            prev = seen[i - 1]["forwarded"]
            same = "same" if strip(s["forwarded"][: len(prev)]) == strip(prev) else "CHANGED"
        tools = TOOL_NAME if i < len(seen) - 1 else (TOOL_NAME + "+" + label.split()[0])
        print(
            f"  {i + 1:>4}  {tools:<25}  {lineage:>7}  {s['frozen']:>6}  {same:>12}"
            f"  {s['read']:>10}  {s['write']:>11}"
        )
    before, after = seen[-2], seen[-1]
    reset = after["tracker"] is not before["tracker"]
    print(
        f"  -> turn 10: {'NEW lineage' if reset else 'same lineage'}, frozen={after['frozen']},"
        f" provider read {after['read']} / wrote {after['write']} tokens"
    )
    return reset


print(
    f"headroom from {os.path.dirname(headroom.__file__)}, mode={os.environ.get('REPRO_MODE', 'token')},"
    f" tool={TOOL_NAME}, upstream={UPSTREAM or 'first-party'}"
)
deferred_reset = report("defer_loading tool added (ToolSearch load)", run(DEFERRED))
eager_reset = report("eager tool added (control)", run(EAGER))
print(
    f"\nRESULT: deferred load resets lineage={deferred_reset}; "
    f"eager tool resets lineage={eager_reset}"
)
sys.exit(0)
```

</details>

## Runtime Rollout Safety

- Rollout-managed feature(s): none. This is a fix in lineage selection,
not a feature in `headroom/rollout.py`.
- Minimum rollout channel: n/a (applies on every channel).
- Stable/default behavior changed: yes, for first-party Anthropic
requests with no translating backend whose `tools` differ from the
previous turn's only by `defer_loading` tools (in practice, the Claude
Code request after a ToolSearch load). They keep the conversation's
lineage and frozen prefix instead of starting at frozen=0.
- Token mode: cache reads up, cache writes down, and tokens saved per
request down, because old history stays verbatim instead of being
recompressed on each load (see the trade table).
- Cache mode: history no longer goes back to its raw form on the load
turn; cache reads up, cache writes down, no compression given up in the
repro.
- Unchanged: third-party Anthropic upstreams and translating backends
(deferred tools reach the provider eagerly, so they still count); any
change to a tool the prompt renders, `tool_choice`, `thinking`,
`output_config` or model; OpenAI, Responses and Gemini; the forwarded
request bytes of every request that did not reset.
- Kill switch / disable path: `HEADROOM_DEFERRED_TOOL_AFFINITY=0` (or
`false`/`no`/`off`), a runtime-env knob read per request: set it in the
proxy environment, let a reusing `headroom wrap` push it, or `POST
/admin/runtime-env` (no restart). Restores main's key exactly
(`test_kill_switch_keys_deferred_tools_again`,
`test_runtime_env_override_keys_deferred_tools_again`).
- Unsafe override required: no.
- Qualification impact: for Claude Code sessions that load deferred
tools against first-party Anthropic, expect `tok_saved/tok_before` and
the list-priced `compression_savings_usd` to FALL (12.1% -> 5.7% in the
A/B) while `cache_read` per request rises and cache writes fall (345k ->
121k), net input cost -43% to -50%. Judge it on both sides; either ratio
alone misleads.
- Rollback path: set `HEADROOM_DEFERRED_TOOL_AFFINITY=0` (no restart
needed), or revert these commits. No persisted state or schema changes.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A, no UI change.

## Additional Notes

- Documentation: no page documents the lineage key or the neighbouring
lineage kill switches (`HEADROOM_TRANSIENT_SYSTEM_LINEAGE`,
`HEADROOM_STABLE_BOUNDARY_BREAKPOINT`), so none was changed; the kill
switch is documented in `cache_key_tools`'s docstring.
- Follow-ups, not in this PR: tools that Headroom's own injector defers
still count in the key, and a per-request route-advice backend is not
seen by the key (see "Interactions checked").
- Separate finding, not in this PR: Claude Code 2.1.296's deferred-tools
shape (client-side `ToolSearch` plus a deferred
`DeferredToolPlaceholder`) carries no search tool Headroom recognizes,
so on default settings every such session logs
`tool_search_deferred_orphan` (at most hourly) although nothing is
wrong. One machine's proxy logs show it on real Claude Code traffic (for
example `deferred=7
names=DeferredToolPlaceholder,Monitor,WebFetch,WebSearch,...`). The
`_CLIENT_TOOL_SEARCH_NAMES` comment says Claude Code sends the
server-side shape when it genuinely defers; this version does not.
- Test hygiene, not in this PR: `tests/conftest.py` clears `HEADROOM_*`
but does not redirect the savings ledger, so handler-level tests that
post through `create_app` (for example
`tests/test_anthropic_fork_lineage_cache.py`) append every proxied
request to the developer's real `~/.headroom/savings_events.jsonl`. The
new test file redirects it itself.
- Review fix 5b83c2ec: the kill switch is a registered runtime-env knob
(see Changes Made), so the rollback reaches a proxy that `headroom wrap`
reuses. `cache_key_tools` stays a pure filter, so `headroom.cache` takes
no dependency on `headroom.proxy`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <tejas@headroomlabs.ai>
2026-10-28 17:55:44 -05:00
09b7e8c1e4 fix(packaging): rename the sandbox extra to lean and correct SECURITY.md claims (#3727)
**Revised 2026-10-05 after review:** tests now patch the deployment root
directly (never HOME) and assert POSIX modes only where enforced, the
supported-versions test follows the latest-release policy, and runner
scripts are recreated rather than truncated so a key is never written at
an old 0755 mode.

## Description

Security-remediation item A-10 (P-21, P-22). Renames the misleading
`sandbox` extra to `lean` (keeping `sandbox` as a deprecated alias for
one release, so `pip install "headroom-ai[sandbox]"` keeps working; not
a breaking change), corrects SECURITY.md's credential claims, and makes
the `headroom install --env` credential-persistence path owner-only and
documented. Details below.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update

## Changes Made

- `pyproject.toml`: `sandbox` extra renamed to `lean`; `sandbox =
["headroom-ai[lean]"]` deprecated alias; `uv.lock` regenerated (`uv lock
--check` clean).
- CI `python-314-wheels`, `run-all-plugins.sh` and the installation
guide now use `[lean]`.
- SECURITY.md: latest-release support policy (from main); discloses the
Copilot OAuth token and the `install --env` persistence path with its
modes.
- `headroom/install/paths.py`: `chmod_owner_only` verifies and reports
the mode it set.
- `headroom/install/state.py`: profile dir 0700 / manifest 0600 on every
save; warns if the directory cannot be narrowed.
- `headroom/install/supervisors.py`: runner scripts 0700 (`.ps1` 0600),
created fresh (old file removed first, so `O_TRUNC` cannot keep a 0755
mode while secrets are written), fail closed on POSIX if the mode cannot
be set.
- Tests: deployment writes are routed through a monkeypatched deploy
root; POSIX-mode assertions are skipped where modes are not enforced
(Windows).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ LITELLM_LOCAL_MODEL_COST_MAP=True uv run --frozen --extra dev --extra proxy pytest -q \
    tests/test_packaging_extras_and_security_docs.py tests/test_install/ tests/test_optional_dependencies.py
209 passed, 7 skipped
$ ruff check / ruff format --check (touched files)   -> clean
$ uv lock --check                                    -> Resolved 251 packages
```

Fail-before (2026-10-05 revision):

```text
test_rewriting_an_old_script_never_writes_secrets_at_its_old_mode   (fix reverted) FAILED assert 493 == 448 (0755 != 0700)
test_security_policy_supports_the_latest_release_without_a_version_number
                                                     (pinned 0.39.x table restored) FAILED
test_unrestrictable_profile_dir_warns_and_still_saves with Path.home() forced to ignore $HOME (as on Windows):
  previous version wrote <fake-home>/.headroom/deploy/default/manifest.json; new version writes nothing outside tmp_path
```

## Real Behavior Proof

- Environment: macOS (Darwin 25.6), CPython 3.14 via `uv run --frozen`,
branch head after merging main.
- Exact command / steps: with `HEADROOM_WORKSPACE_DIR` pointed at a temp
dir containing a pre-existing 0755 `deploy/default/` and 0755
`run-headroom.sh` (simulating a profile from an earlier version), ran
the real `build_manifest(..., extra_env={"ANTHROPIC_API_KEY": ...})` ->
`save_manifest` -> `render_runner_scripts`, then `stat` on every file.
- Observed result: `default/` 0700, `manifest.json` 0600,
`run-headroom.sh` 0700, `ensure-headroom.sh` 0700. `uv.lock` records
`headroom-ai[lean]` under marker `extra == 'sandbox'`, so the alias
resolves.
- Not tested: Windows end-to-end (ACL behaviour; mode assertions are
skipped there by design), a real `pip install "headroom-ai[sandbox]"`
from a published wheel, supervisor registration (launchd/systemd).

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: n/a (install-time file modes and packaging
metadata).
- Stable/default behavior changed: yes - deployment profile dirs and
runner scripts are owner-only after the next `headroom install apply`
(see release note below). `[sandbox]` keeps installing the same
dependencies.
- Kill switch / disable path: none; re-running an older version restores
old modes.
- Unsafe override required: no.
- Qualification impact: none.
- Rollback path: revert the PR; existing files keep their narrowed
modes, which is harmless.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

History of the change and the earlier review rounds follows.

---

Security-remediation item **A-10** (P-21, P-22). Two documentation
defects, each of which leads a reader to a false premise about the
product.

## 1. The `sandbox` extra promises isolation it does not provide

`headroom-ai[sandbox]` is a **dependency profile** — a torch-free subset
of `[all]`. It provides no isolation, no confinement and no privilege
reduction of any kind. The name invites exactly the wrong assumption,
and "we run it in the sandbox extra" is the kind of sentence that ends
up in someone's control narrative.

Renamed to **`lean`**, which describes what it actually is. `sandbox` is
kept as a deprecated alias forwarding to `[lean]` for one release,
because our own `python-314-wheels` CI job installs it
(`.github/workflows/ci.yml:218`) and so may customers. The alias
forwards rather than duplicating the dependency list, so the two cannot
drift.

## 2. SECURITY.md is stale and overclaims

- **Supported-versions table sat at 0.27.x** while the code ships 0.38.0
— four minor releases behind. A lagging table is worse than none: it
tells a reporter their release is unsupported when it is the current
one.
- **"No credential storage: We never store or log API keys"** is broader
than the code supports. `headroom/copilot_auth.py:646` persists the
GitHub Copilot OAuth refresh token to `~/.headroom/copilot_auth.json`.
It does so correctly — `chmod 0600` at `:648` — but it *is* stored, and
a blanket denial is the sort of claim a customer's security reviewer
checks.

The replacement says what is actually true: provider API keys are read
from the environment and forwarded without being written to disk; the
Copilot OAuth token is the one deliberate exception; and
request/response bodies may reach the runtime log when logging is
enabled, so that log is sensitive.

## Tests (original revision)

`tests/test_packaging_extras_and_security_docs.py` — four tests, all
failing before and passing after:

```
FAILED test_sandbox_alias_still_resolves_to_the_lean_profile
FAILED test_lean_profile_stays_torch_free
FAILED test_security_policy_supported_versions_match_the_shipped_version
FAILED test_security_policy_does_not_claim_blanket_no_credential_storage
4 failed  ->  4 passed
```

Method: commit, `git checkout origin/main -- pyproject.toml SECURITY.md`
(keeping the new tests), run, restore.

The version-table test is the one that matters most — main now documents
"latest release on PyPI" as the supported version, and the test (renamed
`test_security_policy_supports_the_latest_release_without_a_version_number`)
pins that and fails if a hard-coded version row returns, so **the table
cannot silently rot again**. That drift, not the wrong number, is the
actual defect.

`test_lean_profile_stays_torch_free` guards the profile's whole reason
for existing: adding a torch-pulling extra to it now fails the build by
name.

## Not done here

The backlog pairs this with a deprecation *warning* on the old extra
name. pip has no mechanism to warn on an extra at install time, so the
alias plus the comment is the available remedy; removal is noted for
after 0.39.x.

Ruff 0.16.7 `check` + `format --check` clean.

---

## Follow-up: the credential-persistence path (review of this PR)

@JerrettDavis blocked the first version because the SECURITY.md
*correction* was
itself inaccurate — it claimed provider API keys are not written to
disk, which
`headroom install --env KEY=VALUE` falsifies. Addressed in 3da8b955 by
disclosing the path accurately **and** narrowing its permissions.

What the trace found: `--env` values are merged into
`DeploymentManifest.base_env`
and persisted verbatim — into `manifest.json` *and*, as cleartext
`export` lines,
into the generated `run-headroom.sh` / `ensure-headroom.sh`. The
manifest was
already 0600, but only incidentally (via `mkstemp`), with nothing
pinning it. The
runner scripts were **0755** and the profile directory **0755** — a live
API key
in a world-readable file.

Persistence is kept: supervisors start from a bare environment, so
`--env` is the
only channel that reaches a supervised proxy. Redacting secret-valued
entries
would leave existing installs starting a proxy with no credential — a
silent 401
after upgrade. Instead: manifest 0600, runner scripts 0700 (0600 for the
`.ps1`),
profile directory 0700, applied on every save so pre-existing profiles
are
narrowed too; scripts created via `os.open(..., mode)` so no
world-readable window
exists between write and chmod.

SECURITY.md now names the files, the modes, that owner-only is not
encryption,
that rotation means re-running `install apply`, and that the Docker
runtime also
places the value in the container command line (`ps`-visible). The
`--env` help
text says the same.

The policy test now drives the real path — `build_manifest` →
`save_manifest` →
`render_runner_scripts` with a provider key against a fake `$HOME` —
asserts the
key really lands in both files, and **parses the documented octal modes
back out
of SECURITY.md** to compare against `stat()`, so the document and the
code cannot
drift apart again.

```
FAILED test_install_env_persists_a_provider_key_exactly_as_the_policy_describes
FAILED test_security_policy_discloses_the_persistent_install_credential_path
FAILED test_save_manifest_writes_owner_only
FAILED test_save_manifest_narrows_a_profile_directory_created_before_this_change
FAILED test_render_runner_scripts_are_owner_only_because_they_export_secrets
       (run-headroom.sh is 0755, exposing an API key)
FAILED test_render_unix_runner_narrows_a_script_written_by_an_earlier_version
6 failed  ->  315 passed
```

### ⚠️ Release note — behaviour change

Existing deployment profiles are **re-permissioned on the next `headroom
install
apply`**: `~/.headroom/deploy/<profile>/` and its runner scripts drop
group and
other bits (0755 -> 0700). Anyone reading or executing `run-headroom.sh`
as a
different, non-root user must now do so as the owner. No supported
supervisor
configuration does this — user scope runs as the installing user, system
scope as
root — but it changes the mode of files that already exist on customer
machines.
`--env` itself is unchanged: no value is dropped, redacted or relocated.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3727"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 14:56:54 -07:00
8d5dde7765 fix(docker): install the redis extra in every image (#4514)
## Description

The 0.42 images ship the replica-shared compress session store, but no
published variant installs the `redis` client. Setting
`HEADROOM_COMPRESS_SESSION_STORE=redis://...` stopped startup with
`ConfigurationError`, and `HEADROOM_CCR_BACKEND=redis` degraded to the
per-process store. This PR adds the `redis` extra to the Dockerfile
default and to all nine `docker-bake.hcl` targets. It locks the
invariant with a release test. It also documents the session fields of
`/v1/compress` and the shared session store.

Closes #4513

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (a fix or feature that changes existing
functionality)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `Dockerfile`, `docker-bake.hcl`: `HEADROOM_EXTRAS` gains `redis` in
the default and in all nine bake targets.
- `tests/test_release_workflows.py`: new
`test_every_docker_variant_includes_the_redis_extra` guards the
Dockerfile default, the bake targets, and the `pyproject.toml` extra.
- `docs/content/docs/installation.mdx`: new `redis` extras row. The
Docker tag table shows the new extras.
- `docs/content/docs/proxy.mdx`, `wiki/proxy.md`: documented the
`session` response fields, `HEADROOM_COMPRESS_SESSION_STORE`, its
failure behavior, and the `session_affinity` interaction.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_release_workflows.py -q
Pytest: 55 passed

$ uvx ruff@0.17.0 check .
All checks passed!
```

## Real Behavior Proof

- Environment: macOS worktree of this repository, Python 3.13.
- Exact command / steps: `python -m pytest
tests/test_release_workflows.py -q` and `uvx ruff@0.17.0 check .`.
- Observed result: 55 tests passed, ruff clean. The new test fails
against the pre-change `Dockerfile`, because its `ARG HEADROOM_EXTRAS`
line lacked `redis`.
- Not tested: the Docker images were not built. The release test asserts
the build args statically, as the sibling Bedrock test does.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. The change touches build args, docs,
and a test.
- Minimum rollout channel: not applicable.
- Stable/default behavior changed: every published image now installs
the `redis` client. With the extra installed,
`HEADROOM_COMPRESS_SESSION_STORE=redis://...` starts instead of failing,
and `HEADROOM_CCR_BACKEND=redis` uses Redis instead of the per-process
fallback.
- Kill switch / disable path: drop `redis` from `HEADROOM_EXTRAS` at
build time, or unset the two environment variables at runtime.
- Unsafe override required: no.
- Qualification impact: none. No runtime code path changed. The `redis`
import stays lazy, so deployments that do not set the variables are
unaffected.
- Rollback path: revert this PR and rebuild.

## Review Readiness

- [x] I performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I performed a self-review of my code
- [x] I commented my code, particularly in hard-to-understand areas
- [x] I made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I added tests that prove my fix is effective or that my feature
works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`. It is generated by
release-please from my Conventional Commit PR title, and a CI guard
enforces this.

## Screenshots (if applicable)

Not applicable.

## Additional Notes

The full test suite and `mypy headroom` were not run. This PR changes no
file under `headroom/`, so I ran the affected suite
(`tests/test_release_workflows.py`) and lint only. The docs half also
answers part of #3184 (documentation of the storage backends). The CI
half of #3184 stays open, and PR #3282 covers it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4514"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <tejas@headroomlabs.ai>
2026-10-28 11:29:27 -07:00
Tejas ChopraandClaude Opus 5.5 4288fbd903 docs(teams): replace the 3-months-free perk with a talk-to-us pitch (#4524)
## Description

Removes the "3 months free with a 12-month plan" offer and every
headroom-perks.vercel.app link. Each place now says: not seeing the
savings you want? Headroom OSS is a framework built to show what's
possible, so it saves less; Headroom Enterprise saves more and adds more
cost savings, tokens secured, on premise deployment, deep attribution
and many more features. Teams and enterprises: talk to us at
hello@headroomlabs.ai or https://headroomlabs.ai.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- README banner (`.github/assets/enterprise-perk.svg`): redrawn simple,
in the `hero.svg` type scale (title 42/700, lead 14.5, eyebrow 10.5); no
animation or chips.
- README "Headroom for teams" section and docs Teams page: the
OSS-vs-Enterprise table is replaced by the pitch, feature list and
contact.
- Docs site banner (`docs/app/layout.tsx`): new copy and link; new
banner id so people who dismissed the old one see it.
- Local dashboard offer (unlicensed installs only): new copy and link;
new localStorage dismissal key (`headroom-teams-contact-dismissed`) so
people who dismissed the old offer see the new one.
- Release-notes footer (`release.yml`) and `llms.txt`: same pitch, email
and site.
- `tests/test_dashboard_teams_perk.py`: asserts the contact email
instead of the perks URL.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) (no Python source changed)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest -q tests/test_dashboard_teams_perk.py -k "not route"
4 passed, 4 deselected in 0.49s
$ ruff check headroom/dashboard tests/test_dashboard_teams_perk.py
All checks passed!
```

The 4 `/dashboard` route tests fail locally with `CancelledError` at
proxy startup; they fail the same way on unmodified `main` in the same
venv, so CI is the judge for those.

## Real Behavior Proof

- Environment: macOS, Python venv of this repo, headless Chrome.
- Exact command / steps: rendered
`get_dashboard_html(licensed=False/True)` and checked the offer block;
rendered the banner with `Google Chrome --headless=new --screenshot
--window-size=900,220 file://.github/assets/enterprise-perk.svg`; `grep
-rn "headroom-perks\|3 months"` over README, docs, llms.txt, headroom/
and .github/.
- Observed result: unlicensed: offer shown, contact email present, no
perks link, new dismissal key. Licensed: offer removed. Banner renders
cleanly within 880×200 with text in the hero font. The grep finds
nothing.
- Not tested: the docs site build (`docs/`) and the release-notes footer
step (runs only on release).

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: only the text and link of the
dashboard offer on unlicensed installs
- Kill switch / disable path: setting `HEADROOM_LICENSE` removes the
offer, as before
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

No test was added for the new dismissal key: it is browser localStorage
read by Alpine, which the Python tests do not run. Past release notes
still carry the old footer; editing them is a separate one-off.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 10:27:33 -07:00
AlessandroSpallinaandClaude Opus 5.5 a90421083a feat(proxy): report the same tool savings on gateway /v1/compress as the proxy path (#4467)
## Description

Gateway-mode `/v1/compress` now reports the same savings as the proxy
path. The why is in #4466.

Closes #4466

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update

## Changes Made

Two commits, one per gap:

1. **`fix(proxy): count tool-schema compaction ...`**
- The gateway turn counts the tool definitions before and after
compaction.
- The handler adds both counts to `tokens_before` / `tokens_after`, with
the proxy path's guard (`0 < after < before`).
   - Tools the client already marked `defer_loading` are skipped.
2. **`feat(proxy): report tool_tokens_saved and tool_tokens_deferred
...`**
- Gateway-mode answers add `tool_tokens_saved` (from the tags `/stats`
sums) and `tool_tokens_deferred` (the estimated share).
   - Docs: `proxy.mdx`, `wiki/proxy.md`.

Non-gateway and fail-open answers do not change.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`, on the changed files)
- [x] Type checking passes (`mypy`, on the changed files)
- [x] New tests added for new functionality

```text
$ pytest -q tests/gateway/ tests/test_gateway_turn_unit.py tests/test_proxy_compress_endpoint.py
380 passed, 5 skipped
```

The tests also pass on the first commit alone (`tests/gateway/`: 280
passed). The new test for client-deferred tools fails without its fix.

## Real Behavior Proof

- Environment: macOS, Python 3.12. Before: image
`headroom:0.41.0-nonroot`. After: this branch, `HEADROOM_OFFLINE=1
headroom proxy`.
- Exact command / steps: start each build, then send it the same
gateway-mode `POST /v1/compress` — one message plus 5 tools whose
schemas carry `title` / `examples` / `default` (the annotation keys
schema compaction strips), with `"gateway": {"can_redrive": false,
"can_relay_response": false}` so the request is a single call. Read
`tokens_before`, `tokens_after`, `tokens_saved` and the `tool_*` fields
out of the JSON answer. No provider is contacted.
- Observed result: same request, same tools. On 0.41.0 the tool
definitions are invisible to the counts, so `tokens_saved` is 0 and
neither `tool_*` field is present. On this branch the compaction is
counted, so `tokens_before` / `tokens_after` carry the tool bytes and
`tokens_saved` is the 240 tokens compaction stripped
(`tool_tokens_saved` / `tool_tokens_deferred` stay 0 here: no
tool-search extension ran, and the extension is what books deferral).

```text
0.41.0:  tokens_before 13,  tokens_after 13,  tokens_saved 0    (no tool_* fields)
branch:  tokens_before 830, tokens_after 590, tokens_saved 240, tool_tokens_saved 0, tool_tokens_deferred 0
```

- Not tested: a live run with the headroom-tool-search extension
(deferral is covered by unit tests), and Responses-API bodies.

## Runtime Rollout Safety

- Rollout-managed feature(s): None gate this change — the gateway
`/v1/compress` accounting is not behind a registry feature. One
interaction: `proxy_output_shaper` (registry feature, available
`stable`, deliberately not default-enabled) consumes the augmented
count. `tokens_before` is folded at
`headroom/proxy/handlers/openai.py:11064` and then handed to the turn at
`:11083`, which passes it as `input_tokens` to `shape_gateway_body`
(`headroom/proxy/gateway_extension.py:213`) → `stratum_key` →
`input_bucket`. A turn whose tool definitions push it across one of the
2k / 8k / 32k / 128k boundaries therefore lands in a bigger stratum,
which picks a different output-savings baseline. The chat path already
has this coupling.
- Minimum rollout channel: n/a — no channel gate, so the change takes
effect wherever gateway mode is served, on every channel.
- Stable/default behavior changed: Yes, gateway-mode answers only.
`tokens_before` / `tokens_after` now include the tool definitions before
and after schema compaction, so `tokens_saved` covers the compaction
saving the same way the chat handlers already do, and two fields are
added. Non-gateway and fail-open answers are byte-identical.
- Kill switch / disable path: None. Without compaction there is nothing
to fold, and the two fields are additive on answers; a gateway that does
not want them ignores them.
- Unsafe override required: No. Nothing crosses a channel boundary.
- Qualification impact: None. No registry entry is added, so
`registry_digest` and `snapshot_digest` are unchanged. The gateway
`tokens_saved` figure itself changes, which a benchmark comparing arms
across this commit will see.
- Rollback path: Revert the two commits.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] I have made corresponding changes to the documentation
- [x] I have added tests that my fix is effective or that my feature
works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

- The fields are emitted for any claimed compress turn. Only the gateway
extension claims turns today.
- If a response-half hook rewrote the tool tags during a re-drive,
`/stats` would no longer match the answer. Core has no such hook.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 08:45:53 -07:00
3fef15d376 fix(offline): keep litellm's pricing table local under HEADROOM_OFFLINE (#4242)
## Description

`HEADROOM_OFFLINE=1` still lets litellm reach GitHub. `import litellm`
downloads `model_prices_and_context_window.json` from
`raw.githubusercontent.com` during the import (three attempts) unless
`LITELLM_LOCAL_MODEL_COST_MAP` is `true`. Headroom imports litellm from
24 places, among them the pricing helpers, the LiteLLM provider and
backend, the proxy's cost and savings trackers, and both Headroom Cloud
integrations. With the air-gap switch on, the first of those imports
dialled out.

Nothing in the chokepoint suite caught it, because no Headroom-built
client is involved, so no `guard_egress` site can dominate it.
`TestHeadroomCloudCompressionOffline` only passed when an earlier test
had already imported litellm. Run on its own (`pytest
tests/test_offline_egress_chokepoint.py -k HeadroomCloud`), it fails on
`main` with `SocketOpened: ... ('raw.githubusercontent.com', 443)`.

The only lever is the env var, and litellm reads it once, during its own
import. So:

- `pin_litellm_cost_map_local()` in `headroom/offline.py` sets
`LITELLM_LOCAL_MODEL_COST_MAP=True` while `HEADROOM_OFFLINE` is on.
- Every litellm import in `headroom/` calls it first, in the same scope.
A pin only at package import would miss a switch turned on later:
`is_offline()` is read when checked, and
`create_app(ProxyConfig(offline=True))` sets it after `import headroom`.
- The pin is Headroom's, so it is undone: called with the switch off, it
restores the value it replaced if the variable still holds the pin. A
process that leaves offline mode before its first litellm import gets
the online pricing refresh back.
- It overwrites the variable rather than using `setdefault`. litellm
only accepts `true`, so an operator value of `1` (the form every
Headroom switch takes), `False` or an empty string would otherwise
reopen the fetch.
- A meta-test fails the build for any litellm import without a preceding
pin, the same pattern as the existing site scanner.

No tracking issue; found while chasing a test that only failed when run
on its own.

Review found a second import-time leak on the same path:
`headroom/integrations/__init__.py` imports the CrewAI integration
whenever CrewAI is installed, and `import crewai` requests a Scarf pixel
from `api.scarf.sh` on a background thread unless
`CREWAI_DISABLE_TELEMETRY` is `true`. So `import
headroom.integrations.litellm_callback` still dialled out under the
switch. `pin_crewai_telemetry_off()` handles it the same way, through a
shared `_OfflineEnvPin`, and the meta-test now covers `crewai` imports
too.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/offline.py`: new `pin_litellm_cost_map_local()` (a no-op
unless `HEADROOM_OFFLINE` is on), documented in the module's list of
refused egress.
- All 24 litellm import sites call it immediately before importing:
`pricing/`, `providers/` (litellm, anthropic, openai, cohere, google),
`backends/litellm.py`, `proxy/` (cost, savings_tracker, route_advice),
`integrations/litellm_callback.py`, `telemetry/session.py`,
`perf/analyzer.py`, `learn/analyzer.py`, `cli/` (evals, learn) and
`evals/`.
- `pin_crewai_telemetry_off()` forces `CREWAI_DISABLE_TELEMETRY=true`
while offline (restored the same way), called before the one `crewai`
import (`headroom/integrations/crewai/agents.py`). Both pins share
`_OfflineEnvPin`.
- `headroom/__init__.py` also calls it, which covers a host that imports
litellm itself after importing Headroom.
- `tests/test_offline_egress_chokepoint.py`, new
`TestLiteLLMCostMapOffline`:
- subprocess probes (litellm only fetches on its first import, which an
earlier test usually did already) for four litellm-backed modules, with
the switch set both before and after `import headroom`;
  - operator values `1`, `False` and empty;
- leaving offline mode restores the replaced value (absent or `False`),
and an operator change made after the pin is kept;
  - `test_every_litellm_import_is_pinned_first`, the AST meta-test;
- the `offline` fixture now restores `LITELLM_LOCAL_MODEL_COST_MAP` at
teardown.
- `TestCrewAITelemetryOffline`: a stand-in `crewai` package with crewai
1.6.1's import-time tracking (no extra or CI job installs CrewAI),
probed early and late for `headroom.integrations.crewai` and
`headroom.integrations.litellm_callback`, plus operator values, a
restore test, the same probes against a real CrewAI when installed, and
`test_every_crewai_import_is_pinned_first`. A sanity test checks that
the stand-in does dial when online.
- `docs/metrics-technical-guide.md` and `docs/content/docs/proxy.mdx`
add the refresh and CrewAI's tracking to the refused list. The guide
also states the limits: a host that imports litellm *before* Headroom
has already made the fetch and must set the variable itself, and offline
prices come from the table bundled with the installed litellm release.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest -q tests/test_offline_egress_chokepoint.py tests/test_proxy_hardening.py
216 passed, 3 skipped, 1 warning in 38.44s

$ pytest -q --continue-on-collection-errors tests/test_providers tests/test_proxy tests/test_pricing* tests/test_perf* \
    tests/test_telemetry* tests/test_learn* tests/test_route_advice* tests/test_backend* tests/test_litellm* tests/test_cost* \
    tests/test_savings* tests/test_counterfactual* tests/test_evals* tests/test_integrations*
1742 passed, 337 skipped, 6 warnings, 1 error in 65.33s
# the 1 error is test_langchain_1x_compat.py failing collection (NameError: tool) because langchain_core
# is not installed in my venv; it fails identically on main df11f274

$ ruff check headroom tests/test_offline_egress_chokepoint.py tests/test_proxy_hardening.py   # ruff 0.17.0
All checks passed!
$ ruff format --check headroom tests/test_offline_egress_chokepoint.py tests/test_proxy_hardening.py
573 files already formatted
$ mypy <the 23 changed headroom/ files>
Success: no issues found in 23 source files

# The new tests fail without the fix: removing one pin call, and reverting to setdefault, gives
FAILED ...::test_operator_value_cannot_reopen_the_fetch[1]       assert ['raw.githubusercontent.com', ...] == []
FAILED ...::test_operator_value_cannot_reopen_the_fetch[False]
FAILED ...::test_operator_value_cannot_reopen_the_fetch[]
FAILED ...::test_every_litellm_import_is_pinned_first           names the unpinned import in headroom/backends/litellm.py
```

## Real Behavior Proof

- Environment: macOS arm64, Python 3.10.18 venv, litellm 1.101.0 (the
`uv.lock` version), `main` at df11f274 in a worktree vs this branch, the
same venv for both, `LITELLM_LOCAL_MODEL_COST_MAP` unset unless shown.
- Exact command / steps: a probe script replaces
`socket.create_connection` with a trap that records the address and
raises, runs `import headroom`, sets `HEADROOM_OFFLINE=1` before
(`early`) or after (`late`) that import, then imports the module and
prints the recorded attempts. Run with `HEADROOM_OFFLINE=1 python
probe2.py early headroom.integrations.asgi`, then the same for
`headroom.backends.litellm`, then `python probe2.py late
headroom.backends.litellm`, then `HEADROOM_OFFLINE=1
LITELLM_LOCAL_MODEL_COST_MAP=1 python probe2.py early
headroom.providers.litellm`.
- Observed result: main df11f274 gives `attempts=3
['raw.githubusercontent.com:443']` for all four runs; this branch gives
`attempts=0 []` for all four, with litellm loaded in every case.
- CrewAI: in a venv with crewai 1.6.1 + litellm 1.101.0 (the review
setup) and CrewAI telemetry at its default, `pytest
tests/test_offline_egress_chokepoint.py -k 'LiteLLMCostMap or CrewAI'`
gives 28 passed, including both real-CrewAI imports. Without the
`agents.py` pin, 10 CrewAI checks fail with `attempts=['api.scarf.sh']`.
- Not tested: a live proxy on an air-gapped host. The probe covers the
import paths; the proxy's lazy imports (`proxy/cost.py`,
`proxy/savings_tracker.py`, `proxy/route_advice.py`) are covered by the
meta-test, not by a running proxy.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: stable.
- Stable/default behavior changed: only with HEADROOM_OFFLINE set;
litellm then uses its bundled pricing table, which is what it already
fell back to after the three failed fetches. Without the switch nothing
changes.
- Kill switch / disable path: unset HEADROOM_OFFLINE.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- A host process that imports litellm before Headroom (a LiteLLM proxy
that loads the Headroom callback) has already fetched by the time any
Headroom code runs. That connection isn't Headroom's, so the docs state
the condition instead of claiming it.
- `HF_HUB_OFFLINE` and `TRANSFORMERS_OFFLINE` keep their `setdefault` in
`apply_offline_env()`: the Hub path is protected by `guard_egress` in
`onnx_runtime.py`, not by those variables.
- Possible follow-up, not in this PR: show which cost map litellm loaded
(litellm's `get_model_cost_map_source_info()`) in the offline startup
banner or `headroom doctor`.

---------

Co-authored-by: Kenneth Wong <17061293+KennethWKZ@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-24 08:45:38 -07:00