Commit Graph
66 Commits
Author SHA1 Message Date
d6df127d57 feat(compress): accept Gemini and Bedrock Converse bodies on the gateway turn (#4679)
## Description

`/v1/compress` in gateway mode (a request with a `gateway` block) now
accepts two more native request bodies:
- Gemini `generateContent`
- Bedrock Converse

It compresses their text slots and returns `body` in the same native
shape. This lets a gateway like LiteLLM compress native Gemini and
Bedrock passthrough traffic, which it couldn't before.

`config.mode: "ccr"` is rejected for these shapes with a 400. Every
other mode is marker-free for them.

(Was stacked on #4617, which has merged; the branch now carries current
`main`.)

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

### How

This works the same way `gateway_responses` does for Codex: it builds a
view instead of translating the body.
- Every compressible text slot becomes one chat-shaped pseudo message.
- Compressed text is written back by its path in the body.
- Everything else stays byte-identical: `thoughtSignature`, thought
parts, `functionCall`, `inlineData`, `toolUse`, `cachePoint`,
`reasoningContent`, `tools`/`toolConfig`.
- Tool calls appear in the view, so tool-name rules still apply. `Read`
stays verbatim, for example.
- A JSON slot is only written back if it still parses to the same JSON
type.
- If the compressed list doesn't line up with the slots, the original
transcript is forwarded and a warning is logged.

### CCR on native bodies (review fix)

A native body can't carry `headroom_retrieve`. Its `tools`/`toolConfig`
are forwarded untouched, and `create_ccr_tool_definition` only builds
chat, Anthropic and Responses shapes. It also never gets a re-drive. So
a CCR marker in a native body points at content the model can never
fetch. Before this fix, `config.mode: "ccr"` on a Gemini or Converse
gateway turn did exactly that: log output came back as a marker with
`ccr_hashes: [<hash>]` and `obligations: []`.

- `handle_compress` now answers `ccr` on a native view with `400
invalid_request`. The message names the shape, explains why, and lists
the supported modes.
- The check runs where the mode is chosen, right after `attach_view` has
recorded the shape. That is before any compression, session work or CCR
store write.
- **The rejection does not depend on `gateway.can_redrive`.** Even a
gateway that can re-drive has no way to put the retrieval tool into a
native request, so there is no capability under which native `ccr`
works.
- **Without a `gateway` block nothing changes.** A Gemini body is
already a 400 (no `messages`). A Converse body goes through the plain
messages path, where its untyped blocks are opaque: 0 tokens saved, no
markers in any mode, `ccr` included (checked). So the rule is
consistent: reject `ccr` wherever Headroom reads a native shape.
- The bypass header still returns the body untouched with no
compression, in any mode.
- I checked every other mode on the same marker-producing input
(default, `lossless`, `lossy_inline`, `lossless_then_lossy`): no marker
and no hash in any of them. These modes build pipelines with
`ccr_inject_marker=False, ccr_enabled=False`. Stale-Read notices are
hashless in marker-free modes since #4678.
- **Second defect, same family:** with `HEADROOM_DEDUPE=1`, cross-turn
dedup wrote `[↑NL same as msg 4]` into a Converse tool result. That `4`
counts view messages (system blocks, tool calls, result slots), not the
caller's `messages`, so it named the wrong message. Native views now set
`cross_turn_dedup_recoverable=False`, as `lossless` mode already did.
- The `not self.native_shape` guards on CCR arming and re-drive in
`GatewayTurn.finish` stay as defence in depth.

### Caller contract

**Gemini**
- Send top-level `contents` as a list, with no `messages` or `input`.
- `systemInstruction`/`system_instruction`, `tools` and
`generationConfig` are optional and pass through.

**Converse**
- Send `messages` where every message has role `user` or `assistant`,
and every content block has no `type` key.
- At least one block must use a Converse key: `text`, `toolUse`,
`toolResult`, `cachePoint`, and so on.
- `system` is optional, as a list of blocks.

**Both shapes**
- `config.mode`: omit it (marker-free default), or use `lossless`,
`lossy_inline` or `lossless_then_lossy`. `ccr` is a 400.
- Only gateway turns get a view. Without a `gateway` block the endpoint
behaves exactly as before. A client-sent `_headroom_native_view` key is
dropped, and `BODY_CONTROL_KEYS` strips the real one before the body is
returned.
- `config.frozen_message_count` and `config.protect_recent` count native
messages: Gemini `contents` entries and Converse `messages`.
`session.frozen_message_count` comes back in native units. The
`/v1/usage` reply's `frozen_message_count` still counts view messages,
and the docs say so.
- No re-drive is ever armed, so `obligations` never contains `redrive`,
and request hooks run stream-safe-only. Tool compaction and output
steering are skipped.
- Relayed usage can be:
- Gemini `usageMetadata`, where `promptTokenCount` already includes
`cachedContentTokenCount`.
- Converse `usage`, where `inputTokens` excludes `cacheReadInputTokens`
and `cacheWriteInputTokens`.
  - The whole provider response that carries either.
- Every gateway answer, fail-open ones included, carries `wire_shape`:
`gemini`, `converse`, `openai_responses` or `messages`. A caller that
sent a native body and gets anything else, or no field at all, is
talking to an older Headroom and should forward its original body.
- Top-level `messages` in the answer is the Converse list for Converse
and the chat-shaped view for Gemini.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`): no new errors; the 8
reported are in `ccr/mcp_server.py` and `cache/prefix_tracker.py` and
are identical on `main`
- [x] New tests added for new functionality
- [x] Manual testing performed (local replay, below)

New in `tests/gateway/test_native_shapes.py`:
- `test_ccr_mode_is_rejected_for_native_bodies[gemini|converse]`: 400,
`invalid_request`, message asserted.
- `test_other_modes_forward_native_bodies_without_ccr_markers`: 4 modes
x 2 shapes, on log output large enough to produce a marker. Checks no
hash, no marker, the shape echoed, non-message keys byte-identical and
the native list mapped back.
- `test_ccr_still_works_on_a_chat_gateway_turn`: the same log output in
a chat body with `ccr` still carries a marker. This proves the input
above is marker-producing, and that chat `ccr` didn't regress.
- `test_native_bodies_get_no_dedup_pointer_into_view_messages`.
- `test_native_turns_are_never_asked_to_redrive` now runs in the default
mode, because `ccr` is a 400.

### Test Output

```text
# before the fix (new tests only, rest of the file passing)
FAILED tests/gateway/test_native_shapes.py::test_ccr_mode_is_rejected_for_native_bodies[gemini]
FAILED tests/gateway/test_native_shapes.py::test_ccr_mode_is_rejected_for_native_bodies[converse]
FAILED tests/gateway/test_native_shapes.py::test_native_bodies_get_no_dedup_pointer_into_view_messages
3 failed, 25 passed

# after
tests/gateway/test_native_shapes.py: 28 passed

# tests/gateway tests/test_gateway_turn_unit.py tests/test_compress_api.py
# tests/test_proxy_compress_endpoint.py tests/test_compress_session_mode.py
# tests/test_compress_session_shared.py tests/test_compress_passthrough.py tests/test_compress_failure.py
1 failed, 507 passed, 4 skipped
# the one failure, test_lossy_inline_emits_no_ccr_markers[1-json_minifier], fails identically on main fda75de5

uvx ruff@0.17.3 check / format --check (changed files): All checks passed! / 3 files already formatted
```

## Real Behavior Proof

These are local synthetic replays. They are not provider qualification.

- Environment: macOS, Python venv, branch head `530c20aa`. Headroom runs
in-process via `create_app(ProxyConfig(optimize=True,
cache_enabled=False, rate_limit_enabled=False,
cost_tracking_enabled=False, log_requests=False, image_optimize=False))`
behind `fastapi.testclient.TestClient` from a loopback client. The
"provider" is an `httpx.MockTransport` that records each body it
receives and answers with a Gemini `generateContent` or Bedrock Converse
response, including usage. `LITELLM_LOCAL_MODEL_COST_MAP=True`. No real
provider calls.
- Exact command / steps: `LITELLM_LOCAL_MODEL_COST_MAP=True python -I -c
"import sys; sys.path.insert(0,'<worktree>');
exec(open('native_replay.py').read())"`. For each shape the script does
three things:
1. POSTs the native body plus `gateway: {can_redrive, session_affinity,
can_relay_response: true}` to `/v1/compress`.
2. Sends the returned `body` through the mock upstream at the real
Gemini or Bedrock URL.
3. POSTs the provider's whole response to `/v1/compress/response` with
the `turn_id`.

It then sends both native bodies again with `config.mode: "ccr"`. The
tool output is the 399-line grep listing used in the tests. The `ccr`
requests use 3000 log lines, which produce a marker in `ccr` mode.
- Observed result:

```text
[gemini] request half: wire_shape=gemini obligations=['relay_usage'] tokens 7246->3863 ccr_hashes=[] marker_in_body=False
[gemini] all 399 grep lines still readable in the forwarded body=True
[gemini] non-message keys unchanged=True first two contents unchanged=True
[gemini] provider got 16125 bytes (raw 29694), control keys forwarded=[], host=generativelanguage.googleapis.com
[gemini] response half: 200 {'action': 'done', ..., 'billed_usage': {'input_tokens': 900, 'output_tokens': 12, 'cache_read_input_tokens': 512}}
[converse] request half: wire_shape=converse obligations=['relay_usage'] tokens 8823->4843 ccr_hashes=[] marker_in_body=False
[converse] all 399 grep lines still readable in the forwarded body=True
[converse] non-message keys unchanged=True first two messages unchanged=True
[converse] provider got 16184 bytes (raw 29761), control keys forwarded=[], host=bedrock-runtime.us-west-2.amazonaws.com
[converse] response half: 200 {'action': 'done', ..., 'billed_usage': {'input_tokens': 300, 'output_tokens': 12, 'cache_read_input_tokens': 600, 'cache_creation_input_tokens': 0}}
[gemini ccr] 400 {'error': {'type': 'invalid_request', 'message': "config.mode='ccr' is not supported for gemini bodies: Headroom cannot add its retrieval tool to a native Gemini or Bedrock Converse request, so the model could not resolve a CCR marker. Use lossless, lossy_inline or lossless_then_lossy, or omit config.mode for the default marker-free mode."}}
[converse ccr] 400 {'error': {'type': 'invalid_request', 'message': "config.mode='ccr' is not supported for converse bodies: ..."}}
provider calls: 2
```

Before the fix, the same `ccr` requests returned 200 with `ccr_hashes`
of length 1, a `hash=` marker in the forwarded body, and `obligations:
[]`.
- Not tested:
- live Gemini (AI Studio or Vertex) and live Bedrock Converse: the
provider was mocked, so provider acceptance of the rewritten bodies is
unverified;
  - real Kong `ai-proxy` or a real LiteLLM gateway in front of Headroom;
- streaming callers (`streamGenerateContent`, `ConverseStream`): the
request half is identical, but no streamed response was relayed;
  - Gemini `thoughtsTokenCount` in output accounting;
  - session mode with a Redis store for native bodies;
- multimodal parts (`inlineData`, `image`, `document`) beyond their
byte-identical pass-through in unit tests.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. Native shapes only activate for
gateway-mode `/v1/compress` callers that send a `gateway` block and a
native body.
- Minimum rollout channel: n/a
- Stable/default behavior changed: no for existing callers. Chat,
Anthropic and Responses bodies are unchanged. A Converse body without a
`gateway` block goes through the plain path as before. Before this PR, a
native body with a `gateway` block was misread as chat (Converse) or
rejected (Gemini).
- Kill switch / disable path: the gateway omits the `gateway` block, or
forwards its original body when `wire_shape` doesn't match.
- Unsafe override required: no
- Qualification impact: none for existing paths
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

Known follow-ups, not in this PR:
- A `functionResponse.response` with any string field never compresses
its nested dict values. The JSON slot is only used when there are no
string fields. This is a missed saving, not a correctness bug.
- `handlers/gemini.py` (the proxy path) and `gateway_native.py` map
Gemini parts with different rules, and their part-key lists are kept in
step by hand. They should share one constant.
- `protect_recent` translation and stream-safe hook gating for native
turns have no dedicated test.
- Gemini `thoughtsTokenCount` is not added to output tokens when usage
is relayed, so output savings attribution undercounts for thinking
models.
- This is not specific to native shapes: the marker-free default folds
very repetitive logs to a single line (for example `[3000 lines omitted:
3000 INFO]`) on chat bodies too, with no retrieval path.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-15 01:52:50 +05:30
33f067a243 feat(proxy): add lossless /v1/compress mode and detect provider by wire shape (#4617)
## Description

Two gaps stop a gateway from using `/v1/compress` well. Both showed up
while fixing LiteLLM's `headroom` guardrail, which is the follow-up PR
that needs this one.

1. **No mode compresses messages losslessly without markers.** A gateway
with no retrieval loop can't resolve a marker or call
`headroom_retrieve`, so it needs a mode with nothing to retrieve. The
existing marker-free modes still run lossy compressors.
2. **Hooks guess the provider from the model name.** A gateway can send
any client's shape to any model. Claude Code's Anthropic body routed to
Gemini looked like OpenAI, so skill condensing found nothing to change.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)

## Changes Made

**`config.mode="lossless"`**
- Runs format-native folds only, with no markers and no CCR store
writes. A registered lossless provider still applies.
- Turns off stale-Read pointers, which drop the file body and need the
retrieval tool.
- Turns off cross-turn dedup pointers, which a client that never numbers
its messages can't resolve.
- Scope: the guarantee covers message compression (lossless codecs, no
CCR). On a gateway turn, tool schema compaction and registered turn
hooks still run as configured, as in every other mode; description
truncation only happens when an operator sets
`HEADROOM_TOOL_DESC_MAX_CHARS`.

**`provider_for_body`**
- Hooks and the session tracker now use the body's wire shape, and fall
back to the model name when the shape proves nothing, such as plain text
with no system prompt or tools.
- Anthropic's own tool types (server tools such as
`web_search_20250305`, client tools such as `bash_20250124`) end in an
eight-digit date and count as the Anthropic shape, so a first-turn body
whose only tool is a server tool is read correctly.
- `route.provider` still follows the model.
- The cache breakpoint budget runs when either the shape or the model is
Anthropic, and never on a Responses body.
- `gateway_response_fields` loses its unused `provider` parameter, and
`RESPONSE_RECEIVED` keeps reporting the model's provider.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`, `ruff format --check .`, on the
pinned 0.17.0)
- [ ] Type checking passes (`mypy headroom`): one error, in
`headroom/integrations/langchain/chat_model.py`, which this PR doesn't
touch (local mypy 1.19.1)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
pytest tests/test_proxy_compress_endpoint.py tests/gateway tests/test_gateway_turn_unit.py tests/test_compress_route_tokenizer_by_model.py
425 passed, 4 skipped

Regression tests with the fixes removed: 8 failed, 7 passed
```

## Real Behavior Proof

- **Environment:** LiteLLM main with the follow-up guardrail change,
this branch as the sidecar, and a fake upstream that simulates prompt
caching. No real provider API was called.
- **Exact command / steps:** replayed a scripted 17-call agent session
with exact-duplicate outputs removed, once direct to the fake upstream
and once through LiteLLM and `/v1/compress`.
- **Observed result:** 0 errors, 0 rewrites of already-sent results,
`Read` and `Edit` results unchanged, about 1.3s added in total.

| Client | Prompt cost vs direct |
|---|---|
| Anthropic Messages, Claude Code cache markers | −3.9% |
| Chat Completions | −3.8% |
| Responses | −4.1% |

The script's outputs are synthetic, so treat these as a comparison
between setups, not a forecast for real traffic.

## Known limits

- The default mode and `lossy_inline` can still return a stale-Read
`Retrieve original: hash=` marker, although the docs call them
marker-free. This predates the PR and needs a separate fix. Proxy
`--lossless` has no read-lifecycle guard either, from reading the code.
- With no `ContentRouter`, `_derived_compress_pipeline` falls back to
the default pipeline, so `lossless` wouldn't hold. Production always has
a router, so only test stubs reach this.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4617"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Maintainer post-merge revisit: main e9331b6b
Published head `e50b001bd05c9bc85405accca49cbef0e7786e51` incorporates
upstream main `e9331b6b963b5b6554f45943abed2366aaa25ea6` after #4142,
#4637 and #4641 merged. 164 endpoint/gateway/tokenizer/admission tests
pass; one json_minifier strategy assertion fails identically on actual
main e9331b6b. Ruff/format/diff pass. Import-only conflict resolved by
retaining both provider_for_body and RequestBodyTooLarge. Both existing
holds reconfirmed: lossless HTTP output removes unique description
suffix via tool_desc_compaction; Anthropic server tool with Gemini model
is classified OpenAI.

Contributor correctness logic was not rewritten. The existing Changes
Requested decision remains applicable; unchanged findings are not
reposted as duplicate reviews. Scoped Windows qualification uses the
existing development native extension, not a fresh build. Hosted checks
must finish independently; no contributor CI action requested and
nothing merged by this review.


## Maintainer post-merge revisit: main cc1eb540
Published `38b871120e318f6eaa1cb2ce16f96d4f6cc2b1e8` includes upstream
main `cc1eb5402aade58204b96b6b4b4d280cdd5456d2`. 90 gateway/tokenizer
tests pass; both existing HTTP/provider counterexamples reconfirmed.
Clean merge; Ruff/format/diff pass. Contributor correctness logic was
not rewritten. The existing Changes Requested decision remains
applicable; unchanged findings are not reposted. Scoped Windows
qualification uses the existing development native extension, not a
fresh build. Hosted checks must finish independently; nothing merged by
this review.


### Maintainer current-main requalification

Exact head `33395b5cdd7ee9facce9f97f2a3250f5e6e49cb2`, upstream main
`ecbaf86d78cf52d185fbffa306157b9a4cc1b906`.

Mechanically merged upstream/main
ecbaf86d78 without changing contributor
policy logic. Re-reviewed the combined gateway/provider delta and
existing comments. All 90 focused gateway/tokenizer tests and ten
inherited Gemini fixture cases pass; Ruff0.17.3 lint/format and
whitespace checks pass. The shared Gemini repair was already present, so
no duplicate fixture commit was added.

Both existing correctness findings independently remain on this combined
tree. An HTTP200 /v1/compress request with config.mode=lossless still
drops UNIQUE_REQUIREMENT_AT_END from a long tool description, reporting
tool_desc_compaction. An Anthropic web_search_20250305 tool body with a
Gemini-named model still returns provider openai. Existing requested
semantic fixes/regressions remain necessary; maintainers are handling CI
and no contributor CI execution is requested. No contributor logic was
rewritten and nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `4c40adb53a68f382e32f00620adcfaf1cff477f5`, upstream main
`8d3e9e5ca7cd645eb73084b848edbea5fc796e1a`.

Clean mechanical sync with upstream
main8d3e9e5ca7cd645eb73084b848edbea5fc796e1a including intervening
release/runtime/cache/CCR/SSE/HTTP1.1 merges. No contributor
gateway/provider policy rewritten. Fresh author/Devin discussion
considered.100focused gateway-contract/tokenizer/Gemini tests
pass;7changed Python files pass Ruff0.17.3check/format and whitespace
checks pass. Last main delta after test start changes only
Rust/OpenClaw, leaving tested Python inputs unchanged.

Both existing blockers remain freshly reproduced from this worktree
(source import path verified): real app POST/v1/compress lossless
returns200 but drops UNIQUE_REQUIREMENT_AT_END from tool description via
tool_desc_compaction; Anthropic server-tool wire shape with Gemini model
returns provider openai. Preserve promised lossless content and classify
server-tool wire forms correctly, with regressions. No external provider
session/nativebuild claimed. Copied extension supports Python startup;
maintainers handle hosted checks. Nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `8b7b2b0d02daf1a031828ecb13095ab315431f7e`, upstream main
`917edc62230c3c4a8a415e2053da68dab1f003dd`.

Current-main integration against917edc62230c3c4a8a415e2053da68dab1f003dd
is clean. Compared entire headroom/crates/Python manifests+lock/tests
inputs to prior qualified4c40adb53: byte-identical. Intervening changes
are reviewed npm maintenance and release-envelope recovery workflow; no
contributor gateway policy changed.75new-base
release-workflow/normalizer tests pass,whitespace passes.
Prior100gateway/tokenizer/Gemini results remain scoped to those
identical inputs rather than relabeled fresh.

Both gateway blockers freshly rechecked from worktree: Anthropic
server-tool+Gemini body classifiesopenai; actual lossless/v1/compress
returns200 but drops unique description suffix via tool_desc_compaction.
Retain existing losslessness/classification corrections. No fresh
nativebuild/provider claim. Maintainers handle exactheadCI;nothing
merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `490ff6defdebddda9106e9e143d42d1e463b6324`, upstream main
`7592e681459f14a2041eb5f524a658cb6ad1dd92`.

Clean integration with upstream/main
7592e68145. Reviewed both intervening
memory fixes and fresh PR discussion. Gateway/runtime/Rust inputs are
unchanged from the previously reviewed head; only the two main memory
fixes and their regressions enter this merge. Nine memory/Gemini
regression tests pass; main-relative whitespace passes.

Fresh actual-app probes retain both existing blockers: Anthropic
server-tool content with a Gemini model classifies as OpenAI; lossless
/v1/compress removes the unique tool-description suffix through
tool_desc_compaction. No contributor policy was rewritten. Existing
native extension used for Python startup, not a fresh native-build
claim. Maintainers handle hosted checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `d3cb3a02af03991a696461160ee0db322dd8bf48`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Clean integration with upstream/main
2d3ee51846 after the external 0.43.1
release merge. Exact headroom/crates/tests comparison against prior
reviewed 490ff6def shows no changes. Re-reviewed generated release
metadata; 78 release/version/MCP tests pass on this composed tree, and
whitespace passes. Previous nine memory/Gemini regressions and fresh
gateway probes remain scoped to identical runtime inputs.

Both established blockers remain: lossless /v1/compress drops the unique
tool-description suffix, and Anthropic server-tool content with a Gemini
model classifies as OpenAI. Runtime probes were not relabeled as rerun
after this metadata-only merge. No contributor policy rewritten; no
fresh native-build claim. Maintainers handle exact-head checks; nothing
merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `e0c873630e9ffc7bc0d0d2fdd22c483b731e49a3`, upstream main
`910588e6b15ab54c4099d03703e5fa20c8abb757`.

Clean mechanical synchronization with upstream
main910588e6b15ab54c4099d03703e5fa20c8abb757, including the externally
merged runtime fixes, docs dependency update and tree-sitter lock
repair. No contributor gateway/provider policy rewritten. Current
author/Devin discussion considered. All90 gateway
contract/tokenizer-by-model tests pass on the preceding composed runtime
tree; the final dependency-only merge changes only pyproject/uv.lock,
leaving those gateway/tokenizer sources unchanged. No redundant rerun on
the final merge is claimed. Changed Python Ruff lint/format and
main-relative whitespace pass.

Both existing blockers freshly reproduce on the composed runtime paths:
lossless /v1/compress returns200 but drops UNIQUE_REQUIREMENT_AT_END
from the tool description and reports tool_desc_compaction; Anthropic
server-tool content with a Gemini model is classified as OpenAI.
Preserve the lossless original payload and classify the full request
protocol consistently, with endpoint regressions for these inputs. The
main fixes do not close either defect.

Tests use the existing native core for startup, not a fresh native
build; no live provider traffic was made. Maintainers handle hosted
checks; no inherited-main pipeline failure attributed and nothing
merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `507433606f698249e546e1418928caee99d0c901`, upstream main
`7877241d84c93b3ec487e07324245838ddae85cc`.

Clean mechanical integration with
upstream/main7877241d84c93b3ec487e07324245838ddae85cc. Gateway
production/contracts remain byte-identical to the preceding reviewed
head; re-reviewed the new CLI/Hermes integration.127 gateway
request/response contract and model-tokenizer tests pass; whitespace
passes. Both existing holds freshly reproduce on this exact tree:
lossless /v1/compress returns200 while dropping
UNIQUE_REQUIREMENT_AT_END and reporting tool_desc_compaction; Anthropic
server-tool content with a Gemini model is classified as OpenAI.
Preserve the lossless original payload and classify the full request
protocol consistently, with endpoint regressions. No contributor
gateway/provider policy rewritten. Existing native core used for
startup; no fresh native build or live provider claim. Maintainers
handle hosted checks; nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `b0daece6b10f623a6e7b0dc3eb01c73e05fa3209`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Final post-Dependabot requalification against upstream/main
c01cf07d94. All six Dependabot updates are
now merged. Relative to this branch's freshly tested prepared tree, the
final main sync changes only the three intended Cargo.lock records;
Python/source/test inputs are unchanged. Re-reviewed the integration and
confirmed complete native source/lock/toolchain objects match the tested
combined build and exact fresh native DLL used for this branch's local
tests.

Focused fresh-native qualification: 127 passed, 3 warnings in 104.78s
(0:01:44). Selected real dependency overlays and native DLL paths are
recorded in final4617-fresh-native-tests.log; the DLL provenance/input
hashes are in final-native-artifact-proof.json. No redundant rerun is
implied after this Cargo-only merge. Whitespace passes. Prior source
review and scoped real-behavior evidence remain applicable to unchanged
contributor inputs.

Both existing blockers freshly reproduce using the combined native DLL:
lossless /v1/compress returns200 but removes UNIQUE_REQUIREMENT_AT_END
and reports tool_desc_compaction; Anthropic server-tool content with a
Gemini model is classified as OpenAI. Preserve the original lossless
payload and classify the complete wire protocol, with endpoint
regressions.

Maintainers handle hosted checks; no external-provider claim,
contributor semantic rewrite or non-Dependabot merge.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `ec6abc0228a93c5eab1881938041dfee344b1e3a`, upstream main
`2973ca593d2fe65c3259941fdff8009175eff456`.

Requalified after human merges #4667/#4669/#4670 against upstream main
ce42754912. All 11 integration patches
exactly match the prior-main-to-current-main delta after ignoring diff
locations; no contributor gateway/provider policy changed. Fresh gateway
request/response contracts, model-tokenizer, gateway unit,
Kompress-disable and must-keep suites: 207 passed, 3 warnings. Six
changed Python files pass Ruff lint/format; offline lock validation and
whitespace pass. Existing matching-core DLL used; native
source/lock/toolchain unchanged by these main merges, with no fresh
native build or live-provider claim. Both existing blockers freshly
reproduce: lossless /v1/compress returns 200 while removing
UNIQUE_REQUIREMENT_AT_END and reporting tool_desc_compaction; Anthropic
server-tool content with a Gemini model is classified as OpenAI.
Preserve original lossless payload and recognize complete wire protocol
with endpoint regressions. Author and Devin discussions checked;
existing findings remain actionable. Maintainers handle hosted checks;
nothing merged.
Subsequent human merge #4671 advanced main to
2973ca593d before publication; guard
stopped the old candidate. Cleanly integrated the new workflow-only
delta and re-reviewed action pinning with 62 workflow/release tests
passing. Runtime/native sources are byte-identical to the immediately
preceding 207-test and real-behavior qualification; those results apply
to unchanged inputs. All prior-main hosted results are historical for
readiness.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `9f857218486fa451ecacc675863a64bc95f45bb0`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Current-main requalification
against2b0315b2d4b0d54947050bbdad2de9e89df06378 after later human
merges. All37 integration patches exactly match2973ca59-to-current-main
delta after diff-location normalization; contributor gateway/provider
policy unchanged.393gateway request/response
contracts,model-tokenizer,gateway
unit,Kompress-disable,must-keep,doctor,search tests
pass;3warnings.27changedPythonfiles pass Ruff lint/format;offline
lock/whitespacepass. Existing pr3982 native DLL verified every tracked
native-input hash/artifactchecksum;no fresh build claimed. Author/Devin
discussion checked. Both correctness blockers freshly reproduce:lossless
/v1/compress returns200while droppingUNIQUE_REQUIREMENT_AT_END and
reportingtool_desc_compaction;Anthropicserver-toolcontent
withGeminimodel classifiedOpenAI. Preserve original lossless payload and
classify full request wire protocol consistently,with endpoint
regressions. No live provider qualification,contributor logic rewrite,or
inherited-main CI fault attributed. Maintainers handle hosted
checks;nothing merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.


### Maintainer current-main requalification

Exact head `41b2145418db7f5980cd52e6235ab330fcd31ac0`, upstream main
`0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`.

Clean integration with upstream/main
0ddaa56c21. All 17 integration patches
exactly match the prior-reviewed-main to current-main delta after
diff-location normalization; contributor gateway/provider policy is
unchanged. Fresh gateway/tokenizer/search/doctor and incoming
Vertex/backend/CLI qualification: 455 passed, 11 skipped, 3 warnings.
Additional gateway request/response contracts and compress-endpoint
suite: 158 passed, 1 warning. Twelve changed Python files pass Ruff
lint/format; offline lock and whitespace checks pass.

Author discussion and outstanding Devin findings read. Both existing
correctness findings freshly reproduce on this exact tree: lossless
/v1/compress returns 200 but drops UNIQUE_REQUIREMENT_AT_END through
tool_desc_compaction; Anthropic server-tool content with a Gemini model
classifies as OpenAI. Existing requested corrections remain; no new
duplicate finding. Local app probes are not live provider proof. Native
DLL reused only after verifying the complete tracked input set, every
recorded input SHA256 and artifact checksum against
pr3982-c01-native-proof.json; no fresh build claimed. Maintainers handle
exact-head hosted checks. No PR merged.


Exact-head hosted checks are being handled by maintainers. Existing
correctness/qualification holds remain; nothing merged.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-07 20:36:11 -06:00
alan blountandJD Davis 0ddaa56c21 fix(vertex): explain Vertex failures in-band, correct the model/region matrix (#2640)
## Description

Vertex rejects onboarding requests for a handful of recurring reasons —
expired ADC, a partner model never enabled in Model Garden, a model not
served in the requested location, a missing optional SDK — and its own
error text names none of the fixes. Users see a bare `404` or an opaque
`500` through their SDK and cannot tell a proxy bug from a
project-config gap.

This makes the proxy explain those failures where it happens, adds a
`[vertex]` extra so the LiteLLM Vertex backend can actually start, and
corrects an example, a test matrix, and three documentation claims that
could not have worked as written.

Closes #2632

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

**Explain Vertex failures in-band**
(`headroom/providers/vertex/diagnostics.py`, new)

- On `401/403/404/429`, attach an actionable hint three ways: a WARNING
in the proxy log, an `x-headroom-hint` response header, and a note
appended to `error.message`. The last is load-bearing — SDKs raise with
the message string and discard headers.
- Successful responses are never modified; annotation is idempotent;
statuses we cannot explain get no hint rather than a plausible-sounding
wrong one.
- Routes opt in via a one-line `@with_vertex_diagnostics` decorator
reading path params FastAPI already injects (**+8 lines** in
`proxy_routes.py`, no re-indentation).
- Same treatment for backend setup failures, where `--backend vertex`
surfaced LiteLLM's raw `No module named 'vertexai'` as an opaque 500.

**Make `--backend vertex` able to start** (`pyproject.toml`,
`registry.py`, `cli/proxy.py`)

- New `[vertex]` extra carrying `google-cloud-aiplatform>=1.38`.
Deliberately **not** folded into `[proxy]`: measured at ~175 MB
installed, and the native passthrough routes never needed it. Follows
the existing `proxy-prod` split.
- Startup preflight keyed on the **resolved provider**, so every alias
normalized at `registry.py:229` (`vertex`, `vertex_ai`,
`litellm-vertex`, `google-vertex`, `googlevertex`) is covered. Skipped
when a backend class is injected, so existing fake-based tests are
unaffected.
- New `BackendUnavailableError` so the CLI prints the remedy and exits
`2` instead of dumping a traceback through startup internals.

**Correct things that could not have worked**

| | was | is |
|---|---|---|
| SDK client | `http_options={"api_endpoint": ...}` | `{"base_url":
"http://..."}` — `api_endpoint` is not a field on `HttpOptions`;
pydantic rejects it before any request is sent |
| Thinking config | `ThinkingConfig(thinking_budget_tokens=)` |
`thinking_budget=` |
| Request payload | `contents: [{parts: [...]}]` | adds `role: "user"` —
the `global` endpoint enforces it, regional was silently lenient |
| Auth guidance | `gcloud auth print-access-token` |
`application-default` — the plain user token gets `401
ACCESS_TOKEN_TYPE_UNSUPPORTED` |
| Test matrix | model × location cross-product | explicit servable pairs
|

- Test matrix is now four pairs that can actually serve:
`global`/`gemini-flash-latest`, `global`/`claude-sonnet-4-6`,
`europe-west2`/`gemini-3.5-flash`, `us-east5`/`claude-sonnet-4-6`.
Gemini 3.x has no US regional endpoint and `-latest` aliases are
global-only, so a pinned `us-central1` Gemini case can never pass. No
Gemini 2.5 — it retires 2026-10-16.
- `test_thinking_levels` asserted only `status_code < 500` while sending
an ignored `thinkingBudgetTokens` key: it passed on 400s and proved
nothing. Now asserts 200 plus scored usage metadata with the correct
`thinkingBudget` spelling.
- Cases the caller's project genuinely cannot serve now **skip** with an
actionable reason instead of reporting a red proxy.

**Docs** (`proxy.mdx`, `installation.mdx`, `wiki/vertex.md`)

- `proxy.mdx` claimed `publisher=anthropic` passthrough "routes through
the same LiteLLM-Vertex path as `--backend vertex_ai`". It does not —
without the flag it is a direct forward, which is why it works on a bare
`[proxy]` install where the LiteLLM path cannot even import.
- Documented the trap of combining `--backend vertex` with native Vertex
URLs: LiteLLM takes project/region/model from its own config and ignores
the URL.
- `installation.mdx` and `wiki/vertex.md` said the SDK is "not included
in any extra".
- Cleared 15 pre-existing `ruff` findings in the two files this PR
rewrites (4 remain in `plugins/headroom-oauth2`, untouched by this
work).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ mypy headroom
Success: no issues found in 511 source files

$ ruff check .
Found 4 errors.        # all in plugins/headroom-oauth2, pre-existing, untouched by this PR

$ pytest tests/test_proxy_vertex_native_integration.py tests/test_vertex_diagnostics.py -q
44 passed, 1 warning in 69.86s          # 8 live integration + 36 offline diagnostics, 0 skipped

$ pytest tests/ -k "anthropic_messages or handler_anthropic or backend or registry or cli_proxy or vertex or proxy_routes"
853 passed, 49 skipped, 9345 deselected, 2 warnings in 28.41s
```

New tests in `tests/test_vertex_diagnostics.py` (36, no credentials
required): status coverage, per-publisher remedy divergence,
idempotency, `content-length` consistency after body rewrite, non-JSON
and body-less/streaming responses, every backend alias preflighted,
other providers unaffected, injected backend bypasses the preflight.

## Real Behavior Proof

- Environment: macOS 14, Python 3.11.15, live GCP project
(`alanblount-sandbox`) with `aiplatform.googleapis.com` enabled and
Claude enabled in Model Garden; ADC via `gcloud auth application-default
login`
- Exact command / steps: `export GCP_PROJECT_ID=alanblount-sandbox &&
export GCP_ACCESS_TOKEN=$(gcloud auth application-default
print-access-token) && uv run --extra proxy --with google-genai python3
examples/vertex_genai_sdk_demo.py && uv run --extra proxy --with
pytest-asyncio pytest tests/test_proxy_vertex_native_integration.py
tests/test_vertex_diagnostics.py -q && headroom proxy --backend vertex`
(exits 2, then retried with `[proxy,vertex]` extra installed)
- Observed result: demo printed `✓ Standard response received
successfully! > 1, 2, 3, 4, 5` and `✓ Thinking response received
successfully!`; 44 tests passed (8 live integration, 36 diagnostics), 0
skipped; `--backend vertex` exited 2 with the remedy message; same
request with the extra installed returned 200; invalid region produced
`[headroom] hint:` in the response body and `x-headroom-hint` header;
success responses were byte-clean
- Not tested: Windows; `us`/`eu` multi-region endpoints and Claude 4.7+
(project cannot serve them); Provisioned Throughput; Rust proxy path;
`--backend vertex` dropping URL project/model on native Vertex routes
(documented as known gap in Additional Notes)


## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — proxy and CLI behavior; terminal output is quoted above.

## Additional Notes

- `uv.lock` intentionally not included: the new extra is opt-in and does
not change default resolution.
- Error-body rewriting is confined to already-failing responses and
wrapped so a diagnostic can never turn a clean upstream error into a
proxy 500.
- The preflight raises where the provider is resolved (`registry.py`)
rather than at the CLI, so programmatic embedders get the same
guarantee; the CLI only formats it.
- Three test files fail to collect on this branch
(`test_hermes_passthrough_compression`, `test_observability_metrics`,
`test_observability_tracing`). They predate this work and are unrelated,
so they were excluded rather than fixed — happy to open a separate
issue.
- `[vertex]` was chosen over adding the dependency to `[proxy]` after
measuring the footprint at ~175 MB. If maintainers prefer it in
`[proxy]`, that is a one-line change.
## Runtime Rollout Safety

- Rollout-managed feature(s): none; diagnostics annotate already-failing
Vertex responses and the backend dependency is opt-in through the vertex
extra.
- Minimum rollout channel: normal stable release after Vertex
diagnostics, route, and backend-path tests pass.
- Stable/default behavior changed: successful responses and native
passthrough remain unchanged; supported Vertex failures gain an
actionable header/message, and selecting the LiteLLM Vertex backend
without its SDK now fails once at startup.
- Kill switch / disable path: use native Vertex passthrough without the
LiteLLM backend, or human-revert the diagnostic/preflight change.
- Unsafe override required: none.
- Qualification impact: offline diagnostics, native integration, and
mocked LiteLLM Vertex request-shaping tests must remain green.
- Rollback path: human-revert the PR; no persisted state or migration is
involved.
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/2640"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `5d7071f348d2462f666fd7ac1ceb8ac18050169a`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Re-reviewed latest integration head c70c1a497 and current author
responses, including fixed public error hints, log escaping, body/header
consistency, all four backend error sites, and native-route versus
LiteLLM SDK dependency behavior. Clean integration with upstream/main
2d3ee51846. No new PR-specific source
blocker found in reviewed scope; no contributor diagnostic/routing logic
rewritten.

232 Vertex diagnostics/native-route/backend/CLI/release/version/MCP
tests pass with11 skips. Offline lock validation passes (268 packages);
all12 changed Python files pass Ruff check/format, and whitespace
passes. These checks cover synthetic error/transport paths and do not
establish live cloud regional availability or billing behavior. Earlier
author live evidence remains scoped to its stated environment; no fresh
cloud request or native build claimed here. Historical shared
cached-read fixture failure was main-reproduced and is not a
contributor-code blocker. Maintainers handle current exact-head hosted
qualification before approval; nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `486a716adbf642ae5e43276df4c2dc1e63b901d7`, upstream main
`c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`.

Clean mechanical sync with upstream main
c01cf07d94. Re-reviewed current
author/Devin discussion and public diagnostic/error boundaries,
native-versus-SDK route separation and startup preflight. This sync does
not alter the PR Vertex diagnostics, backend or route-registry
modules.158 focused diagnostics/native-route/backend/streaming/CLI tests
pass,11live-environment skips,1deprecation warning. Offline uv lock
--check passes;all12 changed Python files pass Ruff lint/format and
main-relative whitespace passes. Fixed hints remain separate from raw
provider error text; multi-worker preflight and non-Google partner
guidance retain resolved regressions. No contributor diagnostic/routing
correctness logic rewritten. Tests use existing current-main DLL and
synthetic errors/transports; no fresh native build, live regional
availability/provider inference or billing qualification claimed. Formal
approval awaits exact-head hosted qualification; maintainers handle CI.
Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `0b75c609f24562c8b6ed2aaeaf58239984b4810c`, upstream main
`ce4275491299231bc9d2907833f72f72ea22b8a9`.

Requalified after human merges #4667,#4669,#4670 against upstream main
ce42754912. Earlier approval and
publication guards stopped stale candidates before either was posted.
Current clean merge retains Vertex diagnostics/native-versus-SDK routing
and main enable_kompress checks, lossless cold-prefix disable, and
Python token-window boundary behavior.190 combined
Vertex/backend/CLI/Kompress-disable/must-keep tests
pass,11live-environment skips,1warning. Offline lock validation,12-file
Ruff lint/format and main-relative whitespace pass. No contributor
diagnostic/routing logic rewritten. Native Rust/Cargo sources are
identical to c01 across these merges; existing matching-core DLL
supports tests, not a fresh native build. No new live Vertex
regional/provider/billing qualification claimed. All prior-main hosted
results are superseded for readiness; approval awaits exact checks on
this published merge head. Maintainers handle CI; no agent merge.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `1fe62702ce614315679d68d9d126e31c57208c67`, upstream main
`00d4f441d3f1d48c463fea57e6c574c2f3671302`.

Requalified after human merges #4667,#4669,#4670 against upstream main
ce42754912. Earlier approval and
publication guards stopped stale candidates before either was posted.
Current clean merge retains Vertex diagnostics/native-versus-SDK routing
and main enable_kompress checks, lossless cold-prefix disable, and
Python token-window boundary behavior.190 combined
Vertex/backend/CLI/Kompress-disable/must-keep tests
pass,11live-environment skips,1warning. Offline lock validation,12-file
Ruff lint/format and main-relative whitespace pass. No contributor
diagnostic/routing logic rewritten. Native Rust/Cargo sources are
identical to c01 across these merges; existing matching-core DLL
supports tests, not a fresh native build. No new live Vertex
regional/provider/billing qualification claimed. All prior-main hosted
results are superseded for readiness; approval awaits exact checks on
this published merge head. Maintainers handle CI; no agent merge.

Final current-main requalification:
07d5add5ba, including human merges #4671
action pinning and #4648 multimodal memory preservation. Clean
mechanical merge; Vertex/backend/CLI inputs retain the preceding
190-test qualification. Fresh workflow/release suite: 62 passed. Fresh
memory wrapper suite: 9 passed. New wrapper retains list payload parts
and extracts textual search inputs; no contributor Vertex logic
rewritten. Existing native input provenance remains unchanged. Hosted
qualification must run on this final head before approval.

Newer human merge #4646 advanced authoritative main to
8cdce172ea before publication. Cleanly
integrated quote-aware read detection; reviewed new masking/segmentation
overlap. Fresh read-command/content-router suite: 188 passed, 1 optional
SOCKS warning. Vertex code remains unchanged by this merge.

Final additional human merge #4067 packaging changes advanced main to
a8905214d: cleanly integrated explicit torch-extras split, retaining
Vertex optional dependencies. Fresh ONNX/optional-dependency contracts:
12 passed. Offline lock check and whitespace pass. No live clean-install
or Torch backend claim. Earlier publication candidates were stopped
before push by fresh-main guard.

Latest human merge #4042 advances authoritative main to
cde584d7fe. Clean merge preserves optional
LangChain availability handling and CI dependency setup. Fresh
binding/compatibility/startup-log/xAI regression suite: 29 passed. New
main integration re-reviewed; no contributor Vertex behavior rewritten.

Subsequent #4049 main merge 00d4f441d adds graceful context lookup/init
failure handling. Cleanly integrated with Vertex handler diagnostics
intact. Fresh Anthropic pre-upstream/memory native ops/OpenAI Codex
routing/WebSocket lifecycle suite: 192 passed, 1 deprecation warning.
Reviewed exception handling and transport overlap; no live provider
qualification claim.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.


### Maintainer current-main requalification

Exact head `6c63811d6de7491ebce9a16f89e197f5f5dd86dd`, upstream main
`2b0315b2d4b0d54947050bbdad2de9e89df06378`.

Requalified against authoritative
main2b0315b2d4b0d54947050bbdad2de9e89df06378 after further human merges.
All10 integration patches exactly match00d4f441-to-current-main changes
after diff-location normalization; contributor Vertex
diagnostics/routing policy unchanged.539combined
Vertex/backend/CLI/Kompress-disable/must-keep/binary/doctor/Windows-supervisor/provider/search/text
tests pass;13platform/live-environment skips,2warnings. Nine changed
Python files pass Ruff lint/format; offline lock and whitespace pass.
Existing pr3982 native DLL verified against every tracked native
build-input hash and artifact checksum; no fresh native build claimed.
Current author/Devin discussion checked; prior fixes remain intact. No
live Vertex regional/provider/billing,IPv6 listener or unsupported cloud
qualification claimed. No new finding in reviewed scope. Exact-head
hosted checks must qualify this published head before formal approval;
maintainers handle them. Nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-06 12:27:47 -06:00
02c4e922f7 fix(cli): resolve active proxy port for read commands (#3522)
## Description

`headroom wrap copilot` can intentionally start a dedicated proxy on
port 8788
(or fall forward when 8787 is busy), but read-only CLI commands
previously
defaulted directly to 8787. As a result, `headroom dashboard` could open
a dead
or unrelated dashboard even though the wrapped session reported its
proxy on
8788.

This change extends the shared `headroom/cli/port_discovery.py` resolver
(introduced by #3855) to include live wrap-client markers, preferring
the
newest healthy session while preserving explicit `--port` and
`HEADROOM_PORT`
overrides.

Closes: N/A (no matching open issue found)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Extended the shared port discovery module from #3855 with live
wrap-client
marker validation (PID liveness and reuse checks), newest-session
ordering,
  and `/health` verification.
- Updated `dashboard`, `inspect`, and the runtime probe used by `learn`
to
  follow the newest healthy live wrap session after explicit and
  `HEADROOM_PORT` overrides.
- Kept `doctor`'s requested-port behavior; it now reports a healthy
proxy
  discovered from wrap-client markers elsewhere, with a `--port` hint.
- Added a warning when `dashboard` cannot reach the resolved port.
- Exported the actual proxy port as `HEADROOM_PORT` into every wrapped
agent
  environment.
- Reused #3855's port-discovery module rather than introducing a
parallel
  resolver.
- Added regression coverage for stale/recycled markers, multiple
sessions,
  explicit/environment overrides, dashboard output, doctor diagnostics,
  inspect/learn routing, and wrapped child environments.
- Documented automatic discovery and wrong-port troubleshooting.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_cli/test_proxy_port_discovery.py \
    tests/test_cli/test_dashboard_port_discovery.py \
    tests/test_cli/test_port_alignment.py \
    tests/test_cli/test_wrap_helpers.py \
    tests/test_cli_doctor.py tests/test_cli_inspect.py tests/test_cli_learn.py -q
267 passed in 5.49s

$ uv run ruff check .
All checks passed!

$ uv run ruff format --check .
1677 files already formatted

$ uv run mypy headroom/cli/port_discovery.py headroom/cli/proxy.py \
    headroom/cli/doctor.py headroom/cli/inspect.py headroom/cli/learn.py
Success: no issues found in 5 source files

$ uv run pre-commit run --all-files
Sync plugin versions.....................................................Passed
Verify Ruff version alignment............................................Passed
check for merge conflicts................................................Passed
ruff (legacy alias)......................................................Passed
ruff format..............................................................Passed
mypy.....................................................................Passed
```

After the first rebase, `make ci-precheck` completed `cargo fmt --check`
and
`cargo clippy --workspace -- -D warnings`, then stalled in the existing
Rust
Magika/detection tests (multiple tests exceeded 60 seconds). The same
stall
recurred during the push hook; it was stopped after confirming the
affected
suite. The final rebase onto #3883 was followed by the Python/CLI
regression
suite and all repository pre-commit gates above, but not another Rust
test run.

## Real Behavior Proof

- Environment: Ubuntu 22.04.5 LTS x86_64; Python 3.10.12; uv 0.12.10;
Headroom proxy 0.40.0-dev, started on isolated port 18788 with an
isolated live-client marker.
- Exact command / steps: Started an isolated process that wrote its live
PID marker and launched `headroom proxy --port 18788` under a temporary
workspace, with `HEADROOM_PORT` unset; then ran `env -u HEADROOM_PORT -u
HEADROOM_PORT_DISCOVERY
HEADROOM_WORKSPACE_DIR=/tmp/headroom-port-discovery-proof-20260930/workspace
./.venv/bin/headroom dashboard --no-open` and curled
`http://127.0.0.1:18788/dashboard`.

```text
$ env -u HEADROOM_PORT -u HEADROOM_PORT_DISCOVERY \
    HEADROOM_WORKSPACE_DIR=/tmp/headroom-port-discovery-proof-20260930/workspace \
    ./.venv/bin/headroom dashboard --no-open
  Detected live proxy on port 18788
  Dashboard: http://127.0.0.1:18788/dashboard

$ curl --max-time 5 -sS -o /dev/null -w '%{http_code}\n' \
    http://127.0.0.1:18788/dashboard
200

# Proxy health before discovery returned:
service=headroom-proxy status=healthy ready=true
```

- Observed result: With `HEADROOM_PORT` unset, the CLI selected the
marker-associated healthy proxy on 18788, printed the matching dashboard
URL, and the endpoint returned HTTP 200. The earlier manual run with a
real wrapped Copilot session also confirmed discovery on 8788 before
this rebase.
- Not tested: Starting a new interactive `wrap copilot` session after
this rebase (avoided launching an agent); Windows/macOS PID-identity
behavior; multiple simultaneous real proxies; browser UI launch. Unit
tests cover marker ordering, PID reuse, and explicit/environment
overrides.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: `dashboard`, `inspect`, and `learn`
now follow
the newest healthy live wrap proxy when no port override is set;
`doctor`
remains on the requested/default port and reports a live proxy
elsewhere.
- Kill switch / disable path: Pass `--port`, set `HEADROOM_PORT`, or set
  `HEADROOM_PORT_DISCOVERY=0` to bypass automatic discovery.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit; commands return to the fixed 8787
default.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from
  my Conventional Commit PR title

## Screenshots (if applicable)

N/A — terminal output above captures the behavior.

## Additional Notes

Port discovery is integrated into #3855's shared `port_discovery.py`;
candidates
are accepted only when `/health` identifies `service: headroom-proxy`,
so an
unrelated local JSON service cannot be mistaken for a Headroom proxy.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3522"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer current-main requalification

Exact head `e66a6393f83278aa07e9965f5b87fed820603677`, upstream main
`2d3ee51846435ef5b320eea4806af14e602b0d8c`.

Re-reviewed latest integration head 420c2d636 and current author
response, including isolated workspace/port fixtures and live-marker
ordering/pruning regressions. Clean integration with upstream/main
2d3ee51846. Explicit CLI ports and
HEADROOM_PORT retain precedence; discovery bounds candidate probes,
verifies Headroom health identity, prefers newest live wrap markers, and
preserves configured-agent discovery. No new PR-specific correctness
blocker found in scope.

All 436
dashboard/port/discovery/wrap/doctor/inspect/learn/release/version/MCP
tests pass. All 14 changed Python files pass Ruff check/format;
whitespace passes. Previous cached-read freeze-boundary failure was
independently reproduced on main and is not a contributor-code blocker.
Maintainers handle fresh exact-head qualification and distinguish
inherited failures; no contributor CI execution requested. Existing
native extension supported startup, not a fresh native build. No
contributor discovery logic rewritten; nothing merged.


Exact-head hosted checks are being handled by maintainers. No new
finding in the reviewed scope; formal approval awaits exact-head hosted
qualification; nothing merged.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-05 21:10:06 -06:00
662a7f6fd8 fix(memory): keep learned evidence until persistence succeeds (#4229)
## Description

Closes #4224 and #4226.

A full queue or failed save no longer makes the learner forget a pattern
permanently. Observations received while the save is queued or running
are included too. `min_evidence=1` saves the first sighting.

The review caught another case: SQLite can commit before vector indexing
fails or the task is cancelled. Retrying that save now reuses the
committed row rather than inserting a duplicate. If index repair fails
again, the pattern and its save token stay pending until repair
succeeds.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update

## Changes Made

- Keep queued patterns as live evidence aggregates until persistence
completes.
- Retain the complete aggregate and its save-attempt token after
failure, cancellation or restart.
- Reconcile only the missing evidence. A delayed SQLite update from a
cancelled task cannot count the same observations twice or move
`last_seen_at` backwards.
- Add real SQLite regressions and document recovery behavior.

## Testing

- [ ] Unit tests pass (pytest) — complete changed learner file verified;
not the entire Python suite locally.
- [ ] Linting passes (ruff check .) — changed-file checks and normal
commit hooks passed.
- [ ] Type checking passes (mypy headroom) — normal mypy commit hook
passed.
- [x] Manual testing performed

### Test Output

```text
Complete tests/test_memory/test_traffic_learner.py:
230 passed in 16.40s
```

The complete learner file ran with the normal installed pytest plugins;
no test selection or warning filters were changed.

The normal full `make ci-precheck` pre-push gate also passed: Rust
formatting, clippy and workspace tests; native extension build; all 181
critical Python tests; commitlint. Rust tests used the supported
`RUST_TEST_THREADS=1` setting; no tests, timeouts or gates were changed
or bypassed.

## Real Behavior Proof

- Environment: Linux x86-64, Python 3.13.14, locally rebuilt native
extension, actual `LocalBackend` with `embedder_backend="onnx"`, SQLite,
and default cache/model settings. No live LLM provider requests.
- Exact command / steps: pass `No, use uv instead of pip for Python
packages.` through the public `TrafficLearner.on_messages` at threshold
three. Make the first three actual vector-index calls raise, then let
the fourth call the original index method. Add observations between
attempts; inspect SQLite, pending ownership and actual search after each
attempt. Persistence, embeddings, cache and retrieval are real; only the
failure boundary is controlled.

The learner tests also cover a rejected SQLite UPDATE, shutdown
continuing after one queued save fails, and index-refresh failure on
already-confirmed evidence.

- Observed result: all phase assertions passed and the standalone
process exited 0.

| Attempt | Rows | SQLite evidence | Pending | Search evidence |
| --- | ---: | ---: | --- | --- |
| Initial index failure | 1 | 3 | yes | not found |
| First repair failure | 1 | 5 | yes | not found |
| Second repair failure | 1 | 6 | yes | not found |
| Successful repair | 1 | 7 | no | 7 |

The original memory ID and save token survive failed repairs. Only the
successful repair marks the pattern saved. Normal evidence-only update
handling is unchanged.

- Not tested: live providers, model quality, Windows/macOS behavior, or
the entire Python suite. Hosted checks for this revision are not claimed
here.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; existing live learning.
- Minimum rollout channel: unchanged.
- Stable/default behavior changed: failed writes retain evidence;
explicitly configured threshold one works; observations during saves are
no longer lost.
- Kill switch / disable path: `--no-learn` or existing memory
configuration.
- Unsafe override required: no.
- Qualification impact: actual persistence and retrieval verified
locally.
- Rollback path: revert the fix; no schema migration.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have made corresponding changes to the documentation
- [x] I did not edit CHANGELOG.md — release-please owns it

## Additional Notes

AI-assisted; no subagents.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4229"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <tejas@headroomlabs.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 22:39:53 -06:00
d9cee45268 fix: [BUG] DirectMem0Adapter.search_memories calls removed mem0 search API (#4518)
<!-- oss-harness:v2 -->
Fixes #4246.

Focused Python fix for: [BUG] DirectMem0Adapter.search_memories calls
removed mem0 search API

Changed implementation: headroom/memory/backends/direct_mem0.py.
Regression tests: tests/test_memory/test_direct_mem0.py.

Verification (working directory `.`): `.venv/bin/python -m pytest -q -x
-p no:cacheprovider -- /workspace/tests/test_memory/test_direct_mem0.py`
- Patched code: 19 tests, zero failures/errors.
- Original code with the same regression tests: 1 failures, zero
collection/setup errors.
- Base commit: `f10725e6b9940d8bc2fcde950c86f8ac97543cb6`.

AI-assisted contribution: OpenCode using Autoloops wrote this patch. The
automated harness independently ran the tests listed above. No human
review is claimed.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4518"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: nandanadileep <116407139+nandanadileep@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Nandana Dileep <nandanadileep@users.noreply.github.com>
2026-11-04 08:33:51 -06:00
6cc0801219 docs: fix generate_report example arguments (#4435)
## Description

The `generate_report()` example in `docs/content/docs/api-reference.mdx`
and `wiki/api.md` passes `format="html"` and `period="day"`, and says it
produces "HTML/Markdown" reports. Neither keyword exists. Running the
documented snippet verbatim on 0.41.x:

```
TypeError: generate_report() got an unexpected keyword argument 'format'
```

Actual signature (`headroom/reporting/generator.py`):
`generate_report(store_url, output_path="report.html", start_time=None,
end_time=None) -> str`. It writes HTML only and returns the output file
path.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Replace the example with the real arguments (`output_path`,
`start_time`), say it returns the path, drop "Markdown" from the
description. Both files, docs only.
- `store_url` uses the `sqlite:////absolute/path/...` form that
`configuration.mdx` already documents: `sqlite:///headroom.db` resolves
to `/headroom.db` at the filesystem root, so the old example also fails
with `OperationalError: unable to open database file` on a normal
machine.
- `start_time` is timezone-aware (`datetime.now(timezone.utc)`), since
`format_timestamp` treats a naive datetime as already UTC (fixed in
0338644 after review).

## Testing

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

No regression test: the change is documentation text only, no code under
`headroom/` is modified, and there is no docs-snippet test harness in
the repo to extend.

### Test Output

```text
naive (old doc) cutoff: 2026-10-22T10:00:00Z
aware (new doc) cutoff: 2026-10-22T15:00:00Z
```

(`format_timestamp` from this branch, simulating a UTC-5 host at
2026-10-23 10:00 local = 15:00Z.)

## Real Behavior Proof

- Environment: headroom 0.41.x, macOS, Python 3.12.
- Exact command / steps: run the old snippet;
`inspect.signature(headroom.generate_report)`; run the new snippet with
an absolute `sqlite:////...` path.
- Observed result: old snippet raises `TypeError`; signature matches the
one above; new snippet writes `report.html` and returns its path.
- Not tested: Windows; the date filter was checked through
`format_timestamp` only, not an end-to-end report against a seeded
store.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a (docs only)
- Stable/default behavior changed: no
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

---------

Signed-off-by: Anton Dziatkovskii <dzyatkovskiy.a@gmail.com>
Co-authored-by: Anton Dziatkovskii <dzyatkovskiy.a@gmail.com>
Co-authored-by: tonydzi <194927794+tonydzi@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-03 22:51:09 -06:00
8d5dde7765 fix(docker): install the redis extra in every image (#4514)
## Description

The 0.42 images ship the replica-shared compress session store, but no
published variant installs the `redis` client. Setting
`HEADROOM_COMPRESS_SESSION_STORE=redis://...` stopped startup with
`ConfigurationError`, and `HEADROOM_CCR_BACKEND=redis` degraded to the
per-process store. This PR adds the `redis` extra to the Dockerfile
default and to all nine `docker-bake.hcl` targets. It locks the
invariant with a release test. It also documents the session fields of
`/v1/compress` and the shared session store.

Closes #4513

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (a fix or feature that changes existing
functionality)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `Dockerfile`, `docker-bake.hcl`: `HEADROOM_EXTRAS` gains `redis` in
the default and in all nine bake targets.
- `tests/test_release_workflows.py`: new
`test_every_docker_variant_includes_the_redis_extra` guards the
Dockerfile default, the bake targets, and the `pyproject.toml` extra.
- `docs/content/docs/installation.mdx`: new `redis` extras row. The
Docker tag table shows the new extras.
- `docs/content/docs/proxy.mdx`, `wiki/proxy.md`: documented the
`session` response fields, `HEADROOM_COMPRESS_SESSION_STORE`, its
failure behavior, and the `session_affinity` interaction.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_release_workflows.py -q
Pytest: 55 passed

$ uvx ruff@0.17.0 check .
All checks passed!
```

## Real Behavior Proof

- Environment: macOS worktree of this repository, Python 3.13.
- Exact command / steps: `python -m pytest
tests/test_release_workflows.py -q` and `uvx ruff@0.17.0 check .`.
- Observed result: 55 tests passed, ruff clean. The new test fails
against the pre-change `Dockerfile`, because its `ARG HEADROOM_EXTRAS`
line lacked `redis`.
- Not tested: the Docker images were not built. The release test asserts
the build args statically, as the sibling Bedrock test does.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. The change touches build args, docs,
and a test.
- Minimum rollout channel: not applicable.
- Stable/default behavior changed: every published image now installs
the `redis` client. With the extra installed,
`HEADROOM_COMPRESS_SESSION_STORE=redis://...` starts instead of failing,
and `HEADROOM_CCR_BACKEND=redis` uses Redis instead of the per-process
fallback.
- Kill switch / disable path: drop `redis` from `HEADROOM_EXTRAS` at
build time, or unset the two environment variables at runtime.
- Unsafe override required: no.
- Qualification impact: none. No runtime code path changed. The `redis`
import stays lazy, so deployments that do not set the variables are
unaffected.
- Rollback path: revert this PR and rebuild.

## Review Readiness

- [x] I performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I performed a self-review of my code
- [x] I commented my code, particularly in hard-to-understand areas
- [x] I made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I added tests that prove my fix is effective or that my feature
works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`. It is generated by
release-please from my Conventional Commit PR title, and a CI guard
enforces this.

## Screenshots (if applicable)

Not applicable.

## Additional Notes

The full test suite and `mypy headroom` were not run. This PR changes no
file under `headroom/`, so I ran the affected suite
(`tests/test_release_workflows.py`) and lint only. The docs half also
answers part of #3184 (documentation of the storage backends). The CI
half of #3184 stays open, and PR #3282 covers it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4514"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <tejas@headroomlabs.ai>
2026-10-28 11:29:27 -07:00
AlessandroSpallinaandClaude Opus 5.5 a90421083a feat(proxy): report the same tool savings on gateway /v1/compress as the proxy path (#4467)
## Description

Gateway-mode `/v1/compress` now reports the same savings as the proxy
path. The why is in #4466.

Closes #4466

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update

## Changes Made

Two commits, one per gap:

1. **`fix(proxy): count tool-schema compaction ...`**
- The gateway turn counts the tool definitions before and after
compaction.
- The handler adds both counts to `tokens_before` / `tokens_after`, with
the proxy path's guard (`0 < after < before`).
   - Tools the client already marked `defer_loading` are skipped.
2. **`feat(proxy): report tool_tokens_saved and tool_tokens_deferred
...`**
- Gateway-mode answers add `tool_tokens_saved` (from the tags `/stats`
sums) and `tool_tokens_deferred` (the estimated share).
   - Docs: `proxy.mdx`, `wiki/proxy.md`.

Non-gateway and fail-open answers do not change.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`, on the changed files)
- [x] Type checking passes (`mypy`, on the changed files)
- [x] New tests added for new functionality

```text
$ pytest -q tests/gateway/ tests/test_gateway_turn_unit.py tests/test_proxy_compress_endpoint.py
380 passed, 5 skipped
```

The tests also pass on the first commit alone (`tests/gateway/`: 280
passed). The new test for client-deferred tools fails without its fix.

## Real Behavior Proof

- Environment: macOS, Python 3.12. Before: image
`headroom:0.41.0-nonroot`. After: this branch, `HEADROOM_OFFLINE=1
headroom proxy`.
- Exact command / steps: start each build, then send it the same
gateway-mode `POST /v1/compress` — one message plus 5 tools whose
schemas carry `title` / `examples` / `default` (the annotation keys
schema compaction strips), with `"gateway": {"can_redrive": false,
"can_relay_response": false}` so the request is a single call. Read
`tokens_before`, `tokens_after`, `tokens_saved` and the `tool_*` fields
out of the JSON answer. No provider is contacted.
- Observed result: same request, same tools. On 0.41.0 the tool
definitions are invisible to the counts, so `tokens_saved` is 0 and
neither `tool_*` field is present. On this branch the compaction is
counted, so `tokens_before` / `tokens_after` carry the tool bytes and
`tokens_saved` is the 240 tokens compaction stripped
(`tool_tokens_saved` / `tool_tokens_deferred` stay 0 here: no
tool-search extension ran, and the extension is what books deferral).

```text
0.41.0:  tokens_before 13,  tokens_after 13,  tokens_saved 0    (no tool_* fields)
branch:  tokens_before 830, tokens_after 590, tokens_saved 240, tool_tokens_saved 0, tool_tokens_deferred 0
```

- Not tested: a live run with the headroom-tool-search extension
(deferral is covered by unit tests), and Responses-API bodies.

## Runtime Rollout Safety

- Rollout-managed feature(s): None gate this change — the gateway
`/v1/compress` accounting is not behind a registry feature. One
interaction: `proxy_output_shaper` (registry feature, available
`stable`, deliberately not default-enabled) consumes the augmented
count. `tokens_before` is folded at
`headroom/proxy/handlers/openai.py:11064` and then handed to the turn at
`:11083`, which passes it as `input_tokens` to `shape_gateway_body`
(`headroom/proxy/gateway_extension.py:213`) → `stratum_key` →
`input_bucket`. A turn whose tool definitions push it across one of the
2k / 8k / 32k / 128k boundaries therefore lands in a bigger stratum,
which picks a different output-savings baseline. The chat path already
has this coupling.
- Minimum rollout channel: n/a — no channel gate, so the change takes
effect wherever gateway mode is served, on every channel.
- Stable/default behavior changed: Yes, gateway-mode answers only.
`tokens_before` / `tokens_after` now include the tool definitions before
and after schema compaction, so `tokens_saved` covers the compaction
saving the same way the chat handlers already do, and two fields are
added. Non-gateway and fail-open answers are byte-identical.
- Kill switch / disable path: None. Without compaction there is nothing
to fold, and the two fields are additive on answers; a gateway that does
not want them ignores them.
- Unsafe override required: No. Nothing crosses a channel boundary.
- Qualification impact: None. No registry entry is added, so
`registry_digest` and `snapshot_digest` are unchanged. The gateway
`tokens_saved` figure itself changes, which a benchmark comparing arms
across this commit will see.
- Rollback path: Revert the two commits.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] I have made corresponding changes to the documentation
- [x] I have added tests that my fix is effective or that my feature
works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

- The fields are emitted for any claimed compress turn. Only the gateway
extension claims turns today.
- If a response-half hook rewrote the tool tags during a re-drive,
`/stats` would no longer match the answer. Core has no such hook.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 08:45:53 -07:00
71663b44e2 fix(learn): pin the learned verbosity level on a cache-mode proxy (#4033)
## Description

`headroom learn --verbosity --apply` hot-enables the output shaper on a
running proxy and prints `level N is live now`. On a proxy in **cache
mode**, which is the default, that claim is false. The proxy never reads
`verbosity.json` there: `resolve_verbosity_level` returns the startup
level (`cache_mode_default`, L2) unless `HEADROOM_VERBOSITY_LEVEL` pins
one. A level that can change mid-conversation would bust the prefix
cache (#3773). So after `--apply` the shaper is on, but at L2 instead of
the learned level, while the CLI reports the learned level as live.

This PR makes `--apply` either do what it says or say what it did:

- **Cache mode, no pin:** the learned level is pinned through
`HEADROOM_VERBOSITY_LEVEL` in the same `/admin/runtime-env` update. That
is exactly the stable-pin path #3773 allows in cache mode.
- **An existing `HEADROOM_VERBOSITY_LEVEL`:** left alone, and the CLI
says which level stays in effect. The one exception is a pin this CLI
provably set. When the proxy confirms a pin (its `applied` map),
`--apply` records `{port, pid, started_at, level}` in
`verbosity_pin.json`. `started_at` is `timestamp - uptime_seconds` from
`/health`, because pids are reused. A later `--apply` replaces the pin
only while that same proxy process still holds that exact value. Hot
overrides die with the process, so a restarted proxy's pin, a reused
pid, or a value changed since stays the operator's.
- **Mode unreadable:** the shaper is still enabled, but the CLI no
longer claims the level is live.
- **Token mode:** unchanged. The proxy reads `verbosity.json` per
request, so no pin is added (a pin would also shadow the verbosity
autotune controller).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/cli/learn.py`:
- New `_query_proxy_verbosity()` reads `config.mode` and
`config.runtime_env.HEADROOM_VERBOSITY_LEVEL` from the local proxy's
`/health`. The proxy includes that config block for loopback callers.
The level is read the way `OutputShaperSettings.from_env` reads it
(parse, default on garbage, clamp to 0–4), so `9` or `04` compare and
print as the level the proxy steers at.
- `_proxy_process()`, `_record_pin()` and `_pin_is_ours()` keep the
ownership record described above. Recording is best-effort: an `OSError`
no longer aborts `--apply`; the CLI reports the activation and notes the
pin wasn't recorded. The record is written with `newline="\n"`.
- `_activate_output_shaper()` returns `(status, port, applied)`, so a
pin the proxy applied while reporting the shaper "blocked" is still
recorded.
- `_activate_output_shaper()` takes an optional `verbosity_level` and
sends it in the same runtime-env request.
- The `--apply` messages distinguish the four cases above. The restart
hints now include `HEADROOM_VERBOSITY_LEVEL=<level>` for cache mode.
- The "learned level is used automatically" wording on the blocked and
no-proxy paths no longer claims cache mode reads the level.
- Those hints no longer prescribe `HEADROOM_ROLLOUT_CHANNEL=beta`:
`proxy_output_shaper` is available on every channel, and beta would also
enable unrelated features. "Blocked" now points at
`HEADROOM_DISABLE_FEATURES`, the only thing that refuses the hot enable.
- `tests/test_cli_learn.py`:
- CLI tests run `learn --verbosity --apply` against a stubbed local
proxy: cache-mode pin, token mode, an operator pin, an unreadable mode,
recording and then replacing this CLI's own pin, and keeping pins it
cannot prove it set. That last case covers an operator pin equal to the
previously learned level, a record from a restarted proxy, and a value
changed since. Also covered: out-of-range pins, re-pinning the same
level, and a proxy that disables the shaper. Each asserts both the
runtime-env body sent and the message printed.
- Further cases: a reused pid, a record without a start time, a pin
applied while the shaper is blocked, a failed record write, and a proxy
without a pid.
- Fallback branches: a non-numeric pin (read as the default level), a
`/health` reply without a config object, no usable proxy start time
(zero or missing uptime, missing or bad timestamp), and a record that
isn't a JSON object. Every new line and branch in
`headroom/cli/learn.py` is covered (`pytest --cov=headroom.cli.learn
--cov-branch`).
- A `windows_newline` test asserts that the ownership record is written
with `newline="\n"`.
- `tests/test_offline_egress_chokepoint.py`: `cli/learn.py` now has two
loopback egress sites (`/health` and `/admin/runtime-env`), recorded in
the allowlist with the same wording as `cli/wrap.py`.
- The existing `--all --apply` test made a real POST to whatever
listened on `127.0.0.1:8787`, which on a developer machine is their own
proxy. It now runs against a stubbed `urlopen`.
- `README.md`, `wiki/learn.md`: state that cache mode does not read the
learned level, and what `--apply` does there. `wiki/learn.md` no longer
says the shaper needs the beta channel.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# Before: origin/main's headroom/cli/learn.py with the new tests
$ pytest tests/test_cli_learn.py -k verbosity_apply -q
FAILED tests/test_cli_learn.py::test_verbosity_apply_pins_the_level_on_a_cache_mode_proxy
FAILED tests/test_cli_learn.py::test_verbosity_apply_leaves_a_token_mode_proxy_reading_the_profile
FAILED tests/test_cli_learn.py::test_verbosity_apply_does_not_override_an_explicit_level
FAILED tests/test_cli_learn.py::test_verbosity_apply_does_not_claim_live_when_the_mode_is_unknown
4 failed, 22 deselected in 1.04s

# Review round 3: the previous head (e10141c3) with this round's tests
$ pytest tests/test_cli_learn.py -q
FAILED tests/test_cli_learn.py::test_activate_output_shaper_reports_effective_rollout_decision[True-live]
FAILED tests/test_cli_learn.py::test_activate_output_shaper_reports_effective_rollout_decision[False-blocked]
FAILED tests/test_cli_learn.py::test_activate_output_shaper_handles_malformed_response
FAILED tests/test_cli_learn.py::test_verbosity_apply_records_the_pin_it_sets
FAILED tests/test_cli_learn.py::test_verbosity_apply_replaces_the_pin_an_earlier_apply_set
FAILED tests/test_cli_learn.py::test_verbosity_apply_keeps_a_pin_it_cannot_prove_it_set[no-start-time]
FAILED tests/test_cli_learn.py::test_verbosity_pin_record_is_written_with_lf
FAILED tests/test_cli_learn.py::test_verbosity_apply_records_a_pin_the_proxy_applied_while_the_shaper_is_disabled
FAILED tests/test_cli_learn.py::test_verbosity_apply_reports_activation_when_the_pin_cannot_be_recorded
FAILED tests/test_cli_learn.py::test_verbosity_apply_does_not_record_a_pin_without_a_process_identity
10 failed, 31 passed in 0.74s

# After: this branch
$ pytest tests/test_cli_learn.py -q
50 passed in 0.71s

# Offline-egress coverage (failed on e10141c3 in CI shard 3)
$ pytest tests/test_offline_egress_chokepoint.py -k test_every_egress_site_is_guarded_or_allowlisted -q
1 passed, 142 deselected in 2.63s

# Related suites (learn, verbosity, output shaper, runtime-env), with branch coverage of the changed file
$ pytest tests/test_*learn*.py tests/test_*verbosity*.py tests/test_*output_shaper*.py tests/test_*runtime_env*.py -q --cov=headroom.cli.learn --cov-branch
207 passed, 3 warnings in 7.83s
# lines/branches added by this PR in headroom/cli/learn.py that are not covered: none

$ ruff check headroom/cli/learn.py tests/test_cli_learn.py
All checks passed!
$ ruff format --check headroom/cli/learn.py tests/test_cli_learn.py
2 files already formatted
```

## Real Behavior Proof

- Environment: macOS (Apple Silicon), Python 3.11. A real `headroom
proxy --port 8799` in default cache mode from this checkout, with an
isolated `HEADROOM_WORKSPACE_DIR` and `--anthropic-api-url` pointed at a
closed local port, so no request left the machine. `headroom learn
--verbosity --apply` ran over real Claude Code transcripts and learned
level 3. Before = `origin/main`'s `headroom/cli/learn.py`; after = this
branch. A fresh proxy was used for each run.
- Exact command / steps: Start the proxy. Run `HEADROOM_PORT=8799
headroom learn --verbosity --apply --project <repo>`. Send one
`/v1/messages` request through the proxy. Read `/health`
`config.runtime_env` and the proxy's `OutputShaper(...)` log line.
- Observed result: before, the CLI says level 3 is live while the proxy
steers at L2. After, the level is pinned and the proxy steers at L3:

  ```text
  ===== BEFORE (origin/main learn.py), proxy in default cache mode
  >> Recommended verbosity level: 3 (confidence: high)
✓ Output shaper enabled on the running proxy (port 8799); level 3 is
live now (while HEADROOM_VERBOSITY_LEVEL is unset).
To keep it on across restarts: export HEADROOM_ROLLOUT_CHANNEL=beta and
HEADROOM_OUTPUT_SHAPER=1 before `headroom wrap ...`.
proxy /health: {"mode": "cache", "HEADROOM_OUTPUT_SHAPER": "1",
"HEADROOM_VERBOSITY_LEVEL": null}
  proxy log: OutputShaper(L2/cache_mode_default)

  ===== AFTER (this branch), proxy in default cache mode
  >> Recommended verbosity level: 3 (confidence: high)
✓ Output shaper enabled on the running proxy (port 8799); level 3 is
live now (pinned with HEADROOM_VERBOSITY_LEVEL: cache mode does not read
the learned level).
Conversations already open re-cache their prompt once, because the
steering text changed.
To keep it on across restarts: export HEADROOM_OUTPUT_SHAPER=1
HEADROOM_VERBOSITY_LEVEL=3 before `headroom wrap ...` or `headroom
proxy`.
  --- verbosity_pin.json: {"port": 8799, "pid": 25345, "level": 3}
  --- second --apply against the same proxy:
✓ Output shaper enabled on the running proxy (port 8799); level 3 is
live now (pinned with HEADROOM_VERBOSITY_LEVEL: cache mode does not read
the learned level).
proxy /health: {"mode": "cache", "HEADROOM_OUTPUT_SHAPER": "1",
"HEADROOM_VERBOSITY_LEVEL": "3"}
  proxy log: OutputShaper(L3/env_pinned)
  ```
- Not tested: a real token-mode proxy, one with an operator-set
`HEADROOM_VERBOSITY_LEVEL`, and a second `--apply` that learns a
*different* level (all covered by the CLI tests against a stubbed proxy;
the real run above re-learns the same level 3); multi-worker proxies,
which reject `/admin/runtime-env` before and after this change; Windows
(the record write is covered by a `windows_newline` test).

## Runtime Rollout Safety

- Rollout-managed feature(s): `proxy_output_shaper` (unchanged). This PR
only changes what the `headroom learn` CLI sends to and prints about a
running proxy.
- Minimum rollout channel: stable.
- Stable/default behavior changed: only `learn --verbosity --apply`
against a running cache-mode proxy with no `HEADROOM_VERBOSITY_LEVEL`.
It now also pins that variable to the learned level, which is what the
CLI already claimed. The proxy itself is unchanged.
- Kill switch / disable path: run `learn --verbosity` without `--apply`;
or POST `{"HEADROOM_VERBOSITY_LEVEL": ""}` / restart the proxy to drop
the hot override.
- Unsafe override required: no.
- Qualification impact: none expected.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

- Type checking was not run locally.
- Pin ownership: `/health` cannot say whether `HEADROOM_VERBOSITY_LEVEL`
came from startup env or a hot override, so `--apply` keeps its own
record. The process start time is derived from `/health` rather than
adding an instance id to the proxy, which keeps this PR CLI-only. The
one remaining ambiguity is an operator hot-POSTing the *same* value to
the *same* proxy process after `--apply` pinned it. That pin would be
treated as ours, but its value is the one `--apply` set anyway.
- Locally, 5 OTLP/Langfuse tests in `test_offline_egress_chokepoint.py`
fail on both `origin/main` and this branch because the dev venv lacks
`opentelemetry-exporter`; they pass in CI.


## Maintainer follow-up at 8c3eba8919

Ownership now compares the exact raw `HEADROOM_VERBOSITY_LEVEL` value
from the same health snapshot as process identity. A changed operator
value is preserved even when it clamps to the same effective level as a
previous CLI pin (for example, CLI4 followed by operator8). The display
still reports the effective clamped level. The new regression fails on
the preceding head and passes here; existing CLI ownership/restart/error
behavior remains covered.

Fresh validation:140 CLI/verbosity/output-shaper/runtime-env tests pass
with one existing warning; Ruff lint/format and Linux-platform mypy
over566files pass. Actual proxy/CLI proof additionally covers learning3,
re-learning1, preserving an operator8 pin, and the
changed-raw/same-clamped-level ownership case. The last case fails
before this follow-up and passes afterward.

Exact commands and full runnable Windows/Python3.12 script: [maintainer
evidence](https://github.com/headroomlabs-ai/headroom/pull/4033#issuecomment-6063120923).
Save the script as `verify_learn_pin.py` in the checkout root and run
`python verify_learn_pin.py` with proxy dependencies installed.
Synthetic Claude transcripts are processed by the real analyzer; no live
LLM judge or provider request is made. Hosted CI and fresh review must
qualify this new head before approval.

---------

Co-authored-by: Kenneth Wong <17061293+KennethWKZ@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-08 10:48:59 -05:00
ee731d7ea8 fix(proxy): refuse token-less non-loopback binds, gate operator routes (#3852)
## Why

Four related gaps in the proxy's network gate (findings 01-F1, 01-F2,
01-F5, 01-F9):

- **Open bind.** A non-loopback bind with no `HEADROOM_PROXY_TOKEN` only
logged a warning, then served the `/v1/*` relay, with the operator's
upstream credentials, to every peer that could reach the port.
- **Missing peer address.** `is_loopback_host(None)` returned `True`. A
Unix-domain socket, an ASGI adapter that leaves `scope["client"]` unset,
or a hand-built request therefore turned off the token gate, the
`/admin` and `/debug` guards and the WebSocket gate.
- **Operator data routes.** `/stats-history` (including CSV export),
`/quota`, `/subscription-window` and `/metrics` were served to any
network caller.
- **WebSocket loopback exemption** checked only the peer IP, not the
`Host` header, so it had no DNS-rebinding defence.

## What changes

- New `headroom/proxy/bind_policy.py`. A non-loopback bind (`0.0.0.0`,
`::`, a LAN address or a hostname) with no token is refused at three
places: `create_app` (embedders and the multi-worker factory),
`run_server` (one exit with code 2 before uvicorn forks) and the
`headroom proxy` CLI (a clear error before the banner, exit code 1). An
absent host means uvicorn's default `127.0.0.1`.
- `HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` acknowledges an open bind when
the runtime already restricts reachability, such as a container
published on `127.0.0.1`. The `proxy_open_bind` warning is still logged
on that path, and the banner says the bind is open.
- `is_loopback_host(None)` now returns `False` (fail closed), matching
`identity.py`.
- `/stats-history`, `/quota` and `/subscription-window` answer only: a
non-loopback caller that passed the token gate, a loopback caller (peer
and `Host` header both loopback), a trusted dashboard client, or, under
an acknowledged token-less bind, the container's own default-gateway TCP
peer with a loopback `Host` (the host browser reaching a
`127.0.0.1`-published container). Everyone else gets 404, as with
`/admin`. Settings writes keep their stricter rule with no token
short-cut.
- `/metrics` answers a non-loopback caller that passed the token gate, a
loopback caller, a connecting peer in
`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` (whatever it forwards), or a
resolved client in the dashboard CIDRs. For CIDR peers a hostname `Host`
is fine, so Prometheus scrapers work; a cross-origin browser request is
still refused.
- The token short-cut applies only to non-loopback requests. The gate
exempts loopback peers without checking `Host`, so a loopback request
has not proven it holds the token and must pass the loopback `Host`
check. This blocks a DNS-rebound browser on the operator's machine.
- The `/dashboard` shell stays ungated. It is a static template, like
the already public `/dashboard/static` assets. Gating it would break the
dashboard for containers published on host loopback, where the peer is
the bridge gateway and `Host` is `localhost`. The data routes behind it
are gated.
- The WebSocket loopback exemption requires a loopback peer and a
loopback `Host` header. A loopback peer with a foreign `Host` can still
connect with the token.
- Launchers that always publish on `127.0.0.1` set the acknowledgement
themselves: `headroom install` (docker preset), `scripts/install.sh` and
`scripts/install.ps1` (through one helper that adds the acknowledgement
only together with the `127.0.0.1` publication), and
`docker/docker-compose.native.yml`.
- Docs: `proxy.mdx` documents the new variable and which routes the
token also gates; `docker-install` (docs and wiki) and the
`docker-compose.yml` comment show the acknowledgement in the
loopback-published example. `e2e/docker-bind-security.sh` sets it and
checks that the same launch without it refuses to start.

## Behaviour change (breaking)

- `headroom proxy --host 0.0.0.0`, or `HEADROOM_HOST` set to any
non-loopback address, without `HEADROOM_PROXY_TOKEN` now refuses to
start. The error names both remedies. Set a token, bind loopback, or set
`HEADROOM_ALLOW_UNAUTHENTICATED_BIND=1` when the port is already
confined. Anyone running the image by hand with `--host 0.0.0.0` and no
token must add one of these.
- Without a token, `/stats-history`, `/quota`, `/subscription-window`
and `/metrics` return 404 to non-loopback callers outside a trusted
CIDR. Prometheus scrapers on another host must be in
`HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS` or send the token.
- Code that relied on a request with no peer address being treated as
loopback now gets the non-loopback path.
- The default loopback run is unchanged.

## Tests

- `tests/test_proxy_bind_policy.py`: the refuse/allow matrix
(non-loopback, loopback, unset host, token, acknowledgement values), the
warning on the acknowledged path, `create_app`, the `run_server` exit
before uvicorn, and the CLI (refusal, `HEADROOM_HOST` from env,
acknowledged start, offline banner, start with a token).
- `tests/test_proxy_loopback_gating.py`: each data route against
non-loopback, loopback, trusted-CIDR and token callers; the dashboard
shell stays reachable; the host of an acknowledged, loopback-published
container reads the data routes while another bridge peer, a forwarded
gateway address and a foreign `Host` do not; CSV export is not served to
network callers; `/metrics` across loopback, both CIDR lists,
out-of-range peers, cross-origin browsers and token callers, including a
trusted gateway relaying a scraper outside its own range. A regression
test shows that a token on the proxy does not exempt a loopback peer
from the `Host` check; it fails without the fix.
- `tests/test_proxy_hardening.py`: the HTTP gate with no peer address,
WebSocket `Host` cases, `require_loopback` and `/debug` with no peer.
- Installer tests: `headroom install` adds the acknowledgement, and the
shell and PowerShell wrappers add it only together with the loopback
publication. Launcher-level tests start the proxy with the environment
and `--host` that `build_runtime_command()` and
`docker-compose.native.yml` produce and check that the host browser
(bridge-gateway peer) gets `/stats-history` and another container does
not.
- Tests that bound `0.0.0.0` for unrelated reasons now set a token or
the acknowledgement. Tests that read operator routes use an explicit
loopback peer.

On `main`, 14 of the gating and hardening tests fail and the bind-policy
module does not exist. Manual runs: token-less `--host 0.0.0.0` exits
with the refusal from both the CLI and `run_server`; with the
acknowledgement, the data routes are 404 to a LAN peer and to a loopback
peer with a foreign `Host`; with a token, a LAN peer gets 401 without it
and 200 with it; the default loopback bind starts as before.

Not tested: a real Unix-domain-socket deployment and a real Prometheus
scrape (both covered by tests only).

## Merge order

Part of a set. Suggested order: #3852 → #3891 → #3849 → #3860 → #3862 →
#3863. This PR goes first: it is the gate the others assume, and it is
the release-note event.

## Not in this PR

- Enforcing the token for loopback callers when one is configured.

Revised 2026-10-02 after review: `/metrics` matches gateway CIDRs
against the TCP peer, and the operator data routes serve the host of a
loopback-published container (bridge-gateway peer under the launchers'
bind acknowledgement) so the default Docker dashboard works.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3852"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-05 08:27:35 -05:00
TrîggeredandJD Davis 4ba231ab3d feat(learn): add agy (Antigravity CLI) as an analysis backend (#3939)
## Description

Adds the Antigravity CLI (`agy`) as a keyless LLM backend for `headroom
learn`, next to `claude`, `gemini` and `codex`. Antigravity subscription
users with no raw API key can now run the analysis through `agy`.

`agy` cannot use the existing stdin path as is. `agy -p` takes the
prompt as a flag argument and never reads stdin, so a large digest would
hit OS argument-length limits. The only stdin path is `--input-format
stream-json`, which requires `--output-format stream-json`. The backend
therefore sends the prompt as a single NDJSON user event on stdin and
reads the answer from `result.response` in the final `result` event.
Everything else (wall-clock timeout, Windows shim fallback, failure
detail, fenced-JSON parsing) reuses the existing `run()` path that
`gemini` and `codex` use.

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/learn/analyzer.py`: add an `agy` / `agy-cli` entry to
`_CLI_BACKENDS` (`agy --input-format stream-json --output-format
stream-json`). `agy` is never auto-detected, and it only runs when
`HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1` is set: `agy` has no zero-tool mode,
so transcript text could drive its tools. Without the opt-in,
`HEADROOM_LEARN_CLI=agy` and `--model agy-cli` fail before any
subprocess starts; with it, a warning is logged on every run.
- `headroom/learn/analyzer.py`: in `_call_cli_llm`, wrap the prompt as
`{"event": "user", "message": {"content": ...}}` for `agy-cli`, and
extract `result.response` from the stream with a new
`_agy_result_text()` helper before the existing `_strip_fenced_json()`
parsing. A missing `result` event surfaces as the existing "unparseable
output" error, with both ends of stdout.
- `tests/test_learn/test_analyzer.py`: tests that auto-detection never
selects `agy` (even when it is the only CLI on PATH and the opt-in is
set), that `HEADROOM_LEARN_CLI=agy` and `_call_cli_llm(..., "agy-cli")`
fail without the opt-in (no subprocess call), `HEADROOM_LEARN_CLI=agy`
with the opt-in, existing CLIs still taking precedence, the exact
command and stdin payload, result-event parsing (including a fenced
response and a literal U+2028), and a missing `result` event.
- `headroom/cli/learn.py`: check the agy opt-in during model resolution,
so `HEADROOM_LEARN_CLI=agy` and `--model agy-cli` without it exit 1 with
an `Error:` line before any sessions are scanned.
- `tests/test_cli_learn.py`: a CLI test for both paths without the
opt-in.
- `wiki/learn.md`: list `agy` among valid `HEADROOM_LEARN_CLI` values
and document the `HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1` opt-in and its
risk.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run --frozen --extra dev pytest tests/test_learn/ tests/test_cli_learn.py -q
5 failed, 345 passed, 4 skipped in 10.78s
# The same 5 fail on unmodified upstream/main on this Windows machine (5 failed, 340 passed),
# all path/git related and unrelated to this change:
#   test_analyzer.py::TestDigestBuilder::test_includes_project_info
#   test_scanner.py::TestDecodeProjectPath::test_home_dir_username_stays_single_component
#   test_writer.py::TestClaudeLocalMdStaysOutOfGit::{test_apply_adds_exclude_entry,
#     test_second_run_does_not_duplicate_the_entry, test_tracked_file_warns_instead_of_excluding}

$ uv run --frozen --extra dev pytest tests/test_learn/test_analyzer.py -q -k agy   # analyzer.py reverted
FAILED TestDetectDefaultModel::test_env_var_selects_agy_with_unsafe_opt_in
FAILED TestDetectDefaultModel::test_env_var_agy_without_unsafe_opt_in_raises
FAILED TestCallCliLlm::test_agy_cli_sends_stream_json_and_parses_result_event
FAILED TestCallCliLlm::test_agy_cli_result_with_unicode_line_separator
FAILED TestCallCliLlm::test_agy_cli_without_unsafe_opt_in_does_not_run
FAILED TestCallCliLlm::test_agy_cli_missing_result_event_raises
6 failed, 2 passed, 97 deselected
# All 8 pass with the change.

$ uvx ruff check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
All checks passed!
$ uvx ruff format --check headroom/learn/analyzer.py tests/test_learn/test_analyzer.py
2 files already formatted
$ uv run --frozen --extra dev mypy headroom/learn/analyzer.py
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: Windows 11, `agy` 1.2.14 signed in via subscription, no
API keys set.
- Exact command / steps: `HEADROOM_LEARN_CLI=agy
HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1 ANTHROPIC_API_KEY= OPENAI_API_KEY=
GEMINI_API_KEY= uv run --frozen --extra dev headroom learn` in this
repository (dry run).
- Observed result: `Analyzing with agy-cli...`, then `Sessions: 3 |
Calls: 109 | Failures: 19 (17.4%)`, `Recommendations: 8`, with `[WOULD
WRITE]` blocks for `CLAUDE.local.md` and `MEMORY.md`; exit 0. Before
wiring the backend I confirmed the protocol by hand: `agy -p -` answers
the literal prompt `-` and ignores stdin, while the stream-json
invocation returns the answer in `{"event": "result", "result":
{"status": "SUCCESS", "response": "..."}}`.
- Not tested: macOS/Linux `agy` installs; `agy` under a permission mode
that prompts for tool approval (my config is `always-proceed`), where a
tool request in print mode would be bounded by the existing
`HEADROOM_LEARN_CLI_TIMEOUT_SECS` wall-clock cap.

## Runtime Rollout Safety

- Rollout-managed feature(s): None (offline `headroom learn` CLI)
- Minimum rollout channel: dev
- Stable/default behavior changed: No. `agy` is never auto-detected; it
is only used when selected with `HEADROOM_LEARN_CLI=agy` / `--model
agy-cli` together with `HEADROOM_LEARN_ALLOW_UNSAFE_AGY=1`.
- Kill switch / disable path: Set an API key, `HEADROOM_LEARN_CLI`, or
`--model` to pick another backend.
- Unsafe override required: None
- Qualification impact: None
- Rollback path: Revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A (CLI change).

## Additional Notes

- No tracking issue exists for an `agy` analysis backend, so `Closes #`
is left empty. Related Antigravity work (#3428 `headroom init -g agy`,
#3610 Antigravity transcripts in `headroom learn`) is independent of
this change.
- Unlike `claude-cli` (#3892), `agy` has no flag to start a session with
zero tools. I deliberately did not add `--dangerously-skip-permissions`:
the digest contains untrusted transcript content, and the backend should
run under the user's own permission settings.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/3939"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-04 13:56:26 -07:00
Tejas ChopraandJerrettDavis 95ae69a85f docs(configuration): document LiteLLM as the source for limits and prices (#2781)
## Description

The ten token/pricing fixes merged this session changed documented
behaviour, and `configuration.mdx` / `wiki/configuration.md` were left
describing the old model. This is docs-only.

**The main problem:** both trees documented resolution as

```
1. Built-in defaults
2. models.json
3. HEADROOM_MODEL_LIMITS
4. SDK constructor arguments
```

with **no mention of LiteLLM at all**. That is misleading in a way that
costs users real work: it reads as though editing `models.json` is the
only way to correct a wrong limit or price. In fact public models
resolve from LiteLLM and stay current on their own — pinning one in
config means signing up to maintain a number by hand forever.

Restated as an explicit first-match-wins list covering **both** limits
and pricing:

```
1. explicit config (models.json / env / SDK args)   <- a decision, beats everything
2. LiteLLM                                          <- live source for public models
3. built-in tables -> pattern -> generic default    <- fallback only
```

…and noted that step 3 exists for installs where LiteLLM is unavailable
(the dependency is gated `python_version < '3.14'`) and for models
LiteLLM doesn't know.

## Changes Made

- **Fixed the example overrides.** Both trees used `"gpt-5": 256000` and
`"gpt-5": [5.00, 15.00]`. Both numbers are wrong (272K input;
$1.25/$10.00), and gpt-5 is now a *known* model — so the example was
teaching readers to override a correct, self-updating value with a stale
hand-written one. Swapped for `ft:gpt-4o:my-org` and
`my-private-deployment`, which are what custom configuration is actually
for, plus a callout saying so.
- **Documented that limits are an INPUT budget.** They come from
LiteLLM's `max_input_tokens`, so `gpt-5` is 272K of prompt room, not
400K — the total window is 272K in + 128K out. I conflated these myself
while writing #2762 and had to correct it, so it's worth stating for
anyone configuring a limit.
- **Updated pattern inference** with `gpt-4.1*`, `gpt-5*`, `o4*`, and
stated that matching prefers the **most specific** prefix — so `gpt-4.1`
isn't treated as `gpt-4` (which is exactly the bug #2762 fixed: an 8K
limit and legacy `gpt-4` pricing).

Kept in sync across both doc trees, since they carried identical stale
text.

## Type of Change

- [x] Documentation update

## Checked and deliberately left alone

- `savings.mdx:50` — already correct: it says cost-avoided uses LiteLLM
list pricing for proxy traffic. That path (`savings_ledger`) was always
LiteLLM-backed; it was the *provider* tables that weren't.
- `opencode.mdx:62` — "GPT-4.1, 1M context, 32K output" is right. Worth
noting the docs had the correct figure all along while the code resolved
8192.
- `wiki/metrics.md` — uses `tokens_input_before/after`, a different
surface from the beacon's `tokens.input` that #2756 touched. No change
needed.

## Testing

- [x] Documentation update

No code changed, so no unit tests. `<Callout type="info">` matches
existing un-imported usage in `savings.mdx`, `ccr.mdx` and
`benchmarks.mdx`; the docs workflow runs `validate-mkdocs` and
`validate-nextjs` (`npm run build`) on any `docs/**` / `wiki/**` PR,
which is the real check for both trees here.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My changes are documentation-only
- [x] I have kept both documentation trees in sync
- [x] I did **not** edit `CHANGELOG.md`

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-10-02 11:06:03 -05:00
eaa16d9f63 fix(proxy): preserve Windows service when Rust core is blocked (#2989)
## Description

When `headroom._core` cannot load, `HEADROOM_REQUIRE_RUST_CORE=false`
should keep the Python proxy available in unoptimized passthrough mode.
A module-level native import made that escape hatch unreachable before
startup could read the setting. This addresses the degraded-service
portion of #2918 and advances #1466; neither issue is closed because
Authenticode/Smart App Control reputation work remains.

## Type of Change

- [x] Bug fix (non-breaking change)
- [x] Documentation update

## Changes Made

- Resolve Rust-backed error-detection registries and patterns lazily
while retaining Rust as the sole source of truth.
- Force `optimize=false` only when the operator explicitly allows a
missing Rust core, keeping traffic available and unmodified.
- Preserve the default fail-loud startup behavior.
- Add Windows Smart App Control, WDAC, antivirus, and Code Integrity
diagnostics.
- Add cold-interpreter coverage for the import boundary and degraded
startup behavior.

## Testing

- [x] Focused tests pass
- [x] Ruff check and formatting pass
- [x] `git diff --check` passes
- [x] New regression tests added

```text
uv run --extra dev pytest -q tests/test_error_detection.py tests/test_rust_core_smoke.py
16 passed in 1.64s (independent exact-head review run)

Additional author verification: 38 focused proxy/core/error-detection/parity/MCP contract tests passed.
```

CI is rerunning on the current head after its merge from `main`; no
failure should be inferred from an unrun or pending job.

## Real Behavior Proof

- Environment: cold CPython 3.12 interpreter with `headroom._core`
marked unavailable and `HEADROOM_REQUIRE_RUST_CORE=false`.
- Exact steps: import `ProxyConfig`/`create_app`, construct the
application with `optimize=True`, and enter the FastAPI lifespan through
the focused degraded-mode tests.
- Observed result: the server module and application construct
successfully; lifespan reports `rust_core=disabled` and changes the
shared config to `optimize=False`. Rust-backed constants still fail
loudly if explicitly used without the extension.
- Not tested: a production Windows host with Smart App Control actively
blocking the shipped `_core.pyd`; the failure boundary is reproduced
deterministically in a child interpreter.

## Runtime Rollout Safety

- Rollout-managed feature: explicit degraded mode controlled by
`HEADROOM_REQUIRE_RUST_CORE=false`.
- Stable/default behavior changed: no; the default still exits loudly
when the required core cannot load.
- Kill switch / disable path: unset the opt-out or set it to true to
restore fail-loud behavior.
- Unsafe override required: no.
- Qualification impact: cold-import, degraded-startup, parity, and
error-detection tests must remain green.
- Rollback path: revert this PR; no persistent data or migration is
involved.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] Code follows project style
- [x] Documentation is updated
- [x] New tests cover the changed behavior
- [x] No manual changelog edit

## Remaining Work

#2918 remains open for Authenticode signing and Smart App Control
reputation. #1466 remains the Windows compatibility umbrella.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-01 16:10:08 -05:00
lorenzozane 7287589d23 fix(install): use safe model backends for persistent services (#3646)
## Description

Native persistent-service manifests now explicitly select CPU for the
memory embedder and ONNX for Kompress by default. This avoids selecting
Metal-specific model backends during supervised startup, while
preserving explicit backend overrides.

Closes #3604

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Add `HEADROOM_EMBEDDER_RUNTIME=cpu` and
`HEADROOM_KOMPRESS_BACKEND=onnx` to native `persistent-service`
manifests only.
- Keep explicit `extra_env` values last so users can intentionally opt
into another backend.
- Add planner regression coverage for service defaults, overrides, and
non-service presets.
- Document the independent backend settings and the explicit MPS opt-in
boundary.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
python -m pytest -q tests/test_install/test_planner.py tests/test_install/test_supervisors.py tests/test_install/test_runtime.py tests/test_cli/test_install_cli.py
133 passed

ruff check headroom/install/planner.py tests/test_install/test_planner.py
All checks passed

ruff format --check headroom/install/planner.py tests/test_install/test_planner.py
All checks passed

git diff --check
passed
```

## Real Behavior Proof

- Environment: Ubuntu/Linux development host; native macOS launchd and
Apple Silicon Metal were not available.
- Exact command / steps: Ran the focused planner, supervisor, runtime,
and install CLI test suite plus Ruff and diff checks listed above.
- Observed result: The planner emits CPU/ONNX defaults for native
persistent services, preserves explicit backend overrides, and leaves
persistent-task and Docker manifests unchanged; the focused suite
passed.
- Not tested: Real Apple Silicon macOS 26 behavior, launchd restart
recovery, and ambient exported backend variables. The current base
source also does not prove that a clean persistent-service install
injected MPS; this PR is defensive safe-default hardening rather than a
claim of reproducing the reported crash.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Standard release.
- Stable/default behavior changed: Native `persistent-service` manifests
use CPU/ONNX model backends by default.
- Kill switch / disable path: Explicit `--env
HEADROOM_EMBEDDER_RUNTIME=pytorch_mps` and/or `--env
HEADROOM_KOMPRESS_BACKEND=pytorch_mps`, subject to platform and
dependency support.
- Unsafe override required: No.
- Qualification impact: Persistent services no longer select
Metal-specific model backends by default.
- Rollback path: Revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — this is a manifest, test, and documentation change.

## Additional Notes

`mypy` was not locally runnable because the maturin editable build
failed: the installed Rust toolchain lacked an applicable cargo
component. The PR does not claim CI success or real macOS runtime
validation.
2026-10-01 15:57:06 -05:00
Yevhen KovalandJerrettDavis 76ec363a60 docs(readme): list C# and PHP as AST-compressed, drop quarantined Perl (#3225)
## Description

The README "What's inside" bullet for CodeCompressor (and the two
matching lists in `wiki/transforms.md` plus the module docstring of
`headroom/transforms/code_compressor.py`) still read `Python, JS/TS, Go,
Rust, Java, C/C++, Perl`. That list predates two changes on `main`:

- **C#** (#1664 → #1926) and **PHP** (#201 → #2423) are first-class
`LangConfig` entries in `CodeAwareCompressor` (`code_compressor.py:440`
and `:469`) but were never added to the lists.
- **Perl** has been quarantined from code-aware compression since #2204
(refs #2185): `_get_parser()` refuses the `perl` grammar
(`code_compressor.py:106`, `:136`) and `compress()` short-circuits Perl
to the Kompress fallback / passthrough (`code_compressor.py:1200`), so
"AST-aware for … Perl" is no longer true.

All four lists now match the canonical docs site
(`docs/content/docs/code-compression.mdx`, which already says `Go, Rust,
Java, C, C++, C#, PHP` for Tier 2 and does not list Perl).

Closes #201 — PHP support itself landed in #2423 (the PR description
mentioned the issue without a closing keyword, so it stayed open); this
PR makes it visible in the README.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `README.md` — CodeCompressor bullet: add C# and PHP, drop Perl.
- `wiki/transforms.md` — "Multi-language support" bullet and the Tier 2
row of the "Supported Languages" table: add C# and PHP.
- `headroom/transforms/code_compressor.py` — module docstring,
"Supported Languages (Tier 2)": add C# and PHP. Docstring only; no
runtime code changes.

## Testing

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ git rev-parse --short HEAD
5a323bd8

$ grep -n "CodeCompressor\*\* — AST-aware" README.md
368:- **CodeCompressor** — AST-aware for Python, JS/TS, Go, Rust, Java, C/C++, C#, PHP.

$ grep -n "Multi-language support\|^| 2 |" wiki/transforms.md
158:- **Multi-language support** — Python, JavaScript, TypeScript, Go, Rust, Java, C, C++, C#, PHP
231:| 2 | Go, Rust, Java, C, C++, C#, PHP | Function body compression |
354:| 2 | SmartCrusher | Compress JSON tool outputs |

$ sed -n "18,19p" headroom/transforms/code_compressor.py
Supported Languages (Tier 2):
- Go, Rust, Java, C, C++, C#, PHP

$ grep -n "Perl" README.md wiki/transforms.md || echo PERL_ABSENT_FROM_LISTS
PERL_ABSENT_FROM_LISTS

$ grep -n "_UNSAFE_TREE_SITTER_LANGUAGES" headroom/transforms/code_compressor.py
106:_UNSAFE_TREE_SITTER_LANGUAGES: frozenset[str] = frozenset({"perl"})
136:    if language in _UNSAFE_TREE_SITTER_LANGUAGES:

$ grep -n "CSHARP = \|PHP = \|CodeLanguage.CSHARP: LangConfig\|CodeLanguage.PHP: LangConfig" headroom/transforms/code_compressor.py
225:    CSHARP = "csharp"
226:    PHP = "php"
440:    CodeLanguage.CSHARP: LangConfig(
469:    CodeLanguage.PHP: LangConfig(

$ python3 -m py_compile headroom/transforms/code_compressor.py && echo PY_COMPILE_OK
PY_COMPILE_OK

$ git log -1 --format=%B | npx -y -p @commitlint/cli -p @commitlint/config-conventional commitlint --config .commitlintrc.json
(exit 0)

$ git diff --stat main..HEAD
 README.md                              | 2 +-
 headroom/transforms/code_compressor.py | 2 +-
 wiki/transforms.md                     | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)
```

## Real Behavior Proof

- Environment: Linux 6.18 (CachyOS), system Python 3.14.7, checkout of
`headroomlabs-ai/headroom` `main` at 34a55175 (v0.36.5), branch
`docs/readme-code-languages`. No `uv`/dev venv on this machine.
- Exact command / steps: the `grep` / `sed` / `py_compile` /
`commitlint` / `git diff --stat` commands pasted above, run after the
patch.
- Observed result: all four language lists name C# and PHP; Perl is
absent from README and wiki; the quarantine guard and the C#/PHP
`LangConfig` entries exist at the quoted lines on current `main`; the
module still byte-compiles; the commit message passes the repo's
commitlint config; diff is 4 lines across 3 files, docs/docstring only.
- Not tested: `uv run pytest` / `ruff` / `mypy` / `make ci-precheck` —
not run locally (no dev environment here); no runtime path is touched,
so the compressor's behavior is unchanged by construction. The docs site
(`docs/`) is untouched because it was already correct.

## Runtime Rollout Safety

- Rollout-managed feature(s): N/A (documentation only)
- Minimum rollout channel: N/A
- Stable/default behavior changed: No
- Kill switch / disable path: N/A
- Unsafe override required: No
- Qualification impact: None
- Rollback path: revert the commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

- N/A checklist items: no code comments / tests to add for a docs-only
change; the test suite was not run locally (see "Not tested").
- `wiki/transforms.md`: per #2784 the wiki is unpublished markdown that
"needs no syncing". I still aligned its two lists because they were
factually wrong in the same way and the file was deliberately kept as a
migration backlog — happy to drop that hunk if you'd rather keep the PR
to README + docstring.
- Perl removal: if the quarantine is ever lifted (the upstream
tree-sitter-perl scanner wedge from #2185), Perl can be re-added to
these lists alongside the code change.
- Already correct and untouched: the `Supported: …` error string in
`_get_parser()` (`code_compressor.py:170`) and the docs site table.

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-09-25 10:37:14 -05:00
Rod BoevandJerrettDavis b0c19a25f0 fix(security): reject unauthenticated public proxy binds (#3316)
## Description

Headroom's published image and maintained Docker launchers could expose
the
proxy on every host interface without a proxy token. This change moves
the
security boundary to the deployment artifacts: published images default
to a
container-loopback listener, supported local launchers publish only on
host
loopback, and deliberate public deployments retain explicit token
protection.
The application factory remains transport-neutral.

Closes #3283

## Type of Change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Set both published image stages to a container-loopback default while
keeping explicit host overrides available.
- Changed native Compose and the maintained Bash and PowerShell
launchers to use an internal `0.0.0.0` listener with host-loopback
publication.
- Preserved the top-level Compose token requirement and the existing
HTTP and WebSocket credential behavior for deliberate public binds.
- Scoped differential-capture token injection to the internal
`headroom-proxy` destination and kept sensitive headers redacted in
captured records.
- Added focused launch-contract, native-core, capture, and Docker
publication checks, plus a base/head harness that tolerates a base
already containing the fix.
- Updated Docker, installer, capture, and mirrored documentation,
including an unauthenticated local loopback quickstart and an
authenticated deliberate-public example.

## Testing

- [x] Focused unit and integration tests pass
- [x] Linting and formatting checks pass
- [ ] Type checking passes
- [x] New tests added for the changed deployment and capture behavior
- [ ] Manual testing performed

### Test Output

```text
uv run python -m pytest tests/test_proxy_hardening.py tests/test_install/test_native_installers.py tests/test_docker_compose_persistence.py tests/test_network_diff_capture.py tests/test_rust_core_smoke.py -q
76 passed, 1 skipped, 1 warning in 23.45s

uv run ruff check tests/test_install/test_native_installers.py tests/test_install/test_runtime.py
All checks passed!

bash -n e2e/docker-bind-security.sh
passed

bash e2e/docker-bind-security.sh --base-image headroom-t3283-base --head-image headroom-t3283-head --fixture tests/fixtures/docker-bind-exposure.json --base-ref origin/main
base_ref=origin/main ... runtime_slim_config=loopback rust_core=loaded local_completion=200 attacker_loopback=000 ws_status=403
docker bind security: fixture forwarding, loopback isolation, built-image health, slim-image loopback config, and token controls passed
```

## Real Behavior Proof

- Environment: Windows, Docker Desktop 29.6.2, current `origin/main`
base, head image, and a real Docker bridge peer.
- Exact command / steps: run the focused security/runtime tests, Compose
configuration checks, then `bash e2e/docker-bind-security.sh` against
the current base and head image definitions.
- Observed result: `76 passed, 1 skipped`; the built-image harness
passed base/head health, local completion, non-loopback isolation,
authenticated public HTTP/WebSocket controls, loopback runtime-slim
configuration, and direct Rust-core loading.
- Not tested: runtime-slim proxy service health on this host, because
its unrelated ONNXRuntime import path exits with code 139; the full
hosted matrix and a future published GHCR image remain CI-owned.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Unsafe override required: no
- Minimum rollout channel: next release containing the Docker and
launcher changes
- Stable/default behavior changed: published images and maintained local
Docker launchers now use loopback-safe publication defaults
- Kill switch / disable path: revert the deployment-artifact changes
before release; deliberate public deployments remain explicitly
operator-controlled
- Qualification impact: Docker users relying on all-interface default
publication must opt into public publication and configure the existing
proxy token
- Deliberate public deployment: use an explicit public host publication,
set `HEADROOM_PROXY_TOKEN`, and send the matching bearer or
`X-Headroom-Proxy-Token` credential
- Local deployment: use the maintained loopback publication and no
mandatory token
- Rollback path: revert the deployment-artifact and documentation
changes together

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove the behavior is effective
- [x] Focused tests pass locally with these changes
- [x] I did **not** edit `CHANGELOG.md`; release-please generates
release notes from the conventional PR title

## Additional Notes

The issue fixture remains provenance for the unauthenticated
publication. Native
`headroom proxy` defaults and the application-level authentication
owners are
unchanged.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-09-24 23:58:46 -05:00
Tejas ChopraandClaude Opus 5 5ff4ea1ef9 fix(metrics): split rate-limited by source and label failures by provider (#3707)
Closes #3696. #3615 moved the outcome-funnel short-circuit from `>= 500`
to `>= 400` — correct, and staying — but it silently changed what two
metrics mean and the docs were never updated.

**⚠️ This changes exported series. See Migration.**

## What was wrong

- `headroom_requests_failed_total` was documented as "Upstream 5xx
errors"; it now counts 4xx too.
- `headroom_requests_rate_limited_total` was documented as "Requests
**Headroom** rejected via its own rate limiter (**not upstream 429s**)"
— #3615 inverted exactly that.
- 4xx also dropped out of `headroom_requests_by_provider` (only
`record_request` touches it), leaving Prometheus with no way to tell
which upstream was failing.

## Changes


**`headroom_requests_rate_limited_total{source="headroom"|"upstream"}`**
— two 429s mean opposite things to an operator: `headroom` is our
limiter refusing (raise the cap), `upstream` is the provider refusing
(back off / shard keys). Closed label set, so cardinality stays bounded;
unknown values clamp to `headroom`, the counter's pre-#3615 meaning, so
an un-updated caller cannot invent a label. Threaded at all five call
sites (`openai.py:3554, 5681`, `anthropic.py:1256`, `gemini.py:405` →
`headroom`; `outcome.py:543`, the #3615 funnel → `upstream`).

**`headroom_requests_failed_total{provider}`** — uses
`outcome.provider`, already Copilot-relabelled by the funnel, so no
second convention. Matches the existing
`upstream_connection_errors_by_provider` shape.

**All four backends**, so no sink reports a differently-shaped counter:
Prometheus, persistent/lifetime
(`SavingsTracker.record_lifetime_rate_limited`), OTel (attribute on
`headroom.proxy.requests.rate_limited`), and `/stats` JSON.
`SCHEMA_VERSION` stays 5 — the new maps are purely additive, and old
state files load with them empty rather than back-filling invented
history.

**Docs**: `docs/metrics-technical-guide.md`,
`docs/content/docs/metrics.mdx`, `wiki/metrics.md`, plus tooltips on the
dashboard's Failed / Rate Limited tiles so the UI stops implying "Rate
Limited" means Headroom only. `requests_total` is now documented as a
completed-request denominator with `inbound_requests_total` named as the
all-traffic counter, and the failure-rate PromQL carries a "why" on its
denominator so nobody "simplifies" it. Checked and found clean:
`examples/grafana/headroom-dashboard.json`, `docs/observability.md`,
`docs/content/docs/proxy.mdx`, `wiki/proxy.md`, `README.md`.

## Second commit: a bug found while reviewing the first

Adding the label turned `headroom_requests_failed_total` from a counter
that **always** exported `0` into one that exports **no sample at all**
on a healthy proxy — confirmed by rendering `/metrics` on a fresh
instance:

```
headroom_requests_failed_total:       0 sample(s)
headroom_requests_rate_limited_total: 2 sample(s)
```

That breaks the failure-rate query documented in this very PR: an empty
numerator empties the whole PromQL expression, so the panel reads "No
data" instead of 0% while everything is fine, and `absent()` alerts
would fire on healthy proxies. The rate-limit series avoided this by
seeding its closed label set; providers are open-ended so the same trick
does not apply.

Fix: seed the `unknown` bucket at zero. It is a real bucket —
`record_failed` uses it for unattributed failures — so this invents no
provider. Guarded by a new test.

## Migration

Both counters now export **only** labelled series; an unlabelled sample
alongside would double-count under `sum()`. Unaggregated queries return
multiple series — `sum without (source)` / `sum without (provider)`
reproduces the previous value exactly. Noted in the docs and an mdx
Callout.

## Verification

- **16 tests fail on pre-change source** (verified by reverting only
`headroom/`), pass after.
- Targeted metrics suites → **58 passed**. Broad sweep `-k "metric or
stats or outcome or rate_limit or savings_tracker or persistent"` → 634
passed / 44 skipped. Handler, dashboard and savings-history suites → 90
+ 79 passed.
- Ruff 0.16.7 (the CI pin, via `scripts/verify-ruff-version.py
--print-version`): clean.

`CHANGELOG.md` intentionally untouched — `changelog-guard.yml` fails any
PR that edits it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 07:26:40 -07:00
Tejas ChopraandClaude Opus 5 b8b222f940 fix(deps): install on Python 3.14 from prebuilt wheels only (#3631)
## Description

Rolled out Python 3.14 and cannot downgrade. Two problems block a
no-compiler install of `headroom-ai[proxy]` on 3.14:

1. **No pricing on 3.14.** litellm is skipped on Python 3.14 (GH #956)
because releases before 1.93 rejected it. So 3.14 installs have no
LiteLLM, and the dashboard shows $0.00 saved. LiteLLM has supported 3.14
since 1.93.0 (BerriAI/litellm#26343), and **1.96.2** is the first
release with prebuilt wheels for Python 3.10 to 3.14 on Linux, macOS and
Windows. Between 1.92 and 1.96.0 some platforms only get an sdist, which
needs a Rust build.
2. **macOS 3.14 needs a C compiler.** `watchdog` 6.0.0, the latest, has
no macOS wheel for Python 3.14 (Linux and Windows have one), so pip
compiles it. It only powers the optional `--code-graph` file watcher,
which already skips itself when watchdog is missing. But
`ensure_proxy_dependencies()` required it, so the proxy wouldn't start
without it.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `pyproject.toml`: `litellm>=1.86.2,<2.0; python_version < '3.14'`
becomes `litellm>=1.96.2,<2.0` (core and `dev`)
- `pyproject.toml`: `watchdog>=4.0.0` becomes `watchdog>=4.0.0;
sys_platform != 'darwin' or python_version < '3.14'`
- `headroom/cli/proxy.py`: `watchdog` removed from the proxy startup
dependency check
- `uv.lock`: refreshed (litellm 1.88.1 to 1.101.0, which moves
boto3/botocore/s3transfer/awscrt; watchdog marker)
- `tests/test_litellm_optional.py`: replaces the "skipped on 3.14"
assertion with tests that litellm applies on every supported Python and
platform with a floor that has 3.14 wheels; that watchdog is skipped
only on macOS 3.14; that the proxy dependency check never imports
watchdog; and that the code graph watcher returns `False` without
watchdog
- `.github/workflows/ci.yml`: new `python-314-wheels` job (Linux +
macOS, runs when packaging changes) that installs `headroom-ai[sandbox]`
with `--only-binary=:all:` on Python 3.14, checks proxy deps and LiteLLM
pricing, and starts the proxy until `/health` answers
- README and `wiki/metrics.md`: removed "pick Python 3.13 for the dollar
figure"; comments that described the 3.14 litellm gate updated

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_litellm_optional.py tests/test_optional_dependencies.py tests/test_pricing_from_litellm.py
18 passed in 3.07s

# new tests without the pyproject/proxy.py change
FAILED tests/test_litellm_optional.py::test_litellm_is_required_on_every_supported_python
FAILED tests/test_litellm_optional.py::test_litellm_floor_has_wheels_for_python_314
FAILED tests/test_litellm_optional.py::test_watchdog_is_skipped_only_where_it_has_no_wheel
FAILED tests/test_litellm_optional.py::test_proxy_starts_without_watchdog
# with the change
6 passed in 0.34s

$ uvx ruff@0.16.7 check <changed files> && uvx ruff@0.16.7 format --check <changed files>
All checks passed!
7 files already formatted
$ mypy --ignore-missing-imports headroom/cli/proxy.py headroom/pricing/litellm_pricing.py
Success: no issues found in 2 source files
$ actionlint .github/workflows/ci.yml
(clean)
```

## Real Behavior Proof

- Environment: macOS arm64, Python 3.14.7, fresh venv,
`PATH=/usr/bin:/bin` (no cargo, no Homebrew toolchain)
- Exact command / steps:
1. took the 0.37.0 wheel and changed only its `Requires-Dist` lines to
match this PR (litellm floor, watchdog marker)
2. `pip install --only-binary=:all: "headroom-ai[sandbox] @
file://…whl"`
  3. ran this branch's `ensure_proxy_dependencies()` in that env
4. started this branch's proxy in that env: `headroom proxy --port
18931`, then `GET /health` and `GET /stats`
- Observed result:
- install exit 0, 0 source builds, litellm 1.101.0, watchdog not
installed
  - dependency check passed without watchdog
  - `/health`: `"status":"healthy","ready":true`
  - `/stats`: `"litellm_available": true`
- Checked separately against PyPI: every one of the 132 to 136 locked
`[sandbox]` + enterprise packages has a Python 3.14 wheel on Linux
x86-64 and Windows x64. On macOS arm64 the only one missing is
`watchdog`, which this PR removes
- Earlier, on Python 3.14.7 with litellm 1.101.0, 0.37.0's 1,236
LiteLLM-related tests produced the same result (1,157 passed) as on
Python 3.12 with litellm 1.91.5
- Not tested: the new CI job itself (runs on this PR); Intel macOS,
Windows on ARM and Alpine, which lack `onnxruntime` or `headroom-ai`
wheels for other reasons

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: Python 3.14 installs now get LiteLLM,
so USD savings are priced instead of $0.00. macOS on 3.14 no longer
installs watchdog, so `--code-graph` live reindexing is off there unless
watchdog is installed manually
- Kill switch / disable path: n/a (dependency metadata)
- Unsafe override required: no
- Qualification impact: litellm floor rises from 1.86.2 to 1.96.2 on all
Pythons
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 10:42:13 -07:00
Tejas ChopraandClaude Opus 5 aceff2ea9d fix(proxy): gateway turn fixes found by benchmarking behind a gateway (#3619)
## Description

Fixes five defects found by running real Claude Code sessions through a
gateway that uses the `/v1/compress` turn contract (#3605).

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- **Prompt text is never compressed on replaying callers.** The coding
profile compresses user messages, and on turn 1 that rewrote a plain
agent's task ("Fix this bug in the library source" became "Fix bug in
source", and test ids lost their `tests/test_` prefix). When
`prefix_replay_guaranteed` is set, `content_router` protects the opening
user messages and the newest user message, for string and list content
alike. The one exception is a newest user message that answers a fenced
shell command in the assistant turn before it: that is a text harness's
tool observation and still compresses. Tool results still compress, and
cache freeze is unchanged.
- **Anthropic's four cache breakpoint limit.** One benchmark request
failed with `max 4 cache_control blocks, found 5`. Session turns in the
OpenAI handler now re-place markers with
`normalize_message_cache_control`. A last-stop guard,
`enforce_cache_breakpoint_budget`, runs in the gateway extension for
Anthropic bodies: it re-places markers, then drops the oldest extras,
and logs `event=cache_breakpoint_budget` when it repairs.
- **Failure relay without a body.** `POST /v1/compress/response`
rejected an upstream 4xx/5xx without a response body as
`missing_response`. A body is now required only on 2xx.
- **Usage totals after a re-drive.** The done payload carries
`billed_usage` summed across rounds. After one or more re-drives,
`response` is never null: it is the hook's replacement or the latest
provider response (including when the hook hands back that same object),
with `usage` set to the totals in its own key shape (chat or Anthropic).
Turns with no re-drive keep `response: null`. Before, the client saw the
last round only.
- **Cold Kompress stall.** When native Kompress init is deferred at
startup, a daemon thread preloads it after a short delay.
`HEADROOM_KOMPRESS_WARMUP=0` disables it. It is off under pytest and
skipped on glibc < 2.28 (the #1908 segfault hosts).
- Docs: `proxy.mdx` and `wiki/proxy.md` describe the 2xx-only body rule
and `billed_usage`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`) on the changed modules
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_transforms_content_router.py tests/test_compress_session_mode.py \
    tests/test_compress_route_tokenizer_by_model.py tests/gateway tests/test_proxy \
    tests/test_kompress_preload_deferral.py
672 passed, 4 skipped in 228.49s

$ uvx ruff@0.16.3 format --check <changed files> && uvx ruff@0.16.3 check <changed files>
12 files already formatted
All checks passed!

$ mypy <changed modules>
Success: no issues found in 6 source files
```

New tests: 5 prompt-protection tests in
`test_transforms_content_router.py` (3 fail without the fix),
`tests/gateway/test_cache_breakpoint_budget.py`, failure-relay and
billed-usage tests in `test_redrive_loop.py`, `billed_usage` in the
response-half contract test,
`tests/test_proxy/test_kompress_background_warmup.py`.

## Real Behavior Proof

- Environment: headroom proxy on this branch behind a gateway, Claude
Code with claude-sonnet-5, real Anthropic calls.
- Exact command / steps: 8 smoke sessions (32 calls) on SWE-bench-style
tasks, with a spend logger recording every provider round.
- Observed result: 0 errors. Task statements reached the provider
verbatim. On every re-driven call (2 and 3 rounds) the client's usage
equalled the sum of the per-round provider usage. Kompress warmed in the
background in about 4 s at startup; in a replay of the slow session the
first code-aware turn went from 5.4 s to 1.3 s.
- Not tested: glibc < 2.28 hosts (warm-up is skipped there by design).
The original body behind the 5-breakpoint error was not captured, so the
guard is covered by unit tests rather than a live repro.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: stable.
- Stable/default behavior changed: yes. Prompt text on replaying callers
is no longer compressed, the done payload adds `billed_usage`, and
Kompress preloads in the background when deferred.
- Kill switch / disable path: `HEADROOM_KOMPRESS_WARMUP=0` for the
warm-up. The other changes are correctness fixes with no switch.
- Unsafe override required: no.
- Qualification impact: none expected; the breakpoint guard only acts on
bodies that would otherwise get a 400.
- Rollback path: revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

Open follow-up, not in this PR: steady-state code-aware compression
falls back to Kompress at 0.3 to 1.3 s per tool result even when it
saves nothing. A gate for that is worth a separate change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 08:24:53 -07:00
Tejas ChopraandClaude Fable 5.1 d9757dcc6b fix(proxy): gateway turn contract for /v1/compress behind a compress-turn seam (#3605)
## Description

Adds a gateway turn contract to `POST /v1/compress`, allowing an API
gateway to submit a complete provider request and later report the
provider response. Requests without a top-level `gateway` object remain
unchanged.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Adds the request half to `/v1/compress` and the response half at
`/v1/compress/response`, including bounded pending turns, idempotent
rounds, usage relay, and redrive obligations.
- Adds a reusable six-stage compress-turn extension seam and turn-hook
priority/provider-header support.
- Keeps native tool-schema deferral in the licensed extension while
sharing protocol-shape helpers with core.
- Accepts Anthropic, OpenAI chat, OpenAI Responses, and Kong-shaped
usage payloads.
- Documents the gateway contract in `docs/content/docs/proxy.mdx` and
`wiki/proxy.md`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
tests/gateway + gateway unit + turn-hook seam + test_issue_746_tool_search:
358 passed

ruff 0.16.3 lint/format and mypy on touched modules:
clean

Kong 3.9 Docker e2e (HEADROOM_KONG_E2E=1):
4 passed
```

## Real Behavior Proof

- Environment: Kong 3.9 Docker e2e and real-provider Claude Code through
Kong on SWE-bench-mini.
- Exact command / steps: `HEADROOM_KONG_E2E=1` gateway e2e;
real-provider traffic through the gateway contract with usage relay and
licensed native deferral.
- Observed result: four Kong e2e tests passed; prefix cache remained
intact with zero busts; relayed usage matched Anthropic token-for-token;
licensed tool-search deferral reduced cache-read tokens per turn by
27–30%.
- Not tested: production rollout at scale.

## Runtime Rollout Safety

- Rollout-managed feature(s): gateway contract and in-process
pending-turn registry.
- Minimum rollout channel: controlled gateway deployment before broader
enablement.
- Stable/default behavior changed: only requests containing a top-level
`gateway` object enter the new contract; ordinary `/v1/compress` bodies
are unchanged.
- Kill switch / disable path: set `HEADROOM_GATEWAY_CONTRACT=0`.
- Unsafe override required: no.
- Qualification impact: gateway callers must redeem advertised `redrive`
and `relay_usage` obligations.
- Rollback path: disable the contract with `HEADROOM_GATEWAY_CONTRACT=0`
or revert the feature commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

Not applicable; this is an API contract change.

## Additional Notes

Pending turns are process-local and bounded by
`HEADROOM_GATEWAY_TURN_TTL_SECONDS`,
`HEADROOM_GATEWAY_MAX_PENDING_TURNS`, and
`HEADROOM_GATEWAY_MAX_REDRIVES`.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-15 16:56:07 -07:00
Raúl 3d3c629a9f fix(proxy): write per-port proxy logs so concurrent instances stop truncating each other (#3204)
## Description

Every proxy instance wrote the same two fixed files:
`~/.headroom/logs/proxy.log` (runtime log, via the `RotatingFileHandler`
in `_setup_file_logging`) and `~/.headroom/logs/proxy-stdio.log`
(subprocess stdout/stderr from `_start_proxy`). The name is a constant
(`paths.py` `_PROXY_LOG_FILE = "proxy.log"`), so N concurrent proxies on
different ports (wrapped copilot, droid, claude; plus any persistent
`install agent` deployment) run N independent rotating handlers over the
*same* file and truncate/rename each other's history.

This fix keys both files by port (`proxy-<port>.log`,
`proxy-stdio-<port>.log`) so each proxy rotates only its own file.
`headroom perf` aggregates every per-port file and still reads a lone
legacy `proxy.log` when that is all that exists (backward compat); the
`proxy-stdio-*` captures (raw stdout/stderr, not PERF records) are
excluded from the parser.

This matters more now that wrap-spawned proxies self-exit after an idle
grace (orphan-proxy watchdog, #3202): the process is gone, so its
per-port log is the only forensic evidence and must survive an unrelated
proxy's rotation.

Closes #3203

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/paths.py`: `proxy_log_path(port=None)` and new
`proxy_stdio_log_path(port=None)` are the single source of truth for the
filenames (`proxy-<port>.log` / `proxy-stdio-<port>.log` with a port,
legacy names without). The `_PROXY_LOG_FILE` constant stays only as the
readers' fallback name.
- `headroom/proxy/helpers.py`: `_setup_file_logging(port=None)` targets
`proxy_log_path(port)`. The duplicate-handler guard runs BEFORE the
`RotatingFileHandler` is constructed (construction opens the file), so a
second `create_app` on a different port in one process cannot leave an
empty stray `proxy-<port>.log` or leak an fd; a handler pointing at a
different port is replaced and closed so per-port keying holds.
- `headroom/proxy/server.py`: `create_app` resolves `config` first, then
calls `_setup_file_logging(config.port)` (the proxy always knows its own
port).
- `headroom/cli/wrap.py`: `_get_log_path(port=None)` /
`_get_proxy_stdio_log_path(port=None)` are per-port; `_start_proxy`
passes its `port`. The stdio filename comes from
`paths.proxy_stdio_log_path` (single source) and its directory from
`_get_log_path`.
- `headroom/perf/analyzer.py`: `parse_log_files` selects PERF logs by a
positive filename match (`_PERF_LOG_FILE_RE` =
`proxy(-<digits>)?.log(.N)?`) rather than a `proxy-stdio` blacklist, so
`proxy-stdio-*.log` and any other `proxy-<word>.log` are never ingested;
ordering is deterministic (mtime, then path).
- `headroom/cli/perf.py`, `wiki/cli.md`: doc/help text updated for
per-port + legacy.
- `tests/test_perf_per_port_logs.py` (new): per-port path helpers; a
second `_setup_file_logging` on a new port reroutes cleanly with no
cross-file leakage (regression for the guard-ordering bug); and
`parse_log_files` aggregates per-port + legacy while excluding stdio and
other non-PERF `proxy-*.log`.
- `tests/test_cli_proxy_env.py`, `tests/test_agent_savings.py`,
`tests/test_cli/test_wrap_{codex,claude_vertex_proxy_env,proxy_detach}.py`:
the `_get_log_path` monkeypatches take the new optional `port`, and the
two stdio-filename assertions/fixtures in `test_cli_proxy_env.py` use
the per-port name.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`) — plus `ruff format --check`
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/python -m pytest tests/test_perf_per_port_logs.py tests/test_cli_proxy_env.py tests/test_agent_savings.py -q -p no:randomly
115 passed, 1 skipped in 4.44s

$ .venv/bin/python -m pytest tests/test_cli_perf_format.py tests/test_learn/test_analyzer.py -q
107 passed in 4.16s

$ .venv/bin/python -m pytest tests/test_cli/test_wrap_proxy_detach.py tests/test_cli/test_wrap_codex.py tests/test_cli/test_wrap_claude_vertex_proxy_env.py -q
120 passed in 171.19s

$ .venv/bin/ruff check <touched> && .venv/bin/ruff format --check <touched>
All checks passed!

$ .venv/bin/mypy headroom/paths.py headroom/cli/wrap.py headroom/proxy/helpers.py headroom/proxy/server.py headroom/perf/analyzer.py
Success: no issues found in 5 source files
```

Rust gate (`cargo clippy/test/fmt`) not run: Python-only change, no
crate touched.

## Real Behavior Proof

- Environment: macOS 26.5 arm64, Python 3.13, local editable source
build.
- Exact command / steps: `headroom proxy --port 8890` and `headroom
proxy --port 8891` started concurrently; inspected `~/.headroom/logs/`;
then `headroom perf --hours 1 --format json`.
- Observed result: each proxy wrote its own file (`proxy-8890.log`,
`proxy-8891.log`, independent), the pre-existing shared `proxy.log` was
not touched, and `headroom perf` reported `log_files_read: 4`,
aggregating per-port + legacy with no error.
`test_setup_file_logging_reroutes_on_port_change` additionally drives
the production helper twice in one process and asserts each port's
records land only in its own file.
- Not tested: Windows path semantics (filename construction is
platform-independent `pathlib`); a legacy `proxy.log` written by an OLD
binary running beside a new per-port one end to end (readers handle it,
but not exercised live).

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A (no flag; readers accept both layouts).
- Stable/default behavior changed: log filenames change from `proxy.log`
to `proxy-<port>.log`; `headroom perf` reads both, so history is not
orphaned.
- Kill switch / disable path: N/A (no flag).
- Unsafe override required: no.
- Qualification impact: none — readers stay backward compatible with a
lone legacy `proxy.log`.
- Rollback path: revert; a downgraded binary writes legacy `proxy.log`
again and readers still aggregate any per-port files left behind.

Disk growth: each per-port family keeps the prior bound (`maxBytes=10 MB
× backupCount=5` = 60 MB), but there is now one family per distinct port
used instead of one global family. Ports are usually stable (default
8787; wrap auto-selects a nearby free port), so in practice a handful of
families; the theoretical worst case is bounded by the port scan
(`_find_available_port`, ≤100) → ≤ ~6 GB. No automatic pruning; stale
`proxy-<port>.log*` families can be deleted safely (perf reads whatever
remains). A retention policy is intentionally out of scope.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A — log-filename change; no UI.

## Additional Notes

Follow-up (separate SDK PR, not here): the TypeScript SDK helper
`sdk/typescript/src/paths.ts::proxyLogPath()` still returns legacy
`proxy.log` with no port argument and is pinned by `paths.test.ts`. It
mirrors `paths.py`, so it should gain an optional port argument in an
SDK PR (kept separate to stay one-surface, and because that package's
vitest suite is the right gate for it).
2026-09-09 15:54:34 -05:00
Tejas ChopraandClaude Opus 5 27bdc2a899 docs(wiki): correct the unpublished tree, and remove invented telemetry (#3390)
`wiki/` **is not published anywhere.** `.github/workflows/docs.yml` says
so in its own header, no mkdocs or sphinx config exists, and `wiki/**`
sits in `paths-ignore` for CI — changes there don't even run a build. It
survives as pending-migration reference material for contributors.

That's why this is separate from the published-site work in #3388:
nothing here reaches a user through `docs.headroomlabs.ai`. It still
matters, because a contributor migrating a page would carry its errors
across, and because the repo is public.

## The finding that justifies this PR on its own

`wiki/benchmarks.md` carried an entire **invented "Production Telemetry"
section**, presented as real-world fleet data:

> real-world production telemetry from 250+ active proxy instances …
**52ms median overhead** … **1.4 billion tokens saved** across 249
instances

Plus P90/P99 timings, ~$4,000 saved, an OS split, a version-share
breakdown, and a dated collection window of March 30 – April 2, 2026.

A repo-wide grep for those figures **matches only that one file.** No
script, test or fixture produces them. The published benchmarks page
never carried the section, so the site owner had already excised it.

Removed rather than re-estimated. The per-category ranges hanging off it
(4.8%, 40–80%, 83–94%, 85–94%, 60–76%) went with it and now point at the
concrete compression table on the same page.

The same page also described `HEADROOM_TELEMETRY` as the opt-in switch
governing fleet-wide numbers. Fleet data would come from
`HEADROOM_BEACON` — a **separate switch that defaults ON**. That note is
gone with the section it footnoted; the accurate description of both
switches shipped to the live site in #3388.

## Other corrections, each verified against source

- **`wiki/agno.md`** constructed `HeadroomAgnoModel(config=...)`; the
field is `headroom_config` (`integrations/agno/model.py:135`). The
sibling `HeadroomPreHook` genuinely does take `config`, which is how
they got conflated.
- **`wiki/ccr.md`** documented CLI flags that don't exist, and a dead
demo script.
- **`wiki/image-compression.md`** had two broken CLI examples.
- **`wiki/getting-started.md`** used `plan.transforms_applied`;
`SimulationResult` exposes `transforms`. (`CompressResult` really does
have `transforms_applied` — two different result types, easy to cross.)
- **`wiki/cli.md`** documents 15 of ~27 top-level commands; an audit
note now says so rather than implying completeness.

## Disposition, for whoever picks up the migration

`wiki/cli.md` is the highest-value page to port — no site page is a full
CLI reference, and `docs.yml` names it as the reason `wiki/` still
exists. `wiki/integration-guide.md` and `wiki/configuration.md` also
cover ground the site doesn't.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01HRaEPjQcCSAwD2xbQMNENu

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-02 15:53:02 +05:30
dependabot[bot]andJerrettDavis ff17961cd7 deps: bump ruff from 0.15.22 to 0.16.2 in the pip-minor-patch group across 1 directory (#2962)
Bumps the pip-minor-patch group with 1 update in the / directory:
[ruff](https://github.com/astral-sh/ruff).

Updates `ruff` from 0.15.22 to 0.16.2
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/astral-sh/ruff/releases">ruff's
releases</a>.</em></p>
<blockquote>
<h2>0.16.2</h2>
<h2>Release Notes</h2>
<p>Released on 2026-08-06.</p>
<h3>Bug fixes</h3>
<ul>
<li>[<code>flake8-pyi</code>] Avoid false positives on
<code>singledispatch</code> functions (<code>PYI041</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/27335">#27335</a>)</li>
</ul>
<h3>Server</h3>
<ul>
<li>Register formatting capabilities dynamically to exclude TOML files
(<a
href="https://redirect.github.com/astral-sh/ruff/pull/27332">#27332</a>)</li>
</ul>
<h3>Contributors</h3>
<ul>
<li><a
href="https://github.com/MeGaGiGaGon"><code>@​MeGaGiGaGon</code></a></li>
<li><a
href="https://github.com/charliermarsh"><code>@​charliermarsh</code></a></li>
<li><a href="https://github.com/epage"><code>@​epage</code></a></li>
<li><a href="https://github.com/sharkdp"><code>@​sharkdp</code></a></li>
<li><a href="https://github.com/ntBre"><code>@​ntBre</code></a></li>
</ul>
<h2>Install ruff 0.16.2</h2>
<h3>Install prebuilt binaries via shell script</h3>
<pre lang="sh"><code>curl --proto '=https' --tlsv1.2 -LsSf
https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-installer.sh
| sh
</code></pre>
<h3>Install prebuilt binaries via powershell script</h3>
<pre lang="sh"><code>powershell -ExecutionPolicy Bypass -c &quot;irm
https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-installer.ps1
| iex&quot;
</code></pre>
<h2>Download ruff 0.16.2</h2>
<table>
<thead>
<tr>
<th>File</th>
<th>Platform</th>
<th>Checksum</th>
</tr>
</thead>
<tbody>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-aarch64-apple-darwin.tar.gz">ruff-aarch64-apple-darwin.tar.gz</a></td>
<td>Apple Silicon macOS</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-aarch64-apple-darwin.tar.gz.sha256">checksum</a></td>
</tr>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-x86_64-apple-darwin.tar.gz">ruff-x86_64-apple-darwin.tar.gz</a></td>
<td>Intel macOS</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-x86_64-apple-darwin.tar.gz.sha256">checksum</a></td>
</tr>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-aarch64-pc-windows-msvc.zip">ruff-aarch64-pc-windows-msvc.zip</a></td>
<td>ARM64 Windows</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-aarch64-pc-windows-msvc.zip.sha256">checksum</a></td>
</tr>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-i686-pc-windows-msvc.zip">ruff-i686-pc-windows-msvc.zip</a></td>
<td>x86 Windows</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-i686-pc-windows-msvc.zip.sha256">checksum</a></td>
</tr>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-x86_64-pc-windows-msvc.zip">ruff-x86_64-pc-windows-msvc.zip</a></td>
<td>x64 Windows</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-x86_64-pc-windows-msvc.zip.sha256">checksum</a></td>
</tr>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-aarch64-unknown-linux-gnu.tar.gz">ruff-aarch64-unknown-linux-gnu.tar.gz</a></td>
<td>ARM64 Linux</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-aarch64-unknown-linux-gnu.tar.gz.sha256">checksum</a></td>
</tr>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-i686-unknown-linux-gnu.tar.gz">ruff-i686-unknown-linux-gnu.tar.gz</a></td>
<td>x86 Linux</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-i686-unknown-linux-gnu.tar.gz.sha256">checksum</a></td>
</tr>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-powerpc64-unknown-linux-gnu.tar.gz">ruff-powerpc64-unknown-linux-gnu.tar.gz</a></td>
<td>PPC64 Linux</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-powerpc64-unknown-linux-gnu.tar.gz.sha256">checksum</a></td>
</tr>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-powerpc64le-unknown-linux-gnu.tar.gz">ruff-powerpc64le-unknown-linux-gnu.tar.gz</a></td>
<td>PPC64LE Linux</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-powerpc64le-unknown-linux-gnu.tar.gz.sha256">checksum</a></td>
</tr>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-riscv64gc-unknown-linux-gnu.tar.gz">ruff-riscv64gc-unknown-linux-gnu.tar.gz</a></td>
<td>RISCV Linux</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-riscv64gc-unknown-linux-gnu.tar.gz.sha256">checksum</a></td>
</tr>
<tr>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-s390x-unknown-linux-gnu.tar.gz">ruff-s390x-unknown-linux-gnu.tar.gz</a></td>
<td>S390x Linux</td>
<td><a
href="https://releases.astral.sh/github/ruff/releases/download/0.16.2/ruff-s390x-unknown-linux-gnu.tar.gz.sha256">checksum</a></td>
</tr>
</tbody>
</table>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md">ruff's
changelog</a>.</em></p>
<blockquote>
<h2>0.16.2</h2>
<p>Released on 2026-08-06.</p>
<h3>Bug fixes</h3>
<ul>
<li>[<code>flake8-pyi</code>] Avoid false positives on
<code>singledispatch</code> functions (<code>PYI041</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/27335">#27335</a>)</li>
</ul>
<h3>Server</h3>
<ul>
<li>Register formatting capabilities dynamically to exclude TOML files
(<a
href="https://redirect.github.com/astral-sh/ruff/pull/27332">#27332</a>)</li>
</ul>
<h3>Contributors</h3>
<ul>
<li><a
href="https://github.com/MeGaGiGaGon"><code>@​MeGaGiGaGon</code></a></li>
<li><a
href="https://github.com/charliermarsh"><code>@​charliermarsh</code></a></li>
<li><a href="https://github.com/epage"><code>@​epage</code></a></li>
<li><a href="https://github.com/sharkdp"><code>@​sharkdp</code></a></li>
<li><a href="https://github.com/ntBre"><code>@​ntBre</code></a></li>
</ul>
<h2>0.16.1</h2>
<p>Released on 2026-07-30.</p>
<h3>Preview features</h3>
<ul>
<li>Add an option to opt out of human-readable names (<a
href="https://redirect.github.com/astral-sh/ruff/pull/27160">#27160</a>)</li>
<li>[<code>flake8-pytest-style</code>] Make fixes safe by default and
unsafe only when comments are present (<code>PT018</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/27201">#27201</a>)</li>
<li>[<code>pyupgrade</code>] Skip fix when a defaulted
<code>TypeVar</code> precedes a non-defaulted one (<code>UP040</code>,
<code>UP046</code>, <code>UP047</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/27133">#27133</a>)</li>
<li>[<code>ruff</code>] Fix false positive with unpacked arguments
(<code>RUF065</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/26959">#26959</a>)</li>
</ul>
<h3>Bug fixes</h3>
<ul>
<li>Bump <code>gen-lsp-types</code> to gracefully handle unknown
enumeration values in LSP messages (<a
href="https://redirect.github.com/astral-sh/ruff/pull/27230">#27230</a>)</li>
<li>[<code>flake8-bugbear</code>] Mark <code>range</code> as immutable
(<code>B008</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/27247">#27247</a>)</li>
<li>[<code>flake8-comprehensions</code>] NFKC-normalize keyword names in
<code>C408</code> fix (<a
href="https://redirect.github.com/astral-sh/ruff/pull/26813">#26813</a>)</li>
<li>[<code>flake8-return</code>] Fix false positive when variable is
read in <code>finally</code> clause (<code>RET504</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/25441">#25441</a>)</li>
<li>[<code>pydocstyle</code>] Skip section detection inside RST
directive bodies (<code>D214</code>, <code>D405</code>,
<code>D413</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/23635">#23635</a>)</li>
<li>[<code>refurb</code>] Parenthesize <code>yield</code> arguments in
the <code>FURB192</code> fix (<a
href="https://redirect.github.com/astral-sh/ruff/pull/27192">#27192</a>)</li>
</ul>
<h3>Rule changes</h3>
<ul>
<li>[<code>flake8-pytest-style</code>] Mark <code>PT022</code> fixes as
unsafe (<a
href="https://redirect.github.com/astral-sh/ruff/pull/26440">#26440</a>)</li>
<li>[<code>refurb</code>] Mark fixes that remove unknown separators as
unsafe (<code>FURB105</code>) (<a
href="https://redirect.github.com/astral-sh/ruff/pull/27200">#27200</a>)</li>
</ul>
<h3>Server</h3>
<ul>
<li>Fix indexing of excluded nested Ruff workspaces (<a
href="https://redirect.github.com/astral-sh/ruff/pull/27303">#27303</a>)</li>
<li>Lint TOML files in the LSP (<a
href="https://redirect.github.com/astral-sh/ruff/pull/26862">#26862</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/astral-sh/ruff/commit/5b48a040974781ba90b47c8df628f8fd9b6c95dd"><code>5b48a04</code></a>
Bump 0.16.2 (<a
href="https://redirect.github.com/astral-sh/ruff/issues/27555">#27555</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/1b9e5fc483b95a01fe02ff104820280b1b32e8ae"><code>1b9e5fc</code></a>
Update Swatinem/rust-cache action to v2.9.2 (<a
href="https://redirect.github.com/astral-sh/ruff/issues/27568">#27568</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/c4e86fc0394c92a9334ba2eb026c77c21db403be"><code>c4e86fc</code></a>
[ty] Add helper extension methods for half-range and equality
constraints (<a
href="https://redirect.github.com/astral-sh/ruff/issues/2">#2</a>...</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/17a00de2e298612201a8fe30790e9399204af1b9"><code>17a00de</code></a>
[ty] Reuse primer commands in memory reports (<a
href="https://redirect.github.com/astral-sh/ruff/issues/27553">#27553</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/6ea296b96923e142eb13af2bc6ad261c280d8eb1"><code>6ea296b</code></a>
[ty] Normalize type labels in structured docstrings (<a
href="https://redirect.github.com/astral-sh/ruff/issues/26923">#26923</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/2fc445f0053f4ec27c717fae0de3671d73c103be"><code>2fc445f</code></a>
[ty] Diagnose invalid <strong>getattr</strong> calls (<a
href="https://redirect.github.com/astral-sh/ruff/issues/27502">#27502</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/22c7823c4e8bffcca97688d8438c9b567d6817d8"><code>22c7823</code></a>
[ty] Enable (but downrank) auto-import completion suggestions from
stub-only ...</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/05160d507f05345a72db9c28ab4edf7c92334819"><code>05160d5</code></a>
[ty] Diagnose invalid descriptor <code>__get__</code> calls (<a
href="https://redirect.github.com/astral-sh/ruff/issues/27400">#27400</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/baea3d0dcec6d6f6d1659321940f3725771c5f45"><code>baea3d0</code></a>
[ty] Expose strict analysis options in the playground (<a
href="https://redirect.github.com/astral-sh/ruff/issues/27543">#27543</a>)</li>
<li><a
href="https://github.com/astral-sh/ruff/commit/c88946ebeb92be6d276087f0d528cd6471df4ead"><code>c88946e</code></a>
[ty] Bump ecosystem-analyzer for strict project settings (<a
href="https://redirect.github.com/astral-sh/ruff/issues/27542">#27542</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/astral-sh/ruff/compare/0.15.22...0.16.2">compare
view</a></li>
</ul>
</details>
<br />

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-08-14 16:38:08 -05:00
JD Davis eafdf11a2c fix(docker): ship Bedrock auth and current registry (#2982)
## Description

Fixes #1551 and #1692.

Every published Headroom Docker image now installs the existing
`bedrock` extra, so `--backend bedrock` can authenticate with temporary
STS, SSO, and credential-process credentials instead of failing because
`botocore` is absent.

Public Docker instructions now consistently use
`ghcr.io/headroomlabs-ai/headroom`. Several still pointed at the old
personal package, which is frozen at 0.27.0 and caused users to report
that no latest image existed.

## Type of Change

- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [x] Documentation update
- [x] Build / CI

## Changes Made

- Add `bedrock` to the standalone Dockerfile default extras.
- Add `bedrock` to all nine root/code/slim/nonroot bake targets.
- Replace obsolete personal GHCR references in README, llms.txt, Compose
guidance, testing guidance, and wiki docs.
- Add release contract tests for Bedrock dependencies and the current
organization registry.

## Testing

- [x] Focused Docker release and Bedrock preflight tests pass.
- [x] Full updater suites pass: 69 tests.
- [x] `uv run ruff check tests/test_release_workflows.py`
- [x] `docker buildx bake --print`
- [x] `git diff --check`

## Real Behavior Proof

Before this change, every published bake target installed only `proxy`
or `proxy,code`, so `AWS_SESSION_TOKEN` selected an unavailable botocore
path. Public copy-paste commands also referenced
`ghcr.io/chopratejas/headroom`, which the existing migration code and
changelog identify as frozen at 0.27.0.

After this change, all nine parsed bake targets install `bedrock`; the
regression resolves that package extra and confirms `boto3` plus
`botocore`. Every public Docker instruction covered by the contract
names `ghcr.io/headroomlabs-ai/headroom`.

## Runtime Rollout Safety

This changes image contents and documentation only; proxy routing and
non-Docker installs are unchanged. Static AWS credentials remain
unaffected. Existing manifests using the deprecated image continue to be
migrated by the established install-state logic. Rollback is a
Docker/bake extras and documentation revert.

## Review Readiness

- [x] Two related Docker blockers batched in one PR
- [x] Regression coverage included
- [x] No unrelated lockfile changes
- [x] Ready for review
2026-08-13 15:06:21 -05:00
3077ac81e8 feat: add deterministic runtime rollout controls (#1490)
## Description

Establish one centrally resolved, observable, deterministic, versioned
runtime rollout-control mechanism for Headroom. Runtime rollout controls
which behaviors an already-built artifact may expose; it does not select
or qualify a Headroom release/version.

## Type of Change

- [x] New feature (non-breaking change that adds functionality)
- [x] Bug fix (non-breaking change that fixes rollout enforcement
regressions)
- [x] Documentation update
- [x] Code refactoring (no functional changes)

## Changes Made

- Added `RolloutChannel`, `HEADROOM_ROLLOUT_CHANNEL`,
`--rollout-channel`, and a versioned immutable `RolloutSnapshot` shared
by Python configuration boundaries.
- Added schema/policy versions, canonical registry and snapshot SHA-256
identities, per-feature decision reasons, disable precedence, unsafe
qualification poisoning, strict CLI validation, and fail-closed
environment handling.
- Added `headroom rollout status --json`, Python `/stats.rollout`, and
Rust `/rollout/status` runtime provenance.
- Added equivalent Rust snapshot semantics and shared Python/Rust policy
vectors while retaining language-specific feature registries.
- Enforced rollout policy at alternate Python server composition roots
so `HEADROOM_READ_MATURATION=1` cannot bypass its beta gate.
- Preserved typed rollout snapshots across multi-worker serialization
with schema, policy, registry, snapshot-digest, type, and feature-name
validation.
- Made loopback runtime output-shaper updates replace the immutable
snapshot atomically for request readers, retain explicit request/disable
provenance, preserve channel and kill-switch precedence, invalidate
cached stats, and return the effective rollout decision.
- Made `headroom learn --verbosity --apply` report a channel-blocked
update instead of claiming the shaper is live.
- Made explicit CLI feature flags fail loudly when their current channel
blocks them.
- Made persistent interceptor installation select canary automatically,
or reject an explicitly insufficient channel unless the break-glass
override is set.
- Updated architecture, proxy, rollout, learn, and output-shaper
documentation with required channels and hot-reload semantics.

## Testing

- [x] Unit tests pass
- [x] Linting passes (`ruff check .` and `ruff format --check .`)
- [x] Type checking passes (`mypy headroom --ignore-missing-imports`)
- [x] New regression tests added for every corrected behavior
- [x] Rust tests and production-target Clippy pass
- [x] Documentation build passes

### Test Output

```text
Focused rollout coverage suite
57 passed; headroom.rollout + rollout CLI: 98% coverage

Affected proxy/rollout/transform/governance suites
222 passed; 0 failed

Final changed regression suites
100 passed; 0 failed

Cross-module hot-reload isolation regression
6 passed; 0 failed

cargo test -p headroom-core -p headroom-proxy --quiet
headroom-core: 924 passed; 1 ignored
headroom-proxy and integration suites: all passed

cargo clippy -p headroom-core -p headroom-proxy --lib --bins -- -D warnings
cargo fmt --all -- --check
ruff check .
ruff format --check .
mypy headroom --ignore-missing-imports
git diff --check
All passed

cd docs && npm run build
Compiled successfully; 164 static pages generated
```

The unsharded Windows-only CI selection exposed unrelated baseline
failures, principally the existing `sqlite:///C:\\...` URL parser
producing an invalid `\\C:\\...` path. At commit `8e793a80`, all 52
completed GitHub checks passed; the only other conclusions are expected
skips and superseded governance jobs.

## Real Behavior Proof

- **Environment:** Windows checkout on Python 3.13.3 and the current
Rust workspace, based on upstream `main` at `93f2d7a2`.
- **Exact command / steps:** Exercised canary and beta feature requests
through CLI status, Python `/stats.rollout`, Rust `/rollout/status`,
multi-worker payload round trips, loopback `/admin/runtime-env`, real
proxy request shaping before/after hot reload, installer manifest
generation, and shared Python/Rust policy vectors.
- **Observed result:** Stable blocks unstable requests; disable wins
over explicit/default/legacy/unsafe paths; unsafe state reports
`qualification_eligible=false`; worker handoff rejects tampering;
running output shaping changes only when the effective beta policy
permits it; explicit blocked flags fail with actionable diagnostics.
- **Not tested:** Live production traffic requiring provider
credentials, or future artifact qualification/promotion automation
(intentionally out of scope).

## Runtime Rollout Safety

- **Rollout-managed features:** Python `tool_result_interceptors`,
`proxy_output_shaper`, `read_maturation`; Rust `native_bedrock`,
`openai_responses_streaming`, `canary_probe`.
- **Minimum rollout channel:** Registry-defined per feature; process
default is `stable`.
- **Stable/default behavior changed:** No unstable feature becomes
enabled by default. Explicit blocked CLI flags now fail instead of
silently doing nothing.
- **Kill switch / disable path:**
`HEADROOM_DISABLE_FEATURES=<comma-separated feature names>`; explicit
disable has highest precedence, including over the unsafe override.
- **Unsafe override required:** No.
`HEADROOM_UNSAFE_ALLOW_UNSTABLE_FEATURES=1` is break-glass only and
makes qualification evidence ineligible.
- **Qualification impact:** Adds machine-readable policy/snapshot
identities and eligibility; does not implement qualification itself.
- **Rollback path:** Set the named disable list for operational
rollback, lower the channel, or revert this PR.

## Review Readiness

- [x] I have performed a full diff review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [x] I have made corresponding documentation changes
- [x] My changes generate no new warnings
- [x] I added tests that reproduce and prevent every regression fixed
during review
- [x] New and existing affected tests pass locally
- [x] I did **not** edit `CHANGELOG.md`; release-please generates it
from the Conventional Commit PR title

## Additional Notes

Out of scope: artifact candidates, benchmark orchestration,
qualification manifests/gates, promotion automation, release branches,
publication guards, and release-risk classification. Those workflows can
consume the rollout registry digest, runtime snapshot digest, decision
reasons, and qualification eligibility through supported black-box
interfaces.

---------

Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
Co-authored-by: JD Davis <jd@JDH-AIR-00.local>
2026-08-12 23:16:54 -05:00
Tejas Chopra 6422a80a58 fix(compress): resolve the /v1/compress tokenizer per model, and document the real contract (#2743)
## Description

`/v1/compress` does no format conversion — callers send whichever wire
shape they already use — but the pipeline pinned **one provider's token
counter for the whole route**.

`OpenAITokenCounter.count_message` walks list content for `text` and
`image_url` only and has **no else branch**, so Anthropic content blocks
contributed literally zero. A 599-token `tool_result` scored 8. A
request that really removed 235 characters reported `tokens_saved: 0` —
so a caller gating on `tokens_saved > 0` concludes compression is broken
while it is working.

Prompted by a Kong integration question ("do you support the Anthropic
native format?"). The answer is that we already did — we just reported
zeros for it, and the docs said otherwise.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Documentation update

## Changes Made

### Tokenizer resolution (no hardcoded lists)

Build the derived pipelines with `provider=None` so `TransformPipeline`
resolves the tokenizer from the **per-model registry**. Every registry
tokenizer derives from `BaseTokenizer`, whose `_count_content_parts`
ends in a serialize-and-count catch-all, which means:

- No block type counts as zero, and there is **no per-provider
block-type list to keep in sync**. An enumerated set was the first thing
I tried and it already missed `mcp_tool_result`,
`web_search_tool_result`, `document`, and `thinking`.
- Gemini / Mistral / DeepSeek / Kimi stop defaulting to a tiktoken count
when the registry already has a calibrated counter for them.
- Gateway aliases matching no vendor pattern still count correctly.

`mode="ccr"` now runs a derived pipeline too, for the same reason —
sharing `openai_pipeline` pinned its provider. Costs that mode its own
cold compression cache; correct metrics win.

### Tokenizer selection stays separate from context-limit resolution

Deliberately not welded together. `model_limit` feeds `context_pressure
-> min_ratio`, so letting a tokenizer decision pick the limit table
changes compression aggressiveness: `gpt-4-32k` answered by the
Anthropic table is **8,192 instead of 32,768**, a 4× under-estimate.
`test_tokenizer_choice_does_not_move_the_context_limit` pins the
independence.

### Docs, rewritten from the code

- **`proxy.mdx`** — the loopback-only default and **404-not-403**
behavior, previously undocumented *anywhere* in `docs/` despite shipping
in #2458 explicitly for gateway sidecars;
`HEADROOM_COMPRESS_ALLOW_REMOTE`; all four request fields; the whole
`config` object including every `mode` value and `frozen_message_count`;
`transforms_summary`; the 400/401/404/503 contract; and the timeout
fail-open shape (`compression_skipped` / `skip_reason`).
- **Corrected "never calls an LLM"** — accurate about *generative*
provider requests, misleading for a sidecar operator. Kompress (a
ModernBERT **encoder**, classification not generation) and Magika run
**in-process**, and `HEADROOM_KOMPRESS_ENDPOINT` offloads inference over
HTTP — **real egress**. Now stated explicitly, with
`HEADROOM_DISABLE_KOMPRESS=1` as the structural-only option.
- **Both wire formats documented as accepted**, and removed
`anthropic-sdk.mdx`'s claim that OpenAI format is "the compression
engine's native format" — the exact misconception that prompted this
work. The SDK's conversion is now framed as an SDK choice, not an API
requirement.
- **`litellm.mdx`** had no mention of the endpoint at all, despite the
code naming LiteLLM's guardrail as its primary consumer. Added the HTTP
deployment path, the `HEADROOM_COMPRESS_ALLOW_REMOTE` requirement, and
why to leave `config.mode` unset.
- **`index.mdx`** printed `compressionRatio * 100` labelled "Saved …%",
so a 77% saving displayed as **23%**. `api-reference.mdx` already
defined it correctly, so the docs contradicted each other.
- `openai-sdk.mdx`, `wiki/proxy.md`, `wiki/typescript-sdk.md` — same
corrections; dropped "any HTTP client", "Cloud", and a CacheAligner
claim (it is detector-only).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/ruff check headroom/ tests/ --exclude headroom/dashboard/templates
All checks passed!

$ .venv/bin/mypy headroom/
Success: no issues found in 511 source files

$ python -m pytest tests/test_compress_route_tokenizer_by_model.py \
    tests/test_proxy_compress_endpoint.py tests/test_compress_api.py \
    tests/test_platform_stabilization_functional.py tests/test_proxy_eager_preload_bind.py -q
99 passed, 2 warnings in 47.15s
```

Broader sweep (`-k "compress or litellm or gateway or guardrail"`):
**1625 passed, 4 failed** — all 4 pre-existing, verified by stashing
this diff and re-running on clean `main` (2 strands hook tests, 1 codex
WS semaphore-tail timing test, 1 unrelated local WIP test).

## Real Behavior Proof

- **Environment:** macOS 26.4 arm64, Python 3.12.6, repo `.venv`, branch
rebased on `upstream/main`.

**(1) Before → after, same request** (60-line grep payload in an
Anthropic `tool_result`):

| model | before | after |
| --- | --- | --- |
| `claude-sonnet-4-6` | `before=28 saved=0` | `before=1223 saved=58` |
| `bedrock/anthropic.claude-3-5-sonnet` | `saved=0` | `before=1037
saved=59` |
| `my-gateway/big-model` (alias) | `saved=0` | `before=1037 saved=59` |
| `gemini-2.5-pro` | `saved=0` | `before=1036 saved=59` |
| `gpt-4o` + OpenAI shape | `before=1225 saved=58` | `before=1225
saved=58` (unchanged) |

All three `config.mode` values verified for each. Response shape
preserved: `type=tool_result`, `tool_use_id` intact.

**(2) Counter-level root cause**, 6.8 KB body, `count_message()`:

```text
OpenAITokenCounter    string-content -> 1406    tool_result block -> 5
registry (BaseTokenizer) claude       tool_result=408  thinking=418  mcp_tool_result=421
                                      web_search_tool_result=421  document=422
```

**(3) Every documented behavior asserted against the running app** — 13
checks, all PASS: 400s for missing `messages`/`model`, invalid
`config.mode`, and all four invalid `frozen_message_count` forms; 200
for valid ones; non-dict `config` ignored; bypass and empty-messages
omit `transforms_summary`; success returns exactly the 8 documented
keys.

- **Not tested:** the docs site was not built (`docs/node_modules`
absent) — MDX was checked for balanced `<Callout>` tags only, so a
reviewer with the site running should eyeball rendering. No live
gateway/Kong request; verification is via `TestClient` against the real
ASGI app.
- **Note:** `HEADROOM_DISABLE_KOMPRESS` is read into `ProxyConfig` at
`server.py:4919` and by the CLI, not by `create_app(ProxyConfig(...))`
directly — I confirmed `disable_kompress=True` does reach the derived
pipeline.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-08-03 12:20:33 -07:00
Tejas Chopra e0ce4b1d48 fix: remove rtk and lean-ctx CLI context tools (#2677)
## Description

Removes both third-party CLI context tools — **rtk** and **lean-ctx** —
and with them the context-tool selector itself. Headroom no longer
downloads, installs or configures either one, and there is no
replacement.

The previous pass (#2344) gated only three entry points inside
`headroom/cli/wrap.py`. That left the feature reachable in practice:

| Gap | Effect |
|---|---|
| `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global
--auto-patch` from bash/PowerShell, **bypassing the Python gate
entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook
regardless of `HEADROOM_RTK` |
| `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was
broken by default**: `rtk_required=True` met a gate returning `None` →
`SystemExit(1)`. Invisible because all 8 openhands tests patched
`_ensure_rtk_binary` to a fake path |
| `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to
`rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker
polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) |
| No cleanup path | Nothing removed artifacts an earlier default had
installed, so a machine that once ran the old default kept rtk in the
loop forever (#1669, #1955) |

Also worth noting: the rtk binary download had **no SHA or signature
verification** — only `rtk --version` as a smoke test.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

**Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages,
`headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` /
`_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` /
`--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap
subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the
dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine
getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers,
`benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path
filters.

**Fails loudly, not silently** — `--context-tool` / `--no-context-tool`
/ `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in
shell profiles, aliases and CI jobs, and accepting them as a no-op would
read as Headroom having quietly stopped working. The installers reject
them too, which matters more than it looks: their arg parsers forward
the first unknown flag **and everything after it** to the wrapped tool,
so a leftover `--no-rtk` would have silently swallowed a following
`--port` and then been ignored downstream.

**New `headroom/context_tool_cleanup.py`** — deleting the code cannot
help a machine that already ran the old default, since the hooks,
binaries and injected guidance are durable on disk.
`purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and
removes the registered hook entries, the generated hook scripts, the
Headroom-managed `~/.local/bin` symlinks, the vendored
`~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server
entry and the marker-fenced instruction blocks. Deliberately
conservative: idempotent, **skips** a malformed config rather than
overwriting it, and only unlinks a symlink resolving inside Headroom's
own bin dir so a user's own build is untouched. It reports on
**stderr**, because `wrap/unwrap openclaw --prepare-only` emit
machine-readable JSON on stdout as their entire contract. Skipped for
`wrap selfheal` (runs from a SessionStart hook; must not race Claude
Code's writer for `~/.claude.json`) and for `--help`, which must stay
read-only.

**Client-config hardening** (discovered while investigating a "corrupted
Serena settings file" report) — `wrap.py` reset a settings file to `{}`
when an existing file would not parse, then wrote that back. One
hand-edited typo or a transient `EACCES`/`EINTR` on a valid file
destroyed the user's `permissions`, `env` and `hooks`, on **every
`headroom wrap claude`**. It now refuses to write. Separately,
`fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`),
fixing all 14 non-atomic client-config writes at once; it follows
symlinks rather than replacing them (dotfile managers) and preserves an
existing file's mode.

**Deliberately kept** — `rtk` stays in the wrapper-peel list in
`transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as
shell-command grammar, so `rtk cat f` is still classified as a file read
for anyone running their own rtk install, which the purge intentionally
leaves alone.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates
All checks passed!

$ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates
1255 files already formatted

$ mypy headroom/
Success: no issues found in 508 source files

$ pytest tests/test_context_tool_cleanup.py -q
11 passed

$ pytest tests/test_fsutil.py -q
12 passed

$ pytest tests/test_cli/test_wrap_codex.py -q            # 89 tests
89 passed in 431.68s
$ pytest tests/test_cli/test_wrap_opencode.py -q
39 passed in 257.46s
$ pytest tests/test_cli/test_wrap_helpers.py -q
45 passed
$ pytest tests/test_paths.py -q
75 passed
$ pytest tests/test_cli/test_unwrap_claude.py -q
14 passed
$ pytest tests/test_proxy_savings_history.py -q
39 passed
$ pytest tests/test_cli/test_wrap_copilot.py -q
27 passed
$ pytest tests/test_cli/test_wrap_zcode.py -q
20 passed
$ pytest tests/test_subscription_tracker.py -q
9 passed
$ pytest tests/test_proxy_dashboard_stats_cache.py -q
5 passed, 1 skipped
```

Repo-wide grep for 14 removed symbols (`headroom.rtk`,
`headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`,
`_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`,
`wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`,
`tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`,
`*.html`: **zero hits**.

Notable test changes: `test_wrap_openhands.py` no longer patches
`_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0
unpatched — the regression that was previously masked.
`test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed
(every test drove RTK instruction injection). A new
`test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed`
proves a pre-removal `subscription_state.json` still loads.

## Real Behavior Proof

- **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @
this branch, real `~/.headroom` and `~/.claude` on the dev machine.
- **Exact command / steps and observed result:**

```text
# 1. Retired flag fails loudly instead of silently no-op'ing
$ headroom wrap codex --prepare-only --context-tool rtk
Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they
rewrote shell commands through a third-party binary Headroom no longer manages.
Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL;
`headroom wrap` uninstalls what they left behind on first run.

$ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only
Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ...

# 2. install.sh rejects the retired flags (extracted parse_wrap_args harness)
['--no-rtk', '--port', '9999']   rc=1  ERROR: CLI context tools ... Drop --no-rtk
['--context-tool=rtk']           rc=1  ERROR: CLI context tools ... Drop --context-tool
$ bash -n scripts/install.sh   # syntax OK

# 3. Purge ran against the real machine, which had all the orphaned artifacts
$ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..."
  removed ~/.headroom/bin/lean-ctx        (51 MB)
  removed ~/.headroom/bin/rtk             (7.7 MB)
  removed ~/.local/bin/rtk                (symlink into ~/.headroom/bin)
  removed ~/.claude/hooks/rtk-rewrite.sh
  removed 8 lean-ctx-* hook scripts
# ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged
# → ~59 MB reclaimed, no unrelated key touched

# 4. stdout stays machine-readable while the purge reports (planted a fake artifact)
$ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err
$ cat out
{"enabled":true,"config":{"proxyPort":8787,...}}     # parses as JSON
$ cat err
Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk

# 5. --help is inert (planted artifact survives), a real run purges
$ headroom wrap codex --help   → artifact survived: CORRECT
$ headroom wrap openclaw --prepare-only → purged: CORRECT

# 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json
top-level keys 90 -> 90;  projects 19 -> 19;  LOST keys: none
all content outside mcpServers byte-identical: True
```

Dashboard rendered via the Playwright test after the panel removal:
"Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`,
and "Token Usage" reads Before Compression → Proxy Removed → After
Compression with no "Filtered (this session)" row. Nothing below the
removed panel broke.

- **Not tested:** Windows and Linux (macOS only) — `install.ps1` is
verified by brace-balance and inspection, not executed, since no `pwsh`
is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated
but not run; it needs the Docker e2e image. `serena project index`
interaction is exercised in the stacked base PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

**Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge
that first; this PR's base should then be retargeted to `main`, or it
will read as containing that fix too.

**Breaking-change migration for users:**
- Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`,
`--context-tool`, `--no-context-tool` from any alias, script or CI job,
and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error
rather than being ignored, so the failure is immediate and
self-explaining.
- Previously-installed artifacts are purged automatically on the next
`wrap`/`unwrap`; no manual cleanup needed.
- `headroom perf --json` no longer carries a `cli_filtering` key, and
`/stats` no longer returns a `context_tool` section.

**Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from
`README.md`,
`docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`,
`docs/observability.md` and the matching `wiki/` pages.
`REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED
rather than deleted, to keep the planning record.

**Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as
dead code — its only caller was the `except KeyboardInterrupt` guarding
the binary download, so with no download there is nothing slow left to
interrupt.
2026-07-30 22:59:41 -07:00
Dávid BalatoniandJerrettDavis 5424e99a65 Clarify uv tool install path on macOS (#1196)
## Description

Clarifies the recommended install path for the Headroom CLI on macOS
Apple Silicon and Linux. The docs now prefer `uv tool install --python
3.13 "headroom-ai[all]"` for host-level CLI use, keep `pip install`
scoped to Python project environments, and call out absolute executable
paths for MCP clients that do not inherit interactive shell `PATH`.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added `uv tool install --python 3.13` guidance to the README, docs
install page, quickstarts, and wiki install pages.
- Documented `uv tool update-shell` for shells that cannot find the
installed `headroom` command.
- Clarified absolute MCP server command paths for clients that do not
inherit the interactive shell `PATH`.
- Pointed Intel macOS users at the Docker-native install path until
native wheel support lands.

## Testing

Describe the tests you ran to verify your changes:

- [ ] Unit tests pass (`pytest`) - not run; docs-only change.
- [ ] Linting passes (`ruff check .`) - not run; docs-only change.
- [ ] Type checking passes (`mypy headroom`) - not run; docs-only
change.
- [ ] New tests added for new functionality - not applicable.
- [x] Manual testing performed
- [x] `git diff --check upstream/main...HEAD`

## Real Behavior Proof

```bash
$ git diff --check upstream/main...HEAD
# exits 0; no whitespace errors
```

`npm --prefix docs run types:check` was also attempted. It regenerated
MDX and route types successfully, then failed in existing docs app code
because `@/lib/...` imports cannot resolve from files such as
`app/(home)/layout.tsx`, `app/api/search/route.ts`, and
`components/button.tsx`. This PR only changes `README.md`,
`docs/content/docs/installation.mdx`,
`docs/content/docs/quickstart.mdx`, and `wiki/*.md` files.

## Review Readiness

- [x] Draft PR; docs wording and install-path accuracy are ready for
review.
- [x] No code or runtime files changed.
- [x] Known docs type-check blocker is documented above.

## Test Output

```bash
$ git diff --check upstream/main...HEAD
# no output
```

```text
$ npm --prefix docs run types:check
[MDX] generated files
✓ Types generated successfully
app/(home)/layout.tsx(2,29): error TS2307: Cannot find module @/lib/layout.shared or its corresponding type declarations.
...
components/button.tsx(4,20): error TS2307: Cannot find module @/lib/cn or its corresponding type declarations.
```

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- not applicable; docs-only change.
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works - not applicable; docs-only change.
- [ ] New and existing unit tests pass locally with my changes - not
run; docs-only change.
- [ ] I have updated the CHANGELOG.md if applicable - not applicable.

## Screenshots (if applicable)

Not applicable.

## Additional Notes

The PR remains a draft while docs verification is limited by the
existing docs app `@/lib/*` resolution issue.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 21:06:30 +00:00
Manmit Singh 942e916368 feat(cli): add headroom inspect to view original vs compressed content (#1595)
## Description

Headroom exposes plenty of *quantitative* compression telemetry (token
counts, ratios, `headroom perf`, `/metrics`) but no way to actually
**see what the compressor changed** in the content. That makes it hard
to trust compression or debug a quality regression ("did it drop
something I cared about?").

This adds a `headroom inspect` command (the issue's Option 1). It reads
the proxy's existing loopback `/transformations/feed` endpoint — which
already carries the pre/post-compression message snapshots when the
proxy runs with `--log-messages` — and renders, per request, the
original vs compressed content for each message with the changed
segments highlighted. No new dependencies (stdlib `difflib`).

```
headroom inspect                 # inspect the most recent request
headroom inspect --last 5        # the 5 most recent
headroom inspect --full          # include unchanged messages
headroom inspect --format json   # raw feed for offline tooling
```

Per request it shows the model, per-request token counts + savings, the
transforms applied, and a colorized unified diff of each changed message
(red = removed, green = added). Clear errors when no proxy is reachable
or when the proxy wasn't started with `--log-messages`.

Side-by-side / interactive rendering (the fuller form of Option 1) can
follow as a polish pass; this lands the core "see what changed"
capability on data Headroom already captures.

Closes #1267

## Type of Change

- [x] New feature (non-breaking change that adds functionality)

## Changes Made

- `headroom/cli/inspect.py`: new `inspect` command +
content-flattening/diff-render helpers.
- `headroom/cli/__init__.py`, `headroom/cli/main.py`: register the
command.
- `tests/test_cli_inspect.py`: unit tests (content extraction, the
no-proxy / no-`--log-messages` / empty-feed paths, text render, json
output).
- `wiki/cli.md`: document the command + options.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] New tests added

### Test Output

```text
$ pytest tests/test_cli_inspect.py -q
7 passed

$ ruff check headroom/cli/ tests/test_cli_inspect.py
All checks passed!
```

## Real Behavior Proof

- Environment: repo main @ HEAD, local venv
- Exact command / steps: invoked the `inspect` command against a mocked
`/transformations/feed` payload (one request, a user message with a line
removed by SmartCrusher).
- Observed result: header shows `req-1 gpt-4o`, `tokens 100 → 40 (saved
60, 60.0%)`, `transforms: SmartCrusher`, and a unified diff with the
removed line on the original side; no-proxy and missing-`--log-messages`
cases raise actionable errors; `--format json` emits the raw feed.
- Not tested: live end-to-end against a real proxy with `--log-messages`
(the data source — the feed endpoint — is exercised via the mocked
payload that mirrors its shape).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove the feature works
2026-07-15 19:58:38 +00:00
nangsontayandJerrettDavis e5b3a634df fix(proxy): dedupe Codex WS request logging for accurate mixed-provider dashboards (#2189)
## Description

Running Claude Code (Anthropic) and Codex (OpenAI) against the **same**
Headroom proxy instance on one port produced incorrect, unstable
dashboard data. The proxy core is provider-isolated and
multi-provider-safe by design; the defect was in the observability
layer. The Codex `/v1/responses` **WebSocket** handler was the only path
in the proxy that wrote to the request logger by hand instead of through
the unified `emit_request_outcome` funnel, and it did so twice per
session close: the per-turn funnel record plus an unconditional
cumulative session-summary `RequestLog`. This PR removes the duplicate
summary log so Codex WS emits exactly one request log per turn, matching
the HTTP provider paths.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Dropped the duplicate cumulative session-summary `RequestLog` in the
Codex WS handler while preserving the per-turn `emit_request_outcome`
path.
- Preserved gated `request_messages` and `turn_id` on residual outcomes
so dashboard telemetry keeps the useful attribution without
double-counting tokens.
- Ensured explicit `--anyllm-provider` wins over a leaked
`HEADROOM_ANYLLM_PROVIDER` environment variable.
- Registered retry delay settings that had drifted out of the settings
registry.
- Hardened tests against developer-shell `HEADROOM_*` /
`ANTHROPIC_CUSTOM_HEADERS` leakage and stabilized several focused
proxy/wrap test fixtures.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ .venv/bin/pytest tests/ -q -p no:cacheprovider
8529 passed, 537 skipped, 5831 warnings in 276.25s (0:04:36)

$ .venv/bin/ruff check <touched files>
All checks passed!
```

## Real Behavior Proof

- Environment: macOS (Darwin 25.4.0), Python 3.13.14, pytest 9.0.3, ruff
via project venv, branch `fix/multi-provider-runtime`.
- Exact command / steps: Ran the full test suite without pytest cache
provider and Ruff on all touched files; used `git stash` to confirm the
stale fake-config failures pre-existed this change.
- Observed result: Full suite passed with no failures; Ruff passed;
Codex WS now routes end-of-session logging through
`emit_request_outcome`, emitting one request log per turn with the same
accounting model as Anthropic HTTP turns.
- Not tested: Live simultaneous Claude + Codex dashboard run. `mypy
headroom` was not run to completion; a scoped run reported one
pre-existing `settings_store.py:470` coercion error outside this diff.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A - server-side observability fix; no UI markup changed.

## Additional Notes

- The proxy's multi-provider routing, header/auth isolation, and
per-model cache keying are already correct and unchanged here; only the
WS observability write path was double-counting.
- Architectural assessment:
`plans/reports/research-260714-0004-multi-provider-upstream-compression-report.md`;
root-cause + resolution trail:
`plans/reports/debug-assessment-260714-0011-dashboard-instability-mixed-claude-codex-report.md`.
- No live simultaneous Claude + Codex dashboard run was performed;
validation is from test coverage and code review of the WS logging path.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-07-15 18:18:34 +00:00
Tejas ChopraandClaude Opus 4.8 60af15f96f feat(content-router): lossless-first dispatch, cross-turn dedup, and A7 lossy-after-fold (#1818)
## Description

<!-- Briefly explain the change and why it is needed. -->

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- 

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
# Paste relevant command output or artifact links here
```

## Real Behavior Proof

- Environment:
- Exact command / steps:
- Observed result:
- Not tested:

## Review Readiness

- [ ] I have performed a self-review
- [ ] This PR is ready for human review

## Checklist

- [ ] My code follows the project's style guidelines
- [ ] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

Add screenshots to help explain your changes.

## Additional Notes

<!-- Mention any N/A checklist items, tradeoffs, follow-ups, or
maintainer context. -->

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 08:32:06 -07:00
ParideboyandClaude Fable 5 1fc5e3d4da docs(proxy): correct --code-aware default to disabled (#1710)
## Description

The wiki proxy page (`wiki/proxy.md`, which feeds the published docs
site) claimed `--code-aware` defaults to **true**. The CLI deliberately
defaults it to **disabled**: `headroom/cli/proxy.py` resolves the paired
flag to off unless `--code-aware` is passed or
`HEADROOM_CODE_AWARE_ENABLED` is truthy, and the Click help text plus
`docs/content/docs/proxy.mdx` and `wiki/cli.md` already document it as
disabled. This PR aligns the one remaining stale table row and collapses
the self-contradictory separate `--no-code-aware` row into a single
paired-flag entry, matching the style used in
`docs/content/docs/proxy.mdx`.

Fixes #1700

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `wiki/proxy.md`: replaced the two flag-table rows claiming
`--code-aware` default `true` / `--no-code-aware` default `false` with
one `--code-aware` / `--no-code-aware` row documenting the actual
default (`disabled`), the `headroom-ai[code]` requirement, and the
`HEADROOM_CODE_AWARE_ENABLED=1` env opt-in.

## Testing

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -c "
from click.testing import CliRunner
from headroom.cli.proxy import proxy
r = CliRunner().invoke(proxy, ['--help'])
print([l.strip() for l in r.output.splitlines() if 'code-aware' in l][0])
"
--code-aware / --no-code-aware  Enable/disable AST-based code compression.

$ grep -n "code-aware" wiki/proxy.md
77:| `--code-aware` / `--no-code-aware` | disabled | Enable or disable AST-based code compression. Requires `headroom-ai[code]` (env: HEADROOM_CODE_AWARE_ENABLED=1 to enable) |
```

## Real Behavior Proof

- Environment: Windows 11, local checkout at `upstream/main` (9fbd47ba),
Python 3.13.
- Exact command / steps: `headroom proxy --help` (via Click test runner)
to confirm the CLI help says "Default: disabled"; inspected
`headroom/cli/proxy.py` flag resolution (explicit flag →
`HEADROOM_CODE_AWARE_ENABLED` → off); `grep -rn "code.aware" wiki/
docs/` to find every doc stating a default.
- Observed result: CLI default is disabled;
`docs/content/docs/proxy.mdx:61` and `wiki/cli.md:255-256` already say
disabled/off; only `wiki/proxy.md:77-78` claimed true. After the change
the table matches actual behavior.
- Not tested: rendered docs-site build (content-only table edit).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 14:19:12 -07:00
DMandAI Agent 64783d8824 fix: skip Magika backend on x86 CPUs without AVX2 (#1162)
## Description

Adds a narrow runtime AVX2 guard before initializing the Magika/ONNX
Runtime detector on x86/x86_64. On x86/x86_64 CPUs without AVX2,
Headroom falls back to existing non-Magika detection tiers instead of
crashing during ONNX Runtime initialization. AVX2-capable systems retain
existing behavior.

Refs #1005

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Adds a Magika/ONNX Runtime CPU support guard before `Session::new()`.
- Returns a normal Magika init error on x86/x86_64 hosts without AVX2,
allowing the existing detection chain to fall through to non-Magika
tiers.
- Keeps AVX2-capable x86/x86_64 behavior unchanged.
- Does not apply the x86-specific AVX2 gate on non-x86 targets.
- Adds CPU-aware Rust tests and a short troubleshooting note.

## Testing

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ cargo test -p headroom-core --lib --locked
833 passed; 0 failed; 1 ignored

$ cargo test --workspace --locked
passed

$ cargo clippy -p headroom-core --locked -- -D warnings
clean
```

## Real Behavior Proof

- Environment: x86_64 Linux host with AVX but no AVX2 (Intel Xeon
E5-2697 v2 on Proxmox), local build from this branch.
- Exact command / steps: `python -X faulthandler -c 'from headroom._core
import detect_content_type; print(detect_content_type("hello world"))'`
- Observed result: before — process exited with `Fatal Python error:
Illegal instruction`; after — command completed successfully returning
`DetectionResult(content_type="text", ...)`, and full `cargo test -p
headroom-core --lib --locked` passed with 833/0/1.
- Not tested: generic no-AVX CPUs, alternate ONNX Runtime builds,
non-x86 platforms.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

This partially addresses #1005 by handling one concrete native crash
class: the Magika detector initializes ONNX Runtime through ort/ort-sys,
whose precompiled runtime can contain AVX2-family instructions. On
AVX-only x86_64 hosts, that initialization can SIGILL before Headroom
can fall back.

Scope:

- This does not introduce generic no-AVX wheels.
- This does not redesign Rust-core packaging.
- This does not disable the Rust core globally.
- This only prevents the Magika/ONNX detector tier from loading on
x86/x86_64 CPUs where AVX2 is unavailable.
- Non-Magika detection tiers continue to run.
- On non-x86 targets, this x86-specific AVX2 gate is not applied.

Changelog omitted: small native detector fallback fix with no public API
change.

Co-authored-by: AI Agent <ai-agent@homelab.internal>
2026-06-30 13:34:17 -05:00
Tejas Chopra 10251b65ca docs: sync README + benchmarks with code (drop retired IntelligentContext/RollingWindow) (#1545)
## Description

Sync the docs with the code after the live-zone realignment. The
`IntelligentContextManager` (ICM), `RollingWindow`, and scoring modules
were deleted in PR #350 (May 2026), but the README and benchmark
docstrings still advertised them as live, and an example still imported
the deleted module (broken on run). This fixes the README + benchmarks
and removes the dead example.

I validated the README against the code with three parallel
static-analysis sub-agents (features/architecture,
CLI/extras/wrap-matrix, public API/integrations). Most of the README
checked out accurate; only the items below were stale/wrong.

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- README: removed the `IntelligentContext` bullet and
`IntelligentContext / RollingWindow` from the transforms list (both
deleted in PR #350).
- README: standardized `Kompress-base` -> `Kompress-v2-base` to match
the HF model id `chopratejas/kompress-v2-base` and the existing badges
(diagram re-aligned).
- README: corrected the CodeCompressor language list to match the
`CodeLanguage` enum (added TS, C, Perl).
- README: softened the unanchored "6 algorithms" tagline to
"content-aware compressors".
- README: Cortex Code is library-mode only — there is no `headroom wrap
cortex`, so the compatibility-matrix row no longer shows a wrap
checkmark.
- Deleted `examples/test_intelligent_context_toin_ccr.py` — it imported
the deleted `IntelligentContextManager` (ImportError on run) and is
unreferenced.
- Removed stale `RollingWindow` mentions from benchmark
docstrings/comments (`benchmarks/__init__.py`, `bench_transforms.py`,
`bench_latency.py`, `scenarios/conversations.py`); the accurate PR-B1
retirement comment is kept.

## Testing

- [ ] Unit tests pass (`pytest`) — N/A, docs/docstring + example
deletion only
- [x] Linting passes — `ruff check` clean on all changed benchmark files
- [ ] Type checking passes — N/A (no type-relevant changes)
- [ ] New tests added — N/A
- [x] Manual testing performed — see Real Behavior Proof

### Test Output

```text
$ ruff check benchmarks/__init__.py benchmarks/bench_transforms.py benchmarks/bench_latency.py benchmarks/scenarios/conversations.py
All checks passed!

# stale refs remaining in README/benchmarks (excluding accurate retirement notes):
$ grep -rn "IntelligentContext|RollingWindow|Kompress-base" README.md benchmarks/ | grep -v retire
(only benchmarks/bench_transforms.py:362 — the accurate PR-B1 retirement comment)

# deleted example is unreferenced anywhere:
$ grep -rn "test_intelligent_context_toin_ccr" --include=*.md --include=*.yml --include=*.py .
(no hits)
```

## Real Behavior Proof

- Environment: macOS (darwin, arm64), Python 3.12 `.venv`, ruff 0.14.x,
repo at branch `docs/sync-readme-with-code` off latest `main`.
- Exact command / steps: (1) three parallel sub-agents
grep/Read-validated README claims vs `headroom/`, `pyproject.toml`,
`sdk/typescript/`; (2) directly verified each flagged mismatch
(`CodeLanguage` enum, `HF_MODEL_ID`, absence of
`IntelligentContext`/`RollingWindow` classes); (3) confirmed the example
imports a deleted module and is unreferenced; (4) `ruff check` on
changed benchmark files; (5) re-grepped README + benchmarks for any
remaining stale refs.
- Observed result: README and benchmark docstrings now match the code;
the only surviving `RollingWindow` string is the accurate retirement
comment; the broken example is removed; ruff passes; the ASCII
architecture diagram still aligns after the `Kompress-v2-base` rename.
- Not tested: rendering of the README on GitHub/PyPI (text-only change);
the separate `docs/content/` and `wiki/` doc sets (see Additional Notes
— out of scope for this PR).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective — N/A
(docs/example cleanup)
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md — N/A (Release Please
auto-generates from the conventional commit)

## Additional Notes

**Larger related finding (NOT in this PR):** the published docs site
(`docs/content/docs/*.mdx`) and the `wiki/*.md` set still document
`IntelligentContextManager`, `RollingWindow`, `RollingWindowConfig`,
`IntelligentContextConfig`, and `ScoringWeights` as live API — with
`from headroom import RollingWindow` / `from headroom.transforms import
IntelligentContextManager` code examples that would `ImportError`. It is
half-migrated (a couple of `.mdx` files already note "removed in 0.9.x"
while neighbors still teach it as current). This is ~15 files and the
fixes require rewriting examples to the live-zone model, not just
deletions — recommended as a focused follow-up PR rather than bundling
it here.
2026-06-28 22:36:41 -07:00
Tejas Chopra c2fc4d3753 fix(ccr): make headroom_retrieve a hash-only full-content lookup (#1532)
The optional `query` parameter on headroom_retrieve routed retrieval
through CompressionStore.search(), which BM25-scored the items inside a
single cached blob and dropped everything below a 0.3 relevance floor.
On small per-blob corpora with conversational queries this returned an
empty result the large majority of the time, so the LLM saw "nothing
found" for content that was actually present — pushing users to turn
compression off entirely.

Retrieval is fundamentally a hash lookup (this already matches the Rust
proxy's CCR store, which is put/get only — "no BM25 search"). Remove the
query/search path end to end and always return the full original
content:

Core (Python proxy):
- tool schemas (anthropic/openai/google) drop the `query` property
- parse_tool_call returns the hash (str | None) instead of (hash, query)
- response handler, proxy POST/GET/tool-call handlers, the MCP retrieve
tool, and the streaming feedback recorders retrieve by hash only
- proactive context-tracker expansion always restores full content
- delete CompressionStore.search() and its BM25 machinery (the bm25
module stays — it is still used by relevance/)
- CCRToolCall.query, CCRToolResult.was_search, and
ExpansionRecommendation.expand_full/search_query are removed

Plugins (advertised a now-defunct query param to the LLM):
- hermes (Python), openclaw + opencode (TypeScript) retrieve tools drop
`query` from their schemas, signatures, request URLs, and tests

Benchmarks/docs:
- ccr_regression + adversarial benchmarks switch from store.search() to
full hash retrieval (search input-injection tests repurposed to the
hash, the only remaining input surface)
- wiki/ARCHITECTURE.md, wiki/ccr.md, docs/content/docs/ccr.mdx,
config.py and store docstrings updated to describe hash-only retrieval

Tests updated to assert full-content retrieval and guard the removed
surface; the full CCR/proxy/store/TOIN suite passes. ruff + mypy clean.

## Description

<!-- Briefly explain the change and why it is needed. -->

Closes #

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- 

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
# Paste relevant command output or artifact links here
```

## Real Behavior Proof

- Environment:
- Exact command / steps:
- Observed result:
- Not tested:

## Review Readiness

- [ ] I have performed a self-review
- [ ] This PR is ready for human review

## Checklist

- [ ] My code follows the project's style guidelines
- [ ] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

Add screenshots to help explain your changes.

## Additional Notes

<!-- Mention any N/A checklist items, tradeoffs, follow-ups, or
maintainer context. -->
2026-06-28 10:32:43 -07:00
Tejas Chopra a639540959 chore: remove committed node_modules + stray/internal markdown (repo hygiene) (#1528)
## Description

Repo hygiene for a public OSS project: removes committed `node_modules`,
stray/internal/draft markdown, and commercial-surface references —
keeping every real doc (the published docs site, the wiki guides, and
all component READMEs) intact. Every file was content-audited before
removal, and load-bearing files were verified against the code/CI and
kept.

Net: **1,695 files changed, +23 / −266,409** (the deletions are
dominated by a committed `node_modules` tree).

Closes # (no tracking issue)

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [x] Code refactoring (no functional changes)

## Changes Made

**Removed (verified to have no code/CI dependencies):**
- `examples/vercel-ai-sdk-pr/` — 1,649 committed `node_modules` files
(zero example source); `node_modules/` added to `.gitignore`.
- `docs/spec/` (23 draft "Living Specification" files — orphaned,
`1.0.0-draft`, drifted from the code), `docs/superpowers/` (2 agent
plans), `docs/proposals/` (2 internal/commercial memos).
- 6 orphan `docs/*.md` (auth-modes, bedrock,
claude-code-vertex-headroom, cortex-code, output-token-reduction-guide,
rtk-loop-weighting).
- `PR.md` (committed PR draft), `ENTERPRISE.md`, `.github/FUNDING.yml`.

**Content scrubs:**
- Removed unreleased "Headroom Cloud" / `api.headroom.ai` / `hr_`
references from `configuration.mdx`, `wiki/configuration.md`,
`wiki/typescript-sdk.md`, `sdk/typescript/README.md` (reworded to
neutral, accurate phrasing).
- Dropped a stale "awaiting maintainer before merge" line from
`plugins/headroom-oauth2/SPEC.md`; tidied `.gitignore` comments (kept
the protective `headroom-managed/` ignore rule).
- Fixed the now-dangling links into removed files (README
nav/`output-token-reduction` link, `scripts/README`, `wiki/vertex`).

**Explicitly KEPT (load-bearing — would orphan in-code citations if
removed):**
- `.changelog.md` — consumed by `.github/workflows/release.yml` (read as
the release-notes file).
- `REALIGNMENT/`, `docs/observability.md`, `docs/rtk-architecture.md`,
`wiki/plans/`, `TESTING-copilot-subscription.md` — referenced by the
Rust core / Python / tests as design docs.

## Testing

- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [ ] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# Docs/markdown + .gitignore only — no Python/Rust source changed, so the
# behavioral test suite is unaffected. Verified the cleanup did not orphan
# references or break the published docs site:

$ git ls-files 'docs/content/docs/*.mdx' | wc -l      # published site intact
42
$ # meta.json nav unchanged; no published page removed.

$ grep -rnI "Headroom Cloud|api.headroom.ai|'hr_" $(git ls-files '*.md' '*.mdx')
>>> none

$ # dangling refs to removed files (excl pre-existing P0/P2 spec stubs that
$ # never existed in git): none remaining.
```

## Real Behavior Proof

- Environment: macOS, local git clone of the repo (markdown/.gitignore
changes only — no runtime).
- Exact command / steps: 4 read-only content-audit agents classified
every `.md`/`.mdx` file; each removal candidate was cross-checked
against the codebase (`grep` for citations in `.rs`/`.py`/tests,
workflows, and configs); only files with no dependents were removed; the
tree was re-grepped after removal to confirm no new dangling references;
verified the published docs site page count (`git ls-files
'docs/content/docs/*.mdx' | wc -l` = 42, unchanged).
- Observed result: the 42-page published docs site and all wiki guides
are untouched; no source or workflow references a removed file;
`.changelog.md` (consumed by release.yml) and the code-cited design docs
were detected as dependencies and kept; the committed `node_modules`
tree is removed and `node_modules/` is gitignored so it can't be
re-committed; zero "Headroom Cloud"/`headroom.dev` references remain.
- Not tested: N/A — no executable code changed (only markdown, `.mdx`,
and `.gitignore`), so the behavioral test suite is unaffected.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

- This branch deletes `.github/FUNDING.yml` while PR #1526 edits it —
the two will be sequenced at merge (delete wins).
- A follow-up option (not in this PR): also remove the internal design
docs that are currently cited by the code (`REALIGNMENT/`,
`docs/observability.md`, `docs/rtk-architecture.md`, `wiki/plans/`) —
that requires scrubbing ~15–20 in-code citations so nothing dangles, so
it's deliberately deferred.
- Untracked local working files (`benchmarks/hf_pilot/`,
`tools/copilot-test/`) are intentionally left out of git (not
committed).
2026-06-27 23:32:54 -07:00
Tejas Chopra bd76235f5c fix(cli): harden all CLI surfaces + fix docs accuracy (#1491)
## Summary

Full CLI audit + documentation accuracy pass. All 5 commits on this
branch:

### CLI Hardening (4 commits)
- **Clean errors instead of tracebacks**: corrupt manifests, missing
Docker, malformed JSONL, bad `--profile`, invalid env-var values all now
raise `click.ClickException` with helpful messages
- **Range validation**: ~25 numeric flags across 10 files now use
`click.IntRange`/`FloatRange` — `--port 0`, `--hours -1`, `--limit 0`
etc. produce clean usage errors instead of silent wrong behavior
- **Flag combination warnings**: conflicting combos (`--no-rate-limit` +
`--rpm`, `--no-optimize` + `--target-ratio`, `--telemetry` +
`--no-telemetry`) emit yellow warnings on stderr
- **`memory --db-path` default fixed**: was resolving to
`headroom_memory.db` (wrong bare file); now uses project store
`./.headroom/memory.db` if present, else `~/.headroom/memory.db`
- **`memory list --search` + filters**: `--scope`/`--session`/`--since`
were silently ignored when `--search` was also set; now filters are
applied to search results
- **`learn --verbosity --apply` now works**: the output shaper is off by
default (`HEADROOM_OUTPUT_SHAPER`); `--apply` now hot-enables it via
`POST /admin/runtime-env` on a running proxy, or prints explicit `export
HEADROOM_OUTPUT_SHAPER=1` instructions when no proxy is running
- **`perf --hours` overflow**: `1e9` hours no longer raises
`OverflowError`; treated as "all data"
- **`evals memory --categories` invalid input**: `abc,1,2` now raises
`BadParameter` instead of a raw `ValueError` traceback

### Documentation (1 commit, 20 files)

Corrected factual errors found by 3 parallel audit agents across root
docs, wiki, and the published Fumadocs site:

**Critical (caused runtime errors or wrong behavior if followed):**
- `simulation.mdx`: `plan.transforms_applied` -> `plan.transforms`;
`plan.savings_percent` -> computed from available fields (both raised
`AttributeError`)
- `shared-context.mdx`: `import { SharedContext } from "headroom"` ->
`"headroom-ai"` (5x `ImportError`)
- `claude-code-azure-foundry.mdx`: `pip install headroom` -> `pip
install headroom-ai`
- `api-reference.mdx` + `configuration.mdx`: `from headroom import
GoogleProvider` -> `from headroom.providers import GoogleProvider`
- `ccr.mdx`: CCR TTL default 300s -> 1800s (30 min)

**Fabricated flags removed:**
- `wiki/proxy.md` + `wiki/cli.md`: `--no-intelligent-context`,
`--no-intelligent-scoring`, `--no-compress-first` (none exist); replaced
with real CCR flags
- `wiki/configuration.md`: `--no-ccr-responses`, `--no-ccr-expansion`
(none exist); replaced with real flags
- `wiki/troubleshooting.md`, `wiki/metrics.md`,
`docs/troubleshooting.mdx`: `headroom proxy --log-level debug` (flag
doesn't exist)

**Stale content corrected:**
- `llms.txt`: telemetry stated as enabled-by-default (it's opt-in); wrap
list had 5 tools (now 11)
- `README.md`: compatibility matrix added 5 missing `wrap` targets;
`unwrap`, `doctor`, `init`/`install`, savings-analytics now mentioned
- `SECURITY.md`: supported version table showed 0.2.x (current: 0.27.x)
- `wiki/learn.md`: 5 missing flags added; verbosity shaper-off behavior
documented
- `wiki/quickstart.md`: "Configuration Reference" linked to `api.md`
(wrong) -> `configuration.md`
- `CacheAlignerConfig.enabled` default corrected: `True` -> `False`
- `opencode.mdx`: `--port` default wrong ("random") -> 8787; `openai`
backend removed
- `CONTRIBUTING.md`: broken Markdown table cell fixed
- `docs/meta.json`: `claude-code-azure-foundry` added to nav (was
unreachable orphan page)
- `configuration.mdx`: SDK modes vs proxy `--mode` now clearly
distinguished

## Test plan

- [x] `python -m pytest tests/ -x -q` — 857 passed, 0 failures
- [x] 41-combination CLI smoke test (all flag combos across 8 commands)
— 0 tracebacks
- [x] `ruff check` on all modified Python files — clean
- [x] Docs changes are removals/corrections of fabricated or stale
content; no new claims introduced
2026-06-27 14:48:43 -07:00
Lucas SantosandJD Davis c30ec4cda8 fix: surface output reduction without a restart, and explain $0.00 savings on Python 3.14 (#1296)
## Description

I was running headroom through pipx on Python 3.14 and hit two issues.

The Proxy $ Saved tile was stuck at $0.00 even though tokens were
tracking fine. Pricing comes from litellm, and litellm does not install
on Python 3.14 because of a version lock, so there was just nothing to
price against. Rather than hardcode a price table that goes stale, I
added a `litellm_available` flag to `/stats` and the tile now tells you
to reinstall on 3.13 when pricing isn't there, like the output-shaper
tile already does.

The other one was Output Tokens Saved showing "—" after I turned on the
shaper. The recorder reads the learned baseline once at startup, so if
you run `learn --verbosity --apply` while the proxy is already up it
never gets picked up, and a later flush writes the empty baseline over
the one learn just saved. Now it re-reads the baseline before estimating
and before each flush, so it works without a restart.

Closes # N/A (no tracking issue)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `output_savings.py`: re-read the baseline from disk before estimating
and before each flush, so a baseline learned while the proxy is running
takes effect (and a re-learn with the same sample count too).
- `server.py`: expose a `litellm_available` flag on `/stats`.
- `dashboard.html`: when savings are zero and litellm is missing, point
to reinstalling on 3.13 instead of showing $0.00.
- tests and docs (`test_output_savings.py`, README, metrics, CHANGELOG).

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_output_savings.py -q
34 passed, 1 warning in 0.11s
```

## Real Behavior Proof

- Environment: macOS, Python 3.13 (litellm present) and 3.14 (litellm
absent), running this branch.
- Exact command / steps: record shaped traffic, write a baseline to the
same file while the recorder is live (no restart), then estimate and
flush.
- Observed result: the recorder goes from `available: False` to
`available: True` once the baseline is written mid-run, and keeps it
after a flush. Before this it stayed `False` and the flush reset the
baseline. Raw output:
  ```text
  shaper traffic recorded, baseline not learned yet -> available: False
  learn --apply wrote baseline while proxy up; restart NOT performed
after baseline write -> available: True | method: estimated | pct: 50.3
  baseline kept after flush -> disk samples: 4
  ```
- Not tested: I did not render the tile hint in a browser, I checked the
flag on `/stats` and read the template instead.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

Just a final note, ruff and mypy are clean on what I changed. The
repo-wide `ruff check .` and `mypy headroom` do report a few problems,
but they're in files I didn't touch and already exist on the base
commit, so I left them alone to keep this small. Happy to do a separate
cleanup PR.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-06-26 12:15:42 -05:00
ParideboyandClaude Opus 4.8 8cc5354f51 docs: use headroom-ai package name in install commands (#1014) (#1257)
## Description

Install commands across the docs referenced the unpublished `headroom`
package instead of the published `headroom-ai`, so copy-pasted `pip
install` commands fail. This corrects them to `headroom-ai` (with
extras).

Closes #1014

## Type of Change

- [x] Documentation update

## Changes Made

- `wiki/getting-started.md`: corrected 4 `pip install headroom` commands
to `headroom-ai` (including the `[proxy]`, `[relevance]`, and `[all]`
extras).
- `docs/content/docs/claude-code-vertex.mdx`: fixed the install command
on line 37.
- `SECURITY.md`: fixed the install command on line 47.

## Testing

- [x] Manual verification

### Test Output

```text
$ rg -n "pip install headroom\b" docs wiki SECURITY.md
(no matches — all bare `headroom` install commands now use `headroom-ai`)
```

## Real Behavior Proof

- Environment: Windows 11, repo working tree on branch
fix/docs-1014-headroom-ai-pkg
- Exact command / steps: Grepped the docs tree for `pip install
headroom` before and after the edits.
- Observed result: Before, several occurrences referenced the
unpublished `headroom`; after, only `headroom-ai` remains (the spec doc
reference is intentionally left untouched).
- Not tested: Did not run a live `pip install headroom-ai` against PyPI
in CI.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 19:25:00 -05:00
nervousapps c0745d4161 feat(proxy): add request timeout config (#738)
## Description

Add --request-timeout-seconds CLI flag and HEADROOM_REQUEST_TIMEOUT
environment variable to the headroom proxy command, allowing users to
configure the upstream request timeout (default: 300s). This is useful
for slow providers such as local LLM servers (Ollama, vLLM, llama.cpp)
where the default timeout may be insufficient.

Fixes #737

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added --request-timeout-seconds option to the proxy command with
HEADROOM_REQUEST_TIMEOUT envvar support
- Passed request_timeout_seconds (default: 300s when not specified)
- Added tests for both CLI flag and environment variable paths

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$  pytest tests/test_cli_proxy_env.py -q
45 passed in 3.46s

$  mypy headroom
Success: no issues found in 356 source files

$  ruff check .
All checks passed!
```

## Real Behavior Proof

- *MISSING*

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

Add screenshots to help explain your changes.

## Additional Notes

Follows the existing pattern used by --connect-timeout-seconds.
Environment variable approach is essential for Docker/Kubernetes
deployments where modifying CLI args requires image rebuilds.


<!-- headroom-maintainer-template-completion:start -->

## Description

This PR prepares `feat(proxy): add request timeout config` for review by
documenting the intended change, validation evidence, and remaining
merge-readiness context.

Linked issues: #737

## Type of Change

- [x] Bug fix
- [ ] New feature
- [ ] Documentation
- [ ] Refactor
- [ ] Tests only

## Changes Made

- Commit: feat(proxy): add request timeout config
- Touches `docs/content/docs/configuration.mdx`
- Touches `docs/content/docs/installation.mdx`
- Touches `headroom/cli/proxy.py`
- Touches `tests/test_cli_proxy_env.py`
- Touches `wiki/cli.md`

## Testing

- [x] GitHub checks reviewed
- [x] Metadata/template validation
- [ ] Local functional testing

### Test Output

```text
gh pr view 738 --repo chopratejas/headroom --json statusCheckRollup
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
- PR Governance / template: FAILURE
```

## Real Behavior Proof

- Environment: GitHub PR metadata and checks for `chopratejas/headroom`
PR #738.
- Exact command / steps: Reviewed PR title, commits, changed files,
linked issues, labels, and check rollup; appended this maintainer
template completion block without replacing the author's original
description.
- Observed result: PR body now contains all required governance
sections, checked readiness fields, and a non-placeholder validation
evidence block.
- Not tested: This pass updated PR metadata only; code validation
remains represented by the linked GitHub checks and any author-provided
evidence above.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

<!-- headroom-maintainer-template-completion:end -->
2026-06-22 14:53:14 -05:00
Tejas Chopra b99869778b fix(telemetry): switch anonymous telemetry to opt-in (off by default) (#1223)
## Description

Anonymous usage telemetry was **on by default** (opt-out). This flips it
to **opt-in**: nothing is collected or shipped unless the user
explicitly turns it on. Small change, but it makes "no data leaves the
proxy by default" the actual default rather than something users have to
discover and disable.

Closes # <!-- N/A: no tracking issue -->

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality) — adds
`--telemetry` opt-in flag
- [x] Breaking change (fix or feature that would cause existing
functionality to change) — telemetry no longer runs unless opted in
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `is_telemetry_enabled()` is now **fail-closed**: only explicit
on-values (`on`/`true`/`1`/`yes`/`enable`/`enabled`) enable telemetry;
unset, empty, or unrecognized values stay disabled. This single
predicate gates both the Supabase beacon and the local `/v1/telemetry`
collector.
- Added `--telemetry` opt-in flag to `headroom proxy` and `headroom
install apply`; kept `--no-telemetry` and `HEADROOM_TELEMETRY=off` for
back-compat. If both are passed, opt-out wins.
- Install manifests now write `HEADROOM_TELEMETRY` explicitly
(`on`/`off`) plus the matching flag, so generated systemd/docker/launchd
deployments are unambiguous and don't rely on the runtime default.
- Startup banner and proxy log show `DISABLED` by default and surface
how to opt in.
- Updated tests for opt-in defaults; added coverage for the default-off
banner, the `--telemetry` flag, and the explicit-on manifest path.
- Updated docs
(proxy/configuration/installation/benchmarks/community-savings mdx, spec
011/015, wiki proxy/cli/benchmarks/metrics) and `CHANGELOG.md`.

## Testing

- [x] Unit tests pass (`pytest`) — targeted telemetry + install-planner
suites
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_telemetry_warning.py tests/test_telemetry.py tests/test_install/test_planner.py -q
81 passed

$ ruff check <changed source + test files>
All checks passed!

$ mypy headroom/telemetry/beacon.py headroom/cli/proxy.py headroom/cli/install.py \
       headroom/install/planner.py headroom/proxy/server.py
Success: no issues found in 5 source files
```

## Real Behavior Proof

- **Environment:** macOS (darwin 25.4.0), project `.venv`, `headroom`
CLI.
- **Exact command / steps:**
  ```
$ python -c "import os; from headroom.telemetry.beacon import
is_telemetry_enabled; \
os.environ.pop('HEADROOM_TELEMETRY', None); print('unset ->',
is_telemetry_enabled()); \
[ (os.environ.__setitem__('HEADROOM_TELEMETRY', v), print(repr(v), '->',
is_telemetry_enabled())) \
        for v in ['on','TRUE','1','yes','off','0','garbage',''] ]"

  $ headroom proxy --help        | grep -i telemetry
  $ headroom install apply --help | grep -i telemetry
  ```
- **Observed result:**
  ```
  unset      -> False      # off by default
  'on'       -> True       'TRUE' -> True   '1' -> True   'yes' -> True
  'off'      -> False      '0' -> False
  'garbage'  -> False      ''  -> False     # fail-closed

proxy: --telemetry Opt in to anonymous usage telemetry — off by default
(env: HEADROOM_TELEMETRY=on)
--no-telemetry Force anonymous usage telemetry off (already the default;
env: HEADROOM_TELEMETRY=off)
install: --telemetry Opt in to anonymous telemetry in the runtime (off
by default).
--no-telemetry Force anonymous telemetry off in the runtime (already the
default).
  ```
- **Not tested:** live Supabase beacon network round-trip (no opt-in
network call was made); full `pytest` suite was not run — only the
telemetry + install-planner targeted suites.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

- "Breaking change" is checked because the default behavior changes
(telemetry stops running unless opted in). It is **not** an API break —
`--no-telemetry` and `HEADROOM_TELEMETRY=off` still work, so existing
opt-out configs are unaffected.
- No tracking issue, so `Closes #` is left N/A.
2026-06-20 21:26:04 -07:00
Tejas Chopra f4bd2fe68f docs(vertex): Claude Code + Vertex via Headroom guide (validated) (#1180)
## Description

Documents the **validated** way to run **Claude Code** against **Claude
models on Google Vertex AI** with **Headroom compressing the context**.
Corrects the prior review's assumption that the "Vertex-mode redirect"
approach would work — Claude Code's client-side `probeVertexModel`
blocks it — and documents the working **Anthropic-mode + LiteLLM
`vertex_ai`** path, verified end-to-end against live Vertex quota (~22%
context compression observed).

Closes # <!-- n/a -->

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **`docs/claude-code-vertex-headroom.md`** (new) — copy-paste runbook:
prerequisites (GCP ADC, `google-cloud-aiplatform`, Vertex quota),
two-terminal setup (proxy `--backend litellm-vertex_ai --region <loc>
--code-aware`; Claude Code in normal Anthropic mode via
`ANTHROPIC_BASE_URL`), verification, a troubleshooting table, and a
section on what `--code-aware` does and what it never touches (local
files / protected `Read`/`Glob`/`Grep`/`Write`/`Edit` output).
- **`wiki/vertex.md`** — new "Claude Code with Headroom compression"
section pointing at the runbook, with the two ⚠️ caveats (Vertex-mode
probe rejects custom URLs; `vertexai` dep + `--code-aware` required).
- **`docs/proposals/vertex-claude-compression-review.md`** — corrected
TL;DR: Setup A is blocked by Claude Code's probe; Setup B is the
validated path.

## Testing

- [ ] Unit tests pass (`pytest`) — **N/A (docs-only, no code changed)**
- [ ] Linting passes (`ruff check .`) — **N/A (no Python changed)**
- [ ] Type checking passes (`mypy headroom`) — **N/A (no Python
changed)**
- [ ] New tests added for new functionality — **N/A (docs)**
- [x] Manual testing performed (live Vertex validation — see below)

### Test Output

```text
# 1) Direct Vertex quota check (global)
POST .../locations/global/publishers/anthropic/models/claude-sonnet-4-6:rawPredict
  -> HTTP 200  {"content":[{"text":"VERTEX OK"}], "model":"claude-sonnet-4-6"}

# 2) Headroom in Anthropic mode -> LiteLLM(vertex_ai) -> Vertex global
POST http://127.0.0.1:8787/v1/messages  (model=claude-sonnet-4-6)
  -> HTTP 200  {"content":[{"text":"LITELLM VERTEX OK"}], "model":"claude-sonnet-4-6"}

# 3) Real Claude Code session (normal mode) through Headroom, --code-aware ON
claude -p "...run two Bash source dumps + summarize..."  (ANTHROPIC_BASE_URL=proxy)
  -> is_error: False, modelUsage: ['claude-sonnet-4-6']
  request_log: orig=9353  saved=2029 (21.7%)  transforms=['router:tool_result:mixed']

# 4) Compressors loaded (GET /debug/warmup)
{'kompress':'loaded', 'code_aware':'loaded', 'tree_sitter':'loaded', 'smart_crusher':'loaded'}
```

## Real Behavior Proof

- **Environment:** macOS (arm64); Claude Code 2.1.181; Headroom 0.27.0;
venv Python 3.12; LiteLLM `vertex_ai` via `google-cloud-aiplatform`
1.158.0; GCP project `eternal-sunset-495505-t0`; Vertex location
`global`; model `claude-sonnet-4-6` (only model with quota on this
project); auth via `gcloud auth application-default login` (ADC).
- **Exact command / steps:** the two-terminal setup in
`docs/claude-code-vertex-headroom.md` — proxy `headroom proxy --port
8787 --backend litellm-vertex_ai --region global --code-aware`; client
`ANTHROPIC_BASE_URL=http://127.0.0.1:8787` +
`ANTHROPIC_MODEL=claude-sonnet-4-6` in normal mode (no
`CLAUDE_CODE_USE_VERTEX`).
- **Observed result:** Claude Code answered via Vertex (`modelUsage:
claude-sonnet-4-6`); ~22% context compression
(`router:tool_result:mixed`) on a code-heavy request forwarded to Vertex
`global`; all compressors loaded.
- **Not tested:** cumulative savings over long multi-turn sessions;
non-global regions (no quota on this project); Opus 4.8 (not enabled in
this project — 404); automated tests for the LiteLLM-vertex path (still
absent — pre-existing gap).

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines (docs)
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
(N/A — docs)
- [x] I have made corresponding changes to the documentation (this *is*
the documentation)
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works — **N/A (docs-only)**
- [ ] New and existing unit tests pass locally with my changes — **N/A
(no code changed)**
- [ ] I have updated the CHANGELOG.md if applicable — **N/A
(docs-only)**

## Additional Notes

- **Docs-only PR** — no Python changed, so `ruff` / `mypy` / `pytest`
are N/A.
- **Base:** branched from latest `origin/main`; clean 3-file diff (the
prerequisite review doc and Vertex wiki content are already on `main`).
- **Follow-ups:** optional `headroom wrap claude` Vertex turnkey; add
automated tests for the LiteLLM-vertex path; consider defaulting
`--code-aware` (or warning when code content is detected but code-aware
is off), since its default-off state makes compression silently no-op on
coding sessions.
2026-06-19 18:14:14 -07:00
Mateus Borges 1e437d781b docs: document HEADROOM_BETA_HEADER_STICKY and HEADROOM_BETA_TRACKER_MAX_SESSIONS (#1060)
## Description

Documents `HEADROOM_BETA_HEADER_STICKY` and
`HEADROOM_BETA_TRACKER_MAX_SESSIONS` — two env vars that exist in source
and tests but are absent from all `.md` / `.mdx` docs. Adds a **Session
Beta Header Tracking** section explaining the `SessionBetaTracker`
behavior, its prefix-cache rationale, and the operator trade-off.

An operator debugging a beta-header-related upstream rejection cannot
discover the knob or the off-switch without reading source.

Closes #1059

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

**`wiki/configuration.md`**
- Two new rows in the Environment Variables table:
`HEADROOM_BETA_HEADER_STICKY` and `HEADROOM_BETA_TRACKER_MAX_SESSIONS`
- New `## Session Beta Header Tracking` section with: what the mechanism
does, why it exists (prefix-cache stability), the operator trade-off,
and how to disable

**`docs/content/docs/configuration.mdx`**
- Same two rows added to the Environment Variables table
- Same `### Session Beta Header Tracking` section (matching heading
level)

## Testing

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .` — N/A, docs-only change to
`.md`/`.mdx` files)
- [ ] Type checking passes (`mypy headroom` — N/A, docs-only)
- [ ] New tests added for new functionality — N/A, docs-only
- [x] Manual testing performed

### Test Output

```text
Manual verification: confirmed env var names, accepted values, defaults, and LRU bound match code (helpers.py:1610-1629, 1613-1614, 1820-1822). Existing tests exercise the behavior (tests/test_anthropic_beta_session_sticky.py:124).
```

## Real Behavior Proof

- Environment: Ubuntu 24.04, Python 3.12, Headroom v0.25.0, provider:
Anthropic (Claude Code via `headroom wrap`)
- Exact command / steps: `grep -ri "HEADROOM_BETA_HEADER_STICKY"
README.md CHANGELOG.md wiki/ docs/` → 0 matches before; source
inspection of helpers.py:1605-1856, anthropic.py:918-961,
prefix_tracker.py:316-335
- Observed result: env vars now documented in both wiki + docs mirrors;
behavior rationale and trade-off explained
- Not tested: end-to-end proxy run with the new docs in place (docs-only
change; behavior unchanged)

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works — N/A, docs-only
- [ ] New and existing unit tests pass locally with my changes — N/A,
docs-only
- [ ] I have updated the CHANGELOG.md if applicable — N/A, changelog is
auto-managed by release-please

## Screenshots (if applicable)

N/A

## Additional Notes

Mirrors the same dual-file pattern used in #579 (wiki + docs mdx). The
documented behavior is contractual — confirmed by
tests/test_anthropic_beta_session_sticky.py:124
(test_beta_seen_turn_1_present_in_turn_2_even_if_client_drops) and the
test module docstring naming Claude Code and Codex CLI as the targeted
clients.
2026-06-16 23:06:41 -07:00
Dashsoap ca23257d1b docs: correct macOS troubleshooting Python floor to 3.10+ (#981)
## Description

`wiki/macos-deployment.md` told users that Headroom "Requires Python
3.9+", but the project's actual floor is Python 3.10. This corrects that
one line to 3.10+ so it matches `pyproject.toml` and every other doc.

Closes #

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `wiki/macos-deployment.md` (troubleshooting → "Common causes"):
`Requires Python 3.9+` → `Requires Python 3.10+`.

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# Ground truth — the real Python floor:
$ grep -n 'requires-python' pyproject.toml
11:requires-python = ">=3.10"

# Every other doc already says 3.10+, and this was the only "3.9" left:
$ grep -rniE 'requires? python *3\.(9|10)' README.md docs/ wiki/ CONTRIBUTING.md
README.md: ... Requires **Python 3.10+**.
docs/content/docs/installation.mdx:16: Headroom requires **Python 3.10+** ...
docs/content/docs/installation.mdx:245: This project requires **Python 3.10+**.
wiki/index.md:405: Requires Python 3.10+.
CONTRIBUTING.md:125: - Python 3.10+. ...
wiki/macos-deployment.md:426: - Python version incompatible: Requires Python 3.10+   # fixed by this PR
```

## Real Behavior Proof

- Environment: local clone at `origin/main`; this is a
documentation-only change.
- Exact command / steps: confirmed `pyproject.toml` declares
`requires-python = ">=3.10"`; grepped all docs and found
`wiki/macos-deployment.md` was the only file claiming `3.9+`; changed
that single line to `3.10+`.
- Observed result: all Python-floor mentions across README,
`docs/content/docs/installation.mdx`, `wiki/index.md`,
`CONTRIBUTING.md`, and now `wiki/macos-deployment.md` agree on 3.10+,
matching `requires-python`.
- Not tested: N/A — single-line prose fix in a Markdown file; no code
paths, no build, no runtime behavior involved.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

Documentation-only change, so `pytest`/`ruff`/`mypy` over the `headroom`
package are N/A — the diff contains no Python source. The fix was a
misleading minimum specifically in the version-incompatibility
troubleshooting step, where the wrong floor (3.9 vs the real 3.10) would
actively mislead a user diagnosing a Python version problem on macOS.
2026-06-15 11:09:59 -05:00
Logan Kang c71592d421 feat(memory): add opt-in Apple-GPU (MPS) embedding runtime (#766)
## Description

On Apple-Silicon Macs — especially fanless models like the MacBook Air
(M5) — running the proxy with memory context injection can pin the CPU
while embedding. The embedding work runs an uncapped session on the CPU,
saturating multiple cores, which starves the proxy's asyncio loop and
leads to request timeouts.

This PR adds an **opt-in** runtime that offloads the memory embedder to
the Apple GPU (MPS). Setting `HEADROOM_EMBEDDER_RUNTIME=pytorch_mps`
routes embedding through the torch `sentence-transformers` backend on
MPS instead of the default ONNX CPU embedder, moving the work off the
CPU and keeping the proxy responsive.
The default behavior is unchanged — the feature is strictly opt-in,
env-var only, and falls through to the existing default embedder
selection (with a warning) whenever MPS or the torch dependencies are
unavailable.

Fixes: N/A — no tracking issue (surfaced while running codex auto-review
through
the proxy on a fanless MacBook Air (M5)).

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [x] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **Runtime selection** (`headroom/proxy/memory_handler.py`): read
`HEADROOM_EMBEDDER_RUNTIME`; when set to `pytorch_mps` **and** MPS is
actually available, route the memory embedder to the torch
`sentence-transformers` backend (Apple GPU).
- If MPS is unavailable or torch/sentence-transformers is not installed,
log a warning and fall through to the existing default embedder
selection (ONNX when available, else the pre-existing local
sentence-transformers fallback) — no crash. The default (env var unset)
is unchanged. Env-var only
- **MPS serialization** (`headroom/memory/adapters/embedders.py`):
`LocalEmbedder` now funnels every `encode()` through a dedicated
single-worker `ThreadPoolExecutor` when the resolved device is MPS.
torch-MPS is not thread-safe, and the existing `run_in_executor(None,
...)` dispatch would otherwise let concurrent proxy requests call MPS
from multiple threads. CPU/CUDA keep the shared default executor
(behavior unchanged). `close()` also drops the cached model so re-use
after close re-initializes cleanly.
- **Packaging** (`pyproject.toml`): new `pytorch-mps` extra (`torch` +
`sentence-transformers`), **platform-gated to macOS** (`; sys_platform
== 'darwin'`) since MPS is Apple-Silicon-only. Deliberately left out of
`[all]` (its deps already arrive via `[ml]`/`[memory]`).
- **Tests** (`tests/test_memory/test_embedder_mps_serialization.py`):
regression coverage for the serialized executor, concurrency safety (no
SIGABRT), CPU-path default behavior, and close/re-use re-initialization.
- **Docs**: `wiki/{configuration,memory,macos-deployment}.md`,
`docs/content/docs/{configuration,installation,memory}.mdx`,
`README.md`, `CHANGELOG.md`.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed (CPU-offload + concurrency profiling on
Apple Silicon)

## Test Output

```
$ pytest -v tests/test_memory/test_embedder_mps_serialization.py
collected 4 items
tests/test_memory/test_embedder_mps_serialization.py::test_cpu_uses_shared_executor PASSED            [ 25%]
tests/test_memory/test_embedder_mps_serialization.py::test_mps_creates_single_worker_executor PASSED  [ 50%]
tests/test_memory/test_embedder_mps_serialization.py::test_mps_concurrent_embeds_do_not_crash PASSED  [ 75%]
tests/test_memory/test_embedder_mps_serialization.py::test_mps_reembed_after_close_recreates_executor PASSED [100%]
============================== 4 passed in 6.92s ===============================

$ ruff check headroom/memory/adapters/embedders.py headroom/proxy/memory_handler.py tests/test_memory/test_embedder_mps_serialization.py
All checks passed!

$ mypy headroom/memory/adapters/embedders.py headroom/proxy/memory_handler.py
Success: no issues found in 2 source files

$ pytest -q tests/test_memory/ tests/test_memory_handler_concurrent_init.py tests/test_memory_handler_native_ops.py
553 passed, 1 skipped in 13.51s
```

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

**Why MPS (and not CoreML or a thread cap):** measured on an
Apple-Silicon Mac, the default uncapped CPU embedding session saturates
the cores; the same model on MPS runs at a fraction of the CPU (≈8x
lower sustained CPU utilization in profiling) while producing
**byte-identical embeddings** (cosine distance ≈ 0 across runtimes), so
relevance/ranking is unchanged. A CoreML execution-provider path was
evaluated and rejected: the default optimized ONNX model uses fused ops
that fall back to CPU under CoreML (no offload), and a full-precision
re-export was impractical (very low throughput + multi-GB memory). MPS
via `sentence-transformers` was the only practical GPU offload.

**Why serialization is mandatory:** torch-MPS is not thread-safe —
concurrent encode calls from a multi-worker executor abort with
`-[IOGPUMetalCommandBuffer validate]: failed assertion 'commit an
already committed command buffer'` (reproduced deterministically; a
single-worker executor resolves it).
Under concurrent load the serialized single-GPU-stream throughput meets
or exceeds the parallel CPU path while using a fraction of the cores.

**Scope / boundary:** this targets the Python **memory** embedder, which
is live on the proxy request path (memory context injection). The
Rust-backed SmartCrusher compression path is unaffected and remains
non-configurable from Python by design.

**Safety:** default behavior is unchanged (ONNX, no torch). The feature
is opt-in, env-var only, macOS-gated at the packaging layer, and
degrades gracefully (warn + the existing default embedder selection)
when MPS or the dependencies are unavailable.
2026-06-11 12:59:20 -05:00
mbachaudandClaude Fable 5 6367d0b722 feat(kompress): warn on unrecognized HEADROOM_KOMPRESS_BACKEND + document backend selection (#204)
## Summary

This PR was originally \"HEADROOM_KOMPRESS_BACKEND env + GPU/MPS
auto-detect\" (for #202). While it sat, main independently shipped the
backend-selection env var in a2ea9648 (\"fix: add Kompress backend and
thread controls\") with a richer backend set (`auto` / `onnx` /
`onnx_cpu` / `onnx_coreml` / `pytorch` / `pytorch_mps` + shorthand
aliases) and an explicit design decision to keep `auto` on the
ONNX-CPU-first path rather than auto-preferring accelerators. Rather
than re-litigate that, this PR has been rebased onto latest main and
rescoped to the two pieces main still lacks:

1. **Warn on unrecognized `HEADROOM_KOMPRESS_BACKEND` values** —
previously typos (`gpu`, `cudaa`, …) silently mapped to `auto`,
indistinguishable from the default. Now a warning names the offending
value and the accepted set; behavior still falls back to `auto`.
2. **Documentation** — the env var and its six backends/aliases were
undocumented outside the source. Added a \"Kompress backend selection\"
section to `wiki/configuration.md` and a CHANGELOG entry.

## Testing

- `pytest tests/test_transforms/test_kompress_compressor.py` — 28 passed
(includes 2 new tests: warning fires on unrecognized value; valid values
and unset stay silent)
- `ruff check` / `ruff format` clean on touched files
- No behavior change beyond the new warning, so no GPU/MPS hardware
validation is required for this scope.

Refs #202

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 20:13:17 -05:00
JD Davis 3c77e52ce4 feat: add Vertex AI proxy routing (#793)
## Description

Adds first-class GCP Vertex AI proxy routing for publisher REST
endpoints so Vertex requests are forwarded to a configurable regional
Vertex host instead of falling through to the generic
OpenAI/Anthropic/Gemini passthrough selection.

Fixes #792

## Type of Change

- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update

## Changes Made

- Added a `vertex` provider target with `VERTEX_TARGET_API_URL` and
`--vertex-api-url` support.
- Registered explicit Vertex publisher routes for Google
`generateContent`, `streamGenerateContent`, `countTokens` and Anthropic
publisher `rawPredict`, `streamRawPredict` passthrough.
- Added startup banner/routing output for Vertex AI.
- Added focused tests for provider target resolution, CLI/env config,
banner output, and route delegation.
- Added `wiki/vertex.md` with usage examples and Google Cloud source
links.

## Sources

- Vertex AI Gemini inference reference:
https://docs.cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference
- Google Cloud REST authentication:
https://docs.cloud.google.com/docs/authentication/rest
- Google Application Default Credentials:
https://docs.cloud.google.com/docs/authentication/application-default-credentials

## Testing

- [x] Linting passes (`python -m ruff check .`)
- [x] New tests added for new functionality
- [x] Focused unit tests pass
- [ ] Full unit suite completed locally
- [ ] Rust tests completed locally
- [ ] Type checking passes locally

## Test Output

```text
$ python -m ruff check .
All checks passed!

$ python -m pytest tests/test_provider_registry.py tests/test_provider_proxy_routes.py tests/test_cli_proxy_env.py tests/test_banner_upstream_targets.py -q
57 passed, 1 warning in 13.75s
```

Local limitations:

- `python -m pytest tests scripts/tests -q` timed out after 1 hour on
this Windows machine before completing.
- `cargo test -p headroom-proxy --test integration_vertex_raw_predict`
could not run because `cargo` is not installed on PATH in this
environment.
- The commit hook's `mypy` step fails locally on an existing Windows
`fcntl` typing issue in `headroom/subscription/tracker.py`; `ruff`,
`ruff-format`, and plugin-version hooks passed, and the commit was made
with only `mypy` skipped.

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have made corresponding changes to the documentation
- [x] I have added tests that prove the feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
2026-06-09 23:05:30 -07:00
JD Davis 11ab5f83a1 feat: add differential network capture harness (#761)
## Summary
- add a containerized differential network capture harness for Claude
Code direct vs Claude Code routed through Headroom
- capture both Headroom client-side traffic and Headroom upstream
traffic with sanitized mitmproxy JSONL output
- add `headroom capture network-diff` to compare captures and produce
Markdown/JSON reports, including Anthropic tool-count/tool-byte deltas
for deferred-tool investigations
- add an on-demand GitHub Actions workflow for the harness; it only runs
via `workflow_dispatch`, with live Claude Code/Anthropic capture gated
on `ANTHROPIC_API_KEY`
- document the workflow and ignore generated capture artifacts

## Validation
- `C:\git\headroom\.venv\Scripts\python.exe -m pytest
tests/test_network_diff_capture.py`
- `ruff check headroom/capture headroom/cli/capture.py
tests/test_network_diff_capture.py`
- `ruff format --check headroom/capture headroom/cli/capture.py
tests/test_network_diff_capture.py`
- `C:\git\headroom\.venv\Scripts\python.exe -m mypy
headroom/capture/network_diff.py headroom/cli/capture.py`
- `docker compose -f
docker/differential-network-capture/docker-compose.yml --profile run
config`
- `docker compose -f
docker/differential-network-capture/docker-compose.yml --profile run
build claude-direct`
- `docker run --rm -e CLAUDE_COMMAND="claude --version"
headroom-network-diff-claude-direct:latest`
- parsed `.github/workflows/network-diff-capture.yml` with PyYAML and
confirmed manual-only trigger

Live Claude API capture was not run locally because `ANTHROPIC_API_KEY`
is not set in this environment. The workflow can run it manually in
GitHub Actions when that secret is present; otherwise it emits a visible
skip warning and uploads a skipped artifact.

## Notes
- Full pre-commit mypy still fails on unrelated Windows `fcntl`
attributes in `headroom/subscription/tracker.py`; the feature commit
skipped only that hook after narrow mypy passed for the new modules.
- `tests/test_release_workflows.py` has two Windows-local failures
because it shells out to a missing Unix/Rust command; unrelated workflow
checks in that file passed before those failures.
- Motivated by
https://github.com/chopratejas/headroom/issues/746#issuecomment-4651276818
/ Issue #746.
2026-06-08 22:18:31 -07:00
Tejas Chopra 18925b8c6e fix(copilot): restore generic endpoint for non-subscription OAuth (#610) (#612)
* fix(copilot): restore generic endpoint for non-subscription OAuth (#610)

0.23.0 re-pointed the shared Copilot OAuth branch from the generic
api.githubcopilot.com host to the account-specific endpoints.api host
returned by /copilot_internal/user, and made resolve_copilot_api_url
ignore the GITHUB_COPILOT_API_URL override whenever a token resolved.

That change was meant to add --subscription, but it also altered the
pre-existing non-subscription OAuth flow that worked on 0.22.4. The
account host does not serve newer models (e.g. gpt-5.4) on the responses
API, so wrapped requests began failing with unsupported-model errors
while plain Copilot and 0.22.4 kept working.

Restore 0.22.4 routing for non-subscription OAuth (generic host, still
overridable) and keep account resolution only for --subscription.
resolve_copilot_api_url now honors GITHUB_COPILOT_API_URL first, so the
override escape hatch works for every path. BYOK is unaffected.

Add a regression suite that mocks a successful user-info response, the
real-world path the prior test never exercised (it relied on the network
call failing in CI and falling back to the generic host).

* fix(copilot): route subscription + OAuth through the generic host (#610)

The 0.23.0 endpoint resolution derived the Copilot API host from
/copilot_internal/user (endpoints.api), which returns a segmented host
(e.g. api.individual.githubcopilot.com) that does not serve newer models
on the responses API and is not the host the official Copilot client
routes with (that comes from the token-exchange endpoint). --subscription
used the identical resolution, so it carried the same latent regression
as the non-subscription OAuth path.

Make Copilot host resolution override -> generic for BOTH --subscription
and the implicit OAuth path, and stop using user-info to route. Accounts
that require a dedicated host (enterprise / data residency) pin it via
GITHUB_COPILOT_API_URL. resolve_copilot_api_url no longer makes a network
call; _fetch_copilot_user_info is retained for token validation.

Update the subscription smoke tests that encoded the old account-host
assumption, and add wrap-level + unit coverage that --subscription routes
to the generic host even when user-info advertises an account host, and
that the GITHUB_COPILOT_API_URL override flows through both paths.

* docs(copilot): document generic-host routing + enterprise override (#610)

Spell out the routing contract introduced by the #610 fix so enterprise
users have a supported path. Headroom routes wrapped Copilot hosted
traffic (--subscription and OAuth) to the generic api.githubcopilot.com,
and accounts on a dedicated host (Enterprise Cloud data residency, egress
proxy) pin it via GITHUB_COPILOT_API_URL.

- copilot --help: note the generic host + GITHUB_COPILOT_API_URL override.
- TESTING-copilot-subscription.md: add "API host & Enterprise / data
  residency" section; correct the stale api.*.githubcopilot.com claim; and
  invite enterprise tenants who want token-exchange-based auto-detection to
  open an issue.
- integration-guide.md: short hosted-host + override note in the Copilot
  section.
2026-06-04 16:27:54 -07:00