Commit Graph
3 Commits
Author SHA1 Message Date
Mansurshakh JaparovandJerrettDavis 50bf775e8c fix(proxy): bypass compression for large admitted histories (#4142)
## Description

Closes #4139.

A history that is too large to compress should still reach the provider.
This keeps the 100 MiB hard admission limit and bypasses request-side
transforms above 16 MiB decoded size. The hard limit still applies
before forwarding.

Current head: `62ffe3defef1684f70d96f1640818d99445a43bb`, merging
current main `a2d4fe72` into `1902d8ef`. The
42-test/native/manual/pre-push results below qualify `1902d8ef`, not
this new merge. New-head shard 4 has a cached-prefix fixture failure
under investigation; coverage uploads are still incomplete.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Performance improvement
- [x] Documentation update

## Changes Made

- Keep the startup override and both wire/decompressed hard bounds.
Large admitted histories skip normalization, image processing, memory
injection, tokenization and compression on Messages, Chat/Responses,
Gemini and Bedrock HTTP routes.
- Keep provider routing and backend translation, including Copilot Chat,
Codex `store:false` and Vertex route-forced streaming. Native forwarding
retains decoded bytes unless the provider's wire contract requires a
rewrite; translation retains the full message text.
- Report actual provider usage, with zero compression savings or
attempted tokens for the bypass. Keep `/v1/compress` and WebSocket
contracts unchanged.
- Add real-provider coverage for large Gemini JSON/SSE requests with
query-key auth, and OpenAI JSON/SSE through the installed LiteLLM
adapter. No admission-policy or CI changes in this follow-up.

## Testing

- [x] Unit tests pass (focused suite and normal critical pre-push suite)
- [x] Linting passes (normal hooks)
- [x] New tests added for admission, recovery and passthrough accounting
- [x] Manual testing performed against the real native build

### Test Output

```sh
python -m pytest -q tests/test_proxy_compression_admission.py tests/test_proxy/test_gemini_plugin_routed_streaming.py --cov=headroom --cov-branch
# 42 passed, 41 warnings
```

Normal pre-push `make ci-precheck` passed: 1668 Rust tests (3 ignored),
181 Python tests, lint/type checks and commitlint. No hook bypass. The
initial deep-module coverage command hit a Pydantic import-isolation
error; using the repo's `--cov=headroom` convention passed both complete
modules without disabling plugins or tests.

The earlier `96c5fe7e` qualification covered the other native protocols,
configured/per-request backends, hard-ceiling recovery and complete
local shards 2/3. Those runs are historical, not a new whole-suite
claim. New-head hosted success is not claimed.

## Real Behavior Proof

- Environment: Linux x86_64, Python 3.13.14, freshly built real
`headroom._core`, public Headroom ASGI routes, a real localhost HTTP
provider and installed LiteLLM. Synthetic credentials, offline mode and
`DO_NOT_TRACK=1`; no live provider calls.
- Exact command / steps: Start
`create_app(ProxyConfig(backend="litellm-openai", ...))` with all
upstream URLs set to the localhost peer and
`litellm.api_base=peer+"/v1"`. Leave the production 16 MiB soft/100 MiB
hard limits unchanged. POST `"history " * (16777216 // 8 + 1024) + "LAST
TURN"` as a user message to Chat Completions, then as a Gemini content
part to generateContent/streamGenerateContent, with streaming off/on.
The peer rejects a changed history, requires Gemini query-key auth and
one `alt=sse`, and returns protocol-valid JSON/SSE with 101 input/1
output tokens. The maintained four small-ceiling cases are in
`tests/test_proxy_compression_admission.py` and run with the command
above.
- Observed result: All four >16 MiB requests returned 200 with the full
history intact. JSON/SSE and Gemini auth survived; streamGenerateContent
forced SSE even with incoming `alt=json`. Each request recorded exactly
101 provider input/1 output tokens, zero compression attempts/savings
and no failures; `/livez` stayed 200. Earlier recorded qualification
also covered all five native HTTP protocols, real translating backends
and 413 recovery.
- Not tested: Live cloud accounts, the reporter's deployed 4 GB VPS,
screenshot/image quality or a zero-JSON-parse path. One envelope/model
JSON parse remains; no proxy-only RSS guarantee is claimed.

## Runtime Rollout Safety

- Rollout-managed feature(s): None; this is the admission/compatibility
fix.
- Minimum rollout channel: Stable/default.
- Stable/default behavior changed: Keep the 100 MiB hard ceiling; bypass
request-side transforms above 16 MiB decoded size on the named HTTP
routes.
- Kill switch / disable path: Set
`HEADROOM_REQUEST_BODY_MAX_BYTES=16777216` (or lower) and restart,
accepting local 413s instead of the large-body path.
- Unsafe override required: No.
- Qualification impact: Local native protocols, backend translation,
hard bounds and recovery exercised; live-provider/VPS qualification
remains external.
- Rollback path: Revert the admission revision and restart; no storage
migration.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] I have made corresponding changes to the documentation
- [x] I did not edit `CHANGELOG.md`

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-04 22:37:21 -06:00
Tejas ChopraandClaude Opus 5.5 72302a3998 fix(proxy): say why proxy.log is missing and why a knob or extension was ignored (#4088)
## Description

Several failures an operator needs to see are swallowed silently or
logged only at DEBUG. This PR makes each of them visible with one
warning. It does not change defaults or control flow.

1. **Missing runtime log.** `_setup_file_logging` caught `OSError` with
`pass`. On a read-only workspace, a wrong owner, or a file where the
logs directory belongs, `proxy.log` simply never appeared and nothing
said why. It now logs `Headroom runtime log disabled: cannot write the
log file: <OSError>. Point HEADROOM_WORKSPACE_DIR at a writable
directory to restore proxy.log.` The OSError text names the path. The
symlink case just above it already warned this way.
2. **Six import-time numeric knobs.** Each one fell back to its default
on an unparseable value, without a word:
   - `HEADROOM_COMPRESSION_TIMEOUT_SECONDS`
   - `HEADROOM_COLD_START_FAST_PASS_TIMEOUT_SECONDS`
   - `HEADROOM_EAGER_PRELOAD_TIMEOUT_SECONDS`
   - `HEADROOM_COMPRESSION_CACHE_MAX_SESSIONS`
   - `HEADROOM_COMPRESSION_CACHE_TTL_SECONDS`
   - `HEADROOM_COMPRESSION_CACHE_MAX_ENTRIES`

Each now logs `Ignoring invalid NAME='value'; using DEFAULT` once, at
import. That matches the idiom in `hard_watchdog.py`.
3. **Extension discovery failure.** When entry-point enumeration fails,
every proxy extension is dropped, Enterprise plugins included. That was
logged at DEBUG. It is now a warning: `proxy extensions: entry-point
enumeration failed (<describe_exception>), so no proxy extensions will
be installed`. `describe_exception` gives the exception types and code
locations, never the message, and no traceback is logged at any level.
It runs at startup only: once in `install_all`, and once more for the
CLI banner.
4. **Pricing lookups in `cost.py`.** `_get_list_price`,
`_get_output_price`, `_get_cache_prices` and the cache-rate lookup in
the savings summary returned `None` on any exception. A pricing bug
therefore showed up as a $0 cost with no trace. All four, plus the
existing `estimate_cost` path, go through one helper,
`_warn_pricing_once(lookup, what, model, exc)`. Example line: `Failed to
get pricing for model 'gpt-4o': RuntimeError at
headroom/proxy/cost.py:... in estimate_cost`. It logs the lookup name,
`safe_id(model)` and `describe_exception(exc)` (types and code
locations, never the message), at WARNING only. No level carries the raw
exception or a traceback, because a pricing backend error can quote
provider responses or credentials. Dedupe is by `(lookup, model)`
through the shared `WarnOnce(1024, "unpriceable models")`. Each pair
warns at most once an hour (the `WarnOnce` window). Past 1024 models,
one overflow notice is logged, and a key is not used up while WARNING is
disabled.
5. **Malformed `HEADROOM_OPENAI_TOOL_SEARCH_MODELS` regex.** The
override was ignored on every request with no log. It now warns once per
bad pattern, and still falls back to the version gate as before. A valid
pattern re-arms the warning, so bad A, then valid B, then bad A again
warns twice.

None of the new lines logs header values, request content or
credentials. They log only knob names, their raw values (operator-set
env), model names, regex patterns, exception type names, the `re.error`
text, and `OSError` text naming a local path. No exception message or
traceback from pricing or extension discovery reaches any log level.

Refs #4136

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/proxy/helpers.py`:
- add `_warn_invalid_env_number(name, default)` and call it from the six
import-time `except ValueError` branches;
  - log the `OSError` in `_setup_file_logging`;
- add a once-per-pattern warning for a malformed
`HEADROOM_OPENAI_TOOL_SEARCH_MODELS`.
- `headroom/proxy/extensions.py`: entry-point enumeration failure logs
at warning with `describe_exception(exc)` instead of at debug.
- `headroom/proxy/cost.py`:
- the four silent pricing `except Exception` branches call
`_warn_pricing_once`;
- `_warn_pricing_once(lookup, what, model, exc)` logs `safe_id(model)`
plus `describe_exception(exc)` at WARNING, with nothing at DEBUG. It
dedupes through `log_safety.WarnOnce`.
- `tests/test_cost_pricing_warning_dedup.py`: the tests reset with a
fresh `WarnOnce`, and match the quoted model id (`for model
'glm-5.2':`).
- Regression tests. Each one attaches a handler to the module logger
directly, because caplog misses `headroom.*` records after proxy
startup. Each positive test fails on main and is paired with a negative
case that passes on both.
  - `tests/test_cache_breakpoint_diagnostics.py`:
- `test_runtime_log_says_why_it_is_missing` makes the logs directory a
regular file. It asserts one warning that names the path and
`HEADROOM_WORKSPACE_DIR`.
    - Negative: `test_runtime_log_stays_quiet_when_writable`.
  - `tests/test_openai_tool_search_deferral.py`:
- `test_malformed_override_warns_once_not_per_request` calls the gate 5
times with a bad pattern. It asserts exactly one warning and that the
version-gate result is unchanged.
    - Negative: a valid override is silent (same test).
- `test_valid_override_rearms_the_malformed_warning`: bad, valid, bad
gives two warnings.
  - `tests/test_proxy/test_compression_timeout_config.py`:
- `test_bad_env_says_it_was_ignored` reloads `helpers` with
`HEADROOM_COMPRESSION_TIMEOUT_SECONDS=not-a-number` and asserts the
exact warning.
- Negative: `test_valid_env_is_silent`. The existing fixture reloads
`helpers` afterwards.
  - `tests/test_proxy_extensions.py`:
- `test_discover_warns_when_entry_point_enumeration_fails` makes
`importlib.metadata.entry_points` raise. It asserts one WARNING that
says no extensions will be installed and names `RuntimeError` but not
the exception text.
    - Negative: `test_discover_is_quiet_when_nothing_is_installed`.
  - `tests/test_cost_cache_prices.py`:
- `test_cache_rate_failure_warns_once_per_model` makes `resolve_rates`
raise. It asserts two `_get_cache_prices` calls give one warning naming
the model and `RuntimeError`, without the exception text.
    - Negative: a resolvable model is silent (same test).
- `test_pricing_error_never_logs_its_text_at_any_level` runs the real
`CostTracker.estimate_cost('gpt-4o', 100, 10)` with a pricing error
whose text holds an Authorization credential canary, with
`headroom.proxy` at DEBUG. It formats every captured record with
`logging.Formatter` (which renders `exc_info`), asserts the canary is
absent and the result is `None`, and asserts the warning is present. On
the previous commit it fails: the formatted DEBUG traceback contains the
canary.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest -q tests/test_cost_cache_prices.py -k never_logs   # previous commit 3202058c (test adapted to its old dedupe set)
E     RuntimeError: upstream rejected request (Authorization: Bearer sk-canary-7f3a9c)']
======================= 1 failed, 25 deselected in 3.48s =======================
$ pytest -q -p no:cacheprovider tests/test_pr208_changes.py tests/test_cost_budget_basis.py tests/test_cost_tracker_counterfactual.py tests/test_cost_tracker_totals.py tests/test_tool_search_client_deferral.py tests/test_cache_breakpoint_diagnostics.py tests/test_proxy_cache_ttl_metrics.py tests/test_proxy_byte_faithful_forwarding.py tests/test_memory_tool_mode.py tests/test_codex_ws_compression_scheduler.py tests/test_proxy_savings_history.py tests/test_compression_cache_registry.py tests/test_proxy_extension_ordering.py tests/test_proxy_eager_preload_bind.py tests/test_perf_per_port_logs.py tests/test_cost_cache_prices.py tests/test_openai_tool_search_deferral.py tests/test_proxy_extensions.py tests/test_direct_chat_ccr_resolution.py tests/test_org_scale_limits.py tests/test_savings_ledger.py tests/test_cost_pricing_warning_dedup.py tests/test_proxy/test_compression_timeout_config.py tests/test_log_safety.py   # 64ff3f03
================== 517 passed, 1 skipped in 85.70s (0:01:25) ===================
$ pytest -q (same pricing/extension/regex/log/knob/log_safety files) after merging log-safety-helpers @ 0ce74a63   # 4ff57af4
======================== 139 passed, 1 skipped in 4.23s ========================
$ uvx ruff@0.16.8 check / format --check headroom/proxy/cost.py headroom/proxy/extensions.py tests/test_cost_cache_prices.py tests/test_cost_pricing_warning_dedup.py tests/test_proxy_extensions.py
All checks passed! / 5 files already formatted
$ mypy headroom/proxy/cost.py headroom/proxy/extensions.py --ignore-missing-imports
Success: no issues found in 2 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.12.13, branch checkout at 4ff57af4
(log-safety-helpers merged at 0ce74a63), using the repo .venv. A temp
HOME, workspace and CODEX_HOME were used each time; the real HOME was
not touched.
- Exact command / steps:
1. Ran the shared smoke harness, which starts a real proxy from this
checkout against a local fake upstream and sends one Anthropic and one
OpenAI request.
2. Ran the real `CostTracker.estimate_cost('gpt-4o', 100, 10)` twice,
with every logger at DEBUG and a LiteLLM stand-in whose `cost_per_token`
raises an error quoting `Authorization: Bearer sk-canary-7f3a9c`.
3. The earlier forced-failure run for the knob, log-directory and regex
warnings (d83e7d5c, below) is unchanged by this commit.
- Observed result:
```text
branch: cleanup/proxy-core-a-surface-ignored-config @ 4ff57af4
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines (<tmp>):
(smoke dir: <tmp>)
```
```text
WARNING headroom.proxy: Failed to get pricing for model 'gpt-4o': RuntimeError at <string>:8 in cost_per_token <- headroom/proxy/cost.py:999 in estimate_cost
result: None None
```
- One payload-safe WARNING appears, with the exception type and code
location, and both calls return `None`.
- The canary appears in no output line at any level: `grep -c canary`
over the whole run, LiteLLM's own DEBUG chatter included, gives 0.
  - Earlier forced-failure run:
```text
branch: cleanup/proxy-core-a-surface-ignored-config @ d83e7d5c
health: 200
POST /v1/messages        -> HTTP200 smoke-ok
POST /v1/chat/completions -> HTTP200 smoke-ok
stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82
proxy.log WARNING/ERROR lines (<tmp>):
(smoke dir: <tmp>)
```
The harness also printed two pre-existing WARNING lines, which I left
out above: offline-mode air-gap active, and the Kompress model prefetch
refused offline. Neither comes from this PR. Output of the direct
command:
```text
WARNING headroom.proxy: Ignoring invalid HEADROOM_COMPRESSION_TIMEOUT_SECONDS='abc'; using 30.0
WARNING headroom.proxy: Headroom runtime log disabled: cannot write the log file: [Errno 17] File exists: '<tmp>/ws/logs'. Point HEADROOM_WORKSPACE_DIR at a writable directory to restore proxy.log.
WARNING headroom.proxy: Ignoring invalid HEADROOM_OPENAI_TOOL_SEARCH_MODELS regex '[bad' (unterminated character set at position 0); using the gpt-5.4+ version gate
gate: True True
```
- The proxy serves both providers, and the normal path adds no new
warnings.
- Each forced failure gives exactly one warning, even though the regex
gate ran twice. The fallbacks are unchanged: 30.0, and the version gate
is True for gpt-5.4.
- Not tested: A real read-only filesystem, a real broken
`importlib.metadata` and a live pricing-catalog failure (the stand-in
above raises from the same call). Each goes through the same `except`
branch that the unit tests force.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A; log output only.
- Stable/default behavior changed: No. Every fallback value and code
path is the same; only new WARNING lines appear, when something was
already failing.
- Kill switch / disable path: N/A. A pricing warning fires at most once
an hour per (lookup, model), through `WarnOnce`; the regex warning fires
once per bad pattern.
- Unsafe override required: No.
- Qualification impact: None.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Part of a maintainer-requested codebase cleanup (dead code, duplication,
debuggability), split into small PRs for review. It is part of
maintainer-requested cleanup #4136.

Depends on #4138; its two files show in this diff until it merges. #4137
also edits `_setup_file_logging`, so whichever of the two merges second
rebases.

Docs are unchanged because no knob's meaning changed. The doc impacts
are recorded in the cleanup docs ledger for the follow-up docs PR. The
six import-time knob warnings have no test: testing them would need
`importlib.reload(helpers)`, which would reset the module globals other
tests depend on.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 11:51:06 -07:00
Eyal MizrachiandClaude Opus 4.8 addebdb29c feat(proxy): make COMPRESSION_TIMEOUT_SECONDS configurable via env (#946) (#991)
## Description

The compression-pipeline timeout was hard-coded at 30s, so slow CPUs and
long Claude Code conversations had no recourse. #946 asks to wire
`HEADROOM_COMPRESSION_TIMEOUT_SECONDS` through. Refs #946.

## Type of Change

- [x] New feature (non-breaking change that adds functionality)

## Changes Made

- Read `HEADROOM_COMPRESSION_TIMEOUT_SECONDS` from the environment
(float), falling back to 30 on an unparseable value.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] New tests added for new functionality

### Test Output

```text
$ pytest tests/test_proxy/test_compression_timeout_config.py -q
4 passed in 0.09s
```

## Real Behavior Proof

- Environment: Linux, Python 3.13, editable build of this branch
- Exact command / steps: `HEADROOM_COMPRESSION_TIMEOUT_SECONDS=88 python
-c "import headroom.proxy.helpers as h;
print(h.COMPRESSION_TIMEOUT_SECONDS)"`
- Observed result: prints `88.0` (default `30.0`; an unparseable value
falls back to `30.0`)
- Not tested: a live compression actually exceeding the configured
timeout under real load

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 21:10:01 -05:00