mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
feat/https-compose-caddy
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
6c0e817bcc |
fix(proxy): discard partial SSE tails during finalization (#3411)
## Description Closes #3405. When an SSE stream is interrupted in the middle of an event or UTF-8 character, the streaming finalizer appended `\n\n` to the residual byte buffer. That synthetic terminator made the incomplete event look complete and sent it through the strict UTF-8 decoder, causing a secondary `UnicodeDecodeError` during cleanup. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (breaking change that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Stop synthesizing an SSE event terminator during Anthropic streaming finalization; discard the unterminated residual tail instead. - Apply the same rule to the OpenAI-via-backend streaming cleanup path. - Keep complete events parsed during normal chunk processing, while preventing partial SSE/UTF-8 data from being decoded as a complete event. - Replace the old truncated-buffer recovery test with a regression covering a truncated UTF-8 tail and asserting finalization completes without raising. ## Testing - [x] Focused unit tests pass - [x] Linting passes - [x] Formatting passes - [x] New regression coverage added - [ ] Full test suite - [ ] Live upstream streaming run ### Test Output ```text python -m pytest tests/test_proxy_streaming_request_logger.py -q 9 passed in 71.37s python -m pytest tests/test_sse_byte_buffer_policy.py tests/test_sse_utf8_split.py -q 8 passed in 10.78s python -m ruff check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_request_logger.py All checks passed! python -m ruff format --check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_request_logger.py 2 files already formatted ``` The focused proxy test used the published `headroom-ai==0.37.0` wheel only for its compatible Rust extension; the source changes and tests ran from this checkout. ## Real Behavior Proof - Environment: Windows 10, Python 3.11.9, pytest 9.1.1. - Exact command / steps: ran the new regression with the production finalizer reverted; it failed with `UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe5 ... unexpected end of data`. Restored the fix and ran the regression plus the existing SSE parser tests. - Observed result: the old implementation fails when finalization appends a terminator to an unterminated event ending in a partial UTF-8 byte; the fixed implementation clears the incomplete tail without raising, and the focused proxy suite passes 9 tests. - Not tested: a live upstream provider disconnect; the regression exercises the exact residual-buffer/finalizer path with a partial UTF-8 sequence. ## Runtime Rollout Safety - Rollout-managed feature(s): None; this is an internal parser/finalization bug fix with no feature flag or staged rollout. - Minimum rollout channel: Standard release containing this bug fix; no separate rollout channel is required. - Stable/default behavior changed: No; complete SSE events remain parsed during normal streaming, while only unterminated residual tails are discarded during finalization. - Kill switch / disable path: Not applicable; revert this PR or the release containing it. - Unsafe override required: No. - Qualification impact: None; there are no API or configuration changes, and focused regression coverage was added. - Rollback path: Revert this change; the prior behavior may reintroduce a secondary UnicodeDecodeError on interrupted partial UTF-8 SSE tails. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] Focused unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes Complete SSE events continue to be parsed during streaming. Only an unterminated residual tail is discarded at finalization, so interrupted cleanup cannot turn incomplete wire data into a parser exception. --------- Signed-off-by: UniversePeak <UniversePeak@users.noreply.github.com> Co-authored-by: UniversePeak <UniversePeak@users.noreply.github.com> |
||
|
|
1c9585d42e |
fix(stats): tag streamed output token source (#2214)
## Description Preserve the existing SSE output-token fallback while making its provenance visible to request logs and downstream statistics. Closes #2213 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Tag provider-reported streaming output tokens with `output_tokens_source=provider`. - Tag the existing `total_bytes // 40` fallback with `output_tokens_source=estimated_bytes`. - Copy incoming tags before adding provenance so caller-owned dictionaries are not mutated. - Add focused coverage for both source values and the unchanged fallback estimate. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --with pytest --with pytest-asyncio --with fastapi --with httpx --with numpy pytest \ tests/test_proxy_streaming_request_logger.py \ tests/test_request_outcome.py \ tests/test_proxy_handler_helpers.py -q 77 passed $ uv run --with ruff ruff check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_request_logger.py All checks passed! $ uv run --with ruff ruff format --check headroom/proxy/handlers/streaming.py tests/test_proxy_streaming_request_logger.py 2 files already formatted ``` ## Real Behavior Proof - Environment: Python 3.13 with the real request logger and synthetic stream state - Exact command / steps: run the focused test set above - Observed result: parsed usage records `provider`; a 200-byte no-usage stream still records 5 output tokens and tags it `estimated_bytes` - Not tested: live provider stream, full repository suite ## Review Readiness - [x] I have performed a self-review - [ ] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes This PR does not change the fallback formula or token totals. Documentation and changelog changes are not needed for the new internal outcome tag. Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
2269e40bde |
feat(proxy): log compressed messages alongside original request (#261)
## Description Expose the post-compression message list that was actually sent upstream as a new `compressed_messages` field on `RequestLog`, paired with the existing (now consistently pre-compression) `request_messages`. Consumers of `/transformations/feed` — dashboards and any downstream observability — can now diff the two sides of a compression to see exactly what the pipeline stripped, replaced, or kept. Turns an abstract "saved N tokens" into a legible before/after. Gated by the same `log_full_messages` flag as `request_messages` so the two sides stay in sync; it's pointless to store one without the other. Also fixes a latent correctness bug: today's `request_messages` field is inconsistent across the four `RequestLog` construction sites — sometimes it's the pre-compression snapshot, sometimes it's the mutated `body["messages"]` (which is the compressed list, because the proxy mutates `body` in place before the log call). After this change, `request_messages` always means pre-compression and `compressed_messages` always means what went upstream. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) Note on "breaking": strictly speaking this is a semantic correction of an inconsistently-populated field, not a schema break. The field name `request_messages` is unchanged and the JSON shape is unchanged; what changes is that the field now consistently holds the pre-compression list. Consumers that treated it as "whatever messages we have" continue to work. Consumers that depended on the accidental post-compression value (if any existed) would shift to `compressed_messages`. ## Changes Made - **`headroom/proxy/models.py`**: `RequestLog` gains `compressed_messages: list[dict] | None = None`. Doc comment explains it's paired with `request_messages` and gated by the same `log_full_messages` flag. - **`headroom/proxy/handlers/anthropic.py`** (2 sites — Bedrock non-streaming and main non-streaming): `request_messages` now consistently sources from `original_messages` (the pre-compression snapshot at line 724), `compressed_messages` sources from `body["messages"]` (the compressed list after in-place mutation at line 1189). Both gated symmetrically. - **`headroom/proxy/handlers/streaming.py`** (2 sites — main streaming in `_finalize_stream_response`, Bedrock streaming in `_stream_response_bedrock`): same treatment. `_stream_response_bedrock` gains a new `original_messages: list[dict] | None = None` parameter so it has access to the pre-compression snapshot; the sole caller in `anthropic.py` now threads it through. - **`headroom/proxy/server.py`**: `/transformations/feed` adds `compressed_messages` to the JSON payload alongside the existing `request_messages` / `response_content`. *Split into a separate preceding commit is a one-time EOL normalization to LF — the file blob in history carries CRLF but `.gitattributes` declares `*.py text eol=lf`, so any contributor editing `server.py` triggers the same whole-file renormalization. Separating the two commits keeps this feature commit's diff at a single line.* - **`headroom/proxy/request_logger.py`**: `compressed_messages` is stripped from the JSONL file log and from `get_recent()` alongside the existing `request_messages` / `response_content` stripping. `get_memory_stats()` also counts it toward the deque's byte budget. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .` and `ruff format --check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed (via the Headroom Desktop client that consumes `/transformations/feed` — confirmed both fields arrive and render) Test coverage added/extended: - `tests/test_proxy/test_request_logger.py` (new file): round-trip unit tests for `RequestLogger`. Confirms `get_recent` strips both sides (pre + post), `get_recent_with_messages` exposes both, and the JSONL file log drops both when `log_full_messages=False`. - `tests/test_proxy/test_transformations_feed.py`: extended to assert `compressed_messages` appears in the endpoint payload alongside `request_messages` / `response_content`. - `tests/test_proxy_streaming_request_logger.py`: existing include/omit tests updated to assert both sides populate when the flag is on and both are `None` when it's off. ## Test Output ``` $ uv run ruff check headroom tests All checks passed! $ uv run ruff format --check headroom tests 614 files already formatted $ uv run pytest tests/test_proxy/test_request_logger.py tests/test_proxy_streaming_request_logger.py tests/test_proxy/test_transformations_feed.py -v ... tests/test_proxy/test_request_logger.py::test_get_recent_strips_compressed_messages_alongside_request_and_response PASSED tests/test_proxy/test_request_logger.py::test_get_recent_with_messages_returns_compressed_messages PASSED tests/test_proxy/test_request_logger.py::test_jsonl_file_strips_both_sides_when_log_full_messages_disabled PASSED tests/test_proxy_streaming_request_logger.py::test_finalize_stream_response_logs_original_and_compressed_messages PASSED tests/test_proxy_streaming_request_logger.py::test_finalize_stream_response_omits_messages_when_log_full_messages_disabled PASSED tests/test_proxy/test_transformations_feed.py::test_transformations_feed_returns_messages PASSED ... ============================== 11 passed in 5.36s ============================== ``` ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas (the two-sided gating at each log site, the `_stream_response_bedrock` parameter addition, and the `get_memory_stats` accounting) - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes ### Non-Anthropic backends `handlers/openai.py` and `handlers/gemini.py` do not currently emit `RequestLog` entries at all — only Anthropic and the shared streaming paths do. This PR therefore only populates `compressed_messages` on Anthropic traffic (which is what `/transformations/feed` shows today). Wiring OpenAI and Gemini into `RequestLogger` end-to-end is a separate, larger gap worth its own PR. ### `server.py` EOL normalization The feature change in `server.py` is a single line. To keep the diff readable, the preceding commit is a whitespace-only `chore(proxy): normalize server.py to LF per .gitattributes` — the file blob was stored with CRLF terminators but `.gitattributes` declares `*.py text eol=lf`. Any contributor touching `server.py` triggers this renormalization; isolating it here keeps the feature commit reviewable. Happy to rebase / drop / reshape as preferred. ### Downstream desktop compatibility The Headroom Desktop client I work on now consumes `compressed_messages` and renders the pre/post pair side-by-side on the "Recent large compression" card. The desktop was updated to handle both shapes: proxies without the field render the legacy single "Request" block; proxies with the field render "Request (original, N tokens)" + "Request (compressed, M tokens)" where N/M come from `input_tokens_original` / `input_tokens_optimized`. No changes needed downstream if this PR lands. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
c7af8307d1 |
Merge branch 'main' into fix/codex-wrap-proxy-mcp-sync
Resolve conflict in tests/test_proxy_streaming_request_logger.py by keeping both new tests: - test_finalize_openai_responses_stream_uses_provider_usage_for_dashboard (this branch — OpenAI Responses stream dashboard usage) - test_finalize_stream_response_recovers_usage_from_truncated_buffer (main — SSE buffer recovery on truncated streams) streaming.py auto-merged cleanly (additive only). |
||
|
|
eaf5980b4a | fix: stabilize codex compression, stats, and proxy lifecycle | ||
|
|
d91254a0f2 |
fix(proxy): recover SSE usage from truncated final event
The per-chunk SSE parser only flushes events terminated by `\n\n`. When upstream truncates mid-event (client disconnect, network drop, RemoteProtocolError), the message_start (cache_read / cache_creation) or message_delta (output_tokens) usage events sit in the residual sse_buffer and never get parsed — the finalizer then logs cache_read=cache_write=0, which the freeze heuristic on the next request reads as "no provider cache, reprocess everything," producing a different prefix and a real cache miss on the *next* turn. Append `\n\n` to the residual buffer at end-of-stream so the existing parser drains the partial event. Only fills None / 0 slots so a real cache_read=0 from earlier in the stream isn't clobbered. |
||
|
|
9bde245c43 |
fix(proxy): log streaming Anthropic requests so feed populates
`_finalize_stream_response` recorded metrics, cost, and prefix-cache stats but never appended a `RequestLog` to the request logger. Because the streaming Anthropic path is what Claude Code uses, this meant `/stats.recent_requests` and `/transformations/feed` were permanently empty for typical traffic — even when the proxy was started with `--log-messages`. Only the non-streaming Anthropic path (`anthropic.py:1167, 1599`) and the Bedrock streaming finalizer (`streaming.py:_stream_response_bedrock`) were logged. Wire the same `RequestLog` shape the other paths build, plumbing `tags` through both `_finalize_stream_response` call sites in `_stream_response` and respecting `config.log_full_messages` for `request_messages`. Adds `tests/test_proxy_streaming_request_logger.py` covering the happy path, both `log_full_messages` branches, the zero-original-tokens edge, and the logger-disabled no-op. |