mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(proxy): strip only one leading space from SSE field values (#4306)
## Description `parse_sse_events_from_byte_buffer` called `.lstrip()` on `event:` and `data:` values, which removes all leading whitespace. The SSE spec ([WHATWG §9.2.6](https://html.spec.whatwg.org/multipage/server-sent-events.html#event-stream-interpretation)) says to remove only one leading U+0020 space; anything after it belongs to the value. So `data: indented` came back as `indented` instead of ` indented`, and `data:\tx` lost its tab. The streaming handler and the CCR response handler both use this parser, so payload whitespace from upstream streams was changed silently. No linked issue; found by reading the parser against the spec. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/sse_byte_buffer_policy.py`: a small `_field_value` helper strips at most one leading space; used for both the `event:` and the `data:` value. - `tests/test_sse_byte_buffer_policy.py`: `test_parse_sse_events_strips_only_one_leading_space`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed The new test fails on `main` and passes with the fix. `ruff check` / `ruff format --check` are clean on the two touched files. ### Test Output ```text $ pytest -q tests/test_sse_byte_buffer_policy.py tests/test_streaming_sse_malformed_events.py tests/test_sse_utf8_split.py ============================= 361 passed in 2.50s ============================== ``` ## Real Behavior Proof - Environment: Linux 6.12, Python 3.13.5, headroom at this branch imported from source, no provider calls. - Exact command / steps: a script that passes this stream through `headroom.proxy.helpers.parse_sse_events_from_byte_buffer` and prints each `(event, data)`: `event: message` / `data: indented code` / `data:<TAB>x = 1` / `data: plain` / blank, then `event: spaced` / `data: {"type":"ping"}` / blank. - Observed result: additional leading spaces and tabs are preserved, as shown below. ```text 'message' ' indented code\n\tx = 1\nplain' ' spaced' '{"type":"ping"}' ``` Same script on `main`: ```text 'message' 'indented code\nx = 1\nplain' 'spaced' '{"type":"ping"}' ``` - Not tested: a live upstream whose payload starts with whitespace, and the full proxy end to end. JSON payloads (the normal Anthropic/OpenAI case) parse the same either way, since JSON ignores leading whitespace. ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: n/a - Stable/default behavior changed: only values with more than one leading space or a leading tab now keep that whitespace, as the spec requires; `data: {...}` is unchanged - Kill switch / disable path: none (parser fix) - Unsafe override required: no - Qualification impact: none expected - Rollback path: revert this PR ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (n/a, internal parser) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes `memory_tool_stream._parse_frame` still `lstrip()`s `data:` values, as Devin noted. Its parsed payloads are only fed to `json.loads` and unchanged frames are forwarded as raw bytes, so the extra stripping there has no visible effect; I left it alone to keep this PR small, but can align it here if you prefer. _AI-assistance note: I found and wrote this change with help from an AI coding assistant. I reviewed it and ran the tests and the script above myself._ Maintainer validation on the current-main integration: Windows/Python 3.12, real loopback HTTP SSE response consumed using `httpx.Client.stream` and `iter_raw(chunk_size=1)`, appending each byte to the shared parser buffer. Input `event: spaced` with data lines containing three leading spaces, a tab, and one space yielded exactly `(" spaced", " indented code\n\tx = 1\nplain")`, with zero residual bytes. All 421 focused parser/malformed-stream/UTF-8/memory-stream tests passed; changed-file lint, formatting, and whitespace checks passed. No live external provider is claimed. Hosted checks for the published sync remain pending. Co-authored-by: kevin9327 <kevin9327@users.noreply.github.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
kevin9327
JD Davis
parent
911bb74074
commit
9226eaf385
@@ -10,6 +10,12 @@ SSE_EVENT_LINE_PREFIX = "event:"
|
||||
SSE_DATA_LINE_PREFIX = "data:"
|
||||
|
||||
|
||||
def _field_value(line: str, prefix: str) -> str:
|
||||
"""Value after ``prefix``; per the SSE spec only one leading space is removed."""
|
||||
value = line[len(prefix) :]
|
||||
return value[1:] if value.startswith(" ") else value
|
||||
|
||||
|
||||
def find_sse_event_terminator(buf: bytearray) -> tuple[int, int] | None:
|
||||
"""Return the earliest complete SSE event terminator in ``buf``."""
|
||||
matches = [
|
||||
@@ -51,9 +57,9 @@ def parse_sse_events_from_byte_buffer(
|
||||
if line.startswith(":"):
|
||||
continue
|
||||
if line.startswith(SSE_EVENT_LINE_PREFIX):
|
||||
event_name = line[len(SSE_EVENT_LINE_PREFIX) :].lstrip()
|
||||
event_name = _field_value(line, SSE_EVENT_LINE_PREFIX)
|
||||
elif line.startswith(SSE_DATA_LINE_PREFIX):
|
||||
data_lines.append(line[len(SSE_DATA_LINE_PREFIX) :].lstrip())
|
||||
data_lines.append(_field_value(line, SSE_DATA_LINE_PREFIX))
|
||||
|
||||
if data_lines:
|
||||
events.append((event_name, "\n".join(data_lines)))
|
||||
|
||||
@@ -31,3 +31,11 @@ def test_parse_sse_events_preserves_split_utf8_tail() -> None:
|
||||
def test_parse_sse_events_raises_on_complete_invalid_utf8_event() -> None:
|
||||
with pytest.raises(UnicodeDecodeError):
|
||||
parse_sse_events_from_byte_buffer(bytearray(b"data: \xff\n\n"))
|
||||
|
||||
|
||||
def test_parse_sse_events_strips_only_one_leading_space() -> None:
|
||||
# SSE spec: "If value starts with a U+0020 SPACE character, remove it".
|
||||
# Further leading whitespace is part of the payload.
|
||||
buf = bytearray(b"event: delta\ndata: indented\ndata:\ttab\ndata:none\n\n")
|
||||
|
||||
assert parse_sse_events_from_byte_buffer(buf) == [("delta", " indented\n\ttab\nnone")]
|
||||
|
||||
Reference in New Issue
Block a user