mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat/https-compose-caddy
1
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
a507249b92 |
fix(proxy): give the upstream write phase its own timeout (#3327)
## Description Partially addresses #3259. `write` inherited `request_timeout_seconds`, so pushing request bytes got the same 300s budget as waiting for a model to answer: ```python httpx.Timeout( connect=config.connect_timeout_seconds, # 10 read=config.request_timeout_seconds, # 300 write=config.request_timeout_seconds, # 300 <- same knob pool=config.connect_timeout_seconds, # 10 ) ``` Sending a request and waiting for a model to think are different operations, and conflating them left the send **effectively unbounded in practice**. When an upstream stops draining — a pooled socket whose peer went away over a laptop sleep, a network change, a NAT timeout — the send blocks until the OS abandons retransmission, ~180-220s on macOS. That is *under* the inherited 300s, so httpx never timed out and the OS ended the request instead. The retry then hit the same wall and stalled again, which is why one incident blocked the reporter's client for 5-10 minutes. `--connect-timeout-seconds` cannot help, exactly as the reporter says: it only guards establishing a *fresh* connection, and a reused one never connects. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Added `ProxyConfig.write_timeout_seconds` (default **150**), with `--write-timeout-seconds` / `HEADROOM_WRITE_TIMEOUT_SECONDS` and a `SettingField` so it appears in the settings registry. - Used it in `_provider_httpx_client_options` **and** in `_anthropic_buffered_request_timeout` — the buffered path sets its own timeout and is the path most likely to carry a large body, so it needed the split too or it would have kept the unbounded write. Behaviour after: ``` BEFORE: connect=10 read=300 write=300 pool=10 AFTER: connect=10 read=300 write=150 pool=10 ``` ### What `write` actually bounds, and why the default is 150s The first revision of this PR used 60s and claimed httpx applies `write` per write operation, so a large body would be unaffected. **That was wrong, and I measured it before shipping the correction.** httpx hands a bytes body to the transport as a *single* write (`ByteStream.__aiter__` yields the whole body; httpcore's h11 `_send_request_body` wraps each yielded chunk in one `anyio.fail_after(write)`), so on HTTP/1.1 the entire upload runs inside one timer. Against a local peer draining a 16MB body at a steady ~1MB/s: ``` write=3.0: WriteTimeout after 3.0s <- healthy peer, still cut off write=60.0: send completed in 19.1s ``` So this is a bound on total upload time, not a stall detector. 60s would have cut off the 7-15MB bodies the reporter sends over a slow uplink — turning a fix into an outage for exactly the person who filed the issue. 150s is the value that satisfies both real constraints: it carries 15MB (the largest body #3259 reports) over a ~1 Mbps uplink, and still fires well before the ~180s OS retransmit ceiling that made the inherited 300s unreachable. `tests/test_upstream_write_timeout.py` pins both ends of that range so neither can drift. On HTTP/2 the body is split by flow control and the waiting-for-window part is charged to `read` (httpcore `_wait_for_outgoing_flow` → `_receive_events`), so only real socket writes count against the bound there. A genuinely per-chunk bound would separate "dead peer" from "slow peer" perfectly, but it needs the body fed as an iterator without changing the wire framing. That is a larger change than this fix warrants; worth a follow-up if slow-uplink users hit the 150s cap. ### Why the read timeout is deliberately left alone Time-to-first-byte is a legitimately long wait for a model, and the buffered Anthropic path already sets its own 600s read for exactly that reason. Lowering `read` globally would break slow legitimate responses to fix a rare hang — the wrong trade. Operators who want a tighter read bound already have `--request-timeout-seconds`; before this PR that knob also silently shortened the write budget, which is the conflation being removed. ### Failover `_retry_request` catches `httpx.TransportError`, and `WriteTimeout` is a `TimeoutException` under it — so a bounded write drops the bad connection and re-sends on a fresh one instead of stalling behind a socket whose peer is gone. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (CI-pinned `ruff` 0.16.3) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality ### Test Output ```text $ pytest tests/test_proxy/ tests/test_proxy_hardening.py tests/test_upstream_write_timeout.py 302 passed in 40.46s $ pytest tests/test_upstream_write_timeout.py tests/test_proxy/test_anthropic_buffered_timeout.py \ tests/test_proxy/test_settings_store.py tests/test_cli_proxy_env.py tests/test_cli_proxy_improvements.py 181 passed, 1 skipped in 11.65s ``` ## Scope This does **not** explain the reporter's follow-up: consistent ~60s stalls on 7-15MB bodies with `HEADROOM_MAX_KEEPALIVE=0`, where there is no pooled socket to inherit. I would not close #3259 on this PR. --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |