mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-23 02:49:17 +08:00
## Description `POST /v1/compress/response` (the response half of the gateway contract) caught every body-read failure and answered `"Invalid JSON in request body."`. That covered malformed JSON, but also an oversize body, a failed decompression, invalid UTF-8 and a JSON array. A Kong or LiteLLM integrator debugging a 400 could not tell which one it was. The other handlers (anthropic, openai, gemini, batch; about 10 call sites) already use `helpers.invalid_request_body_message(exc)`. It picks a fixed string by exception type, so it never echoes the exception text that CodeQL's `py/stack-trace-exposure` flags, and it logs the exception type on the proxy side. This handler now uses it too. That gives specific messages for malformed JSON, a non-object, invalid UTF-8 and an oversize body. A failed decompression (for example a bad gzip stream) still gets the generic `Invalid request body`, as it does on every other handler. A separate decompression message would mean changing the shared helper for all handlers, so that is out of scope here. The `if not isinstance(body, dict)` branch that followed is removed. `_read_request_json` raises `RequestBodyNotObject` for a non-object before it can return one, so the branch could not be reached. Its case is now reported as `Invalid request body: must be a JSON object`. The status code (400) and the error `type` (`invalid_request`) are unchanged. Only the `message` text differs. No test, doc or consumer repo matched on the old string: I grepped `tests`, `docs`, `headroom` and the gateway integrations, which branch on the status code. Refs #4136 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/gateway_turn.py`: `handle_compress_response` answers 400 with `invalid_request_body_message(exc)`, and the unreachable non-dict check is gone. - `tests/gateway/test_contract_response_half.py`: add `test_invalid_body_400_says_why`, parametrized over three bodies. Each case asserts a 400, `invalid_request`, and a message that names the reason: - malformed JSON; - a JSON array; - invalid UTF-8. All three cases fail on main. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ pytest -q -p no:cacheprovider tests/gateway/test_contract_response_half.py -k invalid # before the fix FAILED tests/gateway/test_contract_response_half.py::test_invalid_body_400_says_why[malformed] FAILED tests/gateway/test_contract_response_half.py::test_invalid_body_400_says_why[not-object] FAILED tests/gateway/test_contract_response_half.py::test_invalid_body_400_says_why[not-utf8] ================== 3 failed, 6 passed, 28 deselected in 7.18s ================== $ pytest -q -p no:cacheprovider tests/gateway/ tests/test_gateway_turn_unit.py tests/test_invalid_request_body_message.py # after ================== 323 passed, 4 skipped in 171.41s (0:02:51) ================== $ uvx ruff@0.16.8 check headroom/proxy/gateway_turn.py tests/gateway/test_contract_response_half.py All checks passed! $ mypy headroom/proxy/gateway_turn.py --ignore-missing-imports Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: macOS, Python 3.12.13, branch checkout atae6a419a, using the repo .venv. A temp HOME, workspace and CODEX_HOME were used, with offline mode on. - Exact command / steps: 1. Ran the shared smoke harness, which starts a real proxy from this checkout against a local fake upstream and sends one Anthropic and one OpenAI request. 2. Started a real proxy with `python -m headroom.cli proxy --port <p>` and used curl to POST five bodies to `http://127.0.0.1:<p>/v1/compress/response`: four bad ones, and one valid body with an unknown turn as a control. - Observed result: ```text branch: cleanup/proxy-core-a-compress-response-errors @ae6a419ahealth: 200 POST /v1/messages -> HTTP200 smoke-ok POST /v1/chat/completions -> HTTP200 smoke-ok stats: requests=2 failed=0 tokens_saved=13574 savings_percent=99.82 proxy.log WARNING/ERROR lines (<tmp>): (smoke dir: <tmp>) ``` ```text malformed JSON -> {"error":{"type":"invalid_request","message":"Invalid request body: malformed JSON"}} HTTP400 JSON array -> {"error":{"type":"invalid_request","message":"Invalid request body: must be a JSON object"}} HTTP400 invalid UTF-8 -> {"error":{"type":"invalid_request","message":"Invalid request body: not valid UTF-8 (possibly compressed?)"}} HTTP400 bad gzip -> {"error":{"type":"invalid_request","message":"Invalid request body"}} HTTP400 valid, unknown turn -> {"error":{"type":"unknown_turn","message":"No pending turn 't-x' (never registered, already finished, or expired)."}} HTTP404 ``` Every bad body is a 400 with type `invalid_request`, and the message names the reason. The exception is bad gzip, which gets the generic message, as on every other handler. On main, all four said `Invalid JSON in request body.`. The control request still reaches turn lookup (404 `unknown_turn`). - Not tested: A live Kong or LiteLLM gateway. Both key on the status and `type`, which are unchanged. ## Runtime Rollout Safety - Rollout-managed feature(s): None (the gateway contract is controlled by `HEADROOM_GATEWAY_CONTRACT`, unchanged). - Minimum rollout channel: N/A; error-message text only. - Stable/default behavior changed: Only the `error.message` text on a 400 for a bad body. The status, error type and every success path are unchanged. - Kill switch / disable path: N/A. - Unsafe override required: No. - Qualification impact: None. - Rollback path: Revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes Part of a maintainer-requested codebase cleanup (dead code, duplication, debuggability), split into small PRs for review. It is part of maintainer-requested cleanup #4136. No docs quote the old message, so none needed changing. The change is small and self-explanatory, so no new comments were added. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
477 lines
18 KiB
Python
477 lines
18 KiB
Python
"""Response half of the gateway turn contract: ``POST /v1/compress/response`` (spec section 2).
|
|
|
|
Invariants protected here:
|
|
|
|
* R-LOOKUP - unknown/expired turn -> 404 ``unknown_turn``; a completed turn is
|
|
gone (second post is 404).
|
|
* R-VALID - 400 on missing/invalid ``turn_id``, non-object ``response``,
|
|
non-object/negative/bool usage, and a missing ``response`` on a
|
|
turn that carries ``redrive``.
|
|
* R-DONE - shape of the ``done`` answer; ``response`` is null unless a hook
|
|
replaced it.
|
|
* R-USAGE - Anthropic / OpenAI / Kong usage shapes feed the session tracker
|
|
exactly like ``/v1/usage`` (same ``frozen_message_count``); a
|
|
usage with no cache signal is accepted but not applied.
|
|
* R-OUTCOME - with ``relay_usage`` the RequestOutcome is deferred and recorded
|
|
exactly once, completed with the provider's numbers; without it
|
|
the outcome is recorded at the request half as today; registry
|
|
expiry records the draft.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import time
|
|
from typing import Any
|
|
|
|
import pytest
|
|
|
|
from tests.gateway.conftest import compress, count_loop_tasks, registry_of
|
|
from tests.gateway.samples import big_tool_history
|
|
|
|
RESPONSE_HALF = "/v1/compress/response"
|
|
DONE_KEYS = {
|
|
"action",
|
|
"turn_id",
|
|
"response",
|
|
"frozen_message_count",
|
|
"usage_applied",
|
|
"rounds",
|
|
"billed_usage",
|
|
}
|
|
|
|
|
|
def _open_turn(client, session_id: str | None = "resp-s", **gateway: Any) -> dict[str, Any]:
|
|
gateway.setdefault("can_relay_response", True)
|
|
body: dict[str, Any] = {"model": "gpt-4o", "messages": big_tool_history(), "gateway": gateway}
|
|
if session_id is not None:
|
|
body["config"] = {"session_id": session_id}
|
|
resp = compress(client, body)
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert "relay_usage" in data["obligations"]
|
|
return data
|
|
|
|
|
|
def _post(client, payload: dict[str, Any]):
|
|
return client.post(RESPONSE_HALF, json=payload)
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# R-LOOKUP / R-VALID #
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_unknown_turn_is_404(headroom_client) -> None:
|
|
"""R-LOOKUP"""
|
|
resp = _post(headroom_client, {"turn_id": "0" * 32, "usage": {"prompt_tokens": 1}})
|
|
assert resp.status_code == 404, resp.text
|
|
assert resp.json()["error"]["type"] == "unknown_turn"
|
|
|
|
|
|
def test_unknown_turn_leaves_no_footprint(headroom_client) -> None:
|
|
"""R-LOOKUP: a flood of bogus ids must not grow the registry."""
|
|
registry = registry_of(headroom_client)
|
|
assert registry is not None
|
|
before = len(registry)
|
|
for i in range(10):
|
|
assert _post(headroom_client, {"turn_id": f"{i:032x}"}).status_code == 404
|
|
assert len(registry) == before
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"payload",
|
|
[
|
|
{},
|
|
{"turn_id": None},
|
|
{"turn_id": 42},
|
|
{"turn_id": ""},
|
|
{"turn_id": "x" * 300},
|
|
],
|
|
ids=["missing", "null", "int", "empty", "too-long"],
|
|
)
|
|
def test_invalid_turn_id_is_400(headroom_client, payload: dict[str, Any]) -> None:
|
|
"""R-VALID"""
|
|
resp = _post(headroom_client, payload)
|
|
assert resp.status_code == 400, resp.text
|
|
assert resp.json()["error"]["type"] == "invalid_request"
|
|
|
|
|
|
def test_invalid_json_is_400(headroom_client) -> None:
|
|
"""R-VALID"""
|
|
resp = headroom_client.post(
|
|
RESPONSE_HALF, content=b"nope", headers={"content-type": "application/json"}
|
|
)
|
|
assert resp.status_code == 400
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("content", "reason"),
|
|
[
|
|
(b"nope", "malformed JSON"),
|
|
(b"[1, 2]", "must be a JSON object"),
|
|
(b"\xff\xfe{}", "not valid UTF-8"),
|
|
],
|
|
ids=["malformed", "not-object", "not-utf8"],
|
|
)
|
|
def test_invalid_body_400_says_why(headroom_client, content: bytes, reason: str) -> None:
|
|
"""The 400 names the reason, like every other handler's body errors."""
|
|
resp = headroom_client.post(
|
|
RESPONSE_HALF, content=content, headers={"content-type": "application/json"}
|
|
)
|
|
assert resp.status_code == 400
|
|
error = resp.json()["error"]
|
|
assert error["type"] == "invalid_request"
|
|
assert reason in error["message"]
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"extra",
|
|
[
|
|
{"response": "not-an-object"},
|
|
{"response": [1, 2]},
|
|
{"usage": "nope"},
|
|
{"usage": {"prompt_tokens": -1}},
|
|
{"usage": {"cache_read_input_tokens": True}},
|
|
{"usage": {"prompt_tokens_details": {"cached_tokens": -5}}},
|
|
],
|
|
ids=[
|
|
"response-str",
|
|
"response-list",
|
|
"usage-str",
|
|
"usage-negative",
|
|
"usage-bool",
|
|
"cached-negative",
|
|
],
|
|
)
|
|
def test_malformed_response_or_usage_is_400_and_keeps_turn(
|
|
headroom_client, extra: dict[str, Any]
|
|
) -> None:
|
|
"""R-VALID: a rejected relay must not consume the turn."""
|
|
turn = _open_turn(headroom_client)
|
|
resp = _post(headroom_client, {"turn_id": turn["turn_id"], **extra})
|
|
assert resp.status_code == 400, resp.text
|
|
assert resp.json()["error"]["type"] == "invalid_request"
|
|
assert registry_of(headroom_client).get(turn["turn_id"]) is not None
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# R-DONE #
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_done_with_null_response_and_turn_removed(headroom_client) -> None:
|
|
"""R-DONE + R-LOOKUP: no hook ran, so ``response`` is null; the turn is consumed."""
|
|
turn = _open_turn(headroom_client)
|
|
resp = _post(
|
|
headroom_client,
|
|
{
|
|
"turn_id": turn["turn_id"],
|
|
"status": 200,
|
|
"latency_ms": 12.5,
|
|
"model": "gpt-4o-2024-08-06",
|
|
"usage": {"prompt_tokens": 100, "completion_tokens": 7},
|
|
},
|
|
)
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert set(data) == DONE_KEYS
|
|
assert data["action"] == "done"
|
|
assert data["turn_id"] == turn["turn_id"]
|
|
assert data["response"] is None
|
|
assert data["rounds"] == 0
|
|
assert data["usage_applied"] is False # no cache signal in that usage
|
|
assert data["billed_usage"] == {"input_tokens": 100, "output_tokens": 7}
|
|
assert isinstance(data["frozen_message_count"], int)
|
|
assert registry_of(headroom_client).get(turn["turn_id"]) is None
|
|
again = _post(headroom_client, {"turn_id": turn["turn_id"]})
|
|
assert again.status_code == 404
|
|
assert again.json()["error"]["type"] == "unknown_turn"
|
|
|
|
|
|
def test_done_without_session_has_null_frozen_count(headroom_client) -> None:
|
|
"""R-DONE: stateless turns carry no tracker, so ``frozen_message_count`` is null
|
|
and usage is never applied."""
|
|
turn = _open_turn(headroom_client, session_id=None)
|
|
resp = _post(
|
|
headroom_client,
|
|
{
|
|
"turn_id": turn["turn_id"],
|
|
"usage": {"input_tokens": 10, "output_tokens": 2, "cache_read_input_tokens": 5000},
|
|
},
|
|
)
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert data["action"] == "done"
|
|
assert data["frozen_message_count"] is None
|
|
assert data["usage_applied"] is False
|
|
|
|
|
|
def test_done_with_minimal_payload(headroom_client) -> None:
|
|
"""R-DONE: ``turn_id`` alone is a valid relay (status defaults to 200)."""
|
|
turn = _open_turn(headroom_client)
|
|
resp = _post(headroom_client, {"turn_id": turn["turn_id"]})
|
|
assert resp.status_code == 200, resp.text
|
|
assert resp.json()["action"] == "done"
|
|
assert resp.json()["usage_applied"] is False
|
|
|
|
|
|
def test_done_leaves_no_tasks_behind(headroom_client) -> None:
|
|
"""R-DONE: a hook-less turn never starts a suspended runner, and after ``done``
|
|
the loop has exactly the tasks it had before."""
|
|
baseline = count_loop_tasks(headroom_client)
|
|
turn = _open_turn(headroom_client)
|
|
_post(headroom_client, {"turn_id": turn["turn_id"], "usage": {"prompt_tokens": 1}})
|
|
assert count_loop_tasks(headroom_client) <= baseline
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# R-USAGE #
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
ANTHROPIC_WARM = {
|
|
"input_tokens": 200,
|
|
"output_tokens": 9,
|
|
"cache_read_input_tokens": 0,
|
|
"cache_creation_input_tokens": 50_000,
|
|
}
|
|
OPENAI_WARM = {
|
|
"prompt_tokens": 50_200,
|
|
"completion_tokens": 9,
|
|
"prompt_tokens_details": {"cached_tokens": 50_000},
|
|
}
|
|
OPENAI_TOPLEVEL_CACHED = {"prompt_tokens": 50_200, "completion_tokens": 9, "cached_tokens": 50_000}
|
|
KONG_NESTED = {"usage": OPENAI_WARM} # Kong ai.proxy log statistics keep it nested
|
|
|
|
|
|
def _frozen_after_legacy_usage(client, history, usage: dict[str, Any]) -> int:
|
|
"""The yardstick: what ``/v1/usage`` does with the Anthropic-shaped equivalent."""
|
|
sid = f"legacy-{time.time_ns()}"
|
|
assert (
|
|
compress(
|
|
client, {"model": "gpt-4o", "messages": history, "config": {"session_id": sid}}
|
|
).status_code
|
|
== 200
|
|
)
|
|
resp = client.post("/v1/usage", json={"session_id": sid, "usage": usage})
|
|
assert resp.status_code == 200, resp.text
|
|
assert resp.json()["applied"] is True
|
|
return resp.json()["frozen_message_count"]
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"usage",
|
|
[ANTHROPIC_WARM, OPENAI_WARM, OPENAI_TOPLEVEL_CACHED, KONG_NESTED],
|
|
ids=["anthropic", "openai", "openai-toplevel-cached", "kong-nested"],
|
|
)
|
|
def test_usage_shapes_apply_to_tracker_like_v1_usage(
|
|
headroom_client, usage: dict[str, Any]
|
|
) -> None:
|
|
"""R-USAGE: every accepted shape lands on the tracker and advances the frozen
|
|
count exactly as the equivalent ``/v1/usage`` relay does."""
|
|
history = big_tool_history()
|
|
expected = _frozen_after_legacy_usage(
|
|
headroom_client,
|
|
history,
|
|
{"cache_read_input_tokens": 0, "cache_creation_input_tokens": 50_000},
|
|
)
|
|
turn = _open_turn(headroom_client, session_id="usage-shape")
|
|
resp = _post(headroom_client, {"turn_id": turn["turn_id"], "usage": usage})
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert data["usage_applied"] is True
|
|
assert data["frozen_message_count"] == expected
|
|
assert data["frozen_message_count"] >= 1
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"usage",
|
|
[
|
|
{"prompt_tokens": 120, "completion_tokens": 3},
|
|
{"input_tokens": 120, "output_tokens": 3},
|
|
{"prompt_tokens": 120, "prompt_tokens_details": {"cached_tokens": 0}},
|
|
{"input_tokens": 120, "output_tokens": 3, "cache_read_input_tokens": 0},
|
|
],
|
|
ids=["openai-bare", "anthropic-bare", "openai-cached-zero", "anthropic-read-zero-only"],
|
|
)
|
|
def test_usage_without_cache_signal_is_accepted_but_not_applied(headroom_client, usage) -> None:
|
|
"""R-USAGE: unlike ``/v1/usage`` a signal-free relay is not a 400 - the response
|
|
half still completes the outcome - but it must not touch the tracker (applying
|
|
it would assert a fully-cold prefix the provider never reported)."""
|
|
turn = _open_turn(headroom_client, session_id="usage-nosignal")
|
|
before = turn["session"]["frozen_message_count"]
|
|
resp = _post(headroom_client, {"turn_id": turn["turn_id"], "usage": usage})
|
|
assert resp.status_code == 200, resp.text
|
|
data = resp.json()
|
|
assert data["action"] == "done"
|
|
assert data["usage_applied"] is False
|
|
tracker = headroom_client.app.state.proxy.session_tracker_store.peek(
|
|
"compress\x00usage-nosignal"
|
|
)
|
|
assert tracker is not None
|
|
assert tracker.get_frozen_message_count() == data["frozen_message_count"]
|
|
assert data["frozen_message_count"] <= max(before, len(big_tool_history()))
|
|
|
|
|
|
def test_both_cache_fields_zero_is_a_confirmed_cold_turn(headroom_client) -> None:
|
|
"""R-USAGE: both fields present and 0 is a genuine fully-cold signal (the same
|
|
rule ``/v1/usage`` applies) and IS applied."""
|
|
turn = _open_turn(headroom_client, session_id="usage-cold")
|
|
resp = _post(
|
|
headroom_client,
|
|
{
|
|
"turn_id": turn["turn_id"],
|
|
"usage": {
|
|
"input_tokens": 1,
|
|
"output_tokens": 1,
|
|
"cache_read_input_tokens": 0,
|
|
"cache_creation_input_tokens": 0,
|
|
},
|
|
},
|
|
)
|
|
assert resp.status_code == 200, resp.text
|
|
assert resp.json()["usage_applied"] is True
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# R-OUTCOME #
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_outcome_deferred_then_recorded_once_with_provider_numbers(
|
|
headroom_client, outcome_spy
|
|
) -> None:
|
|
"""R-OUTCOME: with ``relay_usage`` nothing is recorded at the request half; the
|
|
response half records exactly one outcome carrying output/input/cache tokens
|
|
and the gateway-measured latency."""
|
|
outcomes = outcome_spy(headroom_client)
|
|
turn = _open_turn(headroom_client, session_id="outcome-defer")
|
|
assert outcomes == [], "outcome must be deferred while relay_usage is pending"
|
|
resp = _post(
|
|
headroom_client,
|
|
{
|
|
"turn_id": turn["turn_id"],
|
|
"status": 200,
|
|
"latency_ms": 250.0,
|
|
"usage": {
|
|
"prompt_tokens": 1234,
|
|
"completion_tokens": 56,
|
|
"prompt_tokens_details": {"cached_tokens": 1000},
|
|
},
|
|
},
|
|
)
|
|
assert resp.status_code == 200, resp.text
|
|
assert len(outcomes) == 1
|
|
o = outcomes[0]
|
|
assert o.provider == "compress"
|
|
assert o.model == "gpt-4o"
|
|
assert o.output_tokens == 56
|
|
assert o.provider_input_tokens == 1234
|
|
assert o.cache_read_tokens == 1000
|
|
assert o.total_latency_ms >= 250.0
|
|
assert o.original_tokens == turn["tokens_before"]
|
|
assert o.optimized_tokens == turn["tokens_after"]
|
|
assert o.tokens_saved == turn["tokens_saved"]
|
|
assert o.status_code == 200
|
|
|
|
|
|
def test_outcome_anthropic_cache_fields(headroom_client, outcome_spy) -> None:
|
|
"""R-OUTCOME: Anthropic read + write buckets land on the outcome's cache fields."""
|
|
outcomes = outcome_spy(headroom_client)
|
|
turn = _open_turn(headroom_client, session_id="outcome-anthropic")
|
|
_post(
|
|
headroom_client,
|
|
{
|
|
"turn_id": turn["turn_id"],
|
|
"usage": {
|
|
"input_tokens": 300,
|
|
"output_tokens": 12,
|
|
"cache_read_input_tokens": 2000,
|
|
"cache_creation_input_tokens": 500,
|
|
},
|
|
},
|
|
)
|
|
assert len(outcomes) == 1
|
|
o = outcomes[0]
|
|
assert o.output_tokens == 12
|
|
assert o.cache_read_tokens == 2000
|
|
assert o.cache_write_tokens == 500
|
|
# Anthropic's input_tokens is the uncached remainder; the billed prompt is
|
|
# that plus both cache buckets, as the direct handler has always recorded.
|
|
assert o.provider_input_tokens == 2800
|
|
assert o.uncached_input_tokens == 300
|
|
|
|
|
|
def test_outcome_recorded_immediately_without_relay(headroom_client, outcome_spy) -> None:
|
|
"""R-OUTCOME: no ``relay_usage`` -> recorded at the request half, as today."""
|
|
outcomes = outcome_spy(headroom_client)
|
|
body = {
|
|
"model": "gpt-4o",
|
|
"messages": big_tool_history(),
|
|
"gateway": {"can_relay_response": False},
|
|
}
|
|
data = compress(headroom_client, body).json()
|
|
assert data["obligations"] == []
|
|
assert len(outcomes) == 1
|
|
assert outcomes[0].output_tokens == 0
|
|
|
|
|
|
def test_outcome_status_from_provider(headroom_client, outcome_spy) -> None:
|
|
"""R-OUTCOME: a provider 5xx relayed through the response half is recorded with
|
|
that status so it cannot inflate save-rate (same rule as the proxy path)."""
|
|
outcomes = outcome_spy(headroom_client)
|
|
turn = _open_turn(headroom_client, session_id="outcome-5xx")
|
|
resp = _post(headroom_client, {"turn_id": turn["turn_id"], "status": 529, "latency_ms": 5})
|
|
assert resp.status_code == 200, resp.text
|
|
assert resp.json()["action"] == "done"
|
|
assert len(outcomes) == 1
|
|
assert outcomes[0].status_code == 529
|
|
|
|
|
|
def test_expiry_records_draft_and_then_404(headroom_client, outcome_spy) -> None:
|
|
"""R-OUTCOME: a turn the gateway never completes is recorded as-is (output 0)
|
|
when the registry sweeps it, and a late relay is a 404."""
|
|
outcomes = outcome_spy(headroom_client)
|
|
turn = _open_turn(headroom_client, session_id="outcome-expire")
|
|
registry = registry_of(headroom_client)
|
|
assert registry is not None and registry.get(turn["turn_id"]) is not None
|
|
assert outcomes == []
|
|
# In production the sweep runs inside an HTTP handler, i.e. on the server
|
|
# loop, where the draft can be handed to the async outcome funnel. Run it
|
|
# there (the TestClient portal) rather than from the test thread, which has
|
|
# no loop and would only park the draft as an orphan.
|
|
deadline = time.time() + 10_000
|
|
headroom_client.portal.call(lambda: registry.sweep(deadline))
|
|
assert registry.get(turn["turn_id"]) is None
|
|
late = _post(headroom_client, {"turn_id": turn["turn_id"], "usage": {"prompt_tokens": 1}})
|
|
assert late.status_code == 404
|
|
assert late.json()["error"]["type"] == "unknown_turn"
|
|
# The record is scheduled as a task; the request above let the loop run it.
|
|
assert len(outcomes) == 1
|
|
assert outcomes[0].output_tokens == 0
|
|
assert outcomes[0].original_tokens == turn["tokens_before"]
|
|
|
|
|
|
def test_ttl_env_is_honoured(make_headroom_client, monkeypatch, outcome_spy) -> None:
|
|
"""R-OUTCOME: ``HEADROOM_GATEWAY_TURN_TTL_SECONDS`` bounds how long a pending
|
|
turn lives; a lazy sweep on the next lookup reclaims it."""
|
|
monkeypatch.setenv("HEADROOM_GATEWAY_TURN_TTL_SECONDS", "0")
|
|
client = make_headroom_client()
|
|
outcomes = outcome_spy(client)
|
|
turn = _open_turn(client, session_id="ttl-zero")
|
|
time.sleep(0.05)
|
|
late = _post(client, {"turn_id": turn["turn_id"], "usage": {"prompt_tokens": 1}})
|
|
assert late.status_code == 404, late.text
|
|
assert len(outcomes) == 1
|
|
|
|
|
|
def test_registry_stats_shape(headroom_client) -> None:
|
|
"""R-LOOKUP: ``stats()`` exposes at least the pending count for /stats and debugging."""
|
|
registry = registry_of(headroom_client)
|
|
assert registry is not None
|
|
turn = _open_turn(headroom_client, session_id="stats")
|
|
stats = registry.stats()
|
|
assert isinstance(stats, dict)
|
|
assert stats.get("pending") == len(registry) >= 1
|
|
_post(headroom_client, {"turn_id": turn["turn_id"]})
|
|
assert registry.stats().get("pending") == len(registry)
|