mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 12:05:34 +08:00
feat(proxy): add lossless /v1/compress mode and detect provider by wire shape (#4617)
## Description Two gaps stop a gateway from using `/v1/compress` well. Both showed up while fixing LiteLLM's `headroom` guardrail, which is the follow-up PR that needs this one. 1. **No mode compresses messages losslessly without markers.** A gateway with no retrieval loop can't resolve a marker or call `headroom_retrieve`, so it needs a mode with nothing to retrieve. The existing marker-free modes still run lossy compressors. 2. **Hooks guess the provider from the model name.** A gateway can send any client's shape to any model. Claude Code's Anthropic body routed to Gemini looked like OpenAI, so skill condensing found nothing to change. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) ## Changes Made **`config.mode="lossless"`** - Runs format-native folds only, with no markers and no CCR store writes. A registered lossless provider still applies. - Turns off stale-Read pointers, which drop the file body and need the retrieval tool. - Turns off cross-turn dedup pointers, which a client that never numbers its messages can't resolve. - Scope: the guarantee covers message compression (lossless codecs, no CCR). On a gateway turn, tool schema compaction and registered turn hooks still run as configured, as in every other mode; description truncation only happens when an operator sets `HEADROOM_TOOL_DESC_MAX_CHARS`. **`provider_for_body`** - Hooks and the session tracker now use the body's wire shape, and fall back to the model name when the shape proves nothing, such as plain text with no system prompt or tools. - Anthropic's own tool types (server tools such as `web_search_20250305`, client tools such as `bash_20250124`) end in an eight-digit date and count as the Anthropic shape, so a first-turn body whose only tool is a server tool is read correctly. - `route.provider` still follows the model. - The cache breakpoint budget runs when either the shape or the model is Anthropic, and never on a Responses body. - `gateway_response_fields` loses its unused `provider` parameter, and `RESPONSE_RECEIVED` keeps reporting the model's provider. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`, `ruff format --check .`, on the pinned 0.17.0) - [ ] Type checking passes (`mypy headroom`): one error, in `headroom/integrations/langchain/chat_model.py`, which this PR doesn't touch (local mypy 1.19.1) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text pytest tests/test_proxy_compress_endpoint.py tests/gateway tests/test_gateway_turn_unit.py tests/test_compress_route_tokenizer_by_model.py 425 passed, 4 skipped Regression tests with the fixes removed: 8 failed, 7 passed ``` ## Real Behavior Proof - **Environment:** LiteLLM main with the follow-up guardrail change, this branch as the sidecar, and a fake upstream that simulates prompt caching. No real provider API was called. - **Exact command / steps:** replayed a scripted 17-call agent session with exact-duplicate outputs removed, once direct to the fake upstream and once through LiteLLM and `/v1/compress`. - **Observed result:** 0 errors, 0 rewrites of already-sent results, `Read` and `Edit` results unchanged, about 1.3s added in total. | Client | Prompt cost vs direct | |---|---| | Anthropic Messages, Claude Code cache markers | −3.9% | | Chat Completions | −3.8% | | Responses | −4.1% | The script's outputs are synthetic, so treat these as a comparison between setups, not a forecast for real traffic. ## Known limits - The default mode and `lossy_inline` can still return a stale-Read `Retrieve original: hash=` marker, although the docs call them marker-free. This predates the PR and needs a separate fix. Proxy `--lossless` has no read-lifecycle guard either, from reading the code. - With no `ContentRouter`, `_derived_compress_pipeline` falls back to the default pipeline, so `lossless` wouldn't hold. Production always has a router, so only test stubs reach this. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4617" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> ## Maintainer post-merge revisit: maine9331b6bPublished head `e50b001bd05c9bc85405accca49cbef0e7786e51` incorporates upstream main `e9331b6b963b5b6554f45943abed2366aaa25ea6` after #4142, #4637 and #4641 merged. 164 endpoint/gateway/tokenizer/admission tests pass; one json_minifier strategy assertion fails identically on actual maine9331b6b. Ruff/format/diff pass. Import-only conflict resolved by retaining both provider_for_body and RequestBodyTooLarge. Both existing holds reconfirmed: lossless HTTP output removes unique description suffix via tool_desc_compaction; Anthropic server tool with Gemini model is classified OpenAI. Contributor correctness logic was not rewritten. The existing Changes Requested decision remains applicable; unchanged findings are not reposted as duplicate reviews. Scoped Windows qualification uses the existing development native extension, not a fresh build. Hosted checks must finish independently; no contributor CI action requested and nothing merged by this review. ## Maintainer post-merge revisit: maincc1eb540Published `38b871120e318f6eaa1cb2ce16f96d4f6cc2b1e8` includes upstream main `cc1eb5402aade58204b96b6b4b4d280cdd5456d2`. 90 gateway/tokenizer tests pass; both existing HTTP/provider counterexamples reconfirmed. Clean merge; Ruff/format/diff pass. Contributor correctness logic was not rewritten. The existing Changes Requested decision remains applicable; unchanged findings are not reposted. Scoped Windows qualification uses the existing development native extension, not a fresh build. Hosted checks must finish independently; nothing merged by this review. ### Maintainer current-main requalification Exact head `33395b5cdd7ee9facce9f97f2a3250f5e6e49cb2`, upstream main `ecbaf86d78cf52d185fbffa306157b9a4cc1b906`. Mechanically merged upstream/mainecbaf86d78without changing contributor policy logic. Re-reviewed the combined gateway/provider delta and existing comments. All 90 focused gateway/tokenizer tests and ten inherited Gemini fixture cases pass; Ruff0.17.3 lint/format and whitespace checks pass. The shared Gemini repair was already present, so no duplicate fixture commit was added. Both existing correctness findings independently remain on this combined tree. An HTTP200 /v1/compress request with config.mode=lossless still drops UNIQUE_REQUIREMENT_AT_END from a long tool description, reporting tool_desc_compaction. An Anthropic web_search_20250305 tool body with a Gemini-named model still returns provider openai. Existing requested semantic fixes/regressions remain necessary; maintainers are handling CI and no contributor CI execution is requested. No contributor logic was rewritten and nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `4c40adb53a68f382e32f00620adcfaf1cff477f5`, upstream main `8d3e9e5ca7cd645eb73084b848edbea5fc796e1a`. Clean mechanical sync with upstream main8d3e9e5ca7cd645eb73084b848edbea5fc796e1a including intervening release/runtime/cache/CCR/SSE/HTTP1.1 merges. No contributor gateway/provider policy rewritten. Fresh author/Devin discussion considered.100focused gateway-contract/tokenizer/Gemini tests pass;7changed Python files pass Ruff0.17.3check/format and whitespace checks pass. Last main delta after test start changes only Rust/OpenClaw, leaving tested Python inputs unchanged. Both existing blockers remain freshly reproduced from this worktree (source import path verified): real app POST/v1/compress lossless returns200 but drops UNIQUE_REQUIREMENT_AT_END from tool description via tool_desc_compaction; Anthropic server-tool wire shape with Gemini model returns provider openai. Preserve promised lossless content and classify server-tool wire forms correctly, with regressions. No external provider session/nativebuild claimed. Copied extension supports Python startup; maintainers handle hosted checks. Nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `8b7b2b0d02daf1a031828ecb13095ab315431f7e`, upstream main `917edc62230c3c4a8a415e2053da68dab1f003dd`. Current-main integration against917edc62230c3c4a8a415e2053da68dab1f003dd is clean. Compared entire headroom/crates/Python manifests+lock/tests inputs to prior qualified4c40adb53: byte-identical. Intervening changes are reviewed npm maintenance and release-envelope recovery workflow; no contributor gateway policy changed.75new-base release-workflow/normalizer tests pass,whitespace passes. Prior100gateway/tokenizer/Gemini results remain scoped to those identical inputs rather than relabeled fresh. Both gateway blockers freshly rechecked from worktree: Anthropic server-tool+Gemini body classifiesopenai; actual lossless/v1/compress returns200 but drops unique description suffix via tool_desc_compaction. Retain existing losslessness/classification corrections. No fresh nativebuild/provider claim. Maintainers handle exactheadCI;nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `490ff6defdebddda9106e9e143d42d1e463b6324`, upstream main `7592e681459f14a2041eb5f524a658cb6ad1dd92`. Clean integration with upstream/main7592e68145. Reviewed both intervening memory fixes and fresh PR discussion. Gateway/runtime/Rust inputs are unchanged from the previously reviewed head; only the two main memory fixes and their regressions enter this merge. Nine memory/Gemini regression tests pass; main-relative whitespace passes. Fresh actual-app probes retain both existing blockers: Anthropic server-tool content with a Gemini model classifies as OpenAI; lossless /v1/compress removes the unique tool-description suffix through tool_desc_compaction. No contributor policy was rewritten. Existing native extension used for Python startup, not a fresh native-build claim. Maintainers handle hosted checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `d3cb3a02af03991a696461160ee0db322dd8bf48`, upstream main `2d3ee51846435ef5b320eea4806af14e602b0d8c`. Clean integration with upstream/main2d3ee51846after the external 0.43.1 release merge. Exact headroom/crates/tests comparison against prior reviewed490ff6defshows no changes. Re-reviewed generated release metadata; 78 release/version/MCP tests pass on this composed tree, and whitespace passes. Previous nine memory/Gemini regressions and fresh gateway probes remain scoped to identical runtime inputs. Both established blockers remain: lossless /v1/compress drops the unique tool-description suffix, and Anthropic server-tool content with a Gemini model classifies as OpenAI. Runtime probes were not relabeled as rerun after this metadata-only merge. No contributor policy rewritten; no fresh native-build claim. Maintainers handle exact-head checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `e0c873630e9ffc7bc0d0d2fdd22c483b731e49a3`, upstream main `910588e6b15ab54c4099d03703e5fa20c8abb757`. Clean mechanical synchronization with upstream main910588e6b15ab54c4099d03703e5fa20c8abb757, including the externally merged runtime fixes, docs dependency update and tree-sitter lock repair. No contributor gateway/provider policy rewritten. Current author/Devin discussion considered. All90 gateway contract/tokenizer-by-model tests pass on the preceding composed runtime tree; the final dependency-only merge changes only pyproject/uv.lock, leaving those gateway/tokenizer sources unchanged. No redundant rerun on the final merge is claimed. Changed Python Ruff lint/format and main-relative whitespace pass. Both existing blockers freshly reproduce on the composed runtime paths: lossless /v1/compress returns200 but drops UNIQUE_REQUIREMENT_AT_END from the tool description and reports tool_desc_compaction; Anthropic server-tool content with a Gemini model is classified as OpenAI. Preserve the lossless original payload and classify the full request protocol consistently, with endpoint regressions for these inputs. The main fixes do not close either defect. Tests use the existing native core for startup, not a fresh native build; no live provider traffic was made. Maintainers handle hosted checks; no inherited-main pipeline failure attributed and nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `507433606f698249e546e1418928caee99d0c901`, upstream main `7877241d84c93b3ec487e07324245838ddae85cc`. Clean mechanical integration with upstream/main7877241d84c93b3ec487e07324245838ddae85cc. Gateway production/contracts remain byte-identical to the preceding reviewed head; re-reviewed the new CLI/Hermes integration.127 gateway request/response contract and model-tokenizer tests pass; whitespace passes. Both existing holds freshly reproduce on this exact tree: lossless /v1/compress returns200 while dropping UNIQUE_REQUIREMENT_AT_END and reporting tool_desc_compaction; Anthropic server-tool content with a Gemini model is classified as OpenAI. Preserve the lossless original payload and classify the full request protocol consistently, with endpoint regressions. No contributor gateway/provider policy rewritten. Existing native core used for startup; no fresh native build or live provider claim. Maintainers handle hosted checks; nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `b0daece6b10f623a6e7b0dc3eb01c73e05fa3209`, upstream main `c01cf07d943d816cfd9d97e6838ed4e38f1ff9a4`. Final post-Dependabot requalification against upstream/mainc01cf07d94. All six Dependabot updates are now merged. Relative to this branch's freshly tested prepared tree, the final main sync changes only the three intended Cargo.lock records; Python/source/test inputs are unchanged. Re-reviewed the integration and confirmed complete native source/lock/toolchain objects match the tested combined build and exact fresh native DLL used for this branch's local tests. Focused fresh-native qualification: 127 passed, 3 warnings in 104.78s (0:01:44). Selected real dependency overlays and native DLL paths are recorded in final4617-fresh-native-tests.log; the DLL provenance/input hashes are in final-native-artifact-proof.json. No redundant rerun is implied after this Cargo-only merge. Whitespace passes. Prior source review and scoped real-behavior evidence remain applicable to unchanged contributor inputs. Both existing blockers freshly reproduce using the combined native DLL: lossless /v1/compress returns200 but removes UNIQUE_REQUIREMENT_AT_END and reports tool_desc_compaction; Anthropic server-tool content with a Gemini model is classified as OpenAI. Preserve the original lossless payload and classify the complete wire protocol, with endpoint regressions. Maintainers handle hosted checks; no external-provider claim, contributor semantic rewrite or non-Dependabot merge. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `ec6abc0228a93c5eab1881938041dfee344b1e3a`, upstream main `2973ca593d2fe65c3259941fdff8009175eff456`. Requalified after human merges #4667/#4669/#4670 against upstream maince42754912. All 11 integration patches exactly match the prior-main-to-current-main delta after ignoring diff locations; no contributor gateway/provider policy changed. Fresh gateway request/response contracts, model-tokenizer, gateway unit, Kompress-disable and must-keep suites: 207 passed, 3 warnings. Six changed Python files pass Ruff lint/format; offline lock validation and whitespace pass. Existing matching-core DLL used; native source/lock/toolchain unchanged by these main merges, with no fresh native build or live-provider claim. Both existing blockers freshly reproduce: lossless /v1/compress returns 200 while removing UNIQUE_REQUIREMENT_AT_END and reporting tool_desc_compaction; Anthropic server-tool content with a Gemini model is classified as OpenAI. Preserve original lossless payload and recognize complete wire protocol with endpoint regressions. Author and Devin discussions checked; existing findings remain actionable. Maintainers handle hosted checks; nothing merged. Subsequent human merge #4671 advanced main to2973ca593dbefore publication; guard stopped the old candidate. Cleanly integrated the new workflow-only delta and re-reviewed action pinning with 62 workflow/release tests passing. Runtime/native sources are byte-identical to the immediately preceding 207-test and real-behavior qualification; those results apply to unchanged inputs. All prior-main hosted results are historical for readiness. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `9f857218486fa451ecacc675863a64bc95f45bb0`, upstream main `2b0315b2d4b0d54947050bbdad2de9e89df06378`. Current-main requalification against2b0315b2d4b0d54947050bbdad2de9e89df06378 after later human merges. All37 integration patches exactly match2973ca59-to-current-main delta after diff-location normalization; contributor gateway/provider policy unchanged.393gateway request/response contracts,model-tokenizer,gateway unit,Kompress-disable,must-keep,doctor,search tests pass;3warnings.27changedPythonfiles pass Ruff lint/format;offline lock/whitespacepass. Existing pr3982 native DLL verified every tracked native-input hash/artifactchecksum;no fresh build claimed. Author/Devin discussion checked. Both correctness blockers freshly reproduce:lossless /v1/compress returns200while droppingUNIQUE_REQUIREMENT_AT_END and reportingtool_desc_compaction;Anthropicserver-toolcontent withGeminimodel classifiedOpenAI. Preserve original lossless payload and classify full request wire protocol consistently,with endpoint regressions. No live provider qualification,contributor logic rewrite,or inherited-main CI fault attributed. Maintainers handle hosted checks;nothing merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. ### Maintainer current-main requalification Exact head `41b2145418db7f5980cd52e6235ab330fcd31ac0`, upstream main `0ddaa56c21cdc01c4774391fdcc69e987ecc4e5c`. Clean integration with upstream/main0ddaa56c21. All 17 integration patches exactly match the prior-reviewed-main to current-main delta after diff-location normalization; contributor gateway/provider policy is unchanged. Fresh gateway/tokenizer/search/doctor and incoming Vertex/backend/CLI qualification: 455 passed, 11 skipped, 3 warnings. Additional gateway request/response contracts and compress-endpoint suite: 158 passed, 1 warning. Twelve changed Python files pass Ruff lint/format; offline lock and whitespace checks pass. Author discussion and outstanding Devin findings read. Both existing correctness findings freshly reproduce on this exact tree: lossless /v1/compress returns 200 but drops UNIQUE_REQUIREMENT_AT_END through tool_desc_compaction; Anthropic server-tool content with a Gemini model classifies as OpenAI. Existing requested corrections remain; no new duplicate finding. Local app probes are not live provider proof. Native DLL reused only after verifying the complete tracked input set, every recorded input SHA256 and artifact checksum against pr3982-c01-native-proof.json; no fresh build claimed. Maintainers handle exact-head hosted checks. No PR merged. Exact-head hosted checks are being handled by maintainers. Existing correctness/qualification holds remain; nothing merged. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
JD Davis
parent
8f50884bd5
commit
33f067a243
@@ -637,7 +637,7 @@ Keep carrying both yourself and send them upstream unchanged. Two consequences w
|
||||
|
||||
| Field | Type | Default | Description |
|
||||
| --- | --- | --- | --- |
|
||||
| `mode` | string | unset | `ccr`, `lossy_inline`, or `lossless_then_lossy`. Unset selects the default marker-free pipeline. Any other value is a `400`. |
|
||||
| `mode` | string | unset | `ccr`, `lossy_inline`, `lossless_then_lossy`, or `lossless`. Unset selects the default marker-free pipeline. Any other value is a `400`. |
|
||||
| `frozen_message_count` | integer | unset | Pin a prefix: the first N messages are returned byte-for-byte unchanged while staying visible to cross-message transforms like dedup. Set it to the number of messages the provider has already cached so compression cannot rewrite the prefix and bust that cache. Must be a non-negative integer; anything else is a `400`. |
|
||||
| `session_id` | string | unset | Opt into [session mode](#session-mode): Headroom keeps this conversation's replay state itself, so you can resend the raw conversation every turn and get a byte-identical prefix back. Non-empty, at most 256 characters; anything else is a `400`. Cannot be combined with `compress_user_messages` (`400`). |
|
||||
| `compress_user_messages` | boolean | `false` | Also compress user-role messages. |
|
||||
@@ -650,6 +650,7 @@ Keep carrying both yourself and send them upstream unchanged. Two consequences w
|
||||
- **unset (default)** — marker-free. Emits no `<<ccr:…>>` retrieval markers and writes nothing to the CCR store, so you can forward the returned messages straight to a provider. This is the right mode for a gateway or guardrail that just swaps `messages` and forwards.
|
||||
- **`ccr`** — emits CCR markers and writes to the store. Only for callers that also inject the `headroom_retrieve` tool *and* can reach `/v1/retrieve` (itself loopback-only). Markers are a dangling pointer for the model otherwise.
|
||||
- **`lossy_inline`** (alias `lossless_then_lossy`) — runs the lossless byte/data fold first, then compresses the folded remainder. Marker-free.
|
||||
- **`lossless`** — format-native folds only, with no markers and no retrieval. For a sidecar with no retrieval loop; a registered lossless provider still applies. Keep the mode stable within a `session_id`, because session mode replays earlier turns byte-for-byte.
|
||||
|
||||
#### Response
|
||||
|
||||
|
||||
@@ -42,6 +42,7 @@ from headroom.proxy.gateway_turn import (
|
||||
new_turn_id,
|
||||
on_turn_expired,
|
||||
parse_gateway_block,
|
||||
provider_for_body,
|
||||
provider_for_model,
|
||||
register_pending_turn,
|
||||
shape_gateway_body,
|
||||
@@ -80,7 +81,7 @@ class GatewayTurn:
|
||||
self.body = body
|
||||
self.caps = caps
|
||||
self.client = client
|
||||
self.provider = provider_for_model(str(body.get("model") or ""))
|
||||
self.provider = provider_for_body(body, str(body.get("model") or ""))
|
||||
# The wire shape is a property of the body, not of the model name, and
|
||||
# it is fixed for the life of the turn: the response half has to write
|
||||
# a re-drive in the same shape the request half read.
|
||||
@@ -100,7 +101,6 @@ class GatewayTurn:
|
||||
request=self.request,
|
||||
body=self.body,
|
||||
caps=self.caps,
|
||||
provider=self.provider,
|
||||
messages=messages,
|
||||
tools=self.body.get("tools"),
|
||||
obligations=[],
|
||||
@@ -116,7 +116,7 @@ class GatewayTurn:
|
||||
bind_scope(tags, self.request.scope)
|
||||
self.model_name = model_name
|
||||
self.tags = tags
|
||||
self.provider = provider_for_model(model_name)
|
||||
self.provider = provider_for_body(self.body, model_name)
|
||||
self._transformer = RequestTransformer(
|
||||
provider=self.provider,
|
||||
model_name=model_name,
|
||||
@@ -198,7 +198,6 @@ class GatewayTurn:
|
||||
request=self.request,
|
||||
body=self.body,
|
||||
caps=self.caps,
|
||||
provider=self.provider,
|
||||
messages=messages,
|
||||
tools=result.tools,
|
||||
obligations=self._obligations,
|
||||
@@ -213,9 +212,13 @@ class GatewayTurn:
|
||||
input_tokens=tokens_before,
|
||||
transforms=transforms,
|
||||
)
|
||||
if self.provider == "anthropic":
|
||||
if not self.responses_shape and "anthropic" in (
|
||||
self.provider,
|
||||
provider_for_model(self.model_name),
|
||||
):
|
||||
# Last stop before the body leaves: a request over Anthropic's
|
||||
# cache_control budget is a guaranteed 400 on the provider.
|
||||
# cache_control budget is a guaranteed 400 on the provider. The
|
||||
# budget belongs to the upstream, so a chat body to Claude counts.
|
||||
from headroom.proxy.helpers import enforce_cache_breakpoint_budget
|
||||
|
||||
provider_body = self._fields["body"]
|
||||
|
||||
@@ -41,6 +41,7 @@ import dataclasses
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
@@ -50,6 +51,7 @@ from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from headroom.proxy.gateway_responses import VIEW_MARKER as GATEWAY_RESPONSES_VIEW_MARKER
|
||||
from headroom.proxy.gateway_responses import carries_view, is_responses_body
|
||||
from headroom.proxy.outcome import RequestOutcome
|
||||
from headroom.proxy.tool_schema_savings_policy import without_deferral_flags
|
||||
from headroom.proxy.turn_hooks import (
|
||||
@@ -208,11 +210,67 @@ def _parse_request_headers(raw: Any) -> dict[str, str]:
|
||||
|
||||
|
||||
def provider_for_model(model_name: str) -> str:
|
||||
"""The tracker-provider inference ``/v1/compress`` already uses."""
|
||||
"""The provider a model name implies, for routing advice and event pricing."""
|
||||
lowered = (model_name or "").lower()
|
||||
return "anthropic" if ("claude" in lowered or "anthropic" in lowered) else "openai"
|
||||
|
||||
|
||||
_OPENAI_ONLY_ROLES = frozenset({"system", "developer", "tool", "function"})
|
||||
_OPENAI_ONLY_BLOCKS = frozenset({"image_url", "input_audio", "input_text"})
|
||||
_ANTHROPIC_ONLY_BLOCKS = frozenset(
|
||||
{"tool_use", "tool_result", "thinking", "redacted_thinking", "server_tool_use", "document"}
|
||||
)
|
||||
|
||||
|
||||
# Anthropic's own tools (server tools such as ``web_search_20250305`` and
|
||||
# client tools such as ``bash_20250124``) carry a dated ``type`` and no
|
||||
# ``input_schema``; OpenAI chat tools are ``function`` or ``custom``.
|
||||
_ANTHROPIC_DATED_TOOL_TYPE = re.compile(r"_\d{8}$")
|
||||
|
||||
|
||||
def _wire_provider_from_shape(body: dict[str, Any]) -> str | None:
|
||||
"""The provider a body's shape proves, or None when only the model can tell."""
|
||||
if carries_view(body) or is_responses_body(body):
|
||||
return "openai"
|
||||
if body.get("system") is not None:
|
||||
return "anthropic"
|
||||
tools = body.get("tools")
|
||||
for tool in tools if isinstance(tools, list) else ():
|
||||
if isinstance(tool, dict) and "input_schema" in tool:
|
||||
return "anthropic"
|
||||
tool_type = tool.get("type") if isinstance(tool, dict) else None
|
||||
if isinstance(tool_type, str) and _ANTHROPIC_DATED_TOOL_TYPE.search(tool_type):
|
||||
return "anthropic"
|
||||
if isinstance(tool, dict) and "function" in tool:
|
||||
return "openai"
|
||||
messages = body.get("messages")
|
||||
for message in messages if isinstance(messages, list) else ():
|
||||
if not isinstance(message, dict):
|
||||
continue
|
||||
if message.get("role") in _OPENAI_ONLY_ROLES or message.get("tool_calls"):
|
||||
return "openai"
|
||||
content = message.get("content")
|
||||
for block in content if isinstance(content, list) else ():
|
||||
block_type = block.get("type") if isinstance(block, dict) else None
|
||||
if not isinstance(block_type, str):
|
||||
continue
|
||||
if block_type in _ANTHROPIC_ONLY_BLOCKS or block_type.endswith("_tool_result"):
|
||||
return "anthropic"
|
||||
if block_type in _OPENAI_ONLY_BLOCKS:
|
||||
return "openai"
|
||||
return None
|
||||
|
||||
|
||||
def provider_for_body(body: dict[str, Any], model_name: str) -> str:
|
||||
"""The wire shape hooks and steering must treat a gateway body as.
|
||||
|
||||
A gateway can send any client's shape to any model: Claude Code's Anthropic
|
||||
body to Gemini, an OpenAI chat body to Claude. The shape wins over the model
|
||||
name; ``route.provider`` stays advice about the model.
|
||||
"""
|
||||
return _wire_provider_from_shape(body) or provider_for_model(model_name)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Usage normalization #
|
||||
# --------------------------------------------------------------------------- #
|
||||
@@ -560,9 +618,9 @@ class PendingTurn:
|
||||
ccr_armed: bool = False
|
||||
# The wire shape this turn arrived in, when it is not the one `provider`
|
||||
# implies: "openai_responses" for a Codex body. Deliberately separate from
|
||||
# `provider`, which picks pricing, tokenizer and the upstream the gateway
|
||||
# is advised to call -- all three want "openai" for a Responses turn while
|
||||
# the tool-call shape and the re-drive body want "openai_responses".
|
||||
# `provider`, the chat-level wire shape (see `provider_for_body`), which
|
||||
# wants "openai" for a Responses turn while the tool-call shape and the
|
||||
# re-drive body want "openai_responses".
|
||||
wire_shape: str | None = None
|
||||
|
||||
@property
|
||||
@@ -1113,7 +1171,6 @@ def build_provider_body(
|
||||
transcript the provider will reject.
|
||||
"""
|
||||
from headroom.proxy.gateway_responses import apply_view as apply_responses_view
|
||||
from headroom.proxy.gateway_responses import carries_view, is_responses_body
|
||||
|
||||
out = {k: v for k, v in body.items() if k not in BODY_CONTROL_KEYS}
|
||||
if carries_view(body):
|
||||
@@ -1175,7 +1232,6 @@ def gateway_response_fields(
|
||||
request: Any,
|
||||
body: dict[str, Any],
|
||||
caps: GatewayCapabilities,
|
||||
provider: str,
|
||||
messages: list[dict[str, Any]],
|
||||
tools: Any,
|
||||
obligations: list[str],
|
||||
@@ -1188,7 +1244,9 @@ def gateway_response_fields(
|
||||
return {
|
||||
"body": provider_body,
|
||||
"turn_id": turn_id,
|
||||
"route": build_route(request, provider_body, provider),
|
||||
"route": build_route(
|
||||
request, provider_body, provider_for_model(str(provider_body.get("model") or ""))
|
||||
),
|
||||
"obligations": list(obligations),
|
||||
"gateway": caps.echo(),
|
||||
"headers": dict(headers or {}),
|
||||
@@ -1468,7 +1526,7 @@ def _emit_response_received(proxy: Any, turn: PendingTurn, response: Any, status
|
||||
PipelineStage.RESPONSE_RECEIVED,
|
||||
operation="gateway.turn",
|
||||
request_id=(turn.outcome_draft.request_id if turn.outcome_draft else turn.turn_id),
|
||||
provider=turn.provider,
|
||||
provider=provider_for_model(turn.model),
|
||||
model=turn.model,
|
||||
response=response,
|
||||
metadata={
|
||||
|
||||
@@ -79,7 +79,7 @@ import httpx
|
||||
from headroom.agent_savings import proxy_pipeline_kwargs
|
||||
from headroom.ccr.marker_resolution import resolve_markers_in_response
|
||||
from headroom.ccr.tool_calls import has_ccr_retrieve_tool
|
||||
from headroom.config import is_tool_excluded, unwrap_tool_call_name
|
||||
from headroom.config import ReadLifecycleConfig, is_tool_excluded, unwrap_tool_call_name
|
||||
from headroom.copilot_auth import (
|
||||
apply_copilot_api_auth,
|
||||
build_copilot_upstream_url,
|
||||
@@ -1925,9 +1925,8 @@ WS_FIRST_FRAME_TIMEOUT_SECONDS = 60.0
|
||||
|
||||
# Accepted values for ``config.mode`` on POST /v1/compress. Unset/None means
|
||||
# the default marker-free pipeline. Anything else is a 400 rather than a
|
||||
# silent fall-through to the default (a typo or an unsupported mode like
|
||||
# "lossless" would otherwise look like it worked).
|
||||
COMPRESS_MODES = ("ccr", "lossy_inline", "lossless_then_lossy")
|
||||
# silent fall-through to the default (a typo would otherwise look like it worked).
|
||||
COMPRESS_MODES = ("ccr", "lossy_inline", "lossless_then_lossy", "lossless")
|
||||
|
||||
# Max wait for a sidecar session's turn lock, on the executor. MUST stay
|
||||
# well below COMPRESSION_TIMEOUT_SECONDS: with an untimed acquire, a slow
|
||||
@@ -11029,6 +11028,25 @@ class OpenAIHandlerMixin:
|
||||
smart_crusher_lossless_only=False, # keep SmartCrusher lossy
|
||||
)
|
||||
|
||||
def _lossless_pipeline(self) -> Any:
|
||||
"""Pipeline for ``/v1/compress`` ``config.mode="lossless"``.
|
||||
|
||||
The per-request form of the proxy's ``--lossless``: format-native folds
|
||||
only, with no markers and no CCR store, so a sidecar with no retrieval
|
||||
loop loses nothing. A registered lossless provider still competes.
|
||||
``handle_compress`` also turns off cross-turn dedup pointers, which a
|
||||
client that never numbers its messages cannot resolve.
|
||||
"""
|
||||
return self._derived_compress_pipeline(
|
||||
"lossless",
|
||||
lossless=True,
|
||||
smart_crusher_lossless_only=True,
|
||||
ccr_inject_marker=False,
|
||||
ccr_enabled=False,
|
||||
# Stale-Read pointers drop the file body and need the retrieval tool.
|
||||
read_lifecycle=ReadLifecycleConfig(enabled=False),
|
||||
)
|
||||
|
||||
async def handle_compress(self, request: Request) -> JSONResponse:
|
||||
"""Compress messages without calling an LLM.
|
||||
|
||||
@@ -11043,6 +11061,9 @@ class OpenAIHandlerMixin:
|
||||
``headroom_retrieve`` tool and can reach loopback ``/v1/retrieve``.
|
||||
* ``"lossy_inline"`` (alias ``"lossless_then_lossy"``) — marker-free,
|
||||
lossless fold first then Kompress the remainder.
|
||||
* ``"lossless"`` — marker-free, lossless folds only. This covers message
|
||||
compression; on a gateway turn, tool schema compaction and registered
|
||||
turn hooks still run as configured.
|
||||
|
||||
Any other ``config.mode`` value is a 400 (see ``COMPRESS_MODES``).
|
||||
|
||||
@@ -11068,6 +11089,7 @@ class OpenAIHandlerMixin:
|
||||
from headroom.proxy.gateway_responses import build_view as build_responses_view
|
||||
from headroom.proxy.gateway_responses import is_responses_body
|
||||
from headroom.proxy.gateway_responses import mark_view as mark_responses_view
|
||||
from headroom.proxy.gateway_turn import provider_for_body
|
||||
from headroom.proxy.helpers import RequestBodyTooLarge, _read_request_json
|
||||
|
||||
def _gateway_invalid(e: CompressTurnError) -> JSONResponse:
|
||||
@@ -11330,7 +11352,9 @@ class OpenAIHandlerMixin:
|
||||
}
|
||||
},
|
||||
)
|
||||
if mode in ("lossy_inline", "lossless_then_lossy"):
|
||||
if mode == "lossless":
|
||||
pipeline = self._lossless_pipeline()
|
||||
elif mode in ("lossy_inline", "lossless_then_lossy"):
|
||||
pipeline = self._lossy_inline_pipeline()
|
||||
elif mode == "ccr":
|
||||
# Markers + store writes wanted, so inherit the live router config
|
||||
@@ -11347,6 +11371,8 @@ class OpenAIHandlerMixin:
|
||||
"model_limit": context_limit,
|
||||
**proxy_pipeline_kwargs(self.config),
|
||||
}
|
||||
if mode == "lossless":
|
||||
pipeline_kwargs["cross_turn_dedup_recoverable"] = False
|
||||
if compress_user_messages:
|
||||
pipeline_kwargs["compress_user_messages"] = True
|
||||
if target_ratio is not None:
|
||||
@@ -11383,11 +11409,7 @@ class OpenAIHandlerMixin:
|
||||
# trick SessionTrackerStore uses for its synthetic lineage
|
||||
# keys). A plain "compress:" string prefix was spoofable.
|
||||
_session_key = f"compress\x00{session_id}"
|
||||
_tracker_provider = (
|
||||
"anthropic"
|
||||
if ("claude" in model_name.lower() or "anthropic" in model_name.lower())
|
||||
else "openai"
|
||||
)
|
||||
_tracker_provider = provider_for_body(body, model_name)
|
||||
comp_cache = self._get_compression_cache(_session_key)
|
||||
session_tracker = self.session_tracker_store.get_or_create(
|
||||
_session_key, _tracker_provider
|
||||
|
||||
@@ -50,7 +50,7 @@ class TurnContext:
|
||||
(or just sent); a hook's ``on_request`` may mutate them in place.
|
||||
"""
|
||||
|
||||
provider: str # "anthropic" | "openai" | "google" | ...
|
||||
provider: str # the wire shape, not the model's vendor: "anthropic" | "openai" | ...
|
||||
model: str
|
||||
messages: list[dict[str, Any]]
|
||||
tools: Any = None # provider-native tools value (list, or None)
|
||||
|
||||
@@ -9,7 +9,7 @@ export type HeadroomMode = "audit" | "optimize" | "simulate";
|
||||
|
||||
/** Pipeline mode accepted by POST /v1/compress. Unset = default marker-free
|
||||
* pipeline. Any other value is rejected (400) by the proxy. */
|
||||
export type CompressMode = "ccr" | "lossy_inline" | "lossless_then_lossy";
|
||||
export type CompressMode = "ccr" | "lossy_inline" | "lossless_then_lossy" | "lossless";
|
||||
|
||||
/** Config accepted by the POST /v1/compress endpoint. camelCase keys are
|
||||
* snake_cased on the wire (e.g. targetRatio -> target_ratio). Every field is
|
||||
|
||||
@@ -467,3 +467,170 @@ def test_turn_ids_are_unique_per_request(headroom_client) -> None:
|
||||
"""I-KEYS: every request half mints a fresh id, registered or not."""
|
||||
ids = {compress(headroom_client, _body(gateway={})).json()["turn_id"] for _ in range(5)}
|
||||
assert len(ids) == 5
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("label", "shape", "model", "provider", "route_provider"),
|
||||
[
|
||||
(
|
||||
"anthropic_system",
|
||||
{"messages": [{"role": "user", "content": "x"}], "system": SYSTEM_PROMPT},
|
||||
"gemini/gemini-2.5-pro",
|
||||
"anthropic",
|
||||
"openai",
|
||||
),
|
||||
(
|
||||
"anthropic_tools",
|
||||
{"messages": [{"role": "user", "content": "x"}], "tools": anthropic_tools()},
|
||||
"gemini/gemini-2.5-pro",
|
||||
"anthropic",
|
||||
"openai",
|
||||
),
|
||||
(
|
||||
"anthropic_history_with_server_tool",
|
||||
{
|
||||
"messages": anthropic_tool_history(),
|
||||
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
|
||||
},
|
||||
"gpt-5.5",
|
||||
"anthropic",
|
||||
"openai",
|
||||
),
|
||||
(
|
||||
"anthropic_first_turn_server_tool_only",
|
||||
{
|
||||
"messages": [{"role": "user", "content": "x"}],
|
||||
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
|
||||
},
|
||||
"gemini/gemini-2.5-pro",
|
||||
"anthropic",
|
||||
"openai",
|
||||
),
|
||||
(
|
||||
"openai_tools_to_claude",
|
||||
{"messages": [{"role": "user", "content": "x"}], "tools": openai_tools()},
|
||||
"claude-opus-5-5",
|
||||
"openai",
|
||||
"anthropic",
|
||||
),
|
||||
(
|
||||
"openai_tool_row_to_claude",
|
||||
{"messages": [{"role": "tool", "tool_call_id": "c", "content": "x"}]},
|
||||
"claude-opus-5-5",
|
||||
"openai",
|
||||
"anthropic",
|
||||
),
|
||||
(
|
||||
"responses_input_to_claude",
|
||||
{"input": [{"role": "user", "content": "x"}], "instructions": "be brief"},
|
||||
"claude-opus-5-5",
|
||||
"openai",
|
||||
"anthropic",
|
||||
),
|
||||
(
|
||||
"chat_system_role_to_claude",
|
||||
{
|
||||
"messages": [
|
||||
{"role": "system", "content": "be brief"},
|
||||
{"role": "user", "content": "x"},
|
||||
]
|
||||
},
|
||||
"claude-opus-5-5",
|
||||
"openai",
|
||||
"anthropic",
|
||||
),
|
||||
(
|
||||
"chat_image_to_claude",
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "image_url", "image_url": {"url": "https://x.test/a.png"}}
|
||||
],
|
||||
}
|
||||
]
|
||||
},
|
||||
"claude-opus-5-5",
|
||||
"openai",
|
||||
"anthropic",
|
||||
),
|
||||
(
|
||||
"anthropic_thinking_to_gpt",
|
||||
{
|
||||
"messages": [
|
||||
{"role": "user", "content": "x"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": [{"type": "thinking", "thinking": "t", "signature": "s"}],
|
||||
},
|
||||
{"role": "user", "content": "y"},
|
||||
]
|
||||
},
|
||||
"gpt-5.5",
|
||||
"anthropic",
|
||||
"openai",
|
||||
),
|
||||
(
|
||||
"anthropic_server_tool_result_to_gpt",
|
||||
{
|
||||
"messages": [
|
||||
{"role": "user", "content": "x"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": [
|
||||
{"type": "web_search_tool_result", "tool_use_id": "s", "content": []}
|
||||
],
|
||||
},
|
||||
{"role": "user", "content": "y"},
|
||||
],
|
||||
"tools": [{"type": "web_search_20250305", "name": "web_search"}],
|
||||
},
|
||||
"gpt-5.5",
|
||||
"anthropic",
|
||||
"openai",
|
||||
),
|
||||
(
|
||||
"plain_text_follows_model",
|
||||
{"messages": [{"role": "user", "content": "x"}], "system": None},
|
||||
"claude-opus-5-5",
|
||||
"anthropic",
|
||||
"anthropic",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_hooks_see_the_wire_shape_and_route_follows_the_model(
|
||||
headroom_client, label, shape, model, provider, route_provider
|
||||
) -> None:
|
||||
"""A gateway can send any client's shape to any model."""
|
||||
seen: list[str] = []
|
||||
|
||||
class Recorder:
|
||||
name = f"provider_recorder_{label}"
|
||||
stream_safe = True
|
||||
|
||||
def on_request(self, ctx: Any) -> None:
|
||||
seen.append(ctx.provider)
|
||||
|
||||
register_turn_hook(Recorder())
|
||||
data = compress(headroom_client, {**shape, "model": model, "gateway": {}}).json()
|
||||
assert seen == [provider]
|
||||
assert data["route"]["provider"] == route_provider
|
||||
|
||||
|
||||
def test_chat_body_to_claude_stays_within_the_cache_breakpoint_budget(headroom_client) -> None:
|
||||
"""The budget belongs to the upstream model, not to the body's wire shape."""
|
||||
marked = [
|
||||
{"role": "user", "content": [{"type": "text", "text": f"m{i}", "cache_control": {}}]}
|
||||
for i in range(7)
|
||||
]
|
||||
body = _body(messages=marked, tools=openai_tools(), model="claude-opus-5-5", gateway={})
|
||||
data = compress(headroom_client, body).json()
|
||||
sent = [
|
||||
block
|
||||
for message in data["body"]["messages"]
|
||||
for block in message["content"]
|
||||
if isinstance(block, dict) and "cache_control" in block
|
||||
]
|
||||
assert len(sent) <= 4
|
||||
assert "cache_breakpoint_budget" in data["transforms_applied"]
|
||||
|
||||
@@ -121,7 +121,7 @@ def test_openai_shape_still_counted_for_every_model_family(client, model):
|
||||
assert body["tokens_saved"] > 0, f"{model} reported no savings: {body}"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mode", [None, "ccr", "lossy_inline", "lossless_then_lossy"])
|
||||
@pytest.mark.parametrize("mode", [None, "ccr", "lossy_inline", "lossless_then_lossy", "lossless"])
|
||||
def test_every_mode_counts_anthropic_shape(client, mode):
|
||||
"""mode="ccr" used to share the request pipeline, which pinned the OpenAI counter."""
|
||||
payload: dict = {"messages": _anthropic_messages(), "model": "claude-sonnet-4-6"}
|
||||
|
||||
@@ -514,6 +514,91 @@ class TestCompressEndpointLossyInlineMode:
|
||||
with TestClient(app, base_url="http://127.0.0.1", client=("127.0.0.1", 12345)) as c:
|
||||
yield c
|
||||
|
||||
def test_lossless_mode_folds_search_output_and_keeps_json_rows(self, client):
|
||||
grep_output = "\n".join(
|
||||
f"headroom/proxy/handlers/openai.py:{i}: value_{i} = compute(value_{i})"
|
||||
for i in range(1, 400)
|
||||
)
|
||||
messages = [
|
||||
{"role": "user", "content": "Find the values."},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_grep",
|
||||
"type": "function",
|
||||
"function": {"name": "Grep", "arguments": "{}"},
|
||||
}
|
||||
],
|
||||
},
|
||||
{"role": "tool", "tool_call_id": "call_grep", "content": grep_output},
|
||||
*self._big_tool_message(),
|
||||
]
|
||||
response = client.post(
|
||||
"/v1/compress",
|
||||
json={"messages": messages, "model": "gpt-4", "config": {"mode": "lossless"}},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert not body.get("ccr_hashes")
|
||||
folded = body["messages"][2]["content"]
|
||||
assert len(folded) < len(grep_output)
|
||||
assert all(f"value_{i} = compute(value_{i})" in folded for i in range(1, 400))
|
||||
json_rows = json.loads(messages[4]["content"])
|
||||
assert all(row["description"] in body["messages"][4]["content"] for row in json_rows)
|
||||
|
||||
@staticmethod
|
||||
def _read_call(call_id, path, name="Read"):
|
||||
return {
|
||||
"role": "assistant",
|
||||
"content": None,
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": call_id,
|
||||
"type": "function",
|
||||
"function": {"name": name, "arguments": json.dumps({"file_path": path})},
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
def test_lossless_mode_keeps_a_read_the_conversation_later_edited(self, client):
|
||||
source = "\n".join(f"def function_{i}(): return {i}" for i in range(400))
|
||||
messages = [
|
||||
{"role": "user", "content": "Change a.py."},
|
||||
self._read_call("call_read", "/x/a.py"),
|
||||
{"role": "tool", "tool_call_id": "call_read", "content": source},
|
||||
self._read_call("call_edit", "/x/a.py", name="Edit"),
|
||||
{"role": "tool", "tool_call_id": "call_edit", "content": "Edited /x/a.py"},
|
||||
{"role": "user", "content": "Now what?"},
|
||||
]
|
||||
response = client.post(
|
||||
"/v1/compress",
|
||||
json={"messages": messages, "model": "gpt-4", "config": {"mode": "lossless"}},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert not body.get("ccr_hashes")
|
||||
assert body["messages"][2]["content"] == source
|
||||
|
||||
def test_lossless_mode_emits_no_dedup_pointer(self, client, monkeypatch):
|
||||
monkeypatch.setenv("HEADROOM_DEDUPE", "1")
|
||||
output = "\n".join(f"2026-01-01 worker-{i} finished job {i}" for i in range(400))
|
||||
messages = [{"role": "user", "content": "Run it twice."}]
|
||||
for call_id in ("call_1", "call_2"):
|
||||
messages.append(self._read_call(call_id, "/x/run.sh", name="Bash"))
|
||||
messages.append({"role": "tool", "tool_call_id": call_id, "content": output})
|
||||
messages.append({"role": "user", "content": "Same?"})
|
||||
|
||||
response = client.post(
|
||||
"/v1/compress",
|
||||
json={"messages": messages, "model": "gpt-4", "config": {"mode": "lossless"}},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
second = response.json()["messages"][4]["content"]
|
||||
assert "same as msg" not in second
|
||||
assert "finished job 399" in second
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("repetitions", "strategy"), [(1, "json_minifier"), (20, "smart_crusher")]
|
||||
)
|
||||
@@ -597,7 +682,7 @@ class TestCompressEndpointModeValidation:
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad_mode",
|
||||
["lossless", "CCR", "ccr ", "no_ccr", "", 7, ["ccr"]],
|
||||
["LOSSLESS", "CCR", "ccr ", "no_ccr", "", 7, ["ccr"]],
|
||||
)
|
||||
def test_unknown_mode_returns_400(self, client, bad_mode):
|
||||
response = client.post(
|
||||
@@ -613,7 +698,7 @@ class TestCompressEndpointModeValidation:
|
||||
assert data["error"]["type"] == "invalid_request"
|
||||
message = data["error"]["message"]
|
||||
# The message must tell the caller what IS valid.
|
||||
for valid in ("ccr", "lossy_inline", "lossless_then_lossy"):
|
||||
for valid in ("ccr", "lossy_inline", "lossless_then_lossy", "lossless"):
|
||||
assert valid in message
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
|
||||
+1
-1
@@ -283,7 +283,7 @@ Compression-only endpoint. Compresses messages without ever making a **completio
|
||||
"model": "gpt-4o", // tokenizer + context limit
|
||||
"token_budget": 8000, // optional: override the context limit
|
||||
"config": { // optional
|
||||
"mode": "lossy_inline", // ccr | lossy_inline | lossless_then_lossy
|
||||
"mode": "lossy_inline", // ccr | lossy_inline | lossless_then_lossy | lossless
|
||||
"frozen_message_count": 12, // pin an already-cached prefix
|
||||
"session_id": "conv-8f1c", // session mode: Headroom keeps the replay state
|
||||
"compress_user_messages": false,
|
||||
|
||||
Reference in New Issue
Block a user