mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(client): preserve Anthropic create stream flags (#4500)
## Description `HeadroomClient.messages.create(..., stream=False)` raises `TypeError: ... got multiple values for keyword argument 'stream'` before reaching Anthropic. The wrapper hardcodes `stream=False` and forwards the caller's same keyword through `**kwargs`. `stream=True` fails the same way. Consume the flag explicitly and preserve the native entrypoint distinction: `messages.create(stream=True)` returns an iterator of raw events; `messages.stream()` continues to return Anthropic's stream context manager. A private transport selector keeps their dispatch separate while lifecycle extension metadata continues to report the existing `anthropic` API style. No issue is linked: this was reproduced while auditing the existing SDK compatibility contract. Related open #4482 owns existing wrapped-stream cleanup; this change does not replace that cleanup implementation. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Accept and forward an explicit `stream` flag from Anthropic `Messages.create`. - Dispatch streaming create calls through native `messages.create(stream=True)` and reuse the existing stream wrapper. - Preserve non-streaming response metrics and native `messages.stream()` behavior. - Add regression tests and a runnable loopback example using the actual Anthropic SDK and HTTPX. ## Testing - [ ] Unit tests pass (`pytest`): full required suite pending; focused results below. - [x] Linting passes (`ruff check .`): full pinned Ruff0.17.0 passes. - [x] Type checking passes (`mypy headroom`): normal full-package pre-commit hook passed - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text Baseline regression: 3 failed, 1 passed Patched focused client/provider suites: 48 passed Changed-file Ruff lint/format and git diff --check: passed ``` The focused runs used existing Python 3.13.5; changed-file lint/format also passed the pinned Ruff 0.17.0. Normal repository pre-commit and commit-message hooks passed, including full-package mypy, pinned Ruff, plugin-version checks and commitlint. The complete required pre-push checks remain separate pending gates. Global pytest plugin discovery was disabled because an unrelated installed plugin interferes with an existing test's process-wide import monkeypatch; pytest_asyncio.plugin was explicitly enabled. Final focused results: 48 passed in 9.11s without warnings; this is not a full-suite pass. ## Real Behavior Proof - Environment: macOS 27.0.1 arm64, Python 3.13.5, actual Anthropic SDK 0.107.1, HTTPX 0.28.1. Audit mode; cache optimization disabled; offline mode, beacon off and DNT; temporary task-owned workspace/config/cache directories. The example starts a loopback HTTP fixture, with a non-secret fixture API key. - Exact command / steps: `PYTHONPATH=. python examples/anthropic_create_stream_local.py`. - Observed result: native and wrapped non-streaming calls both returned `local fixture`. Native and wrapped streaming create calls yielded the same six event types (`message_start`, `content_block_start`, `content_block_delta`, `content_block_stop`, `message_delta`, `message_stop`). Existing `messages.stream()` produced `local fixture` through its context manager. All five actual HTTP responses were closed after full consumption. Lifecycle events kept `api_style=anthropic`. Before the fix, the native call succeeded while the wrapped call raised the duplicate-keyword TypeError without making another HTTP request. - Not tested: actual Anthropic endpoints, paid models, model quality, production deployment, early-close resource cleanup (owned by #4482), or stream usage accounting. ## Runtime Rollout Safety - Rollout-managed feature(s): N/A; existing Python SDK argument/transport compatibility. - Minimum rollout channel: N/A. - Stable/default behavior changed: explicit `stream=False` is accepted; explicit `stream=True` uses the native create iterator. Omitted flag retains non-streaming behavior. - Kill switch / disable path: use the original SDK client. - Unsafe override required: no. - Qualification impact: no rollout-channel changes. - Rollback path: revert this change. ### Broader check limits and upload exception Rust1.95.0 `cargo fmt --all -- --check` and `cargo clippy --workspace -- -D warnings` passed on the unchanged native sources. The cold workspace-test build was stopped twice by a disk-reserve guard; no full Rust-test or native-dependent Python-suite pass is claimed. Normal commit hooks passed, including pinned Ruff0.17.0, whole-package mypy and commitlint. These patches do not change Rust sources or Cargo dependencies. The [installed pre-push hook](https://github.com/headroomlabs-ai/headroom/blob/17d38e5699b7d7c69ac6addcd762f39e42dd7f00/scripts/install-git-hooks.sh) documents an occasional `git push --no-verify` upload exception. It was used for this independently reviewed Python-only fix after the bounded native-check attempts. Required hosted CI and maintainer review remain pending; the complete pre-push gate has not passed locally. Prepared and independently reviewed with AI assistance; actual runtime proof and its limits are recorded above. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [ ] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title ## Additional Notes No new dependency or provider capability is introduced. The local HTTP fixture is protocol evidence, not a claim of live provider verification. Required full Rust/Python pre-push gates must be reported before publication readiness; normal commit hooks passed. Streaming create returns Headroom’s existing event iterator. Native Stream-specific methods/attributes and early-close behavior are not added or qualified by this change. ### Follow-up native validation The actual Python extension now builds and imports from the unchanged native source using Rust1.95.0 and the matching Xcode26.4 SDK. At this exact PR head, the Makefile pre-push Python test-file selection passes **177 tests with 4 skips**; one existing Anthropic approximation warning is reported. Whole-tree pinned Ruff0.17.0 passes. The complete Rust workspace test build was again stopped by the local disk-reserve guard before completion; this is not a Rust-test pass. The incomplete task-owned build cache was removed after all build processes exited, retaining the source-built extension and logs. Full pytest, complete pre-push success and required hosted CI remain unclaimed. <!-- security-sync-20261028 --> ### Security baseline sync — 2026-10-28 Current exact head: `f85946f309c0c26308b1462bf7006f0b078ac7be`. Normal history merge of upstream `3c6f82b06091206379106f8fe1a54f2400ed7a93` brings in the maintainer-owned dependency remediation [#4517](https://github.com/headroomlabs-ai/headroom/pull/4517), following the prior npm baseline sync. The authored fix and its test/example files are unchanged by this sync; no additional dependency policy or runtime logic was introduced. This imports the maintainer remediation requested by review; it does not establish audit success or approval. Validation on this head (macOS arm64, Python3.13.5, offline mode/beacon off/DNT, existing source-built native extension): `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -p pytest_asyncio.plugin tests/test_anthropic_client_create_stream.py tests/test_providers/test_anthropic.py tests/test_provider_registry_extended.py -q` (58 passed; one existing token-count approximation warning). `PYTHONPATH=. python examples/anthropic_create_stream_local.py` passed against actual Anthropic0.107.1 and HTTPX0.28.1: native/wrapped six-event streaming agreement, native stream context manager, all five HTTP responses closed. `PRE_COMMIT_HOME=<existing task cache> pre-commit run --all-files` passed, including pinned Ruff, whole-package mypy, version and merge-conflict checks. Normal commit hooks and commitlint passed. Existing native sources/Cargo dependencies are unchanged. The documented occasional `git push --no-verify` exception was used for a normal fast-forward upload; full Rust workspace tests, full pytest and complete pre-push qualification were not rerun or claimed. Hosted security checks and maintainer review remain separate gates. Existing human-readiness declarations are preserved; this status update is evidence of local validation and a baseline sync, not human approval or a claim that hosted CI is green. As of `2026-10-28T17:40:32.107305+00:00`, hosted pip-audit and all five npm audits passed on this exact head ([security workflow](https://github.com/headroomlabs-ai/headroom/actions/runs/40737711996/job/122990639723)); cargo audit also passed. Other hosted jobs remain in progress, and the human review state remains `CHANGES_REQUESTED`. This is not complete CI qualification or human approval. Signed-off-by: Rudy Celekli <rudy@gradiahq.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JD Davis
parent
6f958e94c2
commit
3910caabd6
@@ -0,0 +1,217 @@
|
||||
"""Actual SDK + loopback HTTP proof; no remote provider or model calls."""
|
||||
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main():
|
||||
scratch = tempfile.TemporaryDirectory(prefix="headroom-client-proof-")
|
||||
for key in ["HEADROOM_WORKSPACE_DIR", "HEADROOM_CONFIG_DIR", "XDG_CACHE_HOME"]:
|
||||
os.environ[key] = str(Path(scratch.name) / key.lower())
|
||||
os.environ.update(HEADROOM_OFFLINE="1", HEADROOM_BEACON="off", DO_NOT_TRACK="1")
|
||||
|
||||
import httpx
|
||||
from anthropic import Anthropic
|
||||
|
||||
from headroom import AnthropicProvider, HeadroomClient
|
||||
from headroom.config import HeadroomConfig
|
||||
|
||||
requests = []
|
||||
observed_styles = []
|
||||
|
||||
class Observer:
|
||||
def on_pipeline_event(self, event):
|
||||
if "api_style" in event.metadata:
|
||||
observed_styles.append(event.metadata["api_style"])
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
def do_POST(self):
|
||||
data = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
|
||||
requests.append(data)
|
||||
if data.get("stream"):
|
||||
events = [
|
||||
(
|
||||
"message_start",
|
||||
{
|
||||
"type": "message_start",
|
||||
"message": {
|
||||
"id": "msg_fixture",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": data["model"],
|
||||
"content": [],
|
||||
"stop_reason": None,
|
||||
"stop_sequence": None,
|
||||
"usage": {"input_tokens": 1, "output_tokens": 0},
|
||||
},
|
||||
},
|
||||
),
|
||||
(
|
||||
"content_block_start",
|
||||
{
|
||||
"type": "content_block_start",
|
||||
"index": 0,
|
||||
"content_block": {"type": "text", "text": ""},
|
||||
},
|
||||
),
|
||||
(
|
||||
"content_block_delta",
|
||||
{
|
||||
"type": "content_block_delta",
|
||||
"index": 0,
|
||||
"delta": {"type": "text_delta", "text": "local fixture"},
|
||||
},
|
||||
),
|
||||
("content_block_stop", {"type": "content_block_stop", "index": 0}),
|
||||
(
|
||||
"message_delta",
|
||||
{
|
||||
"type": "message_delta",
|
||||
"delta": {"stop_reason": "end_turn", "stop_sequence": None},
|
||||
"usage": {"output_tokens": 2},
|
||||
},
|
||||
),
|
||||
("message_stop", {"type": "message_stop"}),
|
||||
]
|
||||
payload = "".join(
|
||||
f"event: {name}\ndata: {json.dumps(event)}\n\n" for name, event in events
|
||||
).encode()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "text/event-stream")
|
||||
self.send_header("Content-Length", str(len(payload)))
|
||||
self.end_headers()
|
||||
self.wfile.write(payload)
|
||||
return
|
||||
payload = json.dumps(
|
||||
{
|
||||
"id": "msg_fixture",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": data["model"],
|
||||
"content": [{"type": "text", "text": "local fixture"}],
|
||||
"stop_reason": "end_turn",
|
||||
"stop_sequence": None,
|
||||
"usage": {"input_tokens": 1, "output_tokens": 2},
|
||||
}
|
||||
).encode()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(payload)))
|
||||
self.end_headers()
|
||||
self.wfile.write(payload)
|
||||
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
responses = []
|
||||
http = httpx.Client(event_hooks={"response": [responses.append]})
|
||||
original = Anthropic(
|
||||
http_client=http,
|
||||
api_key="local-fixture",
|
||||
base_url=f"http://127.0.0.1:{server.server_port}",
|
||||
max_retries=0,
|
||||
)
|
||||
wrapped = HeadroomClient(
|
||||
original,
|
||||
AnthropicProvider(warn=False),
|
||||
store_url=f"sqlite:///{scratch.name}/metrics.db",
|
||||
enable_cache_optimizer=False,
|
||||
config=HeadroomConfig(
|
||||
store_url=f"sqlite:///{scratch.name}/metrics.db",
|
||||
discover_pipeline_extensions=False,
|
||||
pipeline_extensions=[Observer()],
|
||||
),
|
||||
)
|
||||
kwargs = {
|
||||
"model": "claude-fixture",
|
||||
"messages": [{"role": "user", "content": "hello"}],
|
||||
"max_tokens": 8,
|
||||
"stream": False,
|
||||
}
|
||||
try:
|
||||
native = original.messages.create(**kwargs)
|
||||
print(
|
||||
json.dumps(
|
||||
{"case": "native", "text": native.content[0].text, "requests": len(requests)}
|
||||
)
|
||||
)
|
||||
try:
|
||||
result = wrapped.messages.create(**kwargs)
|
||||
print(
|
||||
json.dumps(
|
||||
{"case": "wrapped", "text": result.content[0].text, "requests": len(requests)}
|
||||
)
|
||||
)
|
||||
kwargs["stream"] = True
|
||||
native_stream = original.messages.create(**kwargs)
|
||||
native_types = [event.type for event in native_stream]
|
||||
native_stream.close()
|
||||
wrapped_stream = wrapped.messages.create(**kwargs)
|
||||
wrapped_types = [event.type for event in wrapped_stream]
|
||||
wrapped_stream.close()
|
||||
assert native_types == wrapped_types
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"case": "create_stream",
|
||||
"native_types": native_types,
|
||||
"wrapped_types": wrapped_types,
|
||||
"native_closed": native_stream.response.is_closed,
|
||||
"requests": len(requests),
|
||||
}
|
||||
)
|
||||
)
|
||||
del kwargs["stream"]
|
||||
with wrapped.messages.stream(**kwargs) as manager:
|
||||
text = "".join(manager.text_stream)
|
||||
assert text == "local fixture"
|
||||
assert manager.response.is_closed
|
||||
assert all(response.is_closed for response in responses)
|
||||
assert observed_styles and set(observed_styles) == {"anthropic"}
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"case": "resources",
|
||||
"http_responses": len(responses),
|
||||
"all_closed": all(response.is_closed for response in responses),
|
||||
}
|
||||
)
|
||||
)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"case": "stream_manager",
|
||||
"text": text,
|
||||
"response_closed": manager.response.is_closed,
|
||||
"requests": len(requests),
|
||||
}
|
||||
)
|
||||
)
|
||||
except Exception as exc:
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"case": "wrapped",
|
||||
"error": type(exc).__name__,
|
||||
"message": str(exc),
|
||||
"requests": len(requests),
|
||||
}
|
||||
)
|
||||
)
|
||||
raise
|
||||
finally:
|
||||
wrapped.close()
|
||||
original.close()
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
scratch.cleanup()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+8
-3
@@ -202,6 +202,7 @@ class Messages:
|
||||
model: str,
|
||||
messages: list[dict[str, Any]],
|
||||
max_tokens: int = 1024,
|
||||
stream: bool = False,
|
||||
# Headroom-specific parameters
|
||||
headroom_mode: str | None = None,
|
||||
headroom_cache_prefix_tokens: int | None = None,
|
||||
@@ -218,6 +219,7 @@ class Messages:
|
||||
model: Model name.
|
||||
messages: List of messages.
|
||||
max_tokens: Maximum tokens in response.
|
||||
stream: Return an iterator of raw message events when True.
|
||||
headroom_mode: Override default mode ("audit" | "optimize").
|
||||
headroom_cache_prefix_tokens: Target cache-aligned prefix size.
|
||||
headroom_output_buffer_tokens: Reserve tokens for output.
|
||||
@@ -231,13 +233,13 @@ class Messages:
|
||||
return self._client._create(
|
||||
model=model,
|
||||
messages=messages,
|
||||
stream=False,
|
||||
stream=stream,
|
||||
headroom_mode=headroom_mode,
|
||||
headroom_cache_prefix_tokens=headroom_cache_prefix_tokens,
|
||||
headroom_output_buffer_tokens=headroom_output_buffer_tokens,
|
||||
headroom_keep_turns=headroom_keep_turns,
|
||||
headroom_tool_profiles=headroom_tool_profiles,
|
||||
api_style="anthropic",
|
||||
api_style="anthropic_create",
|
||||
max_tokens=max_tokens,
|
||||
**kwargs,
|
||||
)
|
||||
@@ -474,6 +476,9 @@ class HeadroomClient:
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Internal implementation of create."""
|
||||
transport_style = api_style
|
||||
if api_style == "anthropic_create":
|
||||
api_style = "anthropic"
|
||||
request_id = generate_request_id()
|
||||
timestamp = datetime.now(timezone.utc).replace(tzinfo=None)
|
||||
mode = HeadroomMode(headroom_mode) if headroom_mode else self._default_mode
|
||||
@@ -675,7 +680,7 @@ class HeadroomClient:
|
||||
# Call underlying client based on API style
|
||||
try:
|
||||
response = call_client_transport(
|
||||
api_style,
|
||||
transport_style,
|
||||
self,
|
||||
model=model,
|
||||
messages=optimized_messages,
|
||||
|
||||
@@ -373,6 +373,34 @@ def _call_openai_transport(
|
||||
return response
|
||||
|
||||
|
||||
def _call_anthropic_create_transport(
|
||||
client: Any,
|
||||
*,
|
||||
model: str,
|
||||
messages: list[dict[str, Any]],
|
||||
stream: bool,
|
||||
metrics: Any,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
if stream:
|
||||
response = client._original.messages.create(
|
||||
model=model,
|
||||
messages=messages,
|
||||
stream=True,
|
||||
**kwargs,
|
||||
)
|
||||
return client._wrap_stream(response, metrics)
|
||||
|
||||
return _call_anthropic_transport(
|
||||
client,
|
||||
model=model,
|
||||
messages=messages,
|
||||
stream=False,
|
||||
metrics=metrics,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
def _call_anthropic_transport(
|
||||
client: Any,
|
||||
*,
|
||||
@@ -409,6 +437,7 @@ def _call_anthropic_transport(
|
||||
_ClientTransport = Callable[..., Any]
|
||||
_CLIENT_TRANSPORTS: dict[str, _ClientTransport] = {
|
||||
"anthropic": _call_anthropic_transport,
|
||||
"anthropic_create": _call_anthropic_create_transport,
|
||||
"openai": _call_openai_transport,
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
"""The SDK wrapper preserves Anthropic create/stream entrypoint contracts."""
|
||||
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.client import Messages
|
||||
from headroom.providers.registry import call_client_transport
|
||||
|
||||
|
||||
@pytest.mark.parametrize("stream", [False, True])
|
||||
def test_messages_create_consumes_explicit_stream_flag(stream):
|
||||
client = SimpleNamespace(_create=Mock(return_value="response"))
|
||||
assert Messages(client).create(model="claude", messages=[], stream=stream) == "response"
|
||||
assert client._create.call_args.kwargs["stream"] is stream
|
||||
assert client._create.call_args.kwargs["api_style"] == "anthropic_create"
|
||||
|
||||
|
||||
def test_create_stream_uses_native_create_iterator_not_stream_manager():
|
||||
events = iter([{"type": "message_start"}, {"type": "message_stop"}])
|
||||
original = SimpleNamespace(
|
||||
messages=SimpleNamespace(create=Mock(return_value=events), stream=Mock())
|
||||
)
|
||||
storage = SimpleNamespace(save=Mock())
|
||||
client = SimpleNamespace(
|
||||
_original=original,
|
||||
_storage=storage,
|
||||
_wrap_stream=Mock(side_effect=lambda value, metrics: value),
|
||||
)
|
||||
metrics = SimpleNamespace(tokens_output=0, cached_tokens=0)
|
||||
|
||||
result = call_client_transport(
|
||||
"anthropic_create",
|
||||
client,
|
||||
model="claude",
|
||||
messages=[],
|
||||
stream=True,
|
||||
metrics=metrics,
|
||||
max_tokens=8,
|
||||
)
|
||||
|
||||
assert list(result) == [{"type": "message_start"}, {"type": "message_stop"}]
|
||||
original.messages.create.assert_called_once_with(
|
||||
model="claude", messages=[], stream=True, max_tokens=8
|
||||
)
|
||||
original.messages.stream.assert_not_called()
|
||||
client._wrap_stream.assert_called_once_with(events, metrics)
|
||||
|
||||
|
||||
def test_messages_stream_keeps_native_context_manager():
|
||||
manager = object()
|
||||
original = SimpleNamespace(
|
||||
messages=SimpleNamespace(create=Mock(), stream=Mock(return_value=manager))
|
||||
)
|
||||
storage = SimpleNamespace(save=Mock())
|
||||
client = SimpleNamespace(_original=original, _storage=storage)
|
||||
metrics = SimpleNamespace(tokens_output=0, cached_tokens=0)
|
||||
|
||||
result = call_client_transport(
|
||||
"anthropic", client, model="claude", messages=[], stream=True, metrics=metrics, max_tokens=8
|
||||
)
|
||||
|
||||
assert result is manager
|
||||
original.messages.stream.assert_called_once_with(model="claude", messages=[], max_tokens=8)
|
||||
original.messages.create.assert_not_called()
|
||||
storage.save.assert_called_once_with(metrics)
|
||||
Reference in New Issue
Block a user