Files
headroom/examples/anthropic_create_stream_local.py
Rudy Mizrahi CelekliandJD Davis 3910caabd6 fix(client): preserve Anthropic create stream flags (#4500)
## Description

`HeadroomClient.messages.create(..., stream=False)` raises `TypeError:
... got multiple values for keyword argument 'stream'` before reaching
Anthropic. The wrapper hardcodes `stream=False` and forwards the
caller's same keyword through `**kwargs`. `stream=True` fails the same
way.

Consume the flag explicitly and preserve the native entrypoint
distinction: `messages.create(stream=True)` returns an iterator of raw
events; `messages.stream()` continues to return Anthropic's stream
context manager. A private transport selector keeps their dispatch
separate while lifecycle extension metadata continues to report the
existing `anthropic` API style.

No issue is linked: this was reproduced while auditing the existing SDK
compatibility contract. Related open #4482 owns existing wrapped-stream
cleanup; this change does not replace that cleanup implementation.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Accept and forward an explicit `stream` flag from Anthropic
`Messages.create`.
- Dispatch streaming create calls through native
`messages.create(stream=True)` and reuse the existing stream wrapper.
- Preserve non-streaming response metrics and native `messages.stream()`
behavior.
- Add regression tests and a runnable loopback example using the actual
Anthropic SDK and HTTPX.

## Testing

- [ ] Unit tests pass (`pytest`): full required suite pending; focused
results below.
- [x] Linting passes (`ruff check .`): full pinned Ruff0.17.0 passes.
- [x] Type checking passes (`mypy headroom`): normal full-package
pre-commit hook passed
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
Baseline regression: 3 failed, 1 passed
Patched focused client/provider suites: 48 passed
Changed-file Ruff lint/format and git diff --check: passed
```

The focused runs used existing Python 3.13.5; changed-file lint/format
also passed the pinned Ruff 0.17.0. Normal repository pre-commit and
commit-message hooks passed, including full-package mypy, pinned Ruff,
plugin-version checks and commitlint. The complete required pre-push
checks remain separate pending gates. Global pytest plugin discovery was
disabled because an unrelated installed plugin interferes with an
existing test's process-wide import monkeypatch; pytest_asyncio.plugin
was explicitly enabled. Final focused results: 48 passed in 9.11s
without warnings; this is not a full-suite pass.

## Real Behavior Proof

- Environment: macOS 27.0.1 arm64, Python 3.13.5, actual Anthropic SDK
0.107.1, HTTPX 0.28.1. Audit mode; cache optimization disabled; offline
mode, beacon off and DNT; temporary task-owned workspace/config/cache
directories. The example starts a loopback HTTP fixture, with a
non-secret fixture API key.
- Exact command / steps: `PYTHONPATH=. python
examples/anthropic_create_stream_local.py`.
- Observed result: native and wrapped non-streaming calls both returned
`local fixture`. Native and wrapped streaming create calls yielded the
same six event types (`message_start`, `content_block_start`,
`content_block_delta`, `content_block_stop`, `message_delta`,
`message_stop`). Existing `messages.stream()` produced `local fixture`
through its context manager. All five actual HTTP responses were closed
after full consumption. Lifecycle events kept `api_style=anthropic`.
Before the fix, the native call succeeded while the wrapped call raised
the duplicate-keyword TypeError without making another HTTP request.
- Not tested: actual Anthropic endpoints, paid models, model quality,
production deployment, early-close resource cleanup (owned by #4482), or
stream usage accounting.

## Runtime Rollout Safety

- Rollout-managed feature(s): N/A; existing Python SDK
argument/transport compatibility.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: explicit `stream=False` is accepted;
explicit `stream=True` uses the native create iterator. Omitted flag
retains non-streaming behavior.
- Kill switch / disable path: use the original SDK client.
- Unsafe override required: no.
- Qualification impact: no rollout-channel changes.
- Rollback path: revert this change.


### Broader check limits and upload exception

Rust1.95.0 `cargo fmt --all -- --check` and `cargo clippy --workspace --
-D warnings` passed on the unchanged native sources. The cold
workspace-test build was stopped twice by a disk-reserve guard; no full
Rust-test or native-dependent Python-suite pass is claimed. Normal
commit hooks passed, including pinned Ruff0.17.0, whole-package mypy and
commitlint. These patches do not change Rust sources or Cargo
dependencies.

The [installed pre-push
hook](https://github.com/headroomlabs-ai/headroom/blob/17d38e5699b7d7c69ac6addcd762f39e42dd7f00/scripts/install-git-hooks.sh)
documents an occasional `git push --no-verify` upload exception. It was
used for this independently reviewed Python-only fix after the bounded
native-check attempts. Required hosted CI and maintainer review remain
pending; the complete pre-push gate has not passed locally.

Prepared and independently reviewed with AI assistance; actual runtime
proof and its limits are recorded above.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [ ] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title

## Additional Notes

No new dependency or provider capability is introduced. The local HTTP
fixture is protocol evidence, not a claim of live provider verification.
Required full Rust/Python pre-push gates must be reported before
publication readiness; normal commit hooks passed.

Streaming create returns Headroom’s existing event iterator. Native
Stream-specific methods/attributes and early-close behavior are not
added or qualified by this change.


### Follow-up native validation

The actual Python extension now builds and imports from the unchanged
native source using Rust1.95.0 and the matching Xcode26.4 SDK. At this
exact PR head, the Makefile pre-push Python test-file selection passes
**177 tests with 4 skips**; one existing Anthropic approximation warning
is reported. Whole-tree pinned Ruff0.17.0 passes. The complete Rust
workspace test build was again stopped by the local disk-reserve guard
before completion; this is not a Rust-test pass. The incomplete
task-owned build cache was removed after all build processes exited,
retaining the source-built extension and logs. Full pytest, complete
pre-push success and required hosted CI remain unclaimed.


<!-- security-sync-20261028 -->
### Security baseline sync — 2026-10-28

Current exact head: `f85946f309c0c26308b1462bf7006f0b078ac7be`. Normal
history merge of upstream `3c6f82b06091206379106f8fe1a54f2400ed7a93`
brings in the maintainer-owned dependency remediation
[#4517](https://github.com/headroomlabs-ai/headroom/pull/4517),
following the prior npm baseline sync. The authored fix and its
test/example files are unchanged by this sync; no additional dependency
policy or runtime logic was introduced. This imports the maintainer
remediation requested by review; it does not establish audit success or
approval.

Validation on this head (macOS arm64, Python3.13.5, offline mode/beacon
off/DNT, existing source-built native extension):
`PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python -m pytest -p
pytest_asyncio.plugin tests/test_anthropic_client_create_stream.py
tests/test_providers/test_anthropic.py
tests/test_provider_registry_extended.py -q` (58 passed; one existing
token-count approximation warning). `PYTHONPATH=. python
examples/anthropic_create_stream_local.py` passed against actual
Anthropic0.107.1 and HTTPX0.28.1: native/wrapped six-event streaming
agreement, native stream context manager, all five HTTP responses
closed.

`PRE_COMMIT_HOME=<existing task cache> pre-commit run --all-files`
passed, including pinned Ruff, whole-package mypy, version and
merge-conflict checks. Normal commit hooks and commitlint passed.
Existing native sources/Cargo dependencies are unchanged. The documented
occasional `git push --no-verify` exception was used for a normal
fast-forward upload; full Rust workspace tests, full pytest and complete
pre-push qualification were not rerun or claimed.

Hosted security checks and maintainer review remain separate gates.
Existing human-readiness declarations are preserved; this status update
is evidence of local validation and a baseline sync, not human approval
or a claim that hosted CI is green.

As of `2026-10-28T17:40:32.107305+00:00`, hosted pip-audit and all five
npm audits passed on this exact head ([security
workflow](https://github.com/headroomlabs-ai/headroom/actions/runs/40737711996/job/122990639723));
cargo audit also passed. Other hosted jobs remain in progress, and the
human review state remains `CHANGES_REQUESTED`. This is not complete CI
qualification or human approval.

Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-29 20:25:48 -05:00

218 lines
7.9 KiB
Python

"""Actual SDK + loopback HTTP proof; no remote provider or model calls."""
import json
import os
import tempfile
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
def main():
scratch = tempfile.TemporaryDirectory(prefix="headroom-client-proof-")
for key in ["HEADROOM_WORKSPACE_DIR", "HEADROOM_CONFIG_DIR", "XDG_CACHE_HOME"]:
os.environ[key] = str(Path(scratch.name) / key.lower())
os.environ.update(HEADROOM_OFFLINE="1", HEADROOM_BEACON="off", DO_NOT_TRACK="1")
import httpx
from anthropic import Anthropic
from headroom import AnthropicProvider, HeadroomClient
from headroom.config import HeadroomConfig
requests = []
observed_styles = []
class Observer:
def on_pipeline_event(self, event):
if "api_style" in event.metadata:
observed_styles.append(event.metadata["api_style"])
class Handler(BaseHTTPRequestHandler):
def log_message(self, *args):
pass
def do_POST(self):
data = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
requests.append(data)
if data.get("stream"):
events = [
(
"message_start",
{
"type": "message_start",
"message": {
"id": "msg_fixture",
"type": "message",
"role": "assistant",
"model": data["model"],
"content": [],
"stop_reason": None,
"stop_sequence": None,
"usage": {"input_tokens": 1, "output_tokens": 0},
},
},
),
(
"content_block_start",
{
"type": "content_block_start",
"index": 0,
"content_block": {"type": "text", "text": ""},
},
),
(
"content_block_delta",
{
"type": "content_block_delta",
"index": 0,
"delta": {"type": "text_delta", "text": "local fixture"},
},
),
("content_block_stop", {"type": "content_block_stop", "index": 0}),
(
"message_delta",
{
"type": "message_delta",
"delta": {"stop_reason": "end_turn", "stop_sequence": None},
"usage": {"output_tokens": 2},
},
),
("message_stop", {"type": "message_stop"}),
]
payload = "".join(
f"event: {name}\ndata: {json.dumps(event)}\n\n" for name, event in events
).encode()
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
return
payload = json.dumps(
{
"id": "msg_fixture",
"type": "message",
"role": "assistant",
"model": data["model"],
"content": [{"type": "text", "text": "local fixture"}],
"stop_reason": "end_turn",
"stop_sequence": None,
"usage": {"input_tokens": 1, "output_tokens": 2},
}
).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
threading.Thread(target=server.serve_forever, daemon=True).start()
responses = []
http = httpx.Client(event_hooks={"response": [responses.append]})
original = Anthropic(
http_client=http,
api_key="local-fixture",
base_url=f"http://127.0.0.1:{server.server_port}",
max_retries=0,
)
wrapped = HeadroomClient(
original,
AnthropicProvider(warn=False),
store_url=f"sqlite:///{scratch.name}/metrics.db",
enable_cache_optimizer=False,
config=HeadroomConfig(
store_url=f"sqlite:///{scratch.name}/metrics.db",
discover_pipeline_extensions=False,
pipeline_extensions=[Observer()],
),
)
kwargs = {
"model": "claude-fixture",
"messages": [{"role": "user", "content": "hello"}],
"max_tokens": 8,
"stream": False,
}
try:
native = original.messages.create(**kwargs)
print(
json.dumps(
{"case": "native", "text": native.content[0].text, "requests": len(requests)}
)
)
try:
result = wrapped.messages.create(**kwargs)
print(
json.dumps(
{"case": "wrapped", "text": result.content[0].text, "requests": len(requests)}
)
)
kwargs["stream"] = True
native_stream = original.messages.create(**kwargs)
native_types = [event.type for event in native_stream]
native_stream.close()
wrapped_stream = wrapped.messages.create(**kwargs)
wrapped_types = [event.type for event in wrapped_stream]
wrapped_stream.close()
assert native_types == wrapped_types
print(
json.dumps(
{
"case": "create_stream",
"native_types": native_types,
"wrapped_types": wrapped_types,
"native_closed": native_stream.response.is_closed,
"requests": len(requests),
}
)
)
del kwargs["stream"]
with wrapped.messages.stream(**kwargs) as manager:
text = "".join(manager.text_stream)
assert text == "local fixture"
assert manager.response.is_closed
assert all(response.is_closed for response in responses)
assert observed_styles and set(observed_styles) == {"anthropic"}
print(
json.dumps(
{
"case": "resources",
"http_responses": len(responses),
"all_closed": all(response.is_closed for response in responses),
}
)
)
print(
json.dumps(
{
"case": "stream_manager",
"text": text,
"response_closed": manager.response.is_closed,
"requests": len(requests),
}
)
)
except Exception as exc:
print(
json.dumps(
{
"case": "wrapped",
"error": type(exc).__name__,
"message": str(exc),
"requests": len(requests),
}
)
)
raise
finally:
wrapped.close()
original.close()
server.shutdown()
server.server_close()
scratch.cleanup()
if __name__ == "__main__":
main()