mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
## Description
When a batch result needs a CCR continuation, `BatchResultProcessor`
made the follow-up call to hard-coded public hosts (`api.anthropic.com`,
`api.openai.com`, `generativelanguage.googleapis.com`). The upstream the
batch was actually submitted to is stored in
`BatchContext.api_base_url`, written by `_store_anthropic_batch_context`
(`ANTHROPIC_API_URL`) and `_store_google_batch_context`
(`GEMINI_API_URL`), but nothing ever read it. With a custom base URL or
gateway, the continuation sent that upstream's stored credential, the
conversation and the retrieved CCR originals to the public host.
Continuations now go to `BatchContext.api_base_url`. They fall back to
the public host only when no upstream was recorded. For Gemini the key
now goes in the `x-goog-api-key` header instead of `?key=` in the URL,
so it stays out of URLs, request logs and `HTTPStatusError` text.
Behaviour change for users: none for default setups (the recorded
upstream is the public host). Batches submitted through a custom
upstream now get their continuations sent there too.
Fix adapted from a proposal by HetCreep.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
## Changes Made
- `headroom/ccr/batch_processor.py`: new `_continuation_base_url()`
(recorded upstream first, then the public default; trailing `/`
stripped). The Anthropic, OpenAI and Google continuations use it. Gemini
sends its key in `x-goog-api-key`.
- `tests/test_ccr_batch_processor.py`: one test per provider. Each
checks the request URL and the auth header when an upstream was
recorded, and for Gemini that the key is not in the URL.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
# new tests against origin/main (c21b0813)
$ pytest tests/test_ccr_batch_processor.py
FAILED ...::TestContinuationCalls::test_anthropic_continuation_uses_configured_api_base_url
FAILED ...::TestContinuationCalls::test_openai_continuation_uses_configured_api_base_url
FAILED ...::TestContinuationCalls::test_google_continuation_uses_configured_api_base_url_and_header_key
========================= 3 failed, 45 passed in 9.67s =========================
# this branch: processor tests + every tests/*batch* file
$ pytest tests/test_ccr_batch_processor.py tests/*batch*
======================= 130 passed, 14 skipped in 39.39s =======================
$ ruff check / ruff format --check (0.17.3): All checks passed!
$ mypy headroom/ccr/batch_processor.py --ignore-missing-imports
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: macOS, Python 3.12. A real `BatchResultProcessor` with a
real `httpx.AsyncClient` on an `httpx.MockTransport` that records
requests (no network).
- Exact command / steps: for each provider,
`_make_continuation_call(...)` with `BatchContext(api_key="demo-key",
api_base_url="http://127.0.0.1:9/gateway")`. Print the recorded request
URL.
- Observed result:
```text
origin/main:
anthropic -> https://api.anthropic.com/v1/messages
openai -> https://api.openai.com/v1/chat/completions
google -> https://generativelanguage.googleapis.com/v1beta/models/m:generateContent key-in-url=True
this branch:
anthropic -> http://127.0.0.1:9/gateway/v1/messages
openai -> http://127.0.0.1:9/gateway/v1/chat/completions
google -> http://127.0.0.1:9/gateway/v1beta/models/m:generateContent key-in-url=False
```
- Real HTTP server check (this branch): a local `http.server` listener
as the "gateway", a real `httpx.AsyncClient` (no mock transport),
`api_base_url=http://127.0.0.1:<port>/gateway`, one continuation per
provider:
```text
anthropic call returned
openai call returned
google call returned
local gateway received: [('/gateway/v1/messages', {'x-api-key': 'demo-key'}),
('/gateway/v1/chat/completions', {'authorization': 'Bearer demo-key'}),
('/gateway/v1beta/models/m:generateContent', {'x-goog-api-key': 'demo-key'})]
```
- Not tested: a live batch against a real provider or gateway.
## Runtime Rollout Safety
- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: see Description
- Kill switch / disable path: none needed
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A
## Additional Notes
Out of scope here: Gemini batch *create* in
`headroom/proxy/handlers/batch.py` still moves `x-goog-api-key` into
`?key=` on the URL. It goes to the configured `GEMINI_API_URL`, so this
is about the key showing up in URLs, not about where it is sent. Worth a
separate follow-up.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
1350 lines
48 KiB
Python
1350 lines
48 KiB
Python
"""Tests for CCR batch result processor.
|
|
|
|
These tests verify that:
|
|
1. Native tool schemas permit real CCR batch continuations
|
|
2. Result parsing for Anthropic, OpenAI, and Google batch formats
|
|
3. CCR tool call detection in batch results
|
|
4. Continuation call handling works for all providers
|
|
5. Error cases and edge cases are handled gracefully
|
|
"""
|
|
|
|
import json
|
|
import re
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
import httpx
|
|
import pytest
|
|
|
|
from headroom.cache import compression_store as compression_store_module
|
|
from headroom.cache.compression_store import CompressionStore, set_request_compression_store
|
|
from headroom.ccr import batch_store as batch_store_module
|
|
from headroom.ccr.batch_processor import (
|
|
BatchResultProcessor,
|
|
process_batch_results,
|
|
)
|
|
from headroom.ccr.batch_store import (
|
|
BatchContext,
|
|
BatchContextStore,
|
|
BatchRequestContext,
|
|
reset_batch_context_store,
|
|
)
|
|
from headroom.ccr.tool_injection import CCR_TOOL_NAME
|
|
|
|
|
|
class TestCustomIdExtraction:
|
|
"""Test _get_custom_id method for different providers."""
|
|
|
|
@pytest.fixture
|
|
def processor(self):
|
|
"""Create a processor instance."""
|
|
http_client = MagicMock(spec=httpx.AsyncClient)
|
|
return BatchResultProcessor(http_client)
|
|
|
|
def test_anthropic_custom_id(self, processor):
|
|
"""Extract custom_id from Anthropic batch result."""
|
|
result = {"custom_id": "anthropic_req_123", "result": {"message": {}}}
|
|
custom_id = processor._get_custom_id(result, "anthropic")
|
|
assert custom_id == "anthropic_req_123"
|
|
|
|
def test_openai_custom_id(self, processor):
|
|
"""Extract custom_id from OpenAI batch result."""
|
|
result = {"custom_id": "openai_req_456", "response": {"body": {}}}
|
|
custom_id = processor._get_custom_id(result, "openai")
|
|
assert custom_id == "openai_req_456"
|
|
|
|
def test_google_custom_id(self, processor):
|
|
"""Extract custom_id from Google batch result (metadata.key)."""
|
|
result = {"metadata": {"key": "google_req_789"}, "response": {}}
|
|
custom_id = processor._get_custom_id(result, "google")
|
|
assert custom_id == "google_req_789"
|
|
|
|
def test_google_missing_metadata(self, processor):
|
|
"""Handle missing metadata in Google result."""
|
|
result = {"response": {}}
|
|
custom_id = processor._get_custom_id(result, "google")
|
|
assert custom_id == ""
|
|
|
|
def test_unknown_provider_fallback(self, processor):
|
|
"""Fallback extraction for unknown provider."""
|
|
result = {"custom_id": "unknown_req", "id": "backup_id"}
|
|
custom_id = processor._get_custom_id(result, "unknown")
|
|
assert custom_id == "unknown_req"
|
|
|
|
def test_unknown_provider_uses_id_fallback(self, processor):
|
|
"""Unknown provider falls back to 'id' field."""
|
|
result = {"id": "id_field_value"}
|
|
custom_id = processor._get_custom_id(result, "unknown")
|
|
assert custom_id == "id_field_value"
|
|
|
|
|
|
class TestResponseExtraction:
|
|
"""Test _extract_response method for different providers."""
|
|
|
|
@pytest.fixture
|
|
def processor(self):
|
|
"""Create a processor instance."""
|
|
http_client = MagicMock(spec=httpx.AsyncClient)
|
|
return BatchResultProcessor(http_client)
|
|
|
|
def test_anthropic_response_extraction(self, processor):
|
|
"""Extract response from Anthropic batch result."""
|
|
result = {
|
|
"custom_id": "req_1",
|
|
"result": {
|
|
"type": "message",
|
|
"message": {
|
|
"content": [{"type": "text", "text": "Hello"}],
|
|
"stop_reason": "end_turn",
|
|
},
|
|
},
|
|
}
|
|
response = processor._extract_response(result, "anthropic")
|
|
assert response is not None
|
|
assert response["content"] == [{"type": "text", "text": "Hello"}]
|
|
|
|
def test_openai_response_extraction(self, processor):
|
|
"""Extract response from OpenAI batch result."""
|
|
result = {
|
|
"custom_id": "req_2",
|
|
"response": {
|
|
"status_code": 200,
|
|
"body": {
|
|
"choices": [{"message": {"content": "Hello"}}],
|
|
},
|
|
},
|
|
}
|
|
response = processor._extract_response(result, "openai")
|
|
assert response is not None
|
|
assert response["choices"][0]["message"]["content"] == "Hello"
|
|
|
|
def test_google_response_extraction(self, processor):
|
|
"""Extract response from Google batch result."""
|
|
result = {
|
|
"metadata": {"key": "req_3"},
|
|
"response": {
|
|
"candidates": [{"content": {"parts": [{"text": "Hello"}]}}],
|
|
},
|
|
}
|
|
response = processor._extract_response(result, "google")
|
|
assert response is not None
|
|
assert response["candidates"][0]["content"]["parts"][0]["text"] == "Hello"
|
|
|
|
def test_anthropic_missing_result(self, processor):
|
|
"""Handle missing result in Anthropic format."""
|
|
result = {"custom_id": "req_4"}
|
|
response = processor._extract_response(result, "anthropic")
|
|
assert response is None
|
|
|
|
def test_openai_missing_body(self, processor):
|
|
"""Handle missing body in OpenAI format."""
|
|
result = {"custom_id": "req_5", "response": {"status_code": 500}}
|
|
response = processor._extract_response(result, "openai")
|
|
assert response is None
|
|
|
|
def test_invalid_response_type(self, processor):
|
|
"""Handle non-dict response."""
|
|
result = {"custom_id": "req_6", "result": {"message": "not_a_dict"}}
|
|
response = processor._extract_response(result, "anthropic")
|
|
assert response is None
|
|
|
|
|
|
class TestCCRToolCallDetectionInBatch:
|
|
"""Test CCR tool call detection within batch results."""
|
|
|
|
@pytest.fixture
|
|
def processor(self):
|
|
"""Create a processor instance."""
|
|
http_client = MagicMock(spec=httpx.AsyncClient)
|
|
return BatchResultProcessor(http_client)
|
|
|
|
def test_detect_anthropic_ccr_in_batch(self, processor):
|
|
"""Detect CCR tool call in Anthropic batch result."""
|
|
response = {
|
|
"content": [
|
|
{"type": "text", "text": "Let me retrieve that data."},
|
|
{
|
|
"type": "tool_use",
|
|
"id": "tool_123",
|
|
"name": CCR_TOOL_NAME,
|
|
"input": {"hash": "abc123"},
|
|
},
|
|
]
|
|
}
|
|
assert processor.ccr_handler.has_ccr_tool_calls(response, "anthropic")
|
|
|
|
def test_detect_openai_ccr_in_batch(self, processor):
|
|
"""Detect CCR tool call in OpenAI batch result."""
|
|
response = {
|
|
"choices": [
|
|
{
|
|
"message": {
|
|
"role": "assistant",
|
|
"content": "Retrieving data...",
|
|
"tool_calls": [
|
|
{
|
|
"id": "call_abc",
|
|
"type": "function",
|
|
"function": {
|
|
"name": CCR_TOOL_NAME,
|
|
"arguments": '{"hash": "def456"}',
|
|
},
|
|
}
|
|
],
|
|
}
|
|
}
|
|
]
|
|
}
|
|
assert processor.ccr_handler.has_ccr_tool_calls(response, "openai")
|
|
|
|
def test_detect_google_ccr_in_batch(self, processor):
|
|
"""Detect CCR tool call in Google batch result."""
|
|
response = {
|
|
"candidates": [
|
|
{
|
|
"content": {
|
|
"parts": [
|
|
{"text": "Retrieving data..."},
|
|
{
|
|
"functionCall": {
|
|
"name": CCR_TOOL_NAME,
|
|
"args": {"hash": "ghi789"},
|
|
}
|
|
},
|
|
]
|
|
}
|
|
}
|
|
]
|
|
}
|
|
assert processor.ccr_handler.has_ccr_tool_calls(response, "google")
|
|
|
|
def test_no_ccr_in_text_only_response(self, processor):
|
|
"""No false positive for text-only response."""
|
|
response = {"content": [{"type": "text", "text": "Just a text response."}]}
|
|
assert not processor.ccr_handler.has_ccr_tool_calls(response, "anthropic")
|
|
|
|
def test_no_ccr_for_other_tools(self, processor):
|
|
"""No false positive for other tool calls."""
|
|
response = {
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "tool_xyz",
|
|
"name": "read_file",
|
|
"input": {"path": "/etc/config"},
|
|
}
|
|
]
|
|
}
|
|
assert not processor.ccr_handler.has_ccr_tool_calls(response, "anthropic")
|
|
|
|
|
|
class TestResultUpdate:
|
|
"""Test _update_result method for different providers."""
|
|
|
|
@pytest.fixture
|
|
def processor(self):
|
|
"""Create a processor instance."""
|
|
http_client = MagicMock(spec=httpx.AsyncClient)
|
|
return BatchResultProcessor(http_client)
|
|
|
|
def test_update_anthropic_result(self, processor):
|
|
"""Update Anthropic batch result with final response."""
|
|
original = {
|
|
"custom_id": "req_1",
|
|
"result": {
|
|
"type": "tool_use",
|
|
"message": {"content": [{"type": "tool_use", "name": CCR_TOOL_NAME}]},
|
|
},
|
|
}
|
|
final_response = {
|
|
"content": [{"type": "text", "text": "Final answer"}],
|
|
"stop_reason": "end_turn",
|
|
}
|
|
|
|
updated = processor._update_result(original, final_response, "anthropic")
|
|
|
|
assert updated["result"]["message"] == final_response
|
|
assert updated["result"]["type"] == "succeeded"
|
|
assert updated["custom_id"] == "req_1"
|
|
|
|
def test_update_openai_result(self, processor):
|
|
"""Update OpenAI batch result with final response."""
|
|
original = {
|
|
"custom_id": "req_2",
|
|
"response": {
|
|
"status_code": 200,
|
|
"body": {
|
|
"choices": [
|
|
{"message": {"tool_calls": [{"function": {"name": CCR_TOOL_NAME}}]}}
|
|
]
|
|
},
|
|
},
|
|
}
|
|
final_response = {"choices": [{"message": {"content": "Final answer"}}]}
|
|
|
|
updated = processor._update_result(original, final_response, "openai")
|
|
|
|
assert updated["response"]["body"] == final_response
|
|
assert updated["custom_id"] == "req_2"
|
|
|
|
def test_update_google_result(self, processor):
|
|
"""Update Google batch result with final response."""
|
|
original = {
|
|
"metadata": {"key": "req_3"},
|
|
"response": {
|
|
"candidates": [{"content": {"parts": [{"functionCall": {"name": CCR_TOOL_NAME}}]}}]
|
|
},
|
|
}
|
|
final_response = {"candidates": [{"content": {"parts": [{"text": "Final answer"}]}}]}
|
|
|
|
updated = processor._update_result(original, final_response, "google")
|
|
|
|
assert updated["response"] == final_response
|
|
|
|
def test_update_creates_missing_containers(self, processor):
|
|
"""Update creates missing result/response containers."""
|
|
original_anthropic = {"custom_id": "req_1"}
|
|
original_openai = {"custom_id": "req_2"}
|
|
final = {"content": "test"}
|
|
|
|
updated_anthropic = processor._update_result(original_anthropic, final, "anthropic")
|
|
updated_openai = processor._update_result(original_openai, final, "openai")
|
|
|
|
assert "result" in updated_anthropic
|
|
assert "response" in updated_openai
|
|
|
|
|
|
class TestMessagesToGoogleContents:
|
|
"""Test _messages_to_google_contents conversion."""
|
|
|
|
@pytest.fixture
|
|
def processor(self):
|
|
"""Create a processor instance."""
|
|
http_client = MagicMock(spec=httpx.AsyncClient)
|
|
return BatchResultProcessor(http_client)
|
|
|
|
def test_convert_simple_text_message(self, processor):
|
|
"""Convert simple text messages."""
|
|
messages = [
|
|
{"role": "user", "content": "Hello"},
|
|
{"role": "assistant", "content": "Hi there"},
|
|
]
|
|
|
|
contents = processor._messages_to_google_contents(messages)
|
|
|
|
assert len(contents) == 2
|
|
assert contents[0]["role"] == "user"
|
|
assert contents[0]["parts"] == [{"text": "Hello"}]
|
|
assert contents[1]["role"] == "model"
|
|
assert contents[1]["parts"] == [{"text": "Hi there"}]
|
|
|
|
def test_skip_system_messages(self, processor):
|
|
"""System messages are skipped (handled separately in Google)."""
|
|
messages = [
|
|
{"role": "system", "content": "You are helpful."},
|
|
{"role": "user", "content": "Hello"},
|
|
]
|
|
|
|
contents = processor._messages_to_google_contents(messages)
|
|
|
|
assert len(contents) == 1
|
|
assert contents[0]["role"] == "user"
|
|
|
|
def test_convert_tool_result_content(self, processor):
|
|
"""Convert structured content with tool results."""
|
|
messages = [
|
|
{
|
|
"role": "user",
|
|
"content": [
|
|
{"type": "tool_result", "tool_use_id": "tool_123", "content": "Result data"}
|
|
],
|
|
}
|
|
]
|
|
|
|
contents = processor._messages_to_google_contents(messages)
|
|
|
|
assert len(contents) == 1
|
|
assert contents[0]["parts"][0]["functionResponse"]["response"]["content"] == "Result data"
|
|
|
|
def test_convert_tool_use_content(self, processor):
|
|
"""Convert content with tool_use blocks."""
|
|
messages = [
|
|
{
|
|
"role": "assistant",
|
|
"content": [{"type": "tool_use", "name": "read_file", "input": {"path": "/test"}}],
|
|
}
|
|
]
|
|
|
|
contents = processor._messages_to_google_contents(messages)
|
|
|
|
assert len(contents) == 1
|
|
assert contents[0]["role"] == "model"
|
|
assert contents[0]["parts"][0]["functionCall"]["name"] == "read_file"
|
|
|
|
def test_preserve_google_format_messages(self, processor):
|
|
"""Messages already in Google format are preserved."""
|
|
messages = [{"role": "model", "parts": [{"text": "Already Google format"}]}]
|
|
|
|
contents = processor._messages_to_google_contents(messages)
|
|
|
|
assert len(contents) == 1
|
|
assert contents[0]["parts"] == [{"text": "Already Google format"}]
|
|
|
|
|
|
class TestProcessResults:
|
|
"""Test process_results method."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def reset_store(self):
|
|
"""Reset global store before each test."""
|
|
reset_batch_context_store()
|
|
yield
|
|
reset_batch_context_store()
|
|
|
|
@pytest.fixture
|
|
def processor(self):
|
|
"""Create a processor instance."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
return BatchResultProcessor(http_client)
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_disabled_processor_passthrough(self, processor):
|
|
"""Disabled processor passes through results unchanged."""
|
|
processor.config.enabled = False
|
|
|
|
results = [{"custom_id": "req_1", "result": {"message": {"content": "test"}}}]
|
|
|
|
processed = await processor.process_results("batch_123", results, "anthropic")
|
|
|
|
assert len(processed) == 1
|
|
assert processed[0].custom_id == "req_1"
|
|
assert processed[0].was_processed is False
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_missing_batch_context_passthrough(self, processor):
|
|
"""Missing batch context passes through results unchanged."""
|
|
results = [{"custom_id": "req_1", "result": {"message": {"content": "test"}}}]
|
|
|
|
processed = await processor.process_results("nonexistent_batch", results, "anthropic")
|
|
|
|
assert len(processed) == 1
|
|
assert processed[0].was_processed is False
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_no_ccr_tool_calls_passthrough(self):
|
|
"""Results without CCR tool calls pass through."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
# Set up batch context
|
|
store = BatchContextStore()
|
|
context = BatchContext(batch_id="batch_123", provider="anthropic")
|
|
context.add_request(
|
|
BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[{"role": "user", "content": "Hi"}],
|
|
model="claude-3-opus",
|
|
)
|
|
)
|
|
await store.store(context)
|
|
|
|
with patch(
|
|
"headroom.ccr.batch_processor.get_batch_context_store",
|
|
return_value=store,
|
|
):
|
|
results = [
|
|
{
|
|
"custom_id": "req_1",
|
|
"result": {
|
|
"message": {
|
|
"content": [{"type": "text", "text": "Hello!"}],
|
|
"stop_reason": "end_turn",
|
|
}
|
|
},
|
|
}
|
|
]
|
|
|
|
processed = await processor.process_results("batch_123", results, "anthropic")
|
|
|
|
assert len(processed) == 1
|
|
assert processed[0].was_processed is False
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_missing_request_context_passthrough(self):
|
|
"""Missing request context for custom_id passes through."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
# Set up batch context without matching request
|
|
store = BatchContextStore()
|
|
context = BatchContext(batch_id="batch_123", provider="anthropic")
|
|
# Don't add any requests
|
|
await store.store(context)
|
|
|
|
with patch(
|
|
"headroom.ccr.batch_processor.get_batch_context_store",
|
|
return_value=store,
|
|
):
|
|
results = [
|
|
{
|
|
"custom_id": "unknown_req",
|
|
"result": {"message": {"content": [{"type": "text", "text": "Test"}]}},
|
|
}
|
|
]
|
|
|
|
processed = await processor.process_results("batch_123", results, "anthropic")
|
|
|
|
assert len(processed) == 1
|
|
assert processed[0].custom_id == "unknown_req"
|
|
assert processed[0].was_processed is False
|
|
|
|
|
|
class TestContinuationCalls:
|
|
"""Test continuation API calls for different providers."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def reset_store(self):
|
|
"""Reset global store before each test."""
|
|
reset_batch_context_store()
|
|
yield
|
|
reset_batch_context_store()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_anthropic_continuation_call(self):
|
|
"""Test Anthropic continuation call format."""
|
|
mock_response = MagicMock()
|
|
mock_response.json.return_value = {"content": [{"type": "text", "text": "Final answer"}]}
|
|
mock_response.raise_for_status = MagicMock()
|
|
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
http_client.post.return_value = mock_response
|
|
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
request_context = BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[{"role": "user", "content": "Hello"}],
|
|
model="claude-3-opus-20240229",
|
|
extras={"max_tokens": 1000},
|
|
)
|
|
batch_context = BatchContext(
|
|
batch_id="batch_123",
|
|
provider="anthropic",
|
|
api_key="test_api_key",
|
|
)
|
|
|
|
messages = [
|
|
{"role": "user", "content": "Hello"},
|
|
{"role": "assistant", "content": "Tool result here"},
|
|
]
|
|
|
|
await processor._anthropic_continuation(messages, None, request_context, batch_context)
|
|
|
|
# Verify API call
|
|
http_client.post.assert_called_once()
|
|
call_args = http_client.post.call_args
|
|
|
|
assert "api.anthropic.com" in call_args.args[0]
|
|
assert call_args.kwargs["headers"]["x-api-key"] == "test_api_key"
|
|
assert call_args.kwargs["json"]["model"] == "claude-3-opus-20240229"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_openai_continuation_call(self):
|
|
"""Test OpenAI continuation call format."""
|
|
mock_response = MagicMock()
|
|
mock_response.json.return_value = {"choices": [{"message": {"content": "Final answer"}}]}
|
|
mock_response.raise_for_status = MagicMock()
|
|
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
http_client.post.return_value = mock_response
|
|
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
request_context = BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[{"role": "user", "content": "Hello"}],
|
|
model="gpt-4",
|
|
)
|
|
batch_context = BatchContext(
|
|
batch_id="batch_123",
|
|
provider="openai",
|
|
api_key="sk-test123",
|
|
)
|
|
|
|
await processor._openai_continuation(
|
|
[{"role": "user", "content": "Hello"}],
|
|
None,
|
|
request_context,
|
|
batch_context,
|
|
)
|
|
|
|
# Verify API call
|
|
http_client.post.assert_called_once()
|
|
call_args = http_client.post.call_args
|
|
|
|
assert "api.openai.com" in call_args.args[0]
|
|
assert "Bearer sk-test123" in call_args.kwargs["headers"]["Authorization"]
|
|
assert call_args.kwargs["json"]["model"] == "gpt-4"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_anthropic_continuation_uses_configured_api_base_url(self):
|
|
"""The continuation goes to the target recorded at submission, not api.anthropic.com."""
|
|
mock_response = MagicMock()
|
|
mock_response.json.return_value = {"content": [{"type": "text", "text": "Final answer"}]}
|
|
mock_response.raise_for_status = MagicMock()
|
|
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
http_client.post.return_value = mock_response
|
|
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
request_context = BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[{"role": "user", "content": "Hello"}],
|
|
model="claude-3-opus-20240229",
|
|
extras={"max_tokens": 1000},
|
|
)
|
|
batch_context = BatchContext(
|
|
batch_id="batch_123",
|
|
provider="anthropic",
|
|
api_key="gateway-credential",
|
|
api_base_url="https://llm-gateway.example.test",
|
|
)
|
|
|
|
await processor._anthropic_continuation(
|
|
[{"role": "user", "content": "Hello"}], None, request_context, batch_context
|
|
)
|
|
|
|
call_args = http_client.post.call_args
|
|
assert call_args.args[0] == "https://llm-gateway.example.test/v1/messages"
|
|
assert call_args.kwargs["headers"]["x-api-key"] == "gateway-credential"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_openai_continuation_uses_configured_api_base_url(self):
|
|
"""A trailing slash on the recorded target does not double the separator."""
|
|
mock_response = MagicMock()
|
|
mock_response.json.return_value = {"choices": [{"message": {"content": "Final answer"}}]}
|
|
mock_response.raise_for_status = MagicMock()
|
|
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
http_client.post.return_value = mock_response
|
|
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
request_context = BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[{"role": "user", "content": "Hello"}],
|
|
model="gpt-4",
|
|
)
|
|
batch_context = BatchContext(
|
|
batch_id="batch_123",
|
|
provider="openai",
|
|
api_key="sk-test123",
|
|
api_base_url="https://llm-gateway.example.test/",
|
|
)
|
|
|
|
await processor._openai_continuation(
|
|
[{"role": "user", "content": "Hello"}], None, request_context, batch_context
|
|
)
|
|
|
|
call_args = http_client.post.call_args
|
|
assert call_args.args[0] == "https://llm-gateway.example.test/v1/chat/completions"
|
|
assert call_args.kwargs["headers"]["Authorization"] == "Bearer sk-test123"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_google_continuation_uses_configured_api_base_url_and_header_key(self):
|
|
"""Gemini: the recorded target is used and the key travels in a header, not the URL."""
|
|
mock_response = MagicMock()
|
|
mock_response.json.return_value = {
|
|
"candidates": [{"content": {"role": "model", "parts": [{"text": "Final answer"}]}}]
|
|
}
|
|
mock_response.raise_for_status = MagicMock()
|
|
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
http_client.post.return_value = mock_response
|
|
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
request_context = BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[{"role": "user", "content": "Hello"}],
|
|
model="gemini-2.5-flash",
|
|
)
|
|
batch_context = BatchContext(
|
|
batch_id="batch_123",
|
|
provider="google",
|
|
api_key="gemini-gateway-key",
|
|
api_base_url="https://gemini-gateway.example.test",
|
|
)
|
|
|
|
await processor._google_continuation(
|
|
[{"role": "user", "content": "Hello"}], None, request_context, batch_context
|
|
)
|
|
|
|
call_args = http_client.post.call_args
|
|
assert call_args.args[0] == (
|
|
"https://gemini-gateway.example.test/v1beta/models/gemini-2.5-flash:generateContent"
|
|
)
|
|
assert "gemini-gateway-key" not in call_args.args[0]
|
|
assert call_args.kwargs["headers"]["x-goog-api-key"] == "gemini-gateway-key"
|
|
|
|
@pytest.mark.asyncio
|
|
@pytest.mark.parametrize("client_tools", ["functions", "siblings", "none"])
|
|
async def test_google_batch_create_to_results_retains_injected_native_tools(
|
|
self, monkeypatch, tmp_path, client_tools
|
|
):
|
|
from starlette.requests import Request
|
|
|
|
from headroom.providers.openai import OpenAIProvider
|
|
from headroom.proxy.server import HeadroomProxy, ProxyConfig
|
|
from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
|
|
from headroom.transforms.pipeline import TransformPipeline
|
|
|
|
monkeypatch.setenv("HEADROOM_WORKSPACE_DIR", str(tmp_path))
|
|
original_records = [
|
|
f"Record {i}: amount={i * 13}, owner=account-{i}, unique source information "
|
|
"with a detailed transaction description."
|
|
for i in range(200)
|
|
]
|
|
original = json.dumps(original_records, indent=2)
|
|
store = CompressionStore(enable_feedback=False)
|
|
previous_store = compression_store_module._request_ccr_store.get()
|
|
set_request_compression_store(store)
|
|
monkeypatch.setattr(batch_store_module, "_batch_context_store", BatchContextStore())
|
|
tools = [] if client_tools == "none" else [{"googleSearch": {}}, {"codeExecution": {}}]
|
|
if client_tools == "functions":
|
|
tools.insert(
|
|
0,
|
|
{
|
|
"functionDeclarations": [
|
|
{
|
|
"name": "lookup",
|
|
"description": "Look up one record.",
|
|
"parameters": {"type": "OBJECT"},
|
|
}
|
|
]
|
|
},
|
|
)
|
|
forwarded = None
|
|
|
|
def admit_wire_request(request):
|
|
nonlocal forwarded
|
|
body = json.loads(request.content)
|
|
if request.url.path.endswith(":batchGenerateContent"):
|
|
forwarded = body["batch"]["input_config"]["requests"]["requests"][0]["request"]
|
|
return httpx.Response(200, json={"name": "batches/native-tools"})
|
|
recovered = [
|
|
part["functionResponse"]["response"].get("original_content")
|
|
for content in body["contents"]
|
|
for part in content["parts"]
|
|
if "functionResponse" in part
|
|
]
|
|
declarations = [
|
|
declaration
|
|
for tool in body.get("tools", [])
|
|
for declaration in tool.get("functionDeclarations", [])
|
|
]
|
|
expected_names = (
|
|
{CCR_TOOL_NAME, "lookup"} if client_tools == "functions" else {CCR_TOOL_NAME}
|
|
)
|
|
admitted = (
|
|
{declaration.get("name") for declaration in declarations} == expected_names
|
|
and body.get("tools") == forwarded.get("tools")
|
|
and all(
|
|
all(declaration in declarations for declaration in tool["functionDeclarations"])
|
|
if "functionDeclarations" in tool
|
|
else tool in body["tools"]
|
|
for tool in tools
|
|
)
|
|
and any(
|
|
isinstance(value, str) and json.loads(value) == original_records
|
|
for value in recovered
|
|
)
|
|
)
|
|
return httpx.Response(
|
|
200 if admitted else 400,
|
|
json={
|
|
"candidates": [
|
|
{"content": {"role": "model", "parts": [{"text": "All records recovered"}]}}
|
|
]
|
|
}
|
|
if admitted
|
|
else {"error": {"message": "Missing effective native tools or original records"}},
|
|
)
|
|
|
|
proxy = HeadroomProxy(
|
|
ProxyConfig(
|
|
optimize=True,
|
|
cache_enabled=False,
|
|
rate_limit_enabled=False,
|
|
cost_tracking_enabled=False,
|
|
log_requests=False,
|
|
retry_enabled=False,
|
|
image_optimize=False,
|
|
)
|
|
)
|
|
proxy.openai_pipeline = TransformPipeline(
|
|
provider=OpenAIProvider(),
|
|
transforms=[
|
|
ContentRouter(
|
|
ContentRouterConfig(
|
|
skip_user_messages=False,
|
|
protect_analysis_context=False,
|
|
enable_kompress=False,
|
|
smart_crusher_max_items_after_crush=12,
|
|
smart_crusher_with_compaction=False,
|
|
)
|
|
)
|
|
],
|
|
)
|
|
payload = json.dumps(
|
|
{
|
|
"batch": {
|
|
"input_config": {
|
|
"requests": {
|
|
"requests": [
|
|
{
|
|
"metadata": {"key": "request"},
|
|
"request": {
|
|
"contents": [
|
|
{"role": "user", "parts": [{"text": original}]}
|
|
],
|
|
"tools": tools,
|
|
},
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
).encode()
|
|
chunks = iter(
|
|
[
|
|
{"type": "http.request", "body": payload[:100], "more_body": True},
|
|
{"type": "http.request", "body": payload[100:], "more_body": False},
|
|
]
|
|
)
|
|
|
|
async def receive():
|
|
return next(chunks, {"type": "http.disconnect"})
|
|
|
|
request = Request(
|
|
{
|
|
"type": "http",
|
|
"method": "POST",
|
|
"path": "/v1beta/models/gemini-2.5-flash:batchGenerateContent",
|
|
"query_string": b"",
|
|
"headers": [(b"content-type", b"application/json")],
|
|
"scheme": "http",
|
|
"server": ("127.0.0.1", 80),
|
|
},
|
|
receive,
|
|
)
|
|
try:
|
|
async with httpx.AsyncClient(
|
|
transport=httpx.MockTransport(admit_wire_request)
|
|
) as client:
|
|
proxy.http_client = client
|
|
created = await proxy.handle_google_batch_create(request, "gemini-2.5-flash")
|
|
assert created.status_code == 200
|
|
hash_key = re.search(
|
|
r"(?:<<ccr:|Retrieve (?:more|original): hash=)([a-f0-9]{12,24})",
|
|
json.dumps(forwarded),
|
|
).group(1)
|
|
response = {
|
|
"candidates": [
|
|
{
|
|
"content": {
|
|
"role": "model",
|
|
"parts": [
|
|
{
|
|
"functionCall": {
|
|
"name": CCR_TOOL_NAME,
|
|
"args": {"hash": hash_key},
|
|
}
|
|
}
|
|
],
|
|
}
|
|
}
|
|
]
|
|
}
|
|
processed = await BatchResultProcessor(client).process_results(
|
|
"batches/native-tools",
|
|
[{"metadata": {"key": "request"}, "response": response}],
|
|
"google",
|
|
)
|
|
assert processed[0].result["response"]["candidates"][0]["content"]["parts"] == [
|
|
{"text": "All records recovered"}
|
|
]
|
|
finally:
|
|
set_request_compression_store(previous_store)
|
|
proxy._compression_executor.shutdown(wait=True)
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_continuation_with_tools(self):
|
|
"""Test continuation includes tools when present."""
|
|
mock_response = MagicMock()
|
|
mock_response.json.return_value = {"content": [{"type": "text", "text": "Done"}]}
|
|
mock_response.raise_for_status = MagicMock()
|
|
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
http_client.post.return_value = mock_response
|
|
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
request_context = BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[],
|
|
model="claude-3-opus",
|
|
extras={"max_tokens": 1000},
|
|
)
|
|
batch_context = BatchContext(
|
|
batch_id="batch_123",
|
|
provider="anthropic",
|
|
api_key="test_key",
|
|
)
|
|
|
|
tools = [
|
|
{"name": "read_file", "input_schema": {"type": "object"}},
|
|
{"name": CCR_TOOL_NAME, "input_schema": {"type": "object"}},
|
|
]
|
|
|
|
await processor._anthropic_continuation(
|
|
[{"role": "user", "content": "test"}],
|
|
tools,
|
|
request_context,
|
|
batch_context,
|
|
)
|
|
|
|
call_args = http_client.post.call_args
|
|
assert "tools" in call_args.kwargs["json"]
|
|
assert len(call_args.kwargs["json"]["tools"]) == 2
|
|
sent_names = [tool.get("name") for tool in call_args.kwargs["json"]["tools"]]
|
|
assert sent_names == sorted(sent_names)
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_make_continuation_call_unknown_provider(self):
|
|
"""Test continuation call raises for unknown provider."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
request_context = BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[],
|
|
model="unknown-model",
|
|
)
|
|
batch_context = BatchContext(
|
|
batch_id="batch_123",
|
|
provider="unknown",
|
|
)
|
|
|
|
with pytest.raises(ValueError, match="Unknown provider"):
|
|
await processor._make_continuation_call(
|
|
[],
|
|
None,
|
|
request_context,
|
|
batch_context,
|
|
"unknown",
|
|
)
|
|
|
|
|
|
class TestProcessSingleResult:
|
|
"""Test _process_single_result method."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def reset_store(self):
|
|
"""Reset global store before each test."""
|
|
reset_batch_context_store()
|
|
yield
|
|
reset_batch_context_store()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_process_single_result_with_ccr(self):
|
|
"""Process a single result containing CCR tool call."""
|
|
# Mock the CCR handler to simulate CCR processing
|
|
mock_response = MagicMock()
|
|
mock_response.json.return_value = {
|
|
"content": [{"type": "text", "text": "Final processed answer"}]
|
|
}
|
|
mock_response.raise_for_status = MagicMock()
|
|
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
http_client.post.return_value = mock_response
|
|
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
# Mock the CCR handler's handle_response
|
|
processor.ccr_handler.handle_response = AsyncMock(
|
|
return_value={"content": [{"type": "text", "text": "Final processed answer"}]}
|
|
)
|
|
|
|
original_result = {
|
|
"custom_id": "req_1",
|
|
"result": {
|
|
"message": {
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "tool_123",
|
|
"name": CCR_TOOL_NAME,
|
|
"input": {"hash": "abc123"},
|
|
}
|
|
]
|
|
}
|
|
},
|
|
}
|
|
response = original_result["result"]["message"]
|
|
|
|
request_context = BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[{"role": "user", "content": "Get data"}],
|
|
tools=[{"name": CCR_TOOL_NAME}],
|
|
model="claude-3-opus",
|
|
)
|
|
batch_context = BatchContext(
|
|
batch_id="batch_123",
|
|
provider="anthropic",
|
|
api_key="test_key",
|
|
)
|
|
|
|
processed = await processor._process_single_result(
|
|
original_result,
|
|
response,
|
|
request_context,
|
|
batch_context,
|
|
"anthropic",
|
|
)
|
|
|
|
assert processed.custom_id == "req_1"
|
|
assert processed.was_processed is True
|
|
|
|
|
|
class TestConvenienceFunction:
|
|
"""Test process_batch_results convenience function."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def reset_store(self):
|
|
"""Reset global store before each test."""
|
|
reset_batch_context_store()
|
|
yield
|
|
reset_batch_context_store()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_process_batch_results_function(self):
|
|
"""Test the convenience function."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
|
|
results = [
|
|
{
|
|
"custom_id": "req_1",
|
|
"result": {"message": {"content": [{"type": "text", "text": "Hello"}]}},
|
|
}
|
|
]
|
|
|
|
processed = await process_batch_results(
|
|
"batch_123",
|
|
results,
|
|
"anthropic",
|
|
http_client,
|
|
)
|
|
|
|
assert len(processed) == 1
|
|
assert processed[0].custom_id == "req_1"
|
|
|
|
|
|
class TestErrorHandling:
|
|
"""Test error handling scenarios."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def reset_store(self):
|
|
"""Reset global store before each test."""
|
|
reset_batch_context_store()
|
|
yield
|
|
reset_batch_context_store()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_continuation_api_error(self):
|
|
"""Handle API errors during continuation."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
http_client.post.side_effect = httpx.HTTPStatusError(
|
|
"Internal Server Error",
|
|
request=MagicMock(),
|
|
response=MagicMock(status_code=500),
|
|
)
|
|
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
request_context = BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[],
|
|
model="claude-3-opus",
|
|
extras={"max_tokens": 1000},
|
|
)
|
|
batch_context = BatchContext(
|
|
batch_id="batch_123",
|
|
provider="anthropic",
|
|
api_key="test_key",
|
|
)
|
|
|
|
with pytest.raises(httpx.HTTPStatusError):
|
|
await processor._anthropic_continuation(
|
|
[{"role": "user", "content": "test"}],
|
|
None,
|
|
request_context,
|
|
batch_context,
|
|
)
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_processing_error_captured(self):
|
|
"""Processing errors are captured in result."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
# Set up batch context
|
|
store = BatchContextStore()
|
|
context = BatchContext(batch_id="batch_123", provider="anthropic")
|
|
context.add_request(
|
|
BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[{"role": "user", "content": "Hi"}],
|
|
model="claude-3-opus",
|
|
)
|
|
)
|
|
await store.store(context)
|
|
|
|
# Mock the handler to raise an error
|
|
processor.ccr_handler.has_ccr_tool_calls = MagicMock(return_value=True)
|
|
processor._process_single_result = AsyncMock(side_effect=Exception("Processing failed"))
|
|
|
|
with patch(
|
|
"headroom.ccr.batch_processor.get_batch_context_store",
|
|
return_value=store,
|
|
):
|
|
results = [
|
|
{
|
|
"custom_id": "req_1",
|
|
"result": {
|
|
"message": {
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "tool_1",
|
|
"name": CCR_TOOL_NAME,
|
|
"input": {"hash": "abc"},
|
|
}
|
|
]
|
|
}
|
|
},
|
|
}
|
|
]
|
|
|
|
processed = await processor.process_results("batch_123", results, "anthropic")
|
|
|
|
assert len(processed) == 1
|
|
assert processed[0].error == "Processing failed"
|
|
assert processed[0].was_processed is False
|
|
|
|
|
|
class TestMultipleResultsProcessing:
|
|
"""Test processing multiple results in a batch."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def reset_store(self):
|
|
"""Reset global store before each test."""
|
|
reset_batch_context_store()
|
|
yield
|
|
reset_batch_context_store()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_mixed_results_processing(self):
|
|
"""Process batch with mix of CCR and non-CCR results."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
# Set up batch context with multiple requests
|
|
store = BatchContextStore()
|
|
context = BatchContext(batch_id="batch_123", provider="anthropic")
|
|
context.add_request(
|
|
BatchRequestContext(
|
|
custom_id="req_1",
|
|
messages=[{"role": "user", "content": "Request 1"}],
|
|
model="claude-3-opus",
|
|
)
|
|
)
|
|
context.add_request(
|
|
BatchRequestContext(
|
|
custom_id="req_2",
|
|
messages=[{"role": "user", "content": "Request 2"}],
|
|
model="claude-3-opus",
|
|
)
|
|
)
|
|
context.add_request(
|
|
BatchRequestContext(
|
|
custom_id="req_3",
|
|
messages=[{"role": "user", "content": "Request 3"}],
|
|
model="claude-3-opus",
|
|
)
|
|
)
|
|
await store.store(context)
|
|
|
|
with patch(
|
|
"headroom.ccr.batch_processor.get_batch_context_store",
|
|
return_value=store,
|
|
):
|
|
results = [
|
|
# Non-CCR result
|
|
{
|
|
"custom_id": "req_1",
|
|
"result": {
|
|
"message": {"content": [{"type": "text", "text": "Simple response"}]}
|
|
},
|
|
},
|
|
# Non-CCR result
|
|
{
|
|
"custom_id": "req_2",
|
|
"result": {
|
|
"message": {"content": [{"type": "text", "text": "Another response"}]}
|
|
},
|
|
},
|
|
# Non-CCR result (other tool)
|
|
{
|
|
"custom_id": "req_3",
|
|
"result": {
|
|
"message": {
|
|
"content": [
|
|
{
|
|
"type": "tool_use",
|
|
"id": "tool_1",
|
|
"name": "read_file",
|
|
"input": {"path": "/test"},
|
|
}
|
|
]
|
|
}
|
|
},
|
|
},
|
|
]
|
|
|
|
processed = await processor.process_results("batch_123", results, "anthropic")
|
|
|
|
assert len(processed) == 3
|
|
# All should not be processed (no CCR tools)
|
|
assert all(not r.was_processed for r in processed)
|
|
assert processed[0].custom_id == "req_1"
|
|
assert processed[1].custom_id == "req_2"
|
|
assert processed[2].custom_id == "req_3"
|
|
|
|
|
|
class TestProviderSpecificFormats:
|
|
"""Test provider-specific batch result formats."""
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def reset_store(self):
|
|
"""Reset global store before each test."""
|
|
reset_batch_context_store()
|
|
yield
|
|
reset_batch_context_store()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_anthropic_batch_format(self):
|
|
"""Test full Anthropic batch result format handling."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
# Typical Anthropic batch result format
|
|
results = [
|
|
{
|
|
"custom_id": "my-request-1",
|
|
"result": {
|
|
"type": "succeeded",
|
|
"message": {
|
|
"id": "msg_123",
|
|
"type": "message",
|
|
"role": "assistant",
|
|
"content": [{"type": "text", "text": "Response text"}],
|
|
"model": "claude-3-opus-20240229",
|
|
"stop_reason": "end_turn",
|
|
"usage": {"input_tokens": 10, "output_tokens": 20},
|
|
},
|
|
},
|
|
}
|
|
]
|
|
|
|
processed = await processor.process_results("batch_1", results, "anthropic")
|
|
|
|
assert processed[0].custom_id == "my-request-1"
|
|
assert processed[0].result["result"]["message"]["content"][0]["text"] == "Response text"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_openai_batch_format(self):
|
|
"""Test full OpenAI batch result format handling."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
# Typical OpenAI batch result format
|
|
results = [
|
|
{
|
|
"id": "batch_req_123",
|
|
"custom_id": "request-1",
|
|
"response": {
|
|
"status_code": 200,
|
|
"request_id": "req_abc",
|
|
"body": {
|
|
"id": "chatcmpl-123",
|
|
"object": "chat.completion",
|
|
"created": 1234567890,
|
|
"model": "gpt-4",
|
|
"choices": [
|
|
{
|
|
"index": 0,
|
|
"message": {
|
|
"role": "assistant",
|
|
"content": "OpenAI response",
|
|
},
|
|
"finish_reason": "stop",
|
|
}
|
|
],
|
|
"usage": {"prompt_tokens": 10, "completion_tokens": 20},
|
|
},
|
|
},
|
|
"error": None,
|
|
}
|
|
]
|
|
|
|
processed = await processor.process_results("batch_1", results, "openai")
|
|
|
|
assert processed[0].custom_id == "request-1"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_google_batch_format(self):
|
|
"""Test full Google batch result format handling."""
|
|
http_client = AsyncMock(spec=httpx.AsyncClient)
|
|
processor = BatchResultProcessor(http_client)
|
|
|
|
# Typical Google batch result format
|
|
results = [
|
|
{
|
|
"metadata": {
|
|
"key": "request-abc",
|
|
},
|
|
"response": {
|
|
"candidates": [
|
|
{
|
|
"content": {
|
|
"parts": [{"text": "Google response text"}],
|
|
"role": "model",
|
|
},
|
|
"finishReason": "STOP",
|
|
"safetyRatings": [],
|
|
}
|
|
],
|
|
"usageMetadata": {
|
|
"promptTokenCount": 10,
|
|
"candidatesTokenCount": 20,
|
|
},
|
|
},
|
|
}
|
|
]
|
|
|
|
processed = await processor.process_results("batch_1", results, "google")
|
|
|
|
assert processed[0].custom_id == "request-abc"
|