mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
fix/plugin-launcher-isolated-python
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
871bbde37a |
fix(proxy): reject Anthropic batch operations on Copilot (#3318)
## Description Anthropic batch routes followed the resolved Anthropic target to GitHub Copilot, where the batch surface isn't available, and returned an opaque upstream failure. This change returns a stable unsupported response before batch parsing or mutation while preserving normal Copilot-hosted messages and explicit non-Copilot Anthropic overrides. Closes #3278 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added one handler-level capability check for Copilot-resolved Anthropic batch routes. - Applied the response consistently to batch create, list, get, cancel, and results routes. - Added regressions for Copilot batch routes and explicit Anthropic overrides. Existing provider-registry coverage continues to verify normal message target resolution. ## Testing - [x] Unit tests pass (`uv run pytest tests/test_proxy_batch_integration.py tests/test_copilot_auth.py tests/test_provider_registry.py -q`) - [x] Linting passes (`uv run ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_batch_integration.py tests/test_copilot_auth.py`) - [ ] Type checking passes (`uv run mypy headroom`) - [x] New tests added for new functionality when applicable - [ ] Manual testing performed ### Test Output ```text uv run python -m pytest tests/test_proxy_batch_integration.py tests/test_copilot_auth.py tests/test_provider_registry.py -q 139 passed, 14 skipped, 1 warning in 7.78s uv run ruff check headroom/proxy/handlers/anthropic.py tests/test_proxy_batch_integration.py tests/test_copilot_auth.py All checks passed! git diff --check completed without whitespace errors ``` ## Real Behavior Proof - Environment: Windows, focused Headroom test client, no live provider credential - Exact command / steps: send Anthropic batch create and the remaining batch routes with the resolved target set to a Copilot host, then repeat with an explicit non-Copilot Anthropic target - Observed result: the five registered Anthropic batch routes return HTTP 501 with `{"type":"error","error":{"type":"api_error","message":"Anthropic batch operations are not supported for Copilot targets."}}` before body parsing or upstream access; with OpenAI forced to `https://api.githubcopilot.com`, the explicit `https://api.anthropic.com` override forwards to `https://api.anthropic.com/v1/messages/batches`. The focused run reported `139 passed, 14 skipped, 1 warning` and the route test reported `6 passed, 17 deselected, 1 warning`. - Not tested: live Copilot tenant ## Runtime Rollout Safety - Rollout-managed feature(s): none - Unsafe override required: no - Minimum rollout channel: standard release after focused CI checks pass - Stable/default behavior changed: Copilot-resolved Anthropic batch routes now return a deterministic unsupported response before upstream access - Kill switch / disable path: revert the handler guard or disable the surrounding batch route configuration - Qualification impact: ordinary Copilot messages and explicit Anthropic targets keep their existing routing paths - Rollback path: revert the handler guard and its focused regressions; no persisted-state migration is required ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes This follows the Anthropic message-target routing introduced in [PR #3258](https://github.com/headroomlabs-ai/headroom/pull/3258). Headroom generates changelog entries from conventional commits. The provider's hosted batch capability remains an external contract; local proof covers Headroom's deterministic guard and route behavior. |
||
|
|
95fd6d8688 |
Add multi-provider batch API support with CCR post-processing
This commit adds comprehensive batch API support for all three major LLM
providers (Anthropic, OpenAI, Google/Gemini) with integrated CCR
(Compress-Cache-Retrieve) functionality for asynchronous batch processing.
## Batch CCR Post-Processing Architecture
When batch APIs are used, responses are processed asynchronously. If the
model calls the CCR retrieval tool (`headroom_retrieve`) within a batch
response, the system now handles this automatically:
1. **Batch Submit**: Request context (messages, tools, model) is stored
in BatchContextStore keyed by batch_id
2. **Batch Results**: When results are retrieved, CCR tool calls are
detected in the responses
3. **Continuation**: For each CCR tool call, the system executes local
retrieval and makes a continuation API call to complete the response
4. **Result Update**: The batch result is updated with the complete
response, transparent to the caller
## New Components
- `headroom/ccr/batch_store.py`: TTL-based context storage for batch
requests, enabling CCR retrieval during result processing
- `headroom/ccr/batch_processor.py`: Processes batch results, detects
CCR tool calls across all provider formats, executes continuations
## Provider Support
### Anthropic
- POST /v1/messages/batches (create with compression)
- GET /v1/messages/batches (list)
- GET /v1/messages/batches/{id} (status)
- GET /v1/messages/batches/{id}/results (with CCR post-processing)
### OpenAI
- POST /v1/batches (create)
- GET /v1/batches (list)
- GET /v1/batches/{id} (status)
- Batch file upload/download support
### Google/Gemini
- Native API support: /v1beta/models/{model}:generateContent
- Batch API: /v1beta/models/{model}:batchGenerateContent
- Token counting: /v1beta/models/{model}:countTokens
- OpenAI-compatible endpoint support
## CCR Enhancements
- Added Google/Gemini format support to response_handler.py
- Extended tool_injection.py with multiple marker patterns for different
compressors (SmartCrusher, TextCompressor, LogCompressor, etc.)
- Added Google functionCall/functionResponse handling
## Proxy Server Updates
- Added Gemini native API handlers alongside OpenAI-compatible endpoints
- Integrated batch context storage on submission
- Added batch result processing with CCR continuation
- Rate limiting and metrics tracking for all providers
## Test Coverage
Added comprehensive integration tests (all skip gracefully without API keys):
- test_proxy_batch_integration.py: Anthropic and OpenAI batch APIs
- test_proxy_gemini_integration.py: Gemini via OpenAI-compatible endpoint
- test_proxy_gemini_native_integration.py: Gemini native API
- test_proxy_count_tokens_integration.py: Token counting endpoint
- test_proxy_openai_responses_integration.py: OpenAI responses API
- test_proxy_passthrough_integration.py: Passthrough endpoints
## Compression Results (Real API Testing)
- Token savings: 83-98% on tool result content
- CCR tool injection: Working across all providers
- Model behavior: OpenAI gpt-4o-mini successfully called headroom_retrieve
when presented with compressed data, proving end-to-end CCR functionality
|