Commit Graph
4 Commits
Author SHA1 Message Date
5f82080329 fix(ccr): send batch continuations to the upstream the batch was submitted to (#4848)
## Description

When a batch result needs a CCR continuation, `BatchResultProcessor`
made the follow-up call to hard-coded public hosts (`api.anthropic.com`,
`api.openai.com`, `generativelanguage.googleapis.com`). The upstream the
batch was actually submitted to is stored in
`BatchContext.api_base_url`, written by `_store_anthropic_batch_context`
(`ANTHROPIC_API_URL`) and `_store_google_batch_context`
(`GEMINI_API_URL`), but nothing ever read it. With a custom base URL or
gateway, the continuation sent that upstream's stored credential, the
conversation and the retrieved CCR originals to the public host.

Continuations now go to `BatchContext.api_base_url`. They fall back to
the public host only when no upstream was recorded. For Gemini the key
now goes in the `x-goog-api-key` header instead of `?key=` in the URL,
so it stays out of URLs, request logs and `HTTPStatusError` text.

Behaviour change for users: none for default setups (the recorded
upstream is the public host). Batches submitted through a custom
upstream now get their continuations sent there too.

Fix adapted from a proposal by HetCreep.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `headroom/ccr/batch_processor.py`: new `_continuation_base_url()`
(recorded upstream first, then the public default; trailing `/`
stripped). The Anthropic, OpenAI and Google continuations use it. Gemini
sends its key in `x-goog-api-key`.
- `tests/test_ccr_batch_processor.py`: one test per provider. Each
checks the request URL and the auth header when an upstream was
recorded, and for Gemini that the key is not in the URL.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# new tests against origin/main (c21b0813)
$ pytest tests/test_ccr_batch_processor.py
FAILED ...::TestContinuationCalls::test_anthropic_continuation_uses_configured_api_base_url
FAILED ...::TestContinuationCalls::test_openai_continuation_uses_configured_api_base_url
FAILED ...::TestContinuationCalls::test_google_continuation_uses_configured_api_base_url_and_header_key
========================= 3 failed, 45 passed in 9.67s =========================

# this branch: processor tests + every tests/*batch* file
$ pytest tests/test_ccr_batch_processor.py tests/*batch*
======================= 130 passed, 14 skipped in 39.39s =======================

$ ruff check / ruff format --check (0.17.3): All checks passed!
$ mypy headroom/ccr/batch_processor.py --ignore-missing-imports
Success: no issues found in 1 source file
```

## Real Behavior Proof

- Environment: macOS, Python 3.12. A real `BatchResultProcessor` with a
real `httpx.AsyncClient` on an `httpx.MockTransport` that records
requests (no network).
- Exact command / steps: for each provider,
`_make_continuation_call(...)` with `BatchContext(api_key="demo-key",
api_base_url="http://127.0.0.1:9/gateway")`. Print the recorded request
URL.
- Observed result:
```text
origin/main:
anthropic -> https://api.anthropic.com/v1/messages
openai    -> https://api.openai.com/v1/chat/completions
google    -> https://generativelanguage.googleapis.com/v1beta/models/m:generateContent  key-in-url=True
this branch:
anthropic -> http://127.0.0.1:9/gateway/v1/messages
openai    -> http://127.0.0.1:9/gateway/v1/chat/completions
google    -> http://127.0.0.1:9/gateway/v1beta/models/m:generateContent  key-in-url=False
```
- Real HTTP server check (this branch): a local `http.server` listener
as the "gateway", a real `httpx.AsyncClient` (no mock transport),
`api_base_url=http://127.0.0.1:<port>/gateway`, one continuation per
provider:
```text
anthropic call returned
openai    call returned
google    call returned
local gateway received: [('/gateway/v1/messages', {'x-api-key': 'demo-key'}),
  ('/gateway/v1/chat/completions', {'authorization': 'Bearer demo-key'}),
  ('/gateway/v1beta/models/m:generateContent', {'x-goog-api-key': 'demo-key'})]
```
- Not tested: a live batch against a real provider or gateway.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: see Description
- Kill switch / disable path: none needed
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this PR

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes

Out of scope here: Gemini batch *create* in
`headroom/proxy/handlers/batch.py` still moves `x-goog-api-key` into
`?key=` on the URL. It goes to the configured `GEMINI_API_URL`, so this
is about the key showing up in URLs, not about where it is sent. Worth a
separate follow-up.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-11-14 10:45:48 +05:30
Mansurshakh JaparovandJD Davis 875745889f fix(batch): preserve native Gemini continuation tools (#4211)
## Description

Closes #4182.

Google batch continuations wrapped native Tool objects in
`functionDeclarations` and could lose the retrieval declaration added
during compression. Keep the tools actually sent upstream, without
replacing the original conversation.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Pass native Google Tool objects directly to continuations.
- Store the effective forwarded tools, including injected retrieval and
sibling `googleSearch`/`codeExecution` tools.
- Cover batch creation through retrieval and the final answer, including
an initially empty tool list.

## Testing

- [ ] Unit tests pass (pytest) — focused suites only.
- [ ] Linting passes (ruff check .) — changed-file hooks only.
- [ ] Type checking passes (mypy headroom)
- [x] Manual testing performed

### Test Output

With this checkout's Rust extension loaded:

```text
pytest tests/test_ccr_batch_processor.py tests/test_ccr_batch_store.py tests/test_proxy_handlers_batch.py -q --no-cov
100 passed in 10.30s
```

The normal push gate also passed Rust workspace checks, rebuilt the
extension, and reported:

```text
181 passed, 5 warnings in 10.39s
ci-precheck PASSED — safe to push.
```

[Independent HTTP before/after
evidence](https://github.com/headroomlabs-ai/headroom/issues/4182#issuecomment-6149567901).

## Real Behavior Proof

- Environment: Linux x86_64, Python 3.13.14, current Rust/PyO3
extension, production batch handlers and retrieval stores; a loopback
HTTP provider, not live Google.
- Exact command / steps: run the focused command above. For the HTTP
probe, create a batch containing 200 string records and native tools,
process the injected retrieval call, and check the continuation's tools
and recovered records.
- Observed result: batch creation returns 200. The forwarded request
shrinks from 22,896 to 1,980 bytes, retains lookup, headroom_retrieve,
googleSearch and codeExecution, recovers all 200 original records, and
returns finalText="All records recovered". Rechecked at head `618811d0`.
- Not tested: live Google acceptance, billing or model quality;
Windows/macOS; successful hosted checks at this head.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; existing Google batch continuations.
- Minimum rollout channel: unchanged.
- Stable/default behavior changed: preserve the native tools and
retrieval declarations sent upstream.
- Kill switch / disable path: existing batch/CCR configuration.
- Unsafe override required: no.
- Qualification impact: real HTTP continuation verified locally; live
provider not tested.
- Rollback path: revert the fix; no persistence migration.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I did not edit CHANGELOG.md — release-please owns it

## Additional Notes

The [maintainer
re-review](https://github.com/headroomlabs-ai/headroom/pull/4211#pullrequestreview-5521733040)
says the retrieval-declaration finding is addressed. Its remaining hold
concerns shared-main rustls advisories; it did not request another code
change here.

AI-assisted; no subagents.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4211"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-15 16:28:18 -05:00
JerrettDavis 7d02829f02 Harden Anthropic prefix cache stability across proxy and batch paths 2026-04-04 13:45:37 -05:00
chopratejasandClaude Opus 4.5 d3298368bf fix: improve error handling and add comprehensive test coverage
Bug fixes:
- Replace bare except handlers with specific exception types and logging
  in proxy/server.py (6 instances for CCR, SSE parsing, cost tracking)
- Fix session_id filtering security bug in memory/backends/local.py
  (sessions were not properly isolated in vector search)

New tests (344 total):
- test_ccr_batch_processor.py: 51 tests for batch result processing
- test_compression_store.py: 76 tests for compression cache
- test_log_compressor.py: 47 tests for log format detection/compression
- test_search_compressor.py: 48 tests for grep output compression
- test_integrations/langchain/: 122 tests for LangChain integration
  (agents, memory, retriever, streaming)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 16:08:36 -08:00