Files
headroom/tests/test_ccr_batch_store.py
Mansurshakh Japarov 2cbdd22a65 fix(batch): preserve live contexts during capacity cleanup (#4210)
## Description

Closes #4181.

Preserve live batch continuation contexts when replacing an existing
entry at capacity, and reclaim expired entries before evicting the
oldest live entry. The verified fork fix is now normally merged onto
main f1bc260 and published at
`e606a044babeed4913cf6c3ecd9ef940347a3b12`; no reset, force push or hook
bypass.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Replacing an existing batch refreshes its request context and TTL
without consuming another capacity slot or evicting an unrelated live
batch.
- Inserting a new batch removes expired contexts before applying the
existing oldest-live eviction policy.
- Keep the configured capacity bounded and retain deterministic
public-store regressions for both data-loss transitions.

## Testing

- [ ] Unit tests pass (pytest) — no entire-project Python suite claim.
- [ ] Linting passes (ruff check .) — normal changed-file hooks passed;
no whole-repository Ruff claim.
- [ ] Type checking passes (mypy headroom) — no separate whole-project
mypy claim.
- [x] Manual testing performed

### Test Output

Fresh current-main focused command: `python -m pytest
tests/test_ccr_batch_store.py tests/test_ccr_batch_processor.py
tests/test_proxy_handlers_batch.py -q`.

```text
110 passed in 25.77s
```

Normal current-main merge/push ran Rust workspace checks, rebuilt the
real native extension, and completed commitlint plus:

```text
181 critical Python tests passed
ci-precheck PASSED — safe to push.
```

Original independent real-store proof and failing-before/passing-after
regressions:
https://github.com/headroomlabs-ai/headroom/issues/4181#issuecomment-6148313526
. That older four-file suite was 119 passed, 14 skipped; it is not
relabelled as the current three-file result.

## Real Behavior Proof

- Environment: Linux x86_64, Python 3.13.14, actual public
BatchContextStore.store/get API, synthetic batch requests; no mocked
store or provider credentials.
- Exact command / steps: standalone asyncio script fills a three-entry
store with a,b,c then replaces c; separately fills a two-entry store
with an older live context and a newer expired context, then inserts a
new batch.
- Observed result: before the fix replacement lost a and insertion lost
the live context; after the fix a,b,c remain and the updated c model is
returned, while insertion retains live+new and removes expired. Original
runtime evidence is linked above; the current-main consumer suites and
full push gates also passed.
- Not tested: live provider continuation quality, cloud invoices,
production throughput, Windows/macOS, or hosted CI for this newly opened
PR.

## Runtime Rollout Safety

- Rollout-managed feature(s): none; capacity policy correction within
the existing batch context store.
- Minimum rollout channel: existing batch-store users; no new channel.
- Stable/default behavior changed: updates no longer discard an
unrelated live context; expired entries are reclaimed before live
eviction.
- Kill switch / disable path: existing batch-processing configuration;
no new override.
- Unsafe override required: no.
- Qualification impact: public-store transitions, downstream batch
consumers and normal full push gates verified; provider execution was
not needed for this store defect.
- Rollback path: revert the capacity-policy fix; no persistence
migration or capacity increase.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I did not edit CHANGELOG.md — release-please owns it

## Additional Notes

This publishes an already-verified fix from the earlier Headroom batch
now that an author PR slot is available. It is not counted toward the
newly requested ten Bend issues. AI-assisted without subagents.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4210"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
2026-10-15 12:53:09 -05:00

173 lines
5.6 KiB
Python

from __future__ import annotations
import sys
from dataclasses import dataclass
from headroom.ccr.batch_store import (
BatchContext,
BatchContextStore,
BatchRequestContext,
get_batch_context_store,
reset_batch_context_store,
)
def test_batch_context_defaults_and_expiry(monkeypatch) -> None:
monkeypatch.setattr("headroom.ccr.batch_store.time.time", lambda: 100.0)
context = BatchContext(batch_id="batch-1", provider="anthropic", created_at=100.0)
assert context.expires_at == 100.0 + 86400
assert context.is_expired is False
request = BatchRequestContext(
custom_id="req-1",
messages=[{"role": "user", "content": "hello"}],
tools=[{"name": "tool"}],
model="gpt-4o",
system_instruction="system",
extras={"x": 1},
)
context.add_request(request)
assert context.get_request("req-1") is request
assert context.get_request("missing") is None
monkeypatch.setattr("headroom.ccr.batch_store.time.time", lambda: context.expires_at + 1)
assert context.is_expired is True
async def test_batch_context_store_core_operations(monkeypatch) -> None:
now = {"value": 100.0}
monkeypatch.setattr("headroom.ccr.batch_store.time.time", lambda: now["value"])
store = BatchContextStore(ttl=10, max_contexts=2)
first = BatchContext(batch_id="b1", provider="anthropic")
second = BatchContext(batch_id="b2", provider="google")
third = BatchContext(batch_id="b3", provider="openai")
await store.store(first)
now["value"] = 101.0
await store.store(second)
assert (await store.get("b1")) is first
now["value"] = 102.0
await store.store(third)
assert await store.get("b1") is None
assert await store.get("b2") is second
assert await store.get("b3") is third
assert await store.remove("b2") is True
assert await store.remove("b2") is False
async def test_replacing_full_store_context_preserves_other_batch_and_refreshes_ttl(
monkeypatch,
) -> None:
now = {"value": 100.0}
monkeypatch.setattr("headroom.ccr.batch_store.time.time", lambda: now["value"])
store = BatchContextStore(ttl=10, max_contexts=2)
first = BatchContext(batch_id="first", provider="openai", created_at=0)
second = BatchContext(batch_id="second", provider="openai", created_at=1)
await store.store(first)
await store.store(second)
now["value"] = 105.0
replacement = BatchContext(batch_id="second", provider="openai")
replacement.add_request(
BatchRequestContext(custom_id="request", messages=[], model="updated-model")
)
await store.store(replacement)
assert await store.get("first") is first
assert (await store.get("second")).get_request("request").model == "updated-model"
now["value"] = 111.0
assert await store.get("first") is None
assert await store.get("second") is replacement
async def test_full_store_reclaims_expired_batch_before_evicting_live_batch(
monkeypatch,
) -> None:
now = {"value": 100.0}
monkeypatch.setattr("headroom.ccr.batch_store.time.time", lambda: now["value"])
store = BatchContextStore(ttl=10, max_contexts=2)
expired = BatchContext(batch_id="expired", provider="openai", created_at=1)
await store.store(expired)
now["value"] = 105.0
live = BatchContext(batch_id="live", provider="openai", created_at=0)
await store.store(live)
now["value"] = 111.0
new = BatchContext(batch_id="new", provider="openai", created_at=2)
await store.store(new)
assert await store.get("live") is live
assert await store.get("expired") is None
assert await store.get("new") is new
async def test_batch_context_store_cleanup_stats_and_memory_stats(monkeypatch) -> None:
now = {"value": 200.0}
monkeypatch.setattr("headroom.ccr.batch_store.time.time", lambda: now["value"])
store = BatchContextStore(ttl=5, max_contexts=10)
first = BatchContext(batch_id="b1", provider="anthropic")
first.add_request(
BatchRequestContext(custom_id="r1", messages=[{"content": "alpha"}], tools=[])
)
second = BatchContext(batch_id="b2", provider="google")
second.add_request(
BatchRequestContext(custom_id="r2", messages=[{"content": ["nested"]}], tools=[{}])
)
await store.store(first)
await store.store(second)
assert await store.stats() == {
"total_contexts": 2,
"max_contexts": 10,
"ttl_seconds": 5,
"providers": {"anthropic": 1, "google": 1},
}
now["value"] = 210.0
assert await store.cleanup_expired() == 2
assert (await store.stats())["total_contexts"] == 0
@dataclass
class FakeComponentStats:
name: str
entry_count: int
size_bytes: int
budget_bytes: int | None
hits: int
misses: int
evictions: int
monkeypatch.setitem(
sys.modules,
"headroom.memory.tracker",
type("TrackerModule", (), {"ComponentStats": FakeComponentStats}),
)
now["value"] = 220.0
third = BatchContext(batch_id="b3", provider="openai")
third.add_request(
BatchRequestContext(
custom_id="r3", messages=[{"content": "payload"}], tools=[{"name": "t"}]
)
)
await store.store(third)
stats = store.get_memory_stats()
assert stats.name == "batch_context_store"
assert stats.entry_count == 1
assert stats.size_bytes > 0
def test_global_batch_context_store_reset() -> None:
reset_batch_context_store()
store_one = get_batch_context_store()
store_two = get_batch_context_store()
assert store_one is store_two
reset_batch_context_store()
store_three = get_batch_context_store()
assert store_three is not store_one