mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(cache/google): tolerate timezone-aware cache expiry timestamps (#3210)
## Description
`GoogleCacheOptimizer` tracks Google `CachedContent` lifecycles via
`CachedContentInfo`, whose `is_expired` / `ttl_remaining_seconds` /
`age_seconds` compared a stored `expires_at` / `created_at` against a
**naive** `datetime.now()`:
```python
@property
def is_expired(self) -> bool:
return datetime.now() >= self.expires_at
```
The documented integration registers the timestamp straight off Google's
SDK:
```python
cached_content = genai.caching.CachedContent.create(model=..., contents=..., ttl=timedelta(hours=1))
optimizer.register_cache(cache_id=cached_content.name, ..., expires_at=cached_content.expire_time)
```
`CachedContent.expire_time` is a timezone-**aware** RFC3339 value.
Comparing or subtracting it against a naive `datetime.now()` raises
`TypeError: can't compare offset-naive and offset-aware datetimes`. This
crashes `register_cache` itself (it logs `ttl_remaining_seconds`), and
every consumer of `is_expired`: `get_reusable_cache`,
`cleanup_expired_caches`, and the `list_*` helpers. A single aware entry
also poisons `cleanup_expired_caches`, which iterates `is_expired` over
the whole registry. `CachedContentInfo.from_dict` reconstructs the same
aware value from a persisted `expires_at`, so the crash survives a
save/restore round-trip.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/cache/google.py`: added a `_now_matching(reference)` helper
that returns `datetime.now(reference.tzinfo)`, and use it in
`CachedContentInfo.is_expired`, `ttl_remaining_seconds`, and
`age_seconds`. "Now" is anchored to the same awareness as the value it
is measured against, so an aware `expires_at` compares against an aware
now, and a naive one keeps the exact prior behavior
(`datetime.now(None)` is naive local time).
- `tests/test_cache/test_google.py`: added
`test_cache_registration_accepts_aware_expiry`,
`test_cache_lookup_expired_aware`, and
`test_from_dict_roundtrip_preserves_aware_expiry`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
### Test Output
```text
tests/test_cache/test_google.py -> 21 passed in 0.72s
uvx ruff@0.16.2 check headroom/cache/google.py tests/test_cache/test_google.py -> All checks passed!
uvx mypy@1.20.2 headroom/cache/google.py -> Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1,
ruff 0.16.2 and mypy 1.20.2 via uvx.
- Exact command / steps: before the fix, `optimizer.register_cache(...,
expires_at=datetime.now(timezone.utc) + timedelta(hours=1))` raised
`TypeError: can't subtract offset-naive and offset-aware datetimes` at
`google.py:108` (`ttl_remaining_seconds`, called from `register_cache`'s
log line). After the fix, the same call registers cleanly,
`get_reusable_cache` returns the entry, `cleanup_expired_caches` returns
`[]`, an aware past-expiry entry reports `is_expired == True`, and naive
registrations behave exactly as before (`is_expired == False`,
`ttl_remaining_seconds > 0`).
- Observed result: aware and naive expiry timestamps both work; no
`TypeError`; naive-path values unchanged.
- Not tested: no live `genai.caching.CachedContent.create` call (the
optimizer intentionally makes no API calls); the aware timestamp Google
returns is reproduced with `datetime.now(timezone.utc)`, which has the
same awareness.
## Runtime Rollout Safety
- Rollout-managed feature(s): none. `GoogleCacheOptimizer` is an
SDK-side cache lifecycle utility (`headroom.cache`), not a
rollout-channel-gated runtime feature.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: no. Naive timestamps take
`datetime.now(None)`, identical to the previous `datetime.now()`; only
the previously-crashing aware case changes, from `TypeError` to a
correct comparison.
- Kill switch / disable path: N/A.
- Unsafe override required: no.
- Qualification impact: none; behavior for existing naive callers is
byte-for-byte unchanged.
- Rollback path: revert this PR; the properties return to a naive
`datetime.now()`.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A:
internal behavior, unchanged semantics)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`
## Additional Notes
`_now_matching` is deliberately minimal: it does not rewrite stored
timestamps or change the serialization format, so a mixed registry of
naive and aware entries each compare correctly on their own terms. This
matches the module's existing convention of leaving caller-supplied
timestamps as-is.
This commit is contained in:
Vendored
+20
-3
@@ -60,6 +60,23 @@ from .base import (
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _now_matching(reference: datetime) -> datetime:
|
||||
"""Return the current time with tzinfo matching ``reference``.
|
||||
|
||||
A ``CachedContentInfo`` timestamp may be naive (the local-time default this
|
||||
module's own examples produce with ``datetime.now() + timedelta(...)``) or
|
||||
timezone-aware (Google's ``genai.caching.CachedContent.expire_time`` is an
|
||||
aware RFC3339 value, and the documented integration registers it directly).
|
||||
Comparing or subtracting a naive ``datetime.now()`` against an aware
|
||||
``expires_at`` raises ``TypeError``, which crashed ``register_cache`` and
|
||||
every ``is_expired`` consumer. Anchoring "now" to the same awareness as the
|
||||
value it is measured against is correct for both: ``datetime.now(None)`` is
|
||||
the naive local time used before, and ``datetime.now(tz)`` is the aware
|
||||
instant for a tz-aware timestamp.
|
||||
"""
|
||||
return datetime.now(reference.tzinfo)
|
||||
|
||||
|
||||
# Google-specific constants
|
||||
GOOGLE_MIN_CACHE_TOKENS = 32_768 # 32K tokens minimum
|
||||
GOOGLE_CACHE_DISCOUNT = 0.75 # 75% discount on cached tokens
|
||||
@@ -100,18 +117,18 @@ class CachedContentInfo:
|
||||
@property
|
||||
def is_expired(self) -> bool:
|
||||
"""Check if cache has expired."""
|
||||
return datetime.now() >= self.expires_at
|
||||
return _now_matching(self.expires_at) >= self.expires_at
|
||||
|
||||
@property
|
||||
def ttl_remaining_seconds(self) -> int:
|
||||
"""Seconds remaining until expiry."""
|
||||
remaining = (self.expires_at - datetime.now()).total_seconds()
|
||||
remaining = (self.expires_at - _now_matching(self.expires_at)).total_seconds()
|
||||
return max(0, int(remaining))
|
||||
|
||||
@property
|
||||
def age_seconds(self) -> int:
|
||||
"""Age of the cache in seconds."""
|
||||
return int((datetime.now() - self.created_at).total_seconds())
|
||||
return int((_now_matching(self.created_at) - self.created_at).total_seconds())
|
||||
|
||||
def to_dict(self) -> dict[str, Any]:
|
||||
"""Serialize to dictionary."""
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Tests for GoogleCacheOptimizer."""
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -123,6 +123,56 @@ class TestGoogleCacheOptimizer:
|
||||
found = optimizer.get_reusable_cache("expired123")
|
||||
assert found is None
|
||||
|
||||
def test_cache_registration_accepts_aware_expiry(self, optimizer):
|
||||
"""A timezone-aware ``expires_at`` must not crash lifecycle checks.
|
||||
|
||||
Google's ``genai.caching.CachedContent.expire_time`` is an aware
|
||||
RFC3339 value, and the documented integration registers it directly.
|
||||
Comparing it against a naive ``datetime.now()`` raised
|
||||
``TypeError: can't compare offset-naive and offset-aware datetimes``,
|
||||
which crashed ``register_cache`` itself (it logs ``ttl_remaining_seconds``)
|
||||
and every ``is_expired`` consumer.
|
||||
"""
|
||||
aware_future = datetime.now(timezone.utc) + timedelta(hours=1)
|
||||
cache_info = optimizer.register_cache(
|
||||
cache_id="aware-cache",
|
||||
content_hash="aware123",
|
||||
token_count=50000,
|
||||
expires_at=aware_future,
|
||||
)
|
||||
assert not cache_info.is_expired
|
||||
assert cache_info.ttl_remaining_seconds > 0
|
||||
assert cache_info.age_seconds >= 0
|
||||
|
||||
# Lookup and cleanup iterate is_expired over the registry.
|
||||
assert optimizer.get_reusable_cache("aware123") is not None
|
||||
assert optimizer.cleanup_expired_caches() == []
|
||||
|
||||
def test_cache_lookup_expired_aware(self, optimizer):
|
||||
"""An already-expired aware cache is recognized as expired, not crashed."""
|
||||
aware_past = datetime.now(timezone.utc) - timedelta(hours=1)
|
||||
cache_info = optimizer.register_cache(
|
||||
cache_id="aware-expired",
|
||||
content_hash="awareexp123",
|
||||
token_count=50000,
|
||||
expires_at=aware_past,
|
||||
)
|
||||
assert cache_info.is_expired
|
||||
assert optimizer.get_reusable_cache("awareexp123") is None
|
||||
|
||||
def test_from_dict_roundtrip_preserves_aware_expiry(self):
|
||||
"""``to_dict``/``from_dict`` of an aware timestamp stays comparable."""
|
||||
info = CachedContentInfo(
|
||||
cache_id="rt",
|
||||
content_hash="rt123",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
expires_at=datetime.now(timezone.utc) + timedelta(hours=1),
|
||||
token_count=50000,
|
||||
)
|
||||
restored = CachedContentInfo.from_dict(info.to_dict())
|
||||
assert restored.is_expired is False
|
||||
assert restored.ttl_remaining_seconds > 0
|
||||
|
||||
def test_cache_lookup_insufficient_ttl(self, optimizer):
|
||||
"""Test that caches with insufficient TTL are not returned."""
|
||||
optimizer.register_cache(
|
||||
|
||||
Reference in New Issue
Block a user