mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(cache): preserve cache_control schema properties (#3348)
## Description
Preserve legitimate JSON Schema properties named `cache_control` when
building semantic response-cache keys. The normalizer now strips only
Anthropic prompt-cache annotations (`{"type": "ephemeral"}` with
optional `ttl`) instead of every key with that name.
Closes #3345
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- Narrow `cache_control` removal to the documented prompt-cache
annotation shape.
- Keep both semantic-cache policy modules synchronized.
- Add regressions proving user-defined schema properties remain in cache
identity.
## Testing
- [x] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
```text
$ .venv/bin/pytest -q tests/test_semantic_cache_key_policy.py tests/test_proxy_semantic_cache_key.py tests/test_proxy_semantic_cache_key_policy.py tests/test_proxy_semantic_cache_key_integration.py
42 passed, 1 warning in 1.74s
$ .venv/bin/ruff check <changed files>
All checks passed!
$ .venv/bin/ruff format --check <changed files>
4 files already formatted
$ .venv/bin/mypy headroom/proxy/semantic_cache_key.py headroom/proxy/semantic_cache_key_policy.py
Success: no issues found in 2 source files
```
## Real Behavior Proof
- Environment: Linux, Python 3.14, current `main` plus this patch,
in-memory `SemanticCache`.
- Exact command / steps: instantiated production `SemanticCache`,
computed keys for otherwise-identical tool schemas with and without a
boolean property named `cache_control`, and compared the keys.
- Observed result: the two production cache keys are distinct, so the
schemas no longer collide.
```text
collision=False
keys_distinct=True
```
- Not tested: live Anthropic API traffic; full repository test suite.
## Runtime Rollout Safety
- Rollout-managed feature(s): Semantic response cache key normalization.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: Yes, only for tool/schema payloads
containing a non-annotation `cache_control` key.
- Kill switch / disable path: disable semantic caching with
`cache_enabled=False`.
- Unsafe override required: No.
- Qualification impact: cache entries for affected schemas are correctly
partitioned after deployment.
- Rollback path: revert this commit.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Screenshots (if applicable)
N/A.
## Additional Notes
Documentation is unchanged because this corrects internal cache-key
normalization without changing the public API. The warning above is the
existing Starlette `TestClient` deprecation warning.
## Maintainer refresh (2026-09-16)
Current main removed the unused duplicate `semantic_cache_key.py`; this
refresh preserves that deletion and applies the schema-property fix only
in `semantic_cache_key_policy.py`, which the proxy imports. Both
policy-level and SemanticCache integration regressions remain. All 35
focused cache-key tests pass, and focused Ruff checks pass. Reported
governance checks pass; the local test results are recorded above.
---------
Co-authored-by: Jerrett Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Jerrett Davis
parent
fc5a09e1b3
commit
b84c4c9f71
@@ -8,11 +8,25 @@ from typing import Any
|
||||
|
||||
|
||||
def strip_cache_control(obj: Any) -> Any:
|
||||
"""Recursively drop ``cache_control`` annotations before hashing."""
|
||||
"""Recursively drop prompt-cache annotations, preserving schema properties."""
|
||||
return _strip_cache_control(obj, preserve_key=False)
|
||||
|
||||
|
||||
def _strip_cache_control(obj: Any, *, preserve_key: bool) -> Any:
|
||||
"""Strip directives while retaining names inside JSON Schema ``properties``.
|
||||
|
||||
``cache_control`` is also a valid user-defined JSON Schema property name.
|
||||
Its value still needs normal recursion because that property's schema may
|
||||
itself contain prompt-cache annotations.
|
||||
"""
|
||||
if isinstance(obj, dict):
|
||||
return {k: strip_cache_control(v) for k, v in obj.items() if k != "cache_control"}
|
||||
return {
|
||||
k: _strip_cache_control(v, preserve_key=k == "properties")
|
||||
for k, v in obj.items()
|
||||
if k != "cache_control" or preserve_key
|
||||
}
|
||||
if isinstance(obj, list):
|
||||
return [strip_cache_control(item) for item in obj]
|
||||
return [_strip_cache_control(item, preserve_key=False) for item in obj]
|
||||
return obj
|
||||
|
||||
|
||||
|
||||
@@ -42,6 +42,28 @@ def test_different_tools_distinct_keys():
|
||||
assert _key(cache, tools=tools_a) != _key(cache, tools=tools_b)
|
||||
|
||||
|
||||
def test_tool_schema_cache_control_property_produces_distinct_key():
|
||||
"""A user-defined schema property is not a prompt-cache annotation."""
|
||||
tools_without = [{"name": "maintain", "input_schema": {"type": "object", "properties": {}}}]
|
||||
tools_with = [
|
||||
{
|
||||
"name": "maintain",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"cache_control": {
|
||||
"type": "boolean",
|
||||
"description": "Flush the application cache",
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
]
|
||||
|
||||
cache = SemanticCache()
|
||||
assert _key(cache, tools=tools_without) != _key(cache, tools=tools_with)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"field,a,b",
|
||||
[
|
||||
|
||||
@@ -55,3 +55,17 @@ def test_strip_cache_control_recurses_through_dicts_and_lists() -> None:
|
||||
"system": [{"type": "text", "text": "sys"}],
|
||||
"tools": [{"name": "read"}],
|
||||
}
|
||||
|
||||
|
||||
def test_strip_cache_control_preserves_user_defined_schema_property() -> None:
|
||||
schema = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"cache_control": {
|
||||
"type": "boolean",
|
||||
"description": "Flush the application cache",
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
assert strip_cache_control(schema) == schema
|
||||
|
||||
Reference in New Issue
Block a user