mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-23 02:49:17 +08:00
feat/https-compose-caddy
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
e4a1ec19be |
refactor(transforms): deprecate unused compress_with_stats, extract_batch and align_for_cache (#4098)
## Description
`DiffCompressor.compress_with_stats`, `HTMLExtractor.extract_batch` and
`cache_aligner.align_for_cache` are public. The first two are methods on
classes exported from `headroom.transforms`, and the third's docstring
called it a stable public API. None is called anywhere in Headroom, the
plugins or the consumer repos; only tests call them. Each now emits a
`DeprecationWarning` that names the replacement.
Refs #4136
## Type of Change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)
## Changes Made
- `DiffCompressor.compress_with_stats` warns and points to
`DiffCompressor.compress`. Unlike `compress()`, it never wrote its CCR
entry to the compression store, so a retrieval marker in its output
could dangle. The docstring now says so.
- `HTMLExtractor.extract_batch` warns and points to calling
`HTMLExtractor.extract` per page.
- `align_for_cache` warns and points to
`CacheAligner(config).apply(messages, tokenizer)` and the
`stable_prefix_hash:` marker.
- The existing tests now assert the warnings. A new test checks that
`align_for_cache` warns at the caller and still returns the messages and
the hash.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ pytest tests/test_cache_aligner_detector_only.py tests/test_cache_aligner_prefix_stability.py tests/test_transforms/test_html_extractor.py tests/test_transforms/test_diff_compressor_rust_parity.py tests/test_transforms/test_diff_compressor.py
118 passed in 3.87s
$ ruff check / format: All checks passed; mypy: Success
```
## Real Behavior Proof
- Environment: macOS, Python 3.12, repo `.venv`, this branch's worktree
with the prebuilt `headroom._core`. Temp `HOME`,
`HEADROOM_WORKSPACE_DIR` and `CODEX_HOME`; nothing touched the real
home.
- Exact command / steps: (1) shared smoke harness: a real proxy from
this checkout against a local fake upstream, with one Anthropic and one
OpenAI request carrying a compressible tool result (`smoke.sh
<worktree>`); (2) `python -W always -c 'align_for_cache(...);
DiffCompressor().compress_with_stats(...)'`.
- Observed result: The proxy serves both requests (200) with the same
savings as main. Each deprecated call emits one DeprecationWarning
attributed to the caller and still returns its result. Output below.
- Not tested: Third-party callers; none is known.
```text
branch: cleanup/transforms-deprecate-unused-methods @
|
||
|
|
be5d9f7baa |
fix(tests): update CacheAligner detector-only tests for F2.2 5-field CompressionPolicy
The two F2.1 fixtures constructed CompressionPolicy with only 2 fields; F2.2 added three required tuning fields (volatile_token_threshold, max_lossy_ratio, toin_read_only). Updated subscription-disabled fixture to use Subscription defaults (32 / 0.25 / True) and the PAYG-enabled fixture to use PAYG defaults (128 / 0.45 / False), mirroring policy_for_mode(). Prefer the fixture update over default-valued fields in the dataclass so the F2.2 parity-test invariant (every CompressionPolicy is fully specified per-mode) stays load-bearing. |
||
|
|
de8e245990 |
fix(transforms): Python parity port of CompressionPolicy + cache_aligner gate (F2.1 c4/5)
Phase F2.1, commit 4 of 5 (consolidated from the 6-commit plan after
finding the Rust dispatcher gate is a no-op in F2.1 — Subscription
still gets live-zone compression by design, so the only behaviour
change is on the Python detector side).
What lands:
- New `headroom/transforms/compression_policy.py` — hand-mirror of
`headroom_core::compression_policy::CompressionPolicy`. Two fields,
`policy_for_mode(AuthMode)` constructor, `policy_default_payg()`
helper for the enforcement-flag-off path. Source of truth is the
Rust crate; a parity test guards against silent drift.
- `CacheAligner.should_apply` now reads `kwargs["compression_policy"]`
and returns `False` when `policy.cache_aligner_enabled is False`.
This is THE behaviour change for F2.1: subscription requests stop
triggering volatility warnings and the per-pipeline-instance
`_previous_prefix_hash` is no longer updated by them.
Hidden state caveat: the hash field is per-pipeline-instance, not
per-request. Clearing it on skip would race with concurrent PAYG
requests on the same pipeline, so we don't. The behaviour is "skip
silently" — exactly what cache-stability-sensitive callers want.
- 6 new tests in `tests/test_compression_policy.py`:
- per-mode field assertions (mirror Rust unit tests)
- `oauth_matches_payg_today` canary for F2.2 divergence
- immutability check (`@dataclass(frozen=True)`)
- field-set parity guard against Rust + Python drift
- 2 new tests in `tests/test_cache_aligner_detector_only.py`:
- subscription policy short-circuits should_apply
- PAYG policy does NOT short-circuit (sanity)
Note: this commit does NOT yet plumb the policy from the proxy
handlers into `pipeline.apply(...)` kwargs. That happens in c5/5
alongside the flag default flip. Until c5/5 lands, the gate is
reachable but unfired — `kwargs["compression_policy"]` is absent
from every production call site, so `policy.get(...)` returns None
and current behaviour is preserved bit-for-bit.
Verified: 28/28 affected Python tests pass.
|
||
|
|
cf5a71571c |
fix(security): allowlist GitGuardian-flagged test fixtures
GitGuardian flagged two strings on PR #350 as leaked secrets. Both are synthetic fixtures, NOT real credentials: 1. tests/test_cache_aligner_detector_only.py:215 — the canonical `eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c` JWT (header `{"alg":"HS256"}`, payload `{"sub":"1"}`) used to verify our `detect_volatile_content` recognises JWT-shaped strings. 2. tests/test_realignment_live_multi_turn.py:1091 — Anthropic-shaped tokens whose payloads literally contain "fixture" (`sk-ant-api03-payg-fixture`, `sk-ant-oat01-oauth-fixture`, `sk-ant-api03-payg-bearer-fixture`). Used to assert the auth-mode classifier routes PAYG / OAuth headers correctly. No live API call is ever made with these tokens — the test only inspects header shape. Two-layer remediation: * `.gitguardian.yaml` (new) — explicit allowlist with the literal match strings, each tagged with the file it lives in and the rationale. Anything else GG flags should be treated as a real incident; this file is the audit trail. * Inline `# ggignore` + `# noqa: S105` comments on each fixture line so a reviewer reading the test in isolation sees the intent without having to cross-reference the config. Per-feedback memory: secrets are routed via `.env`; the user's keys were never in chat or version control. These rows document the classifier-sweep false positive without weakening the detection rule. |
||
|
|
704fb2f19d |
fix: A2 — system prompt immutable; memory routes to live-zone tail; cache_aligner detector-only
P0-1: Delete `_inject_system_context` from `proxy/server.py`. Memory context now routes exclusively to the first text block of the latest non-frozen user message via `_append_context_to_latest_non_frozen_user_turn` (promoted to the canonical default in handlers/anthropic.py). Mirror applied to OpenAI Responses API at handlers/openai.py: `body["instructions"]` is no longer mutated; memory context appends to the latest user item in `body["input"]`. P2-23: Replace `headroom/transforms/cache_aligner.py` with a detector-only implementation. The legacy rewrite path (~400 LOC) is removed. The volatile- content detector uses no regex — UUIDs via `uuid.UUID`, ISO 8601 via `datetime.fromisoformat`, JWT shape via base64url segment-count check, hex hashes via length + `int(token, 16)` validation. Volatile findings surface through `cache_metrics`/`warnings`/`logger.warning`; the prompt is never mutated. Configurability: new env var `HEADROOM_MEMORY_INJECTION_MODE` with values `live_zone_tail` (default) and `disabled`. No `system_prompt` value — that path is permanently retired. Structured logs: every memory injection emits `event=memory_injection` with `decision`, `bytes_injected`, `query_hash` (BLAKE2b, never raw query), `session_id`, `request_id`. Auth is never logged. Tests: - Add `tests/test_proxy_system_prompt_immutable.py` (7 tests). - Add `tests/test_cache_aligner_detector_only.py` (20 tests). - Replace `tests/test_transforms/test_cache_aligner.py` (rewrite-path tests, 58 cases) with detector-only behavior. - Update `tests/test_acceptance.py::TestDateTrap` to pin the new detector-only contract. Acceptance: - `git grep -n "_inject_system_context\|_inject_to_system_or_instructions" headroom/` returns nothing. - `git grep -n "import re\|from re import" headroom/transforms/cache_aligner.py` returns nothing. - Targeted suite (`test_proxy_system_prompt_immutable.py`, `test_cache_aligner_detector_only.py`, `test_proxy_anthropic_cache_stability.py`, `test_acceptance.py::TestDateTrap`, `test_memory*.py`, `test_cli/`) green. |