mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 15:10:00 +08:00
feat/https-compose-caddy
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
603d873992 |
fix(config): use the longest matching model_context_limits prefix (#4288)
## Description
`HeadroomConfig.get_context_limit()` resolves a user's
`model_context_limits` override by prefix, but it returned the **first**
key that prefixes the model name, in dict insertion order. A more
specific override listed after a general one was never used:
```python
HeadroomClient(..., model_context_limits={"gpt-4": 8192, "gpt-4o": 128000})
client._get_context_limit("gpt-4o-2024-08-06") # 8192 on main, 128000 expected
```
`HeadroomClient._get_context_limit()` uses this value as the context
budget, so a dated `gpt-4o` snapshot was compressed against an 8K window
instead of 128K. `ModelRegistry.get()` already prefers the longest
prefix for the same reason; this applies the same rule to the
user-override path.
Closes: N/A (no matching issue found)
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/config.py`: `get_context_limit()` collects every key that
prefixes the model and returns the longest one's limit. Exact matches
and the "no match → `None`" path are unchanged, and the existing loose
prefix behaviour (`"gpt-4"` still matches `"gpt-4o"` when it is the only
key) is kept, so `test_get_context_limit_prefix_match` passes unchanged.
- `tests/test_config.py`:
`test_get_context_limit_prefers_longest_prefix` puts the general key
first (`gpt-4` before `gpt-4o`, `claude-3` before `claude-3-5-sonnet`)
and checks that the specific one wins, and that the general one still
applies when it is the only match.
## Testing
- [x] Unit tests pass (`pytest`): `tests/test_config.py`, not the full
suite
- [x] Linting passes (`ruff check .`): on the changed files
- [ ] Type checking passes (`mypy headroom`): not run
- [x] New tests added for new functionality
- [x] Manual testing performed
### Test Output
New test on `main` (fix reverted, test kept):
```text
$ python -m pytest tests/test_config.py -q -k context_limit
assert config.get_context_limit("gpt-4o-2024-08-06") == 128000
E AssertionError: assert 8192 == 128000
================== 1 failed, 4 passed, 32 deselected in 4.71s ==================
```
With the fix:
```text
$ python -m pytest tests/test_config.py -q
============================== 37 passed in 2.20s ==============================
$ ruff check headroom/config.py tests/test_config.py && ruff format --check headroom/config.py tests/test_config.py
All checks passed!
2 files already formatted
```
## Real Behavior Proof
- Environment: macOS 27.0.1 (arm64), Python 3.12.11, source tree on this
branch (native `headroom._core` from the 0.41.0 wheel), real
`openai.OpenAI` client (no request sent) and the real `OpenAIProvider`.
- Exact command / steps: built a real
`HeadroomClient(original_client=openai.OpenAI(api_key=...),
provider=OpenAIProvider(), store_url="sqlite:///…",
model_context_limits={"gpt-4": 8192, "gpt-4o": 128000})` and printed
`client._get_context_limit(model)` for three models, on `main` and on
this branch.
- Observed result: on `main` the dated `gpt-4o-2024-08-06` snapshot gets
the 8192 `gpt-4` limit; on this branch it gets the 128000 `gpt-4o`
limit, and the other two models are unchanged.
```text
== main
gpt-4-0613 -> 8192
gpt-4o -> 128000
gpt-4o-2024-08-06 -> 8192 # wrong: gpt-4o override ignored
== this branch
gpt-4-0613 -> 8192
gpt-4o -> 128000
gpt-4o-2024-08-06 -> 128000
```
- Not tested: a live `chat.completions.create(headroom_mode="optimize")`
call against a real API; the full test suite.
## Runtime Rollout Safety
- Rollout-managed feature(s): none
- Minimum rollout channel: N/A
- Stable/default behavior changed: only for users whose
`model_context_limits` has two keys where one prefixes the other; the
more specific key now wins
- Kill switch / disable path: N/A (remove the overlapping key from
`model_context_limits`)
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert this commit
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)
## Additional Notes
No docs change needed: the documented behaviour ("prefix matching for
versioned model names") is what this makes true.
<!-- devin-review-badge-begin -->
---
<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4288"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
|
||
|
|
47fd8cb3ba |
fix(client): store metrics at HeadroomConfig.store_url (#4270)
## Description
`HeadroomClient` ignored `HeadroomConfig.store_url`. The constructor
copied `config.store_url` into `self._store_url`, but then opened
storage with its own `store_url` argument:
```python
if config.store_url:
self._store_url = config.store_url # saved here
...
self._storage = create_storage(store_url) # but storage uses the argument
```
So `HeadroomClient(...,
config=HeadroomConfig(store_url="sqlite:///mine.db"))` wrote metrics to
`<system temp>/headroom.db`, and `mine.db` was never created.
`validate_setup()` reads `self._store_url`, so it reported `mine.db` as
the storage URL. The client said it stored metrics in one place and
actually stored them in another. `get_metrics()` / `get_summary()` also
read from the temp database, so anything reading `mine.db` directly (for
example `generate_report(store_url="sqlite:///mine.db")`) found nothing.
The constructor docstring says the config "takes precedence over
individual settings like store_url".
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/client.py`: storage is opened with `self._store_url`, the
URL the constructor already resolves and reports.
- `headroom/client.py`: the client no longer writes its resolved URL
back into a caller-supplied config (the old `else:
self._config.store_url = store_url`).
- **Why:** a caller can reuse one `HeadroomConfig` object for several
clients. Client A would fill `shared.store_url` with its own URL, and
client B would then read that as an explicit config value and store into
A's database despite passing its own `store_url=`. (Reported by Devin
review; the write-back was harmless on `main` only because the old
truthy default meant it never ran and storage ignored the config
anyway.)
- The client keeps its resolved URL in `self._store_url`, which is all
it uses. A client built without a config still records the URL on the
`HeadroomConfig` it creates for itself.
- `headroom/config.py`: `HeadroomConfig.store_url` now defaults to
`None` instead of `"sqlite:///headroom.db"`. `None` means "let the
client decide": its `store_url` argument, else `headroom.db` in the
system temp directory.
- **Why the default had to change:** the old default was always truthy,
so with only the one-line fix every
`HeadroomClient(config=HeadroomConfig())` would have started writing
`headroom.db` into the current working directory. v0.2.0 (
|
||
|
|
fef99cc5b4 |
feat(compress): add per-message compression diagnostics (#3058)
## Description Adds `CompressConfig.diagnostics` (bool, default `False`) and a companion `HEADROOM_DIAGNOSTICS=1` environment variable that enable per-message decision logging on every call to `compress()`. When active, `CompressResult.diagnostics` is a list of `MessageDecision` objects — one per input message — each describing: | Field | Description | |---|---| | `message_index` | Zero-based position in the original messages list | | `role` | `"user"` / `"assistant"` / `"tool"` / `"system"` / … | | `tokens_before` | Token count before the ContentRouter acted on this message | | `tokens_after` | Token count after (equals `tokens_before` for protected/passthrough) | | `action` | What was done: `"compressed:kompress:0.72"`, `"protected:user_message"`, `"passthrough:frozen"`, `"passthrough:small"`, etc. | Closes #2855. ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **`headroom/config.py`** — add `MessageDecision` dataclass; add `message_decisions: list[MessageDecision]` field to `TransformResult` - **`headroom/compress.py`** — add `CompressConfig.diagnostics`; add `CompressResult.diagnostics`; derive `collect_diagnostics` from config + env var; pass to pipeline; collect from `TransformResult` - **`headroom/transforms/pipeline.py`** — accumulate `message_decisions` from each transform result - **`headroom/transforms/content_router.py`** — instrument every decision branch in `apply()` with `_diag[i] = "action:..."` strings when `collect_diagnostics=True`; build `MessageDecision` list after Pass 3 (zero overhead when flag is off) - **`headroom/__init__.py`** — export `MessageDecision` in `__all__` and lazy-import map - **`tests/test_compress_diagnostics.py`** — 14 tests covering the full `compress() <-> pipeline` contract via a mock pipeline ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_compress_diagnostics.py -v ========================= test session starts ========================== collected 14 items tests/test_compress_diagnostics.py::test_message_decision_dataclass PASSED tests/test_compress_diagnostics.py::test_message_decision_importable_from_headroom PASSED tests/test_compress_diagnostics.py::test_transform_result_has_message_decisions_field PASSED tests/test_compress_diagnostics.py::test_transform_result_message_decisions_populated PASSED tests/test_compress_diagnostics.py::test_compress_config_diagnostics_defaults_false PASSED tests/test_compress_diagnostics.py::test_compress_config_diagnostics_can_be_set PASSED tests/test_compress_diagnostics.py::test_compress_result_diagnostics_none_by_default PASSED tests/test_compress_diagnostics.py::test_compress_passes_collect_diagnostics_when_config_true PASSED tests/test_compress_diagnostics.py::test_compress_result_diagnostics_list_when_config_true PASSED tests/test_compress_diagnostics.py::test_compress_result_diagnostics_contains_decisions PASSED tests/test_compress_diagnostics.py::test_env_var_enables_diagnostics PASSED tests/test_compress_diagnostics.py::test_env_var_absent_keeps_diagnostics_none PASSED tests/test_compress_diagnostics.py::test_env_var_overrides_config_false PASSED tests/test_compress_diagnostics.py::test_no_diagnostics_when_optimize_false PASSED ========================== 14 passed in 0.42s ========================== $ ruff check . All checks passed! ``` ## Real Behavior Proof - Environment: Python 3.12, headroom dev install - Exact command / steps: `python -m pytest tests/test_compress_diagnostics.py -v` - Observed result: 14 tests pass; diagnostics list is populated when `CompressConfig(diagnostics=True)` or `HEADROOM_DIAGNOSTICS=1` - Not tested: ContentRouter integration tests (require compiled `headroom._core` extension — run in CI) ## Runtime Rollout Safety - Rollout-managed feature(s): None — diagnostics are opt-in via `CompressConfig.diagnostics=True` or `HEADROOM_DIAGNOSTICS=1` - Minimum rollout channel: N/A (additive, opt-in only) - Stable/default behavior changed: No — `CompressConfig.diagnostics` defaults to `False`; `CompressResult.diagnostics` is `None` unless explicitly requested - Kill switch / disable path: Simply omit `diagnostics=True` / unset `HEADROOM_DIAGNOSTICS` - Unsafe override required: No - Qualification impact: None - Rollback path: Revert this PR; no schema or data migration involved ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A — no UI changes. ## Additional Notes The `collect_diagnostics` flag incurs zero overhead in the hot path when not set: the `_diag` dict in `ContentRouter.apply()` is never allocated and no `MessageDecision` objects are constructed. Signed-off-by: Radha Krishnan P <gingeekrishna@gmail.com> --------- Signed-off-by: Radhakrishnan Pachyappan <radhakrishnan.p@op.tech> Signed-off-by: Radhakrishnan P <gingeekrishna@gmail.com> |
||
|
|
f9285766dd |
feat: attribute reread waste to over-compression via marker check (#901)
## Description Fixes #899. The `reread` signal (#853/#854) counts re-served tool results but cannot answer the question that motivated it: **did Headroom cause the re-read?** A re-read after an intact first serve is agent behavior; a re-read after Headroom markerized the first serve is over-compression cost. This PR splits the signal so the actionable part is visible. Request-local, no store lookups: the client resends full history each turn and the pipeline recompresses it deterministically, so the current request already holds the evidence. `TransformPipeline.apply` passes `current_messages` into `parse_messages(compressed_messages=...)`. For each counted reread group, if the transformed copy of the **first serve** carries a CCR retrieval marker and its original text is gone, the group's counted repeats go into `reread_compressed_tokens`. Lossless reshaping (no marker) is deliberately not attributed. Closes #899. ## Type of Change - [x] New feature (non-breaking change that adds functionality) ## Changes Made - `parser.py`: `parse_messages` gains an optional `compressed_messages` param; the content-hash reread loop accumulates per-group `counted_tokens` and attributes them to `reread_compressed_tokens` when the first serve's transformed copy carries a CCR marker (`CCR_RETRIEVAL_MARKER_RE`, kept local to avoid a transforms import cycle). - `transforms/pipeline.py`: pass `current_messages` (post-transform copy) into the existing waste-detection `parse_messages` call. - `config.py`: new `reread_compressed_tokens` WasteSignals field; `dashboard.html` + `reporting/generator.py` surface it. - Tests: `tests/test_reread_attribution.py` + WasteSignals contract update. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] New tests added for new functionality - [x] New and existing unit tests pass locally with my changes ### Test Output ```text $ pytest tests/test_reread_attribution.py tests/test_parser.py tests/test_gemini_function_response_waste.py tests/test_codex_responses_waste_signals.py -q 122 passed in 1.50s $ pytest tests/ -k "waste or pipeline or reporting or config or reread" -q 348 passed, 33 skipped, 6010 deselected # (1 unrelated env-dependent failure: test_proxy_gemini_native_integration::test_generation_config — 404, reproduces on main without these changes; needs a Gemini key locally) $ ruff check headroom/parser.py headroom/transforms/pipeline.py All checks passed! ``` ## Real Behavior Proof - Environment: local macOS, repo .venv, Python 3.11.9 - Exact command / steps: rebased onto current main to resolve conflicts with #909 (merged), then ran the reread + parser + waste suites above - Observed result: a reread whose first serve is markerized attributes to `reread_compressed_tokens`; an intact first serve and a lossless (no-marker) reshape do not. #909's re-issued-call detection (same call, different bytes) continues to count and dedup correctly alongside it — all 122 targeted tests pass. - Not tested: live proxy traffic; the one gemini-native route test above (environmental 404, not introduced here). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Additional Notes **Rebased onto current main after #909 merged.** #909 added a re-issued-call reread pass *after* the original content-hash loop this PR modifies — the conflict was textual/adjacent, not a re-architecture. Resolution preserves #909's `counted_results` dedup contract and leaves its new pass unchanged; #901's attribution stays scoped to the content-hash groups it was reviewed against (attributing #909's call-key pass too would be a separate follow-up). The diff differs from the prior approval only by this reshape — worth a quick re-glance. |
||
|
|
5f1d88ad27 |
feat: detect re-served tool results as over-compression waste signal (#854)
Closes #853 ## What Adds a `reread` waste signal: identical `tool_result` content appearing at more than one message position means the agent re-fetched something already in context — the dominant failure signature of over-compression (Manus context-engineering; JetBrains "Complexity Trap", arXiv:2508.21433). Per-request savings can't see this cost; this signal makes it visible. - `WasteSignals.reread_tokens` — new field, in `total()`, exported as `"reread"` in `to_dict()`. - `parse_messages()` groups `tool_result` blocks by their **existing** `content_hash` and counts every repeat beyond the first serve. No new hashing or tokenization; one O(blocks) dict pass. - `REREAD_MIN_TOKENS = 50` guard: short outputs ("ok", empty diffs) legitimately repeat and are skipped. Duplicates within a single message (same `source_index`) are not counted. - Works across all formats the parser already normalizes to `tool_result` blocks: OpenAI `role=tool`, Anthropic `tool_result`, Strands/Bedrock `toolResult` (#813/#815). - Flows through existing generic plumbing with zero handler changes: pipeline → `RequestOutcome.waste_signals` → Prometheus `headroom_waste_signal_tokens_total{signal="reread"}` → dashboard "Waste Detected" panel. Dashboard gains label/color entries for the new key. ## Tests 7 new tests in `tests/test_parser.py::TestRereadDetection` (red before, green after): OpenAI + Anthropic format detection, repeat-counting semantics (first serve free), single-occurrence, short-duplicate guard, same-message guard, `total()`/`to_dict()` participation. Updated 2 exact-shape assertions in `tests/test_config.py`. Local runs: `tests/test_parser.py` (72 passed), `tests/test_config.py` + outcome/reporting/observability/storage/proxy-hooks suites (190 passed), `tests/test_canonical_pipeline.py` + `tests/test_proxy_pipeline_lifecycle.py` (11 passed). `ruff check` + `ruff format --check` clean. ## Real behavior proof **Setup:** macOS (Darwin 25.5), Python 3.11.9, this branch, real proxy server (`python -m headroom.proxy.server --port 18970 --anthropic-api-url http://127.0.0.1:18971`) with a local mock Anthropic upstream returning a canned `/v1/messages` response (no real key needed). **Steps:** POSTed an Anthropic-format conversation to the live proxy: agent fetches a 14 KB JSON log array via `get_logs` tool, then fetches the identical content again under a different `tool_use_id` (the re-read). **Observed result** — `curl http://127.0.0.1:18970/metrics` after the request: ``` # HELP headroom_waste_signal_tokens_total Tokens attributed to detected waste signals # TYPE headroom_waste_signal_tokens_total counter headroom_waste_signal_tokens_total{signal="json_bloat"} 9858 headroom_waste_signal_tokens_total{signal="reread"} 4935 ``` `reread` = 4935 tokens, exactly the second serve of the ~4.9k-token tool result (json_bloat counts both occurrences ≈ 2×). `/stats` shows the same: `"waste_signals": {"json_bloat": 9858, "reread": 4935, ...}` — which is what the dashboard panel renders. Also verified the negative path live: a conversation whose tool results contain non-compressible plain code text produced no waste-signal entries (the pipeline only attributes waste when compression actually engaged, unchanged behavior). **Not tested:** Gemini `functionResponse` path (parser doesn't produce `tool_result` blocks for it — pre-existing gap tracked in #819); dashboard rendering only verified via the `/stats` payload the panel binds to, not a browser screenshot. ## Out of scope (per #853) Tool-call argument matching, compression-marker attribution, tokens-per-task metric, cache hit-rate panel. --------- Co-authored-by: Ash Rhodes <ashley.rhodes@king.com> |
||
|
|
967b0db439 |
fix: B1 — retire ICM, RollingWindow, scoring, relevance + dependents
Phase B step 1 of the live-zone-only realignment. Removes ~10K LOC of
"drop messages from history" machinery that became unreachable after
PR-A1 made `/v1/messages` a passthrough on the proxy. Live-zone-only
compression (PR-B2..B7) operates on content blocks within messages;
message-list mutation no longer happens in the pipeline.
Python deletes:
- headroom/transforms/intelligent_context.py (1077 LOC)
- headroom/transforms/rolling_window.py (395 LOC)
- headroom/transforms/progressive_summarizer.py (508 LOC)
- headroom/transforms/scoring.py (459 LOC)
- headroom/transforms/tool_crusher.py (338 LOC)
- 5 corresponding tests/test_transforms/* and tests/test_proxy_intelligent_context.py
Rust deletes:
- crates/headroom-core/src/context/* (manager, config, workspace,
candidate, ccr_drop, strategy/, mod) + safety.rs replaced
- crates/headroom-core/src/scoring/* (mod, score, scorer, traits, weights)
- MessageScorerComparator from crates/headroom-parity (PR #338/#343
becomes deletable; sunk cost stays sunk)
- 13 message_scorer fixtures + record_message_scorer.py
Rust adds (move + rewrite):
- crates/headroom-core/src/transforms/safety.rs — `tool_pair_indices`
preserves the OpenAI/Anthropic tool_use ↔ tool_result pairing rule
the live-zone dispatcher (PR-B2) needs. No IcmConfig dependency.
Surface refactors:
- HeadroomConfig: drop `tool_crusher`, `rolling_window`,
`intelligent_context` fields; hoist `output_buffer_tokens` to top
level (used by client.py).
- ProxyConfig: drop `intelligent_context*` fields.
- `headroom wrap` proxy server: retire IntelligentContextManager
and RollingWindow imports + branch; pipeline is CacheAligner →
ContentRouter (smart_routing) or CacheAligner → SmartCrusher
(legacy).
- CLI: drop `--no-intelligent-context`, `--no-intelligent-scoring`,
`--no-compress-first` flags.
- LangChain memory integration: rename `_apply_rolling_window` →
`_apply_compression`, drop RollingWindowConfig dep. Threshold is
now advisory — B6 will rework the contract.
- TransformPipeline.create_pipeline now takes only cache_aligner_config.
- headroom/__init__.py + headroom/transforms/__init__.py: strip
exports of deleted symbols.
Bug fixes uncovered by full pytest sweep:
- providers/copilot/wrap.py: `environ or os.environ` collapsed
empty-dict to falsy → callers passing `environ={}` accidentally
pulled from os.environ. Use `environ if environ is not None else
os.environ`.
Test correctness fixes:
- _DummyAnthropicHandler._retry_request gains **_kwargs to match
the real handler signature post-A8.
- test_ws_http_fallback extracts JSON from `content=` (post-A3
byte-faithful) rather than the obsolete `json=` kwarg.
- test_ccr_response_handler_extra fixture joins SSE events with
`\n\n` per spec (post-A8 byte-buffer parser requirement).
- test_proxy_responses_phase_preservation: capture via direct
handler attached to the named logger, so the assertion is
order-independent (proxy `_setup_file_logging` flips
`headroom.propagate=False` once any earlier test triggers it).
- conftest.py autouse fixture resets `headroom.propagate=True`
before each test as a defensive measure for the same pollution.
- test_wrap_copilot_translated_backend_still_requires_byok:
monkeypatch.delenv every provider key so the BYOK error
actually fires.
- test_native_installers: skip when system bash < 4.3 (macOS ships 3.2).
- TestGeminiEmbedContent / TestGeminiBatchEmbedContents:
pytest.mark.skip — proxy currently has no :embedContent route;
feature gap, not regression.
Acceptance:
- cargo build --workspace + cargo clippy + cargo fmt --check: green.
- cargo test --workspace --exclude headroom-py: 777 passed.
- pytest: 4892 passed, 240 skipped, 0 failed.
- git grep returns only intentional comments referencing the deletion.
Per-PR-B1 plan: REALIGNMENT/04-phase-B-live-zone.md.
|
||
|
|
08d81f2e2c |
fix: dashboard metrics, TTFB tracking, eager LLMLingua loading, and multi-provider consistency
Dashboard was showing wildly incorrect metrics (99.5% savings, 3ms overhead) due to using Anthropic API's non-cached input_tokens instead of optimized_tokens, and dividing overhead by total request count instead of optimized-only count. Key fixes: - Use optimized_tokens (what we sent) for dashboard aggregation, not API's input_tokens which excludes cached portion - Track overhead_count separately from latency_count for correct averages - Add TTFB (time to first byte) measurement, replace full stream latency in UI - Eager-load LLMLingua model at proxy startup (eliminates 5.9s first-request delay) - Simplify CostTracker to token-based accounting with counterfactual cost display - Add two-tier compression cache to ContentRouter (skip set + result cache) - Fix compression pinning to detect both CCR and ReadLifecycle markers - Clamp tokens_saved to max(0, ...) across all provider paths - Add per-transform timing instrumentation to pipeline - Guard against over-aggressive code compression (<5% ratio) - Fix ReadLifecycle partial read supersede logic (_read_covers range check) - Disable CacheAligner and compress_superseded by default - Fix all pre-existing mypy errors (CompressionCache return types) - Fix test mocks to accept **kwargs for cache token parameters |
||
|
|
e4a41faa33 |
Fix all ruff lint and format errors for CI
- Fix E402: Move module-level imports to top of file - Fix F401: Add noqa for availability check imports - Fix F402: Rename loop variables shadowing imports - Fix E722: Replace bare except with except Exception - Fix B904: Add exception chaining (from e) - Fix F811: Remove duplicate imports - Fix B027: Add noqa for empty close() method - Fix E741: Rename ambiguous variable l -> label - Fix I001: Import sorting issues - Apply ruff format to all 106 files All 902 tests pass. |
||
|
|
c1feb60595 |
feat: Add CCR architecture, TOIN telemetry, and DevEx improvements
## Core Features ### Compress-Cache-Retrieve (CCR) Architecture - Implement reversible compression with automatic retrieval support - Add CompressionStore for caching original content with TTL-based eviction - Add CompressionFeedback for learning from retrieval patterns - Implement tool injection for LLM retrieval capability - Add MCP server support for CCR operations - Track retrieval rates to dynamically adjust compression aggressiveness ### Tool Output Intelligence Network (TOIN) - Implement cross-session pattern learning for tool compression - Add ToolSignature for structural hashing of tool outputs - Track compression success rates per strategy (top_n, sample, truncate, etc.) - Implement privacy-preserving telemetry with SHA256 hashing - Add persistent storage with JSON file backend - Support network-effect learning across tool types ### SmartCrusher Enhancements - Add crushability analysis with variance/uniqueness detection - Implement statistical anomaly detection for outlier preservation - Add relevance-based item prioritization using BM25 scoring - Support multiple compression strategies with quality retention - Add change point detection for time-series data - Implement constant factoring for homogeneous datasets ## Developer Experience Improvements ### Exception Hierarchy - Add HeadroomError base class for all custom exceptions - Add specific exceptions: ConfigurationError, ProviderError, StorageError, CompressionError, TokenizationError, CacheError, ValidationError, TransformError ### Client Enhancements - Add validate_setup() for configuration verification - Add get_stats() for in-memory session metrics without DB query - Track session statistics (requests, tokens saved, cache hits) ### Logging Infrastructure - Add structured logging to TransformPipeline with token savings - Add logging to RollingWindow for dropped message tracking - Add logging to ToolCrusher for compression events - Add logging to CacheAligner for cache hit/miss detection - Add logging to SmartCrusher for strategy selection ## Bug Fixes (from deep analysis) ### Critical Fixes - Fix eviction heap memory leak with stale entry tracking - Fix hash collision detection in compression store - Fix strategy truncation desync in TOIN - Fix non-deterministic set truncation with sorted iteration - Fix race conditions in lazy initialization with proper locking - Fix user count double-counting in TOIN metrics ### High Priority Fixes - Fix unbounded strategy_success_rates growth with LRU eviction - Fix mutable pattern references with defensive copying - Fix lock held during file I/O with copy-then-write pattern - Fix state divergence on eviction with success event recording - Fix TOIN skip check order for CPU efficiency - Fix preserve_fields type mismatch (set vs list) - Fix prioritize_indices exceeding max_items limit - Fix instance ID collision risk (32-bit to 64-bit hash) ## Testing - Add comprehensive test suites for CCR, TOIN, and telemetry - Add crushability detection tests - Add quality retention tests for compression - Add integration tests for cross-component data flow - All 902 tests passing |
||
|
|
9c7d4512d6 |
Initial commit: Headroom SDK - LLM context optimization toolkit
A comprehensive SDK for optimizing LLM context windows, reducing token usage while preserving critical information for AI agents. Core Features: - SmartCrusher: Statistical compression of tool outputs (70-85% reduction) - CacheAligner: Prefix optimization for prompt cache hits - RollingWindow: Intelligent context window management - BM25/Hybrid relevance scoring for smart item selection Integrations: - OpenAI and Anthropic provider support - LangChain integration (ChatModel, Callbacks, Runnable) - MCP (Model Context Protocol) integration for tool compression Test Coverage: - 372 tests passing across all modules - 35 performance benchmarks - Real-world agent evaluations with 88% token savings Key Components: - headroom/transforms/: Core compression transforms - headroom/providers/: OpenAI and Anthropic support - headroom/integrations/: LangChain and MCP integrations - headroom/relevance/: BM25 and hybrid scoring - headroom/pricing/: Model pricing registry - benchmarks/: Performance benchmark suite - examples/: Usage examples and demos |