Commit Graph
7 Commits
Author SHA1 Message Date
Nandana Dileep d8ff0fe9d2 fix: [BUG] Volatile SQLite metrics stores lose their schema before the first save (#4633)
<!-- oss-harness:v2 -->
Fixes #4262.

Focused Python fix for: [BUG] Volatile SQLite metrics stores lose their
schema before the first save

Changed implementation: headroom/storage/sqlite.py.
Regression tests: tests/test_storage/test_sqlite.py.

Verification (working directory `.`): `.venv/bin/python -m pytest -q -x
-p no:cacheprovider -- /workspace/tests/test_storage/test_sqlite.py`
- Patched code: 35 tests, zero failures/errors.
- Original code with the same regression tests: 1 failures, zero
collection/setup errors.
- Base commit: `12dca26aa11d99b3af6c6d68cdf96ff90ef415aa`.

AI-assisted contribution: OpenCode using Autoloops wrote this patch. The
automated harness independently ran the tests listed above. No human
review is claimed.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4633"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

### Maintainer real behavior proof

Reviewed `55a6c8e3807c3d59da81a457514c82d4a3b48032` against main
`12dca26aa11d99b3af6c6d68cdf96ff90ef415aa` in isolated Windows
worktrees, Python 3.12.13. Current main is an ancestor; no branch update
was needed.

Executed `.venv/Scripts/python.exe -m pytest
tests/test_storage/test_sqlite.py -q -k 'not test_creates_db_file'
--disable-warnings`: **34 passed**, including disk migration/concurrency
and volatile-store regressions. Ruff and `git diff --check` passed. The
excluded test fails before exercising storage because its fixture holds
the temporary file open on Windows; the identical WinError 32 was
reproduced independently on current main.

Also ran a direct API smoke script, separately loading `git show
upstream/main:headroom/storage/sqlite.py` and `git show
HEAD:headroom/storage/sqlite.py`. For each of
`SQLiteStorage(':memory:')` and `SQLiteStorage('')`, it saved one
`RequestMetrics` row, checked equality with `get`, `count == 1`, one
result from `query`, `get_summary_stats()['total_requests'] == 1`, then
closed and verified a recreated store has zero rows. **Main: first save
raises `OperationalError('no such table: requests')` for both paths. PR:
all operations pass for both paths.**

Limits: this is a direct local storage API exercise, not a running proxy
or production-load qualification. Hosted CI's four test shards and
applicable checks passed on the exact reviewed head. No release or merge
performed.
2026-11-04 12:28:05 -06:00
b4c9bdf100 fix: [BUG] Saved request metrics lose their cache optimization fields (#4584)
<!-- oss-harness:v2 -->
Fixes #4261.

Focused Python fix for: [BUG] Saved request metrics lose their cache
optimization fields

Changed implementation: headroom/storage/jsonl.py,
headroom/storage/sqlite.py.
Regression tests: tests/test_storage/test_jsonl.py,
tests/test_storage/test_sqlite.py, tests/test_storage_backends.py.

Verification (working directory `.`): `.venv/bin/python -m pytest -q -x
-p no:cacheprovider -- /workspace/tests/test_storage/test_jsonl.py
/workspace/tests/test_storage/test_sqlite.py
/workspace/tests/test_storage_backends.py`
- Patched code: 57 tests, zero failures/errors.
- Original code with the same regression tests: 1 failures, zero
collection/setup errors.
- Base commit: `1fe8549f9ca77a2bc1280e4319980e0b9b31f5b9`.

AI-assisted contribution: OpenCode using Autoloops wrote this patch. The
automated harness independently ran the tests listed above. No human
review is claimed.

<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4584"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

## Maintainer verification: Real behavior proof

Tested the current synced head
`0ad8c55662fb905cb3464fe82a58aec1335a637b` on Windows with CPython 3.12.
A standalone script (outside pytest, no mocks/provider calls)
instantiated `RequestMetrics` with all seven cache-optimization fields,
saved it through each real `SQLiteStorage` and `JSONLStorage`, closed
the store, opened a new instance against the same on-disk file, and
retrieved the request by ID. Exact command:
`G:/git/headroom/.venv/Scripts/python.exe
C:/Users/jd/AppData/Local/Temp/headroom-4584-durable-proof.py` from the
isolated branch checkout.

Both backends returned `cache_optimizer_used='anthropic'`,
`cache_optimizer_strategy='explicit'`, `cacheable_tokens=1200`,
`breakpoints_inserted=2`, `estimated_cache_hit=True`,
`estimated_savings_percent=81.5`, and `semantic_cache_hit=True` after
reopening. All seven values matched the saved input. This checks real
local disk persistence; live provider/proxy traffic, other operating
systems, and multiprocess migration were not part of this standalone
observation. The targeted storage tests separately cover
legacy/concurrent migration: 57 passed; the excluded Windows temp-file
unlink test fails identically on main.

---------

Co-authored-by: nandanadileep <116407139+nandanadileep@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Nandana Dileep <nandanadileep@users.noreply.github.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-04 08:41:23 -06:00
bf34ccf588 fix(storage): normalize timezone-aware JSONL time filters (#4063)
## Description

Passing a timezone-aware datetime to JSONLStorage.query(), count() or
get_summary_stats() raises TypeError. Stored rows are read as naive UTC,
but the filter was compared directly with them. SQLite already
normalizes its filter through format_timestamp().

A private `_utc_bound()` helper normalizes JSONL boundaries once before
iteration using `astimezone(timezone.utc).replace(tzinfo=None)`,
matching the conversion in `format_timestamp()` without a string round
trip. This preserves the existing naive-UTC contract and microseconds
without changing the reader or SQLite.

Closes #4062. Related: #3116 fixed UTC timestamp serialization; this
addresses the remaining JSONL filter comparison.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Normalize non-null start/end boundaries in the three JSONL filter
entry points.
- Add three focused JSONL regression cases, one per method, covering
+05:30 start-only, end-only, and equal start/end bounds. Records one
microsecond before and after the boundary prove inclusive bounds and
preserved precision.

## Testing

- [x] New tests added for new functionality
- [x] Manual testing performed
- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)

### Necessary boundary coverage follow-up (e9f33fe5)

- Retain three parameterized method cases. Each checks start-only,
end-only, and equal start/end bounds, inclusive boundaries, and
microsecond precision.
- Storage and timestamp utility tests: 44 passed. Ruff, formatting, and
whitespace checks passed.
- This preserves normal single-bound API usage while keeping the reduced
suite requested by the maintainer.
- Full Python suite was not rerun; hosted checks for this head remain
separate from local evidence.

### Maintainer-requested revision validation (f206b72c)

- Storage suite: 37 passed, including all three focused regression
cases.
- The three focused cases fail with the original pre-fix source
(TypeError).
- Ruff, formatting, whitespace checks and targeted mypy passed.
- Tests import the current workspace source with published native
dependencies. Editable source installation was blocked by the local
Cargo registry mirror connection; this revision does not change native
code.
- Full Python suite was not rerun for this revision. The evidence below
describes the original revision and its larger test matrix.

### Original revision test output

```text
Storage + timestamp baseline: 51 passed
With the new cases and original implementation: 36 failed, 60 passed
After fix: 147 passed
Fix-only stash regression check: 36 failed, 60 passed
96 new cases: 96 passed
Storage/timestamp suite after source-extension install: 147 passed
make ci-precheck: passed (Rust gates, editable source install, 177 Python passed / 4 skipped, commitlint)
ruff check .: All checks passed!
ruff format --check headroom/storage/jsonl.py tests/test_storage/test_jsonl.py: 2 files already formatted
mypy headroom/storage/jsonl.py --follow-imports=silent: Success
pre-commit run --files headroom/storage/jsonl.py tests/test_storage/test_jsonl.py: all six hooks passed (including isolated mypy)
mypy headroom: one pre-existing truthy-function error in headroom/integrations/langchain/chat_model.py:394, reproduced with the fix stashed
cargo fmt --all -- --check: passed
cargo clippy --workspace --locked -- -D warnings: passed
cargo test --workspace --locked: 1655 passed, 3 ignored
Full suite after fix: 43 failed, 16306 passed, 718 skipped, 4 errors (2302.22s)
Full baseline: 41 failed, 16212 passed, 718 skipped, 4 errors (2026.35s)
All 41 baseline failures and 4 errors also occur after the fix.
Two revised-only failures (bundled tool install and Groq pricing) pass on individual reruns in both trees; their original state/order dependence is not proven.
Collection delta: 96 new cases; full-run delta: 94 passes + 2 transient failures.
```

## Real Behavior Proof

- Environment: macOS 26.3.1 arm64, Python 3.11.5, real JSONL files and
SQLite databases, model field gpt-4o; no LLM provider calls or storage
mocks.
- Exact command / steps: PYTHONPATH="$PWD" .venv/bin/python
/tmp/headroom-gc-ship/real_behavior.py; the self-contained script is
included below.
- Observed result: Both backends return only the boundary record, count
1 and token savings 200 for a +05:30 start and -07:00 end representing
the same UTC instant; JSONL raised TypeError before the fix.
- Not tested: Windows or live provider traffic. The full Python baseline
comparison uses the same published v0.40.0 native wheel in both runs; a
wheel built from this source revision also succeeds. After installing
the source extension (SHA-256 matched the locally built wheel), the 147
storage/timestamp tests, disk-backed proof and make ci-precheck passed.

```text
JSONLStorage: ids=['boundary'], count=1, saved=200
SQLiteStorage: ids=['boundary'], count=1, saved=200
```

### Reproducible disk-backed check

```python
from datetime import datetime, timedelta, timezone
from tempfile import TemporaryDirectory

from headroom.config import RequestMetrics
from headroom.storage import JSONLStorage, SQLiteStorage

boundary = datetime(2026, 4, 23, 23, 59, 59, 500000, tzinfo=timezone.utc)
start = boundary.astimezone(timezone(timedelta(hours=5, minutes=30)))
end = boundary.astimezone(timezone(timedelta(hours=-7)))
with TemporaryDirectory() as directory:
    for backend in (JSONLStorage, SQLiteStorage):
        with backend(f'{directory}/{backend.__name__}') as storage:
            for name, seconds in [('before', -1), ('boundary', 0), ('after', 1)]:
                storage.save(RequestMetrics(
                    request_id=name, timestamp=boundary + timedelta(seconds=seconds),
                    model='gpt-4o', stream=False, mode='audit',
                    tokens_input_before=1000, tokens_input_after=800,
                ))
            filters = dict(start_time=start, end_time=end)
            ids = [row.request_id for row in storage.query(**filters)]
            count = storage.count(**filters)
            stats = storage.get_summary_stats(**filters)
            assert (ids, count, stats['total_tokens_saved']) == (['boundary'], 1, 200)
            print(f'{backend.__name__}: ids={ids}, count={count}, saved={stats["total_tokens_saved"]}')
```

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: Aware JSONL time filters now compare
in UTC rather than raising TypeError; naive UTC filters retain their
behavior.
- Kill switch / disable path: N/A.
- Unsafe override required: No.
- Qualification impact: Local storage filtering only; no provider
routing or compression changes.
- Rollback path: Revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A: no
public API or configuration change)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

The full suite is not green in this environment. Baseline failures
include CLI/deployment fixtures, local auth configuration, network/TLS
and macOS Unix socket paths. The two additional failures from the
revised full run did not reproduce individually in either tree; I have
not established their original state/order dependence. This PR is ready
for human review with the verification limits above explicitly
documented.

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-10-24 08:44:07 -07:00
charlieannaandJerrettDavis 91237cae70 fix(storage): page JSONL queries after sorting (#3872)
## Description

`JSONLStorage.query()` applied `limit` and `offset` while reading the
append-only file, then sorted only the truncated page. SQLite sorts the
full filtered set newest-first before paging, so the two storage
backends returned different records for the same page.

Closes #3822

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring

## Changes Made

- Sort the complete filtered JSONL result set newest-first before
applying offset and limit.
- Remove the early read-time truncation that returned old rows.
- Add comprehensive JSONL/SQLite parity coverage for multiple pages and
filters, plus focused JSONL tests.

## Testing

- [x] Unit tests pass
- [x] Linting passes
- [x] Type checking passes
- [x] New tests added
- [ ] Manual testing performed

### Test Output

```text
41 passed: tests/test_storage_backends.py tests/test_storage tests/test_storage/test_jsonl.py
ruff check: passed
ruff format --check: passed
mypy headroom/storage/jsonl.py: passed
```

## Real Behavior Proof

- Environment: macOS arm64, Python 3.12.14 using the headroom project
virtual environment; current main (`ffc6edb4`) merged into this branch.
The test uses local temporary JSONL and SQLite files, with no network or
provider credentials.
- Exact command / steps: From the repository root run `PYTHONPATH=.
/Users/ankurkothari/Documents/workspace/headroom/.venv/bin/python -m
pytest -q tests/test_storage_backends.py tests/test_storage`. The
regression fixture appends 250 timestamped requests oldest-first into
both storage backends, then queries `limit=5` at offsets 0, 5, and 10
and repeats with model, mode, and combined filters.
- Observed result: `41 passed` on the updated branch. Both backends
return identical newest-first pages; the first JSONL page starts at
`req-249`, and adjacent pages do not overlap. On the unfixed baseline,
JSONL's first page contains old records instead.
- Not tested: performance and peak memory for multi-gigabyte JSONL
files, or Windows path handling. This change necessarily scans the
filtered JSONL data before paging.

## Runtime Rollout Safety

- Rollout-managed feature(s): None.
- Minimum rollout channel: Stable; the existing JSONL backend receives
the correction without a feature flag.
- Stable/default behavior changed: JSONL pages now match the existing
SQLite newest-first contract.
- Kill switch / disable path: Not applicable.
- Unsafe override required: No.
- Qualification impact: Storage backend query parity and existing
storage tests; no provider or proxy change.
- Rollback path: Revert this PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review
- [x] I have added tests proving the fix
- [x] New and existing tests pass locally
- [x] I did not edit `CHANGELOG.md`

## Screenshots (if applicable)

Not applicable.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
2026-09-30 21:31:08 -05:00
chopratejas 967b0db439 fix: B1 — retire ICM, RollingWindow, scoring, relevance + dependents
Phase B step 1 of the live-zone-only realignment. Removes ~10K LOC of
"drop messages from history" machinery that became unreachable after
PR-A1 made `/v1/messages` a passthrough on the proxy. Live-zone-only
compression (PR-B2..B7) operates on content blocks within messages;
message-list mutation no longer happens in the pipeline.

Python deletes:
- headroom/transforms/intelligent_context.py (1077 LOC)
- headroom/transforms/rolling_window.py (395 LOC)
- headroom/transforms/progressive_summarizer.py (508 LOC)
- headroom/transforms/scoring.py (459 LOC)
- headroom/transforms/tool_crusher.py (338 LOC)
- 5 corresponding tests/test_transforms/* and tests/test_proxy_intelligent_context.py

Rust deletes:
- crates/headroom-core/src/context/* (manager, config, workspace,
  candidate, ccr_drop, strategy/, mod) + safety.rs replaced
- crates/headroom-core/src/scoring/* (mod, score, scorer, traits, weights)
- MessageScorerComparator from crates/headroom-parity (PR #338/#343
  becomes deletable; sunk cost stays sunk)
- 13 message_scorer fixtures + record_message_scorer.py

Rust adds (move + rewrite):
- crates/headroom-core/src/transforms/safety.rs — `tool_pair_indices`
  preserves the OpenAI/Anthropic tool_use ↔ tool_result pairing rule
  the live-zone dispatcher (PR-B2) needs. No IcmConfig dependency.

Surface refactors:
- HeadroomConfig: drop `tool_crusher`, `rolling_window`,
  `intelligent_context` fields; hoist `output_buffer_tokens` to top
  level (used by client.py).
- ProxyConfig: drop `intelligent_context*` fields.
- `headroom wrap` proxy server: retire IntelligentContextManager
  and RollingWindow imports + branch; pipeline is CacheAligner →
  ContentRouter (smart_routing) or CacheAligner → SmartCrusher
  (legacy).
- CLI: drop `--no-intelligent-context`, `--no-intelligent-scoring`,
  `--no-compress-first` flags.
- LangChain memory integration: rename `_apply_rolling_window` →
  `_apply_compression`, drop RollingWindowConfig dep. Threshold is
  now advisory — B6 will rework the contract.
- TransformPipeline.create_pipeline now takes only cache_aligner_config.
- headroom/__init__.py + headroom/transforms/__init__.py: strip
  exports of deleted symbols.

Bug fixes uncovered by full pytest sweep:
- providers/copilot/wrap.py: `environ or os.environ` collapsed
  empty-dict to falsy → callers passing `environ={}` accidentally
  pulled from os.environ. Use `environ if environ is not None else
  os.environ`.

Test correctness fixes:
- _DummyAnthropicHandler._retry_request gains **_kwargs to match
  the real handler signature post-A8.
- test_ws_http_fallback extracts JSON from `content=` (post-A3
  byte-faithful) rather than the obsolete `json=` kwarg.
- test_ccr_response_handler_extra fixture joins SSE events with
  `\n\n` per spec (post-A8 byte-buffer parser requirement).
- test_proxy_responses_phase_preservation: capture via direct
  handler attached to the named logger, so the assertion is
  order-independent (proxy `_setup_file_logging` flips
  `headroom.propagate=False` once any earlier test triggers it).
- conftest.py autouse fixture resets `headroom.propagate=True`
  before each test as a defensive measure for the same pollution.
- test_wrap_copilot_translated_backend_still_requires_byok:
  monkeypatch.delenv every provider key so the BYOK error
  actually fires.
- test_native_installers: skip when system bash < 4.3 (macOS ships 3.2).
- TestGeminiEmbedContent / TestGeminiBatchEmbedContents:
  pytest.mark.skip — proxy currently has no :embedContent route;
  feature gap, not regression.

Acceptance:
- cargo build --workspace + cargo clippy + cargo fmt --check: green.
- cargo test --workspace --exclude headroom-py: 777 passed.
- pytest: 4892 passed, 240 skipped, 0 failed.
- git grep returns only intentional comments referencing the deletion.

Per-PR-B1 plan: REALIGNMENT/04-phase-B-live-zone.md.
2026-05-02 12:23:17 -07:00
chopratejas e4a41faa33 Fix all ruff lint and format errors for CI
- Fix E402: Move module-level imports to top of file
- Fix F401: Add noqa for availability check imports
- Fix F402: Rename loop variables shadowing imports
- Fix E722: Replace bare except with except Exception
- Fix B904: Add exception chaining (from e)
- Fix F811: Remove duplicate imports
- Fix B027: Add noqa for empty close() method
- Fix E741: Rename ambiguous variable l -> label
- Fix I001: Import sorting issues
- Apply ruff format to all 106 files

All 902 tests pass.
2026-01-10 15:33:44 -08:00
chopratejas 9c7d4512d6 Initial commit: Headroom SDK - LLM context optimization toolkit
A comprehensive SDK for optimizing LLM context windows, reducing token
usage while preserving critical information for AI agents.

Core Features:
- SmartCrusher: Statistical compression of tool outputs (70-85% reduction)
- CacheAligner: Prefix optimization for prompt cache hits
- RollingWindow: Intelligent context window management
- BM25/Hybrid relevance scoring for smart item selection

Integrations:
- OpenAI and Anthropic provider support
- LangChain integration (ChatModel, Callbacks, Runnable)
- MCP (Model Context Protocol) integration for tool compression

Test Coverage:
- 372 tests passing across all modules
- 35 performance benchmarks
- Real-world agent evaluations with 88% token savings

Key Components:
- headroom/transforms/: Core compression transforms
- headroom/providers/: OpenAI and Anthropic support
- headroom/integrations/: LangChain and MCP integrations
- headroom/relevance/: BM25 and hybrid scoring
- headroom/pricing/: Model pricing registry
- benchmarks/: Performance benchmark suite
- examples/: Usage examples and demos
2026-01-06 23:16:58 -08:00