mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(storage): normalize timezone-aware JSONL time filters (#4063)
## Description Passing a timezone-aware datetime to JSONLStorage.query(), count() or get_summary_stats() raises TypeError. Stored rows are read as naive UTC, but the filter was compared directly with them. SQLite already normalizes its filter through format_timestamp(). A private `_utc_bound()` helper normalizes JSONL boundaries once before iteration using `astimezone(timezone.utc).replace(tzinfo=None)`, matching the conversion in `format_timestamp()` without a string round trip. This preserves the existing naive-UTC contract and microseconds without changing the reader or SQLite. Closes #4062. Related: #3116 fixed UTC timestamp serialization; this addresses the remaining JSONL filter comparison. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Normalize non-null start/end boundaries in the three JSONL filter entry points. - Add three focused JSONL regression cases, one per method, covering +05:30 start-only, end-only, and equal start/end bounds. Records one microsecond before and after the boundary prove inclusive bounds and preserved precision. ## Testing - [x] New tests added for new functionality - [x] Manual testing performed - [ ] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) ### Necessary boundary coverage follow-up (e9f33fe5) - Retain three parameterized method cases. Each checks start-only, end-only, and equal start/end bounds, inclusive boundaries, and microsecond precision. - Storage and timestamp utility tests: 44 passed. Ruff, formatting, and whitespace checks passed. - This preserves normal single-bound API usage while keeping the reduced suite requested by the maintainer. - Full Python suite was not rerun; hosted checks for this head remain separate from local evidence. ### Maintainer-requested revision validation (f206b72c) - Storage suite: 37 passed, including all three focused regression cases. - The three focused cases fail with the original pre-fix source (TypeError). - Ruff, formatting, whitespace checks and targeted mypy passed. - Tests import the current workspace source with published native dependencies. Editable source installation was blocked by the local Cargo registry mirror connection; this revision does not change native code. - Full Python suite was not rerun for this revision. The evidence below describes the original revision and its larger test matrix. ### Original revision test output ```text Storage + timestamp baseline: 51 passed With the new cases and original implementation: 36 failed, 60 passed After fix: 147 passed Fix-only stash regression check: 36 failed, 60 passed 96 new cases: 96 passed Storage/timestamp suite after source-extension install: 147 passed make ci-precheck: passed (Rust gates, editable source install, 177 Python passed / 4 skipped, commitlint) ruff check .: All checks passed! ruff format --check headroom/storage/jsonl.py tests/test_storage/test_jsonl.py: 2 files already formatted mypy headroom/storage/jsonl.py --follow-imports=silent: Success pre-commit run --files headroom/storage/jsonl.py tests/test_storage/test_jsonl.py: all six hooks passed (including isolated mypy) mypy headroom: one pre-existing truthy-function error in headroom/integrations/langchain/chat_model.py:394, reproduced with the fix stashed cargo fmt --all -- --check: passed cargo clippy --workspace --locked -- -D warnings: passed cargo test --workspace --locked: 1655 passed, 3 ignored Full suite after fix: 43 failed, 16306 passed, 718 skipped, 4 errors (2302.22s) Full baseline: 41 failed, 16212 passed, 718 skipped, 4 errors (2026.35s) All 41 baseline failures and 4 errors also occur after the fix. Two revised-only failures (bundled tool install and Groq pricing) pass on individual reruns in both trees; their original state/order dependence is not proven. Collection delta: 96 new cases; full-run delta: 94 passes + 2 transient failures. ``` ## Real Behavior Proof - Environment: macOS 26.3.1 arm64, Python 3.11.5, real JSONL files and SQLite databases, model field gpt-4o; no LLM provider calls or storage mocks. - Exact command / steps: PYTHONPATH="$PWD" .venv/bin/python /tmp/headroom-gc-ship/real_behavior.py; the self-contained script is included below. - Observed result: Both backends return only the boundary record, count 1 and token savings 200 for a +05:30 start and -07:00 end representing the same UTC instant; JSONL raised TypeError before the fix. - Not tested: Windows or live provider traffic. The full Python baseline comparison uses the same published v0.40.0 native wheel in both runs; a wheel built from this source revision also succeeds. After installing the source extension (SHA-256 matched the locally built wheel), the 147 storage/timestamp tests, disk-backed proof and make ci-precheck passed. ```text JSONLStorage: ids=['boundary'], count=1, saved=200 SQLiteStorage: ids=['boundary'], count=1, saved=200 ``` ### Reproducible disk-backed check ```python from datetime import datetime, timedelta, timezone from tempfile import TemporaryDirectory from headroom.config import RequestMetrics from headroom.storage import JSONLStorage, SQLiteStorage boundary = datetime(2026, 4, 23, 23, 59, 59, 500000, tzinfo=timezone.utc) start = boundary.astimezone(timezone(timedelta(hours=5, minutes=30))) end = boundary.astimezone(timezone(timedelta(hours=-7))) with TemporaryDirectory() as directory: for backend in (JSONLStorage, SQLiteStorage): with backend(f'{directory}/{backend.__name__}') as storage: for name, seconds in [('before', -1), ('boundary', 0), ('after', 1)]: storage.save(RequestMetrics( request_id=name, timestamp=boundary + timedelta(seconds=seconds), model='gpt-4o', stream=False, mode='audit', tokens_input_before=1000, tokens_input_after=800, )) filters = dict(start_time=start, end_time=end) ids = [row.request_id for row in storage.query(**filters)] count = storage.count(**filters) stats = storage.get_summary_stats(**filters) assert (ids, count, stats['total_tokens_saved']) == (['boundary'], 1, 200) print(f'{backend.__name__}: ids={ids}, count={count}, saved={stats["total_tokens_saved"]}') ``` ## Runtime Rollout Safety - Rollout-managed feature(s): None. - Minimum rollout channel: N/A. - Stable/default behavior changed: Aware JSONL time filters now compare in UTC rather than raising TypeError; naive UTC filters retain their behavior. - Kill switch / disable path: N/A. - Unsafe override required: No. - Qualification impact: Local storage filtering only; no provider routing or compression changes. - Rollback path: Revert this commit. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation (N/A: no public API or configuration change) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes The full suite is not green in this environment. Baseline failures include CLI/deployment fixtures, local auth configuration, network/TLS and macOS Unix socket paths. The two additional failures from the revised full run did not reproduce individually in either tree; I have not established their original state/order dependence. This PR is ready for human review with the verification limits above explicitly documented. --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
Codex
JD Davis
parent
11f743dd90
commit
bf34ccf588
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
import heapq
|
||||
import json
|
||||
from collections.abc import Iterator
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
@@ -14,6 +14,13 @@ from ..utils import format_timestamp, parse_timestamp
|
||||
from .base import Storage
|
||||
|
||||
|
||||
def _utc_bound(dt: datetime | None) -> datetime | None:
|
||||
"""Normalize a filter bound to the naive UTC used by stored timestamps."""
|
||||
if dt is not None and dt.tzinfo is not None:
|
||||
return dt.astimezone(timezone.utc).replace(tzinfo=None)
|
||||
return dt
|
||||
|
||||
|
||||
class JSONLStorage(Storage):
|
||||
"""JSONL file-based metrics storage."""
|
||||
|
||||
@@ -130,6 +137,10 @@ class JSONLStorage(Storage):
|
||||
if limit == 0:
|
||||
return []
|
||||
|
||||
# Stored timestamps are naive UTC; apply the same normalization to filters.
|
||||
start_time = _utc_bound(start_time)
|
||||
end_time = _utc_bound(end_time)
|
||||
|
||||
def _matches(metrics: RequestMetrics) -> bool:
|
||||
if start_time is not None and metrics.timestamp < start_time:
|
||||
return False
|
||||
@@ -156,6 +167,9 @@ class JSONLStorage(Storage):
|
||||
mode: str | None = None,
|
||||
) -> int:
|
||||
"""Count metrics matching filters."""
|
||||
start_time = _utc_bound(start_time)
|
||||
end_time = _utc_bound(end_time)
|
||||
|
||||
count = 0
|
||||
|
||||
for metrics in self.iter_all():
|
||||
@@ -194,6 +208,9 @@ class JSONLStorage(Storage):
|
||||
end_time: datetime | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Get summary statistics."""
|
||||
start_time = _utc_bound(start_time)
|
||||
end_time = _utc_bound(end_time)
|
||||
|
||||
total_requests = 0
|
||||
total_tokens_before = 0
|
||||
total_tokens_after = 0
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Tests for JSONL storage query paging (shared newest-first contract)."""
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -64,6 +65,31 @@ def _ids(metrics: list[RequestMetrics]) -> list[str]:
|
||||
return [m.request_id for m in metrics]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("method", ["query", "count", "get_summary_stats"])
|
||||
def test_jsonl_aware_time_bounds_are_inclusive(method: str, tmp_path: Path) -> None:
|
||||
"""Aware bounds preserve their UTC instant and include the exact boundary."""
|
||||
boundary = datetime(2026, 4, 23, 23, 59, 59, 500000, tzinfo=timezone.utc)
|
||||
local_boundary = boundary.astimezone(timezone(timedelta(hours=5, minutes=30)))
|
||||
with JSONLStorage(str(tmp_path / "metrics.jsonl")) as storage:
|
||||
for request_id, delta in [("before", -1), ("boundary", 0), ("after", 1)]:
|
||||
storage.save(_metrics(request_id, boundary + timedelta(microseconds=delta)))
|
||||
scenarios = [
|
||||
({"start_time": local_boundary}, ["after", "boundary"]),
|
||||
({"end_time": local_boundary}, ["boundary", "before"]),
|
||||
({"start_time": local_boundary, "end_time": local_boundary}, ["boundary"]),
|
||||
]
|
||||
for filters, expected_ids in scenarios:
|
||||
result = getattr(storage, method)(**filters)
|
||||
if method == "query":
|
||||
assert _ids(result) == expected_ids, filters
|
||||
elif method == "count":
|
||||
assert result == len(expected_ids), filters
|
||||
else:
|
||||
assert result["total_requests"] == len(expected_ids), filters
|
||||
assert result["total_tokens_before"] == 1000 * len(expected_ids), filters
|
||||
assert result["total_tokens_after"] == 800 * len(expected_ids), filters
|
||||
|
||||
|
||||
def _assert_timestamp_desc(metrics: list[RequestMetrics]) -> None:
|
||||
stamps = [m.timestamp for m in metrics]
|
||||
assert stamps == sorted(stamps, reverse=True)
|
||||
|
||||
Reference in New Issue
Block a user