mirror of
https://github.com/akitaonrails/ai-memory.git
synced 2026-10-02 03:24:46 +08:00
fix(backfill): store each event's original time instead of the import time
Every session/observation `ai-memory backfill` imported was stamped with started_at/ended_at/created_at at import time, discarding each transcript event's own timestamp even though the workstream adapters already captured it (`NewWorkstreamEvent::occurred_at`) — a session imported today from a transcript recorded weeks ago showed up as having happened today. `NewSession` and `NewObservation` gain an optional `occurred_at` (microseconds); the store falls back to `now()` when it is `None`, so live hook capture is unaffected. This includes `admit_hook_session_event`'s own session-row INSERT (the real path a live `/hook` session-start event takes, separate from `begin_session_row`) — missing that one meant a backfilled session's `ended_at` (original, past) could sit before its `started_at` (import time). The `/hook` body accepts an RFC 3339 `occurred_at`, read from the top level of the body only (not the nested `payload`/`event`/`properties`/`info`/`path` search other hook fields use, so a harness payload that happens to carry an `occurred_at` key elsewhere in its own structure is never mistaken for this field). It is client-controlled input arriving over the hook endpoint, so `HookEnvelope::occurred_at_micros` bounds it (must be > 0 and no more than five minutes ahead of server time) before it is trusted; anything else — missing, unparsable, or out of bounds — resolves to `None` rather than erroring, keeping hooks fire-and-forget. It is numeric metadata, not text, so it never goes through the sanitizer. Backfill validates each transcript event's own timestamp (an unparsable one is treated as missing) and threads the resolved time through `map_event` and into the session-start/session-end items: a missing timestamp inherits the nearest preceding valid one, an event before the first valid timestamp inherits that first one, and the session's start/end times are the earliest/ latest valid event time in the transcript rather than assuming it is already time-sorted. Because a backfilled session's `ended_at` can land well in the past, it can sit below the auto-improve review watermark and the cross-session experience-pass anchor (both keyed on `ended_at`), so a freshly imported session may not get an automatic review pass until a newer session moves those forward; an opt-in retention window measured from an observation's own time can also make an old backfilled observation immediately prunable rather than only after it ages in place; and the "most recently active project" restart fallback, which looks at how recent observations are, may not pick a project that was just backfilled. These are documented consequences, not regressions introduced here — they follow directly from timestamps now being honest.
This commit is contained in:
@@ -252,6 +252,26 @@ and older clients cannot bypass it. The typed sanitizer boundary then applies a
|
||||
16 KiB backstop to every durable observation body after redaction. The
|
||||
separately gated Claude Code assistant/Stop excerpt remains capped at 2 KB.
|
||||
|
||||
An optional RFC 3339 `occurred_at` on the hook body lets a client supply the
|
||||
event's own original time — currently only `ai-memory backfill`, replaying a
|
||||
transcript's per-event timestamps, so imported sessions/observations are
|
||||
stamped with when they actually happened instead of import time. It is read
|
||||
from the top level of the body only (unlike the nested `payload`/`event`/
|
||||
`properties`/`info`/`path` search other hook fields use), so a real harness
|
||||
payload that happens to carry an `occurred_at` key somewhere in its own
|
||||
structure is never mistaken for this field. It is numeric metadata, not text,
|
||||
so it never goes through the sanitizer (outside invariant #6's boundary); it
|
||||
is still client-controlled input over `/hook`, so
|
||||
`HookEnvelope::occurred_at_micros` bounds it (must be > 0 and no more than
|
||||
five minutes ahead of server time) before trusting it. Anything else —
|
||||
missing, unparsable, or out of bounds — resolves to `None`, which the store
|
||||
treats as "now", never an error, keeping hooks fire-and-forget (invariant #5).
|
||||
A backfilled session's `ended_at` can therefore land well in the past, which
|
||||
can push it below the auto-improve watermark and the experience-pass anchor
|
||||
(both keyed on `ended_at`), and a retention window measured from an
|
||||
observation's own time can make an old backfilled observation immediately
|
||||
prunable rather than only after it ages in place.
|
||||
|
||||
## Storage architecture
|
||||
|
||||
**Two layers, one source of truth.**
|
||||
|
||||
Reference in New Issue
Block a user