fix(backfill): store each event's original time instead of the import time

Every session/observation `ai-memory backfill` imported was stamped with
started_at/ended_at/created_at at import time, discarding each transcript
event's own timestamp even though the workstream adapters already captured
it (`NewWorkstreamEvent::occurred_at`) — a session imported today from a
transcript recorded weeks ago showed up as having happened today.

`NewSession` and `NewObservation` gain an optional `occurred_at`
(microseconds); the store falls back to `now()` when it is `None`, so live
hook capture is unaffected. This includes `admit_hook_session_event`'s own
session-row INSERT (the real path a live `/hook` session-start event takes,
separate from `begin_session_row`) — missing that one meant a backfilled
session's `ended_at` (original, past) could sit before its `started_at`
(import time).

The `/hook` body accepts an RFC 3339 `occurred_at`, read from the top level
of the body only (not the nested `payload`/`event`/`properties`/`info`/`path`
search other hook fields use, so a harness payload that happens to carry an
`occurred_at` key elsewhere in its own structure is never mistaken for this
field). It is client-controlled input arriving over the hook endpoint, so
`HookEnvelope::occurred_at_micros` bounds it (must be > 0 and no more than
five minutes ahead of server time) before it is trusted; anything else —
missing, unparsable, or out of bounds — resolves to `None` rather than
erroring, keeping hooks fire-and-forget. It is numeric metadata, not text, so
it never goes through the sanitizer.

Backfill validates each transcript event's own timestamp (an unparsable one
is treated as missing) and threads the resolved time through `map_event` and
into the session-start/session-end items: a missing timestamp inherits the
nearest preceding valid one, an event before the first valid timestamp
inherits that first one, and the session's start/end times are the earliest/
latest valid event time in the transcript rather than assuming it is already
time-sorted.

Because a backfilled session's `ended_at` can land well in the past, it can
sit below the auto-improve review watermark and the cross-session
experience-pass anchor (both keyed on `ended_at`), so a freshly imported
session may not get an automatic review pass until a newer session moves
those forward; an opt-in retention window measured from an observation's own
time can also make an old backfilled observation immediately prunable rather
than only after it ages in place; and the "most recently active project"
restart fallback, which looks at how recent observations are, may not pick a
project that was just backfilled. These are documented consequences, not
regressions introduced here — they follow directly from timestamps now being
honest.
This commit is contained in:
Maxsuel Einstein
2026-09-25 15:27:03 -03:00
parent 1417119eec
commit 191a8eea11
34 changed files with 928 additions and 30 deletions
+20
View File
@@ -252,6 +252,26 @@ and older clients cannot bypass it. The typed sanitizer boundary then applies a
16 KiB backstop to every durable observation body after redaction. The
separately gated Claude Code assistant/Stop excerpt remains capped at 2 KB.
An optional RFC 3339 `occurred_at` on the hook body lets a client supply the
event's own original time — currently only `ai-memory backfill`, replaying a
transcript's per-event timestamps, so imported sessions/observations are
stamped with when they actually happened instead of import time. It is read
from the top level of the body only (unlike the nested `payload`/`event`/
`properties`/`info`/`path` search other hook fields use), so a real harness
payload that happens to carry an `occurred_at` key somewhere in its own
structure is never mistaken for this field. It is numeric metadata, not text,
so it never goes through the sanitizer (outside invariant #6's boundary); it
is still client-controlled input over `/hook`, so
`HookEnvelope::occurred_at_micros` bounds it (must be > 0 and no more than
five minutes ahead of server time) before trusting it. Anything else —
missing, unparsable, or out of bounds — resolves to `None`, which the store
treats as "now", never an error, keeping hooks fire-and-forget (invariant #5).
A backfilled session's `ended_at` can therefore land well in the past, which
can push it below the auto-improve watermark and the experience-pass anchor
(both keyed on `ended_at`), and a retention window measured from an
observation's own time can make an old backfilled observation immediately
prunable rather than only after it ages in place.
## Storage architecture
**Two layers, one source of truth.**