Files
OpenMAIC/lib/runtime/learner-key.ts
T
wyuc 34448beb6c feat(storage): server-backed runtime — pluggable backend seam, HTTP contract, Postgres backend, reference server (#939) (#946)
* ci: run PR checks against the runtime-server-backend integration branch (#939)

* feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B) (#940)

* feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B)

- documented JSON HTTP contract for every RuntimeStore operation
  (server-assigned seq, learnerKey derived from auth — never trusted
  from the request, machine-readable error codes, idempotency notes)
- HttpRuntimeStore client with injected fetch + auth headers hook;
  session reads (including createSession responses) migrate forward
  on the runtime line so an older server cannot leak stale envelopes
- conformance test server bridging the contract onto the browser
  backend; full runRuntimeStoreContract green over HTTP plus error
  mapping cases

* fix(storage): harden HTTP runtime backend after cross-review (#939 Part B)

Cross-review fixes:
- appendRecord gates payloads through assertJsonValue: fail loud on
  values JSON cannot represent (Map, Date, NaN, nested undefined, NUL
  strings) instead of silently mangling them in transit; payload
  domain documented in the contract
- appendRecord/listRecords responses validate via validateRuntimeRecord
  (server-assigned seq is no longer trusted verbatim) and listRecords
  sorts by seq like listSessions already sorted defensively
- request headers no longer require an ambient Headers constructor
  when fetch is injected
- conformance server classifies errors structurally (existence checks,
  DSL validators, version stamps) instead of regex-sniffing messages
  that interpolate caller-controlled ids
- router preserves empty path segments so empty-key calls keep browser
  no-op semantics instead of shifting onto other routes
- stray runtimeDslVersion/seq in request bodies are ignored (store-
  assigned wins), matching the browser reference; docs updated
- loopback integration test exercises the real listening server
- docs state the conformance server is test-only; auth-derived
  learnerKey is Part D's reference server

* fix(storage): round-2 cross-review hardening for the HTTP backend (#939 Part B)

- json-value guard rewritten: additionally rejects -0, symbol-keyed and
  non-enumerable own properties, and non-index own properties on arrays;
  U+2028/U+2029 are accepted again (they round-trip through JSON per
  RFC 8259 — rejecting them lost legitimate pasted text)
- conformance server classifies racing duplicate creates as 409 via a
  structural post-check, gates payloads and merge learner keys as 400
- segment() rejects '.'/'..' ids the URL layer would fold away
- list responses must be arrays and mergeLearner's moved must be a
  finite number, else a typed MALFORMED_RESPONSE error

* fix(storage): round-3 cross-review hardening for the HTTP backend (#939 Part B)

- json-value guard: NUL is rejected in object keys too, and unpaired
  UTF-16 surrogates are rejected in string values and keys (jsonb
  refuses both; other JSON stacks corrupt lone surrogates to U+FFFD)
- typed storage errors survive non-object 200 responses instead of
  degrading into TypeErrors on .id access
- mergeLearner's moved must be a non-negative integer
- conformance server classifies missing/non-object request bodies as
  400 VALIDATION_FAILED
- createSession/appendRecord gate the full init envelope, not only the
  payload (stray Date/undefined properties, NUL in ids)

* fix(storage): round-4 convergence fixes for the HTTP backend (#939 Part B)

- envelope JSON gate tolerates explicitly-undefined optional anchors
  (sceneId: undefined behaves like omission, matching the browser)
- body-carried identifiers reject '.'/'..' so nothing persists that
  the URL layer cannot later address; conformance server mirrors it
- mergeLearner keys pass the JSON-domain gate on client and server
- json-value guard v4: rejects enumerable accessors (validation/serialize
  TOCTOU), Array subclasses and null-proto arrays, prototype-supplied
  array indices; sparse-array test now constructs a genuine hole

* fix(storage): round-5 convergence fixes for the HTTP backend (#939 Part B)

- undefined-stripping is limited to the DSL-declared optional anchors
  (sceneId/actionIndex/subAnchor); any other undefined member fails the
  JSON gate loud instead of being dropped silently
- json-value guard: array and object prototype checks are realm-agnostic
  (chain-shape instead of identity), so ordinary values from another
  realm are accepted while subclasses stay rejected; shared
  isLosslessJsonString predicate exported for SQL key guards
- conformance server merge route validates target-key addressability

* fix(storage): close the toJSON prototype channel in the JSON guard (#939)

toJSON is the single channel through which a prototype can alter JSON
output — prototype properties never serialize and own accessors are
already rejected — so refusing any value with a callable toJSON closes
prototype influence on serialization entirely, including prototypes
crafted to pass the realm-agnostic chain-shape check.

* fix(storage): probe toJSON via descriptor walk, not property read (#939)

Reading value.toJSON would execute an inherited accessor, letting a
stateful getter hide from validation and reappear at stringify time.
The probe now walks own descriptors up the prototype chain without
invoking any user code. Post-validation mutation of the caller's object
graph is documented as out of scope — it is equally unpreventable for
every other validated property.

* fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939)

The own-most descriptor decides: absent is safe, a non-callable data
value is an ordinary shadowing member and is safe, a callable data
value is rejected, and an accessor is rejected because it cannot be
inspected without invoking it. No daylight remains between the guard
and the serializer on this property.

* feat(storage): PgRuntimeStore — Postgres runtime backend (#939 Part C) (#941)

* feat(storage): PgRuntimeStore — Postgres runtime backend over an injected queryable (#939 Part C)

- PgRuntimeStore implements RuntimeStore over a minimal injected
  Queryable (node-postgres and PGlite both satisfy it) — the package
  keeps zero runtime dependencies beyond @openmaic/dsl
- runtime_sessions / runtime_records schema exported as
  RUNTIME_PG_SCHEMA with idempotent ensureSchema()
- appends serialize per session via a session-row lock; MAX(seq)+1
  and the insert share one transaction, UNIQUE(session_id, seq) plus
  bounded retry backstop non-cooperating writers
- envelope semantics mirror the browser backend: version stamping,
  validation gates, migrate-on-read, fail-loud on future-stamped rows
- full runRuntimeStoreContract green on PGlite plus PG-specific cases
  (concurrent-append seq atomicity, ensureSchema and mergeLearner
  idempotence)

* fix(storage): require pinned transactions + JSON payload gate in PG backend (#939 Part C)

Cross-review fixes:
- withTransaction is now required; the BEGIN/COMMIT fallback is removed
  (on a pg Pool it spread BEGIN/body/COMMIT across different connections
  — no real transaction, leaked idle-in-transaction clients, broken
  mergeLearner atomicity; on a shared pinned client concurrent calls
  interleaved transactions)
- single-statement deletes no longer wrap in a transaction hook call
- appendRecord gates payloads through assertJsonValue: fail loud on
  values JSON cannot represent (Map, Date, NaN, nested undefined, NUL
  strings) instead of silently persisting something different
- append retry also covers 40001/40P01; READ COMMITTED assumption
  documented at the retry loop
- ensureSchema documented as create-only; redundant stage index dropped
- deterministic interleaving test proves the 23505 retry path; real
  PostgreSQL contract lane added (postgres:16 service workflow, pg
  driver suite skipped locally without PG_CONTRACT_URL)

* fix(storage): round-2 cross-review hardening for the PG backend (#939 Part C)

- json-value guard rewritten (shared with the HTTP backend): additionally
  rejects -0, symbol-keyed and non-enumerable own properties, and
  non-index own properties on arrays; accepts U+2028/U+2029
- storage-pg-contract workflow now triggers for the runtime-server-backend
  integration branch and fails loud (STORAGE_PG_CONTRACT_REQUIRED=1)
  when PG_CONTRACT_URL is missing instead of silently skipping
- loadSession distinguishes corrupt non-object rows from absent rows so
  getSession fails loud instead of reporting the session missing

* fix(storage): round-3 cross-review hardening for the PG backend (#939 Part C)

- json-value guard (shared): NUL rejected in object keys, unpaired
  UTF-16 surrogates rejected in values and keys
- createSession/appendRecord gate the full persisted envelope through
  assertJsonValue, not only the payload — stray Date/undefined
  properties and NUL in ids fail loud instead of silently diverging
  from the returned record or leaking raw 22P05
- real-PG lane covers a genuine 23505 conflict from a second
  connection and recovery after an aborted transaction
- retry-set asymmetry and mergeLearner's unbounded lock set documented

* fix(storage): round-4 convergence fixes for the PG backend (#939 Part C)

- NUL/lone-surrogate lookup and delete keys resolve to absent/no-op
  instead of leaking 22021 driver errors; setSessionStatus reports the
  session missing; mergeLearner from-key moves 0
- mergeLearner destination key passes the JSON-domain gate fail-loud
- envelope JSON gate tolerates explicitly-undefined optional anchors,
  matching the browser backend
- json-value guard v4 (shared) + tests for accessors, Array subclasses,
  prototype-supplied indices

* fix(storage): round-5 convergence fixes for the PG backend (#939 Part C)

- appendRecord pre-checks the session key like every other lookup path,
  so a NUL/lone-surrogate sessionId reports 'no session' instead of
  leaking a 22021 driver error
- the queryable-key predicate now structurally reuses the shared
  isLosslessJsonString export instead of restating the string rule
- undefined-stripping limited to the DSL-declared optional anchors
- json-value guard: realm-agnostic prototype checks (shared)

* fix(storage): close the toJSON prototype channel in the JSON guard (#939)

Shared guard change with the HTTP backend branch; adds the crafted
null-proto-prototype regression test. Both final-gate reviewers
independently converged on this same channel.

* fix(storage): probe toJSON via descriptor walk, not property read (#939)

Reading value.toJSON would execute an inherited accessor, letting a
stateful getter hide from validation and reappear at stringify time.
The probe now walks own descriptors up the prototype chain without
invoking any user code. Post-validation mutation of the caller's object
graph is documented as out of scope — it is equally unpreventable for
every other validated property.

* fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939)

The own-most descriptor decides: absent is safe, a non-callable data
value is an ordinary shadowing member and is safe, a callable data
value is rejected, and an accessor is rejected because it cannot be
inspected without invoking it. No daylight remains between the guard
and the serializer on this property.

* fix(storage): conform HTTP/PG backends to the post-#926 RuntimeStore interface (#943)

The chat cutover (#926) added deleteAllRuntime() to the RuntimeStore
contract while the HTTP and Postgres backends were developed in
parallel against the pre-#926 interface. Implements the method on both
backends (single-statement wipe on PG via the FK cascade; DELETE
/runtime on the HTTP contract, documented as an operator-gated
administrative endpoint), restoring a green typecheck and contract
suite on the integration branch.

* feat(runtime): injectable RuntimeStore backend + learner-key provider (#939 Part A) (#944)

* wip(runtime): backend injection seam — pending gate verification

* fix(runtime): cross-review hardening for the storage injection seam

- stage-deletion IndexedDB probe applies only to the default browser
  backend; an injected store always receives deleteStageRuntime
- explicit kv argument takes precedence over the configured learner-key
  provider, matching the store seam's explicit-beats-global rule
- configured learner-key resolution is latched with in-flight dedup,
  mirroring the store singleton; identity changes require app-level
  handling
- factory retry semantics documented; seal errors explain the
  module-level bootstrap requirement; isRuntimeStorageConfigured()
  probe and a test-only reset added
- client-bootstrap-only contract documented (SSR/HMR caveats)

* fix(runtime): snapshot configuration and make the test reset complete

- configureRuntimeStorage copies the option fields so mutating the
  caller's object after configuring cannot swap the sealed backend or
  identity provider
- resetRuntimeStorageForTests now clears every latched consumer cache
  (store singleton, learner-key in-flight promise) via a reset-hook
  registry, so a reset-then-reconfigure test actually gets the new
  backend

* fix(runtime): reset also clears the default learner-key caches

resetRuntimeStorageForTests left defaultInFlight/defaultKv latched, so
a default-path test could leak its anonymous key or KV store into the
next test despite the documented full-reset promise.

* feat(storage): runtime reference server — auth-derived learnerKey (#939 Part D) (#945)

* wip(storage): reference server — pending deleteAllRuntime route + gates

* feat(storage): runtime reference server — auth-derived learnerKey enforcement (#939 Part D)

- createRuntimeHttpHandler(store, options) wires the documented HTTP
  contract onto any injected RuntimeStore over node http
- authenticate is required; every learner-scoped operation verifies the
  path/body learnerKey against the authenticated principal (403
  FORBIDDEN_LEARNER) — the client-supplied value is never trusted
- mergeLearner requires an explicit authorizeMerge grant and admin
  surfaces (stage cascade, DELETE /runtime) require authorizeAdmin;
  both default-deny
- runnable reference entry demonstrates a pg Pool withTransaction and
  bearer-token authentication, marked as demo-only
- contract suite green through HttpRuntimeStore -> listening reference
  handler -> PgRuntimeStore(pglite); security matrix tested (401/403
  paths, admin default-deny); threat model documented

* fix(storage): cross-review hardening for the reference server (#939 Part D)

- principal learnerKey is optional: admin/merge-only credentials no
  longer fabricate learner identity; learner-scoped routes 403 without
  ownership
- full session/record envelopes pass the JSON-domain gate at the
  handler, so NUL/lone-surrogate identifiers map to 400 instead of 500
- payload validation follows the injected store's validator map
  (options.payloadValidators) instead of imposing DSL defaults
- reference factory accepts authenticate/authorizeMerge/authorizeAdmin/
  payloadValidators overrides; docs no longer claim the factory binds
  to localhost; demo-impersonation warning hardened
- 500 responses carry a generic message; details go to the server log
- ownership checks precede version checks and unowned sessions read as
  404, closing existence/version oracles; cross-learner denial matrix
  tested per route
- merge/delete concurrency documented as linearizable-equivalent with
  the ownership re-check narrowed to the delete call

* fix(storage): align reference-server semantics with the store contract

- future-stamped sessions read and delete through unchanged; 409
  FUTURE_VERSION applies only to guarded writes (status, append, merge)
- check-then-write races reclassify structurally via a post-failure
  re-fetch: missing session 404, non-active session 400, never a
  message-sniffed or generic 500

* fix(storage): close the reference CLI's pg pool on startup failure

ensureSchema opens connections inside createReferenceRuntimeServer, so
a failed schema init or occupied port left the pool holding database
resources until its idle timeout.

* fix(storage): close the records-route existence oracle

cosarah's review point on #946: the records list answered 200 [] for an
absent session but 404 for another learner's, so the 404 leaked that an
id exists. Absent and foreign sessions now answer identically (404
SESSION_NOT_FOUND) and the HTTP client maps that code back to an empty
list, preserving the store contract's absent-lists-as-empty semantics.
Contract doc records the server MAY/SHOULD and the client MUST.
2026-07-17 16:15:57 +08:00

102 lines
4.1 KiB
TypeScript

/**
* Device-anonymous learner identity for the runtime layer (#869).
*
* `learnerKey` partitions all learner-runtime data (RuntimeStore sessions).
* Until sign-in exists it is a per-device anonymous key: minted once and kept
* in the KV `device` scope, which never syncs across devices — a synced key
* would merge two people's runtime into one partition. When sign-in lands,
* `RuntimeStore.mergeLearner(anonKey, accountKey)` is the migration path.
*
* Client-only: the default KV store lazily touches `localStorage`. Server
* code must not import this without injecting its own `KVStore`.
*/
import { BrowserKVStore, type KVStore } from '@openmaic/storage';
import { registerRuntimeStorageResetHook, resolveConfiguredLearnerKey } from './config';
export const LEARNER_KEY_KV_KEY = 'runtime.learnerKey';
const LEARNER_KEY_LOCK = 'maic:learner-key';
let defaultKv: KVStore | undefined;
let defaultInFlight: Promise<string> | undefined;
let configuredInFlight: Promise<string> | undefined;
registerRuntimeStorageResetHook(() => {
configuredInFlight = undefined;
defaultInFlight = undefined;
defaultKv = undefined;
});
function mintLearnerKey(): string {
const uuid =
typeof crypto !== 'undefined' && 'randomUUID' in crypto
? crypto.randomUUID()
: `${Date.now()}-${Math.random().toString(36).slice(2)}`;
return `anon:${uuid}`;
}
async function mintPersisted(store: KVStore): Promise<string> {
const minted = mintLearnerKey();
await store.set(LEARNER_KEY_KV_KEY, minted, 'device');
// Return the PERSISTED value, not the local mint: if another writer raced
// us, everyone converges on the stored winner instead of keeping an
// orphaned key of their own.
return (await store.get<string>(LEARNER_KEY_KV_KEY, 'device')) ?? minted;
}
async function readOrMint(store: KVStore): Promise<string> {
const existing = await store.get<string>(LEARNER_KEY_KV_KEY, 'device');
if (existing) return existing;
if (typeof navigator !== 'undefined' && navigator.locks) {
// Cross-tab mutual exclusion: only one tab mints under the lock; a tab
// that lost the race re-reads the winner's key inside its own grant. An
// existing key is never overwritten, so the per-tab memo stays safe.
return await navigator.locks.request(LEARNER_KEY_LOCK, async () => {
const won = await store.get<string>(LEARNER_KEY_KV_KEY, 'device');
return won ?? mintPersisted(store);
});
}
// No Web Locks (older browsers, non-window contexts): read-after-write
// only. Residual race: a tab whose re-read lands before another tab's
// write keeps an orphaned key. Accepted here — it merely splits one
// anonymous learner's local history, and these contexts are rare.
return mintPersisted(store);
}
/**
* Resolve the client session's learner partition key.
*
* An explicit KV store takes priority over app-wide configuration. Without an
* explicit store, a configured provider is invoked only on first resolution:
* concurrent calls share its in-flight promise and the first resolved value is
* retained for the session. Identity changes mid-session belong in the
* application layer (reload or a `mergeLearner` flow).
*
* Client-only: this singleton is not SSR-safe or intended for request-scoped
* identity. Partial dev-mode HMR can recreate this cache independently of the
* configuration module; reload after changing bootstrap configuration.
*/
export function getLearnerKey(kv?: KVStore): Promise<string> {
// Injected stores (tests, server-side callers) bypass the memo but stay
// race-safe through the lock / read-after-write above.
if (kv) return readOrMint(kv);
const configured = configuredInFlight ?? resolveConfiguredLearnerKey();
if (configured) {
configuredInFlight ??= configured.catch((error) => {
configuredInFlight = undefined;
throw error;
});
return configuredInFlight;
}
// Concurrent same-bundle callers share one in-flight read/mint. A failure
// is not cached — a transient storage error must not pin every later call.
defaultInFlight ??= readOrMint((defaultKv ??= new BrowserKVStore())).catch((error) => {
defaultInFlight = undefined;
throw error;
});
return defaultInFlight;
}