mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-18 05:40:25 +08:00
@openmaic/dsl@0.6.2
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
b4e3145c90 |
feat(roster): single-source the generated agent roster on the stage document (#994)
* feat(roster): single-source the generated agent roster on the stage document The generated agent roster used to live in two places at once: the stage document carried generatedAgentConfigs, while a per-stage IndexedDB mirror (db.generatedAgents) was the actual read authority — and the only home of each agent's voice binding. The split caused real losses: voices vanished on device switches, exports, and shares (the mirror never travels), the load path trusted a device-local table over the portable document, and roster edits persisted through a bespoke debounce with no beforeunload flush. Make the stage document the single source of truth: - Contract: add optional voiceConfig/voiceDesign to GeneratedAgentConfig in @openmaic/dsl (VoiceDesign moves into the contract; the app re-exports it). Additive optional fields — not a breaking serialized-shape change, so DSL_VERSION stays put per the version policy. - Writes: remove every mirror writer (registry saveGeneratedAgents, the stage store's debouncedSaveAgents bypass, classroom-load hydration, import direct writes). setStageAgents now only marks the stage dirty for the shared persistence scheduler plus synchronous in-memory registry/selection mirrors; generation and import embed the roster on the stage. - Reads: classroom load hydrates the registry from the loaded stage's generatedAgentConfigs; export reads the stage roster directly. - Lazy migration: the mirror is retained read-only. On load, a roster (or voice fields) missing from the document is backfilled from the mirror, committed onto the in-memory stage, and persisted by the next flush — idempotent, and gated on the current stage id after every await so a classroom switch cannot leak a stale roster into the registry. - Deletion: deleteStageData discards the deleted stage's pending persistence work so a queued flush cannot resurrect the document. - Round trip: classroom ZIP manifests now carry the voice fields, so export/import preserves agent voices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): harden deletion, migration, selection, and ingress paths Review hardening on top of the roster single-source change: - Deletion vs in-flight persistence: deleting a stage now registers a session-scoped tombstone (lib/utils/deleted-stages.ts) before the cascade starts. Every persistence landing point checks it — the scheduled flush and the departing-stage retry drop their snapshots, and the aggregate/incremental saves re-check under the document lock, including the incremental path's full-save fallback that would otherwise rebuild a deleted document from the in-memory snapshot. A deletion that fails before the document is removed lifts the tombstone so the still-existing stage keeps persisting. - Lazy migration termination: a fruitless legacy-mirror probe (nothing to merge — e.g. server-generated rosters, which are voiceless by design) is remembered per stage for the session instead of re-querying the mirror on every load forever. Successful merges are not memoized so a failed flush is retried on the next load. Docstrings now state the real lifecycle. - Selection provenance: setStageAgents no longer overwrites a user-set agent selection with the full roster. Stage-derived selections still track the roster; a user-set auto selection is only narrowed to surviving agents (newcomers are not auto-selected, the user-set flag is untouched), and a user-set preset selection is left alone — matching restoreAgentSelection. - Mirror hygiene: deleteStageData clears the deleted stage's rows from the legacy roster mirror (best-effort), and the mirror table's docs now describe its actual access pattern (migration reads + deletion hygiene). - Registry persistence: the agent registry's persist snapshot partializes generated agents out of localStorage, making the stage document the roster's only durable home; the rehydration merge filter remains as defense in depth. - Ingress validation: imported manifest voice fields are structurally validated (malformed bindings dropped per field, the agent survives), and applyGeneratedAgentsToRegistry validates providerId against the known TTS provider registry instead of casting, treating unknown providers as "no bound voice". - Contract: drop the speculative AgentVoiceConfig.modelId — an audit of every roster producer, current and historical, found none that ever emitted it. Contract fields are added once a producer exists, not before. New tests cover the deletion tombstone at both the store and storage layers (including the missing-destination full-save pin and the lock-race re-check), the fruitless-probe memo, the selection-provenance matrix, persistence exclusion of generated agents, and voice-field sanitization on import. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): close tombstone lifecycle, probe-memo, and selection-intent gaps Second hardening pass over the deletion/migration/selection fixes: - Tombstone lifecycle: a deleted classroom id can legitimately come back (deletion is client-side only). Explicit (re)creation points now lift the tombstone — server-copy restore in applyClassroomStageAndScenes and backup restore in importDatabase — so edits after a same-session restore persist again, while in-flight flushes stay fenced. - Failed deletion now restores the dirt it discarded: deleteStageData snapshots the pending map plus the in-flight round before tombstoning, and re-marks it when the delete fails with the document still present. - Legacy-mirror probe memo now distinguishes a FAILED read (null, retried on the next load) from a confirmed-empty mirror (memoized), so a transient IndexedDB error cannot suppress migration for the whole session. - Incremental save tail (currentScene KV row + chat sessions) is tombstone- fenced like the aggregate path, and both paths re-check the tombstone immediately before every write, with comments scoped honestly for the lock-free (no Web Locks) best-effort LWW fallback. - Roster-edit selection mirror: an empty intersection now falls back to the full roster and clears the user-set flag (matching restoreAgentSelection's length > 0 gate), and a user-set selection that equals the pre-edit full roster keeps tracking the roster wholesale so newcomers are not excluded forever by the AgentBar auto-toggle snapshot. - TTS/ASR provider lookups use Object.hasOwn instead of `in`/bare indexing, so prototype-chain keys ('toString', 'constructor', ...) no longer pass the provider whitelist or resolve to Object.prototype members. - Documented the per-tab limit of the tombstone set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): replace the boolean deletion tombstone with a deletion-generation model A boolean tombstone cannot express the deletion lifecycle once a deleted id legitimately comes back (server-copy restore on revisit, backup import): after the restore lifts the flag, a pre-delete flush still in flight is indistinguishable from a post-restore edit and can overwrite the restored document with pre-delete content. Two rounds of point fixes kept leaking variants of this race, so this restructures the fence instead. deleted-stages.ts now keeps a per-stage monotonic deletion epoch plus a deleted flag. markStageDeleted bumps the epoch; unmarkStageDeleted clears only the flag and never rewinds the epoch. Every persistence path captures the epoch at the moment it captures the data it will write (flush-round snapshot, departing-stage snapshot, aggregate save entry) and re-checks "captured epoch is current AND not deleted" immediately before each individual write. Invariants: - A write captured before a deletion can never land after it, even when a same-id restore has lifted the deleted flag (the captured epoch is permanently stale). - A write capturing data after a restore observes the current epoch and persists normally. - A failed deletion lifts only the flag; the restored pending changes are re-queued as descriptors, so their flush re-captures current store state under the current epoch and is neither dropped nor a stale replay. Also fixed in the same restructuring: - Incremental tail writes (currentScene, chats) each re-check independently; a delete landing while the first tail write is awaiting now fences the chat write instead of riding the earlier check. - Deleting a classroom evicts it from the warm in-memory store, and loadFromStorage treats a warm-but-deleted stage as not loaded (discarding the ghost), so navigating Back to a deleted classroom reaches the server-restore path instead of rendering an editable ghost whose every edit is silently dropped. - importDatabase records each document's pre-import deletion state and reinstates it when a failed import rolls the document back, so an outstanding flush cannot recreate the rolled-back document. - Docstrings on the failed-deletion restore path now state that only dirt captured before the deletion is restored; edits refused during the deletion window are not part of the snapshot. New tests cover the epoch invariants at the storage layer (pre-delete rounds dropped across restores on the incremental, aggregate, and tail paths; post-restore captures landing; strictly increasing epochs), the scheduler layer (departing-retry delete+restore straddle; restored dirt flushing under the current epoch), the warm-ghost restore driven through the real loadFromStorage, and the import lift/rollback pair. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): report fenced drops, gate ghost eviction on cascade settlement Edge hardening around the deletion-epoch fence; the epoch model itself is unchanged. - An epoch-stale drop is now reported, not disguised: saveStageData / saveStageDataIncremental return a distinct 'stale-dropped' status (entry check, every in-mutation re-check, and both tail-write re-checks) instead of a shape identical to success. saveToStorage reacts by skipping all success bookkeeping - no chatSnapshot rebind to a snapshot that never landed, no pending-dirt clearing - and returns false, keeping its "true means verified write" contract honest. persistDirtySnapshot keeps its existing (correct) semantics explicitly: a drop has nothing to retry. - Warm-ghost eviction is settlement-gated: the deletion cascade records an in-flight bit (begun by deleteStageData, settled in its finally), and the deleted-warm branch of loadFromStorage keeps the warm state while the cascade is unsettled - a delete can still fail before removing the document, in which case the warm state plus the restored pending dirt is the only copy of the pre-delete edits. Once settled, the ghost discard and server-restore path behave as before. - Read-side landing re-check: loadFromStorage re-checks isStageDeleted after hydration, immediately before set(), mirroring the write-side "re-check immediately before landing" discipline - a lock-free mid-cascade read can no longer re-materialize a ghost classroom. - clearStoreForDeletedStage re-checks isStageDeleted at eviction time, so a same-id restore completing inside the cascade-tail window is never wiped. - capturedEpoch is now a required parameter on both storage entry points: the default (call-time capture) silently reopened the capture-point/ validation-point split for future callers; the type system now enforces the pairing. All production callers already passed it explicitly. New tests cover the dropped-status contract, the cascade in-flight lifecycle (including both failure shapes), keep-warm during an unsettled delete with end-to-end pending restore after a failed delete, settlement- then-eviction on success, the eviction guard against a tail-window restore, and the read-side hydration re-check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): settle the deleted-warm load by awaiting the cascade outcome Hardening around the deletion settlement gate introduced previously; the epoch model is unchanged. - The deleted-warm branch of loadFromStorage no longer completes the load while the deletion cascade is undecided (which exposed a classroom whose edits were refused without being restorable, and left a successful deletion's eviction with no follow-up reload). It now parks on a per-stage settlement promise (stageDeletionSettled) and branches on the outcome: a failed delete keeps the warm, restored classroom; a successful delete falls through to a full cold load so the server-restore path recovers the route; a navigation that moved on during the park is stopped by the load token guard. The eviction (clearStoreForDeletedStage) deliberately stops claiming a new load token so the parked load stays current - ghost re-materialization is fenced by the read-side deleted re-checks instead. No deadlock: a parked load holds no document lock, and the cascade never waits on a load. - 'stale-dropped' now propagates through the debounced flush path: persistDirtySnapshot returns the sentinel instead of an empty failure set, and startFlushRound skips the chatSnapshot rebind on it (mirroring saveToStorage) - a falsely rebound baseline would let later chat saves no-op-skip chats that never landed and corrupt cross-tab conflict detection. Pending-map clearing keeps its empty-failure semantics (restored descriptors mint fresh revisions). - The cascade in-flight state is a counter (begin++/settle--) instead of a boolean, and deleteStageData is single-flight per stage (a concurrent second call joins the first cascade), so overlapping deletions can never expose an undecided cascade as settled. - The StaleDroppedSave docstring no longer overpromises: a tail saveCurrentScene can land after the cascade's clearCurrentScene; the orphaned cursor row is ignored by the load path rather than removed by the cascade. Net +8 tests: overlapping-cascade counting, settlement-promise resolution (success and failure), concurrent double-delete single-flight join, the parked mid-cascade Back in both outcomes plus the navigation-away guard, and the fenced flush round keeping an honest chat baseline end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): re-verify store identity on keep-warm resume, re-run joined deletes after a mid-cascade restore Three hardening refinements on the deletion/persistence fencing: - loadFromStorage's parked keep-warm branch now mirrors the success path's store-identity guard: if a tokenless writer (e.g. a database import) replaced the stage during the park, a failed deletion falls through to the cold load instead of reporting another classroom as the kept warm state. The ghost-discard block is correspondingly gated on the deleted flag so the fall-through does not mislabel a surviving document as a ghost. - deleteStageData: a joined call carries its own deletion intent. When the first cascade fulfills but a same-id restore lifted the deleted flag in the meantime, the joined caller now runs one fresh cascade against the restored document instead of reporting the pre-restore outcome. Exactly one re-check per call (no recursion); a rejected first cascade still propagates unchanged to every joined caller. - Narrowed the failed-delete restore-ownership comments: the restore covers only the pending map plus an in-flight flush round's dirt; a departing-stage snapshot is outside that capture and is fenced and dropped by design. Comment-only, no behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): gate deleted-warm handling on stage identity, keep an explicit empty roster authoritative Final-audit fixes, three items: - loadFromStorage keyed its deleted-warm settlement handling (park / keep-warm / ghost discard) behind the scenes.length > 0 shortcut, so a zero-scene warm ghost of a deleted classroom skipped it entirely: the cold load found no document, the ghost stayed in the store, and the classroom loader's server-fallback gate (!getCurrentStage()) never ran — the silently-uneditable trap this branch exists to close, in its scene-less form. Deletion handling now keys on stage identity alone, before any scene-count shortcut; the non-deleted warm skip still requires scenes. - rosterNeedsLegacyFallback collapsed "roster field absent" (document predates roster persistence) with "explicitly persisted empty roster". A future writer persisting [] on a device whose read-only mirror still held stale rows would have its emptied roster resurrected on every load. The full-lift branch now triggers only when the field is absent (undefined); an explicit [] is authoritative and never probes the mirror. No current writer produces [], but the read side no longer depends on that invariant. - The GeneratedAgentConfig docstring claimed the voice fields are not a breaking change; that only holds for this codebase's tolerant structural validators. The generated stage.schema.json sets additionalProperties: false, so consumers pinning an older published schema artifact reject documents carrying the new fields. The docstring now states both sides honestly. Tests: a zero-scene deleted warm ghost is restored through the real loadFromStorage (fails pre-fix), an explicit empty roster does not resurrect stale mirror rows (fails pre-fix), and the rosterNeedsLegacyFallback unit matrix covers undefined vs []. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): re-queue aggregate persistence when a failed deletion is recovered Direct aggregate saves (saveToStorage: generation completion, server-restore hydration) run outside the pending map and the flush round, so the deletion path's pre-delete dirt snapshot cannot describe them. When a deletion bumped the epoch while such a save was in flight, the save was correctly fenced ('stale-dropped') — but if the deletion then failed before removing the document, the recovery only re-queued the snapshotted scheduler descriptors. The fenced aggregate-only content stayed memory-only with no retry anywhere, and a later reload lost it. The failed-deletion restore now merges a full re-mark of the aggregate (structure / stage / outline / currentScene / chats plus a descriptor for every current scene) into the recovery set, still guarded on the store holding the stage. Because the flush recaptures the CURRENT store state under the CURRENT epoch, this necessarily carries whatever the fenced aggregate save held — and, for the same reason, edits refused by the scheduler during the deletion window, which previously stayed memory-only as well; the scope docstrings now say so instead of declaring them excluded. New integration coverage drives the real store scheduler and the real storage layer end to end: a genuinely fenced in-flight saveToStorage (generation completion) plus a pre-removal deletion failure ends with the completion flag durable on the next flush; a deletion-window scene edit survives the same way; and a successful deletion re-marks nothing (control). Both recovery tests fail against the previous restore logic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
|
be3d313bc3 |
feat(document): dirty-set incremental saves + operation-level flush (#983)
* feat(document): dirty-set incremental saves + operation-level flush + unload protection Every store mutation used to trigger a debounced FULL aggregate rewrite — editing one word rewrote the whole document, scene switches during playback rewrote it for nothing, and concurrent editors clobbered each other at document granularity. Mutations now mark precise dirt (scene:<id> / structure / stage / outline / currentScene / chats): scene edits flush as version-gated putScene, stage metadata as putStage, structure and outline changes keep the full-save path, and currentScene/chats no longer touch the document at all. Dirt entries are revisioned so a mutation landing during a write is never accidentally cleared, and failed writes retain their dirt for retry. flushStageSave() drains the debounce single-flight; agent apply operations await it so each AI edit is durable before its tool result returns; hidden-page/unload kick a best-effort immediate flush. The PendingChange type stays neutral about the change unit so a future operation-log backend can replace the flush internals at one seam. Co-authored-by: Codex <codex@openai.com> * fix(document): incremental-persistence review round — bypass-proofing, true drain, no stranded dirt Structural mutations carry the cursor they changed (and cursor resolution validates membership on load, so a stale device cursor can never resurrect a deleted scene). The Stage API now wraps the real store in a guard that diffs pre/post state and centrally classifies every raw write — whiteboard, element, canvas, and action-engine mutations can no longer bypass dirty tracking, and new API namespaces inherit the guard by construction (with a lint-like inventory test). flushStageSave is a bounded true drain: an awaiter whose mutation landed mid-flight loops until its own dirt is durable. setStage hands the old document's dirt to a departing flush instead of discarding it, and a failed flush always reschedules. Mixed dirty batches (scene+stage/outline) use the atomic full save — the incremental fast path is homogeneous-only. Chat-half failures keep the chats dirt for retry without changing the tolerated-split outer semantics. Co-authored-by: Codex <codex@openai.com> * fix(document): reviewer round — drain covers caller dirt, full-path chat retry, stamping unification flushStageSave's early-return now only fires when the failing round's snapshot actually covered the caller's dirt — an agent edit landing beside an in-flight failing round is written before the promise resolves, keeping the tool-result durability guarantee unconditional. The full-save path reports chat failures the same way the incremental path does (chats dirt survives and reschedules; the tolerated-split semantics are unchanged). The departing-stage flush retries once and its doc-comment states the best-effort semantics honestly. Stage and scene stamping are extracted and shared — fixing the incremental putScene branch silently dropping the order ?? index normalization the full path applies — and the dead outline mark in setGenerationComplete is gone. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
|
d0d227424b |
feat(document): complete the document-persistence cutover — stage/scene/outline onto DocumentStore (#965)
* feat(runtime): persist quiz attempts in RuntimeStore * fix(runtime): coalesce quiz draft snapshots * fix(runtime): recover concurrent quiz attempts * fix(runtime): handle quiz completion races * fix(runtime): commit quiz review atomically * fix(runtime): dedupe concurrent quiz writes * fix(runtime): drop stale quiz drafts * fix: harden quiz runtime review recovery * fix: serialize quiz attempt identity * feat: read quiz state from runtime store * fix: persist quiz retries before resetting * fix: preserve authoritative quiz outcomes * fix: preserve legacy quiz retries during cutover * fix: reconcile legacy quiz snapshots safely * fix: drain rollover quiz write queues * fix: drain completed quiz retry queues * fix: reuse concurrent quiz retries * fix(quiz): preserve drafts across abrupt reloads * fix(quiz): recover empty retry sessions * fix(chat): abort stalled runtime state reads * fix(quiz): expose queued phases to readers * fix(quiz): retain concurrent writer tails * fix(quiz): canonicalize retry branches * fix(quiz): keep retry rollovers monotonic * fix(quiz): validate skipped retry siblings * fix(quiz): close read cutover races * test(classroom): cover legacy quiz summaries * fix(quiz): reset async consumers on scene changes * fix(quiz): close scene transition windows * test(pbl): cover launch freshness guards * fix(quiz): close cutover concurrency gaps * fix(quiz): harden retry and context freshness * fix(quiz): reject malformed legacy answers * fix(quiz): validate legacy answer values * test(quiz): cover legacy multi-answer migration * fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover Main's three call sites built static storeState blocks that runAgentLoopFn never consumed (it always rebuilds fresh state via getStoreState); the quiz read cutover replaced that callback with the async two-phase RuntimeStore read, leaving the template field with zero consumers. Drop it. * feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError semantics existed only in the browser backend; server-backed deployments would silently accept conflicting appends and leave completed sessions active. Forward the options over the wire, detect conflicts atomically under the PG transaction, map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error client-side so quiz retry logic works across every backend. Co-authored-by: Codex <codex@openai.com> * fix(chat): Pi single requests build storeState via the async runtime quiz read Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the request template straight to /api/chat/pi, which rejects bodies without storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read with the scene-transition guard) and call it on the Pi path too. * ci: whitelist the runtime-data-cutover integration trunk for PR checks * feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956) * feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869) The fourth and last runtime family. Consumed-discussion facts become append-only 'playback' records folded into a set at read (at-least-once appends, no conflict machinery); the resume cursor is device-scoped last-write-wins KV per the amended #779/#869 split. sessionStorage keeps same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie playbackState machinery is retired, with a one-time lazy migration of any legacy row (cursor half + facts half) before deletion, and stage deletion now clears both the KV cursor and any unmigrated legacy row. Co-authored-by: Codex <codex@openai.com> * test(runtime): include playbackState in the stage-delete db mock --------- Co-authored-by: Codex <codex@openai.com> * fix(playback): persist discussion facts on every consumption path (#957) * fix(playback): persist discussion facts on every consumption path (final-review P0+P1s) - The engine now publishes a progress snapshot the moment a discussion is consumed (join / skip / unselected-agent auto-skip). onProgress otherwise fires before the discussion action executes and a discussion is the scene's last action, so the fact never reached persistence. - Reads fold records across ALL playback sessions in the learner partition (mergeLearner deliberately preserves same-kind sessions from both keys). - Legacy migration appends only not-yet-durable facts, so an interrupted migration resumes instead of dropping the tail. - recordConsumedDiscussion reports durability; the component drops failed ids from its observed set so a later progress tick retries (at-least-once). * test(e2e): live verification of the playback persistence chain Seeds a deterministic stage straight into the Dexie DB, starts the lecture via the canvas overlay, and asserts the full chain: discussion auto-skip appends a discussionConsumed record to maic-runtime, the device cursor lands in KV, and both survive a fresh browsing context (empty sessionStorage). * refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959) Product ruling on #869's fourth family: playback learner state is front-end ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined discussions' content already lives in chat runtime records, and no replay export / analytics consumer exists — so durable facts bought nothing over in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half with real UX value), the engine's consumption-time progress snapshot (cursor freshness), and the legacy Dexie retirement (cursor half migrates, row deletes, consumed ids are dropped). * ci: whitelist the document-cutover integration trunk for PR checks * feat(document): widened-stage contracts + app document seam (#860 follow-up, PR1) (#961) Package: MaicDocument/DocumentStore/StageRow gain a TStage parameter (defaults keep existing users unchanged), BrowserDocumentStore accepts an injectable validateStage, and a version-guarded putStage closes the rename read-modify-write race — same current-version gate as putScene. App: lib/document/ seam — AppDocument types (currentSceneId is device state, not document data, and the app stage validator rejects it), a four-kind scene validator (slide/quiz delegate to the DSL; interactive/pbl validate envelope + discriminant binding only), and pure legacy canonicalizers (type rebound from content.type, singular whiteboard → whiteboards, outline envelope). No consumer switches off Dexie yet. Co-authored-by: Codex <codex@openai.com> * feat(document): stage-storage cutover + locked lazy migration (PR2) (#962) * feat(document): cut stage/scene persistence over to DocumentStore with locked lazy migration (PR2) stage-storage now persists through the lib/document seam: aggregate saveDocument with a mandatory outline envelope (omission means deletion in the contract; ordinary saves fall back to the stored envelope), lazy per-stage migration under a Web Lock (destination authoritative, one coherent legacy snapshot, save-verify-marker, legacy rows retained read-only; no-lock environments read legacy without migrating), currentSceneId in device KV, rename via version-guarded putStage, and a list that merges legacy-only stages until they migrate. stageOutlines' second write authority folds into the document service, making the final scene set and generationComplete one atomic commit. Co-authored-by: Codex <codex@openai.com> * fix(document-store): relocate the persistence seam out of lib/document lib/document/ is the server-side content-extraction domain (sharp, cloud SDKs); exporting the persistence seam from its barrel dragged node-only modules into the client bundle and broke the Turbopack build. The seam now lives in lib/document-store/ with its own barrel, and the extraction barrel is restored. Client-only guards become capability probes (indexedDB / localStorage presence) so injected node test environments keep working while a true server render still fails loud. * fix(document-store): no-lock write classification + lock-order inversion (PR2 fix round) Lock order is now per-stage document lock → global runtime epoch (a save never waits on a document lock while occupying the shared epoch, which deadlocked against fair-queued maintenance). Without Web Locks, mutation classifies instead of refusing wholesale: destination-backed and genuinely new documents accept the product's established lock-free/LWW risk, while a legacy-only document stays read-only so two authorities cannot fork — matching the chat-cutover-era acceptance suite, whose save assertions now verify through the public API instead of legacy Dexie rows. Co-authored-by: Codex <codex@openai.com> * style: prettier on document-store seam files --------- Co-authored-by: Codex <codex@openai.com> * feat(document): cut secondary consumers over to the document seam (PR3) (#963) ZIP classroom/video exports read stage metadata through the seam; classroom import builds one canonical aggregate under the per-stage lock and writes canonical 'whiteboards' (fixing the import-side singular-alias defect the manifest never had); backups become versioned document aggregates that finally include outline envelopes, while old {stages, scenes} backups are accepted through the legacy canonicalizers; clearDatabase now clears maic-documents and the seam's KV keys so clear-cache cannot strand documents; deleteStageWithRelatedData is deprecated in favor of deleteStageData. Four e2e seeders target maic-documents directly; classroom-interaction stays legacy-seeded to exercise real-UI lazy migration. Co-authored-by: Codex <codex@openai.com> * fix(document): final-review hardening — restore pre-images, honest markers, clear-race exclusion (3 P1 + 2 P2) (#964) - Backup rollback restores pre-existing destination documents (and imported current-scene KV values) from captured pre-images instead of deleting them - The skip-migration path only writes its marker after reconciling the legacy snapshot against the authoritative destination; a divergent snapshot logs and stays discoverable instead of being falsely certified - Migration's destination write enrolls in the global shared storage epoch (per-stage -> global order), so clearDatabase's exclusive epoch can no longer race a lazy migration into resurrecting a document - listStages drops legacy entries whose snapshot vanished mid-merge Co-authored-by: Codex <codex@openai.com> * fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966) * fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair) Review findings on #955: didActiveSceneRemainUnchanged compared the active scene by object identity, so a store update reallocating the scene during the async quiz read dropped the learner's graded answers from that turn's request — the scene id is the real boundary. The records route now classifies a malformed sessionTransition as a validation failure instead of letting the store's throw surface as a 500. * fix(playback): superseded-engine cursor guard + migration write-window recheck Second-vendor review of the #959 shrink (requested after the cross-review noted it had single-vendor coverage) found: an engine orphaned by a scene switch during async lecture resume could pass the idle-only recheck, be resurrected, and publish its old scene's progress over the new scene's debounced cursor — the resume continuation now requires identity with the installed engine, and onProgress drops snapshots from superseded engines. The legacy cursor migration also rechecks KV immediately before its write so a concurrent tab's newer cursor cannot be overwritten and orphaned by the legacy-row delete. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> * fix(review): approval follow-up P3 nits (#967) * release: v0.3.1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard - The records route's sessionTransition guard now requires an ISO updatedAt (isIsoTimestamp), matching the sibling PATCH /status route - Dead vi.mock factories for the deleted playback-storage module dropped - A corrupt legacy playback timestamp falls back to 'now' instead of wedging migration into a permanent re-throw that disabled resume --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(document): backup covers no-lock legacy courses; clears fence stale saves (review P1s) (#968) Backup assembly appends read-only canonicalized aggregates for legacy-only stages the lazy migration could not persist (no-Web-Locks environments) — courses visible in the app can no longer vanish from an export. A device-scoped storage generation now fences document mutations: clears bump it inside their exclusive section, and any save or migration commit that entered before the clear re-reads it inside the shared epoch and fails loud instead of repopulating a wiped store. Co-authored-by: Codex <codex@openai.com> * fix(document): review round 2 — legacy-only chat backup coverage + full-width generation fence (#972) Backups now enumerate runtime chat sessions for every exported document, not just store-backed ones — a legacy-only course's chat history (the chat and document cutovers are independent seams) no longer drops out of the export; the runtime read injects an empty legacy chat store since legacy rows are collected by direct table reads, which also keeps the export working without Web Locks. The storage-generation fence now guards every mutating store method, with the exclusive-epoch delete cascades passing storageSharedLockHeld so the guard cannot self-deadlock against their own hold. --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
3eea9dc542 |
feat(runtime): complete the #869 learner-data cutover — quiz + playback onto RuntimeStore (#955)
* feat(runtime): persist quiz attempts in RuntimeStore * fix(runtime): coalesce quiz draft snapshots * fix(runtime): recover concurrent quiz attempts * fix(runtime): handle quiz completion races * fix(runtime): commit quiz review atomically * fix(runtime): dedupe concurrent quiz writes * fix(runtime): drop stale quiz drafts * fix: harden quiz runtime review recovery * fix: serialize quiz attempt identity * feat: read quiz state from runtime store * fix: persist quiz retries before resetting * fix: preserve authoritative quiz outcomes * fix: preserve legacy quiz retries during cutover * fix: reconcile legacy quiz snapshots safely * fix: drain rollover quiz write queues * fix: drain completed quiz retry queues * fix: reuse concurrent quiz retries * fix(quiz): preserve drafts across abrupt reloads * fix(quiz): recover empty retry sessions * fix(chat): abort stalled runtime state reads * fix(quiz): expose queued phases to readers * fix(quiz): retain concurrent writer tails * fix(quiz): canonicalize retry branches * fix(quiz): keep retry rollovers monotonic * fix(quiz): validate skipped retry siblings * fix(quiz): close read cutover races * test(classroom): cover legacy quiz summaries * fix(quiz): reset async consumers on scene changes * fix(quiz): close scene transition windows * test(pbl): cover launch freshness guards * fix(quiz): close cutover concurrency gaps * fix(quiz): harden retry and context freshness * fix(quiz): reject malformed legacy answers * fix(quiz): validate legacy answer values * test(quiz): cover legacy multi-answer migration * fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover Main's three call sites built static storeState blocks that runAgentLoopFn never consumed (it always rebuilds fresh state via getStoreState); the quiz read cutover replaced that callback with the async two-phase RuntimeStore read, leaving the template field with zero consumers. Drop it. * feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError semantics existed only in the browser backend; server-backed deployments would silently accept conflicting appends and leave completed sessions active. Forward the options over the wire, detect conflicts atomically under the PG transaction, map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error client-side so quiz retry logic works across every backend. Co-authored-by: Codex <codex@openai.com> * fix(chat): Pi single requests build storeState via the async runtime quiz read Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the request template straight to /api/chat/pi, which rejects bodies without storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read with the scene-transition guard) and call it on the Pi path too. * ci: whitelist the runtime-data-cutover integration trunk for PR checks * feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956) * feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869) The fourth and last runtime family. Consumed-discussion facts become append-only 'playback' records folded into a set at read (at-least-once appends, no conflict machinery); the resume cursor is device-scoped last-write-wins KV per the amended #779/#869 split. sessionStorage keeps same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie playbackState machinery is retired, with a one-time lazy migration of any legacy row (cursor half + facts half) before deletion, and stage deletion now clears both the KV cursor and any unmigrated legacy row. Co-authored-by: Codex <codex@openai.com> * test(runtime): include playbackState in the stage-delete db mock --------- Co-authored-by: Codex <codex@openai.com> * fix(playback): persist discussion facts on every consumption path (#957) * fix(playback): persist discussion facts on every consumption path (final-review P0+P1s) - The engine now publishes a progress snapshot the moment a discussion is consumed (join / skip / unselected-agent auto-skip). onProgress otherwise fires before the discussion action executes and a discussion is the scene's last action, so the fact never reached persistence. - Reads fold records across ALL playback sessions in the learner partition (mergeLearner deliberately preserves same-kind sessions from both keys). - Legacy migration appends only not-yet-durable facts, so an interrupted migration resumes instead of dropping the tail. - recordConsumedDiscussion reports durability; the component drops failed ids from its observed set so a later progress tick retries (at-least-once). * test(e2e): live verification of the playback persistence chain Seeds a deterministic stage straight into the Dexie DB, starts the lecture via the canvas overlay, and asserts the full chain: discussion auto-skip appends a discussionConsumed record to maic-runtime, the device cursor lands in KV, and both survive a fresh browsing context (empty sessionStorage). * refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959) Product ruling on #869's fourth family: playback learner state is front-end ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined discussions' content already lives in chat runtime records, and no replay export / analytics consumer exists — so durable facts bought nothing over in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half with real UX value), the engine's consumption-time progress snapshot (cursor freshness), and the legacy Dexie retirement (cursor half migrates, row deletes, consumed ids are dropped). * fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966) * fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair) Review findings on #955: didActiveSceneRemainUnchanged compared the active scene by object identity, so a store update reallocating the scene during the async quiz read dropped the learner's graded answers from that turn's request — the scene id is the real boundary. The records route now classifies a malformed sessionTransition as a validation failure instead of letting the store's throw surface as a 500. * fix(playback): superseded-engine cursor guard + migration write-window recheck Second-vendor review of the #959 shrink (requested after the cross-review noted it had single-vendor coverage) found: an engine orphaned by a scene switch during async lecture resume could pass the idle-only recheck, be resurrected, and publish its old scene's progress over the new scene's debounced cursor — the resume continuation now requires identity with the installed engine, and onProgress drops snapshots from superseded engines. The legacy cursor migration also rechecks KV immediately before its write so a concurrent tab's newer cursor cannot be overwritten and orphaned by the legacy-row delete. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> * fix(review): approval follow-up P3 nits (#967) * release: v0.3.1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard - The records route's sessionTransition guard now requires an ISO updatedAt (isIsoTimestamp), matching the sibling PATCH /status route - Dead vi.mock factories for the deleted playback-storage module dropped - A corrupt legacy playback timestamp falls back to 'now' instead of wedging migration into a permanent re-throw that disabled resume --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
wyuc
|
34448beb6c |
feat(storage): server-backed runtime — pluggable backend seam, HTTP contract, Postgres backend, reference server (#939) (#946)
* ci: run PR checks against the runtime-server-backend integration branch (#939) * feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B) (#940) * feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B) - documented JSON HTTP contract for every RuntimeStore operation (server-assigned seq, learnerKey derived from auth — never trusted from the request, machine-readable error codes, idempotency notes) - HttpRuntimeStore client with injected fetch + auth headers hook; session reads (including createSession responses) migrate forward on the runtime line so an older server cannot leak stale envelopes - conformance test server bridging the contract onto the browser backend; full runRuntimeStoreContract green over HTTP plus error mapping cases * fix(storage): harden HTTP runtime backend after cross-review (#939 Part B) Cross-review fixes: - appendRecord gates payloads through assertJsonValue: fail loud on values JSON cannot represent (Map, Date, NaN, nested undefined, NUL strings) instead of silently mangling them in transit; payload domain documented in the contract - appendRecord/listRecords responses validate via validateRuntimeRecord (server-assigned seq is no longer trusted verbatim) and listRecords sorts by seq like listSessions already sorted defensively - request headers no longer require an ambient Headers constructor when fetch is injected - conformance server classifies errors structurally (existence checks, DSL validators, version stamps) instead of regex-sniffing messages that interpolate caller-controlled ids - router preserves empty path segments so empty-key calls keep browser no-op semantics instead of shifting onto other routes - stray runtimeDslVersion/seq in request bodies are ignored (store- assigned wins), matching the browser reference; docs updated - loopback integration test exercises the real listening server - docs state the conformance server is test-only; auth-derived learnerKey is Part D's reference server * fix(storage): round-2 cross-review hardening for the HTTP backend (#939 Part B) - json-value guard rewritten: additionally rejects -0, symbol-keyed and non-enumerable own properties, and non-index own properties on arrays; U+2028/U+2029 are accepted again (they round-trip through JSON per RFC 8259 — rejecting them lost legitimate pasted text) - conformance server classifies racing duplicate creates as 409 via a structural post-check, gates payloads and merge learner keys as 400 - segment() rejects '.'/'..' ids the URL layer would fold away - list responses must be arrays and mergeLearner's moved must be a finite number, else a typed MALFORMED_RESPONSE error * fix(storage): round-3 cross-review hardening for the HTTP backend (#939 Part B) - json-value guard: NUL is rejected in object keys too, and unpaired UTF-16 surrogates are rejected in string values and keys (jsonb refuses both; other JSON stacks corrupt lone surrogates to U+FFFD) - typed storage errors survive non-object 200 responses instead of degrading into TypeErrors on .id access - mergeLearner's moved must be a non-negative integer - conformance server classifies missing/non-object request bodies as 400 VALIDATION_FAILED - createSession/appendRecord gate the full init envelope, not only the payload (stray Date/undefined properties, NUL in ids) * fix(storage): round-4 convergence fixes for the HTTP backend (#939 Part B) - envelope JSON gate tolerates explicitly-undefined optional anchors (sceneId: undefined behaves like omission, matching the browser) - body-carried identifiers reject '.'/'..' so nothing persists that the URL layer cannot later address; conformance server mirrors it - mergeLearner keys pass the JSON-domain gate on client and server - json-value guard v4: rejects enumerable accessors (validation/serialize TOCTOU), Array subclasses and null-proto arrays, prototype-supplied array indices; sparse-array test now constructs a genuine hole * fix(storage): round-5 convergence fixes for the HTTP backend (#939 Part B) - undefined-stripping is limited to the DSL-declared optional anchors (sceneId/actionIndex/subAnchor); any other undefined member fails the JSON gate loud instead of being dropped silently - json-value guard: array and object prototype checks are realm-agnostic (chain-shape instead of identity), so ordinary values from another realm are accepted while subclasses stay rejected; shared isLosslessJsonString predicate exported for SQL key guards - conformance server merge route validates target-key addressability * fix(storage): close the toJSON prototype channel in the JSON guard (#939) toJSON is the single channel through which a prototype can alter JSON output — prototype properties never serialize and own accessors are already rejected — so refusing any value with a callable toJSON closes prototype influence on serialization entirely, including prototypes crafted to pass the realm-agnostic chain-shape check. * fix(storage): probe toJSON via descriptor walk, not property read (#939) Reading value.toJSON would execute an inherited accessor, letting a stateful getter hide from validation and reappear at stringify time. The probe now walks own descriptors up the prototype chain without invoking any user code. Post-validation mutation of the caller's object graph is documented as out of scope — it is equally unpreventable for every other validated property. * fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939) The own-most descriptor decides: absent is safe, a non-callable data value is an ordinary shadowing member and is safe, a callable data value is rejected, and an accessor is rejected because it cannot be inspected without invoking it. No daylight remains between the guard and the serializer on this property. * feat(storage): PgRuntimeStore — Postgres runtime backend (#939 Part C) (#941) * feat(storage): PgRuntimeStore — Postgres runtime backend over an injected queryable (#939 Part C) - PgRuntimeStore implements RuntimeStore over a minimal injected Queryable (node-postgres and PGlite both satisfy it) — the package keeps zero runtime dependencies beyond @openmaic/dsl - runtime_sessions / runtime_records schema exported as RUNTIME_PG_SCHEMA with idempotent ensureSchema() - appends serialize per session via a session-row lock; MAX(seq)+1 and the insert share one transaction, UNIQUE(session_id, seq) plus bounded retry backstop non-cooperating writers - envelope semantics mirror the browser backend: version stamping, validation gates, migrate-on-read, fail-loud on future-stamped rows - full runRuntimeStoreContract green on PGlite plus PG-specific cases (concurrent-append seq atomicity, ensureSchema and mergeLearner idempotence) * fix(storage): require pinned transactions + JSON payload gate in PG backend (#939 Part C) Cross-review fixes: - withTransaction is now required; the BEGIN/COMMIT fallback is removed (on a pg Pool it spread BEGIN/body/COMMIT across different connections — no real transaction, leaked idle-in-transaction clients, broken mergeLearner atomicity; on a shared pinned client concurrent calls interleaved transactions) - single-statement deletes no longer wrap in a transaction hook call - appendRecord gates payloads through assertJsonValue: fail loud on values JSON cannot represent (Map, Date, NaN, nested undefined, NUL strings) instead of silently persisting something different - append retry also covers 40001/40P01; READ COMMITTED assumption documented at the retry loop - ensureSchema documented as create-only; redundant stage index dropped - deterministic interleaving test proves the 23505 retry path; real PostgreSQL contract lane added (postgres:16 service workflow, pg driver suite skipped locally without PG_CONTRACT_URL) * fix(storage): round-2 cross-review hardening for the PG backend (#939 Part C) - json-value guard rewritten (shared with the HTTP backend): additionally rejects -0, symbol-keyed and non-enumerable own properties, and non-index own properties on arrays; accepts U+2028/U+2029 - storage-pg-contract workflow now triggers for the runtime-server-backend integration branch and fails loud (STORAGE_PG_CONTRACT_REQUIRED=1) when PG_CONTRACT_URL is missing instead of silently skipping - loadSession distinguishes corrupt non-object rows from absent rows so getSession fails loud instead of reporting the session missing * fix(storage): round-3 cross-review hardening for the PG backend (#939 Part C) - json-value guard (shared): NUL rejected in object keys, unpaired UTF-16 surrogates rejected in values and keys - createSession/appendRecord gate the full persisted envelope through assertJsonValue, not only the payload — stray Date/undefined properties and NUL in ids fail loud instead of silently diverging from the returned record or leaking raw 22P05 - real-PG lane covers a genuine 23505 conflict from a second connection and recovery after an aborted transaction - retry-set asymmetry and mergeLearner's unbounded lock set documented * fix(storage): round-4 convergence fixes for the PG backend (#939 Part C) - NUL/lone-surrogate lookup and delete keys resolve to absent/no-op instead of leaking 22021 driver errors; setSessionStatus reports the session missing; mergeLearner from-key moves 0 - mergeLearner destination key passes the JSON-domain gate fail-loud - envelope JSON gate tolerates explicitly-undefined optional anchors, matching the browser backend - json-value guard v4 (shared) + tests for accessors, Array subclasses, prototype-supplied indices * fix(storage): round-5 convergence fixes for the PG backend (#939 Part C) - appendRecord pre-checks the session key like every other lookup path, so a NUL/lone-surrogate sessionId reports 'no session' instead of leaking a 22021 driver error - the queryable-key predicate now structurally reuses the shared isLosslessJsonString export instead of restating the string rule - undefined-stripping limited to the DSL-declared optional anchors - json-value guard: realm-agnostic prototype checks (shared) * fix(storage): close the toJSON prototype channel in the JSON guard (#939) Shared guard change with the HTTP backend branch; adds the crafted null-proto-prototype regression test. Both final-gate reviewers independently converged on this same channel. * fix(storage): probe toJSON via descriptor walk, not property read (#939) Reading value.toJSON would execute an inherited accessor, letting a stateful getter hide from validation and reappear at stringify time. The probe now walks own descriptors up the prototype chain without invoking any user code. Post-validation mutation of the caller's object graph is documented as out of scope — it is equally unpreventable for every other validated property. * fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939) The own-most descriptor decides: absent is safe, a non-callable data value is an ordinary shadowing member and is safe, a callable data value is rejected, and an accessor is rejected because it cannot be inspected without invoking it. No daylight remains between the guard and the serializer on this property. * fix(storage): conform HTTP/PG backends to the post-#926 RuntimeStore interface (#943) The chat cutover (#926) added deleteAllRuntime() to the RuntimeStore contract while the HTTP and Postgres backends were developed in parallel against the pre-#926 interface. Implements the method on both backends (single-statement wipe on PG via the FK cascade; DELETE /runtime on the HTTP contract, documented as an operator-gated administrative endpoint), restoring a green typecheck and contract suite on the integration branch. * feat(runtime): injectable RuntimeStore backend + learner-key provider (#939 Part A) (#944) * wip(runtime): backend injection seam — pending gate verification * fix(runtime): cross-review hardening for the storage injection seam - stage-deletion IndexedDB probe applies only to the default browser backend; an injected store always receives deleteStageRuntime - explicit kv argument takes precedence over the configured learner-key provider, matching the store seam's explicit-beats-global rule - configured learner-key resolution is latched with in-flight dedup, mirroring the store singleton; identity changes require app-level handling - factory retry semantics documented; seal errors explain the module-level bootstrap requirement; isRuntimeStorageConfigured() probe and a test-only reset added - client-bootstrap-only contract documented (SSR/HMR caveats) * fix(runtime): snapshot configuration and make the test reset complete - configureRuntimeStorage copies the option fields so mutating the caller's object after configuring cannot swap the sealed backend or identity provider - resetRuntimeStorageForTests now clears every latched consumer cache (store singleton, learner-key in-flight promise) via a reset-hook registry, so a reset-then-reconfigure test actually gets the new backend * fix(runtime): reset also clears the default learner-key caches resetRuntimeStorageForTests left defaultInFlight/defaultKv latched, so a default-path test could leak its anonymous key or KV store into the next test despite the documented full-reset promise. * feat(storage): runtime reference server — auth-derived learnerKey (#939 Part D) (#945) * wip(storage): reference server — pending deleteAllRuntime route + gates * feat(storage): runtime reference server — auth-derived learnerKey enforcement (#939 Part D) - createRuntimeHttpHandler(store, options) wires the documented HTTP contract onto any injected RuntimeStore over node http - authenticate is required; every learner-scoped operation verifies the path/body learnerKey against the authenticated principal (403 FORBIDDEN_LEARNER) — the client-supplied value is never trusted - mergeLearner requires an explicit authorizeMerge grant and admin surfaces (stage cascade, DELETE /runtime) require authorizeAdmin; both default-deny - runnable reference entry demonstrates a pg Pool withTransaction and bearer-token authentication, marked as demo-only - contract suite green through HttpRuntimeStore -> listening reference handler -> PgRuntimeStore(pglite); security matrix tested (401/403 paths, admin default-deny); threat model documented * fix(storage): cross-review hardening for the reference server (#939 Part D) - principal learnerKey is optional: admin/merge-only credentials no longer fabricate learner identity; learner-scoped routes 403 without ownership - full session/record envelopes pass the JSON-domain gate at the handler, so NUL/lone-surrogate identifiers map to 400 instead of 500 - payload validation follows the injected store's validator map (options.payloadValidators) instead of imposing DSL defaults - reference factory accepts authenticate/authorizeMerge/authorizeAdmin/ payloadValidators overrides; docs no longer claim the factory binds to localhost; demo-impersonation warning hardened - 500 responses carry a generic message; details go to the server log - ownership checks precede version checks and unowned sessions read as 404, closing existence/version oracles; cross-learner denial matrix tested per route - merge/delete concurrency documented as linearizable-equivalent with the ownership re-check narrowed to the delete call * fix(storage): align reference-server semantics with the store contract - future-stamped sessions read and delete through unchanged; 409 FUTURE_VERSION applies only to guarded writes (status, append, merge) - check-then-write races reclassify structurally via a post-failure re-fetch: missing session 404, non-active session 400, never a message-sniffed or generic 500 * fix(storage): close the reference CLI's pg pool on startup failure ensureSchema opens connections inside createReferenceRuntimeServer, so a failed schema init or occupied port left the pool holding database resources until its idle timeout. * fix(storage): close the records-route existence oracle cosarah's review point on #946: the records list answered 200 [] for an absent session but 404 for another learner's, so the 404 leaked that an id exists. Absent and foreign sessions now answer identically (404 SESSION_NOT_FOUND) and the HTTP client maps that code back to an empty list, preserving the store contract's absent-lists-as-empty semantics. Contract doc records the server MAY/SHOULD and the client MUST. |
||
|
wyuc
|
65bf20a84b |
feat(runtime): cut chat sessions over to RuntimeStore (#926)
* feat(runtime): persist chat sessions in RuntimeStore * fix(runtime): harden chat persistence ordering * fix(runtime): protect cross-tab chat snapshots * fix(runtime): compare structured chat values safely * fix(runtime): preserve chat backup and reset semantics * fix(runtime): restore imported chat snapshots * fix(runtime): isolate chat load failures * fix(runtime): make cache clearing lossless * fix(runtime): normalize empty chat saves * fix(runtime): serialize legacy chat migration * fix(chat): serialize backup restoration * test(chat): cover multi-stage restore locking * fix(chat): coordinate fallback migrations across tabs * fix(chat): harden cross-tab migration locking * fix(database): retire abandoned chat lease schema * fix(chat): preserve monotonic local edits * fix(chat): protect unseen runtime sessions * fix(chat): order lifecycle transitions monotonically * fix(chat): serialize runtime maintenance * fix(chat): order queued writes before maintenance * fix(chat): preserve cross-version lock compatibility * fix(chat): preserve streamed and backup data * fix(chat): isolate empty no-lock saves * fix(chat): isolate partial runtime snapshots * fix(runtime): close cross-store cutover races * fix(runtime): close remaining cutover races * fix(chat): retain caller-visible conflict baseline * fix(chat): preserve read-only legacy autosaves * fix(chat): avoid restore lock inversion * fix(runtime): enroll writes before maintenance * fix(runtime): retain maintenance ordering * fix(runtime): close restore and deletion races * fix(chat): order partition queues within lock epochs * fix(chat): preserve recovery and deletion intent * fix(runtime): recover interrupted chat restores * fix(backup): scope chat deduplication by stage * fix(backup): stage chats by runtime partition * fix(backup): clear staged chats with stages |
||
|
wyuc
|
667f3b0c51 |
feat(runtime): device-anonymous learner identity + RuntimeStore app bootstrap (#869 Part C, step 1) (#885)
* feat(runtime): device-anonymous learnerKey + app RuntimeStore singleton (#869) * feat(runtime): cascade stage deletion into the runtime store (#869) deleteStageWithRelatedData now cascades into the runtime layer after its Dexie transaction completes. The runtime data lives in a separate IndexedDB database (maic-runtime), so it cannot join the transaction, and the cascade goes through deleteStageRuntimeSafely — a helper that never throws (warns with context instead): a broken or hung runtime DB must not brick stage deletion in the main app DB. deleteStageRuntime is idempotent, so a failed cascade can simply be retried. Covered at the helper seam with a stub RuntimeStore (success + throwing); the repo has no Dexie-in-node harness for database.ts and this change does not invent one. * fix(runtime): wire the live deletion path, serialize learner-key minting (#869) Cross-review fixes on the C1 bootstrap: - The runtime cascade was wired into deleteStageWithRelatedData, which has zero callers; the UI classroom-deletion flow goes through deleteStageData in stage-storage. Wire deleteStageRuntimeSafely into that live path too (after its Dexie work, same isolation rationale), and cover the wiring by running the real deleteStageData with its module deps mocked — the repo's established pattern for database-touching code. - getLearnerKey minted twice under concurrency. Same-bundle callers now share one in-flight promise on the default path (failures are not cached), and every path re-reads after writing and returns the PERSISTED key, so a cross-tab race converges on the stored winner instead of keeping an orphaned local mint. - Guard crypto.randomUUID with the house fallback pattern. - Honest docs: the cascade comment no longer claims a retry path exists (orphaned rows are inert today; a startup sweep is deferred to Part C2), and both lib/runtime modules note they are client-only. * fix(runtime): cross-tab mint lock and bounded deletion cascade (#869) - Read-after-write alone still let a tab keep an orphaned learner key when its re-read landed before the other tab's write. Minting now runs under the Web Locks API ('maic:learner-key') where available: grants are mutually exclusive across tabs, the loser re-reads the winner's key inside its grant, and an existing key is never overwritten (so the per-tab memo stays safe). Without navigator.locks (older browsers, non-window contexts) the memo + read-after-write behavior remains, with the residual race named in a comment and accepted — it merely splits one anonymous learner's local history. - deleteStageRuntimeSafely awaited without a bound, so a hung runtime IndexedDB could block the live deletion path — the exact failure the helper exists to isolate. The cascade now races a 5s timeout: on timeout it warns and resolves (orphaned rows stay inert), and the still -pending cascade carries a swallow handler so a late rejection cannot become an unhandled rejection. * fix(runtime): probe for the runtime DB before cascading a stage deletion (#869) deleteStageRuntimeSafely reached openDb() unconditionally, and opening CREATES the maic-runtime database — so deleting a classroom on a device that never wrote runtime data paid an open-or-create of a second IndexedDB DB, and in degraded environments burned the full 5s bound for zero cleanup value. Probe first without creating: where indexedDB.databases() is available, a missing maic-runtime entry returns immediately; where the probe API is unavailable (older Firefox), fall through to the bounded cascade — skipping there would strand real cleanup once Part C2 adds writers. The probe shares the existing try/catch + timeout envelope, so a hanging databases() cannot brick deletion either. The DB name is a module const passed explicitly to BrowserRuntimeStore so probe and store can never drift. |