9 Commits

Include renames

Author	SHA1	Message	Date
3 people	ceb05adb41	feat(media): allocate generated assets through the registry (#1007 part 2, step b) (#1039) ... * feat(media): establish shared asset ownership primitives Introduce global browser asset-pool ownership, asset-reference collection, stage reclamation planning, and lease-based URL access. Carry allocated media identity through storage and generation boundaries with regression coverage. * fix(media): enforce safe resolution across every consumer Route image, video, thumbnail, presentation, and video-export consumers through one resolution state machine. Prevent opaque allocated or generated references from reaching render and export sinks, with fallback and ownership tests. * fix(media): protect document-owned assets across mutations Allocate pool bytes before compatibility writes and document commits, then roll back uncommitted generations safely. Preserve document ownership across edits, retries, imports, speech generation, scene changes, and stage deletion. * fix(media): scope retries and tighten ownership guard Scope retries to the target scene and slide and refuse ambiguous shared-reference mutations. Expand retry rendering coverage and keep direct pool URL resolution behind the shared lease owner. * fix(storage): avoid nested lock during stage cleanup Execute prepared reclamation plans against an explicitly deleted document so the compatibility cascade cannot re-enter the per-document lock. Cover deletion of stage-owned media rows even when the document has no references. * refactor(media): confine reclamation to stage deletion Remove inline pool and compatibility-row cleanup from element, speech, scene, and audio replacement flows. Keep whole-stage reclamation behind explicit stage ownership, preserve stage-less legacy audio rows, and document deferred document-truth sweeping. * fix(media): close retry and resolution gaps Restore shared source tasks after successful forks, scope retries across both whiteboard locations, and wait for parallel TTS workers before rollback. Resolve background media through import, rendering, and PPTX export paths while keeping retry controls visible over last-good bytes. * test(media): execute the consumer safety matrix Replace source-substring checks with resolver seam execution across all six UI consumers, stage hydration, and both exporters. Pin each rollback layer independently and harden the ownership guard against aliased pool imports. * chore(packages): publish the additive DSL field Bump the DSL patch version for the optional speech-action field. Keep the transitional reclamation policy app-owned and document the legitimate transaction rollback removals there. * fix(media): make retry rollback task-safe Delay allocation task re-keying until final document reconciliation succeeds, restore shared source tasks on failed forks, and require exact stage-whiteboard targets before falling through from a missed scene. * fix(media): clear private assets and refresh leases Delete the asset-pool database during the confirmed local-data wipe. Notify the app lease layer after same-id replacement so mounted consumers re-resolve current bytes without reaching into storage internals. * test(media): pin closure safety guards Exercise the CSS allocation boundary, unfiltered legacy-row ownership, delete-and-undo byte survival, and real distinct consumer seams. Restore the video-only manifest overwrite condition and share the direct video resolution hook across both element variants. * test(media): narrow rollback element assertion Narrow the reconciled slide element to an image before checking its source so the rollback regression remains type-safe under the full root compiler configuration. * fix(media): close lease refresh races Gate the first batch publication by unique resolved refs, then publish every replacement snapshot without mutating the prior React state object. Serialize invalidation behind pending releases, evict rejected refreshes, and register replacement observation at the pool boundary. * fix(media): reopen pool after clear failures Always evict the singleton once its store has been closed, including blocked and failed database deletion paths. Report blocked deletion as deferred and prove a later write uses a fresh live store. * fix(media): isolate shared retry progress Track forked regeneration under the selected element until it receives a fresh asset identity, leaving the shared source task and bytes untouched. Surface targeted failures through renderer task lookup and skip the redundant fork reconciliation lock. * fix(media): close final asset retry gaps Keep blocked asset clears fail-loud until a successful retry, with actionable settings guidance. Clear failed shared-fork state across durable and live key spaces, and pin renderer lookups, lease publication identity, and committed-ref rollback protection. * fix: preserve actionable retry failures Localize the blocked cache-clear recovery hint across every supported locale and pin the deferred-error mapping. Retain durable fork failure rows while retries run, deleting them only after successful generation so unstructured failures survive reload. * fix(media): hydrate legacy stored video thumbnails Home-page recent-video thumbnails regressed for legacy Dexie mediaFiles rows keyed by gen_vid placeholders: the reworked hydration resolved the row's bytes through the sealed resolver but never surfaced the stored blob (and its poster) as object URLs for the preview card, so the CI recent-video-thumbnail e2e specs found no visible element. Hydration now materializes legacy stored video rows into blob URLs for both the element src and poster while keeping the resolver invariants: opaque refs still never reach a DOM src, and concrete addresses are never blanked. Unit pins cover the seam so the vitest suite catches this class without a browser. * fix(media): resolve sole restored legacy video Classroom playback restored tasks by exact document media references. Legacy gen_vid references can outlive the key used by the one persisted video row, leaving the player on a placeholder even though bytes were restored. Select the sole completed stage video only for legacy sequential refs after exact and reconciled matches. Keep exact failures authoritative, refuse ambiguous candidates, and cover success, ambiguity, and failure precedence in unit tests. * fix(media): scope legacy video recovery Decide restored legacy video recovery once from the complete document and record it through the shared task lookup consumed by playback, editing, and resolved slides. Keep ambiguous documents as placeholders, apply the same decision to thumbnail hydration, and preserve exact failure precedence. * fix(media): exclude claimed video recovery tasks Model restored legacy recovery as a two-pass match across document video elements and task rows. Remove tasks claimed by exact, targeted, or placeholder lookup before applying the sole-candidate fallback, and cover the ownership/cardinality matrix. * fix(media): unify video element resolution Centralize source, task, poster, and legacy recovery decisions for every video consumer. Ensure direct URLs win over opaque refs and play_video waits on element-targeted retry tasks. * refactor(media): route legacy recovery through resolver Let document-aware consumers request legacy video recovery through the unified element binding API, keeping thumbnail hydration on a single decision path. * fix(media): preserve import refs and prefer pool bytes Recognize unambiguous extensionless relative media addresses during classroom import. Resolve allocated export and thumbnail assets from the shared pool before falling back to lagging compatibility rows. * fix(export): preserve concrete video sources Route PPTX video elements through the shared media binding resolver so an unresolved opaque reference cannot replace a playable source. Complete browser and persisted-store cleanup when asset-pool deletion is deferred, while retaining distinct hard-failure behavior. * fix(media): fork retries without exclusive ownership Enumerate logical asset owners across every persisted document before allowing global pool replacement. Thread explicit targets through fresh-id rewrites and cover cross-document aliases plus unreadable ownership. * fix(media): guard global asset reclamation Share a fail-closed persisted-document liveness check between stage deletion and retry replacement. Preserve cross-document pool aliases while deleting stage-owned compatibility rows and cover enumeration failures. * fix(media): cover complete slide asset references Route slide media traversal through a shared mutable slot contract so backgrounds participate in export, thumbnail hydration, collection, and rewrite lifecycles. Snapshot complete surviving-document refs once per reclamation and preserve manifest-only owners while retaining fail-closed behavior. * fix(media): preserve exclusive retry asset ids Allow targeted retries to replace exclusively owned pool assets in place. Keep shared and unprovable ownership paths on fresh allocations, and pin production-shaped retries plus compatibility-row cleanup. * fix(media): revalidate asset bindings at completion Recheck repository-wide ownership before replacing generated media and fork scoped retries when exclusivity changed. Route video export selection through the unified resolver and keep concrete posters independent of task state. * fix(media): count unflushed owners and broadcast replacements The completion-time exclusivity proof read only the persisted document, but slide duplication updates the Zustand aggregate synchronously and schedules persistence behind a debounce. A retry finishing inside that window saw a single persisted owner and replaced the bytes behind a reference the duplicate also held. The proof now also counts owners in the live stage snapshot when that snapshot represents the stage being retried, so an unflushed duplicate forks instead. Same-id replacement notifications were realm-local, so a second tab showing the same classroom kept its lease pinned to the superseded blob URL. The notification now travels over a BroadcastChannel; each receiving realm runs its own observers against its own pool, so a spoofed message can at most force a re-resolve. A missing or failing channel never fails the replacement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): bind replacement listeners and spare shared audio rows A realm that only renders never sends a replacement, so binding the channel from the sender path left passive tabs deaf to peers. Binding now happens where the observer is registered, when the asset-pool module loads, and the receiving realm resolves its own pool lazily so a cleared or unavailable pool degrades to the next resolve instead of throwing. Observer notifications ran under Promise.all, so a rejection surfaced after BrowserAssetStore.replace had already committed and turned a durable success into a reported failure. They are settled individually now; the callback is wrapped because a synchronous throw would otherwise escape before allSettled sees the array. audioFiles rows are keyed globally by audioId, so deleting a stage removed the sole row for an id a surviving document still referenced — playback and both export paths read that table directly and cannot fall back to the preserved pool blob. Rows are now filtered against surviving references, while a failed enumeration still withholds only the irreversible pool removal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): serve replaced bytes from the pool everywhere Unknown survivor liveness deleted every planned audioFiles row. Those rows are keyed globally by audioId, and playback plus both export paths read the table directly, so losing one is as irreversible for them as removing the pool entry. Unknown liveness now preserves the rows too, leaving bounded garbage for a later pass that can prove exclusivity. The earlier pool-first change covered PPTX, video collection and thumbnail hydration but missed classroom ZIP export, which still serialized the stale compatibility row after a lagged same-id replacement, shipping media the classroom no longer renders. Auditing every direct reader of the media and audio tables surfaced the same gap in playback: speech regeneration also replaces bytes under a stable id and does not roll the pool back when the compatibility write fails, so the player kept serving superseded narration. It now resolves the pool first and falls back to stored rows for legacy and imported audio. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(audio): replace exclusively owned speech clips in place regenerateSpeechAudio accepted the action's audioId but always passed undefined as replaceAssetId, so regenerating an exclusively owned pool-backed clip allocated a new asset every time, rewrote the action and orphaned the previous pool entry and compatibility row until stage reclamation — contradicting the stable-id path generateAndStoreTTS already implements for media. Ownership is now established before synthesis, and the rule itself moved to the shared reference module so media retries, poster replacement and speech regeneration consume one implementation instead of restating it. An exclusively owned clip keeps its id and has its bytes replaced; a shared clip, a legacy id with no pool entry, or unprovable ownership still gets a fresh allocation so other holders keep their audio. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(media): drop the import left behind by the ownership move Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): resolve audio pool-first and fence peer realms Stable-id TTS regeneration commits replaced narration to the pool before the audioFiles mirror write, so a failed mirror leaves the row stale. AudioPlayer already resolved pool-first, but audioObjectUrl, collectAudioFiles and the video timeline dependencies still read Dexie directly and would serve the superseded clip. All allocated-audio readers now share one resolver, with Dexie kept as the fallback for legacy and imported rows. The exclusivity proof modelled unflushed owners in the active realm only, so another tab duplicating the same asset during its save debounce could still be observed as a single owner and have its bytes replaced globally. A peer's pending state cannot be read across realms, so presence is probed instead: any realm holding the stage forces the fork path. A deferred asset-pool deletion no longer reloads the page. The database is still on disk and the guidance asks the user to close the other tab and retry, which the reload discarded. The decision moved into a shared helper so the rule is pinned rather than living inline in the component. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): fail closed when presence cannot be probed The presence helper documented that an unanswerable probe must count as a peer, but every unavailable path — no BroadcastChannel, a constructor that threw, a send that failed, and the window before the pool's asynchronous binding completes — returned false, so the ownership proof cleared a single local owner and replaced globally shared bytes in place. Probing now returns present, absent or unknown, and only a probe that was actually sent and went unanswered is absent; the ownership decision treats unknown exactly like present. The pool declares its binding intent synchronously so a probe issued during the load-time window waits for the bind instead of concluding that presence is unavailable, and releases that gate if the import fails. Coverage reaches the write boundary: with presence unknown, a production-shaped targeted retry forks to a fresh id instead of calling replace, and the original bytes stay intact for a peer's unflushed owner. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(dsl): bump to 0.6.3 after the release dedupe took 0.6.2 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(i18n): add the deferred-clear guidance to fr-FR The locale landed on main after this branch added the key, so the alignment check flagged it as the one missing translation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>	2026-08-06 18:49:34 +08:00
wyuc	ad30061b6e	fix(chat): harden runtime sync, legacy migration, and record validation (#1050) ... * fix(chat): harden runtime sync and legacy migration * fix(storage): validate HTTP runtime appends client-side * fix(storage): harden runtime validation and legacy guards * fix(chat): skip malformed legacy messages	2026-08-05 06:23:45 -04:00
3 people	b4e3145c90	feat(roster): single-source the generated agent roster on the stage document (#994) ... * feat(roster): single-source the generated agent roster on the stage document The generated agent roster used to live in two places at once: the stage document carried generatedAgentConfigs, while a per-stage IndexedDB mirror (db.generatedAgents) was the actual read authority — and the only home of each agent's voice binding. The split caused real losses: voices vanished on device switches, exports, and shares (the mirror never travels), the load path trusted a device-local table over the portable document, and roster edits persisted through a bespoke debounce with no beforeunload flush. Make the stage document the single source of truth: - Contract: add optional voiceConfig/voiceDesign to GeneratedAgentConfig in @openmaic/dsl (VoiceDesign moves into the contract; the app re-exports it). Additive optional fields — not a breaking serialized-shape change, so DSL_VERSION stays put per the version policy. - Writes: remove every mirror writer (registry saveGeneratedAgents, the stage store's debouncedSaveAgents bypass, classroom-load hydration, import direct writes). setStageAgents now only marks the stage dirty for the shared persistence scheduler plus synchronous in-memory registry/selection mirrors; generation and import embed the roster on the stage. - Reads: classroom load hydrates the registry from the loaded stage's generatedAgentConfigs; export reads the stage roster directly. - Lazy migration: the mirror is retained read-only. On load, a roster (or voice fields) missing from the document is backfilled from the mirror, committed onto the in-memory stage, and persisted by the next flush — idempotent, and gated on the current stage id after every await so a classroom switch cannot leak a stale roster into the registry. - Deletion: deleteStageData discards the deleted stage's pending persistence work so a queued flush cannot resurrect the document. - Round trip: classroom ZIP manifests now carry the voice fields, so export/import preserves agent voices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): harden deletion, migration, selection, and ingress paths Review hardening on top of the roster single-source change: - Deletion vs in-flight persistence: deleting a stage now registers a session-scoped tombstone (lib/utils/deleted-stages.ts) before the cascade starts. Every persistence landing point checks it — the scheduled flush and the departing-stage retry drop their snapshots, and the aggregate/incremental saves re-check under the document lock, including the incremental path's full-save fallback that would otherwise rebuild a deleted document from the in-memory snapshot. A deletion that fails before the document is removed lifts the tombstone so the still-existing stage keeps persisting. - Lazy migration termination: a fruitless legacy-mirror probe (nothing to merge — e.g. server-generated rosters, which are voiceless by design) is remembered per stage for the session instead of re-querying the mirror on every load forever. Successful merges are not memoized so a failed flush is retried on the next load. Docstrings now state the real lifecycle. - Selection provenance: setStageAgents no longer overwrites a user-set agent selection with the full roster. Stage-derived selections still track the roster; a user-set auto selection is only narrowed to surviving agents (newcomers are not auto-selected, the user-set flag is untouched), and a user-set preset selection is left alone — matching restoreAgentSelection. - Mirror hygiene: deleteStageData clears the deleted stage's rows from the legacy roster mirror (best-effort), and the mirror table's docs now describe its actual access pattern (migration reads + deletion hygiene). - Registry persistence: the agent registry's persist snapshot partializes generated agents out of localStorage, making the stage document the roster's only durable home; the rehydration merge filter remains as defense in depth. - Ingress validation: imported manifest voice fields are structurally validated (malformed bindings dropped per field, the agent survives), and applyGeneratedAgentsToRegistry validates providerId against the known TTS provider registry instead of casting, treating unknown providers as "no bound voice". - Contract: drop the speculative AgentVoiceConfig.modelId — an audit of every roster producer, current and historical, found none that ever emitted it. Contract fields are added once a producer exists, not before. New tests cover the deletion tombstone at both the store and storage layers (including the missing-destination full-save pin and the lock-race re-check), the fruitless-probe memo, the selection-provenance matrix, persistence exclusion of generated agents, and voice-field sanitization on import. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): close tombstone lifecycle, probe-memo, and selection-intent gaps Second hardening pass over the deletion/migration/selection fixes: - Tombstone lifecycle: a deleted classroom id can legitimately come back (deletion is client-side only). Explicit (re)creation points now lift the tombstone — server-copy restore in applyClassroomStageAndScenes and backup restore in importDatabase — so edits after a same-session restore persist again, while in-flight flushes stay fenced. - Failed deletion now restores the dirt it discarded: deleteStageData snapshots the pending map plus the in-flight round before tombstoning, and re-marks it when the delete fails with the document still present. - Legacy-mirror probe memo now distinguishes a FAILED read (null, retried on the next load) from a confirmed-empty mirror (memoized), so a transient IndexedDB error cannot suppress migration for the whole session. - Incremental save tail (currentScene KV row + chat sessions) is tombstone- fenced like the aggregate path, and both paths re-check the tombstone immediately before every write, with comments scoped honestly for the lock-free (no Web Locks) best-effort LWW fallback. - Roster-edit selection mirror: an empty intersection now falls back to the full roster and clears the user-set flag (matching restoreAgentSelection's length > 0 gate), and a user-set selection that equals the pre-edit full roster keeps tracking the roster wholesale so newcomers are not excluded forever by the AgentBar auto-toggle snapshot. - TTS/ASR provider lookups use Object.hasOwn instead of `in`/bare indexing, so prototype-chain keys ('toString', 'constructor', ...) no longer pass the provider whitelist or resolve to Object.prototype members. - Documented the per-tab limit of the tombstone set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): replace the boolean deletion tombstone with a deletion-generation model A boolean tombstone cannot express the deletion lifecycle once a deleted id legitimately comes back (server-copy restore on revisit, backup import): after the restore lifts the flag, a pre-delete flush still in flight is indistinguishable from a post-restore edit and can overwrite the restored document with pre-delete content. Two rounds of point fixes kept leaking variants of this race, so this restructures the fence instead. deleted-stages.ts now keeps a per-stage monotonic deletion epoch plus a deleted flag. markStageDeleted bumps the epoch; unmarkStageDeleted clears only the flag and never rewinds the epoch. Every persistence path captures the epoch at the moment it captures the data it will write (flush-round snapshot, departing-stage snapshot, aggregate save entry) and re-checks "captured epoch is current AND not deleted" immediately before each individual write. Invariants: - A write captured before a deletion can never land after it, even when a same-id restore has lifted the deleted flag (the captured epoch is permanently stale). - A write capturing data after a restore observes the current epoch and persists normally. - A failed deletion lifts only the flag; the restored pending changes are re-queued as descriptors, so their flush re-captures current store state under the current epoch and is neither dropped nor a stale replay. Also fixed in the same restructuring: - Incremental tail writes (currentScene, chats) each re-check independently; a delete landing while the first tail write is awaiting now fences the chat write instead of riding the earlier check. - Deleting a classroom evicts it from the warm in-memory store, and loadFromStorage treats a warm-but-deleted stage as not loaded (discarding the ghost), so navigating Back to a deleted classroom reaches the server-restore path instead of rendering an editable ghost whose every edit is silently dropped. - importDatabase records each document's pre-import deletion state and reinstates it when a failed import rolls the document back, so an outstanding flush cannot recreate the rolled-back document. - Docstrings on the failed-deletion restore path now state that only dirt captured before the deletion is restored; edits refused during the deletion window are not part of the snapshot. New tests cover the epoch invariants at the storage layer (pre-delete rounds dropped across restores on the incremental, aggregate, and tail paths; post-restore captures landing; strictly increasing epochs), the scheduler layer (departing-retry delete+restore straddle; restored dirt flushing under the current epoch), the warm-ghost restore driven through the real loadFromStorage, and the import lift/rollback pair. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): report fenced drops, gate ghost eviction on cascade settlement Edge hardening around the deletion-epoch fence; the epoch model itself is unchanged. - An epoch-stale drop is now reported, not disguised: saveStageData / saveStageDataIncremental return a distinct 'stale-dropped' status (entry check, every in-mutation re-check, and both tail-write re-checks) instead of a shape identical to success. saveToStorage reacts by skipping all success bookkeeping - no chatSnapshot rebind to a snapshot that never landed, no pending-dirt clearing - and returns false, keeping its "true means verified write" contract honest. persistDirtySnapshot keeps its existing (correct) semantics explicitly: a drop has nothing to retry. - Warm-ghost eviction is settlement-gated: the deletion cascade records an in-flight bit (begun by deleteStageData, settled in its finally), and the deleted-warm branch of loadFromStorage keeps the warm state while the cascade is unsettled - a delete can still fail before removing the document, in which case the warm state plus the restored pending dirt is the only copy of the pre-delete edits. Once settled, the ghost discard and server-restore path behave as before. - Read-side landing re-check: loadFromStorage re-checks isStageDeleted after hydration, immediately before set(), mirroring the write-side "re-check immediately before landing" discipline - a lock-free mid-cascade read can no longer re-materialize a ghost classroom. - clearStoreForDeletedStage re-checks isStageDeleted at eviction time, so a same-id restore completing inside the cascade-tail window is never wiped. - capturedEpoch is now a required parameter on both storage entry points: the default (call-time capture) silently reopened the capture-point/ validation-point split for future callers; the type system now enforces the pairing. All production callers already passed it explicitly. New tests cover the dropped-status contract, the cascade in-flight lifecycle (including both failure shapes), keep-warm during an unsettled delete with end-to-end pending restore after a failed delete, settlement- then-eviction on success, the eviction guard against a tail-window restore, and the read-side hydration re-check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): settle the deleted-warm load by awaiting the cascade outcome Hardening around the deletion settlement gate introduced previously; the epoch model is unchanged. - The deleted-warm branch of loadFromStorage no longer completes the load while the deletion cascade is undecided (which exposed a classroom whose edits were refused without being restorable, and left a successful deletion's eviction with no follow-up reload). It now parks on a per-stage settlement promise (stageDeletionSettled) and branches on the outcome: a failed delete keeps the warm, restored classroom; a successful delete falls through to a full cold load so the server-restore path recovers the route; a navigation that moved on during the park is stopped by the load token guard. The eviction (clearStoreForDeletedStage) deliberately stops claiming a new load token so the parked load stays current - ghost re-materialization is fenced by the read-side deleted re-checks instead. No deadlock: a parked load holds no document lock, and the cascade never waits on a load. - 'stale-dropped' now propagates through the debounced flush path: persistDirtySnapshot returns the sentinel instead of an empty failure set, and startFlushRound skips the chatSnapshot rebind on it (mirroring saveToStorage) - a falsely rebound baseline would let later chat saves no-op-skip chats that never landed and corrupt cross-tab conflict detection. Pending-map clearing keeps its empty-failure semantics (restored descriptors mint fresh revisions). - The cascade in-flight state is a counter (begin++/settle--) instead of a boolean, and deleteStageData is single-flight per stage (a concurrent second call joins the first cascade), so overlapping deletions can never expose an undecided cascade as settled. - The StaleDroppedSave docstring no longer overpromises: a tail saveCurrentScene can land after the cascade's clearCurrentScene; the orphaned cursor row is ignored by the load path rather than removed by the cascade. Net +8 tests: overlapping-cascade counting, settlement-promise resolution (success and failure), concurrent double-delete single-flight join, the parked mid-cascade Back in both outcomes plus the navigation-away guard, and the fenced flush round keeping an honest chat baseline end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): re-verify store identity on keep-warm resume, re-run joined deletes after a mid-cascade restore Three hardening refinements on the deletion/persistence fencing: - loadFromStorage's parked keep-warm branch now mirrors the success path's store-identity guard: if a tokenless writer (e.g. a database import) replaced the stage during the park, a failed deletion falls through to the cold load instead of reporting another classroom as the kept warm state. The ghost-discard block is correspondingly gated on the deleted flag so the fall-through does not mislabel a surviving document as a ghost. - deleteStageData: a joined call carries its own deletion intent. When the first cascade fulfills but a same-id restore lifted the deleted flag in the meantime, the joined caller now runs one fresh cascade against the restored document instead of reporting the pre-restore outcome. Exactly one re-check per call (no recursion); a rejected first cascade still propagates unchanged to every joined caller. - Narrowed the failed-delete restore-ownership comments: the restore covers only the pending map plus an in-flight flush round's dirt; a departing-stage snapshot is outside that capture and is fenced and dropped by design. Comment-only, no behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): gate deleted-warm handling on stage identity, keep an explicit empty roster authoritative Final-audit fixes, three items: - loadFromStorage keyed its deleted-warm settlement handling (park / keep-warm / ghost discard) behind the scenes.length > 0 shortcut, so a zero-scene warm ghost of a deleted classroom skipped it entirely: the cold load found no document, the ghost stayed in the store, and the classroom loader's server-fallback gate (!getCurrentStage()) never ran — the silently-uneditable trap this branch exists to close, in its scene-less form. Deletion handling now keys on stage identity alone, before any scene-count shortcut; the non-deleted warm skip still requires scenes. - rosterNeedsLegacyFallback collapsed "roster field absent" (document predates roster persistence) with "explicitly persisted empty roster". A future writer persisting [] on a device whose read-only mirror still held stale rows would have its emptied roster resurrected on every load. The full-lift branch now triggers only when the field is absent (undefined); an explicit [] is authoritative and never probes the mirror. No current writer produces [], but the read side no longer depends on that invariant. - The GeneratedAgentConfig docstring claimed the voice fields are not a breaking change; that only holds for this codebase's tolerant structural validators. The generated stage.schema.json sets additionalProperties: false, so consumers pinning an older published schema artifact reject documents carrying the new fields. The docstring now states both sides honestly. Tests: a zero-scene deleted warm ghost is restored through the real loadFromStorage (fails pre-fix), an explicit empty roster does not resurrect stale mirror rows (fails pre-fix), and the rosterNeedsLegacyFallback unit matrix covers undefined vs []. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): re-queue aggregate persistence when a failed deletion is recovered Direct aggregate saves (saveToStorage: generation completion, server-restore hydration) run outside the pending map and the flush round, so the deletion path's pre-delete dirt snapshot cannot describe them. When a deletion bumped the epoch while such a save was in flight, the save was correctly fenced ('stale-dropped') — but if the deletion then failed before removing the document, the recovery only re-queued the snapshotted scheduler descriptors. The fenced aggregate-only content stayed memory-only with no retry anywhere, and a later reload lost it. The failed-deletion restore now merges a full re-mark of the aggregate (structure / stage / outline / currentScene / chats plus a descriptor for every current scene) into the recovery set, still guarded on the store holding the stage. Because the flush recaptures the CURRENT store state under the CURRENT epoch, this necessarily carries whatever the fenced aggregate save held — and, for the same reason, edits refused by the scheduler during the deletion window, which previously stayed memory-only as well; the scope docstrings now say so instead of declaring them excluded. New integration coverage drives the real store scheduler and the real storage layer end to end: a genuinely fenced in-flight saveToStorage (generation completion) plus a pre-removal deletion failure ends with the completion flag durable on the next flush; a deletion-window scene edit survives the same way; and a successful deletion re-marks nothing (control). Both recovery tests fail against the previous restore logic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>	2026-07-27 14:39:19 +08:00
3 people	be3d313bc3	feat(document): dirty-set incremental saves + operation-level flush (#983) ... * feat(document): dirty-set incremental saves + operation-level flush + unload protection Every store mutation used to trigger a debounced FULL aggregate rewrite — editing one word rewrote the whole document, scene switches during playback rewrote it for nothing, and concurrent editors clobbered each other at document granularity. Mutations now mark precise dirt (scene:<id> / structure / stage / outline / currentScene / chats): scene edits flush as version-gated putScene, stage metadata as putStage, structure and outline changes keep the full-save path, and currentScene/chats no longer touch the document at all. Dirt entries are revisioned so a mutation landing during a write is never accidentally cleared, and failed writes retain their dirt for retry. flushStageSave() drains the debounce single-flight; agent apply operations await it so each AI edit is durable before its tool result returns; hidden-page/unload kick a best-effort immediate flush. The PendingChange type stays neutral about the change unit so a future operation-log backend can replace the flush internals at one seam. Co-authored-by: Codex <codex@openai.com> * fix(document): incremental-persistence review round — bypass-proofing, true drain, no stranded dirt Structural mutations carry the cursor they changed (and cursor resolution validates membership on load, so a stale device cursor can never resurrect a deleted scene). The Stage API now wraps the real store in a guard that diffs pre/post state and centrally classifies every raw write — whiteboard, element, canvas, and action-engine mutations can no longer bypass dirty tracking, and new API namespaces inherit the guard by construction (with a lint-like inventory test). flushStageSave is a bounded true drain: an awaiter whose mutation landed mid-flight loops until its own dirt is durable. setStage hands the old document's dirt to a departing flush instead of discarding it, and a failed flush always reschedules. Mixed dirty batches (scene+stage/outline) use the atomic full save — the incremental fast path is homogeneous-only. Chat-half failures keep the chats dirt for retry without changing the tolerated-split outer semantics. Co-authored-by: Codex <codex@openai.com> * fix(document): reviewer round — drain covers caller dirt, full-path chat retry, stamping unification flushStageSave's early-return now only fires when the failing round's snapshot actually covered the caller's dirt — an agent edit landing beside an in-flight failing round is written before the promise resolves, keeping the tool-result durability guarantee unconditional. The full-save path reports chat failures the same way the incremental path does (chats dirt survives and reschedules; the tolerated-split semantics are unchanged). The departing-stage flush retries once and its doc-comment states the best-effort semantics honestly. Stage and scene stamping are extracted and shared — fixing the incremental putScene branch silently dropping the order ?? index normalization the full path applies — and the dead outline mark in setGenerationComplete is gone. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>	2026-07-24 00:00:14 -04:00
3 people	d0d227424b	feat(document): complete the document-persistence cutover — stage/scene/outline onto DocumentStore (#965) ... * feat(runtime): persist quiz attempts in RuntimeStore * fix(runtime): coalesce quiz draft snapshots * fix(runtime): recover concurrent quiz attempts * fix(runtime): handle quiz completion races * fix(runtime): commit quiz review atomically * fix(runtime): dedupe concurrent quiz writes * fix(runtime): drop stale quiz drafts * fix: harden quiz runtime review recovery * fix: serialize quiz attempt identity * feat: read quiz state from runtime store * fix: persist quiz retries before resetting * fix: preserve authoritative quiz outcomes * fix: preserve legacy quiz retries during cutover * fix: reconcile legacy quiz snapshots safely * fix: drain rollover quiz write queues * fix: drain completed quiz retry queues * fix: reuse concurrent quiz retries * fix(quiz): preserve drafts across abrupt reloads * fix(quiz): recover empty retry sessions * fix(chat): abort stalled runtime state reads * fix(quiz): expose queued phases to readers * fix(quiz): retain concurrent writer tails * fix(quiz): canonicalize retry branches * fix(quiz): keep retry rollovers monotonic * fix(quiz): validate skipped retry siblings * fix(quiz): close read cutover races * test(classroom): cover legacy quiz summaries * fix(quiz): reset async consumers on scene changes * fix(quiz): close scene transition windows * test(pbl): cover launch freshness guards * fix(quiz): close cutover concurrency gaps * fix(quiz): harden retry and context freshness * fix(quiz): reject malformed legacy answers * fix(quiz): validate legacy answer values * test(quiz): cover legacy multi-answer migration * fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover Main's three call sites built static storeState blocks that runAgentLoopFn never consumed (it always rebuilds fresh state via getStoreState); the quiz read cutover replaced that callback with the async two-phase RuntimeStore read, leaving the template field with zero consumers. Drop it. * feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError semantics existed only in the browser backend; server-backed deployments would silently accept conflicting appends and leave completed sessions active. Forward the options over the wire, detect conflicts atomically under the PG transaction, map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error client-side so quiz retry logic works across every backend. Co-authored-by: Codex <codex@openai.com> * fix(chat): Pi single requests build storeState via the async runtime quiz read Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the request template straight to /api/chat/pi, which rejects bodies without storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read with the scene-transition guard) and call it on the Pi path too. * ci: whitelist the runtime-data-cutover integration trunk for PR checks * feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956) * feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869) The fourth and last runtime family. Consumed-discussion facts become append-only 'playback' records folded into a set at read (at-least-once appends, no conflict machinery); the resume cursor is device-scoped last-write-wins KV per the amended #779/#869 split. sessionStorage keeps same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie playbackState machinery is retired, with a one-time lazy migration of any legacy row (cursor half + facts half) before deletion, and stage deletion now clears both the KV cursor and any unmigrated legacy row. Co-authored-by: Codex <codex@openai.com> * test(runtime): include playbackState in the stage-delete db mock --------- Co-authored-by: Codex <codex@openai.com> * fix(playback): persist discussion facts on every consumption path (#957) * fix(playback): persist discussion facts on every consumption path (final-review P0+P1s) - The engine now publishes a progress snapshot the moment a discussion is consumed (join / skip / unselected-agent auto-skip). onProgress otherwise fires before the discussion action executes and a discussion is the scene's last action, so the fact never reached persistence. - Reads fold records across ALL playback sessions in the learner partition (mergeLearner deliberately preserves same-kind sessions from both keys). - Legacy migration appends only not-yet-durable facts, so an interrupted migration resumes instead of dropping the tail. - recordConsumedDiscussion reports durability; the component drops failed ids from its observed set so a later progress tick retries (at-least-once). * test(e2e): live verification of the playback persistence chain Seeds a deterministic stage straight into the Dexie DB, starts the lecture via the canvas overlay, and asserts the full chain: discussion auto-skip appends a discussionConsumed record to maic-runtime, the device cursor lands in KV, and both survive a fresh browsing context (empty sessionStorage). * refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959) Product ruling on #869's fourth family: playback learner state is front-end ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined discussions' content already lives in chat runtime records, and no replay export / analytics consumer exists — so durable facts bought nothing over in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half with real UX value), the engine's consumption-time progress snapshot (cursor freshness), and the legacy Dexie retirement (cursor half migrates, row deletes, consumed ids are dropped). * ci: whitelist the document-cutover integration trunk for PR checks * feat(document): widened-stage contracts + app document seam (#860 follow-up, PR1) (#961) Package: MaicDocument/DocumentStore/StageRow gain a TStage parameter (defaults keep existing users unchanged), BrowserDocumentStore accepts an injectable validateStage, and a version-guarded putStage closes the rename read-modify-write race — same current-version gate as putScene. App: lib/document/ seam — AppDocument types (currentSceneId is device state, not document data, and the app stage validator rejects it), a four-kind scene validator (slide/quiz delegate to the DSL; interactive/pbl validate envelope + discriminant binding only), and pure legacy canonicalizers (type rebound from content.type, singular whiteboard → whiteboards, outline envelope). No consumer switches off Dexie yet. Co-authored-by: Codex <codex@openai.com> * feat(document): stage-storage cutover + locked lazy migration (PR2) (#962) * feat(document): cut stage/scene persistence over to DocumentStore with locked lazy migration (PR2) stage-storage now persists through the lib/document seam: aggregate saveDocument with a mandatory outline envelope (omission means deletion in the contract; ordinary saves fall back to the stored envelope), lazy per-stage migration under a Web Lock (destination authoritative, one coherent legacy snapshot, save-verify-marker, legacy rows retained read-only; no-lock environments read legacy without migrating), currentSceneId in device KV, rename via version-guarded putStage, and a list that merges legacy-only stages until they migrate. stageOutlines' second write authority folds into the document service, making the final scene set and generationComplete one atomic commit. Co-authored-by: Codex <codex@openai.com> * fix(document-store): relocate the persistence seam out of lib/document lib/document/ is the server-side content-extraction domain (sharp, cloud SDKs); exporting the persistence seam from its barrel dragged node-only modules into the client bundle and broke the Turbopack build. The seam now lives in lib/document-store/ with its own barrel, and the extraction barrel is restored. Client-only guards become capability probes (indexedDB / localStorage presence) so injected node test environments keep working while a true server render still fails loud. * fix(document-store): no-lock write classification + lock-order inversion (PR2 fix round) Lock order is now per-stage document lock → global runtime epoch (a save never waits on a document lock while occupying the shared epoch, which deadlocked against fair-queued maintenance). Without Web Locks, mutation classifies instead of refusing wholesale: destination-backed and genuinely new documents accept the product's established lock-free/LWW risk, while a legacy-only document stays read-only so two authorities cannot fork — matching the chat-cutover-era acceptance suite, whose save assertions now verify through the public API instead of legacy Dexie rows. Co-authored-by: Codex <codex@openai.com> * style: prettier on document-store seam files --------- Co-authored-by: Codex <codex@openai.com> * feat(document): cut secondary consumers over to the document seam (PR3) (#963) ZIP classroom/video exports read stage metadata through the seam; classroom import builds one canonical aggregate under the per-stage lock and writes canonical 'whiteboards' (fixing the import-side singular-alias defect the manifest never had); backups become versioned document aggregates that finally include outline envelopes, while old {stages, scenes} backups are accepted through the legacy canonicalizers; clearDatabase now clears maic-documents and the seam's KV keys so clear-cache cannot strand documents; deleteStageWithRelatedData is deprecated in favor of deleteStageData. Four e2e seeders target maic-documents directly; classroom-interaction stays legacy-seeded to exercise real-UI lazy migration. Co-authored-by: Codex <codex@openai.com> * fix(document): final-review hardening — restore pre-images, honest markers, clear-race exclusion (3 P1 + 2 P2) (#964) - Backup rollback restores pre-existing destination documents (and imported current-scene KV values) from captured pre-images instead of deleting them - The skip-migration path only writes its marker after reconciling the legacy snapshot against the authoritative destination; a divergent snapshot logs and stays discoverable instead of being falsely certified - Migration's destination write enrolls in the global shared storage epoch (per-stage -> global order), so clearDatabase's exclusive epoch can no longer race a lazy migration into resurrecting a document - listStages drops legacy entries whose snapshot vanished mid-merge Co-authored-by: Codex <codex@openai.com> * fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966) * fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair) Review findings on #955: didActiveSceneRemainUnchanged compared the active scene by object identity, so a store update reallocating the scene during the async quiz read dropped the learner's graded answers from that turn's request — the scene id is the real boundary. The records route now classifies a malformed sessionTransition as a validation failure instead of letting the store's throw surface as a 500. * fix(playback): superseded-engine cursor guard + migration write-window recheck Second-vendor review of the #959 shrink (requested after the cross-review noted it had single-vendor coverage) found: an engine orphaned by a scene switch during async lecture resume could pass the idle-only recheck, be resurrected, and publish its old scene's progress over the new scene's debounced cursor — the resume continuation now requires identity with the installed engine, and onProgress drops snapshots from superseded engines. The legacy cursor migration also rechecks KV immediately before its write so a concurrent tab's newer cursor cannot be overwritten and orphaned by the legacy-row delete. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> * fix(review): approval follow-up P3 nits (#967) * release: v0.3.1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard - The records route's sessionTransition guard now requires an ISO updatedAt (isIsoTimestamp), matching the sibling PATCH /status route - Dead vi.mock factories for the deleted playback-storage module dropped - A corrupt legacy playback timestamp falls back to 'now' instead of wedging migration into a permanent re-throw that disabled resume --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(document): backup covers no-lock legacy courses; clears fence stale saves (review P1s) (#968) Backup assembly appends read-only canonicalized aggregates for legacy-only stages the lazy migration could not persist (no-Web-Locks environments) — courses visible in the app can no longer vanish from an export. A device-scoped storage generation now fences document mutations: clears bump it inside their exclusive section, and any save or migration commit that entered before the clear re-reads it inside the shared epoch and fails loud instead of repopulating a wiped store. Co-authored-by: Codex <codex@openai.com> * fix(document): review round 2 — legacy-only chat backup coverage + full-width generation fence (#972) Backups now enumerate runtime chat sessions for every exported document, not just store-backed ones — a legacy-only course's chat history (the chat and document cutovers are independent seams) no longer drops out of the export; the runtime read injects an empty legacy chat store since legacy rows are collected by direct table reads, which also keeps the export working without Web Locks. The storage-generation fence now guards every mutating store method, with the exclusive-epoch delete cascades passing storageSharedLockHeld so the guard cannot self-deadlock against their own hold. --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-22 23:18:24 +08:00
3 people	3eea9dc542	feat(runtime): complete the #869 learner-data cutover — quiz + playback onto RuntimeStore (#955) ... * feat(runtime): persist quiz attempts in RuntimeStore * fix(runtime): coalesce quiz draft snapshots * fix(runtime): recover concurrent quiz attempts * fix(runtime): handle quiz completion races * fix(runtime): commit quiz review atomically * fix(runtime): dedupe concurrent quiz writes * fix(runtime): drop stale quiz drafts * fix: harden quiz runtime review recovery * fix: serialize quiz attempt identity * feat: read quiz state from runtime store * fix: persist quiz retries before resetting * fix: preserve authoritative quiz outcomes * fix: preserve legacy quiz retries during cutover * fix: reconcile legacy quiz snapshots safely * fix: drain rollover quiz write queues * fix: drain completed quiz retry queues * fix: reuse concurrent quiz retries * fix(quiz): preserve drafts across abrupt reloads * fix(quiz): recover empty retry sessions * fix(chat): abort stalled runtime state reads * fix(quiz): expose queued phases to readers * fix(quiz): retain concurrent writer tails * fix(quiz): canonicalize retry branches * fix(quiz): keep retry rollovers monotonic * fix(quiz): validate skipped retry siblings * fix(quiz): close read cutover races * test(classroom): cover legacy quiz summaries * fix(quiz): reset async consumers on scene changes * fix(quiz): close scene transition windows * test(pbl): cover launch freshness guards * fix(quiz): close cutover concurrency gaps * fix(quiz): harden retry and context freshness * fix(quiz): reject malformed legacy answers * fix(quiz): validate legacy answer values * test(quiz): cover legacy multi-answer migration * fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover Main's three call sites built static storeState blocks that runAgentLoopFn never consumed (it always rebuilds fresh state via getStoreState); the quiz read cutover replaced that callback with the async two-phase RuntimeStore read, leaving the template field with zero consumers. Drop it. * feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError semantics existed only in the browser backend; server-backed deployments would silently accept conflicting appends and leave completed sessions active. Forward the options over the wire, detect conflicts atomically under the PG transaction, map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error client-side so quiz retry logic works across every backend. Co-authored-by: Codex <codex@openai.com> * fix(chat): Pi single requests build storeState via the async runtime quiz read Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the request template straight to /api/chat/pi, which rejects bodies without storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read with the scene-transition guard) and call it on the Pi path too. * ci: whitelist the runtime-data-cutover integration trunk for PR checks * feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956) * feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869) The fourth and last runtime family. Consumed-discussion facts become append-only 'playback' records folded into a set at read (at-least-once appends, no conflict machinery); the resume cursor is device-scoped last-write-wins KV per the amended #779/#869 split. sessionStorage keeps same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie playbackState machinery is retired, with a one-time lazy migration of any legacy row (cursor half + facts half) before deletion, and stage deletion now clears both the KV cursor and any unmigrated legacy row. Co-authored-by: Codex <codex@openai.com> * test(runtime): include playbackState in the stage-delete db mock --------- Co-authored-by: Codex <codex@openai.com> * fix(playback): persist discussion facts on every consumption path (#957) * fix(playback): persist discussion facts on every consumption path (final-review P0+P1s) - The engine now publishes a progress snapshot the moment a discussion is consumed (join / skip / unselected-agent auto-skip). onProgress otherwise fires before the discussion action executes and a discussion is the scene's last action, so the fact never reached persistence. - Reads fold records across ALL playback sessions in the learner partition (mergeLearner deliberately preserves same-kind sessions from both keys). - Legacy migration appends only not-yet-durable facts, so an interrupted migration resumes instead of dropping the tail. - recordConsumedDiscussion reports durability; the component drops failed ids from its observed set so a later progress tick retries (at-least-once). * test(e2e): live verification of the playback persistence chain Seeds a deterministic stage straight into the Dexie DB, starts the lecture via the canvas overlay, and asserts the full chain: discussion auto-skip appends a discussionConsumed record to maic-runtime, the device cursor lands in KV, and both survive a fresh browsing context (empty sessionStorage). * refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959) Product ruling on #869's fourth family: playback learner state is front-end ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined discussions' content already lives in chat runtime records, and no replay export / analytics consumer exists — so durable facts bought nothing over in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half with real UX value), the engine's consumption-time progress snapshot (cursor freshness), and the legacy Dexie retirement (cursor half migrates, row deletes, consumed ids are dropped). * fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966) * fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair) Review findings on #955: didActiveSceneRemainUnchanged compared the active scene by object identity, so a store update reallocating the scene during the async quiz read dropped the learner's graded answers from that turn's request — the scene id is the real boundary. The records route now classifies a malformed sessionTransition as a validation failure instead of letting the store's throw surface as a 500. * fix(playback): superseded-engine cursor guard + migration write-window recheck Second-vendor review of the #959 shrink (requested after the cross-review noted it had single-vendor coverage) found: an engine orphaned by a scene switch during async lecture resume could pass the idle-only recheck, be resurrected, and publish its old scene's progress over the new scene's debounced cursor — the resume continuation now requires identity with the installed engine, and onProgress drops snapshots from superseded engines. The legacy cursor migration also rechecks KV immediately before its write so a concurrent tab's newer cursor cannot be overwritten and orphaned by the legacy-row delete. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> * fix(review): approval follow-up P3 nits (#967) * release: v0.3.1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard - The records route's sessionTransition guard now requires an ISO updatedAt (isIsoTimestamp), matching the sibling PATCH /status route - Dead vi.mock factories for the deleted playback-storage module dropped - A corrupt legacy playback timestamp falls back to 'now' instead of wedging migration into a permanent re-throw that disabled resume --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-22 15:28:04 +08:00
wyuc	34448beb6c	feat(storage): server-backed runtime — pluggable backend seam, HTTP contract, Postgres backend, reference server (#939) (#946) ... * ci: run PR checks against the runtime-server-backend integration branch (#939) * feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B) (#940) * feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B) - documented JSON HTTP contract for every RuntimeStore operation (server-assigned seq, learnerKey derived from auth — never trusted from the request, machine-readable error codes, idempotency notes) - HttpRuntimeStore client with injected fetch + auth headers hook; session reads (including createSession responses) migrate forward on the runtime line so an older server cannot leak stale envelopes - conformance test server bridging the contract onto the browser backend; full runRuntimeStoreContract green over HTTP plus error mapping cases * fix(storage): harden HTTP runtime backend after cross-review (#939 Part B) Cross-review fixes: - appendRecord gates payloads through assertJsonValue: fail loud on values JSON cannot represent (Map, Date, NaN, nested undefined, NUL strings) instead of silently mangling them in transit; payload domain documented in the contract - appendRecord/listRecords responses validate via validateRuntimeRecord (server-assigned seq is no longer trusted verbatim) and listRecords sorts by seq like listSessions already sorted defensively - request headers no longer require an ambient Headers constructor when fetch is injected - conformance server classifies errors structurally (existence checks, DSL validators, version stamps) instead of regex-sniffing messages that interpolate caller-controlled ids - router preserves empty path segments so empty-key calls keep browser no-op semantics instead of shifting onto other routes - stray runtimeDslVersion/seq in request bodies are ignored (store- assigned wins), matching the browser reference; docs updated - loopback integration test exercises the real listening server - docs state the conformance server is test-only; auth-derived learnerKey is Part D's reference server * fix(storage): round-2 cross-review hardening for the HTTP backend (#939 Part B) - json-value guard rewritten: additionally rejects -0, symbol-keyed and non-enumerable own properties, and non-index own properties on arrays; U+2028/U+2029 are accepted again (they round-trip through JSON per RFC 8259 — rejecting them lost legitimate pasted text) - conformance server classifies racing duplicate creates as 409 via a structural post-check, gates payloads and merge learner keys as 400 - segment() rejects '.'/'..' ids the URL layer would fold away - list responses must be arrays and mergeLearner's moved must be a finite number, else a typed MALFORMED_RESPONSE error * fix(storage): round-3 cross-review hardening for the HTTP backend (#939 Part B) - json-value guard: NUL is rejected in object keys too, and unpaired UTF-16 surrogates are rejected in string values and keys (jsonb refuses both; other JSON stacks corrupt lone surrogates to U+FFFD) - typed storage errors survive non-object 200 responses instead of degrading into TypeErrors on .id access - mergeLearner's moved must be a non-negative integer - conformance server classifies missing/non-object request bodies as 400 VALIDATION_FAILED - createSession/appendRecord gate the full init envelope, not only the payload (stray Date/undefined properties, NUL in ids) * fix(storage): round-4 convergence fixes for the HTTP backend (#939 Part B) - envelope JSON gate tolerates explicitly-undefined optional anchors (sceneId: undefined behaves like omission, matching the browser) - body-carried identifiers reject '.'/'..' so nothing persists that the URL layer cannot later address; conformance server mirrors it - mergeLearner keys pass the JSON-domain gate on client and server - json-value guard v4: rejects enumerable accessors (validation/serialize TOCTOU), Array subclasses and null-proto arrays, prototype-supplied array indices; sparse-array test now constructs a genuine hole * fix(storage): round-5 convergence fixes for the HTTP backend (#939 Part B) - undefined-stripping is limited to the DSL-declared optional anchors (sceneId/actionIndex/subAnchor); any other undefined member fails the JSON gate loud instead of being dropped silently - json-value guard: array and object prototype checks are realm-agnostic (chain-shape instead of identity), so ordinary values from another realm are accepted while subclasses stay rejected; shared isLosslessJsonString predicate exported for SQL key guards - conformance server merge route validates target-key addressability * fix(storage): close the toJSON prototype channel in the JSON guard (#939) toJSON is the single channel through which a prototype can alter JSON output — prototype properties never serialize and own accessors are already rejected — so refusing any value with a callable toJSON closes prototype influence on serialization entirely, including prototypes crafted to pass the realm-agnostic chain-shape check. * fix(storage): probe toJSON via descriptor walk, not property read (#939) Reading value.toJSON would execute an inherited accessor, letting a stateful getter hide from validation and reappear at stringify time. The probe now walks own descriptors up the prototype chain without invoking any user code. Post-validation mutation of the caller's object graph is documented as out of scope — it is equally unpreventable for every other validated property. * fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939) The own-most descriptor decides: absent is safe, a non-callable data value is an ordinary shadowing member and is safe, a callable data value is rejected, and an accessor is rejected because it cannot be inspected without invoking it. No daylight remains between the guard and the serializer on this property. * feat(storage): PgRuntimeStore — Postgres runtime backend (#939 Part C) (#941) * feat(storage): PgRuntimeStore — Postgres runtime backend over an injected queryable (#939 Part C) - PgRuntimeStore implements RuntimeStore over a minimal injected Queryable (node-postgres and PGlite both satisfy it) — the package keeps zero runtime dependencies beyond @openmaic/dsl - runtime_sessions / runtime_records schema exported as RUNTIME_PG_SCHEMA with idempotent ensureSchema() - appends serialize per session via a session-row lock; MAX(seq)+1 and the insert share one transaction, UNIQUE(session_id, seq) plus bounded retry backstop non-cooperating writers - envelope semantics mirror the browser backend: version stamping, validation gates, migrate-on-read, fail-loud on future-stamped rows - full runRuntimeStoreContract green on PGlite plus PG-specific cases (concurrent-append seq atomicity, ensureSchema and mergeLearner idempotence) * fix(storage): require pinned transactions + JSON payload gate in PG backend (#939 Part C) Cross-review fixes: - withTransaction is now required; the BEGIN/COMMIT fallback is removed (on a pg Pool it spread BEGIN/body/COMMIT across different connections — no real transaction, leaked idle-in-transaction clients, broken mergeLearner atomicity; on a shared pinned client concurrent calls interleaved transactions) - single-statement deletes no longer wrap in a transaction hook call - appendRecord gates payloads through assertJsonValue: fail loud on values JSON cannot represent (Map, Date, NaN, nested undefined, NUL strings) instead of silently persisting something different - append retry also covers 40001/40P01; READ COMMITTED assumption documented at the retry loop - ensureSchema documented as create-only; redundant stage index dropped - deterministic interleaving test proves the 23505 retry path; real PostgreSQL contract lane added (postgres:16 service workflow, pg driver suite skipped locally without PG_CONTRACT_URL) * fix(storage): round-2 cross-review hardening for the PG backend (#939 Part C) - json-value guard rewritten (shared with the HTTP backend): additionally rejects -0, symbol-keyed and non-enumerable own properties, and non-index own properties on arrays; accepts U+2028/U+2029 - storage-pg-contract workflow now triggers for the runtime-server-backend integration branch and fails loud (STORAGE_PG_CONTRACT_REQUIRED=1) when PG_CONTRACT_URL is missing instead of silently skipping - loadSession distinguishes corrupt non-object rows from absent rows so getSession fails loud instead of reporting the session missing * fix(storage): round-3 cross-review hardening for the PG backend (#939 Part C) - json-value guard (shared): NUL rejected in object keys, unpaired UTF-16 surrogates rejected in values and keys - createSession/appendRecord gate the full persisted envelope through assertJsonValue, not only the payload — stray Date/undefined properties and NUL in ids fail loud instead of silently diverging from the returned record or leaking raw 22P05 - real-PG lane covers a genuine 23505 conflict from a second connection and recovery after an aborted transaction - retry-set asymmetry and mergeLearner's unbounded lock set documented * fix(storage): round-4 convergence fixes for the PG backend (#939 Part C) - NUL/lone-surrogate lookup and delete keys resolve to absent/no-op instead of leaking 22021 driver errors; setSessionStatus reports the session missing; mergeLearner from-key moves 0 - mergeLearner destination key passes the JSON-domain gate fail-loud - envelope JSON gate tolerates explicitly-undefined optional anchors, matching the browser backend - json-value guard v4 (shared) + tests for accessors, Array subclasses, prototype-supplied indices * fix(storage): round-5 convergence fixes for the PG backend (#939 Part C) - appendRecord pre-checks the session key like every other lookup path, so a NUL/lone-surrogate sessionId reports 'no session' instead of leaking a 22021 driver error - the queryable-key predicate now structurally reuses the shared isLosslessJsonString export instead of restating the string rule - undefined-stripping limited to the DSL-declared optional anchors - json-value guard: realm-agnostic prototype checks (shared) * fix(storage): close the toJSON prototype channel in the JSON guard (#939) Shared guard change with the HTTP backend branch; adds the crafted null-proto-prototype regression test. Both final-gate reviewers independently converged on this same channel. * fix(storage): probe toJSON via descriptor walk, not property read (#939) Reading value.toJSON would execute an inherited accessor, letting a stateful getter hide from validation and reappear at stringify time. The probe now walks own descriptors up the prototype chain without invoking any user code. Post-validation mutation of the caller's object graph is documented as out of scope — it is equally unpreventable for every other validated property. * fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939) The own-most descriptor decides: absent is safe, a non-callable data value is an ordinary shadowing member and is safe, a callable data value is rejected, and an accessor is rejected because it cannot be inspected without invoking it. No daylight remains between the guard and the serializer on this property. * fix(storage): conform HTTP/PG backends to the post-#926 RuntimeStore interface (#943) The chat cutover (#926) added deleteAllRuntime() to the RuntimeStore contract while the HTTP and Postgres backends were developed in parallel against the pre-#926 interface. Implements the method on both backends (single-statement wipe on PG via the FK cascade; DELETE /runtime on the HTTP contract, documented as an operator-gated administrative endpoint), restoring a green typecheck and contract suite on the integration branch. * feat(runtime): injectable RuntimeStore backend + learner-key provider (#939 Part A) (#944) * wip(runtime): backend injection seam — pending gate verification * fix(runtime): cross-review hardening for the storage injection seam - stage-deletion IndexedDB probe applies only to the default browser backend; an injected store always receives deleteStageRuntime - explicit kv argument takes precedence over the configured learner-key provider, matching the store seam's explicit-beats-global rule - configured learner-key resolution is latched with in-flight dedup, mirroring the store singleton; identity changes require app-level handling - factory retry semantics documented; seal errors explain the module-level bootstrap requirement; isRuntimeStorageConfigured() probe and a test-only reset added - client-bootstrap-only contract documented (SSR/HMR caveats) * fix(runtime): snapshot configuration and make the test reset complete - configureRuntimeStorage copies the option fields so mutating the caller's object after configuring cannot swap the sealed backend or identity provider - resetRuntimeStorageForTests now clears every latched consumer cache (store singleton, learner-key in-flight promise) via a reset-hook registry, so a reset-then-reconfigure test actually gets the new backend * fix(runtime): reset also clears the default learner-key caches resetRuntimeStorageForTests left defaultInFlight/defaultKv latched, so a default-path test could leak its anonymous key or KV store into the next test despite the documented full-reset promise. * feat(storage): runtime reference server — auth-derived learnerKey (#939 Part D) (#945) * wip(storage): reference server — pending deleteAllRuntime route + gates * feat(storage): runtime reference server — auth-derived learnerKey enforcement (#939 Part D) - createRuntimeHttpHandler(store, options) wires the documented HTTP contract onto any injected RuntimeStore over node http - authenticate is required; every learner-scoped operation verifies the path/body learnerKey against the authenticated principal (403 FORBIDDEN_LEARNER) — the client-supplied value is never trusted - mergeLearner requires an explicit authorizeMerge grant and admin surfaces (stage cascade, DELETE /runtime) require authorizeAdmin; both default-deny - runnable reference entry demonstrates a pg Pool withTransaction and bearer-token authentication, marked as demo-only - contract suite green through HttpRuntimeStore -> listening reference handler -> PgRuntimeStore(pglite); security matrix tested (401/403 paths, admin default-deny); threat model documented * fix(storage): cross-review hardening for the reference server (#939 Part D) - principal learnerKey is optional: admin/merge-only credentials no longer fabricate learner identity; learner-scoped routes 403 without ownership - full session/record envelopes pass the JSON-domain gate at the handler, so NUL/lone-surrogate identifiers map to 400 instead of 500 - payload validation follows the injected store's validator map (options.payloadValidators) instead of imposing DSL defaults - reference factory accepts authenticate/authorizeMerge/authorizeAdmin/ payloadValidators overrides; docs no longer claim the factory binds to localhost; demo-impersonation warning hardened - 500 responses carry a generic message; details go to the server log - ownership checks precede version checks and unowned sessions read as 404, closing existence/version oracles; cross-learner denial matrix tested per route - merge/delete concurrency documented as linearizable-equivalent with the ownership re-check narrowed to the delete call * fix(storage): align reference-server semantics with the store contract - future-stamped sessions read and delete through unchanged; 409 FUTURE_VERSION applies only to guarded writes (status, append, merge) - check-then-write races reclassify structurally via a post-failure re-fetch: missing session 404, non-active session 400, never a message-sniffed or generic 500 * fix(storage): close the reference CLI's pg pool on startup failure ensureSchema opens connections inside createReferenceRuntimeServer, so a failed schema init or occupied port left the pool holding database resources until its idle timeout. * fix(storage): close the records-route existence oracle cosarah's review point on #946: the records list answered 200 [] for an absent session but 404 for another learner's, so the 404 leaked that an id exists. Absent and foreign sessions now answer identically (404 SESSION_NOT_FOUND) and the HTTP client maps that code back to an empty list, preserving the store contract's absent-lists-as-empty semantics. Contract doc records the server MAY/SHOULD and the client MUST.	2026-07-17 16:15:57 +08:00
wyuc	65bf20a84b	feat(runtime): cut chat sessions over to RuntimeStore (#926) ... * feat(runtime): persist chat sessions in RuntimeStore * fix(runtime): harden chat persistence ordering * fix(runtime): protect cross-tab chat snapshots * fix(runtime): compare structured chat values safely * fix(runtime): preserve chat backup and reset semantics * fix(runtime): restore imported chat snapshots * fix(runtime): isolate chat load failures * fix(runtime): make cache clearing lossless * fix(runtime): normalize empty chat saves * fix(runtime): serialize legacy chat migration * fix(chat): serialize backup restoration * test(chat): cover multi-stage restore locking * fix(chat): coordinate fallback migrations across tabs * fix(chat): harden cross-tab migration locking * fix(database): retire abandoned chat lease schema * fix(chat): preserve monotonic local edits * fix(chat): protect unseen runtime sessions * fix(chat): order lifecycle transitions monotonically * fix(chat): serialize runtime maintenance * fix(chat): order queued writes before maintenance * fix(chat): preserve cross-version lock compatibility * fix(chat): preserve streamed and backup data * fix(chat): isolate empty no-lock saves * fix(chat): isolate partial runtime snapshots * fix(runtime): close cross-store cutover races * fix(runtime): close remaining cutover races * fix(chat): retain caller-visible conflict baseline * fix(chat): preserve read-only legacy autosaves * fix(chat): avoid restore lock inversion * fix(runtime): enroll writes before maintenance * fix(runtime): retain maintenance ordering * fix(runtime): close restore and deletion races * fix(chat): order partition queues within lock epochs * fix(chat): preserve recovery and deletion intent * fix(runtime): recover interrupted chat restores * fix(backup): scope chat deduplication by stage * fix(backup): stage chats by runtime partition * fix(backup): clear staged chats with stages	2026-07-16 18:30:40 +08:00
wyuc	667f3b0c51	feat(runtime): device-anonymous learner identity + RuntimeStore app bootstrap (#869 Part C, step 1) (#885) ... * feat(runtime): device-anonymous learnerKey + app RuntimeStore singleton (#869) * feat(runtime): cascade stage deletion into the runtime store (#869) deleteStageWithRelatedData now cascades into the runtime layer after its Dexie transaction completes. The runtime data lives in a separate IndexedDB database (maic-runtime), so it cannot join the transaction, and the cascade goes through deleteStageRuntimeSafely — a helper that never throws (warns with context instead): a broken or hung runtime DB must not brick stage deletion in the main app DB. deleteStageRuntime is idempotent, so a failed cascade can simply be retried. Covered at the helper seam with a stub RuntimeStore (success + throwing); the repo has no Dexie-in-node harness for database.ts and this change does not invent one. * fix(runtime): wire the live deletion path, serialize learner-key minting (#869) Cross-review fixes on the C1 bootstrap: - The runtime cascade was wired into deleteStageWithRelatedData, which has zero callers; the UI classroom-deletion flow goes through deleteStageData in stage-storage. Wire deleteStageRuntimeSafely into that live path too (after its Dexie work, same isolation rationale), and cover the wiring by running the real deleteStageData with its module deps mocked — the repo's established pattern for database-touching code. - getLearnerKey minted twice under concurrency. Same-bundle callers now share one in-flight promise on the default path (failures are not cached), and every path re-reads after writing and returns the PERSISTED key, so a cross-tab race converges on the stored winner instead of keeping an orphaned local mint. - Guard crypto.randomUUID with the house fallback pattern. - Honest docs: the cascade comment no longer claims a retry path exists (orphaned rows are inert today; a startup sweep is deferred to Part C2), and both lib/runtime modules note they are client-only. * fix(runtime): cross-tab mint lock and bounded deletion cascade (#869) - Read-after-write alone still let a tab keep an orphaned learner key when its re-read landed before the other tab's write. Minting now runs under the Web Locks API ('maic:learner-key') where available: grants are mutually exclusive across tabs, the loser re-reads the winner's key inside its grant, and an existing key is never overwritten (so the per-tab memo stays safe). Without navigator.locks (older browsers, non-window contexts) the memo + read-after-write behavior remains, with the residual race named in a comment and accepted — it merely splits one anonymous learner's local history. - deleteStageRuntimeSafely awaited without a bound, so a hung runtime IndexedDB could block the live deletion path — the exact failure the helper exists to isolate. The cascade now races a 5s timeout: on timeout it warns and resolves (orphaned rows stay inert), and the still -pending cascade carries a swallow handler so a late rejection cannot become an unhandled rejection. * fix(runtime): probe for the runtime DB before cascading a stage deletion (#869) deleteStageRuntimeSafely reached openDb() unconditionally, and opening CREATES the maic-runtime database — so deleting a classroom on a device that never wrote runtime data paid an open-or-create of a second IndexedDB DB, and in degraded environments burned the full 5s bound for zero cleanup value. Probe first without creating: where indexedDB.databases() is available, a missing maic-runtime entry returns immediately; where the probe API is unavailable (older Firefox), fall through to the bounded cascade — skipping there would strand real cleanup once Part C2 adds writers. The probe shares the existing try/catch + timeout envelope, so a hanging databases() cannot brick deletion either. The DB name is a module const passed explicitly to BrowserRuntimeStore so probe and store can never drift.	2026-07-09 14:54:21 +08:00