trending/OpenMAIC
1
0
Fork 0
mirror of https://github.com/THU-MAIC/OpenMAIC.git synced 2026-10-17 05:29:46 +08:00
Code Issues Packages Projects Releases Wiki Activity
474 Commits 293 Branches 128 Tags
@openmaic/storage@0.26.0
Commit Graph
84 Commits
Author SHA1 Message Date
wyucandClaude Fable 5 04621578de release: OpenMAIC 1.0.0 — the agent workbench (#1228) 
* feat(storage): add an agent-session store with PG backend and layered contracts (#1163)

* feat(storage): add agent-session store with PG backend and layered contracts

* test(storage): avoid BigInt literals for pre-ES2020 root typecheck

* fix(storage): close agent-session store review findings

* docs(storage): align hook ordering and contention-probe claims with the code

* ci: run on the agent-workbench integration branch

* chore(storage): bump to 0.5.0 for the agent-session store

* fix(storage): carry replay compaction across page boundaries

* feat(agent): add the driver model contract and stage route dialect (#1165)

* feat(agent): add the driver model contract and stage route dialect

* fix(agent): validate route context windows and clarify dialect precedence

* feat(agent): adapt the agent-session store and runtime foundations (#1167)

* feat(agent): adapt the agent-session store and runtime foundations

* feat(agent): resolve request owner identity via an anonymous cookie

* docs(agent): document the opt-in compaction default and harden edge cases

* feat(agent): add the background session runner (#1169)

* feat(agent): add the background session runner

* feat(agent): wire the runner into startup behind feature flags

* fix(agent): stop clean interruptions from consuming the attempt budget

* fix(storage): charge the attempt budget for abandoned leases but not clean parks

* docs(storage): document the attempt-charging contract and decouple its tests

* feat(agent): add agent session and owner event streams (#1170)

* feat(agent): add agent session and owner event streams

* fix(agent): close the session-existence oracle and document the owner seam

* feat(agent): add agent session lifecycle routes (#1171)

* feat(agent): add agent session lifecycle routes

* fix(agent): validate session-create input and preserve the owner cookie on errors

* refactor(storage): drop the unused active-stage API from the agent-session contract (#1174)

* refactor(storage): drop the unused active-stage API from the agent-session contract

Tools address stages explicitly on every call, so the store keeps no
mutable session-level stage pointer. Removes resolveActiveStage and
setActiveStage from the store interface, their PG implementations, the
active_stage_changed lifecycle event, the session_active_stage owner
event variant, and the contract tests pinning them. The active_stage_id
column and the DDL check constraint stay untouched for schema
compatibility.

* chore(storage): bump @openmaic/storage to 0.7.0 for the contract removal

* docs: document the agent runtime configuration surface (#1176)

* fix(agent): repair orphaned and late tool results across interruption boundaries (#1180)

* fix(agent): repair orphaned and late tool results across interruption boundaries

A crash, shutdown, or provider failure can leave the durable transcript with
tool calls that have no result, or with results ordered illegally for the
provider. Three failure modes were fixed:

- Orphaned tool calls: a run that died between an assistant tool-call frame
  and its result left a dangling call in the entry tree. Resume no longer
  synthesizes and persists receipts for it: interrupted results are a
  read-time provider view owned by a shared read-boundary repair, which
  returns the original array for a healthy transcript and never mutates the
  tree.

- Late parallel results: a parallel tool can finish while pi unwinds an
  aborted assistant frame, leaving result(A), assistant(aborted), result(B)
  in durable order. Strict providers reject non-contiguous results, so the
  read-boundary repair moves existing results next to their owning assistant
  frame (in call order), omits incomplete unwind frames, and synthesizes
  receipts only for genuinely missing calls.

- Interrupted calls at the write boundary: a call still in flight when the
  run winds down (shutdown, lease loss, cancellation, provider failure) had
  no receipt at all. The runner now tracks in-flight calls from their
  assistant frames and, before the terminal flush, appends an
  interrupted-result receipt for each still-orphaned call through the same
  attempt-fenced write chain, so a lease-stealing zombie never writes and
  the next claim sees a provider-safe transcript.

* test(agent): pin the runner wiring for interruption-boundary tool repair

* feat(agent): add neutral tool foundation libraries (#1184)

* feat(agent): register a web_search tool on the session runner (#1185)

* feat(storage): add a per-session URL trust gate (#1186)

* feat(agent): add the skills system (#1189)

* feat(agent): add the skills system (builtin directories and durable user skills)

* fix(storage): serialize the user-skill quota check-and-insert per owner

Two concurrent creates at the 50-skill boundary both counted 49 rows and
both inserted (READ COMMITTED, no lock), overshooting the quota contract.
The create transaction now takes a per-owner pg_advisory_xact_lock first,
and the same-name idempotency check runs before the count check so an
at-least-once retry of the create that committed as the owner's 50th row
still returns its durable receipt instead of a quota error. The 23505
backstop is retained for writes that do not take the lock.

* fix(agent): share unstorable-character validation and align skill lookup

* feat(agent): add session materials and a fetch_url tool behind the URL trust gate (#1190)

* feat(agent): add session materials and a fetch_url tool behind the URL trust gate

* fix(agent): harden session material fetching

* feat(storage): add an ownership scope to stage documents (#1191)

* feat(agent): add material read and search tools (#1192)

* feat(agent): add stage read and patch tools (#1194)

* feat(agent): add page generation and deck editing tools (#1198)

* test(storage): keep the PG contract suite order-independent (#1200)

* fix(agent): revoke deleted-session URL authority and reject private ISATAP endpoints (#1199)

* fix(storage): revoke deleted session URL authority

* fix(ssrf): reject private ISATAP endpoints in strict fetches

* chore(storage): bump to 0.11.1 for the session-URL authority fix

* feat(agent): add roster and voice registration tools (#1201)

* feat(agent): add folder organisation tools (#1202)

* feat(api): add stage and material HTTP routes (#1203)

* feat(workbench): add the client data layer (#1204)

* feat(workbench): add the client data layer

* docs(workbench): write the ported comments in English

* chore(edit): remove the in-editor agent panel (#1210)

* chore(edit): remove the in-editor agent panel

* style: apply prettier formatting

* fix(agent): report the runtime as unusable without a database (#1207)

* fix(agent): report the runtime as unusable without a database

* style: apply prettier formatting

* feat(agent): add image, video and pptx import tools (#1211)

* feat(workbench): add the agent chat surface (#1205)

* feat(workbench): add the agent chat surface

* docs(workbench): write the ported comments in English

* fix(workbench): label the folder and rename tools on the timeline

* fix(workbench): label the roster and voice tools on the timeline

The reconciliation test iterates every tool the runner registers and
requires a display label of its own. The roster and voice-clone tools
(list_voices, set_roster, clip_audio, register_voice) reached the
integration base with the roster/voice-registration tools but never
gained presentation rows, so they fell through to the default branch
and rendered their wire names. Port their rows from the reference
implementation (labels and i18n keys verbatim) and extend the
reconciliation allowlist with ROSTER_TOOL_NAMES and
VOICE_CLONE_TOOL_NAMES, so a future tool cannot enter the product
without a label.

* feat(agent): add the material extraction lifecycle (#1212)

* feat(storage): add material extraction lifecycle

* feat(agent): execute queued material extraction

* style: apply prettier formatting

* style: satisfy prefer-const in the extraction runner

* test: give material fixtures the extraction lifecycle fields

The media-tools slice and the extraction lifecycle slice were each green
in isolation but never compiled together: the lifecycle made derivedFrom
and extraction required on AgentSessionMaterial while the media-tool
fixtures predate them.

* chore: remove stray task notes

* fix(workbench): label the extraction lifecycle tools on the timeline

* feat(workbench): add the workspace shell (#1206)

* feat(workbench): add the workspace shell

* docs(workbench): write the ported comments in English

* i18n(workbench): align workspace keys across locales

* fix(workbench): adopt the landed data layer and label the extraction tools

- replace the sibling-slice seam stubs with the real data-layer modules
- drop ambient declarations now shadowed by landed files
- port timeline labels for the extraction lifecycle tools from the reference
- align the new i18n keys across all locales

* ci: retrigger

* feat(api): folder routes, stage-meta viewer surfaces, and the material upload contract (#1215)

* fix(storage): restore capability-based stage access

* fix(api): bind document access to request owner

* fix(agent): restore three-state stage access on the tool layer

Port probeStageAccess and the three-state StageAccess (owned / foreign /
missing / tombstoned) and gate every stageId-bearing stage tool on an owned
probe, mirroring the reference per tool:

- move_to_folder, rename_stage, read_stage_outline refuse a non-owned stage
  with the single not-yours message before touching the store.
- The course/DSL toolset and the roster toolset are wrapped by
  withOwnerStageAuthorization: read_stage, patch_stage, grep_stage and every
  writer refuse a foreign stage with the same message and refusal shape.
- Scene preview keeps its own probe and its own refusal text, and is
  registered beside the course toolset (never double-gated).
- The runner injects one probe factory at the three call sites.

Tests: the dsl cross-owner test premise (a foreign stage is readable by id)
encoded an invented capability-read policy that the reference does not have
at the tool layer; it now asserts foreign read/patch/grep are all refused
while the owner still reads. Curriculum cross-owner assertions were already
the reference's and now pass with the probes in place.

* docs: correct per-file test counts in the fidelity report

* test: fix type errors in stage-access fidelity test

* test: adapt media-tool and gate suites to the owner-scoped store seam

* feat(api): add owner-scoped course-folder HTTP routes

Port the reference implementation's /api/folders family (list, create,
rename, delete with ungroup/remove modes, and folder membership) onto the
owner-bound document store, replacing its provider-based auth with the
existing withRequestOwnerId / owner-scoped store seams.

The storage package's folder store grows the pieces the routes need:
DocumentFolder.order (schema column + max+1 assignment + ordering),
renameFolder, deleteFolder(mode) with captured member ids, and
setStageFolder(stageId, folderId | null) with idempotent un-filing.
FolderNameError moves into folder-name-validation.ts (stage-storage re-exports
it, keeping import sites intact).

Every route gates on the configured agent runtime (plain 404 when off or
unconfigured), keeps the reference's machine codes and envelopes, and is
covered by gate tests plus a behavior suite.

* feat(api): add stage-meta viewer surfaces for the classroom

Port the reference implementation's viewer-facing stage state — can-edit /
collected / published / generation-complete — on top of the stage-access
base (stage_meta + tombstones). stage_meta gains published_at and
generation_complete columns plus a stage_bookmarks table; the reference's
deployment-specific origin/claimed_at columns are stripped.

New gated routes: GET /api/stage-meta/[stageId] (per-viewer facts, 404 for
absent/tombstoned, never returns the owner id), GET /api/stages/[id]/status,
POST generation-complete / publish / unpublish (owner-only), POST
/api/bookmarks. The resolver lives in lib/server/stage-access.ts.

Wiring: a fetchStageMeta client with the reference's three-outcome contract,
stage-store isOwner/isBookmarked/readOnly fields (upstream single-user
defaults, no-op until the sidecar answers) plus setViewerAccess, the
classroom apply path computing readOnly = !(isOwner || isBookmarked), the
Stage editability gate, and a sidecar probe after each classroom load. A
sidecar 'absent' answer keeps the editable default here because the
classroom also serves local-only courses; server writes stay owner-enforced.

* feat(api): port the reference material upload contract

Rewrite POST /api/materials to the reference implementation's upload shape
so the workbench uploader (uploadWorkbenchMaterial, which posts no session
id and expects a flat 201 view) works unchanged: owner-scoped upload with
mime normalization/validation (415), per-class size caps checked on the
declared content-length and the streamed body (413), empty body (400),
quota (429), sha256 reserve->store->finalize lifecycle with abandon on
failure, flat { materialId, originalName, bytes, mime, extraction } 201,
and an x-request-id echo.

Adds the owner-scoped material library (owner_material table + quota +
24h lazy sweep, bytes in the host's asset registry as the neutral
replacement for the reference's object-storage byte path) and the material
cap configuration. The session-scoped GET list is left as-is; the
reference's owner-material extraction worker is not ported (the branch's
session-material extraction lifecycle already covers extraction).

Gate tests now cover all 23 persistence routes across the three runtime
env states; the materials behavior suite pins the new contract.

* feat(media): add an optional local ffmpeg media extractor (#1213)

Adds a local ffmpeg/ffprobe pipeline as a second media extraction
provider behind the extractor registry, ported faithfully from the
reference implementation: duration probing, keyframe-safe chunking,
per-chunk ASR with timeout and deadline budgets, and timestamped
transcript assembly.

- Availability probing feeds the registry's candidate selection: the
  provider simply is not a candidate when ffmpeg/ffprobe are absent.
- With neither ffmpeg nor a cloud provider configured, extraction fails
  with an actionable message naming both enablement paths.
- Media materials route through the same extraction lifecycle and lease
  fence as documents; no parallel queue.
- Tests inject the executable resolver so the missing-ffmpeg path is the
  default-tested one; the real pipeline test is skip-if-unavailable.
- @openmaic/storage 0.13.0 -> 0.14.0 (media routing in the material
  lifecycle surface).

* feat(storage): per-scene monotonic revisions via database triggers (#1214)

* feat(storage): per-scene monotonic revisions via database triggers

Restore the reference implementation's freshness granularity: a
per-scene monotonic revision maintained by database triggers, so every
writer (HTTP routes, agent tools, jobs, manual SQL) bumps it without
application cooperation.

- Companion revision tables + trigger functions in the storage package's
  idempotent schema bootstrap, with the lock-order invariant, pg_notify
  wakeup and the suppression switch for batch writers.
- ensureDocumentSchema gained a dollar-quote-aware statement splitter.
- The freshness and manifest routes serve per-scene revisions.
- Mutation-verified: dropping the triggers turns the revision tests red.
- @openmaic/storage 0.13.0 -> 0.14.0.

* fix: forward the freshness manifest through the owner-bound store

* feat(workbench): add the Pro entry points and preserve the mode-transition semantics (#1208)

* feat(workbench): add the Pro entry points

* feat(workbench): preserve Pro mode transition semantics

* fix(workbench): drop ambient declarations shadowed by landed slices

* fix(workbench): drop ambient declarations shadowed by the landed shell

* feat: port workspace shell sibling modules

Port the 16 leaf modules the Pro workspace shell imports but that were only
ambient-declared, replacing the compile-time bridge with real implementations
adapted from the sibling-slice reference: pure workbench helpers (session
title, rail tab, course-chat bootstrap, created-course tabs, course-tabs
memory, workspace navigation, pane navigation, pro-edit sizing,
existing-course minting, first-message session), the neutral brand context and
course-rename server API, the server-action session delete, the home
discovery hook, the classroom pane host with its load-policy leaf, the theme
toggle and floating-layer owner, plus the floating-layer-owner wiring the
dialog/dropdown/tooltip portals stamp.

Also add the workbench-shell locale copy for all 12 locales, port the
reference tests for the ported modules, and drop
types/workbench-sibling-slices.d.ts now that every declaration has a real
implementation.

* docs: keep ported comments in English and deployment-neutral

* docs: announce 1.0.0 and refresh the feature overview (#1216)

* docs: announce 1.0.0 and refresh the feature overview

* docs: finalize 1.0.0 README after feature merge

* fix(agent): control-plane routes answer 404, not 500, without a database

The agent control-plane routes gated only on the runtime flag, so an
enabled-but-unconfigured deployment (flag on, DATABASE_URL empty)
answered 500 from a store that cannot connect. Gate them on the
configured check instead, matching the stage/material routes: the
whole surface is cleanly absent until both the flag and the database
are present. The status probe keeps reporting both bits.

* test: mock both runtime gate exports in the control-plane route suites

* fix(agent): abort in-flight TTS on cancel and bound each provider request with a timeout (#1217)

The generate_tts / scene-tts path checked the runner's AbortSignal between
actions but never created the provider HTTP requests with it, so a session
cancel left a hung synthesis fetch in flight until a restart repaired the
tool result. Thread the signal end-to-end: TTSModelConfig carries an
optional signal, generateTTS combines it with a per-request timeout
(TTS_REQUEST_TIMEOUT_MS, default 30s, ported from the reference runtime's
TTS bounds) via AbortSignal.any, and every provider fetch (openai, azure,
glm, qwen incl. voice-clone + audio download, voxcpm, minimax, doubao,
elevenlabs, lemonade) is created with that signal.

A timeout now fails the tool call with TTSRequestTimeoutError (a clear
retryable error) instead of wedging the session; a caller cancel propagates
as the interruption so the runner settles the session as cancelled without
a restart.

Tests: hung-provider simulation rejects at the timeout with the retryable
error; abort mid-flight aborts the captured request signal and surfaces the
interrupted shape; removing the signal wiring makes the abort tests fail
(red), restoring them turns green.

* fix(workbench): PG-mode home listing via owner stages; keep the interrupted terminal course card (#1218)

Finding 1: with server persistence on, listStages resolved to the generic
GET /api/persistence/documents listing, which the capability model
deliberately answers 403 FORBIDDEN_DOCUMENTS for (reads by id, listings
owner-only). The home/workspace library now lists through the owner-scoped
GET /api/stages surface (same anonymous-owner cookie the workbench uses)
when server persistence is enabled; the server-side 403 is untouched.

Finding 2: a run interrupted (session_interrupted) and repaired
(session_resumed) that ends cancelled before agent_end stranded its pending
classroom sightings, so the timeline's terminal card lost the course the
answer produced. session_end (cancelled) now flushes the pending sightings
into the same course card set agent_end paints, before the stopped caption.

* chore(workbench): remove the bookmark concept and the saved-courses drawer (#1219)

* chore(classroom): remove the bookmark ('collected') concept entirely

The stage-meta viewer port introduced a bookmark surface (stage_bookmarks
table, POST /api/bookmarks, the isBookmarked sidecar field, and a
readOnly rule that let a saved course stay editable). The product has no
such concept, so remove it as a closure:

- delete the /api/bookmarks route and the stage_bookmarks table plus its
  query helpers from the persistence bootstrap
- drop isBookmarked from GET /api/stage-meta/[stageId]
- simplify the classroom read-only rule to readOnly = !isOwner across the
  sidecar client, ownership signal, classroom load, stage store and the
  classroom page
- keep publish/unpublish, generation-complete, isOwner and isPublic
  exactly as they were
- update the gate and stage-meta route suites and the README mentions

The workspace rail's Bookmark glyphs and comments describe the upstream
saved-courses (favorites) section, which is driven by isOwner and renders
no collect affordance; they are kept as unrelated homonyms.

* chore(workbench): remove the saved-courses drawer UI

The first pass removed the bookmark data model but kept the rail's
"Saved courses" drawer, judging it a separate surface driven by
`isOwner === false`. The home/workspace listing is owner-scoped, so that
flag can never occur: `allSaved` is permanently empty and the drawer
(plus the collapsed-rail Bookmark mini-button) is a dead affordance.
Remove it: the SavedDrawer component and its mount, the savedOpen /
savedSection state, the allSaved / matchedSaved derivations, the 'saved'
variant of the course-list renderers, the mini Bookmark glyph, the
drawer-only CSS, and the drawer's i18n keys from all 12 locales. The
courses tab is now exactly one folders tree.

The authored/favorites split in workspace-tree.ts goes with it; the tree
module no longer reads `isOwner`. The discovery course type keeps the
field — the shell still reads it for read-only gating.

Upstream has no collect concept; the drawer could only ever render empty
here. The reference implementation HAS this drawer (its favorites come
from its account system), so this removal is a deliberate upstream
product decision, not a fidelity bug.

* fix(workbench): restore the attach entry, add the rail settings entry, pin all three entry points (#1221)

* fix(workbench): restore the composer attach entry by gating it on the live runtime

The AttachButton's rollout probe read a `materialsEnabled` field that this
branch's /api/agent/runtime never answers (the materials routes gate on the
runtime itself, like the stages), so the gate could never pass and the attach
button never rendered — the Pro launch and chat composers showed only the
@-mention and enhance glyphs.

Substitute the field with the runtime's `enabled` value, which IS the upload
action's precondition: POST /api/materials answers 404 whenever it is false,
so the render condition now equals the action precondition (no dead button).

The button's label (`proMode.attach`) is a user-visible string that becomes
visible again; port the reference implementation's own translations verbatim
into the 11 locales that still carried the Chinese copy.

* feat(workbench): add the settings entry to the rail's bottom-left cluster

The reference's rail foot carries a cluster of utilities (its saved-courses
drawer, the language switcher, the display toggle). This branch removed the
drawer — it could only ever render empty here — and the product decision is
to fill that freed spot with the settings entry.

Add a settings trigger to the foot cluster (expanded rail, beside the
language and display toggles, and on the collapsed strip) and mount the
model/provider SettingsDialog in the rail, wired to the trigger. It is the
same dialog the classic home opens from its header pill; the workspace had no
settings entry of its own, so nothing is duplicated within a surface.

* test(workbench): pin the restored upload, attach, and settings entry points

Covers the three restored entry points:

- the courses-tab upload control: rendered beside the course name filter,
  wired to the discovery hook's ZIP import trigger, disabled while an import
  runs, and gated by the same condition as its action (the courses tab);
- the composer attach control: an actual render of AttachButton under both
  probe answers (visible when the runtime says the upload path is live,
  hidden otherwise), its mounts in the launch and chat composers, the
  branch's runtime-field substitution in the probe, and the reference's own
  `proMode.attach` copy in all 12 locales;
- the settings entry: the trigger in the rail's foot cluster (expanded and
  collapsed), beside the language and display toggles, opening the
  SettingsDialog the rail mounts.

* chore(config): the Pro workbench flag implies the MAIC Editor gate (#1223)

A workbench build without the editor toggle has no way to edit a course:
enabling NEXT_PUBLIC_PRO_WORKBENCH_ENABLED while forgetting
NEXT_PUBLIC_MAIC_EDITOR_ENABLED produced exactly that split-brain bundle.
The workbench IS Pro mode, so its flag now implies the editor gate; the
standalone flag remains for deployments that want the classroom editor
without the workbench. Documents both flags in .env.example.

* fix(agent): wake SSE tails and the runner on durable deltas (streaming fidelity) (#1222)

The Pro workbench chat did not stream: the session/owner SSE routes polled
the durable event log on a 5s/30s clock with no wakeup, so message_update
deltas (written at 150ms cadence) reached the browser in poll-sized blocks
and the thinking strip only mounted after the whole reasoning text had
accumulated.

Port the reference's LISTEN/NOTIFY delta path:

- storage: add in-transaction wake hooks (onSessionEventAppended,
  onOwnerEventAppended, onCancelRequested) so a host queues pg_notify in
  the same transaction as the durable append; align readEventsAfterForReplay
  to rank the bounded page so the first delta after the cursor is always
  kept (the live tail can never starve). Bump @openmaic/storage to 0.18.0.
- app: port the process-wide event-notify bus (dedicated LISTEN client,
  self-check probe, reconnect backoff; notify through the storage
  transaction surface), wire the store hooks, subscribe both SSE routes
  before the initial read with the reference's initializing gate, and give
  the runner one {kind:'session'} subscription whose wake runs the cancel
  check and the message drain. Polls stay as the lossy-NOTIFY backstop.
- lifecycle: start/stop the bus from instrumentation.

Tests: storage hook + compaction contract; route wakeup latency; runner
wakeup wiring with a fake agent; bus unit tests; PG contracts proving a
real append wakes the routes and a live SSE route forwards a message_update
on the wakeup, and that a rolled-back append never wakes. Also fix the
pre-existing park-attempt-budget PG test TRUNCATE (missing CASCADE against
newer FK tables).

* fix(storage): asset writes self-deadlocked against pooled PostgreSQL (#1225)

* fix(storage): refuse the non-transactional byte-write deadlock configuration

A byte store whose plain write() runs on its own pooled connection cannot be
invoked from inside a registry write transaction: after the transaction has
claimed the blob-row lock, that write blocks on the lock the transaction just
took while the transaction waits on the write - a self-deadlock PostgreSQL
cannot detect (one side is idle in transaction).

There is no lock-safe ordering for such a writer: bytes must be written after
the row claim (writing before it lets the collector delete the bytes while the
upsert waits), and any second-connection write after the claim is the
deadlock. The configuration is therefore detected and refused:

- AssetByteStore gains writesOutsideRegistryDatabase?: true, declaring that the
  layer's plain byte operations cannot contend for the registry's row locks.
- PgAssetStore refuses put()/replace() up front (and defends coordinatedWrite)
  when the byte store has no writeWith and does not declare the flag, throwing
  a clear configuration error before any row is claimed.
- The collector mirrors the guard on its delete path (deleteWith or a declared
  out-of-registry layer, else a configuration error).
- The object store declares the flag (its out-of-transaction write remains
  legitimate); the in-registry PostgreSQL byte column provides writeWith /
  deleteWith instead.
- Write transactions (put/replace/remove) set SET LOCAL lock_timeout = 30s so
  any future lock-contention variant fails loudly instead of hanging.

Bumps @openmaic/storage to 0.18.0.

* fix(persistence): forward the transactional byte methods through the lazy asset byte-store wrapper

The no-bucket case of lazyAssetByteStore returned a bare { write, read,
delete } and dropped writeWith/readWith even though the underlying
PgAssetByteStore has them. The registry's hasTransactionalWriter duck check
then failed and put() fell back to the byte store's own pooled connection,
which blocks forever on the blob-row lock the registry transaction just took
when the bytes live in the same PostgreSQL - the production self-deadlock.

The no-bucket layer is statically PgAssetByteStore, so its transaction-pinned
methods are forwarded eagerly (typed against the real signatures via
PgForwardedByteStore). The bucket case keeps its lazy-probing semantics: no
transactional writer exists there, the signed-URL method stays absent or lazy
exactly as documented, and the wrapper now declares writesOutsideRegistryDatabase
so the registry may run the plain write inside its transaction.

New tests pin the wrapper's transactional capability red-to-green and assert
put()/resolve() route byte traffic through the transaction-pinned queryable.

* fix(home): cap the generate-prep ingest drain at 3s so Generate never waits the full server budget

The classic home flow's Generate click drained in-flight ingests for the full
15s server budget. Cap the wait at GENERATE_DRAIN_CAP_MS (3000ms, documented as
a UX bound) and reuse the existing timeout fallback: sources that miss the cap
proceed on the legacy byte path and each late-resolving id is released.

* chore(storage): bump to 0.19.0 over the concurrently landed 0.18.0

* fix(agent): bound every tool call with a timeout; never resurrect a cancelled session (#1226)

* fix(agent): bound every tool call with a global timeout and settle it on cancel

A tool await that neither resolves nor rejects wedges the session forever:
the lease keeps heartbeating and the driver never reaches its next cancel
checkpoint. Race every tool execution (in buildAgent) against a hard budget
(OPENMAIC_AGENT_TOOL_TIMEOUT_MS, default 10 min, per-tool overrides for
known long runners) and against the caller's AbortSignal, so even a
signal-ignoring await cannot keep a cancelled session running.

On timeout the call rejects with AgentToolTimeoutError; the agent loop turns
the rejection into a structured error tool-result the agent can retry or
proceed from, and the abort signal is delivered to the tool's in-flight work
through a derived controller. Zombie-tool updates after settlement are
dropped.

* fix(storage): never re-lease a cancel-requested session; settle it as cancelled on claim

The claim scan treated a session with cancel_requested_at set as a normal
claim candidate: after a restart it re-leased the same session for attempt
N+1 and resumed generating despite the pending cancel. claimNextSession now
settles such candidates as cancelled under the claim lock (status cancelled,
attempt reset, lease and cancel request cleared, terminal session_end event
and owner projection) instead of leasing them, then keeps scanning.

Bump @openmaic/storage to 0.18.0.

* docs: takeaway-style 1.0.0 announcement with bilingual guide links

The 1.0.0 head is now a short takeaway block — badge links to the
official user guides (English and Chinese), five one-line highlights,
and pointers into Features and the workbench setup section — instead of
six dense paragraphs. The detailed provider-neutrality and freshness
notes move into the Features workbench section, phrased database-
neutrally (the announcement no longer names a specific database).
Release date corrected to August 27.

* fix(workbench): restore editor chrome, mode transition, streaming, materials, mentions, folders (#1229)

* fix(workbench): wire workspace folder routes

* fix(editor): restore reference workbench chrome

* fix(workbench): persist composer materials and course refs

* fix(workbench): preserve live reasoning frames

* fix(persistence): back off failed streaming saves

* chore(workbench): retire stale slice seams

* test(editor): cover element pin layer

* chore(storage): bump to 0.21.0 for the user-message ref/material fields

* chore(editor): translate ported code comments to English

* fix(agent): fence durable tool writes and consume cancel requests atomically (#1230)

* fix(agent): enforce provider force-off in agent tools and scrub vendor identity from tool results (#1231)

* fix(materials): serialize per-owner quota reservations and make crashed uploads reclaimable (#1232)

* fix(editor): resolve dock-bar i18n keys, remove dock height drag, wire element referencing (#1233)

* fix(workbench): send the opening session message exactly once with refs intact (#1234)

* feat(editor): port timeline TTS preview single-flight and voice-all state latching (#1235)

* fix(media): restore the reference classic media chain (#1236)

* fix(import): adapt imported PPTX canvas size so decks render without overflow (#1237)

* fix(editor): complete element referencing — renderer DOM contract and GenUI picking aligned with the reference (#1238)

* test(providers): reconcile the provider-config vendor-token debt count after the main merge

The integration line's AK/SK fallback for the managed document provider adds
occurrences that main's allowlist snapshot predates. Same mixed-composition
debt category the group already documents; no new vendor behavior.

* test(providers): reconcile vendor-token debt counts with the integration line

The main-merge brought main's neutrality-guard snapshot next to integration
features it predates (media-extractor fallback chain, local voice-profile
deletion semantics, the enabled-TTS helper). Same debt categories the guard
already documents; counts updated to the guard's own tally and two grouped
entries added. No new vendor behavior.

* fix(agent): carry reasoning through the completions dialect so the thinking strip renders (#1239)

* feat(skills): add Feynman and spiral curriculum methods (#1240)

* feat(agent): port missing reference tools and skills (parity audit) (#1241)

* feat(media): retire asset-registry wiring; media and materials follow the reference byte model (#1242)

* fix(classroom): center adapted canvases in the stage and send back navigation home during generation (#1243)

* feat(settings): skill management with real list, download, delete, and upload (#1244)

* feat(settings): skill management section with real list, detail, and zip download

* feat(skills): owner skill delete and upload across storage, API, and settings

* fixup! feat(settings): skill management section with real list, detail, and zip download

chore: neutralize a reference note in the settings header comment

* fix(media): persist origin-independent classroom-media references from the agent runtime (#1245)

* feat(editor): float the insert toolbar in the outer frame with collapse (#1246)

The insert strip was bounded to the slide card, so it could only ever sit on
top of slide content: the card's overflow clipped it and it could not be parked
in the padding beside the slide. Move it into the studio frame the element
picker's panel already roams (CanvasOverlayPortal + the frame selector), so both
canvas overlays share one bounding container and their handles behave the same.
While picking, the strip rises over the picker and goes inert, which is the
z-order CANVAS_OVERLAY_Z already documents.

Add a fold beside the grip: the chevron collapses the strip to that grip row and
back, with the buttons unmounted rather than hidden. The fold is session-local
state owned by EditShell, next to the drag offset, so a surface swap keeps it;
nothing is persisted. Expanding a strip parked at the bottom edge re-clamps
through the same bounds rule the keyboard move uses.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* fix(workbench): align the chat timeline's left edge with the composer (#1247)

* fix(agent): fence session claims while an ask_user question is outstanding (#1248)

* fix(agent): settle-time rescue tracks real delivery instead of a count offset (#1249)

* fix(persistence): migrate owner_material to oss_key and drop legacy asset_id (#1250)

* docs(readme): surface the 1.0.0 user guide badges at the top (#1253)

* fix(workbench): show newly created folders in the sidebar without reload (#1254)

* docs(readme): add the release version prefix and drop the opt-in framing

* fix(workbench): single-source the chat gutter so timeline and composer share a left edge (#1255)

The transcript and the composer each established their own column: their own
`px-*` gutter and their own `mx-auto w-full max-w-*` centering wrapper. Equal
padding values were never enough, because the two columns are centered inside
different containing blocks — the transcript's is a scroll container, whose
content box is narrower than the composer footer's by the scrollbar's width:

  transcript text left = pad + (pane - 2*pad - scrollbar - measure) / 2
  composer box  left   = pad + (pane - 2*pad             - measure) / 2

The padding cancels out of the difference and what remains is `-scrollbar/2` at
every padding value, so the transcript sat half a scrollbar to the left of the
composer and tuning the two paddings against each other could not move it.

The column is now established once, by the nearest common ancestor of both
(`chatColumn`), and the scroll viewport and the composer footer are siblings
inside it that add no horizontal inset of their own. The cap carries the gutter
on top of the 760px reading measure, so the text column keeps its width. The
handed-over question row drops the padding that indented it past the agent's
prose; framed rows keep their own inner padding, which is what a card's border
sitting on the column edge means.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* fix(workbench): lock pane-embedded classroom to edit mode (#1256)

The workspace right pane painted the full learning chrome — speed control,
play button, learner avatars, mic bar — for a course the agent had just
created, then flipped to edit once the first scene landed.

resolveStageChromeMode treated playback as the DEFAULT branch for a hosted
classroom, so every shortfall fell into it: a course whose tab opens at
stage_link time has no scenes yet, so currentSceneId is null and
isHostedSceneEditable is false. A folded pane parked the playback root
behind the fold and cross-faded it out over the pane on unfold, and a
failed editor chunk dropped into playback permanently.

Lock it at the pane instead of defaulting per entry path:

- WorkbenchPanelProvider — the single element that mounts a classroom into
  the workspace — publishes editPinned (visible && !playback). Every entry
  path passes through it, so none of them decides.
- The hosted resolution can no longer degrade to playback. Start Learning
  (workbenchLearning, new input, split out from pane visibility) is the one
  door; everything else resolves between the neutral loading shell and edit.
- Stage's chrome dispatch is exhaustive on chromeMode, so the playback root
  is no longer the else-branch of a condition about the current scene.

No flicker: chromeMode is resolved during render, and preloadEditor now
answers synchronously (isEditorPreloaded) so a remount with the chunk
already registered paints edit on the first frame. A failed import is no
longer cached forever, so the lock cannot strand the pane.

Standalone classrooms keep their stored mode unchanged.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-27 19:26:25 +08:00
wyuc 97ceb11a14 fix(storage): asset writes self-deadlock against pooled PostgreSQL (#1227) 
* fix(storage): refuse the non-transactional byte-write deadlock configuration

A byte store whose plain write() runs on its own pooled connection cannot be
invoked from inside a registry write transaction: after the transaction has
claimed the blob-row lock, that write blocks on the lock the transaction just
took while the transaction waits on the write - a self-deadlock PostgreSQL
cannot detect (one side is idle in transaction).

There is no lock-safe ordering for such a writer: bytes must be written after
the row claim (writing before it lets the collector delete the bytes while the
upsert waits), and any second-connection write after the claim is the
deadlock. The configuration is therefore detected and refused:

- AssetByteStore gains writesOutsideRegistryDatabase?: true, declaring that the
  layer's plain byte operations cannot contend for the registry's row locks.
- PgAssetStore refuses put()/replace() up front (and defends coordinatedWrite)
  when the byte store has no writeWith and does not declare the flag, throwing
  a clear configuration error before any row is claimed.
- The collector mirrors the guard on its delete path (deleteWith or a declared
  out-of-registry layer, else a configuration error).
- The object store declares the flag (its out-of-transaction write remains
  legitimate); the in-registry PostgreSQL byte column provides writeWith /
  deleteWith instead.
- Write transactions (put/replace/remove) set SET LOCAL lock_timeout = 30s so
  any future lock-contention variant fails loudly instead of hanging.

Bumps @openmaic/storage to 0.5.0.

* fix(persistence): forward the transactional byte methods through the lazy asset byte-store wrapper

The no-bucket case of lazyAssetByteStore returned a bare { write, read,
delete } and dropped writeWith/readWith even though the underlying
PgAssetByteStore has them. The registry's hasTransactionalWriter duck check
then failed and put() fell back to the byte store's own pooled connection,
which blocks forever on the blob-row lock the registry transaction just took
when the bytes live in the same PostgreSQL - the production self-deadlock.

The no-bucket layer is statically PgAssetByteStore, so its transaction-pinned
methods are forwarded eagerly (typed against the real signatures via
PgForwardedByteStore). The bucket case keeps its lazy-probing semantics: no
transactional writer exists there, the signed-URL method stays absent or lazy
exactly as documented, and the wrapper now declares writesOutsideRegistryDatabase
so the registry may run the plain write inside its transaction.

New tests pin the wrapper's transactional capability red-to-green and assert
put()/resolve() route byte traffic through the transaction-pinned queryable.

* test(providers): reconcile the neutrality guard with current main and this fix

Two changes that were each green alone broke together on main: the
provider audit removed the lib/storage barrel and types entry points
and reshaped provider references while the guard still pinned the old
file list and counts. Point the file list at the surviving
lib/storage/client.ts, drop the satisfied sora debt, and update the
extract-document mineru count. The asset-byte-store pg count grows
because the wrapper now forwards the transactional byte methods of the
concrete store, which necessarily names it.
 2026-08-27 12:50:31 +08:00
wyucandClaude Fable 5 4f00cf7ad6 feat(materials): material library manifest and id-addressed generation images (#1153 part 2) (#1168) 
* feat(materials): material library manifest and id-addressed generation images (#1153 part 2)

* fix(materials): library-owned references and failure-path parity (#1153 part 2)

* fix(generation): resolve-then-slice vision images and close allocation leaks (#1153 part 2)

* fix(generation): bound vision resolution and make library writes leak-safe (#1153 part 2)

Bound the resolve-with-refill phase with an aggregate 15s budget raced
against every probe plus a 3-consecutive-failure fuse — either stop strips
unresolved and unprobed ids from the mapping and proceeds (text-only at
worst), never failing the request. Restore the original interleaved text
ordering for non-vision runs with a mapping present. Guard the library's
failed-entry-write release with an ambiguous-commit re-read: release only
when the entry provably does not point at the new id, keep it when the
write actually landed, and prefer a recoverable leak over a dangling entry
when the re-read fails. Drop the unused package export.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(generation): bump @openmaic/generation to 0.3.1 (#1153 part 2)

The package's inputs changed (partitionImagesForVision, resolvedVisionImages
option, non-vision ordering fix); the merge-time gate requires a version bump.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-24 18:08:40 +08:00
xuyuanwei678andwyuc 838a214484 fix(editor): allow dragging selected lines (#1166) 
* fix(editor): allow dragging selected lines

* chore(editor): bump package version to 0.0.4

* test(editor): remove hardcoded package version assertion

* test(editor): preserve manifest provenance coverage

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-24 16:43:42 +08:00
wyuc e3fd494658 feat(tts): add Qwen TTS voice cloning (#1160) 
* feat(tts): add Qwen TTS voice cloning via the registration adapter

* feat(settings): add Qwen TTS clone-voice manager

* fix(tts): harden Qwen voice cloning integration

* fix(tts): enforce voice-bound Qwen synthesis

* fix(tts): close Qwen voice clone tail issues

* fix(tts): close final voice clone review issues

* fix(agents): harden voice token parsing

* fix(settings): hide the voice-clone model from manual selection

* fix(agents): pin the narrator voice to the user's selected voice

* fix(agents): keep narrator fallback alive when the pinned voice is unusable

* fix(i18n): correct the narration fallback notice copy

* fix(tts): bound the narrator voice retry to a single fallback hop

* fix(tts): tolerate transient vendor errors in the voice existence precheck
 2026-08-24 14:33:43 +08:00
LING DUANandwyuc a0dccbe78d feat(pi): add Native Child additive whiteboard tools (#1156) 
* feat(pi): add Native Child additive whiteboard tools

* chore(renderer): bump package version to 0.1.3

* fix(pi): reject invisible whiteboard mutations

* fix(pi): close native whiteboard review blockers

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-24 12:12:07 +08:00
Hugo Wongandwyuc 1632897103 Fix Traditional Chinese translations in importer index.html (#1127) 
* Fix Traditional Chinese translations in importer index.html

Convert Simplified Chinese characters to Traditional Chinese:
- 文件 → 檔案
- 数据 → 資料
- 支持 → 支援
- 依赖 → 依賴
- 复制 → 複製
- 下载 → 下載
- 点击 → 點擊
- 页面 → 頁
- 服务器端 → 伺服器端
- 浏览器 → 瀏覽器
- 执行 → 執行

Also fixes layout資訊 → 布局資訊

This ensures the web interface displays correct Traditional Chinese
for users in Hong Kong and Taiwan.

* fix: convert remaining Simplified Chinese to Traditional Chinese (zh-HK)

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-22 13:20:37 +08:00
wyucand杨慎 e96e2fbf7a feat(dsl): standardize the asset manifest and converge the export paths (#1007 part 3) (#1117) 
* feat(dsl): standardize the document asset manifest

Add asset-manifest.ts to @openmaic/dsl: the canonical AssetManifestEntry
shape (ref, kind, and byteSize/mimeType/duration/voice/prompt metadata
where available) plus enumerateAssetManifest, the pure document-to-manifest
enumeration. An entry's ref is the reference exactly as the document holds
it -- the manifest is the id-based reference enumeration with metadata,
not a content hash and not a resolution result. The traversal walks the
stage whiteboard, each scene's canvas/whiteboards/speech actions, and the
stage video-manifest keys in document order, with logical-owner reference
counts that match the accounting duplication-safe replacement uses.

This settles the media-ref + asset-manifest schema question (#779 open
question 4) on the side the asset-pool RFC already implied: the schema is
a function of the id semantics decided there. The type lives in the dsl
rather than a new @openmaic/exporter package because the enumeration is
pure over document types the dsl already owns (Stage/Scene/Slide/Action),
so a separate package would add a published artifact and release-workflow
surface without adding a capability; the storage contract comment now
points at the module.

Refs #1007

* refactor(export): drive the classroom ZIP from the asset manifest

collectMediaFiles used to scan the whole mediaFiles table for the stage,
so any row the document no longer references -- an orphan left by an edit
or a superseded regeneration -- rode along into the archive. Both ZIP
collectors now take their reference sets from the standardized asset
manifest (buildStageAssetManifest wraps the dsl enumeration with the
compatibility rows' metadata): only referenced assets are archived, and a
referenced asset whose bytes exist only in the pool is still collected via
a synthesized record. Byte resolution is unchanged: pool first through
resolveStoredBytes / resolveAudioBlob, with the compatibility row kept as
the legacy byte fallback and as the metadata source.

mediaIndex is now a serialized view of the manifest, and the missing-audio
report derives from the manifest's audio entries instead of a second
action walk. The audioRef mapping and the legacy audioUrl fetch path
(collectLegacyAudioForExport) are untouched.

Refs #1007

* refactor(video-export): take the timeline's reference sets from the manifest

createVideoTimelineDeps scanned the whole mediaFiles table for the stage
and derived its audio id set from its own action walk -- a third,
independent answer to "which media does this course use?". Both record
loads now key off the standardized asset manifest: media rows are read per
manifest ref by compound key instead of by table scan, and the audio id
set is the manifest's audio entries. Orphan rows were never reachable
through the scene-scoped elementId-to-mediaRef bridge; now they are not
even read.

The bridge itself is untouched: element ids recur across scenes, so the
elementId-to-mediaRef mapping stays scoped per scene, and the legacy
audioUrl fallback keeps its own action walk because a URL is not a
manifest ref. AssetPlan remains the video IR's view of the same
references.

Refs #1007

* refactor(export): resolve PPTX media through the shared resolver only

Each PPTX element branch carried its own resolution chain: a task-state
renderable-URL lookup first, then -- gated on the legacy placeholder
predicate -- a stored-bytes override, with the poster block repeating the
pattern. One helper now owns resolution for backgrounds, images, video /
audio sources, and posters: opaque refs (allocated ids and legacy
placeholders alike, no placeholder-pattern gate) resolve pool-first
through resolveStoredBytes and embed as data URLs, concrete addresses
resolve through the media state machine and keep the caller's fetch path.
exportMediaResolution and the resolveStoredMediaBlob wrapper fold into
the helper; resolvePptxMediaBinding stays as the state-machine entry the
resolution-surface test matrix drives.

Refs #1007

* refactor(export): retire the export-side Dexie byte fallbacks

Export call sites no longer read bytes off compatibility rows directly.
The ZIP collectors and the video timeline's audio load resolve bytes only
through the shared resolvers (resolveStoredBytes / resolveAudioBlob),
which answer pool-first and keep the compatibility row as their internal
legacy fallback level; the row reads that remain at the call sites supply
metadata (format/duration/voice/mime/size/prompt) only. The rows
themselves stay for legacy and regeneration readers -- what goes is the
export paths' own fallback logic.

One observable tightening: a failed media row (error set, empty
placeholder blob) no longer ships a 0-byte file into the classroom ZIP,
and an evicted row no longer ships its empty local blob; referenced-but-
byteless assets are simply absent from the archive, as they already were
when no row existed.

Refs #1007

* test(media): cover the enriched stage asset manifest builder

Pins the join between the pure dsl enumeration and the compatibility
rows: metadata attaches by ref, rows no document reference names never
appear, and a referenced asset with no row keeps a metadata-free entry.

Refs #1007

* fix(video-export): widen the deps stage input for the manifest enumeration

enumerateAssetManifest reads the stage's whiteboard and videoManifest, so
createVideoTimelineDeps declares them on its input instead of the bare id;
callers pass only the id today and the optional fields stay absent. Also
applies the repo prettier formatting to the files this branch touched.

Refs #1007

* fix(dsl): enumerate slide audio elements in the asset manifest

Slide audio elements carry their own src, and the manifest skipped
them, so a manifest-driven collector could never archive their bytes.
The audio slot maps to kind 'audio' alongside narration ids.

Refs #1007

* fix(media): harden ref-keyed lookups against prototype-named asset refs

AssetRef is an unconstrained string alias, so a media reference can legitimately be "__proto__", "constructor", or any other Object.prototype member. Plain objects keyed by such refs silently drop assignments or answer lookups with the prototype object, which rewrite paths then accept as a mapped id.

Convert the remaining ref-keyed lookup tables introduced by the export convergence to prototype-safe structures: the classroom import media/poster alias maps and the legacy-conversion video-manifest reconstruction now use Map / null-prototype containers with explicit membership checks, and every consumed value is validated as a string before it is written into a src / mediaRef / audioId slot. The shared media-task lookup receives the same treatment: one centralized own-property-checked lookupMediaTask now serves the stored-bytes resolver, the PPTX embeddable-src path, the video collection path, and the element/background task resolution, so a prototype-named placeholderRef can no longer hide a re-keyed task from the fallback chain.

Adversarial tests drive "__proto__" and "constructor" refs through the import round trip, the PPTX fallback path, the legacy conversion commit path, and the media-task fallback end to end, including a buildPptxBlob regression with a task re-keyed to an allocated id while retaining a prototype-named placeholderRef.

* fix(export): use safe archive asset paths

* refactor(dsl): centralize slide media slot roles

* refactor(export): derive consumer refs from manifest

* fix(export): sanitize classroom archive extensions

* fix(video-export): preserve narration speech order

* fix(export): enforce kind-coherent archive media

* fix(export): define media coherence boundary

* fix(export): carry task-owned poster binding for PPTX export

A video element with no explicit poster falls back to its media task's
generated poster URL, but resolveVideoMediaForElement left posterTask
undefined for that case, so the PPTX manifest guard saw a foreign URL
with no task-ownership exemption and dropped the video element instead
of using the established runtime poster fallback.

Carry the poster task binding whenever the task poster is the effective
poster: the task-owned URL then satisfies the guard's objectUrl
exemption end to end. A concrete explicit element poster still stays
element-owned and never borrows the binding, and the guard's foreign-ref
rejection is preserved (and exported as a directly testable predicate).

Coverage: an element with no poster plus a task-provided poster embeds
the task poster as the PPTX cover (red at the pre-fix head, green now),
and a genuinely unrelated URL with no task ownership is still rejected
by the guard.

* fix(export): preserve legacy narration source refs in the media index

The explicit sourceRef contract was partial: primary audio and generated
media entries carried it, but legacy URL narration serialized no source
ref. The legacy URL itself is the natural source ref — it is known at
fetch time — so wire it through the collected blob into the mediaIndex
entry. Import already registers serialized sourceRefs as aliases, so the
URL now round-trips as an explicit mapping instead of being reconstructed
only from the action's audioRef.

Poster siblings are deliberately NOT given their own mediaIndex entry:
a sibling poster (media/asset-<n>.poster.<ext>) is a legacy byte copy
written from the video record and is not an independently referenced
document asset — when the poster is a real document asset it already has
its own indexed entry with a sourceRef, and import reconstructs the
sibling by path derivation from its parent video entry, reusing the
poster's own indexed allocation when one exists. The PR description is
narrowed to match; corrected paragraph:

"Archive names never interpolate refs — sequential safe paths
(media/asset-<n>.<ext>, audio/audio-<n>.<ext>) with the original ref
preserved through an explicit sourceRef mapping on every independently
indexed media entry: generated media assets, poster assets, primary
narration, and legacy URL narration (the legacy URL itself is the
entry's sourceRef). Extensions are allowlisted per kind. The one
exception is the legacy sibling poster byte copy
(media/asset-<n>.poster.<ext>, written next to its video when the video
record still carries the pre-pool poster bytes): it is not an
independently referenced document asset, so it has no mediaIndex entry
or sourceRef of its own — its identity is derivable from its parent
video entry (same index), and import reconstructs it by sibling-path
derivation from that video entry, reusing the poster's own indexed
allocation when one exists."

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-08-19 20:43:14 +08:00
xuyuanwei678 4eb435dc40 fix(editor): align editable shape labels with text (#1137) 
* fix(editor): align editable shape labels with text

* fix(editor): preserve plain-text line breaks on edit

* fix(editor): preserve entities in plain text editing

* style(editor): format shape editing changes

* test(editor): update manifest version expectation
 2026-08-19 10:22:11 +08:00
xuyuanwei678 221df3595d fix(renderer): preserve literal text line breaks (#1132) 
* fix(renderer): preserve literal text line breaks

* fix(renderer): preserve empty text paragraphs

* fix(renderer): scope line-break preservation to plain text

* fix(renderer): detect non-tag HTML markup

* style(renderer): format rich text tests
 2026-08-17 21:45:46 +08:00
wyuc e15780835a feat(media): convert legacy references to allocated asset ids (#1101) 
Lands epic #1007 part 2(c): the app-side reference converter, the `audioUrl`
removal, and the `DSL_VERSION` bump as one delivery unit.

`convertDocumentAssetRefs` walks a loaded document (slide canvases, whiteboards,
video-manifest keys, speech actions) and rewrites legacy handles to allocated
pool ids: placeholders and `audioId`s with local bytes are ingested, a co-present
`audioUrl`/`audioId` pair collapses to one asset, a definitively dead URL
converts to an emptied reference, and a transient failure keeps both handles for
a later retry. Unavailable bytes leave the document untouched for a later open,
and a converted document returns by identity with no writes.

Conversion runs lazily on document open and at fetch time for server-generated
classrooms, with conversion and the first document save sharing the per-stage
document lock: once the document commits it owns every allocation, and every
failure exit rolls its pass ledger back. Legacy URL probes are bounded by a
per-probe timeout and a shared 60-second pass budget.

`SpeechAction.audioUrl` is removed from the DSL and every consumer reads the
converted shape; the server classroom generator keeps its derived-id + serving-URL
pair as an explicitly typed pre-conversion transport, consumed before persistence.
`DSL_VERSION` becomes 0.2.0 with the first real ladder entry, and
`@openmaic/dsl` goes to 0.9.0 per the release rule.

Deferred follow-ups are tracked in #1129: production principal derivation with
document/asset ownership aligned, conditional document writes, and hoisting the
aggregate probe budget into the export and video-timeline paths.

Refs #1007
 2026-08-15 17:38:04 +08:00
wyucand杨慎 0a10af11cf feat(storage): opt-in indirect asset byte egress (#1007) (#1100) 
* feat(storage): add opt-in indirect asset byte egress

A deployment can now opt an asset byte GET into a 302 to a short-lived
signed URL when the byte layer can sign. It is off by default, and
byte-for-byte unchanged when off. The byte layer gains an optional
signReadUrl capability; S3AssetByteStore implements it through the new
optional @aws-sdk/s3-request-presigner peer, resolved lazily exactly
like the client SDK, with a signer seam mirroring the commands seam so
tests can bind doubles. The signed URL pins the contract's response
headers -- the media type after the renderable allowlist, the fixed
disposition, and the no-store cache posture -- via S3 response-header
overrides. PgAssetStore mints the URL from the same ownership-checked
transactional read a direct resolve performs, so authorization still
runs per read before any URL exists, and a store whose byte layer
cannot sign falls back to direct bytes. The contract documents the
shape and the disclosure tradeoff: a Location into a hash-keyed byte
layer names the content hash, so deployments that need the
no-disclosure property keep direct egress.

Refs #1007

* feat(app): wire ASSET_BYTE_EGRESS into the persistence route

ASSET_BYTE_EGRESS=redirect opts asset byte GETs into the storage
package's indirect egress; unset, direct, or an unrecognized value
keeps the default byte-for-byte behavior, with a warning for the
unrecognized case. The lazy byte-store wrapper now forwards
signReadUrl and answers undefined when the resolved layer has no
signer, so the PostgreSQL byte column degrades to direct bytes and
the S3 layer signs without the wrapper knowing which it holds. The
collector path is untouched: it builds its byte store through
createAssetByteStore as before and only ever calls delete.

Refs #1007

* fix(storage): let the packaged client survive redirect egress

Cross-review round 1 on the indirect byte egress change found two
problems.

A byte response that followed a redirect carries the pinned content
type but no X-Asset-Revision, and HttpAssetStore required both from
the same response, so every cold resolve under redirect egress failed
MALFORMED_RESPONSE. The client now detects the shape once -- a
redirected byte response with no revision -- latches it, and probes
HEAD before each cold GET from then on, taking the label from the
probe. The probe predates the download on purpose: a replacement in
between labels newer bytes with an older revision, which the next
revalidation detects and corrects, while the reverse order could pin
stale bytes under a fresh revision with no signal to repair it. The
probe doubles as the miss check, so a redirect-mode miss costs no
download. The direct path is byte-for-byte unchanged; the one double
download per client lifetime on first contact is documented in the
contract, which gains the client-side paragraph this behavior
implements.

signedUrlTtlSeconds also accepted lifetimes beyond the seven-day
SigV4 presigning maximum, minting redirects the object store would
reject. The option is now capped at construction.

Refs #1007

* fix(storage): cap signed URL lifetime below the reclamation grace

A signed URL whose lifetime exceeds the collector grace period can
outlive its object: the last reference goes, the grace elapses, the
collector deletes the object, and the still-valid URL errors at the
object store. The previous ceiling -- the seven-day SigV4 maximum --
made that window days wide against the one-hour default grace. The
cap is now fifteen minutes, and the contract text states both the
ceiling and the rule for deployments that shorten their grace.

Refs #1007

* fix(storage): decline signing when the optional presigner is absent

An unresolvable @aws-sdk/s3-request-presigner made signReadUrl throw,
which the registry surfaced as a failed asset read -- a 500 for a
deployment whose only fault is a missing optional peer. The contract's
answer for a byte layer that cannot sign is the capability fallback:
return undefined and let the caller serve the bytes directly. Signing
errors from a resolved signer still fail loud; only the missing
capability declines.

Refs #1007

* fix(storage): answer redirect egress as a descriptor to asking clients

Following a 302 is not header-neutral: the platform fetch forwards the
original request's headers to the object store's origin, stripping only
Authorization, so a deployment whose credential travels in a custom
header would hand it to the object store -- and the preflight the
forwarded custom headers provoke commonly fails there besides. This
replaces the latch-and-probe client shape with an explicit one: the
client sends X-Asset-Egress: descriptor on every byte GET, a
redirect-egress server answers 200 with a JSON { url, revision } body
instead of a Location, and the client fetches the signed URL with no
deployment headers at all. The revision comes from the descriptor, so
the probe-first ordering and its one wasted download are gone too. The
302 remains the answer for consumers that did not ask; the descriptor
response is marked by its own header so a JSON-media asset can never
parse as one.

Refs #1007

* fix(storage): negotiate the descriptor through Accept, not a custom header

A custom X-Asset-Egress request header is not CORS-safelisted, so every
byte GET to a cross-origin server became a preflighted request -- a
regression for direct-egress deployments that never opted into
anything, against servers with no reason to allow the header. The
negotiation now rides Accept with a vendor media type, which is
safelisted and costs no preflight, and the descriptor answer is
identified by that media type as its Content-Type rather than by a
marker header.

Refs #1007

* fix(storage): sign outside the registry transaction and reserve the descriptor type

Two review findings on the indirect egress path.

resolveIndirect awaited the signer inside the coordinated read, but a
signer on refreshable credentials can wait on the network, and a
database connection plus the blob row's FOR SHARE lock would be held
across that delay. The read now closes first -- the lock still puts
the hash, label and revision in one snapshot -- and signing runs
afterwards, which cannot observe anything the read did not.

The descriptor media type is also reserved from renderableTypes: the
client identifies a descriptor answer by that exact Content-Type, and
an allowlisted asset served with it would be misread as a descriptor
and fail MALFORMED_RESPONSE. The handler now rejects the configuration
at construction, and the contract states the reservation.

Refs #1007

* fix(storage): ship the signing deps and negotiate without ambiguity

Three review findings on the indirect egress path.

The app now declares @aws-sdk/client-s3 and
@aws-sdk/s3-request-presigner as runtime dependencies, and the
standalone build force-includes them for the persistence route: both
are reached through deliberately untraced dynamic imports, so the
shipped image could not resolve them, and a deployment opting into S3
or redirect egress would have found the capability silently absent.

The descriptor negotiation matches exactly on both sides: the client
compares the Content-Type essence, so a longer media type that merely
begins with the reserved value still serves as bytes, and the server
parses Accept media ranges, so a descriptor range sent with q=0
selects the redirect instead of the descriptor it explicitly rejects.

Refs #1007

* fix(app): refuse redirect egress when the collection grace is shorter

A signed URL must never outlive its object: with the handler's
60-second default lifetime, a deployment that sets
ASSET_COLLECTION_GRACE_MS below ten minutes lets the collector delete
an object while a URL minted against it is still valid. The route now
refuses the combination at handler initialization, naming both
variables -- a deterministic misconfiguration, loud at startup rather
than silent at read time.

Refs #1007

* fix(app): ship the signing SDKs without loading them

Two findings on the standalone deployment of redirect egress.

The outputFileTracingIncludes globs did not follow pnpm's scoped
symlinks, so the standalone image carried neither AWS package and both
S3 mode and redirect egress could not resolve their SDK at runtime.
The packages are now server-external and referenced from literal --
but never called -- dynamic import thunks in the persistence byte-store
wiring, which gets them traced into the image while module resolution
still happens only on first S3 use. Verified against a local
standalone build: node_modules/@aws-sdk now contains both packages,
and the route tests that pin the never-resolve-unless-configured
discipline still pass.

Media type comparisons are also case-insensitive now, as HTTP
requires, on both the Accept parse and the descriptor recognition.

Refs #1007

* fix(app): treat an empty collection grace as unset

Number('') is 0, so a present-but-empty ASSET_COLLECTION_GRACE_MS
failed the redirect-egress coordination check and took persistence
down with it, while the collector's own parsing reads the same value
as the one-hour default. The check now trims and treats empty as
unset, matching durationEnv.

Refs #1007

* fix(storage): accept bytes too, omit signing on the known-PG wrapper, state the full tradeoff

Three review findings.

The client's descriptor request now advertises both representations --
the vendor descriptor type preferred, */* accepted -- so a strict
content-negotiating layer can never answer 406 to a client that does
in fact consume plain bytes.

The lazy byte-store wrapper no longer advertises signReadUrl when no
bucket is configured: the layer is known to be PostgreSQL at
construction, and carrying the method made resolveIndirect take its
ownership query and blob-row lock just to decline, then repeat them
in resolve, on every cold GET. With a bucket configured the lazy
validation is preserved and the wrapper still declines when the
resolved layer cannot sign.

And the contract's disclosure tradeoff now states the second edge:
object-store GETs carry ETag and Last-Modified, response overrides
cannot strip them, and because deduplicated PUTs rewrite the
hash-keyed object, Last-Modified moves when another principal
re-uploads the same bytes -- shared-object write timing, beyond byte
equality. Deployments for whom that signal is sensitive must keep
direct egress.

Refs #1007

* fix(app): degrade on bad grace, share the ttl invariant, cover the real boundary

The latest review round, four findings.

A misconfigured ASSET_COLLECTION_GRACE_MS now degrades redirect
egress to direct with a warning instead of failing the shared
handler's initialization -- the asset backend is optional, and its
misconfiguration must never take document and runtime traffic down.

The package exports assertSignedUrlTtlWithinGrace so a deployment
wiring the handler and the collector separately validates the
invariant once at its own boundary: the handler caps the lifetime,
but only the deployment knows both numbers.

The contract states that nosniff does not survive the redirect --
response-header overrides cannot set it -- and why the client-minted
blob makes that safe for conforming clients.

And the route gains a real-boundary test: the Fetch<->Node adapter,
the real createStorageHttpHandler, and the egress wiring, exercised
with an actual byte GET through the composed stack instead of mocks.

Refs #1007

* fix(storage): make the unsafe egress configuration unrepresentable

Three findings on the indirect egress path, all resolved by moving a
rule into a place where it cannot be bypassed rather than by adding a
check a consumer has to remember.

The grace/TTL invariant now lives in the option shape. byteEgress takes
'direct' or { mode: 'redirect', collectionGraceMs, signedUrlTtlSeconds? },
so enabling indirect egress without declaring the reclamation window it
lives inside no longer type-checks, and the handler validates both
numbers at construction. The flat signedUrlTtlSeconds option, its
cross-field "requires byteEgress redirect" check, and the standalone
900-second cap as the only guard all go away; the ceiling stays, but
alongside the ratio rather than in place of it. This is what the earlier
exported assertSignedUrlTtlWithinGrace could not do -- it protected only
consumers who called it -- and it costs nothing in compatibility because
the option is new in this branch. The app resolves the grace through one
shared parser with the collector, so the number the handler checks is the
number the collector runs with.

A signed URL whose object is gone is now a miss. The client maps a 404
from the byte fetch to ASSET_NOT_FOUND, matching the direct path: the
entry was owned and readable when the URL was minted, so a reclaimed
object is the same physical state the direct read reports as a miss, and
what a read means must not depend on the deployment's egress setting.
Signing still does not probe for the object -- that would restore the
round trip this feature removes and make the mint's price vary with prior
presence -- so the contract instead requires the byte layer to answer 404
for an absent object, and states that S3 needs s3:ListBucket to do so.

Route coverage now spans the whole egress matrix through the real
composed handler rather than only the direct case: 302 for a consumer
that did not ask, descriptor for one that did, direct bytes when the
byte layer declines to sign, and direct bytes when a short grace degraded
the configured mode.

Refs #1007

* feat(storage): forward byte egress options through reference server

* fix(storage): fail closed on redirects and unconfirmed 404s in indirect byte egress

A signed-fetch 404 now maps to a miss only when the object store's XML
body declares NoSuchKey; every other 404 from the signed fetch fails loud,
so a wrong bucket, access point, or endpoint can no longer make a live
asset read as absent. The descriptor byte GET is sent with redirect:
'manual' and any 3xx answer is treated as an error, so a server that
ignores the descriptor negotiation can never forward the deployment's
custom credential headers to a redirect target.

The ASSET_BYTE_EGRESS switch is now documented in .env.example and the
server-persistence deployment section, together with the object-store CORS
and s3:ListBucket prerequisites it requires, and the asset descriptor
media type is exported from the package root and the asset/http subpath.

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-08-14 23:57:21 +08:00
Yizuki_AmeandCursor 13d94a52a1 feat(ai): add Grok 4.6 to the model registry (#1113) 
* feat(ai): add Grok 4.6 to the model registry

The Grok picker still topped out at 4.5, so the new flagship ID was not selectable with its documented xhigh reasoning effort.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(docs): stop the docs Next app from loading app instrumentation

Next 16 discovers the repo-root instrumentation.ts from packages/docs, then fails because that file imports app persistence modules. Give the standalone docs app a no-op register() so the Grok catalog docs can build.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-13 16:59:06 +08:00
wyucClaude Opus 5杨慎
 		969574bee4 feat(storage): wire the asset backend into the app (#1007) (#1089) 
* fix(persistence): stop corrupting binary response bodies in the route adapter

The embedded persistence route bridges a Node `RequestListener` to the
Fetch API by hand. Its response object is cast with `as unknown as
ServerResponse`, so the compiler checks none of that surface, and two
parts of it were wrong in ways only a byte-carrying handler would hit.

`end` decoded any `Uint8Array` chunk with `Buffer.from(chunk).toString()`,
which is UTF-8. `ServerResponse.end` accepts a `Uint8Array`, and those
bytes are not necessarily valid UTF-8, so every unpaired byte became
U+FFFD -- silent corruption with no error anywhere. The adapter now
buffers chunks as bytes and builds the response from them.

`write` was absent entirely, making any chunked handler a runtime
TypeError rather than a compile error. It is part of the surface this
object claims to implement, so it is now implemented.

Neither reaches a caller today: the document and runtime handlers only
ever end with JSON strings, which took the string branch. Both are traps
laid for the first handler that carries bytes.

Both cases are pinned by tests that fail before this change -- the
binary one on a byte comparison, the chunked one with a 500 from the
TypeError. The existing adapter round-trip test, whose comment already
called this the most bug-prone code in the route, covered only the
string path.

* feat(storage): wire the asset backend into the app (#1007)

The server asset registry shipped with #1007 but was reachable from
nothing: the persistence route never passed an `assetStore`, and no file
under `app/`, `lib/` or `components/` imported the asset server or client
modules. This connects it, additively -- no DSL change, no version bump,
and no behavioural change when server persistence is off.

The route now builds a `PgAssetStore` on the same pool and transaction as
the document store, with `PgAssetByteStore` by default and
`S3AssetByteStore` when `ASSET_S3_BUCKET` opts a deployment into it. The
AWS SDK stays an optional peer, reached only through a dynamic import on
that branch. The offline collector is deliberately not scheduled here,
and the route says so where a reader would look for it, because leaving
it unscheduled grows storage without bound and scheduling it is the
deployment's decision.

On the browser side the asset pool gains a single-shot configuration seam
modelled on the document store's, so that with server persistence enabled
the pool is an `HttpAssetStore` carrying the same auth headers as the
other stores.

Three browser-specific mechanisms needed decisions rather than
translation:

`clearAssetPool` deletes an IndexedDB database today. In server mode
there is nothing local to delete, and a "clear cache" action must not
remove server-side assets -- that would destroy user data from a button
promising to free space. It now revokes local object URLs and closes the
client, and deletes nothing remote.

The cross-tab replacement broadcast is unchanged. Its job is to make
another tab drop a warm object URL when the bytes behind an id change,
and that need is identical in server mode.

The exclusivity proof that decides whether an in-place `replace` is safe
reasons over this browser only: local documents, unflushed state, and a
cross-tab presence probe. In server mode those cover a strict subset of
the holders, since another device can reference the same id and no probe
can see it -- and asking the server who else references an id would be an
existence oracle over other principals. The proof therefore returns not
exclusive unconditionally in server mode, and regeneration takes the
existing fork path. That is the fail-closed direction and a graceful
degradation rather than a break.

The development authenticator had to grow the asset principal's required
`key`; without it every asset request would have been correctly denied.
It derives from the same learner partition, and a principal without one
still gets no asset access. The route now states plainly that this key
comes from a client-supplied header, so the cross-principal isolation the
asset contract describes is not in force under this authenticator -- that
warning previously lived only in the auth module's own header, where
nobody mounting the route would see it.

* fix(storage): close three defects in the asset app wiring (#1007)

**The S3 branch could not load, and isolated nothing.** The route
dynamically imported the storage package's S3 byte store, but that module
statically imported the AWS SDK, so merely resolving it pulled the SDK
into resolution and bundling. The route then separately performed an
ignored native import of the SDK from the app's own scope, which fails
with ERR_MODULE_NOT_FOUND because the SDK lives under the storage
workspace package -- so a deployment that set the bucket got a broken
store rather than an S3 one. There is now one lazy loader in one
resolution scope, the storage module no longer imports the SDK at module
load, and the bucket is validated at initialization rather than treated
as valid because it is non-empty. A test mocks the SDK to throw on
resolution and asserts that importing the byte-store module never
resolves it.

**A cross-tab replacement could leave a mounted lease stale.** The
broadcast handler re-resolves, and `HttpAssetStore.resolve` coalesces
concurrent calls for one id onto a single in-flight request -- correct on
its own, and required by the contract. Together they meant a resolve
triggered by the broadcast could merge into a GET that started before the
peer's replacement, return the old revision, and leave the consumer on
the old URL with nothing to dislodge it. The client gains an invalidation
that retires its cached snapshot and advances the per-id generation
without revoking any URL already issued; `release` would have been wrong
here, since the contract keeps an issued URL valid until its holder
releases it. Removing the invalidation call fails the new race test.

**Clearing a server-backed pool left the closed client installed.** In
server mode `clearAssetPool` closed the client and returned before
clearing the module singleton, so every later `getAssetPool()` handed
back a closed store until a page reload happened to intervene. Only the
IndexedDB deletion is skipped in server mode now; the singleton is always
cleared so the next call rebuilds through the configured factory.

Also adds the tests whose absence let these through: a route round-trip
carrying invalid UTF-8, the SDK-resolution assertion above, the broadcast
race, and reopening the pool after a clear. The earlier tests were not
wrong so much as aimed past the defect -- the route test used JSON text,
and the S3 test injected a loader rather than exercising the imports.

* chore(storage): bump package version to 0.3.0

* fix(persistence): make the route adapter faithful where it carries bytes

The embedded persistence route bridges a Node `RequestListener` to the
Fetch API by hand, and its response object is cast
`as unknown as ServerResponse`, so the compiler checks none of that
surface. Five divergences are closed here; the first two are the ones
that corrupt data, the last three were raised in review.

**`end` decoded bytes as text.** It ran `Buffer.from(chunk).toString()`
on a `Uint8Array` chunk, which is UTF-8, so every byte outside that
encoding became U+FFFD -- silent corruption with no error anywhere. The
adapter now buffers chunks as bytes.

**`write` was absent**, making any chunked handler a runtime `TypeError`
rather than a compile error.

**`write` callbacks ran inline.** Node invokes them after the chunk is
handed off. A handler that writes its next chunk from each callback
therefore recursed synchronously and could overflow the stack, and a
throwing callback threw in the wrong execution phase. They are now
deferred.

**The encoding argument was ignored.** `write(s, 'latin1')` emitted
`0xc3 0xa9` where Node emits `0xe9`, corrupting the body and potentially
invalidating a `Content-Length` the handler had set. Both `write` and
`end` now parse their full overload set and pass the encoding through.
Node's own behaviour for an invalid encoding was established by probing
`ServerResponse` on Node 22 rather than assumed: it throws
`ERR_UNKNOWN_ENCODING` synchronously, so the adapter validates with
`Buffer.isEncoding` and lets `Buffer.from` raise the native error.

**Bodyless statuses were not suppressed.** Node discards writes after
`writeHead(204)` or `writeHead(304)`; this passed the buffered bytes to
the Fetch `Response` constructor, which throws for those statuses, so the
request became a 500 instead of the intended 204 or 304. Buffered data is
now discarded for 204, 205 and 304, and for `HEAD`.

None of the five reaches a caller today -- the document and runtime
handlers only ever end with JSON strings. They are traps laid for the
first handler that carries bytes, which is why they surfaced while
wiring the asset server backend, whose byte routes hit the first two
immediately.

Each is pinned by a test that fails when its fix is reverted.

* fix(storage): keep { client, bucket } working for the S3 byte store

`S3AssetByteStoreOptions.commands` was required and the store's methods
called it directly, so the published `./asset/s3-bytes` export stopped
accepting the `{ client, bucket }` construction it had always taken.
Anyone not going through `loadS3AssetByteStore` either failed to compile
or hit an undefined command factory at runtime.

Make `commands` optional. Omitted, the store resolves the AWS SDK's own
command constructors through a dynamic import on its first `write` /
`read` / `delete` and caches them, so `{ client, bucket }` works again.
Importing the module and constructing a store still never reach the
optional peer dependency: a deployment that does not select S3 never
resolves it. When the SDK cannot be resolved the call rejects naming
`@aws-sdk/client-s3` rather than crashing on an undefined property, and
the resolution is left uncached so installing the dependency fixes an
already-constructed store.

The lazily bound store is run against the byte-store contract, and the
new isolation tests pin both that construction does not resolve the SDK
and that an unresolvable SDK is reported by name.

With the break gone this branch's change to the package is additive, so
the version becomes a patch above main (0.2.4 -> 0.2.5) rather than the
minor it was carrying.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(persistence): run the asset collector in the shipped deployment

`PgAssetStore.remove`, and a `replace` that changes content, only stamp
`asset_blobs.unreferenced_at`. `AssetCollector.collect` is the only path
that deletes anything, and the persistence route deliberately did not
schedule it, on the grounds that scheduling is the deployment's decision.

The deployment this repository ships is docker-compose.yml: the app and
PostgreSQL, and nothing else that could make that decision. "The
deployment decides" therefore meant "it never runs", and ordinary asset
churn retained PostgreSQL bytes or S3 objects forever.

Schedule it from instrumentation.ts, which Next runs once per server
process — a route module can be instantiated more than once and has no
shutdown hook, so a schedule started from one is really started per
instantiation. Next 15 and later pick the file up with no config, so
next.config.ts is unchanged.

Collection is on by default with a 15-minute interval and the package's
one-hour grace period, so the Compose stack is correct with no operator
action. ASSET_COLLECTION_INTERVAL_MS, ASSET_COLLECTION_GRACE_MS, and
ASSET_COLLECTION_ENABLED change or disable it, each documented where it
is read. Without DATABASE_URL nothing is scheduled at all, and a failed
pass is logged and retried on the next tick rather than ending the
schedule or the process.

Several instances may collect concurrently: each candidate blob row is
re-checked and locked FOR UPDATE inside its own transaction, so
collectors serialize on the row rather than racing. The comment says so,
so nobody adds a distributed lock that is not needed.

The byte-store selection moves to lib/persistence/asset-byte-store so the
collector reclaims through the same layer the request path wrote through.
A collector on the PostgreSQL byte store while the route wrote to S3
would drop the blob row and orphan the object permanently.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(storage): bound each collector pass and report whether it filled

An unbounded collect() is sized by however long a deployment ran before
collection was scheduled: one statement selecting every eligible blob,
then a transaction and a byte-layer delete each, in a loop nothing
interrupts. The first pass over the Compose deployment that grew a
backlog before collection was scheduled is exactly that pass.

A pass now takes at most batchSize blobs (default one thousand) and
returns. Ordinary churn between two scheduled passes is far below the
cap, so a healthy deployment behaves as it did when a pass was
unbounded; a capped pass costs the remainder one scheduling interval,
which is what the interval is for.

collect() still answers with the count, and the count alone cannot tell
an empty backlog from a full batch -- a re-referenced or concurrently
taken candidate is skipped, so even a full batch can collect less than
batchSize. collectPass() returns the count together with capped, which
is exactly "this batch was full, run again"; a caller draining a
backlog loops while it is true.

Candidates are taken oldest-unreferenced first, with the content hash
as tiebreaker within one timestamp. Ordering by the hash alone would
starve: digests are uniformly distributed, so a high-sorting blob
waits behind every lower digest stamped after it, and under steady
arrivals those keep coming. The tests pin the ordering against a
planner that happens to answer the right rows without an ORDER BY.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(persistence): keep asset backend faults off the shared handler

Two review findings on the app wiring.

A concrete asset pool instance is single-lifecycle, but
resolveConfiguredAssetPoolStore handed the same object out again after
clearAssetPool() had closed it, reinstalling a dead store as the live
pool. The second handout now refuses loudly and points at the factory
form, which rebuilds on every resolution.

createPersistenceHandler eagerly awaited the asset byte store, so an
invalid ASSET_S3_BUCKET or an unresolvable AWS SDK rejected the shared
persistence handler and took document and runtime traffic down with an
optional backend. Byte-store construction is now deferred to the first
asset byte operation, and a failed construction is not cached, so the
next asset request retries -- the route's own no-poisoned-singleton
rule.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-11 13:01:16 +08:00
xuyuanwei678 3f210095d5 fix(editor): add npm provenance metadata (#1098) 2026-08-11 11:51:34 +08:00
xuyuanwei678 c38da84ef6 feat(renderer): decouple editor UI from the app (#1072) 
* docs: define renderer editor task 1 integration

* docs: plan renderer editor task 1

* feat: gate renderer slide editor

* feat: adapt renderer editor intents

* feat: connect renderer editor to slide surface

* test: complete renderer intent shape fixture

* fix: restore renderer editor action previews

* fix: render editor alignment guides

* perf: smooth renderer element dragging

* feat: align renderer hidden element behavior

* feat: add renderer canvas host commands

* feat: add renderer canvas context interactions

* test: cover renderer context menu interaction

* fix: preserve renderer canvas interaction semantics

* docs: design renderer text editing

* docs: plan renderer text editing

* feat(renderer): add ProseMirror text core

* feat(renderer): add rich text editor controller

* feat(renderer): render editable text in place

* feat(renderer): enter text editing on click

* feat: connect renderer rich text editing

* feat: finalize renderer text edit lifecycle

* fix: stabilize renderer text edit commits

* fix(renderer): move elements from selected borders

* docs: design renderer editing UI text toolbar

* feat(renderer): publish editing UI contracts

* feat(renderer): add controlled text format toolbar

* style(renderer): format editing UI contracts

* fix(renderer): address Task 2 review findings

* fix(renderer): handle zero font sizes in toolbar

* feat(renderer): add editing UI color picker

* fix(renderer): address Task 3 color picker review

* fix(renderer): commit native color input changes

* fix(renderer): coordinate native color picker interaction

* feat(renderer): anchor text toolbar to canvas elements

* feat(renderer): compose editable canvas with text UI

* feat(editor): use renderer text toolbar UI

* fix(renderer): make editing UI tests type-safe

* fix(renderer): harden editing UI integration

* feat(renderer): add editable text UI

* feat(renderer): add image and shape editing

* feat(editor): extend renderer editing controls

* feat(editor): add renderer chart insertion

* docs: design renderer latex editor

* docs: plan renderer latex editor

* feat(renderer): add latex editor contract

* feat(renderer): add latex editor dialog

* feat(renderer): compose latex insert and edit UI

* feat(editor): integrate renderer latex editing

* fix(editor): consolidate latex toolbar actions

* fix(renderer): show toolbar hover labels

* test(renderer): cover localized latex tooltips

* chore(renderer): hide latex sample palette

* feat(renderer): add video editing controls

* docs: define renderer audio editing

* feat(renderer): render audio elements

* feat(renderer): add audio editing UI

* feat(renderer): add preview audio controls

* feat(editor): add renderer element clipboard

* feat(renderer): decouple editor UI from app

* chore(renderer): release 0.0.6

* fix(renderer): clear editor CI gates

* fix(editor): adapt renderer canvas to media resolution API

* fix(i18n): complete French editor labels

* fix(renderer): hide toolbar vertical overflow

* chore(renderer): address review cleanup

* chore(docs): remove planning artifacts from pr

* feat(editor): add transactional core package

* chore(editor): ignore local test cache

* build(editor): publish core entrypoint

* refactor(editor): route renderer changes through transactions

* refactor(editor): split renderer editing into editor package

* ci(editor): register editor package for release

* docs(editor): document package dependencies

* ci: rerun validation after editor release registration

* test(editor): validate deletion against existing element

* fix(editor): address transaction and package review findings

* fix(editor): validate transactions and restore editor styles

* fix(editor): protect required updates and react styles

* fix(editor): validate required transaction fields

* fix(editor): prevent insert popovers shifting canvas

* refactor(editor): own renderer editing capabilities

* fix(editor): harden editing transaction boundaries

* refactor(editor): modularize editor UI integration

* docs(editor): harden third-party integration

* fix(editor): preserve video resize aspect ratio

* fix(editor): discard text draft before deletion

* fix(editor): preserve renderer picking and line presets

* chore(editor): remove unused pick-layer import
 2026-08-10 19:56:24 +08:00
wyuc 3b4567076f refactor(generation): consume the package everywhere and delete lib/generation (#1090) 
Part D of #1057 — the switch. All consumers import @openmaic/generation; lib/generation deleted (net -11,900 lines). PBL wiring preserved (single-call -> classified fallback -> injected loop), streaming outline route deduplicated onto buildOutlinePrompt, serverExternalPackages wired, and the change verified by a real-model four-kind end-to-end run against the production build with a rendered-classroom screenshot check. Closes #1057.
 2026-08-10 03:58:33 -04:00
wyuc b64e787749 feat(generation): move scene generation and PBL single-call planning into the package (#1087) 
Part C of #1057. Scene content/actions/widget generation, retry, action parsing, interactive post-processing, buildCompleteScene with injectable sceneId, and PBL single-call planning re-seated on AICallFn (pure core + kernel + package-relative planner assets; loop planner stays app-side behind the injected pblLoopFallback with provider/abort classification). Purely additive — app source untouched until Part D. Package 0.2.0.
 2026-08-10 00:58:35 -04:00
wyucand杨慎 6825363de1 feat(storage): server asset registry over a pluggable byte layer (#1007) (#1077) 
* docs(storage): specify the asset registry HTTP contract (#1007)

Adds the contract document and the server-side store, byte-layer, and
handler types for the asset layer, the only layer with no server backend.
Runtime behaviour is unchanged: the new modules are deliberately
unreferenced until the backends land.

Reclamation moves offline, and that decision carries the design. `remove`
deletes a registry entry and nothing else -- it never reads, counts, or
deletes bytes. A separate collector takes byte rows that have had no
references for a grace period. Three things follow. The hardest race in
the design disappears, because no request can delete bytes another
request is adopting, so no request path needs a lock on the byte row and
the write path never has to reconcile taking one with never branching on
whether the bytes already existed. A side channel closes, because a
`remove` that only deletes a row costs the same whether or not another
principal holds the same bytes. And nothing is given up: under global
deduplication `remove` could never promise the bytes were destroyed, so
deleting them synchronously bought far less than it appeared to. The cost
is that storage grows until the collector runs, which a deployment must
configure rather than inherit.

Offline reclamation is also what makes the byte layer pluggable, so this
keeps #1007's pluggable-backend model rather than narrowing it. With
collection out of every request path, the requirement on a byte layer is
an ordering rule rather than an atomicity one: bytes are written before
the row that references them, deleted after the last row referencing
them, and the reference count is serialized on the blob row that every
implementation keeps regardless of where bytes sit. Two implementations
are planned -- bytes in a column of the transactional store, and an
object store keyed by content hash -- which is why the interface exists;
an interface with one implementation is not a seam, and this package has
removed such a seam before. The browser backend keeps no such seam,
because it reclaims inline and its bytes therefore cannot leave its
transaction; four comments that stated this as a storage difference now
state it as a reclamation one.

Metadata crosses the wire as a multipart part. It is open-ended and
callers already fill it with generated-narration text and image prompts,
so a query parameter would put unbounded user content into the request
target and into every intermediary's logs. That also motivates a rule
the other layers do not need: metadata must never appear in a URL, a
response header, a log line, or an error message. Metadata gets a value
domain, and the document is explicit that this is a capability
reduction -- the browser backend's structured clone carries Date, Map,
Set, and cycles faithfully, so a deployment moving to a server backend
must audit its metadata rather than expect a quietly lossy path to
tighten.

Resolution yields a client-minted object URL, superseding the resolution
that a server deployment resolves to a self-hosted proxy path. A media
element sends only ambient credentials, so a proxy path works for
cookie-session deployments and silently fails for header-authenticated
ones; closing that gap means minting URL tokens, a second credential
form inside a storage library. Fetching bytes over the client's own
authenticated request needs none and removes a class of hash-disclosure
surface. The client obligations that make it preserve the browser
backend's semantics are specified, and split into the two the shared
suite already pins and the four an HTTP backend must pin itself. The
cost, no range requests and no progressive playback, is stated.

Channels that could carry a principal or a content hash are closed
structurally rather than by enumeration: the route table admits no
segment that could carry either, and no route takes a query string at
all. Headers are closed by not reading them -- the contract assigns no
meaning to any header beyond content type and length, and a handler must
not derive identity from one outside its authenticate hook. Rejecting
header names by pattern was tried and removed: no pattern broad enough
to catch X-Remote-User spares User-Agent, and the forwarded-identity
headers such a rule would reject are exactly what an authenticating
reverse proxy injects for that hook to read.

Ids that cannot be a path segment are resolved by the client as misses
rather than raised as errors, a deliberate divergence from the KV
contract. An unknown id is a miss by this layer's id-domain rule and the
shared suite pins the empty string as one, so a client that threw would
fail that suite and reintroduce the shape oracle the rule prevents.

Also specified: response values may not derive from the shared byte row
(Last-Modified off that row is an existence oracle), a collision-
resistant digest is required because writes are unconditional and an
object layer keys on that digest, byte responses carry the revision and
refuse Range, HEAD errors carry the code in a header since they have no
body, quota exhaustion has a status and a code, and relabelling a
non-renderable type is not refusal -- resolve still returns a URL.

* feat(storage): add pluggable server asset store (#1007)

* fix(storage): write asset bytes only after claiming the blob row (#1007)

The write path wrote bytes twice: once before the registry transaction
and once inside it, after the upsert that claims the blob row. Only the
second one carries any safety, and the first is what a large-media
deployment pays for on every upload.

The ordering that matters is claim, write, reference. Claiming the blob
row takes its lock, so a collector already holding that lock finishes
before the write proceeds; writing bytes before the claim instead lets
the collector delete them while the claim waits, leaving a fresh entry
that points at nothing. Writing them before the entry keeps every
surviving entry backed by bytes that were actually stored. Dropping the
first write preserves both ends of that order and removes a crash window
that produced orphans for no reason.

For a byte layer inside the transactional store this also means a failed
write now leaves nothing at all, rather than an orphan the collector had
to sweep later.

The four tests this changed were each pinned to the removed write:

- the statement-sequence test hardcoded the old four-statement shape; it
  now pins claim, write, entry, and still asserts the two existence paths
  are identical, which is the property that matters;
- the quota test asserted two writes per accepted put;
- the crash-window test asserted an orphan that a transactional byte
  layer no longer produces. It is now two tests: a transactional layer
  leaves nothing, and a non-transactional one strands an object with no
  blob row -- which the collector deliberately cannot see, since
  recovering it is deployment housekeeping rather than reference
  counting. It also no longer depends on state left behind by earlier
  tests in the file;
- the collector-race test drove its interleaving off the removed write.
  It was choreography rather than concurrency in any case: PGlite is
  single-connection, and the test serialized transactions, so no row lock
  was ever contended. It is replaced by the two invariants that actually
  make the race safe -- the byte write is unconditional, and bytes the
  collector has already removed are re-stored by an adopting put. The
  contended-lock case belongs to the real-PostgreSQL suite, where it can
  be tested rather than mimed.

Reverting the unconditional write fails two of them, so they pin it.

The contract said a failed write leaves no partially written bytes while
its byte-layer section permits exactly that orphan; the two are
reconciled, and the ordering rule now states the claim step and the
unconditional write it depends on, along with the cost that follows --
the byte write happens with the registry transaction open, so an object
store holds a row lock across a network upload.

* test(storage): fix the real-PostgreSQL asset tests against the new write order

Both were written against the removed pre-transaction byte write, and
both failed once it went away -- one of them by hanging, which also
timed out the next test's TRUNCATE.

The lock-contention test waited for the adopting put to start writing
bytes before releasing the collector. Bytes are now written after the
upsert that claims the blob row, and the collector holds that lock, so
the write could never start: a circular wait. It now waits for a backend
to actually appear in pg_stat_activity blocked on a lock, which is the
condition it meant to wait for, observed rather than signalled. Its final
assertion carries real weight now -- had the bytes been written before
the claim, the collector would have deleted them and the adopting entry
would resolve to nothing.

The orphan test asserted that a failed registry transaction strands
bytes. With this byte layer inside the registry's transaction it strands
nothing, so it now asserts that instead. Object storage does strand an
object, and that case is covered where it belongs.

All of this ran against PostgreSQL 16: 781 tests pass with 3 skipped,
and those 3 pass against a real S3-compatible server. Nothing in the
suite is now unverified for lack of infrastructure.

* feat(storage): add asset HTTP backend (#1007)

* docs(storage): separate trusting a header from reading one (#1007)

The header rule said a server reads exactly Content-Type and
Content-Length, while the size rules require reading Content-Encoding in
order to reject it -- the two cannot both be literal, and the
implementation had to pick one.

What the rule means is narrower: no request header may be trusted to say
who is asking, outside the authenticate hook. Reading a header to decide
how to frame or refuse a request is a different thing, and the transport
headers are read and acted on.

* chore(storage): bump to 0.2.3 for the asset server backend (#1007)

Additive: new asset registry, byte layers, collector, HTTP handler and
client, and their entry points. Nothing existing is removed or changed
incompatibly, so this is a patch under the pre-1.0 rule.

* fix(storage): close four defects found reviewing the asset backend (#1007)

**Concurrent writes could exceed a principal's logical quota.** The check
ran on the pool before the write transaction, so two concurrent puts both
read the old total and both passed; enough concurrency amplified it
arbitrarily. It now runs inside the write transaction behind a
transaction-scoped advisory lock on the principal, taken only when a quota
is configured -- a branch on deployment configuration, never on data.
Pinned by a real-PostgreSQL test: four concurrent six-byte writes against
a ten-byte quota, of which exactly one may be accepted. Removing the lock
accepts three.

**A part name smuggled inside a quoted filename was read as a real part
name.** `Content-Disposition` was scanned with a regex that does not
understand quoted strings, so `filename="x; name=meta; y"` parsed as a
`name` parameter here while an RFC-aware intermediary sees an unnamed
part. That is exactly the parser differential the contract requires be
rejected. Replaced with a tokenizer that honours quoted strings and
escapes and requires exactly one real `name`.

**Any S3 404 was reported as an absent object.** `NoSuchBucket`, a
misdirected endpoint, and a revoked access point all answer 404 while the
bytes still exist, so a storage outage surfaced as `404 ASSET_NOT_FOUND`
and a caller clearing its reference on that would turn it into real data
loss -- the same failure the contract spells out for `401`. Only
key-absent codes map to a miss now; everything else propagates to
`500 INTERNAL_ERROR`.

**Exceeding `maxParts` answered `400` rather than `413`.** It is one of
the declared multipart resource limits and now answers like the other
three.

Verified against PostgreSQL 16 and a real S3-compatible server:
840 tests, none skipped.

* fix(storage): repair two defects the previous fix round introduced (#1007)

Both were created by the fixes themselves rather than surviving them,
which is the failure mode a second review round exists to catch.

**An over-quota replace answered 500.** Moving the quota check inside the
write transaction put it under a catch that re-threw only
AssetNotFoundError, so AssetQuotaExceededError was collapsed into a
generic registry failure and the handler mapped it to INTERNAL_ERROR --
losing the status and code the contract gives that condition. Both typed
errors now survive the catch.

**The hand-written disposition tokenizer accepted malformed input and
trusted it.** The quoted-value loop never checked that it found a closing
quote, so `name="meta` ran to the end of the header and returned `meta`;
`name="meta"junk` returned `meta` as well; and an extended `name*` form
competing with a plain one was silently ignored. Each recreates the
parser differential the tokenizer was written to remove. A quoted value
now requires its closing quote, rejects a dangling escape, and permits
only whitespace and a separator after it, and an extended name form is
refused rather than resolved.

Also widened the quota lock key from `hashtext` to `hashtextextended`.
The 32-bit form collides -- two unrelated principals sharing a key would
block each other for the whole transaction, which spans a byte write that
may be a network upload.

Regression tests cover all three: an over-quota replace raises the quota
error and leaves the original bytes, and each of the three malformed
dispositions is refused.

Verified against PostgreSQL 16 and a real S3-compatible server:
842 tests, none skipped.

* fix(storage): narrow the multipart disposition surface and the error boundary (#1007)

Three review rounds have each found a defect in the part-disposition
parser: a regex that misread quoted strings, then a tokenizer that
accepted an unterminated quote, then RFC 2231 continuation forms
(`name*0*=UTF-8''bytes`) and malformed empty parameter slots. Patching it
a fourth time would have been the wrong move.

This contract needs exactly one parameter, so it now accepts exactly one:
a part disposition is `form-data` plus a single `name` whose value is
`meta` or `bytes`, and everything else is a validation failure. That
removes the continuation forms, the encoded forms, the empty slots, and
`filename` -- the parameter every one of these attacks travelled in --
without having to model RFC 2231 at all.

That rule turned out to bind our own client too: it emitted a fixed
`filename="asset"`. Nothing reads it, and the contract already forbids
deriving a response disposition filename from caller data, so the client
no longer sends one.

Separately, `instanceof` was being used as provenance. The byte layer is
pluggable, so a byte store raising a same-named `AssetQuotaExceededError`
or `AssetNotFoundError` inside the transaction was re-thrown as a logical
registry outcome, carrying the byte layer's own message to a direct
caller. The registry's checks now raise module-private sentinels, and
only those are converted -- at the boundary, into fresh public errors
with this package's fixed messages. Anything else, whatever its name,
collapses to the generic registry failure.

Also pinned the advisory-lock key width. Reverting `hashtextextended` to
the 32-bit `hashtext` left every test green, so the emitted lock
statement is now asserted. A SQL-text assertion is deliberate here: the
behavioural difference is a collision between two particular keys under a
database-internal hash, which is not a stable thing to assert against.

Verified against PostgreSQL 16 and a real S3-compatible server:
845 tests, none skipped.

* fix(storage): define multipart disposition grammar (#1007)

* fix(storage): delegate multipart parsing to Fetch (#1007)

* fix(storage): require both multipart parts to be files (#1007)

Delegating the framing to the platform parser cost two rules for a
metadata part sent without a filename: such a part comes back as a
string with its headers discarded, so its `application/json` type could
no longer be checked, and its bytes had already been replacement-decoded
-- `{"x":"<0xFF>"}" was stored as `{"x":"\uFFFD"}` instead of refused.

The bytes part already had to be a file for binary safety. Metadata is
now symmetric: the client sends a fixed filename on both, the server
requires both to be files, and with the part preserved the media type is
checked and the bytes are decoded fatally before parsing. Neither
filename is read anywhere.

The limits are also described honestly now. `maxParts`, `maxMetaBytes`
and `maxAssetBytes` are validated after the parser has materialized
every part, so they bound what is accepted rather than what is parsed,
and `maxRequestBytes` is what caps memory. The contract said `maxParts`
bounded parser work; it does not. Restoring that would mean adopting a
streaming parser with its own grammar, which is the disagreement with
intermediaries that delegating to the platform exists to remove -- so
the trade is stated rather than reversed.

Also corrected the media-type wording: the two branches are metadata
present versus absent, and retention of an untyped replacement's prior
type is a browser-backend behaviour the HTTP path cannot reproduce,
because a conforming parser supplies a default type for a file part
whose header omits one.

Verified against PostgreSQL 16 and a real S3-compatible server:
853 tests, none skipped.

* docs(storage): the asset server backend ships in this branch (#1007)

The contract still described the server backend as not yet shipped, which
this branch is what changes. The conformance server remains test-only.

* fix(storage): make asset HEAD bytes-free and pin reads (#1007)

* fix(storage): harden asset write boundaries (#1007)

* chore(storage): bump to 0.2.4 for the asset server backend (#1007)

main merged in a 0.2.3 of its own, so this branch's bump no longer
increased the version. Additive relative to the merged base -- new asset
registry, byte layers, collector, HTTP handler and client, with nothing
removed or narrowed -- so a patch bump under the pre-1.0 rule.

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-08-10 12:30:06 +08:00
wyuc 12814079ba refactor(pbl): consume the contract's PBL project types (#1079) 
Third and final PR for #1061. lib/pbl/v2/types.ts becomes a barrel over @openmaic/dsl plus a recursive app-side runtime overlay (plain Omit & Overlay after the dsl contract types were made precise again — index signatures removed, schema tolerance moved into generation post-processing). Write validator byte-identical to base; negative type tests and keyof sentinels pin the contract surface. dsl 0.8.0, storage 0.2.3 (comment-only). Closes #1061.
 2026-08-09 23:15:03 -04:00
wyuc 0ae21aa559 refactor(pbl): retire v1 write path and dead UI, converge on v2 (#1060) 
* refactor(pbl): retire v1 write path and dead UI (#1058)

Remove the legacy generator, renderer branches, chat API, and compatibility mirror writes.

Keep stored v1 scenes readable through documented legacy adapters, and surface ordinary PBL planner double failures instead of degrading.

* fix(pbl): contain planner failure to the scene, support legacy scenes in action regeneration, tighten legacy guards

- Contain typed planner failures per scene while preserving retry and skip behavior.
- Upgrade legacy PBL content before regenerating scene actions.
- Apply video-export import boundaries to legacy readers.
- Harden legacy rendering and empty-state handling.
- Restore legacy upgrade compatibility coverage.
- Remove retired translations and refresh fallback narration and legacy cover documentation.

* fix(pbl): keep tolerant legacy configs renderable, guard invalid chat timestamps

* chore(dsl): bump to 0.6.3 for README accuracy fix

The README no longer references the deleted PBLProjectConfig symbol; the version gate requires a bump for any publishable package input change.

* fix(pbl): harden legacy upgrade and persistence contract

* fix(pbl): align action prompt with v2 workflow and validate project containers

* fix(i18n): remove retired PBL v1 keys from the new fr-FR locale

The fr-FR locale landed on main after this branch removed the v1 UI key families; prune the same 38 keys so all 10 locales align with en-US.

* fix(generation): sync packaged pbl-actions template with the v2 wording

The generation package that landed on main carries a byte-identical copy of the app's pbl-actions template; sync the v2 rewording and bump the package version.

* fix(pbl): normalize legacy content and preserve planner status

* fix(pbl): propagate provider status and ground action prompts

* fix(pbl): classify planner failures for retry, default the project summary prompt input

* fix(pbl): fall back to legacy data when v2 payloads are damaged, harden summary building

* refactor(pbl): resolve hybrid content through a validity-aware resolver

* fix(pbl): keep one v2 acceptance criterion across validator, resolver and persistence

Persistence gates on the same container predicate as the validator and the renderer's resolver: payloads the renderer will not treat as v2 are inert bytes everywhere, avoiding runtime side effects for documents the write barrier would reject. An explicit null proficiencyAssessment stays processable.

* fix(pbl): require runnable projects and harden legacy fallback

* test(pbl): exercise the real save reload and hydration chain

* fix(pbl): require an instructor and usable microtasks before v2 takes precedence

* fix(pbl): fall back to the loop planner unless the provider failed or the user cancelled

* fix(pbl): require role and task identities before v2 takes precedence

Runtime normalization can repair status and create the instructor thread, but it cannot synthesize the role id that binds the thread, the microtask ids used by lookup and update paths, or the labels the workspace renders. The runnable predicate and its video-export mirror now require them, so identity-less payloads fall back to usable legacy data.
 2026-08-08 23:26:17 -04:00
wyuc 7aba740a64 refactor(app): consume the contract's interactive and widget types (#1073) 
Second of three PRs for #1061 (after #1070). WidgetType re-exports from @openmaic/dsl; the app's rich widget configs extend WidgetConfigBase; InteractiveContent aliases the contract generic; the write validator adopts the contract's html-or-url rule while staying lenient over historical widget shapes (with a primitive-config barrier matching the hydration crash class); widget-config extraction seeds the contract type; @openmaic/generation drops its WidgetType copy and gains the dsl dependency (0.1.1).
 2026-08-06 22:53:16 -04:00
wyuc 587deb2bde feat(dsl): promote the interactive and pbl content kinds into the contract (#1070) 
First of three PRs for #1061. InteractiveContent (generic widget-config extension, open schema surface) and PBLContent/PBLProject (design-time definition plus planner-seeded skeleton; opaque deprecated legacy payload; historical runtime-overlay fields tolerated) enter @openmaic/dsl with validators, normalizePBLProject, guards, and a four-kind scene schema. DSL_VERSION unchanged (additive); npm 0.6.3 -> 0.7.0.
 2026-08-06 08:04:17 -04:00
wyucClaude Opus 5杨慎
 		ceb05adb41 feat(media): allocate generated assets through the registry (#1007 part 2, step b) (#1039) 
* feat(media): establish shared asset ownership primitives

Introduce global browser asset-pool ownership, asset-reference collection, stage reclamation planning, and lease-based URL access. Carry allocated media identity through storage and generation boundaries with regression coverage.

* fix(media): enforce safe resolution across every consumer

Route image, video, thumbnail, presentation, and video-export consumers through one resolution state machine. Prevent opaque allocated or generated references from reaching render and export sinks, with fallback and ownership tests.

* fix(media): protect document-owned assets across mutations

Allocate pool bytes before compatibility writes and document commits, then roll back uncommitted generations safely. Preserve document ownership across edits, retries, imports, speech generation, scene changes, and stage deletion.

* fix(media): scope retries and tighten ownership guard

Scope retries to the target scene and slide and refuse ambiguous shared-reference mutations. Expand retry rendering coverage and keep direct pool URL resolution behind the shared lease owner.

* fix(storage): avoid nested lock during stage cleanup

Execute prepared reclamation plans against an explicitly deleted document so the compatibility cascade cannot re-enter the per-document lock. Cover deletion of stage-owned media rows even when the document has no references.

* refactor(media): confine reclamation to stage deletion

Remove inline pool and compatibility-row cleanup from element, speech, scene, and audio replacement flows. Keep whole-stage reclamation behind explicit stage ownership, preserve stage-less legacy audio rows, and document deferred document-truth sweeping.

* fix(media): close retry and resolution gaps

Restore shared source tasks after successful forks, scope retries across both whiteboard locations, and wait for parallel TTS workers before rollback. Resolve background media through import, rendering, and PPTX export paths while keeping retry controls visible over last-good bytes.

* test(media): execute the consumer safety matrix

Replace source-substring checks with resolver seam execution across all six UI consumers, stage hydration, and both exporters. Pin each rollback layer independently and harden the ownership guard against aliased pool imports.

* chore(packages): publish the additive DSL field

Bump the DSL patch version for the optional speech-action field. Keep the transitional reclamation policy app-owned and document the legitimate transaction rollback removals there.

* fix(media): make retry rollback task-safe

Delay allocation task re-keying until final document reconciliation succeeds, restore shared source tasks on failed forks, and require exact stage-whiteboard targets before falling through from a missed scene.

* fix(media): clear private assets and refresh leases

Delete the asset-pool database during the confirmed local-data wipe. Notify the app lease layer after same-id replacement so mounted consumers re-resolve current bytes without reaching into storage internals.

* test(media): pin closure safety guards

Exercise the CSS allocation boundary, unfiltered legacy-row ownership, delete-and-undo byte survival, and real distinct consumer seams. Restore the video-only manifest overwrite condition and share the direct video resolution hook across both element variants.

* test(media): narrow rollback element assertion

Narrow the reconciled slide element to an image before checking its source so the rollback regression remains type-safe under the full root compiler configuration.

* fix(media): close lease refresh races

Gate the first batch publication by unique resolved refs, then publish every replacement snapshot without mutating the prior React state object. Serialize invalidation behind pending releases, evict rejected refreshes, and register replacement observation at the pool boundary.

* fix(media): reopen pool after clear failures

Always evict the singleton once its store has been closed, including blocked and failed database deletion paths. Report blocked deletion as deferred and prove a later write uses a fresh live store.

* fix(media): isolate shared retry progress

Track forked regeneration under the selected element until it receives a fresh asset identity, leaving the shared source task and bytes untouched. Surface targeted failures through renderer task lookup and skip the redundant fork reconciliation lock.

* fix(media): close final asset retry gaps

Keep blocked asset clears fail-loud until a successful retry, with actionable settings guidance.

Clear failed shared-fork state across durable and live key spaces, and pin renderer lookups, lease publication identity, and committed-ref rollback protection.

* fix: preserve actionable retry failures

Localize the blocked cache-clear recovery hint across every supported locale and pin the deferred-error mapping.

Retain durable fork failure rows while retries run, deleting them only after successful generation so unstructured failures survive reload.

* fix(media): hydrate legacy stored video thumbnails

Home-page recent-video thumbnails regressed for legacy Dexie mediaFiles
rows keyed by gen_vid placeholders: the reworked hydration resolved the
row's bytes through the sealed resolver but never surfaced the stored
blob (and its poster) as object URLs for the preview card, so the CI
recent-video-thumbnail e2e specs found no visible element.

Hydration now materializes legacy stored video rows into blob URLs for
both the element src and poster while keeping the resolver invariants:
opaque refs still never reach a DOM src, and concrete addresses are
never blanked. Unit pins cover the seam so the vitest suite catches this
class without a browser.

* fix(media): resolve sole restored legacy video

Classroom playback restored tasks by exact document media references. Legacy gen_vid references can outlive the key used by the one persisted video row, leaving the player on a placeholder even though bytes were restored.

Select the sole completed stage video only for legacy sequential refs after exact and reconciled matches. Keep exact failures authoritative, refuse ambiguous candidates, and cover success, ambiguity, and failure precedence in unit tests.

* fix(media): scope legacy video recovery

Decide restored legacy video recovery once from the complete document and record it through the shared task lookup consumed by playback, editing, and resolved slides.

Keep ambiguous documents as placeholders, apply the same decision to thumbnail hydration, and preserve exact failure precedence.

* fix(media): exclude claimed video recovery tasks

Model restored legacy recovery as a two-pass match across document video elements and task rows. Remove tasks claimed by exact, targeted, or placeholder lookup before applying the sole-candidate fallback, and cover the ownership/cardinality matrix.

* fix(media): unify video element resolution

Centralize source, task, poster, and legacy recovery decisions for every video consumer. Ensure direct URLs win over opaque refs and play_video waits on element-targeted retry tasks.

* refactor(media): route legacy recovery through resolver

Let document-aware consumers request legacy video recovery through the unified element binding API, keeping thumbnail hydration on a single decision path.

* fix(media): preserve import refs and prefer pool bytes

Recognize unambiguous extensionless relative media addresses during classroom import. Resolve allocated export and thumbnail assets from the shared pool before falling back to lagging compatibility rows.

* fix(export): preserve concrete video sources

Route PPTX video elements through the shared media binding resolver so an unresolved opaque reference cannot replace a playable source. Complete browser and persisted-store cleanup when asset-pool deletion is deferred, while retaining distinct hard-failure behavior.

* fix(media): fork retries without exclusive ownership

Enumerate logical asset owners across every persisted document before allowing global pool replacement. Thread explicit targets through fresh-id rewrites and cover cross-document aliases plus unreadable ownership.

* fix(media): guard global asset reclamation

Share a fail-closed persisted-document liveness check between stage deletion and retry replacement. Preserve cross-document pool aliases while deleting stage-owned compatibility rows and cover enumeration failures.

* fix(media): cover complete slide asset references

Route slide media traversal through a shared mutable slot contract so backgrounds participate in export, thumbnail hydration, collection, and rewrite lifecycles. Snapshot complete surviving-document refs once per reclamation and preserve manifest-only owners while retaining fail-closed behavior.

* fix(media): preserve exclusive retry asset ids

Allow targeted retries to replace exclusively owned pool assets in place. Keep shared and unprovable ownership paths on fresh allocations, and pin production-shaped retries plus compatibility-row cleanup.

* fix(media): revalidate asset bindings at completion

Recheck repository-wide ownership before replacing generated media and fork scoped retries when exclusivity changed. Route video export selection through the unified resolver and keep concrete posters independent of task state.

* fix(media): count unflushed owners and broadcast replacements

The completion-time exclusivity proof read only the persisted document, but
slide duplication updates the Zustand aggregate synchronously and schedules
persistence behind a debounce. A retry finishing inside that window saw a
single persisted owner and replaced the bytes behind a reference the
duplicate also held. The proof now also counts owners in the live stage
snapshot when that snapshot represents the stage being retried, so an
unflushed duplicate forks instead.

Same-id replacement notifications were realm-local, so a second tab showing
the same classroom kept its lease pinned to the superseded blob URL. The
notification now travels over a BroadcastChannel; each receiving realm runs
its own observers against its own pool, so a spoofed message can at most
force a re-resolve. A missing or failing channel never fails the replacement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): bind replacement listeners and spare shared audio rows

A realm that only renders never sends a replacement, so binding the channel
from the sender path left passive tabs deaf to peers. Binding now happens
where the observer is registered, when the asset-pool module loads, and the
receiving realm resolves its own pool lazily so a cleared or unavailable
pool degrades to the next resolve instead of throwing.

Observer notifications ran under Promise.all, so a rejection surfaced after
BrowserAssetStore.replace had already committed and turned a durable success
into a reported failure. They are settled individually now; the callback is
wrapped because a synchronous throw would otherwise escape before allSettled
sees the array.

audioFiles rows are keyed globally by audioId, so deleting a stage removed
the sole row for an id a surviving document still referenced — playback and
both export paths read that table directly and cannot fall back to the
preserved pool blob. Rows are now filtered against surviving references,
while a failed enumeration still withholds only the irreversible pool
removal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): serve replaced bytes from the pool everywhere

Unknown survivor liveness deleted every planned audioFiles row. Those rows
are keyed globally by audioId, and playback plus both export paths read the
table directly, so losing one is as irreversible for them as removing the
pool entry. Unknown liveness now preserves the rows too, leaving bounded
garbage for a later pass that can prove exclusivity.

The earlier pool-first change covered PPTX, video collection and thumbnail
hydration but missed classroom ZIP export, which still serialized the stale
compatibility row after a lagged same-id replacement, shipping media the
classroom no longer renders.

Auditing every direct reader of the media and audio tables surfaced the same
gap in playback: speech regeneration also replaces bytes under a stable id
and does not roll the pool back when the compatibility write fails, so the
player kept serving superseded narration. It now resolves the pool first and
falls back to stored rows for legacy and imported audio.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(audio): replace exclusively owned speech clips in place

regenerateSpeechAudio accepted the action's audioId but always passed
undefined as replaceAssetId, so regenerating an exclusively owned
pool-backed clip allocated a new asset every time, rewrote the action and
orphaned the previous pool entry and compatibility row until stage
reclamation — contradicting the stable-id path generateAndStoreTTS already
implements for media.

Ownership is now established before synthesis, and the rule itself moved to
the shared reference module so media retries, poster replacement and speech
regeneration consume one implementation instead of restating it. An
exclusively owned clip keeps its id and has its bytes replaced; a shared
clip, a legacy id with no pool entry, or unprovable ownership still gets a
fresh allocation so other holders keep their audio.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(media): drop the import left behind by the ownership move

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): resolve audio pool-first and fence peer realms

Stable-id TTS regeneration commits replaced narration to the pool before the
audioFiles mirror write, so a failed mirror leaves the row stale. AudioPlayer
already resolved pool-first, but audioObjectUrl, collectAudioFiles and the
video timeline dependencies still read Dexie directly and would serve the
superseded clip. All allocated-audio readers now share one resolver, with
Dexie kept as the fallback for legacy and imported rows.

The exclusivity proof modelled unflushed owners in the active realm only, so
another tab duplicating the same asset during its save debounce could still
be observed as a single owner and have its bytes replaced globally. A peer's
pending state cannot be read across realms, so presence is probed instead:
any realm holding the stage forces the fork path.

A deferred asset-pool deletion no longer reloads the page. The database is
still on disk and the guidance asks the user to close the other tab and
retry, which the reload discarded. The decision moved into a shared helper so
the rule is pinned rather than living inline in the component.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): fail closed when presence cannot be probed

The presence helper documented that an unanswerable probe must count as a
peer, but every unavailable path — no BroadcastChannel, a constructor that
threw, a send that failed, and the window before the pool's asynchronous
binding completes — returned false, so the ownership proof cleared a single
local owner and replaced globally shared bytes in place.

Probing now returns present, absent or unknown, and only a probe that was
actually sent and went unanswered is absent; the ownership decision treats
unknown exactly like present. The pool declares its binding intent
synchronously so a probe issued during the load-time window waits for the
bind instead of concluding that presence is unavailable, and releases that
gate if the import fails.

Coverage reaches the write boundary: with presence unknown, a
production-shaped targeted retry forks to a fresh id instead of calling
replace, and the original bytes stay intact for a peer's unflushed owner.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(dsl): bump to 0.6.3 after the release dedupe took 0.6.2

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(i18n): add the deferred-clear guidance to fr-FR

The locale landed on main after this branch added the key, so the alignment
check flagged it as the one missing translation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-08-06 18:49:34 +08:00
wyuc 9556a035b1 feat(generation): move outline generation into the package (#1065) 
Part B of #1057. Outline primitives, buildOutlinePrompt (single-source prompt construction with golden snapshots), logger injection, JSON repair, and the bare-Node second-consumer smoke script with CI coverage. Shim-first: no app-side behavior changes; duplicated prompt assets guarded by a CI parity check until Part D.
 2026-08-05 23:29:36 -04:00
wyuc 335180ec9c feat(generation): scaffold @openmaic/generation with pipeline types and packaged prompt assets (#1063) 
Part A of #1057. Package skeleton (ESM, tsc, dsl/storage pattern), pipeline type contracts, prompt loader with package-relative asset resolution, 13 templates + 7 snippets byte-identical with golden tests, allowlist boundary lint, full publish/CI wiring, installed-tarball smoke coverage.
 2026-08-05 22:22:54 -04:00
wyuc ad30061b6e fix(chat): harden runtime sync, legacy migration, and record validation (#1050) 
* fix(chat): harden runtime sync and legacy migration

* fix(storage): validate HTTP runtime appends client-side

* fix(storage): harden runtime validation and legacy guards

* fix(chat): skip malformed legacy messages
 2026-08-05 06:23:45 -04:00
zhifu gaoandLauraGPT 2b3f2eaa37 feat(asr): add local FunASR provider (#1044) 
Signed-off-by: LauraGPT <199134975+LauraGPT@users.noreply.github.com>
Co-authored-by: LauraGPT <199134975+LauraGPT@users.noreply.github.com>
 2026-08-05 03:03:56 -04:00
xppcnnandwyuc 378579e9df fix(slide-renderer): keep arrowheads visible when toggling thumbnails… (#1045) 
* fix(slide-renderer): keep arrowheads visible when toggling thumbnails or fullscreen

* fix(renderer): bump version to 0.0.5

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-04 22:57:04 -04:00
wyuc 90f5f3942b fix(release): dedupe the published dsl, close two release-path blind spots (#1019) 
* fix(release): dedupe the published dsl, close two release-path blind spots

Three independent defects on the release path.

1. `@openmaic/storage`, `@openmaic/renderer` and `@openmaic/importer`
   declared `"@openmaic/dsl": "workspace:*"`, which pnpm publishes as an
   EXACT pin. The tarballs on the registry today say storage@0.1.1 needs
   dsl 0.5.0, renderer@0.0.3 needs dsl 0.4.0 and importer@0.1.1 needs dsl
   0.4.0, so installing storage together with renderer yields two copies
   of the dsl. The dsl carries the schema, the validators and the version
   constants, so a document produced against one copy can be validated by
   the other's schema revision.

   `workspace:^` publishes as `^0.5.0`. Under 0.x that admits 0.5.x, so a
   dsl patch reaches dependents without republishing them while a dsl
   minor still requires a deliberate dependent release. All three
   packages take a patch version increase, both because
   check-package-version-bumps.mjs requires one and because the corrected
   range only reaches consumers through a new release.

   The tarball smoke test now asserts the packed range, so an exact pin
   cannot come back unnoticed.

2. The publish workflow's build-integrity guard ran `git diff --quiet --
   packages/@openmaic`, which compares the worktree to the INDEX. The
   steps before it run third-party package code, which needs no
   credential to rewrite a tracked file and `git add` it, after which the
   guard reported clean and the rewritten bytes were packed and
   published. Both copies of the guard now compare against HEAD, the
   commit actually being released.

3. `@openmaic/storage`'s PostgreSQL contract suites refuse to skip when
   STORAGE_PG_CONTRACT_REQUIRED=1, but that refusal is a `throw` inside
   the test modules, so it never fires if vitest stops collecting them.
   Collection is decided by `packages/@openmaic/storage/vitest.config.ts`,
   which is on the ignore list of publishable inputs, so excluding
   `*.pg.test.ts` there turns both PostgreSQL backends off with a green
   run and no version bump.

   scripts/assert-pg-contract-suites.mjs reads vitest's json results and
   requires both suite files to be present, passed and non-empty, keyed
   on the file names rather than on a count. It runs in the publish
   workflow's validate job and in the storage PostgreSQL contract
   workflow.

Merging this publishes @openmaic/storage@0.1.2, @openmaic/renderer@0.0.4
and @openmaic/importer@0.1.2. That republish is the fix for (1), not a
side effect of it: the corrected dependency range exists only in a
tarball that has not been cut yet.

* fix(release): audit the contract database, pin the format version to a minor

Follow-up on cross-vendor review of the three release-path fixes.

1. The PostgreSQL assertion proved far less than it claimed. Vitest's
   `assertionResults` are test CASES, not `expect()` calls, so one
   `test('x', () => {})` per required filename satisfied it — and the
   whole surface is reachable without a version bump, because `test/` is
   on the same ignore list as `vitest.config.ts`. A `vi.mock('pg', ...)`
   in the already-wired `test/setup.ts`, or a stubbed suite body, made it
   print "ran against a real database" having checked nothing of the
   sort.

   The script now runs in two phases. Phase 1 keeps the file/status
   checks. Phase 2 connects to PG_CONTRACT_URL from outside the vitest
   process and requires all five tables the two backends own to exist and
   to show inserts in pg_stat_user_tables. Nothing inside `test/` can
   forge that. Cumulative insert counters rather than surviving rows,
   because the suites clean up after themselves. The success message now
   states only what was checked.

2. Validation packed with lifecycle scripts while publishing runs
   `--ignore-scripts`, so committing `workspace:*` plus a `prepack` that
   rewrites it to `workspace:^` passed the tarball assertion and then
   published the exact pin. Both the smoke test and the workflow's
   dry-run pack now pass `--config.ignore-scripts=true`, so validation
   and publication have identical lifecycle semantics.

3. `workspace:^` unpins the serialized-format version that the exact pin
   was silently holding. DSL_VERSION and RUNTIME_DSL_VERSION are
   decoupled from the npm version by design, and storage compares them by
   value across the package boundary. Under `^0.5.0` a dsl PATCH could
   therefore hand a new format to an already-published storage: one
   install resolves it and stamps the new version into document_stages,
   another install pinned to the older patch reads that row and hard-
   fails. Two installs of one published storage version, data-
   incompatible.

   check-package-version-bumps.mjs now requires a change to either
   constant to carry at least a MINOR increase of the dsl package
   version, since a caret does not cross a 0.x minor. The rule is
   documented next to both constants. That doc edit is itself a
   publishable change, so dsl takes a patch bump to 0.5.1.

4. The range and build-integrity guards ran only in the publish workflow,
   so a pull request restoring `workspace:*` or leaving a stale generated
   file failed at release time, after a version number had been spent.
   ci.yml gains a cheap source-level range check
   (scripts/check-internal-dependency-ranges.mjs) and the
   `git diff --quiet HEAD` integrity check after its install step. The
   tarball assertion stays where it is as the release-time proof of
   actual published behaviour.

Merging this now publishes @openmaic/dsl@0.5.1 as well as
@openmaic/storage@0.1.2, @openmaic/renderer@0.0.4 and
@openmaic/importer@0.1.2.

* fix(release): fail the gates closed, and state what they actually prove

Confirmation round on the release-path fixes.

1. The format-version rule failed open if version.ts moved. It hard-coded
   one source path and returned successfully when either revision lacked
   it, so renaming the file, changing DSL_VERSION and bumping dsl by a
   patch passed both checks. It now reports an error whenever the file is
   absent at either revision while dsl has any publishable change. The
   constant regex is anchored to the start of a line, so a commented-out
   declaration cannot be read as the value, and a duplicate declaration is
   an error rather than a coin flip.

2. "At least a MINOR" was wrong past 1.0.0: `^1.0.0` admits minors, so a
   format change shipped as a minor would float into published dependents
   exactly as the rule exists to prevent. The rule is now stated as what
   it needs to be — an increase the dependents' caret range will NOT admit
   — computed from the pre-change version: a minor while dsl is 0.x, a
   major once it reaches 1.0.0. The error message and the docs in
   version.ts say the same thing.

3. Internal declarations outside `dependencies` bypassed both range
   checks. Keeping `dependencies` at `workspace:^` while adding an exact
   `peerDependencies` or `optionalDependencies` entry published a second,
   tighter constraint and passed. An owned package must now appear exactly
   once, in `dependencies`; entries in the peer and optional fields are
   rejected outright, in the source check and in the packed-manifest
   assertion. The global count test is replaced by naming the three
   dependents that must be present.

4. The format rule compared against the base tip, so an un-rebased branch
   produced an inverted message. It now uses the merge base, matching the
   `base...HEAD` form the rest of diff mode uses. Deliberately not applied
   to the version comparison, where the tip is the correct reference.

5. Phase 2 had no baseline, so cumulative counters from an earlier run
   satisfied it forever against a non-ephemeral database. It now captures
   the counters before the vitest step and requires a positive delta.
   Both workflows gained the capture step.

6. Phase 2 overclaimed. It proves rows were inserted into those five
   tables during the run, not that the built PgDocumentStore and
   PgRuntimeStore inserted them — `test/` is on the publishable-input
   ignore list, so test code could write directly. The success message now
   says exactly that, and the script documents the threat model: this
   catches accidental silencing, which is the defect it was written for;
   it is not a defence against someone who can merge changes to `test/`,
   who can alter production code just as easily.

Also: scripts/openmaic-packages.mjs is now the single source for the
owned package list, consumed by all three checking scripts, and it
cross-checks itself against both the packages directory and
publish-packages.yml so a package added in one place and forgotten in
another fails rather than going quietly exempt. The workflow's own YAML
lists cannot import it, which is why they are cross-checked instead.

* fix(release): let the format-version source move without opening the gate

Failing closed on a missing version.ts is right, but on its own it made a
rename impossible to merge: the new path does not exist at the base
revision either, so the check refused both before and after the move.

DSL_VERSION_SOURCES is now an ordered candidate list, resolved per
revision against the first path that exists there. A rename lands by
prepending the new path in the same change, and the comparison still
happens across the move. Absent everywhere at either revision, while dsl
has any publishable change, remains an error.

* fix(release): report package-list drift instead of crashing on it

Every check in check-internal-dependency-ranges.mjs reads the shared
package list, so a list naming a package that does not exist threw an
ENOENT out of readManifest before any finding was printed. It failed
closed, but unreadably. The list is now validated first and fatally, with
its own headline.

* fix(release): anchor the caret to the base branch, and check the gates' own inputs

Final confirmation round.

1. The caret escape was computed from the merge base, which is the wrong
   reference for it. With merge base dsl 0.5.1, an ordinary minor landing
   on main taking the tip to 0.6.0, and HEAD at 0.6.1 with a changed
   DSL_VERSION, the rule computed the escape from 0.5.1, got 0.6.0, and
   accepted 0.6.1 — but dependents released against 0.6.0 publish
   `^0.6.0`, which admits 0.6.1, so the new format reached them anyway.
   0.6.1 is also the minimum the ordinary version check allows, making
   that the default outcome rather than an unlucky one.

   The escape is now computed from the HIGHEST dsl version reachable on
   the base branch. Whether the format moved still comes from the merge
   base, because that is a question about the branch. The comment on
   mergeBaseWithHead now records that it is a no-op in both current CI
   invocations (checkout uses the merge ref on pull_request, and the
   before-SHA is an ancestor on push) rather than claiming a case that is
   not exercised there.

2. The workflow cross-check matched raw text, so a commented-out trigger
   path still satisfied it while silently disabling that package's
   release. Full-line comments are now stripped first; the `on.push.paths`
   list and every `for pkg in ...` loop are compared as exact sets, so an
   unexpected entry fails as well as a missing one; and `--filter` usage
   is checked more loosely, because individual steps legitimately filter
   subsets. The limits of a textual check are stated where it lives.

3. `locate()` returned the first candidate format-version source that
   existed, so during a half-finished rename the gate could compare a file
   that is no longer the exported one. It now resolves every candidate per
   revision and fails unless exactly one declares the constants.

4. INTERNAL_DEPENDENTS was checked in one direction only, so deleting an
   entry exempted that package from the source check and from the packed
   assertion that iterates the same map. It is now cross-checked both
   ways, an owned package in another owned package's devDependencies is
   rejected outright, and multiple owned dependencies no longer collapse
   to whichever was declared last.

5. assertPackageListIsComplete() now also runs in release mode, which is
   the gate that decides what gets published; and the PostgreSQL script
   consumes flag values properly instead of treating the first non-flag
   argument as the results path.

Also documents, in openmaic-packages.mjs, the threat model these gates
share: they catch mistakes, and anyone who can merge edits to scripts/
can edit production code directly, so deliberate subversion is out of
scope and cannot be closed by adding further checks.
 2026-08-02 22:44:34 -04:00
wyuc 84876ef723 feat(dsl): unify media asset reference types (#1007 part 2, step a) (#1038) 
* feat(dsl): unify media asset reference types

* fix(document-store): migrate verification baseline

* fix(dsl): address asset reference review findings

* fix(dsl): close asset reference review gaps
 2026-08-02 02:45:37 -04:00
wyuc 3204051d09 chore(storage): address asset registry review follow-ups (#1037) 2026-08-02 11:08:38 +08:00
wyuc ee1bd6bbee feat(storage): asset registry — allocated ids over the content-addressed blob layer (#1007 part 1) (#1024) 
* refactor(dsl): restate the asset seam as allocated, non-disclosing refs

The asset seam's docstrings described a ref as a handle whose scheme was the
backend's business, and recommended content-addressed hashing so identical
bytes de-duplicate. Both are now wrong in the same place: a content-addressed
ref *is* a statement about the bytes, so returning one from `put` tells the
caller whether those bytes were already stored — an existence oracle over data
the caller never wrote.

State the contract the global asset pool needs instead: `put` allocates, every
call returns a new ref, and nothing the caller receives reveals whether the
bytes pre-existed. A provider may still de-duplicate internally; that is a
property of its storage layer, not of the ref. Also spell out that the ref
domain is unconstrained, so an unrecognized ref is a miss rather than an error.

Comments only — no interface signature changes.

* feat(storage): add the asset registry — allocated ids over the blob layer

Part 1 of the global asset pool (#1007): the browser registry core.

`BrowserAssetStore` is now the package's asset backend. It maps an allocated
`AssetId` to a registry entry (`contentHash`, `mime`, `meta`), and the registry
maps that hash to bytes, giving three properties at once:

- an id survives the bytes behind it being regenerated or replaced, and one set
  of bytes can back several ids with different metadata;
- identical bytes occupy one row however many ids name them;
- the content hash never leaves the package, so "whoever knows the hash can
  reach the bytes" — the threat pure content-addressing has to defend against —
  structurally does not arise, and the blob layer beneath can stay global and
  metadata-free without being an isolation hole.

`put` always allocates a new id and always writes the blob unconditionally, so
the first-store and de-duplication-hit paths run the same statements, issue the
same requests, and return the same shape. The caller cannot infer that someone
else's bytes are already present.

Both stores live in one IndexedDB database, so reference counting on `remove`
happens inside the transaction that deletes the entry, over an index on
`contentHash`. IndexedDB serializes readwrite transactions over the same
stores, so a last-reference removal can never reclaim bytes a concurrent `put`
has just adopted.

Images, audio and video share one id space; the medium is a `mime` column, not
a partition. There is no `principal` column: a principal is derived server-side
from an authenticated session, and a value the client supplies about itself
authorizes nothing — the column arrives with the server backend.

The id domain is opaque and unvalidated, taking the KV key domain's conclusion
as its starting point. `toAssetId` brands without checking; an id this store
never issued is a miss, never an error.

`BrowserAssetProvider` is demoted to an internal blob backend and dropped from
the public exports (0.x, and it had no consumers): its content-addressed ref is
precisely the de-duplication signal an outward API must not emit.

Tests split along the same seam. `runAssetStoreContract` pins the outward
semantics — allocation, an existence-disclosure matrix compared facet by facet
rather than asserted in prose, reference-counted release, and a table of
foreign ids that must all miss without throwing — while `runBlobStoreContract`
keeps the content-addressed semantics for the layer underneath. Facts the
outward API must not disclose (one blob row per byte string, reclamation at the
last reference, the put/remove race) are asserted against the rows.

* chore(packages): bump @openmaic/dsl to 0.5.1 and @openmaic/storage to 0.2.0

`storage` takes a minor bump rather than a patch: `BrowserAssetProvider` is no
longer exported from the package entry point, which is breaking under 0.x
convention even though nothing consumed it.

* fix(storage): address asset registry review findings

* fix(storage): complete asset registry lifecycle

* fix(storage): harden asset cache coherence and lifecycle

* fix(storage): close pending asset mint edges

* refactor(storage): adopt immutable asset URL snapshots

* fix(storage): close asset snapshot lifecycle gaps

* test(storage): pin asset lifecycle contracts

* docs(storage): document 0.1.x asset upgrade

* fix(storage): reject incompatible asset database schemas

* fix(storage): enforce asset allocation independence

* fix(storage): instrument production asset allocation

* test(storage): cover allocation ordinals across histories

* refactor(storage): remove standalone blob backend seam
 2026-08-01 22:21:23 -04:00
wyuc 091989c48b feat(storage): promote KV HTTP contract + clients to main (Part A of #1000) (#1020) 
* feat(storage): KV HTTP contract and clients (#1000 Part A) (#1002)

* feat(storage): KV + Asset HTTP contracts and clients

Adds the client half of the server backends for the two components in the
storage RFC that still had a browser backend only.

Asset contract: the ref is `sha256-<hex>` over the bytes, computed by one
shared rule both backends call, and it *is* the address bytes are written
to (`PUT /assets/{ref}`). The server re-hashes what it received and refuses
a mismatch, so content addressing is enforced end to end rather than
assumed, and uploads become idempotent. `resolve` returns a URL per call,
either an absolute signed URL passed through untouched or a root-relative
proxied path joined to the base origin — the document never stores a raw
URL either way.

KV contract: account scope only, with no scope path segment, query
parameter or body field anywhere on the wire. `HttpAccountKV` is the only
object that can reach it and none of its methods takes a scope, so a
device value is not a request the transport can express; `HttpKVStore`
routes `device` to a local backend it requires at construction. Tests
assert the invariant both ways — `@ts-expect-error` on every scope-passing
call (unused directives fail the build if a scope is ever added) and a
fetch spy proving device traffic does not exist.

Both clients pass the existing `runKVStoreContract` and
`runStorageProviderContract` suites against an in-memory conformance
server, the asset one in both the proxied and signed URL shapes.

* fix(storage): close the review findings on the KV + asset HTTP clients

Cross-review found the device-scope guarantee resting on its weakest layer,
and several places where the contract promised more than the code or the
conformance server could deliver.

deviceStore: the two type-level layers only documented intent — the barrier
that actually kept device values off the wire was the injected store, typed
as `KVStore`, which `HttpAccountKV` satisfies structurally (its methods are
the same methods minus an optional parameter). Injecting the transport as
the device backend type-checked and put device values on the network. The
device scope now requires a branded `LocalKVStore`, the transport declares
itself not to be one, and a runtime check refuses what a cast smuggles past.

resolve URLs: a media `src` carries cookies, never the client's headers
hook, so the proxied shape is only sound for cookie-authenticated
deployments — the contract now says so instead of claiming the content route
sits behind "the same authentication", and the conformance reader uses a
cookie rather than a header no browser could send. The client also validates
what it resolves: http(s) schemes only, and no `//host` or `/\host` slipping
through the relative branch to another origin.

Identifiers: refs and keys arrive percent-decoded, so `..%2F..%2F` was a
traversal waiting for the first backend to join it to a path. Both are now
constrained on both sides, and the contract states the MUST that keeps a
storage layer from being the first to look.

Also: the JSON gate runs before anything reads the value (a `JSON.stringify`
pre-flight ran caller code first and reclassified `{ toJSON: () => undefined }`
as a delete); scope routing fails closed; an upload's Content-Type can no
longer be relabelled by the headers hook; a mutation invalidates in-flight
resolution; unparseable 2xx bodies become typed errors rather than native
SyntaxErrors; signed tokens are validated rather than merely present; and the
contracts settle the per-principal claim model, the media-type allowlist, the
literal prefix rule, and the identifier charset that Part B would otherwise
have had to guess at.

* fix(storage): close the second review round on the KV + asset HTTP clients

The device-scope guard was in one place and needed to be in three. The
transport is exported and structurally a `KVStore`, so `kvPersistStorage(new
HttpAccountKV(...), 'device')` and a plain `KVStore`-typed reference both
compiled without a cast, dropped the scope argument on the floor, and put a
device value on the wire. The transport now takes the scope it was pretending
not to have — typed as `'account'`, refused at runtime for anything else, since
bivariant method parameters mean the type alone cannot hold — and the persist
adapter requires a `LocalKVStore` when paired with `'device'`.

The resolved-URL checks were text tests where only parsing works. The URL
parser strips ASCII tab, LF and CR before parsing, so `/<tab>/evil/x` walked
past the `//` prefix check and loaded cross-origin — in the app-mounted shape,
where a path-only base URL leaves that check as the only guard. The absolute
branch let through `https://` with no host and `https:host/x`, which names one
place standalone and another when a browser resolves it against a same-scheme
page. Both branches are now decided by the parser: control characters refused
outright, relative references validated against a probe origin, absolute ones
required to parse as `scheme://host` with an http(s) scheme — and still
returned byte-for-byte, so signatures survive.

The media-type MUST had no implementation and contradicted itself: the
conformance server stored and served `text/html` verbatim, and a metadata-less
upload became `application/octet-stream`, which cannot be on a renderable
allowlist. The server now implements the allowlist, serves everything else as
an opaque attachment, and the contract scopes its "usable as an img src"
promise to allowlisted types.

The KV key domain was HTTP-only, which made "the same suite proves the backends
equivalent" untrue: `a/b`, an empty key and a 600-byte key all passed in the
browser and would have become unreachable on a server. An audit of the app's
keys (learner key, storage generation, current-scene and migration markers,
persist names) found none in violation, so the rules are lifted to the
primitive and both backends enforce them, with the shared suite asserting it.

Also: the claim model the last round wrote into the contract is now implemented
and covered case by case; malformed percent-escapes are a 400 rather than a
500; the in-flight identity guard has a test that can fail; and the Content-Type
conflict raises a typed error.

* fix(storage): final review round — backend parity, URL probe, credentials

Four small things the final pass found, plus the contract patches Part B would
otherwise inherit.

The `toJSON` delete semantics were documented backwards. `BrowserKVStore.set`
was the trial-serialize the KV contract said no backend does, so
`{ toJSON: () => undefined }` was a silent delete there and a rejection over
HTTP — opposite answers to the same call. The browser backend now validates
scope and key before anything can run caller code, recognizes the delete case by
type, and refuses a value that still fails to serialize. Both cases are in the
shared suite, so the two backends cannot drift apart again.

The probe-origin check could not vouch for a URL naming the probe host itself:
`//asset-url-probe.invalid/x` resolves onto the probe origin and passed, then
resolves somewhere else entirely against a real base. Any authority-bearing
form is refused outright, so the check no longer depends on the probe host
being unguessable.

Cross-origin cookie deployments had no way to send a cookie: `fetch` omits
them cross-origin by default and the headers hook cannot help, since `Cookie`
is a forbidden header name. Both clients accept `credentials` and pass it to
`fetch`; the contract records the CORS obligations that come with it. Rather
than restricting cookie deployments to same-origin, which the asset contract
explicitly does not require.

The conformance server's dot-segment check was reading a path the URL parser
had already normalized, so `/assets/%2e%2e/x` arrived as `/x` and the documented
400 was really post-routing behaviour. It now validates the raw request target,
and the injected transport preserves it instead of handing over a normalized one.

Contract: the prefix rule names backslash (PostgreSQL's default LIKE escape,
and the one an implementer skips because a legal key cannot contain it, though
a prefix can), and acquiring a claim MUST leave the bytes present — a
de-duplicating server that inserts only the claim can race reclamation of the
last previous claim and leave a document resolving to a 404.

Also: base URLs are checked for an http(s) scheme at construction, the ref
domain moved next to the KV key domain and is enforced by both asset backends
with shared suite coverage, the KV client refuses a hook-supplied Content-Type
like the asset one, and CI runs the package typecheck that makes the
`@ts-expect-error` guards mean something.

* fix(storage): probe guard spelling, prefix validator, base url shape

Three one-line-level holes from the closing review.

The probe-host guard was spelled `startsWith('//')`, so the backslash forms
`/\host/x` and `/\/host/x` reached the origin comparison, passed it by naming
the probe host, and were stopped only by that host failing to resolve. It now
refuses any second character that can open an authority — which is also what
makes the comment true rather than aspirational: a path-absolute reference
always inherits the base's origin, so with no authority expressible the
comparison cannot be satisfied cross-origin, whatever the probe host is.

The conformance server validated `?prefix` with the path-segment validator,
rejecting `keys('.')` and `keys('..')` — legitimate prefixes of keys like
`.hidden`, which the client rightly allows. A prefix travels in the query
string and is never a path segment, so it now has its own validator: the key
rules minus the non-empty and dot-segment requirements. The contract says the
same, and the shared suite pins both backends to it.

Base URL validation moved into one shared helper both clients call, since two
copies of a security rule is how they drift. It now also refuses
`https:host/path` — which parses standalone to that host but is resolved
against the current document by `fetch`, sending requests and their credentials
somewhere the operator never wrote — and a base carrying a query or fragment,
which silently swallows the path concatenated onto it.

* fix(storage): close two holes in the shared base-url check

The path-only branch returned its input unexamined, so `//evil.example` and
`/\evil.example` were accepted as paths. A browser reads both as an authority,
which means every request built on such a base — and every credential the
headers hook attaches — leaves the origin. A base is now required to be a
single `/` followed by something that is not `/` or `\`, the same refusal the
resolved-URL check already makes, for the same reason. Control characters are
refused first: the parser strips or re-encodes them, and `/<tab>/evil.example`
becomes `//evil.example` after it does.

The query/fragment check read the parsed `search` and `hash`, which are empty
strings for a base ending in a bare `?` or `#` — so exactly the inputs that
swallow the concatenated path into a query or fragment were the ones that got
through. It reads the raw string now, which also makes the post-parse check
redundant rather than merely unreachable.

* fix(storage): bypass HTTP caches on reads, and require a 404 for not-found

Two mechanisms the clients were missing, on both sides.

Reads are marked `cache: 'no-store'`. The asset resolve request answers two
mutable questions — does this principal hold a claim, and which URL is valid
now — so a cached answer returns an expired signed URL or replays a negatively
cached 404 for an asset written since; the in-flight coalescing that would
otherwise cover this lives in the client and cannot reach the HTTP cache layer.
KV reads have the same problem for a different reason: `account` is by
definition the scope another device may have just written, and nothing here can
invalidate a cached copy when it does. The contracts now require `no-store` on
those responses. Asset content is left cacheable — bytes at a content-addressed
ref never change — and writes are unmarked, having nothing to read from a cache.

The not-found mappings now require the status as well as the code. A proxy or
gateway answering 401, 403 or 500 while echoing the body's error code was being
read as "this does not exist", and `null` is indistinguishable from a real
miss — so an outage reached the caller as data loss.

* fix(storage): reject unusable base urls, and make the harness meet the contract

Two configuration checks and two harness gaps.

A base URL carrying userinfo built a client that threw a native TypeError on
every operation, because Fetch refuses to construct a request from a URL with
embedded credentials — one legible error where the deployment is configured
beats a per-call failure with no explanation. A base with surrounding
whitespace was worse than useless: `new URL` strips it, so validation passed,
and then concatenation moved the space inside the URL where it is no longer
trailing and no longer stripped, breaking every request.

The conformance server now sends `Cache-Control: no-store` on the three reads
the contracts require it on. The requirement landed last round with nothing
asserting it, so the suite would have passed against exactly the implementation
it exists to catch. And `DELETE /kv/entries/{key}` refuses a request body
outright: the query and the header were already checked for a smuggled scope,
leaving the body as the one channel through which a delete could still describe
one.

* fix(storage): allow a device-routing composite to persist device-scoped state

cosarah found kvPersistStorage(httpKVStore, "device") wrongly rejected. An
HttpKVStore is a full dual-scope composite: it routes device operations to the
LocalKVStore it requires at construction and only reaches the network for
account, so device values never leave the machine. The guard tested
isLocalKVStore — "is the whole store local" — and so turned away a store that
is safe for device precisely because it is not fully local.

The guard now turns on the capability that actually matters: does the device
scope stay local. A new DeviceSafeKVStore brand (servesDeviceScopeLocally: true)
marks a store whose device writes never leave the device — held by a fully-local
LocalKVStore (which extends it) and by HttpKVStore, and declared false by
HttpAccountKV, the pure account transport with no local device backend. The
persist device overload requires DeviceSafeKVStore; the account overload is
unchanged.

The security boundary the red team opened this whole line on is untouched:
HttpAccountKV used for device is still doubly refused — its false brand is not
assignable to the true DeviceSafeKVStore requires (type), and the runtime check
rejects the cast that would erase it. The brand is structurally unforgeable, not
a flag the transport could set; and the injected device backend of a composite
still demands the stronger, fully-local LocalKVStore, so composites cannot nest
into a router loop with no local floor.

Regression test (cosarah's ask): kvPersistStorage(httpKVStore, "device")
set/get/remove reach the injected device backend, land device-scoped there, and
make zero HTTP requests. Mutation-checked: widening the brand so HttpAccountKV
is accepted, or neutering the guard, turns the device-out reject test red.

* refactor(storage): scope #1002 down to KV — remove the asset HTTP backend

Per the maintainer decision, the AssetProvider server design moves to the
global-resource-pool RFC (#1007), where AssetRef changes from content-addressed
to an allocated global id and the contract model itself changes. Continuing to
harden the content-addressed asset client here would be work on code that is
being replaced, so the asset half of #1002 is withdrawn and the PR is pure KV.

Removed (all introduced by #1002):
- docs/asset-http-contract.md
- src/asset/http.ts (HttpAssetProvider) and its index / package.json exports
- src/asset/content-ref.ts (assertAssetRef / computeAssetRef / the ref shape)
- test/http-asset-provider.test.ts and the asset half of the conformance server

Reverted to the #858 baseline, untouched:
- src/asset/browser.ts — back to opaque resolve/remove, no assertAssetRef guard.
  This directly removes cosarah's latest asset finding: an opaque ref such as
  `bucket/path` no longer throws, restoring the DSL AssetRef-opaque contract.
- test/asset-contract.ts, test/asset-browser.test.ts — 0-diff from origin/main.

Kept (all KV): kv-http-contract.md, src/kv/* (HttpKVStore / HttpAccountKV /
the DeviceSafeKVStore brand), the KV tests, and the KV conformance server —
renamed kv-conformance-server.ts and reduced to KV routing over the shared
harness framework (body caps, no-store, scope-channel and dot-segment guards,
auth hooks). base-url.ts stays; KV still uses it.

* fix(storage): treat KV keys as opaque; document account-sync prerequisites

cosarah found the key validation we added for traversal defense breaks
legitimate callers: DSL identifiers (stageId/sceneId/learnerKey) are
unconstrained strings — verified, the DSL only requires typeof === 'string' —
and app callers compose keys like `editor-current-scene:<stageId>`, so a valid
stage id `stage/one` now throws in BrowserKVStore before localStorage.

Fix, matching the asset-opaque precedent: a KV key is an opaque string again.
The `/` and `\` rejection is removed from assertKVKey / assertKVKeyPrefix and
from the conformance server. Traversal is defended by encoding, not rejection —
the HTTP client already percent-encodes the whole key into one path segment, so
a separator never opens a new segment on the wire, and the server stores the
decoded key as a plain map key (bound value), never a path. `a%2F..%2F..%2Fb`
decodes to the single opaque key `a/../../b`, which addresses one entry and
traverses nothing; a raw unencoded `..` path produces extra segments that match
no route (404). The rules that remain are only what encoding cannot cover and
the transport cannot carry (empty, whole-key `.`/`..`, NUL, lone surrogate,
length) — none producible by `prefix + id`.

Tests: the shared suite now round-trips keys with `/`, `\`, `%`, `:`, spaces
across both backends; a `stage/one` key round-trips through the real conformance
server; the traversal input is proven to store as one opaque key and the raw
path to 404. Mutation-checked: re-adding the `/` rejection reddens exactly the
round-trip cases.

Docs (kv-http-contract.md):
- "A key is an opaque string" replaces the identifier-shape section — keys may
  contain separators; traversal is an encoding + server-bound-value duty.
- Account settings sync now documents its security prerequisite next to the
  account scope: a deployment MUST provide real per-user isolation of stored
  rows and encryption at rest before enabling sync; the reference server is not
  that posture.
- kvPersistStorage(device) doc corrected from LocalKVStore to DeviceSafeKVStore
  (the device-safe brand accepts the HttpKVStore composite; HttpAccountKV stays
  double-rejected).

* fix(storage): close the scope/key/prefix contract gaps with a coverage matrix

Three completeness gaps, nailed by an executable matrix so an untested
channel/shape can't slip through the next pass rather than patched one by one.

1. Scope via a GET body was not rejected. `assertNoScopeChannel` covered the
   query and header, but the bodyless GET routes never read their body, so
   `GET /kv/keys` with `{"scope":"device"}` returned 200 []. Bodyless methods
   (GET, DELETE) now reject any request body up front — that is the channel a
   scope would hide in — and only `set` reads a body, checking it for a scope
   field. The DELETE route's own body check folds into the shared one.

2. The 512-byte key bound wrongly rejected legitimate keys. A DSL id is an
   unconstrained string (verified: the DSL requires only typeof === 'string'),
   and `editor-current-scene:` + a 500-char stage id is 521 bytes, which
   BrowserKVStore rejected before localStorage. Keys are length-unconstrained;
   the bound is raised to a generous 8 KiB DoS ceiling and documented as a
   resource guard, not part of the key domain — not a derived/hashed key, which
   would break the literal-prefix semantics of keys(prefix). Both backends and
   the conformance server move in lockstep.

3. The prefix contract text contradicted itself: it claimed a legal key can
   never contain a backslash, while the key section (correctly) makes keys
   opaque, backslash included. Reworded — a prefix is opaque like a key — and
   the shared literal-prefix matrix now covers `\` (PostgreSQL's LIKE escape),
   which it had skipped, alongside `%`, `_`, and combinations.

Coverage matrix (CI-executable):
- scope-rejection: {get,set,remove,keys} × {query,header,body}. query/header
  run in-process; the body cells use a raw HTTP client (fetch cannot put a body
  on a GET — exactly the non-conforming client the finding described) against a
  loopback listener, skipped where the sandbox forbids binding.
- key-validity: `/ \ % _ : .` space, BMP + astral Unicode, `%2F`, a 500-char
  composed key, and all-at-once — each round-trips (set/get, keys(prefix)
  match, remove) on both browser and HTTP backends via the shared suite.
- prefix-charset: `%`, `_`, `\`, and combinations match literally, both backends.
- Boundaries still rejected: whole-key `.`/`..`, NUL, lone surrogate, empty,
  and the DoS ceiling.

Mutation-checked: removing the bodyless-body rejection reddens the three GET/
DELETE body cells; lowering the ceiling back to 512 reddens the long-key
round-trip on both backends.

Bumps @openmaic/storage to 0.1.2 (src/kv/* changed).

* fix(storage): bound the KV key DoS ceiling by encoded size, not UTF-8 bytes

Follow-up to the 8 KiB ceiling: measuring it in UTF-8 bytes ignored
percent-encoding expansion and reopened a backend-parity break. A non-ASCII key
is small in bytes but large on the wire — `€` is 3 UTF-8 bytes and `%E2%82%AC`,
9, encoded — so `'€'.repeat(2730)` (8190 bytes, under the old cap) encoded to
~24.6 KB, which the Node HTTP server rejects with 431 before routing while
`BrowserKVStore`, which never builds a URL, accepted it.

The ceiling is now measured on `encodeURIComponent(key).length` — the size the
key occupies as a request target — at 4096, far above any real id (a 500-char id
under `editor-current-scene:` encodes to ~523) and well below Node's default
16 KiB `maxHeaderSize` (empirically the 431 trigger on Node 22), so no key that
passes validation can reach a 431. Both backends and the conformance server use
the same encoded-size limit, so a key is accepted iff its encoded form fits,
whichever backend it reaches — parity restored. `MAX_KV_KEY_BYTES` becomes
`MAX_KV_KEY_ENCODED_LENGTH`.

Tests, against a real node:http server for the 431 boundary:
- the max-legal non-ASCII key (455×€, encoded 4095) round-trips over real HTTP
  with no 431, and the browser accepts the same key — parity on accept;
- one code point over (456×€, encoded 4104) is rejected by both backends at
  validation, before any request — the exact parity case a byte bound missed
  (1368 bytes, would have passed, then 431'd over HTTP);
- a raw client bypassing client validation with an over-ceiling (under 16 KiB)
  encoded key gets a clean 400, not a 431.
Shared suite's key-domain matrix now bounds by encoded size on both backends.

Mutation-checked: reverting the ceiling to UTF-8 bytes reddens the three
encoded-length parity tests. Bumps @openmaic/storage to 0.1.3 (src/kv/* changed).

* fix(storage): make the KV key domain truly opaque; keep transport limits out

The recurring key-limit findings (512 → 8 KiB → 4096, each patch exposing the
next) shared one root cause: transport-driven rules kept being added to the key
domain — the browser primitive included — breaking the "opaque unconstrained
string key" contract. This is the structural fix, per cosarah: "keep HTTP
transport limits out of the browser primitive and preserve the existing string
key domain."

Approach (a), the minimal one: a KV key is any string, validated nowhere. The
browser primitive is a Map/Storage that stores any key; the HTTP backend carries
the key as a URL path segment and inherits that transport's limits, documented
as HTTP-deployment concerns that never constrain the key domain. Every real key
(a `prefix:id` from an unconstrained DSL string) round-trips on both backends;
only pathological keys hit a transport limit, and none of those are keys a caller
produces. (b) — keys in the request body for accept-parity — was unnecessary:
verified that no real key needs it.

Removed from the key domain (browser + HTTP + conformance server):
- the encoded-size / length ceiling entirely — there is no key-length limit;
- the NUL exclusion — NUL percent-encodes to %00, is preserved end to end, and
  is a legitimate Map/Storage key (a false "cannot represent" premise);
- the `.`/`..`/empty/`/`/`\` rejections — all opaque, all round-trip.
`assertKVKey` / `assertKVKeyPrefix` / `MAX_KV_KEY_*` are gone.

Kept, at the transport layer only (HttpAccountKV), never reaching the primitive:
- an unpaired UTF-16 surrogate genuinely has no percent-encoding, so the HTTP
  client surfaces a clear transport error (KEY_NOT_ENCODABLE) while the browser
  stores the same key — verified: encodeURIComponent throws on it.

Scope completeness (P2): the "reject scope in every channel" guarantee had two
holes — a scope path segment (`/kv/device/keys`) 404'd instead of 400, and only
`x-scope` was checked among header spellings. The server now rejects a scope
path segment and the whole prohibited header set (scope, x-scope, kv-scope,
x-kv-scope). The coverage matrix is {get,set,remove,keys} × {path, query, each
header spelling, body}.

Coverage matrix reworked to verify arbitrary opaque keys (empty, NUL, control
chars, separators, long, Unicode) round-trip on both backends — not to verify a
limit. A real node:http test proves NUL and long keys round-trip end to end, and
that a lone surrogate is a transport limit (browser accepts, HTTP client errors).
Mutation-checked: re-adding a NUL rejection, removing the path-segment check, or
dropping a header spelling each reddens the matching cells.

Cleanup: package.json description updated (KV HTTP delivered, not "later");
network-server bind/skip/close lifecycle deduped into one helper. Bumps
@openmaic/storage to 0.1.4 (src/kv/* changed).

* fix(storage): reject whole-key "."/".." in the HTTP client — they alias entries

Follow-up to making the key domain opaque: removing the `.`/`..` key-domain
rejection was right (the browser stores them fine), but the HTTP client then let
them reach `fetch`, where URL path normalization silently rewrites the target
before it is sent. `encodeURIComponent('.')` is `.`, so `/kv/entries/.` collapses
to `/kv/entries/` (the empty key) and `/kv/entries/..` walks up a level — so
`get`/`set`/`remove` on a whole-key `.` or `..` silently read, overwrite, or
delete a *different* entry, with no error. A data-correctness bug.

Fix, same pattern as the unpaired-surrogate limit already handled: a whole-key
`.`/`..` is something the URL-path transport structurally cannot carry, so the
client's path-segment encoder refuses it with `KEY_NOT_ENCODABLE` before a
request is built. Split the encoder in two: `encodeKeyPathSegment` (key → path
segment, rejects whole-key `.`/`..`) and `encodeComponent` (the `keys()` prefix →
query value, where `.`/`..` are legitimate and untouched). A key that merely
*contains* a dot (`a.b`, `prefix:id`) is unaffected and round-trips. The browser
primitive still stores `.`/`..` opaquely.

The HTTP transport's limits are now complete — three cases, all fail-loud at the
transport layer, all storable on the browser: (1) an unpaired surrogate (no
percent-encoding), (2) a whole-key `.`/`..` (URL normalization aliases it), and
(3) an over-long key (deployment request-target ceiling). The contract lists all
three as HTTP-deployment limits, not key-domain constraints.

Tests: whole-key `.`/`..` — browser stores and round-trips (distinct from the
empty key); HTTP client throws `KEY_NOT_ENCODABLE` and builds no request; and a
real-server test asserts the empty-key entry is never read, overwritten, or
deleted by a `.`/`..` operation. A dot-containing key (`a.b`) round-trips.
Mutation-checked: dropping the whole-key check reddens the alias tests.

Bumps @openmaic/storage to 0.1.5 (src/kv/* changed).

* fix(storage): reconcile the settings/profile persist consumer with the KV API

The integration merge (#1002 KV into main) surfaced two things the merge itself
could not resolve mechanically:

- `lib/store/kv-persist.ts` (added on main by #1001) called
  `kvPersistStorage(kv, scope)` with a *variable* scope, which #1002's overloads
  deliberately reject — `'device'` must be paired with a device-safe store, so
  the overloads take the scope as a literal. Branch on the scope: `'account'`
  takes any `KVStore`; `'device'` narrows the backend to `DeviceSafeKVStore` via
  its brand (the adapter re-checks at runtime) and otherwise fails loud. The app
  wires only `'account'` today; the `'device'` arm keeps the generic helper
  honest. `ControllableKV` in the persist test wraps a local `BrowserKVStore`,
  so it is genuinely device-safe and now declares the brand.

- `packages/@openmaic/storage` version bumped 0.1.5 → 0.1.6: the merge changed
  storage inputs (README + a pg-schema test carried from main) relative to the
  previous integration tip, so the version-bump guard requires an increase above
  #1002's 0.1.5. Still ahead of main's 0.1.1.

Verified: root `tsc --noEmit` clean, the persist test green (40 passed), and the
storage package suite green on Node 22.
 2026-07-30 22:31:17 -04:00
wyucClaude Opus 4.8杨慎wyuc
 		f378c25a53 feat(storage): route settings/user-profile persist through KVStore (no legacy migration) (#1001) 
* feat(storage): route zustand persist through KVStore

The settings and user-profile stores persisted through zustand's default
`localStorage` storage, while the rest of the app's small keyed values had
already moved to `@openmaic/storage`'s `KVStore` — two unrelated mechanisms
over the same browser API, which is the split-brain the storage RFC set out
to remove. `kvPersistStorage` shipped with the package but had no importer.

Add `lib/store/kv-persist.ts`, the app seam that adapts a `KVStore` into a
zustand `PersistStorage`, and wire both stores to it under an explicit
`account` scope. Store logic is unchanged: `version` / `migrate` / `merge`
still belong to the stores, only the bytes' destination moves.

Existing installs are adopted rather than orphaned. On first read the seam
moves the raw `localStorage` entry into the KV scope — write first, then
drop the raw key, so an interrupted migration leaves the value readable
from one side or the other but never from neither. Once KV holds the entry
it is authoritative, so a stale raw entry reappearing cannot roll back
newer state, and re-running can neither duplicate nor resurrect anything.
An entry that is not a readable persist envelope is left exactly where it
is.

Hydration is now asynchronous, because a `KVStore` may be remote. The
browser backend resolves within a few microtasks of the store module being
evaluated, so nothing user-visible changes, but tests that assert on
persisted state now await `persist.rehydrate()` instead of relying on a
synchronous read.

The four pre-persist keys (`llmModel` / `providersConfig` / `ttsModel` /
`selectedAgentIds`) are deliberately untouched: they are read-only inputs
to a migration that never owned them, and `providersConfig` is still read
independently by `lib/ai/providers.ts`.

Part of #857.

* fix(storage): close the async-storage failure modes in the KV persist seam

Cross-review found that routing `persist` through an asynchronous store
opens failure modes a synchronous `localStorage` never had. Each one below
is now covered by a test that fails without its fix.

Data loss on a failed or empty hydration. A KV read that threw was
indistinguishable from an empty scope, so the store stayed on defaults
while an always-on initializer's `set()` persisted those defaults; the
next load then read them as a completed migration and deleted the raw
original, leaving nothing. Reads now separate "nothing there" from "the
backend could not answer": a failed read serves the raw entry read-only,
adopts nothing and deletes nothing, and leaves the key unwritable for the
session so no un-hydrated state can be persisted over an original. Raw
entries are additionally only dropped against a recorded completion
marker, so a KV entry of unknown provenance can never authorize deleting
the only copy of the user's data.

Lost adoption across tabs. A reader whose KV read answered before another
tab's adoption write, and whose localStorage read landed after that tab's
delete, saw nothing in either place and hydrated defaults. Against a
remote backend that interleaving is ordinary rather than rare, so an
apparent "nothing anywhere" is now re-checked before it is believed.

Out-of-order writes. Two rapid `setItem`s could complete in either order
and let the older value win. Reads and writes are serialized per key.

Pre-persist migration resurrecting on every load. `migrateFromOldStorage`
was guarded by the existence of the raw `settings-storage` key — the very
key the migration removes — so once the blob moved into the KVStore it ran
on every load and republished whatever the pre-persist keys still held,
including API keys the user had since deleted, into first-frame state. It
is now the persist layer's last-resort source instead of a synchronous
read in the store initializer: its result is adopted like any other
migration, so the adopted entry is the marker that stops it. Initial state
is plain defaults, and nothing stored reaches observable state before the
storage layer has vouched for it. The source keys are still only read,
never deleted, since `lib/ai/providers.ts` remains a reader.

Also: KV reads and writes are guarded so a corrupt entry surfaces instead
of throwing through zustand's silent catch; clearing the cache now clears
the migrated stores through their own storage, which a server-backed
scope would need and a blanket `localStorage.clear()` would only appear to
do; `removeItem` drops the KV copy before the raw one; and the e2e suite
gains post-migration steady-state coverage, which every existing spec
missed by seeding the pre-cutover key.

Part of #857.

* fix(storage): scope migration bookkeeping to the device, gate writes on call

Second review round. The findings shared one root cause: two of the three
sentinels recorded a fact about *this machine's* files but were stored
beside the data, in the store's own scope. Under a syncing `account`
backend that is a category error with teeth — one device's history would
travel to another and authorize deleting files it says nothing about.

Sentinels are now always `device`-scoped and carry a reserved `__` prefix,
whatever scope the store uses. The worst combination this closes: device A
migrates, its marker syncs, device B — which has never run this build —
loads and deletes the raw entry that is still the only copy of its own
settings. A test drives exactly that sequence against two devices sharing
one account scope and nothing else.

The pre-persist fallback gains a device sentinel of its own. Its source
keys are device-local and deliberately left in place, so "KV and raw are
both empty" was not a safe trigger: clearing the account entry, from the
cache button or a server-side wipe, let any device re-read those keys and
republish credentials the user had rotated. It is now consulted at most
once per device, and `removeItem` deliberately does not clear that
sentinel — clearing the entry is precisely the case it exists for. The
fallback also no longer writes the adoption marker: that marker is a
licence to delete a raw persist entry, and the fallback has adopted no
such thing. Were it written, a raw entry later restored by an older bundle
would be deleted unread.

The write gate is now read when `setItem` is called rather than when its
queued turn arrives. Evaluated inside the queue, a write issued during
hydration waited behind `getItem`, found the gate opened by the very read
it raced, and persisted its pre-hydration snapshot on top of the stored
value.

Refused writes are no longer a dead end: the first one asks the store to
rehydrate, since the backend may have recovered, and if that does not work
the user is told through a durable notice rather than losing every change
on reload. Each refusal is logged, not just the first — a once-per-session
warning hides how long a session has been silently unsaved.

`removeItem` now propagates backend failures, so clearing the cache cannot
report success for a clear that did not happen, and that flow reloads
immediately instead of leaving a window in which a background `set()`
re-persists the credentials just deleted.

Also: the recovery hook is bound after each store is constructed, because
naming the store inside its own definition made its type circular and
silently widened every selector to `any`; the `serial` comment no longer
claims an unreachable rejection handler; and the pre-persist version stamp
documents that it skips `migrate`'s unversioned normalization too, which
`merge` redoes anyway.

Part of #857.

* refactor(storage): make the KV persist seam a per-key state machine

Three review rounds kept turning up the same defect class in new places:
a backend failure read as "there is nothing there", and the result of an
operation that nobody looked at. Patching instances of it was not working,
because both are possible whenever a call site is free to decide for itself
what a failed operation meant. This makes that decision unavailable to
call sites.

Every backend call now returns an `Outcome` whose payload is a private
field and whose only reader demands a `KeyState`. Feeding the machine is
not a convention to remember; it is the only way to get the value out.
What comes back is the value or `UNAVAILABLE` — deliberately not `null`,
so "the backend could not answer" is not spellable as "nothing is there".
The machine then decides, in one place, what failure means: any read
failure is unavailability, any write failure closes the key, and either
raises the health signal and asks for recovery.

The scattered closure flags (a settled set, a warned set, a recovery
latch) become one explicit phase per key — unhydrated, settled,
unavailable, clearing — with the transition table written out in the
docstring and a test per row.

What that consumes:

- A write failing after the key settled used to be ignored; its outcome
  now closes the key, signals, and the value is remembered for replay.
- `removeItem` enters `clearing` synchronously at call time, so a `set()`
  racing a clear cannot be admitted, queue behind the delete, and write
  the just-deleted value back. Nothing is replayed across a clear.
- The pre-persist sentinel is recorded by *every* settling path, not only
  the one that consults the old keys. A device that had only ever loaded
  an already-populated account entry recorded nothing, and so stayed
  eligible to republish its own stale credentials the moment that entry
  was cleared — which the docstring already claimed it would not.
- Two places treated a failed read as an answer: the sentinel read that
  decides whether the fallback runs, and the confirming re-read before
  adopting. Both now stop.
- An adoption whose marker did not land no longer settles, and never
  deletes the raw entry: no proof, no licence.
- Refused and failed writes are remembered and replayed once storage
  recovers, but only when the store was holding real data at the time —
  replaying a snapshot taken over defaults would overwrite the stored
  value, which is the failure this seam exists to prevent. When a
  snapshot cannot be replayed the user is told, because rehydration is
  about to discard those edits.
- Health notices publish on a later task, so a recovery that lands
  moments later cancels a warning nobody should have seen, and a
  subscriber mounting after the failure is caught up once its toast host
  exists. `StorageHealthNotice` now sits after the `Toaster` for the same
  reason.

Also: adoption re-reads before committing so a concurrent write is not
rolled back; a shadowed raw entry is only deleted if it parses, matching
the promise never to delete bytes we could not read; the unavailable copy
points at the control that resolves it rather than suggesting a reload
that cannot; and the version pin, the pre-persist retention rationale, and
the cache-clear comment say what is actually true.

Part of #857.

* fix(storage): separate the contexts settle() was conflating

Convergence round on the state machine. The findings share a root: settle()
treated every read that concluded as licence to open the key, without
asking what else was true at the time.

- A read finishing while a clear is outstanding no longer settles. Under a
  remote backend a recovery rehydrate in flight during a clear is ordinary,
  and opening the gate there admits a write that queues behind the pending
  delete and restores the credentials just removed. Only finishClear opens
  a cleared key.
- The raw entry served read-only after a failed read no longer counts as
  the authoritative value. A shadowed raw entry is by construction the
  older copy — long-term coexistence with a newer KV value is a designed
  state — so an edit taken against it must never be replayed, or one
  transient read failure on one device rolls the whole account back.
- A write that lands supersedes an earlier failed one. Both were admitted
  before the key closed, so the queue ordered them; replaying the older
  snapshot on recovery would undo the newer value.
- Unreachable browser storage is a failure, not an empty store. Privacy
  modes make localStorage throw; treating that as "nothing stored" hydrated
  defaults, refused every write and said nothing until the user reloaded
  and found the work gone. SSR stays a silent no-op.
- The recovery latch re-arms when storage answers, so the three recovery
  transitions are not empty promises from the second failure onward.
- A refusal carries the phase it happened in. Cold-start timing — an
  always-on initializer writing before the first read resolves, on a
  backend that never failed — is a log line, not a sticky red toast; only
  a refusal under `unavailable` reports lost changes.

The health channel stops delivering stale catch-ups: its timers are
cancellable and re-read the current state when they fire, so a recovery
cannot be overtaken by a warning with nothing left to dismiss it. And
`changes-lost` is now a fait accompli rather than a condition — a later
recovery has nothing to retract and leaves it standing.

Also: a throwing `localStorage` is unavailability rather than absence
wherever the raw entry is read; clearing records the pre-persist sentinel
like every other settling path, making that docstring true; the raw entry
is dropped before the marker so a failed marker clear leaves something
inert; and the pre-persist fallback and its four source reads can no
longer throw into zustand's silent catch.

Part of #857.

* fix(storage): hold a replay until it lands, and prove a raw copy before deleting it

Three failure-path corners.

A replay was retired the moment its write was attempted. Readable but not
writable is a real state — a quota-exhausted backend answers reads
perfectly — so a recovering read could succeed, the replay write fail, and
the snapshot be gone with nothing owed: the next rehydrate served the older
stored value over the user's edit and said nothing. The snapshot is now
held until the write is durable; a failed replay goes back to being a
refused write, so the next recovery tries again and a permanent failure is
eventually reported.

The adoption marker was treated as proof that a raw entry is the
migration's leftover. It proves only that a migration happened on this
device once. A rolled-back deployment, or a tab still running the old
bundle, writes the raw key again with settings the migration never saw, and
the next load on the new bundle deleted them unread. A raw entry is now
dropped only when its content equals the value shadowing it — anything else
is data, and keeping a copy beats deleting an original here as everywhere
else in this file.

`changes-lost` was being cancelled by a later recovery. The two statuses
answer different questions: "storage is down" describes the world now and
stops being true when storage works, while "your edits were lost" describes
something that already happened and no amount of recovery undoes. They are
now tracked on independent lines per key and rendered as two toasts with
distinct ids, so retracting the fault cannot take the acknowledgement with
it; the loss notice goes away only when the user closes it.

Part of #857.

* fix(storage): bound recovery, keep a live store from walking backwards

Three interactions the state machine did not have an answer for.

Recovery had no budget. A backend that reads but cannot write — a
quota-exhausted one does exactly that — turned it into a treadmill: the
recovering read succeeds, the key settles, the replay write fails, and that
failure asks for recovery again, forever. Attempts now follow a backoff
schedule whose length is the cap, and the budget is re-armed only on real
progress: a settle that left nothing owed. A settle reached *by* a recovery
with a replay still queued is not progress, and counting it as such is
precisely what made every lap look like a fresh start. When the budget runs
out the key is left read-only and the loss is reported, rather than
retrying where nobody can see it failing.

A failed read could walk a live store backwards. Once a session has loaded
the authoritative value, a transient read failure was still falling back to
the shadowed raw copy — which is legitimately older, and legitimately
long-lived now that a differing raw entry is kept rather than deleted — and
handing it to zustand overwrote a live store with stale data. Worse, the
store had already latched "holding real data", so edits made on top of that
stale value counted as replayable. A failed read after an authoritative one
now returns nothing, which is exactly "keep what you have", and the raw
fallback stays where it belongs: the case where nothing authoritative has
been seen at all.

Dismissing a lost-changes toast only closed the UI. The key stayed in the
lost set, so the same store losing changes a second time was swallowed as a
duplicate and any later subscriber resurrected a notice the user had
already dealt with. The close button now clears the latch, so a second loss
is reported and an acknowledged one stays gone.

Part of #857.

* fix(storage): close the last two provisional and clear-ordering gaps

An abandoned migration was handing back an authoritative value. When the
confirming re-read fails, or the adoption write does, `adopt` returns the
raw envelope unadopted — the same footing as the raw copy served after any
other failed read — but it was returned bare, so the store latched "holding
real data" and an edit taken on top of it counted as replayable. Another
device publishing to the account scope in that window would have been
rolled back. `adopt` now reports provisionality like every other read path,
and the shape is a named `LoadedValue` so the next path cannot forget.

Clearing swallowed a failed raw-entry deletion. The account entry was
already gone by then and the marker was cleared afterwards, so the
operation reported success while leaving an empty scope beside a live raw
entry — precisely the shape the next hydration re-adopts, handing back the
settings the user had just deleted. The raw entry now goes first and its
failure propagates, which also fixes the ordering: failing before anything
is removed leaves the KV value authoritative and the clear simply
unperformed, which the caller can retry. Adoption keeps the tolerant
variant, where a leftover raw entry is a duplicate rather than a loss.

Part of #857.

* fix(storage): make the shadowed-raw delete atomic against a concurrent tab

cosarah reproduced a data loss: both raw-entry deletes verified the value,
then awaited (reading or writing the adoption marker), then removed the key
by name. A tab running an old bundle — which takes no Web Lock, so a lock
could not have helped — overwrites the raw key with newer settings during
that await, and the unconditional `removeItem` then deletes the newer value.

The raw side is synchronous `localStorage`, so the fix needs no lock. Both
deletes now finish every await first, then run a single synchronous
compare-and-delete: re-read the raw bytes, and remove only if they still
equal the exact string that was verified. A competing tab is a separate
context whose writes surface here only at event-loop boundaries, so its
write lands wholly before the re-read (seen — bytes differ — kept) or after
the remove, never in between. A changed value is newer data, not a
migration leftover, and is kept.

Both sites cosarah named are covered: the shadowed-cleanup path
(`dropShadowedLegacy`) and the adoption cleanup in `adopt`. `readLegacyEntry`
now carries the exact bytes alongside the parse so the re-check is
byte-exact rather than comparing a re-serialized value. The clear path's
delete is unchanged: it is a deliberate full delete, not a conditional
leftover cleanup.

Part of #857.

* fix(storage): stop deleting the raw entry — adopt by copy (Plan B)

Maintainer decision after cross-vendor + red-team review: `localStorage`
has no cross-process atomic compare-and-delete (the storage mutex was
dropped from the spec), so a tab on an old bundle can insert a write
between our re-read and our removeItem in a genuinely parallel process. The
synchronous compare-and-delete narrowed that window but could not close it
— as its own comment admitted. Since the raw-cleanup cannot be done safely
on localStorage while an old bundle may be live, this stops doing it.

Adoption is now a copy: the value is written to KV and the raw entry is
left in place. The delete race is therefore impossible — there is no
delete. KV stays authoritative (once it holds the value, reads take the KV
path and never consume the raw one), so the shadow duplicate is inert,
left for a wholesale raw-cleanup at migration exit.

Removed with the two deletes:
- `dropShadowedLegacy` (its only job was the gated delete) and its three
  call sites.
- `deleteLegacyIfUnchanged` and the `{raw,value}` shape of the legacy read
  (`readLegacyEntry` → back to `readLegacy` returning the value).
- The adoption marker (`__persist-adopted:`), whose only reader was that
  delete gate. It did not prevent re-adoption — the presence of the KV
  value does — so removing it breaks no invariant. `dropRaw` / `expectedRaw`
  on `adopt` and the marker-clear step in the clear path go with it.

The clear path (user deletes all data) still deletes the raw entry
strictly — that is a deliberate full delete, not a migration cleanup.

Tests: the delete/marker cases become "the raw shadow always survives, KV
is authoritative"; a two-device and an old-bundle-concurrent-write case
guard it, and re-adding a delete on either adoption path turns them red.

Part of #857.

* fix(storage): add a monotonic migrated marker to stop shadow resurrection

Cross-vendor + red-team review independently caught a P1 the previous
"adopt-as-copy" revision introduced. Because the raw entry is now kept and
writes only ever reach KV, the raw copy becomes a frozen pre-cutover value.
If the account KV is later emptied by something other than a local clear —
a server-side wipe, an account switch, KV eviction — the next load saw
`stored === null` beside a live raw entry and re-adopted that stale copy,
refilling the wiped account with ancient settings/credentials and syncing
them to every device. Deterministic, not a race; it would fire once a
server-backed account backend exists.

Root cause: "keep the raw entry" inherently requires a record that it has
already been migrated, so an empty-KV load can tell "first migration" from
"external wipe". Removing the old delete-authorization marker in the copy
revision dropped that record — Plan B was only half done.

Adds a monotonic, device-scoped `__migrated:<name>` marker:
- Read on every load. `stored === null` + raw present + marker set ⇒ respect
  the wipe, do not adopt (fall through to the confirm re-read, so a
  concurrent repopulation is still served). Only a genuine first migration
  (no marker, raw present) adopts.
- Written best-effort after a successful KV write. A failed write degrades
  to the KV value's own presence check, so resurrection needs both a failed
  marker write and a later external wipe.
- Read failure fails closed (do not adopt), never "assume not migrated".
- Monotonic: only ever set true, cleared only by a deliberate full clear —
  no compare-and-delete race, which is why it is safe where deleting the raw
  entry was not. Live state with a real reader, not the delete-authorization
  marker removed earlier.

The clear path clears the marker too (back to a genuine first-load state);
best-effort, since with the raw entry already gone a leftover marker is
inert.

Tests: resurrection guard (external wipe does not re-adopt; concurrent
post-wipe write is served; first migration records the marker; marker read
fails closed) + the clear-path marker clearing. Mutation-checked: removing
the gate resurrects the shadow, dropping the record leaves it unguarded.
P3: the inert `__persist-adopted:` test rot is retargeted to the real
`__migrated:` path.

Part of #857.

* refactor(storage): remove automatic legacy migration — pure KV wiring

Maintainer decision (terminal): after repeated review rounds kept finding
new P1s in the legacy-adoption path — its state space is too large to keep
correct — the PR is narrowed from "migrate + wire" to "wire only". Settings
and user-profile persist through the KVStore; pre-cutover data is not
migrated; an upgrading user reconfigures once (the data is reconstructable
and the cost is one-time).

Removed the entire legacy-adoption subsystem from the adapter:
- `adopt()`, `readLegacy()`, the `legacyStorage` dep, and the raw-fallback
  read on a failed KV read.
- The `__migrated:` marker and its resurrection guard, the confirm-re-read,
  and the `provisional`/`LoadedValue` machinery.
- The `prePersistFallback` dep, `recordPrePersistHandled`, the
  `__pre-persist-consulted:` sentinel, and `readSentinel`/`writeSentinel`.
- In settings.ts, `readPrePersistSettings` (the llmModel/providersConfig
  reader wired as that fallback) — reading legacy keys is migration, which
  this PR no longer does. `lib/ai/providers.ts` still reads `providersConfig`
  independently, so those source keys are left in place.

The adapter now reads and writes the KV scope and touches no legacy key: an
empty scope hydrates the store to defaults (ordinary zustand). `load()` is a
single KV read; a failed read stays unsettled so defaults are never
persisted over an unread value; the clear path removes only the KV entry.

Kept, because they are async-storage *steady state* a remote KV backend
needs, not migration scaffolding: the KeyState machine (hydration gate,
write gate, recovery, replay), the Outcome discipline, per-key
serialization, and the persist-health channel.

Added `purgeLegacyPersistKey`: a best-effort, fire-and-forget removal of the
old raw persist blob, called once by each store. Never read, so a leftover
is pure garbage — and the old settings blob held plaintext provider API
keys, so clearing it is a small security win.

Net: the adapter drops ~460 lines. Tests are rewritten to steady-state KV
coverage plus an explicit "old raw key is ignored, not migrated" proof at
the store level; the e2e seed fixtures move from the raw key to the KV scope
key.

Part of #857.

* docs(storage): align README and comments with the no-migration behavior

Follow-up requested on the approved PR: the migration/adoption logic was
removed, but a few comments and one README roadmap line still referred to
those deleted concepts. Documentation only — no behaviour change.

- README roadmap: the settings/user-profile line said "adopting their
  pre-cutover raw keys"; it now states there is no automatic migration
  (legacy keys ignored and best-effort purged, users reconfigure once).
- kv-persist.ts: the per-key serial chain comment no longer claims to keep
  "adoption" from interleaving; it serializes a key's read/write/clear.
- settings.ts: `SETTINGS_PERSIST_VERSION` no longer documents a
  "pre-persist fallback" that no longer exists.
- Test/fixture headers that described seeding "the pre-cutover raw key" for
  "one-time adoption" now describe seeding the KV `account` scope, with no
  legacy-key migration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(storage): bump version to 0.1.1 for README doc alignment

The README is a publishable input, and the no-migration doc alignment
changed it, so the package version must increase. Patch bump: documentation
only, no code or API change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
Co-authored-by: wyuc <zdq1204@gmail.com>
 2026-07-30 15:12:51 +08:00
wyucandClaude Opus 5 cc95851dc5 chore(packages): publish storage and enforce version bumps (#998) 
* chore(packages): publish storage and enforce version bumps

* fix(packages): cover all publish inputs and tarballs

* fix(packages): scope version guard to release inputs

* fix(packages): fail closed on new package inputs

* docs(packages): clarify version guard boundaries

* fix(packages): harden release guard edge cases

* fix(packages): align smoke peers and git inputs

* docs(packages): make release smoke policy explicit

* fix(packages): anchor version guard at repository root

* fix(packages): report version guard setup errors

* fix(packages): harden release validation

* fix(ci): validate complete main pushes

* test(storage): pin the exported PostgreSQL schemas

DOCUMENT_PG_SCHEMA and RUNTIME_PG_SCHEMA are public API. A deployment that
provisions these tables with its own migration tooling has to reproduce the
DDL exactly for ensureDocumentSchema() / ensureSchema() to stay the intended
no-op against an already-provisioned database.

Nothing guarded that today: every statement is CREATE ... IF NOT EXISTS, so
PostgreSQL silently accepts whatever table already exists under the name. A
column type, a nullability, an index or a FK action can drift apart from a
downstream migration without an error, and the first symptom is a store query
failing in production or succeeding against the wrong types.

Pin both constants verbatim, and assert every statement stays IF NOT EXISTS
guarded so the ensure functions remain idempotent. The pin does not judge the
DDL; it makes changing it impossible to do by accident.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): gate publishing on the version check

The version check ran only in ci.yml. publish-packages.yml is a separate
workflow, so it could reach `pnpm publish` on a main push, a matching tag, or
a manual run with dry_run=false even when CI was red, skipped, or never ran.

Run the validation inside the publish job, before publishing, in a new release
mode that works for every trigger. A push range only exists for branch pushes,
so release mode judges each package against the registry instead: a version
that is not published yet is a release, and a version that is already
published is accepted only when the package source has not moved since that
release. `pnpm publish` skips an already-published version, so without that
second half a drifted package releases as a silent no-op.

The anchor for "since that release" is the @openmaic/<name>@<version> tag the
job now writes after every successful publish, falling back to the push range
for packages released before those tags existed. With neither anchor the
release stops instead of guessing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): make the release gate prove only what it can

Three independent reviews of the previous revision converged on one root
cause: it used git tags and a push range as the record of which tree produced
a published version. Neither is trustworthy. Tags are mutable and can be
created or moved by hand, so "the tag resolves" was being read as "the tag is
authentic"; pnpm records no gitHead; and a push range describes one push, not
the origin of a release. Packages released before the scheme existed had no
anchor at all, which made the tag and manual triggers fail closed from the
first run with no way to self-heal, dry runs included.

Drop the anchor machinery. Drift is prevented where it is provable, in diff
mode at merge time, which sees every commit before it can be released. The
pre-publish gate is now limited to claims a release can establish: every
version it publishes is new and moves forward, an already-published version is
reported and left alone, and any registry answer that is not a definitive
404 stops the release.

Around it, close what the reviews found in the workflow:

- real publishes only from a commit contained in main, so a tag or manual run
  cannot ship an unreviewed ref while holding NPM_TOKEN
- one repository-wide concurrency group, so a main push and a tag push cannot
  each decide the same version is unpublished
- publish and mark each package individually, so a partial failure stays
  retryable instead of leaving published packages unrecorded
- fail if the build rewrites tracked package files, which would otherwise
  publish content that is not in the released commit
- compare against every published version when ordering, and refuse to guess
  when the registry holds versions this check cannot order

The schema pin also asserted the constants without asserting what the ensure
functions execute, so those could diverge. Both are now run against a
recording queryable and their exact statement sequence is asserted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): refuse to release from a tree behind the registry

Simplifying the gate introduced a hole that a mutation check caught: a package
whose local version is already published took the "already published, skip"
path before any ordering check, so a tree rolled back to an older published
version passed silently. That is not a harmless no-op, because pnpm publish
rewrites each workspace dependency to the version in the tree, and a sibling
released alongside it would be published declaring a dependency on the older
package.

Order every package against the registry first, then decide: at the highest
published version it is a legitimate skip, below it the release stops.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): couple publishing to a green CI run for the same commit

Second review round, on the reworked design. The headline defect was a hard
blocker: `git fetch --depth=0` is not valid git (that is an actions/checkout
convention), so under `set -e` every real publish aborted before the gate ever
ran. Remove the depth argument; the checkout already has full history.

The rest closes the gap both reviewers reached independently, that being on
main is not the same as having passed the gate:

- wait for this exact commit's CI conclusion and require success. ci.yml runs
  concurrently with this workflow on a push to main and blocks nothing, so an
  unpublished version whose source drifted could be released while its own
  version check was still running or already red.
- require the commit on main's FIRST-PARENT history. Plain reachability also
  accepts every intermediate commit of every branch merged with a merge commit.
  A pull request that adds a bad tree and reverts it in the next commit leaves
  that tree an ancestor of main forever, and a tag pointing at it published it.
- do not cancel CI runs on main. Each run validates only its own push range, so
  cancelling one drops the range that contained the change, and its replacement
  compares against the cancelled tip and sees nothing.
- run the version check on every main push. A branch creation or force push
  reports an unusable `before`, which used to skip the gate entirely; fall back
  to the first parent instead.
- publish with --ignore-scripts. Every package's prepublishOnly reruns the
  build, deleting and regenerating the dist that was just verified and
  smoke-tested, so npm could receive bytes nothing had checked.
- run storage's PostgreSQL contract suites before publishing it. They skip
  themselves without a database, so the one backend that needs a real
  PostgreSQL was shipping unexercised.
- order registry versions with real semver, prereleases included. Rejecting
  every non-x.y.z version meant one historical prerelease anywhere in one
  package's history would refuse every future release of every package.
- warn when a release marker exists but points at another commit.

Note what is NOT enforceable here: a workflow_dispatch or tag run executes the
workflow definition from the selected ref, so a branch that edits this file can
delete these checks. The boundary has to be a GitHub Environment holding
NPM_TOKEN behind a main-only deployment branch rule. The job now declares that
environment and the header states the required setup; the in-file checks are
defence in depth.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): stop the first-parent check failing on SIGPIPE

Third review round. The blocker this time: `git rev-list --first-parent | grep
-qx` under `set -o pipefail`. grep exits at the first match, rev-list then dies
of SIGPIPE, and the pipeline returns 141, so a commit that IS on main is read
as "not on main" and every real publish is refused. It does not reproduce in a
shallow clone, where main's history fits in the pipe buffer; with the full
history this job checks out it always will. Count matches instead, which reads
the whole stream.

Also from this round:

- identify the CI run by workflow file and triggering event rather than by a
  check-run display name. Names are not unique, so any other workflow or app
  publishing a check called "Lint, Typecheck & Unit Tests" could stand in for a
  red version gate.
- fail the main-push version check when the push range is unusable instead of
  substituting HEAD^. A force push can replace many commits at once, so the
  previous commit is not the range that needs checking, and a green run here is
  what publishing depends on.
- correct the stated limitation. It claimed the package-directory input model
  was exact for dsl and storage; it is not. Their dist is whatever the
  lockfile's TypeScript emits, and dsl's shipped JSON schema comes from the
  lockfile's ts-json-schema-generator, so a toolchain bump can change any of the
  four tarballs with no diff under the package directory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(packages): make a merged version bump the only release input

The tag trigger was incoherent. Its own documentation admitted that any
@openmaic/* tag republishes the whole family and that which packages actually
go out is decided by the manifests, so the tag name never participated in the
decision. It was a second entry point to one decision function, it bypassed
review entirely because a tag is not a diff and can be created by anyone with
write access on any commit, and it cannot be guarded: environment deployment
rules match GITHUB_REF, so refs/tags/* and a main-only rule are mutually
exclusive.

Drop it. A version bump that landed on main is now the only release input, and
`@openmaic/<name>@<version>` tags are an output written after a package
reaches the registry. Nothing is lost: a manual dispatch from main still
republishes without a new commit, and that path is inside the protected
environment.

Split the workflow so the token has a boundary. `validate` holds everything
that does not need NPM_TOKEN and runs from any ref, which keeps dry runs
useful to contributors. `publish` is the only job declaring the `release`
environment, so the token is never attached to a run that is merely
validating. It rebuilds rather than sharing state, which is the right trade
for a release.

Also add the `actions: read` permission the CI-conclusion query needs; it was
switched from the checks API to the Actions API without updating the scope,
which would have failed with 403 on every real publish.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(contributing): explain when a package version must be bumped

CONTRIBUTING said nothing about versioning or releases, while this branch adds
a CI check that fails a contributor's PR with "publishable package inputs
changed but version did not increase". A gate that rejects work without
telling anyone the rule is the kind that ends up disabled.

State the rule, the exact failure message, which files are exempt, and how to
choose the number, with a specific warning for @openmaic/dsl: it is the
contract the other packages validate against, so narrowing what an existing
document may contain is breaking even when the diff is small.

Also state what contributors do NOT do: publishing is automatic once a bump
lands on main, and the release tag is a marker written afterwards rather than
something to push.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): run renderer's tests before publishing renderer

The pre-publish gate ran renderer's typecheck but never its test suite, so the
one package whose behaviour is hardest to typecheck was the one shipping
unexercised. It has 25 test files and 242 tests; they just were not in the
filter list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): keep the git credential away from package code

Review finding: the publish job held `contents: write` while
actions/checkout had persisted that credential, and it then ran
`pnpm install` and the package builds. Any dependency or build script
executed with a credentialed git remote in reach and a write scope to use
it, which is a wider blast radius than publishing needs.

Separate the two capabilities so no job holds both:

- `validate` and `publish` run package code. Both now check out with
  `persist-credentials: false`, `validate` is pinned to `contents: read`,
  and `publish` drops write scope entirely, keeping only the `actions: read`
  it needs for the CI conclusion and `id-token: write` for provenance.
- a new `mark` job is the only holder of `contents: write` and a git
  credential, and it installs nothing and builds nothing.

`mark` also reconciles instead of only recording this run. A marker whose
push failed previously could never be repaired before, because the release
plan excludes versions already on the registry and the marker logic sat
behind that plan. It now marks any registry version whose tag is missing,
and leaves an existing tag alone rather than moving it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-29 23:54:23 -04:00
xuyuanwei678 f4c1b46642 fix(importer): handle custom shapes without paths (#1015) 2026-07-30 10:53:45 +08:00
sophietao20-star 0cf2a33041 docs: refresh and synchronize documentation (#996) 
Refresh the Chinese documentation against current functionality, synchronize all localized guides, document deployment and configuration behavior, and align the PPTX import feature flag implementation.
 2026-07-27 22:45:15 -04:00
xuyuanwei678andwyuc ff95e6683d feat(renderer): add optional playback canvas (#958) 
* feat(renderer): add optional playback canvas

* style(renderer): format playback tests

* fix(renderer): center text by default

* fix(renderer): address playback review feedback

* docs(renderer): align playback canvas API docs

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-07-27 04:15:07 -04:00
杨慎andClaude Opus 4.8 6d29241bb0 feat(video-export): fidelity polish — spotlight geometry, video clips, formulas, subtitles (#952) 
* fix(video-export): bridge play_video element id → media ref

Generated-media records are keyed by the element's media ref (gen_vid_…),
but a play_video action targets the slide element by its .id. The app-side
compiler deps looked media/durations up by the raw element id, so every
generated video missed → present:false → no <video> emitted, no bytes
collected → the clip vanished from the export (only its static first frame,
baked into the base PNG, remained).

Build an elementId → mediaRef bridge across all slide scenes in
createVideoTimelineDeps (mirroring the live engine's resolveMediaPlaceholderId)
and resolve both media() and videoDurationMs() through it.

Refs #867.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(video-export): re-measure KaTeX fit-scale after fonts settle

KaTeX sizes large delimiters (\left\{, \begin{cases}) from its KaTeX_Size
faces, which load asynchronously. KatexContent measured its shrink-to-fit
scale once on mount, against the fallback font, and never recomputed — so a
cold slide snapshot baked a stale scale and the big brace desynced from the
piecewise body ("大括号后面的分段函数与大括号错位").

- Re-run the fit-scale measurement on document.fonts.ready and on each
  `loadingdone`, keeping the shrink-only (cap-at-1) behavior.
- slideToPng waits two extra frames before capture so the font-triggered
  React re-measure commits into the DOM html2canvas reads.

Refs #867.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(video-export): place spotlight/laser/video on the rendered content box

The pure geometry helper resolves an element's authored outer box against a
fixed 1000×562.5 base, but the live overlays — and the exported frame PNG —
measure the element's `.element-content` box, which for horizontal text is
auto-height plus 10px content padding. So a spotlight/laser sat offset from
where the text actually renders ("spotlight 与对应元素的位置有偏差", #867 item 5).

Add an optional GeometryProbe to the compiler's DI boundary: when supplied, the
geometry pass prefers each element's measured content-box geometry (video keeps
its rotation from the authored element), degrading to the pure authored-box calc
on a miss so the compiler stays deterministic in unit contexts.

App side: measureSlideElementGeometry (new, in @openmaic/renderer/snapshot)
mounts the slide off-screen and reads the same `.element-content`-against-
container box the live SpotlightOverlay uses; createVideoTimelineDeps
pre-measures every spotlight/laser/play_video target per scene and serves the
probe as a synchronous table lookup.

Tests: geometry.test.ts quantifies the outer-box vs content-box delta and proves
probe-wins / fallback / rotation-preservation; timeline-deps.test.ts covers the
pre-measure wiring.

Refs #867.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(video-export): decode a first-frame poster for posterless videos

A play_video clip only shows during its play window; the rest of the time the
static base-frame PNG backs that spot. But html2canvas can't draw a <video>, so
the snapshot swaps it for its poster <img> — and generated videos usually carry
no poster, leaving the video area blank whenever the clip isn't playing.

When a video record has no stored poster bytes, decode the video blob's first
frame (seek slightly off 0, draw to a canvas) and set element.poster before the
base snapshot, so that spot shows the first frame — the "paused on frame one"
look — instead of blank. Falls back to no poster on decode failure / timeout /
CORS taint, never failing the export.

Refs #867.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(video-export): split subtitles, make burn-in optional, centered modal

Burned-in captions used one cue per whole narration paragraph, so a long cue
became a 4–6 line block covering the lower half of the slide, and there was no
way to turn it off.

- Split cues (pure, compiler layer): lib/video-export/split-cue.ts splits each
  cue on sentence → clause → hard-wrap to ≤~40 CJK-equiv units, distributing the
  parent window by character weight (last piece pinned to the parent end, sub-1.2s
  slivers merged). Wired into the timeline pass, so the burned-in overlay and the
  SRT/VTT sidecar share one split track and can't drift.
- Burn-in optional (default off, #867 item 2): emitHyperframes gains
  burnInSubtitles; off → clean video + sidecar SRT/VTT only. Caption layout hard-
  clamped to 2 lines so an outlier can't grow tall again.
- Download subtitles without rendering: compileSubtitles() runs only the compiler
  (no assets/zip/service); new useDownloadSubtitles saves an .srt.
- Centered modal: video-export-dialog.tsx replaces the cramped dropdown section,
  grouping Output / Subtitles / Actions / Progress with a stable lifecycle for
  multi-minute renders. header-controls opens it from an "Export Video" entry;
  old video-export-menu.tsx removed. i18n keys added across all 8 locales.

Refs #867.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style: prettier format video-export fidelity changes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: drop unused playVideo helper in timeline-deps test

The media-bridge tests assert via deps.assets.media(...) directly, so the
playVideo helper (and its PlayVideoAction import) were dead. Flagged by the
code-quality bot on #952.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(video-export): resolve cross-review findings + lower subtitle band

- emit-hyperframes: drop duplicate `display:-webkit-box` on subtitle cues so
  they start truly `display:none` (a second declaration overrode the `none`,
  showing every cue stacked at t=0 when burn-in was enabled).
- geometry: don't feed the measured content-box AABB to a rotated video clip —
  it already encloses the rotation, so re-applying `rotate` doubled it. Rotated
  elements fall back to the authored box (one, un-doubled source of truth).
- build-export-zip: extract shared `compileStageIr` so buildExportZip and
  compileSubtitles can't drift on the timing/assets/geometry wiring.
- split-cue: join merged CJK cue pieces without a spurious ASCII space.
- subtitles: lower the caption band (bottomRatio 0.055 → 0.01) and hoist the
  band's magic numbers into a named SUBTITLE constants block.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(video-export): stop the snapshot onclone override from reshaping KaTeX

The export snapshot injects a neutral-kerning / geometricPrecision style over
`.slide-renderer-prose *` to keep CJK prose+table glyphs from mis-advancing
under html2canvas-pro. KaTeX formulas (output:'html') also live inside
`.slide-renderer-prose`, so the override was reshaping their math glyph
advances — a burned-in formula rendered subtly differently from the live
canvas, which applies no such override.

Empirical check (Playwright + system Chrome, cases+fraction formula, native
paint vs html2canvas): the override raised the formula's mean per-pixel delta
vs the live paint from 120.1 to 139.2 (~16%) and moved ~75% of inked pixels.
Restoring native font-kerning/text-rendering for `.katex` subtrees brings the
formula's delta back to 120.1 (bit-identical to the no-override capture) while
prose and table text keep the override. The residual 120.1 is the deeper
html2canvas rasterization gap (vlist/frac-line/delimiter), a separate track.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(video-export): render slides via native paint, bundle KaTeX font embed

Root-fix the exported-vs-live formula distortion (and the filter/mask/video
gaps) by rasterizing slides with native Chrome paint instead of html2canvas's
reimplementation.

- slideToPng now captures via html-to-image (foreignObject → the same Chrome
  engine that paints the live classroom rasterizes the same DOM). KaTeX
  formulas, CSS filter, soft-edge masks, and mixed CJK/Latin text come out as
  the classroom shows them. html2canvas-pro stays as a fallback (with the
  filter/mask bakes + KaTeX text-rendering reset) for when native paint can't
  run — e.g. a cross-origin image taints the canvas.

- A foreignObject SVG can't reach the document font registry, so fonts must be
  inlined. KaTeX math faces are prepended from the bundled woff2
  (KATEX_FONT_EMBED_CSS, generated by scripts/generate-katex-fonts.mjs), NOT
  left to runtime getFontEmbedCSS — which reads cssRules and silently drops a
  cross-origin KaTeX stylesheet, collapsing a large brace to a fallback glyph
  (the observed "formula render failure" on e.g. a \begin{cases} with \text{}).
  A per-face guard verifies every KaTeX face the formula references is embedded,
  falling back to html2canvas otherwise.

Verified in a Playwright spike (native vs html2canvas vs html-to-image): formula
inked-delta 143→53; filter/mask/video match native; and with getFontEmbedCSS
forced empty, the bundled embed alone still renders the failing formula
correctly. Renderer package tests: 226 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(video-export): prettier-format the generated KaTeX font-embed module

The generator emitted an unformatted TS file (double quotes, long unwrapped CSS
literal) that failed CI `prettier --check`. Run the generated output through the
repo's Prettier config before writing, so every rebuild produces CI-clean,
stable output. Regenerated the committed file to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(lint): rename reserved `module` var in katex-fonts generator; ignore .scratch

- generate-katex-fonts.mjs assigned to a variable named `module`, tripping
  @next/next/no-assign-module-variable (the one hard eslint error). Renamed to
  `fileContents`; generated output is byte-identical (deterministic).
- Add `.scratch/**` to the eslint globalIgnores (alongside .claude/.worktrees)
  so local throwaway verification scaffolding doesn't pollute `pnpm lint`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(video-export): address PR #952 review — scene-scoped media bridge, safer cue splitting

Resolves the two blocking correctness issues and three follow-ups from the
cross-review on #952.

- Scope the elementId→mediaRef bridge by scene (was deck-wide/last-writer-wins):
  slide-local ids like `video_001` now resolve within their own scene. `assets.media`
  uses its scene arg; `timing.videoDurationMs` resolves via action object identity.
  Adds a cross-scene collision test.
- Stop treating every ASCII period as a sentence boundary: a lone `.` splits only
  when not between word chars (3.14, v1.2) and not an abbreviation dot (e.g.),
  collapsing repeated-punctuation runs. Adds decimal/version/abbrev/ellipsis tests.
- Share the module-level in-flight guard across ZIP export and subtitle download
  (extracted to export-in-flight.ts); dialog `busy` folds in `downloading`.
- Skip the off-screen geometry render for the subtitles-only path (skipGeometry);
  duration probes are kept so sidecar cue timing stays in sync with the video.
- Align measure.ts font settling with slideToPng: force-load each used face before
  fonts.ready so measured content-boxes don't bake fallback advance widths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(video-export): prettier-format split-cue test

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(video-export): enable beginFrame capture via PRODUCER_BROWSER_GPU_MODE=hardware

The producer defaults to `software` GPU mode, which silently force-enables the
CPU-bound `Page.captureScreenshot` fallback: SwiftShader pins every core on
rasterization and caps frame capture at ~10 fps. Selecting `hardware` keeps
Chromium's `HeadlessExperimental.beginFrame` capture active instead.

Measured on a 900-frame (30s) clip, no real GPU on the host:
- capture ~10 -> ~19 fps, total 100s -> 62s (~38% faster)
- CPU peak ~810% -> ~98%; peak mem 2.2 GiB -> 0.9 GiB
- static frames pixel-identical; animation frames differ only in sub-pixel
  edge antialiasing (same-path re-render is deterministic/inf PSNR)

No real GPU is required — the win is the faster capture API, not hardware
rasterization; beginFrame falls back gracefully if the host lacks a GPU. Also
eases the RENDER_JOB_DEADLINE_MS timeout risk on long videos.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(video-export): sentence splitting via Intl.Segmenter + abbreviation merge

Addresses the PR #952 follow-up: separate sentence-boundary detection from cue
shaping and stop growing a custom punctuation heuristic.

- splitSentences now delegates boundary detection to Intl.Segmenter (locale-
  agnostic ICU), the multilingual/CJK baseline. It already preserves 3.14, v1.2,
  e.g., U.S., ellipses, and CJK 。！？… without any hand-rolled period scan.
- ICU still ends a sentence after a titlecase abbreviation before a capitalized
  word (Dr. Smith, Fig. 3) — the edge case the reviewer flagged. A small
  abbreviation-aware post-merge re-joins those, driven by an ABBREVIATIONS set.
- Semicolons are no longer sentence ends (they join clauses); moved ；; into the
  clause-level SECONDARY splitter so over-budget sentences still break there.
- Cue shaping (budget / clause / hard-wrap / weighted timing) is unchanged, so
  detection and shaping are now distinct stages.

Not using `sbd` as suggested: lib/video-export/*.ts sits behind the machine-
enforced eslint purity boundary (imports limited to @openmaic/dsl, zod,
../choreography, siblings), so a new npm dep is rejected. Intl.Segmenter is a
pure Node >=20.9 builtin, matching the reviewer's "small abbreviation-aware
post-merge layer" alternative.

Tests: add Dr. Smith / Fig. 3 / U.S. coverage; the rhetorical ellipsis now stays
one under-budget cue (was split); add a semicolon over-budget case. 22/22 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(video-export): prettier-format split-cue sentence refactor

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* i18n(es-MX): add 7 subtitle/video export keys to align with en-US

main added the Spanish (Mexico) locale (#942) after this branch's subtitle
keys landed, so es-MX was the only locale missing export.subtitles* and
export.videoBurnSubtitles* — which failed the i18n key-alignment CI check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(video-export): restrict title post-merge so it never deletes real boundaries

Addresses the blocking re-review on the Intl.Segmenter change: the abbreviation
post-merge unconditionally rejoined the next ICU segment whenever the previous
one ended in a listed abbreviation, which erased genuine sentence boundaries —
`It was made by Acme Inc. Next point.` collapsed into one cue, and even an
explicit newline (`Acme Inc.\nNext topic`) was rejoined.

The merge now fires only for the case ICU actually mis-splits — a title
(Mr/Mrs/Ms/Dr/Prof/Sr/Jr) before a capitalized proper noun — and only when:
  - the previous segment does not end at a newline (hard boundary), and
  - the next segment starts with a capital letter.

Sentence-trailing abbreviations (Inc./Ltd./etc./vs./No.) are dropped from the
set entirely: they legitimately end sentences, and ICU already keeps a title
before a number (No. 5, Fig. 3) or a lowercase continuation (etc. and…) whole
without any merge, so nothing there needs rejoining.

Regression tests: Inc./etc. boundary preserved, newline-after-title preserved,
lowercase continuation kept whole, plus the retained Dr. Smith / Fig. 3 / U.S.
cases. 26 split-cue tests pass; full video-export suite 123 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-27 02:58:04 -04:00
wyucClaude Fable 5杨慎
 		b4e3145c90 feat(roster): single-source the generated agent roster on the stage document (#994) 
* feat(roster): single-source the generated agent roster on the stage document

The generated agent roster used to live in two places at once: the stage
document carried generatedAgentConfigs, while a per-stage IndexedDB mirror
(db.generatedAgents) was the actual read authority — and the only home of
each agent's voice binding. The split caused real losses: voices vanished on
device switches, exports, and shares (the mirror never travels), the load
path trusted a device-local table over the portable document, and roster
edits persisted through a bespoke debounce with no beforeunload flush.

Make the stage document the single source of truth:

- Contract: add optional voiceConfig/voiceDesign to GeneratedAgentConfig in
  @openmaic/dsl (VoiceDesign moves into the contract; the app re-exports it).
  Additive optional fields — not a breaking serialized-shape change, so
  DSL_VERSION stays put per the version policy.
- Writes: remove every mirror writer (registry saveGeneratedAgents, the
  stage store's debouncedSaveAgents bypass, classroom-load hydration, import
  direct writes). setStageAgents now only marks the stage dirty for the
  shared persistence scheduler plus synchronous in-memory registry/selection
  mirrors; generation and import embed the roster on the stage.
- Reads: classroom load hydrates the registry from the loaded stage's
  generatedAgentConfigs; export reads the stage roster directly.
- Lazy migration: the mirror is retained read-only. On load, a roster (or
  voice fields) missing from the document is backfilled from the mirror,
  committed onto the in-memory stage, and persisted by the next flush —
  idempotent, and gated on the current stage id after every await so a
  classroom switch cannot leak a stale roster into the registry.
- Deletion: deleteStageData discards the deleted stage's pending
  persistence work so a queued flush cannot resurrect the document.
- Round trip: classroom ZIP manifests now carry the voice fields, so
  export/import preserves agent voices.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(roster): harden deletion, migration, selection, and ingress paths

Review hardening on top of the roster single-source change:

- Deletion vs in-flight persistence: deleting a stage now registers a
  session-scoped tombstone (lib/utils/deleted-stages.ts) before the cascade
  starts. Every persistence landing point checks it — the scheduled flush and
  the departing-stage retry drop their snapshots, and the aggregate/incremental
  saves re-check under the document lock, including the incremental path's
  full-save fallback that would otherwise rebuild a deleted document from the
  in-memory snapshot. A deletion that fails before the document is removed
  lifts the tombstone so the still-existing stage keeps persisting.
- Lazy migration termination: a fruitless legacy-mirror probe (nothing to
  merge — e.g. server-generated rosters, which are voiceless by design) is
  remembered per stage for the session instead of re-querying the mirror on
  every load forever. Successful merges are not memoized so a failed flush is
  retried on the next load. Docstrings now state the real lifecycle.
- Selection provenance: setStageAgents no longer overwrites a user-set agent
  selection with the full roster. Stage-derived selections still track the
  roster; a user-set auto selection is only narrowed to surviving agents
  (newcomers are not auto-selected, the user-set flag is untouched), and a
  user-set preset selection is left alone — matching restoreAgentSelection.
- Mirror hygiene: deleteStageData clears the deleted stage's rows from the
  legacy roster mirror (best-effort), and the mirror table's docs now describe
  its actual access pattern (migration reads + deletion hygiene).
- Registry persistence: the agent registry's persist snapshot partializes
  generated agents out of localStorage, making the stage document the roster's
  only durable home; the rehydration merge filter remains as defense in depth.
- Ingress validation: imported manifest voice fields are structurally
  validated (malformed bindings dropped per field, the agent survives), and
  applyGeneratedAgentsToRegistry validates providerId against the known TTS
  provider registry instead of casting, treating unknown providers as "no
  bound voice".
- Contract: drop the speculative AgentVoiceConfig.modelId — an audit of every
  roster producer, current and historical, found none that ever emitted it.
  Contract fields are added once a producer exists, not before.

New tests cover the deletion tombstone at both the store and storage layers
(including the missing-destination full-save pin and the lock-race re-check),
the fruitless-probe memo, the selection-provenance matrix, persistence
exclusion of generated agents, and voice-field sanitization on import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(roster): close tombstone lifecycle, probe-memo, and selection-intent gaps

Second hardening pass over the deletion/migration/selection fixes:

- Tombstone lifecycle: a deleted classroom id can legitimately come back
  (deletion is client-side only). Explicit (re)creation points now lift the
  tombstone — server-copy restore in applyClassroomStageAndScenes and backup
  restore in importDatabase — so edits after a same-session restore persist
  again, while in-flight flushes stay fenced.
- Failed deletion now restores the dirt it discarded: deleteStageData
  snapshots the pending map plus the in-flight round before tombstoning, and
  re-marks it when the delete fails with the document still present.
- Legacy-mirror probe memo now distinguishes a FAILED read (null, retried on
  the next load) from a confirmed-empty mirror (memoized), so a transient
  IndexedDB error cannot suppress migration for the whole session.
- Incremental save tail (currentScene KV row + chat sessions) is tombstone-
  fenced like the aggregate path, and both paths re-check the tombstone
  immediately before every write, with comments scoped honestly for the
  lock-free (no Web Locks) best-effort LWW fallback.
- Roster-edit selection mirror: an empty intersection now falls back to the
  full roster and clears the user-set flag (matching restoreAgentSelection's
  length > 0 gate), and a user-set selection that equals the pre-edit full
  roster keeps tracking the roster wholesale so newcomers are not excluded
  forever by the AgentBar auto-toggle snapshot.
- TTS/ASR provider lookups use Object.hasOwn instead of `in`/bare indexing,
  so prototype-chain keys ('toString', 'constructor', ...) no longer pass the
  provider whitelist or resolve to Object.prototype members.
- Documented the per-tab limit of the tombstone set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(roster): replace the boolean deletion tombstone with a deletion-generation model

A boolean tombstone cannot express the deletion lifecycle once a deleted id
legitimately comes back (server-copy restore on revisit, backup import): after
the restore lifts the flag, a pre-delete flush still in flight is
indistinguishable from a post-restore edit and can overwrite the restored
document with pre-delete content. Two rounds of point fixes kept leaking
variants of this race, so this restructures the fence instead.

deleted-stages.ts now keeps a per-stage monotonic deletion epoch plus a
deleted flag. markStageDeleted bumps the epoch; unmarkStageDeleted clears
only the flag and never rewinds the epoch. Every persistence path captures
the epoch at the moment it captures the data it will write (flush-round
snapshot, departing-stage snapshot, aggregate save entry) and re-checks
"captured epoch is current AND not deleted" immediately before each
individual write. Invariants:

- A write captured before a deletion can never land after it, even when a
  same-id restore has lifted the deleted flag (the captured epoch is
  permanently stale).
- A write capturing data after a restore observes the current epoch and
  persists normally.
- A failed deletion lifts only the flag; the restored pending changes are
  re-queued as descriptors, so their flush re-captures current store state
  under the current epoch and is neither dropped nor a stale replay.

Also fixed in the same restructuring:

- Incremental tail writes (currentScene, chats) each re-check independently;
  a delete landing while the first tail write is awaiting now fences the
  chat write instead of riding the earlier check.
- Deleting a classroom evicts it from the warm in-memory store, and
  loadFromStorage treats a warm-but-deleted stage as not loaded (discarding
  the ghost), so navigating Back to a deleted classroom reaches the
  server-restore path instead of rendering an editable ghost whose every
  edit is silently dropped.
- importDatabase records each document's pre-import deletion state and
  reinstates it when a failed import rolls the document back, so an
  outstanding flush cannot recreate the rolled-back document.
- Docstrings on the failed-deletion restore path now state that only dirt
  captured before the deletion is restored; edits refused during the
  deletion window are not part of the snapshot.

New tests cover the epoch invariants at the storage layer (pre-delete rounds
dropped across restores on the incremental, aggregate, and tail paths;
post-restore captures landing; strictly increasing epochs), the scheduler
layer (departing-retry delete+restore straddle; restored dirt flushing under
the current epoch), the warm-ghost restore driven through the real
loadFromStorage, and the import lift/rollback pair.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(deletion): report fenced drops, gate ghost eviction on cascade settlement

Edge hardening around the deletion-epoch fence; the epoch model itself is
unchanged.

- An epoch-stale drop is now reported, not disguised: saveStageData /
  saveStageDataIncremental return a distinct 'stale-dropped' status (entry
  check, every in-mutation re-check, and both tail-write re-checks) instead
  of a shape identical to success. saveToStorage reacts by skipping all
  success bookkeeping - no chatSnapshot rebind to a snapshot that never
  landed, no pending-dirt clearing - and returns false, keeping its
  "true means verified write" contract honest. persistDirtySnapshot keeps
  its existing (correct) semantics explicitly: a drop has nothing to retry.
- Warm-ghost eviction is settlement-gated: the deletion cascade records an
  in-flight bit (begun by deleteStageData, settled in its finally), and the
  deleted-warm branch of loadFromStorage keeps the warm state while the
  cascade is unsettled - a delete can still fail before removing the
  document, in which case the warm state plus the restored pending dirt is
  the only copy of the pre-delete edits. Once settled, the ghost discard
  and server-restore path behave as before.
- Read-side landing re-check: loadFromStorage re-checks isStageDeleted
  after hydration, immediately before set(), mirroring the write-side
  "re-check immediately before landing" discipline - a lock-free
  mid-cascade read can no longer re-materialize a ghost classroom.
- clearStoreForDeletedStage re-checks isStageDeleted at eviction time, so a
  same-id restore completing inside the cascade-tail window is never wiped.
- capturedEpoch is now a required parameter on both storage entry points:
  the default (call-time capture) silently reopened the capture-point/
  validation-point split for future callers; the type system now enforces
  the pairing. All production callers already passed it explicitly.

New tests cover the dropped-status contract, the cascade in-flight
lifecycle (including both failure shapes), keep-warm during an unsettled
delete with end-to-end pending restore after a failed delete, settlement-
then-eviction on success, the eviction guard against a tail-window restore,
and the read-side hydration re-check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(deletion): settle the deleted-warm load by awaiting the cascade outcome

Hardening around the deletion settlement gate introduced previously; the
epoch model is unchanged.

- The deleted-warm branch of loadFromStorage no longer completes the load
  while the deletion cascade is undecided (which exposed a classroom whose
  edits were refused without being restorable, and left a successful
  deletion's eviction with no follow-up reload). It now parks on a per-stage
  settlement promise (stageDeletionSettled) and branches on the outcome:
  a failed delete keeps the warm, restored classroom; a successful delete
  falls through to a full cold load so the server-restore path recovers the
  route; a navigation that moved on during the park is stopped by the load
  token guard. The eviction (clearStoreForDeletedStage) deliberately stops
  claiming a new load token so the parked load stays current - ghost
  re-materialization is fenced by the read-side deleted re-checks instead.
  No deadlock: a parked load holds no document lock, and the cascade never
  waits on a load.
- 'stale-dropped' now propagates through the debounced flush path:
  persistDirtySnapshot returns the sentinel instead of an empty failure set,
  and startFlushRound skips the chatSnapshot rebind on it (mirroring
  saveToStorage) - a falsely rebound baseline would let later chat saves
  no-op-skip chats that never landed and corrupt cross-tab conflict
  detection. Pending-map clearing keeps its empty-failure semantics
  (restored descriptors mint fresh revisions).
- The cascade in-flight state is a counter (begin++/settle--) instead of a
  boolean, and deleteStageData is single-flight per stage (a concurrent
  second call joins the first cascade), so overlapping deletions can never
  expose an undecided cascade as settled.
- The StaleDroppedSave docstring no longer overpromises: a tail
  saveCurrentScene can land after the cascade's clearCurrentScene; the
  orphaned cursor row is ignored by the load path rather than removed by
  the cascade.

Net +8 tests: overlapping-cascade counting, settlement-promise resolution
(success and failure), concurrent double-delete single-flight join, the
parked mid-cascade Back in both outcomes plus the navigation-away guard,
and the fenced flush round keeping an honest chat baseline end-to-end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(deletion): re-verify store identity on keep-warm resume, re-run joined deletes after a mid-cascade restore

Three hardening refinements on the deletion/persistence fencing:

- loadFromStorage's parked keep-warm branch now mirrors the success
  path's store-identity guard: if a tokenless writer (e.g. a database
  import) replaced the stage during the park, a failed deletion falls
  through to the cold load instead of reporting another classroom as
  the kept warm state. The ghost-discard block is correspondingly
  gated on the deleted flag so the fall-through does not mislabel a
  surviving document as a ghost.

- deleteStageData: a joined call carries its own deletion intent. When
  the first cascade fulfills but a same-id restore lifted the deleted
  flag in the meantime, the joined caller now runs one fresh cascade
  against the restored document instead of reporting the pre-restore
  outcome. Exactly one re-check per call (no recursion); a rejected
  first cascade still propagates unchanged to every joined caller.

- Narrowed the failed-delete restore-ownership comments: the restore
  covers only the pending map plus an in-flight flush round's dirt;
  a departing-stage snapshot is outside that capture and is fenced
  and dropped by design. Comment-only, no behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(roster): gate deleted-warm handling on stage identity, keep an explicit empty roster authoritative

Final-audit fixes, three items:

- loadFromStorage keyed its deleted-warm settlement handling (park /
  keep-warm / ghost discard) behind the scenes.length > 0 shortcut, so a
  zero-scene warm ghost of a deleted classroom skipped it entirely: the
  cold load found no document, the ghost stayed in the store, and the
  classroom loader's server-fallback gate (!getCurrentStage()) never ran
  — the silently-uneditable trap this branch exists to close, in its
  scene-less form. Deletion handling now keys on stage identity alone,
  before any scene-count shortcut; the non-deleted warm skip still
  requires scenes.

- rosterNeedsLegacyFallback collapsed "roster field absent" (document
  predates roster persistence) with "explicitly persisted empty roster".
  A future writer persisting [] on a device whose read-only mirror still
  held stale rows would have its emptied roster resurrected on every
  load. The full-lift branch now triggers only when the field is absent
  (undefined); an explicit [] is authoritative and never probes the
  mirror. No current writer produces [], but the read side no longer
  depends on that invariant.

- The GeneratedAgentConfig docstring claimed the voice fields are not a
  breaking change; that only holds for this codebase's tolerant
  structural validators. The generated stage.schema.json sets
  additionalProperties: false, so consumers pinning an older published
  schema artifact reject documents carrying the new fields. The
  docstring now states both sides honestly.

Tests: a zero-scene deleted warm ghost is restored through the real
loadFromStorage (fails pre-fix), an explicit empty roster does not
resurrect stale mirror rows (fails pre-fix), and the
rosterNeedsLegacyFallback unit matrix covers undefined vs [].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(deletion): re-queue aggregate persistence when a failed deletion is recovered

Direct aggregate saves (saveToStorage: generation completion, server-restore
hydration) run outside the pending map and the flush round, so the deletion
path's pre-delete dirt snapshot cannot describe them. When a deletion bumped
the epoch while such a save was in flight, the save was correctly fenced
('stale-dropped') — but if the deletion then failed before removing the
document, the recovery only re-queued the snapshotted scheduler descriptors.
The fenced aggregate-only content stayed memory-only with no retry anywhere,
and a later reload lost it.

The failed-deletion restore now merges a full re-mark of the aggregate
(structure / stage / outline / currentScene / chats plus a descriptor for
every current scene) into the recovery set, still guarded on the store
holding the stage. Because the flush recaptures the CURRENT store state under
the CURRENT epoch, this necessarily carries whatever the fenced aggregate
save held — and, for the same reason, edits refused by the scheduler during
the deletion window, which previously stayed memory-only as well; the scope
docstrings now say so instead of declaring them excluded.

New integration coverage drives the real store scheduler and the real storage
layer end to end: a genuinely fenced in-flight saveToStorage (generation
completion) plus a pre-removal deletion failure ends with the completion flag
durable on the next flush; a deletion-window scene edit survives the same
way; and a successful deletion re-marks nothing (control). Both recovery
tests fail against the previous restore logic.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-07-27 14:39:19 +08:00
wyucand杨慎 16dad613c9 fix(storage): bind default fetch to globalThis in both HTTP store clients (#984) 
* fix(storage): bind default fetch to globalThis in both HTTP store clients

Browsers require fetch to be invoked with this === globalThis; storing the
global as an instance field and calling this.fetchImpl(...) throws
'Illegal invocation' on the very first request — while node's undici does
not enforce the receiver, which is exactly why every node-side contract
suite stayed green. Found by a downstream end-to-end test: the persistence
bootstrap configured the HTTP stores correctly, but the first page load
crashed on fetch and nothing ever reached PostgreSQL. A strict-receiver
regression test now guards the gap node cannot catch natively.

* fix(storage): validate the fetch option before binding it (review P2)

.bind on a non-function throws a native TypeError that preempted the
documented 'requires a fetch implementation' error; both constructors now
validate the selected implementation first.

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-07-24 14:09:24 +08:00
wyucandCodex e39c64cd9e feat(persistence): one-command server-backed stack (compose profile + embedded API + docs) (#982) 
* feat(persistence): one-command server-backed stack — embedded API + compose postgres profile

docker compose --profile server-persistence up --build runs exactly two
containers: the app (persistence HTTP server embedded at /api/persistence,
enabled only when DATABASE_URL is set — unset keeps today's browser-only
behavior byte-for-byte) and PostgreSQL 16. Client bootstrap configures both
storage seams against the same-origin route when NEXT_PUBLIC_PERSISTENCE=1,
riding the existing lazy migrations so prior browser data reaches the
server per-course on first open. Dev auth (bearer token + client-asserted
learner header) lives in one overridable module and is loudly documented
as development-grade. Verified end-to-end against the real stack:
document PUT/GET and runtime session create land in Postgres; bad tokens 401.

Co-authored-by: Codex <codex@openai.com>

* fix(persistence): review round — structural bootstrap ordering, retryable init, honest failure surfaces

The bootstrap side-effect import moves into the two storage seam entry
modules, so any module graph that can resolve a store necessarily
evaluates it first — no client-entry ordering luck, and the two configure
calls are all-or-nothing. A failed server init clears its cached promise
(next request retries instead of permanent 500s). listStages rethrows
backend failures and the dashboard surfaces a persistence-unavailable
error instead of rendering an empty list that reads as data loss. Docs
state the build-time nature of NEXT_PUBLIC_PERSISTENCE, the real token
boundary (any visitor can extract it and impersonate any learner —
localhost/trusted-network only), and the correct postgres password
rotation. Dev-token compare is timing-safe; the handler singleton lives
on globalThis so HMR cannot leak pools.

Co-authored-by: Codex <codex@openai.com>

* fix(persistence): review response — no-confidentiality wording + adapter round-trip coverage

The token docs now state plainly that the NEXT_PUBLIC token is compiled
into the public bundle and provides no confidentiality or user isolation
(its only role is keeping unrelated scanners off a trusted network). The
Fetch/Node adapter — the route's most bug-prone code — gains a round-trip
integration test: PUT body streaming, 201 with multi-value headers, 204
empty body, and encoded path segments reaching the handler un-decoded.

---------

Co-authored-by: Codex <codex@openai.com>
 2026-07-23 16:28:03 +08:00
wyucandCodex 5cef056d17 feat(storage): server-backed documents — HTTP contract, Postgres backend, one reference server for both stores (#979) 
* ci: whitelist the document-server-backend integration trunk

* feat(storage): DocumentStore HTTP contract + client + reference-server routes (1B PR1) (#975)

Mirrors the runtime line's server-backed delivery for the document store:
a documented /documents HTTP contract (validation/not-found/FUTURE_VERSION
taxonomy; authorization stays a deployment concern via injected hooks with
an any-authenticated default), an HttpDocumentStore that runs the same
validators and migrate-on-read semantics as the browser backend before and
after the wire, and document routes on the shared reference server — one
server, both stores. The backend-agnostic document contract suite now runs
against HTTP → reference server → browser store.

Co-authored-by: Codex <codex@openai.com>

* ci: whitelist the trunk in the storage PG contract workflow too

* feat(storage): PgDocumentStore — Postgres document backend (1B PR2) (#976)

Implements the full DocumentStore contract over document_stages /
document_scenes / document_outlines, reusing the pure adapter split so
every backend shares one aggregate semantics: saveDocument replaces the
aggregate in one transaction under the stage row's FOR UPDATE lock
(vanished scenes deleted, outline row added/removed — omission means
deletion), incremental putStage/putScene/deleteScene take the same
current-version gate as the browser backend, loads read FOR SHARE and
migrate without writing back, listings never touch content, and deletes
cascade by FK. The document contract suite is PG-gated the same way the
runtime one is; local coverage comes from stubbed-queryable unit tests.

Co-authored-by: Codex <codex@openai.com>

* feat(document-store): sealed bootstrap injection seam for the document backend (1B PR3) (#977)

configureDocumentStorage mirrors the runtime seam: module-level bootstrap
injection sealed after first resolution, a factory that receives the app
validators so a server-backed store cannot silently skip them, and
reset-for-tests wiring. A deployment swaps the browser document backend
for HttpDocumentStore without touching app code. Storage README roadmap
updated — the document HTTP/PG backend items are done.

Co-authored-by: Codex <codex@openai.com>

* fix(storage): 1B final-review hardening — error classification, JSON-safety, body limits, parity coverage (#978)

Unversioned legacy incremental writes classify as 400 instead of leaking a
500; document/scene reads assert JSON safety before serialization so a
structured-clone value fails loud (500 NOT_JSON_SAFE naming the path)
instead of silently converting; a shared streaming body reader bounds
request bodies (default 32 MiB, 413 PAYLOAD_TOO_LARGE, option exposed
through every handler factory); the putStage stale/future guards and
corrupt-stamp listing tolerance move into the shared contract suite so all
four backends prove them; both HTTP store errors retain structured details.

Co-authored-by: Codex <codex@openai.com>

* refactor(storage): typed document version/not-found errors replace prose classification (#980)

DocumentVersionError (future | not-current, storedVersion) and
DocumentNotFoundError now carry the classification the server previously
regex-derived from message prose — instanceof matching mirrors the runtime
sibling's RuntimeAppendConflictError pattern, so a wording change can no
longer silently break status mapping. The prose fallback remains, after
the typed checks, solely for third-party DocumentStore implementations.
The HTTP client rematerializes 409s as DocumentVersionError for symmetry,
and http.ts's private validator fields are renamed to match the public
option names all three backends already share.

Co-authored-by: Codex <codex@openai.com>

---------

Co-authored-by: Codex <codex@openai.com>
 2026-07-23 12:56:26 +08:00
wyucCodexClaude Fable 5
 		d0d227424b feat(document): complete the document-persistence cutover — stage/scene/outline onto DocumentStore (#965) 
* feat(runtime): persist quiz attempts in RuntimeStore

* fix(runtime): coalesce quiz draft snapshots

* fix(runtime): recover concurrent quiz attempts

* fix(runtime): handle quiz completion races

* fix(runtime): commit quiz review atomically

* fix(runtime): dedupe concurrent quiz writes

* fix(runtime): drop stale quiz drafts

* fix: harden quiz runtime review recovery

* fix: serialize quiz attempt identity

* feat: read quiz state from runtime store

* fix: persist quiz retries before resetting

* fix: preserve authoritative quiz outcomes

* fix: preserve legacy quiz retries during cutover

* fix: reconcile legacy quiz snapshots safely

* fix: drain rollover quiz write queues

* fix: drain completed quiz retry queues

* fix: reuse concurrent quiz retries

* fix(quiz): preserve drafts across abrupt reloads

* fix(quiz): recover empty retry sessions

* fix(chat): abort stalled runtime state reads

* fix(quiz): expose queued phases to readers

* fix(quiz): retain concurrent writer tails

* fix(quiz): canonicalize retry branches

* fix(quiz): keep retry rollovers monotonic

* fix(quiz): validate skipped retry siblings

* fix(quiz): close read cutover races

* test(classroom): cover legacy quiz summaries

* fix(quiz): reset async consumers on scene changes

* fix(quiz): close scene transition windows

* test(pbl): cover launch freshness guards

* fix(quiz): close cutover concurrency gaps

* fix(quiz): harden retry and context freshness

* fix(quiz): reject malformed legacy answers

* fix(quiz): validate legacy answer values

* test(quiz): cover legacy multi-answer migration

* fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover

Main's three call sites built static storeState blocks that runAgentLoopFn
never consumed (it always rebuilds fresh state via getStoreState); the quiz
read cutover replaced that callback with the async two-phase RuntimeStore
read, leaving the template field with zero consumers. Drop it.

* feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions

The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError
semantics existed only in the browser backend; server-backed deployments would
silently accept conflicting appends and leave completed sessions active. Forward
the options over the wire, detect conflicts atomically under the PG transaction,
map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error
client-side so quiz retry logic works across every backend.

Co-authored-by: Codex <codex@openai.com>

* fix(chat): Pi single requests build storeState via the async runtime quiz read

Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the
request template straight to /api/chat/pi, which rejects bodies without
storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read
with the scene-transition guard) and call it on the Pi path too.

* ci: whitelist the runtime-data-cutover integration trunk for PR checks

* feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956)

* feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869)

The fourth and last runtime family. Consumed-discussion facts become
append-only 'playback' records folded into a set at read (at-least-once
appends, no conflict machinery); the resume cursor is device-scoped
last-write-wins KV per the amended #779/#869 split. sessionStorage keeps
same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie
playbackState machinery is retired, with a one-time lazy migration of any
legacy row (cursor half + facts half) before deletion, and stage deletion
now clears both the KV cursor and any unmigrated legacy row.

Co-authored-by: Codex <codex@openai.com>

* test(runtime): include playbackState in the stage-delete db mock

---------

Co-authored-by: Codex <codex@openai.com>

* fix(playback): persist discussion facts on every consumption path (#957)

* fix(playback): persist discussion facts on every consumption path (final-review P0+P1s)

- The engine now publishes a progress snapshot the moment a discussion is
  consumed (join / skip / unselected-agent auto-skip). onProgress otherwise
  fires before the discussion action executes and a discussion is the scene's
  last action, so the fact never reached persistence.
- Reads fold records across ALL playback sessions in the learner partition
  (mergeLearner deliberately preserves same-kind sessions from both keys).
- Legacy migration appends only not-yet-durable facts, so an interrupted
  migration resumes instead of dropping the tail.
- recordConsumedDiscussion reports durability; the component drops failed ids
  from its observed set so a later progress tick retries (at-least-once).

* test(e2e): live verification of the playback persistence chain

Seeds a deterministic stage straight into the Dexie DB, starts the lecture
via the canvas overlay, and asserts the full chain: discussion auto-skip
appends a discussionConsumed record to maic-runtime, the device cursor lands
in KV, and both survive a fresh browsing context (empty sessionStorage).

* refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959)

Product ruling on #869's fourth family: playback learner state is front-end
ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined
discussions' content already lives in chat runtime records, and no replay
export / analytics consumer exists — so durable facts bought nothing over
in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the
RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half
with real UX value), the engine's consumption-time progress snapshot (cursor
freshness), and the legacy Dexie retirement (cursor half migrates, row
deletes, consumed ids are dropped).

* ci: whitelist the document-cutover integration trunk for PR checks

* feat(document): widened-stage contracts + app document seam (#860 follow-up, PR1) (#961)

Package: MaicDocument/DocumentStore/StageRow gain a TStage parameter
(defaults keep existing users unchanged), BrowserDocumentStore accepts an
injectable validateStage, and a version-guarded putStage closes the rename
read-modify-write race — same current-version gate as putScene.

App: lib/document/ seam — AppDocument types (currentSceneId is device
state, not document data, and the app stage validator rejects it), a
four-kind scene validator (slide/quiz delegate to the DSL; interactive/pbl
validate envelope + discriminant binding only), and pure legacy
canonicalizers (type rebound from content.type, singular whiteboard →
whiteboards, outline envelope). No consumer switches off Dexie yet.

Co-authored-by: Codex <codex@openai.com>

* feat(document): stage-storage cutover + locked lazy migration (PR2) (#962)

* feat(document): cut stage/scene persistence over to DocumentStore with locked lazy migration (PR2)

stage-storage now persists through the lib/document seam: aggregate
saveDocument with a mandatory outline envelope (omission means deletion in
the contract; ordinary saves fall back to the stored envelope), lazy
per-stage migration under a Web Lock (destination authoritative, one
coherent legacy snapshot, save-verify-marker, legacy rows retained
read-only; no-lock environments read legacy without migrating),
currentSceneId in device KV, rename via version-guarded putStage, and a
list that merges legacy-only stages until they migrate. stageOutlines'
second write authority folds into the document service, making the final
scene set and generationComplete one atomic commit.

Co-authored-by: Codex <codex@openai.com>

* fix(document-store): relocate the persistence seam out of lib/document

lib/document/ is the server-side content-extraction domain (sharp, cloud
SDKs); exporting the persistence seam from its barrel dragged node-only
modules into the client bundle and broke the Turbopack build. The seam now
lives in lib/document-store/ with its own barrel, and the extraction barrel
is restored. Client-only guards become capability probes (indexedDB /
localStorage presence) so injected node test environments keep working
while a true server render still fails loud.

* fix(document-store): no-lock write classification + lock-order inversion (PR2 fix round)

Lock order is now per-stage document lock → global runtime epoch (a save
never waits on a document lock while occupying the shared epoch, which
deadlocked against fair-queued maintenance). Without Web Locks, mutation
classifies instead of refusing wholesale: destination-backed and genuinely
new documents accept the product's established lock-free/LWW risk, while a
legacy-only document stays read-only so two authorities cannot fork —
matching the chat-cutover-era acceptance suite, whose save assertions now
verify through the public API instead of legacy Dexie rows.

Co-authored-by: Codex <codex@openai.com>

* style: prettier on document-store seam files

---------

Co-authored-by: Codex <codex@openai.com>

* feat(document): cut secondary consumers over to the document seam (PR3) (#963)

ZIP classroom/video exports read stage metadata through the seam; classroom
import builds one canonical aggregate under the per-stage lock and writes
canonical 'whiteboards' (fixing the import-side singular-alias defect the
manifest never had); backups become versioned document aggregates that
finally include outline envelopes, while old {stages, scenes} backups are
accepted through the legacy canonicalizers; clearDatabase now clears
maic-documents and the seam's KV keys so clear-cache cannot strand
documents; deleteStageWithRelatedData is deprecated in favor of
deleteStageData. Four e2e seeders target maic-documents directly;
classroom-interaction stays legacy-seeded to exercise real-UI lazy
migration.

Co-authored-by: Codex <codex@openai.com>

* fix(document): final-review hardening — restore pre-images, honest markers, clear-race exclusion (3 P1 + 2 P2) (#964)

- Backup rollback restores pre-existing destination documents (and imported
  current-scene KV values) from captured pre-images instead of deleting them
- The skip-migration path only writes its marker after reconciling the
  legacy snapshot against the authoritative destination; a divergent
  snapshot logs and stays discoverable instead of being falsely certified
- Migration's destination write enrolls in the global shared storage epoch
  (per-stage -> global order), so clearDatabase's exclusive epoch can no
  longer race a lazy migration into resurrecting a document
- listStages drops legacy entries whose snapshot vanished mid-merge

Co-authored-by: Codex <codex@openai.com>

* fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966)

* fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair)

Review findings on #955: didActiveSceneRemainUnchanged compared the active
scene by object identity, so a store update reallocating the scene during
the async quiz read dropped the learner's graded answers from that turn's
request — the scene id is the real boundary. The records route now
classifies a malformed sessionTransition as a validation failure instead of
letting the store's throw surface as a 500.

* fix(playback): superseded-engine cursor guard + migration write-window recheck

Second-vendor review of the #959 shrink (requested after the cross-review
noted it had single-vendor coverage) found: an engine orphaned by a scene
switch during async lecture resume could pass the idle-only recheck, be
resurrected, and publish its old scene's progress over the new scene's
debounced cursor — the resume continuation now requires identity with the
installed engine, and onProgress drops snapshots from superseded engines.
The legacy cursor migration also rechecks KV immediately before its write
so a concurrent tab's newer cursor cannot be overwritten and orphaned by
the legacy-row delete.

Co-authored-by: Codex <codex@openai.com>

---------

Co-authored-by: Codex <codex@openai.com>

* fix(review): approval follow-up P3 nits (#967)

* release: v0.3.1

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard

- The records route's sessionTransition guard now requires an ISO
  updatedAt (isIsoTimestamp), matching the sibling PATCH /status route
- Dead vi.mock factories for the deleted playback-storage module dropped
- A corrupt legacy playback timestamp falls back to 'now' instead of
  wedging migration into a permanent re-throw that disabled resume

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* fix(document): backup covers no-lock legacy courses; clears fence stale saves (review P1s) (#968)

Backup assembly appends read-only canonicalized aggregates for legacy-only
stages the lazy migration could not persist (no-Web-Locks environments) —
courses visible in the app can no longer vanish from an export. A
device-scoped storage generation now fences document mutations: clears bump
it inside their exclusive section, and any save or migration commit that
entered before the clear re-reads it inside the shared epoch and fails loud
instead of repopulating a wiped store.

Co-authored-by: Codex <codex@openai.com>

* fix(document): review round 2 — legacy-only chat backup coverage + full-width generation fence (#972)

Backups now enumerate runtime chat sessions for every exported document,
not just store-backed ones — a legacy-only course's chat history (the chat
and document cutovers are independent seams) no longer drops out of the
export; the runtime read injects an empty legacy chat store since legacy
rows are collected by direct table reads, which also keeps the export
working without Web Locks. The storage-generation fence now guards every
mutating store method, with the exclusive-epoch delete cascades passing
storageSharedLockHeld so the guard cannot self-deadlock against their own
hold.

---------

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-22 23:18:24 +08:00
wyucClaude Fable 5Codex
 		3eea9dc542 feat(runtime): complete the #869 learner-data cutover — quiz + playback onto RuntimeStore (#955) 
* feat(runtime): persist quiz attempts in RuntimeStore

* fix(runtime): coalesce quiz draft snapshots

* fix(runtime): recover concurrent quiz attempts

* fix(runtime): handle quiz completion races

* fix(runtime): commit quiz review atomically

* fix(runtime): dedupe concurrent quiz writes

* fix(runtime): drop stale quiz drafts

* fix: harden quiz runtime review recovery

* fix: serialize quiz attempt identity

* feat: read quiz state from runtime store

* fix: persist quiz retries before resetting

* fix: preserve authoritative quiz outcomes

* fix: preserve legacy quiz retries during cutover

* fix: reconcile legacy quiz snapshots safely

* fix: drain rollover quiz write queues

* fix: drain completed quiz retry queues

* fix: reuse concurrent quiz retries

* fix(quiz): preserve drafts across abrupt reloads

* fix(quiz): recover empty retry sessions

* fix(chat): abort stalled runtime state reads

* fix(quiz): expose queued phases to readers

* fix(quiz): retain concurrent writer tails

* fix(quiz): canonicalize retry branches

* fix(quiz): keep retry rollovers monotonic

* fix(quiz): validate skipped retry siblings

* fix(quiz): close read cutover races

* test(classroom): cover legacy quiz summaries

* fix(quiz): reset async consumers on scene changes

* fix(quiz): close scene transition windows

* test(pbl): cover launch freshness guards

* fix(quiz): close cutover concurrency gaps

* fix(quiz): harden retry and context freshness

* fix(quiz): reject malformed legacy answers

* fix(quiz): validate legacy answer values

* test(quiz): cover legacy multi-answer migration

* fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover

Main's three call sites built static storeState blocks that runAgentLoopFn
never consumed (it always rebuilds fresh state via getStoreState); the quiz
read cutover replaced that callback with the async two-phase RuntimeStore
read, leaving the template field with zero consumers. Drop it.

* feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions

The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError
semantics existed only in the browser backend; server-backed deployments would
silently accept conflicting appends and leave completed sessions active. Forward
the options over the wire, detect conflicts atomically under the PG transaction,
map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error
client-side so quiz retry logic works across every backend.

Co-authored-by: Codex <codex@openai.com>

* fix(chat): Pi single requests build storeState via the async runtime quiz read

Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the
request template straight to /api/chat/pi, which rejects bodies without
storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read
with the scene-transition guard) and call it on the Pi path too.

* ci: whitelist the runtime-data-cutover integration trunk for PR checks

* feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956)

* feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869)

The fourth and last runtime family. Consumed-discussion facts become
append-only 'playback' records folded into a set at read (at-least-once
appends, no conflict machinery); the resume cursor is device-scoped
last-write-wins KV per the amended #779/#869 split. sessionStorage keeps
same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie
playbackState machinery is retired, with a one-time lazy migration of any
legacy row (cursor half + facts half) before deletion, and stage deletion
now clears both the KV cursor and any unmigrated legacy row.

Co-authored-by: Codex <codex@openai.com>

* test(runtime): include playbackState in the stage-delete db mock

---------

Co-authored-by: Codex <codex@openai.com>

* fix(playback): persist discussion facts on every consumption path (#957)

* fix(playback): persist discussion facts on every consumption path (final-review P0+P1s)

- The engine now publishes a progress snapshot the moment a discussion is
  consumed (join / skip / unselected-agent auto-skip). onProgress otherwise
  fires before the discussion action executes and a discussion is the scene's
  last action, so the fact never reached persistence.
- Reads fold records across ALL playback sessions in the learner partition
  (mergeLearner deliberately preserves same-kind sessions from both keys).
- Legacy migration appends only not-yet-durable facts, so an interrupted
  migration resumes instead of dropping the tail.
- recordConsumedDiscussion reports durability; the component drops failed ids
  from its observed set so a later progress tick retries (at-least-once).

* test(e2e): live verification of the playback persistence chain

Seeds a deterministic stage straight into the Dexie DB, starts the lecture
via the canvas overlay, and asserts the full chain: discussion auto-skip
appends a discussionConsumed record to maic-runtime, the device cursor lands
in KV, and both survive a fresh browsing context (empty sessionStorage).

* refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959)

Product ruling on #869's fourth family: playback learner state is front-end
ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined
discussions' content already lives in chat runtime records, and no replay
export / analytics consumer exists — so durable facts bought nothing over
in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the
RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half
with real UX value), the engine's consumption-time progress snapshot (cursor
freshness), and the legacy Dexie retirement (cursor half migrates, row
deletes, consumed ids are dropped).

* fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966)

* fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair)

Review findings on #955: didActiveSceneRemainUnchanged compared the active
scene by object identity, so a store update reallocating the scene during
the async quiz read dropped the learner's graded answers from that turn's
request — the scene id is the real boundary. The records route now
classifies a malformed sessionTransition as a validation failure instead of
letting the store's throw surface as a 500.

* fix(playback): superseded-engine cursor guard + migration write-window recheck

Second-vendor review of the #959 shrink (requested after the cross-review
noted it had single-vendor coverage) found: an engine orphaned by a scene
switch during async lecture resume could pass the idle-only recheck, be
resurrected, and publish its old scene's progress over the new scene's
debounced cursor — the resume continuation now requires identity with the
installed engine, and onProgress drops snapshots from superseded engines.
The legacy cursor migration also rechecks KV immediately before its write
so a concurrent tab's newer cursor cannot be overwritten and orphaned by
the legacy-row delete.

Co-authored-by: Codex <codex@openai.com>

---------

Co-authored-by: Codex <codex@openai.com>

* fix(review): approval follow-up P3 nits (#967)

* release: v0.3.1

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard

- The records route's sessionTransition guard now requires an ISO
  updatedAt (isIsoTimestamp), matching the sibling PATCH /status route
- Dead vi.mock factories for the deleted playback-storage module dropped
- A corrupt legacy playback timestamp falls back to 'now' instead of
  wedging migration into a permanent re-throw that disabled resume

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-22 15:28:04 +08:00
wyuc 34448beb6c feat(storage): server-backed runtime — pluggable backend seam, HTTP contract, Postgres backend, reference server (#939) (#946) 
* ci: run PR checks against the runtime-server-backend integration branch (#939)

* feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B) (#940)

* feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B)

- documented JSON HTTP contract for every RuntimeStore operation
  (server-assigned seq, learnerKey derived from auth — never trusted
  from the request, machine-readable error codes, idempotency notes)
- HttpRuntimeStore client with injected fetch + auth headers hook;
  session reads (including createSession responses) migrate forward
  on the runtime line so an older server cannot leak stale envelopes
- conformance test server bridging the contract onto the browser
  backend; full runRuntimeStoreContract green over HTTP plus error
  mapping cases

* fix(storage): harden HTTP runtime backend after cross-review (#939 Part B)

Cross-review fixes:
- appendRecord gates payloads through assertJsonValue: fail loud on
  values JSON cannot represent (Map, Date, NaN, nested undefined, NUL
  strings) instead of silently mangling them in transit; payload
  domain documented in the contract
- appendRecord/listRecords responses validate via validateRuntimeRecord
  (server-assigned seq is no longer trusted verbatim) and listRecords
  sorts by seq like listSessions already sorted defensively
- request headers no longer require an ambient Headers constructor
  when fetch is injected
- conformance server classifies errors structurally (existence checks,
  DSL validators, version stamps) instead of regex-sniffing messages
  that interpolate caller-controlled ids
- router preserves empty path segments so empty-key calls keep browser
  no-op semantics instead of shifting onto other routes
- stray runtimeDslVersion/seq in request bodies are ignored (store-
  assigned wins), matching the browser reference; docs updated
- loopback integration test exercises the real listening server
- docs state the conformance server is test-only; auth-derived
  learnerKey is Part D's reference server

* fix(storage): round-2 cross-review hardening for the HTTP backend (#939 Part B)

- json-value guard rewritten: additionally rejects -0, symbol-keyed and
  non-enumerable own properties, and non-index own properties on arrays;
  U+2028/U+2029 are accepted again (they round-trip through JSON per
  RFC 8259 — rejecting them lost legitimate pasted text)
- conformance server classifies racing duplicate creates as 409 via a
  structural post-check, gates payloads and merge learner keys as 400
- segment() rejects '.'/'..' ids the URL layer would fold away
- list responses must be arrays and mergeLearner's moved must be a
  finite number, else a typed MALFORMED_RESPONSE error

* fix(storage): round-3 cross-review hardening for the HTTP backend (#939 Part B)

- json-value guard: NUL is rejected in object keys too, and unpaired
  UTF-16 surrogates are rejected in string values and keys (jsonb
  refuses both; other JSON stacks corrupt lone surrogates to U+FFFD)
- typed storage errors survive non-object 200 responses instead of
  degrading into TypeErrors on .id access
- mergeLearner's moved must be a non-negative integer
- conformance server classifies missing/non-object request bodies as
  400 VALIDATION_FAILED
- createSession/appendRecord gate the full init envelope, not only the
  payload (stray Date/undefined properties, NUL in ids)

* fix(storage): round-4 convergence fixes for the HTTP backend (#939 Part B)

- envelope JSON gate tolerates explicitly-undefined optional anchors
  (sceneId: undefined behaves like omission, matching the browser)
- body-carried identifiers reject '.'/'..' so nothing persists that
  the URL layer cannot later address; conformance server mirrors it
- mergeLearner keys pass the JSON-domain gate on client and server
- json-value guard v4: rejects enumerable accessors (validation/serialize
  TOCTOU), Array subclasses and null-proto arrays, prototype-supplied
  array indices; sparse-array test now constructs a genuine hole

* fix(storage): round-5 convergence fixes for the HTTP backend (#939 Part B)

- undefined-stripping is limited to the DSL-declared optional anchors
  (sceneId/actionIndex/subAnchor); any other undefined member fails the
  JSON gate loud instead of being dropped silently
- json-value guard: array and object prototype checks are realm-agnostic
  (chain-shape instead of identity), so ordinary values from another
  realm are accepted while subclasses stay rejected; shared
  isLosslessJsonString predicate exported for SQL key guards
- conformance server merge route validates target-key addressability

* fix(storage): close the toJSON prototype channel in the JSON guard (#939)

toJSON is the single channel through which a prototype can alter JSON
output — prototype properties never serialize and own accessors are
already rejected — so refusing any value with a callable toJSON closes
prototype influence on serialization entirely, including prototypes
crafted to pass the realm-agnostic chain-shape check.

* fix(storage): probe toJSON via descriptor walk, not property read (#939)

Reading value.toJSON would execute an inherited accessor, letting a
stateful getter hide from validation and reappear at stringify time.
The probe now walks own descriptors up the prototype chain without
invoking any user code. Post-validation mutation of the caller's object
graph is documented as out of scope — it is equally unpreventable for
every other validated property.

* fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939)

The own-most descriptor decides: absent is safe, a non-callable data
value is an ordinary shadowing member and is safe, a callable data
value is rejected, and an accessor is rejected because it cannot be
inspected without invoking it. No daylight remains between the guard
and the serializer on this property.

* feat(storage): PgRuntimeStore — Postgres runtime backend (#939 Part C) (#941)

* feat(storage): PgRuntimeStore — Postgres runtime backend over an injected queryable (#939 Part C)

- PgRuntimeStore implements RuntimeStore over a minimal injected
  Queryable (node-postgres and PGlite both satisfy it) — the package
  keeps zero runtime dependencies beyond @openmaic/dsl
- runtime_sessions / runtime_records schema exported as
  RUNTIME_PG_SCHEMA with idempotent ensureSchema()
- appends serialize per session via a session-row lock; MAX(seq)+1
  and the insert share one transaction, UNIQUE(session_id, seq) plus
  bounded retry backstop non-cooperating writers
- envelope semantics mirror the browser backend: version stamping,
  validation gates, migrate-on-read, fail-loud on future-stamped rows
- full runRuntimeStoreContract green on PGlite plus PG-specific cases
  (concurrent-append seq atomicity, ensureSchema and mergeLearner
  idempotence)

* fix(storage): require pinned transactions + JSON payload gate in PG backend (#939 Part C)

Cross-review fixes:
- withTransaction is now required; the BEGIN/COMMIT fallback is removed
  (on a pg Pool it spread BEGIN/body/COMMIT across different connections
  — no real transaction, leaked idle-in-transaction clients, broken
  mergeLearner atomicity; on a shared pinned client concurrent calls
  interleaved transactions)
- single-statement deletes no longer wrap in a transaction hook call
- appendRecord gates payloads through assertJsonValue: fail loud on
  values JSON cannot represent (Map, Date, NaN, nested undefined, NUL
  strings) instead of silently persisting something different
- append retry also covers 40001/40P01; READ COMMITTED assumption
  documented at the retry loop
- ensureSchema documented as create-only; redundant stage index dropped
- deterministic interleaving test proves the 23505 retry path; real
  PostgreSQL contract lane added (postgres:16 service workflow, pg
  driver suite skipped locally without PG_CONTRACT_URL)

* fix(storage): round-2 cross-review hardening for the PG backend (#939 Part C)

- json-value guard rewritten (shared with the HTTP backend): additionally
  rejects -0, symbol-keyed and non-enumerable own properties, and
  non-index own properties on arrays; accepts U+2028/U+2029
- storage-pg-contract workflow now triggers for the runtime-server-backend
  integration branch and fails loud (STORAGE_PG_CONTRACT_REQUIRED=1)
  when PG_CONTRACT_URL is missing instead of silently skipping
- loadSession distinguishes corrupt non-object rows from absent rows so
  getSession fails loud instead of reporting the session missing

* fix(storage): round-3 cross-review hardening for the PG backend (#939 Part C)

- json-value guard (shared): NUL rejected in object keys, unpaired
  UTF-16 surrogates rejected in values and keys
- createSession/appendRecord gate the full persisted envelope through
  assertJsonValue, not only the payload — stray Date/undefined
  properties and NUL in ids fail loud instead of silently diverging
  from the returned record or leaking raw 22P05
- real-PG lane covers a genuine 23505 conflict from a second
  connection and recovery after an aborted transaction
- retry-set asymmetry and mergeLearner's unbounded lock set documented

* fix(storage): round-4 convergence fixes for the PG backend (#939 Part C)

- NUL/lone-surrogate lookup and delete keys resolve to absent/no-op
  instead of leaking 22021 driver errors; setSessionStatus reports the
  session missing; mergeLearner from-key moves 0
- mergeLearner destination key passes the JSON-domain gate fail-loud
- envelope JSON gate tolerates explicitly-undefined optional anchors,
  matching the browser backend
- json-value guard v4 (shared) + tests for accessors, Array subclasses,
  prototype-supplied indices

* fix(storage): round-5 convergence fixes for the PG backend (#939 Part C)

- appendRecord pre-checks the session key like every other lookup path,
  so a NUL/lone-surrogate sessionId reports 'no session' instead of
  leaking a 22021 driver error
- the queryable-key predicate now structurally reuses the shared
  isLosslessJsonString export instead of restating the string rule
- undefined-stripping limited to the DSL-declared optional anchors
- json-value guard: realm-agnostic prototype checks (shared)

* fix(storage): close the toJSON prototype channel in the JSON guard (#939)

Shared guard change with the HTTP backend branch; adds the crafted
null-proto-prototype regression test. Both final-gate reviewers
independently converged on this same channel.

* fix(storage): probe toJSON via descriptor walk, not property read (#939)

Reading value.toJSON would execute an inherited accessor, letting a
stateful getter hide from validation and reappear at stringify time.
The probe now walks own descriptors up the prototype chain without
invoking any user code. Post-validation mutation of the caller's object
graph is documented as out of scope — it is equally unpreventable for
every other validated property.

* fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939)

The own-most descriptor decides: absent is safe, a non-callable data
value is an ordinary shadowing member and is safe, a callable data
value is rejected, and an accessor is rejected because it cannot be
inspected without invoking it. No daylight remains between the guard
and the serializer on this property.

* fix(storage): conform HTTP/PG backends to the post-#926 RuntimeStore interface (#943)

The chat cutover (#926) added deleteAllRuntime() to the RuntimeStore
contract while the HTTP and Postgres backends were developed in
parallel against the pre-#926 interface. Implements the method on both
backends (single-statement wipe on PG via the FK cascade; DELETE
/runtime on the HTTP contract, documented as an operator-gated
administrative endpoint), restoring a green typecheck and contract
suite on the integration branch.

* feat(runtime): injectable RuntimeStore backend + learner-key provider (#939 Part A) (#944)

* wip(runtime): backend injection seam — pending gate verification

* fix(runtime): cross-review hardening for the storage injection seam

- stage-deletion IndexedDB probe applies only to the default browser
  backend; an injected store always receives deleteStageRuntime
- explicit kv argument takes precedence over the configured learner-key
  provider, matching the store seam's explicit-beats-global rule
- configured learner-key resolution is latched with in-flight dedup,
  mirroring the store singleton; identity changes require app-level
  handling
- factory retry semantics documented; seal errors explain the
  module-level bootstrap requirement; isRuntimeStorageConfigured()
  probe and a test-only reset added
- client-bootstrap-only contract documented (SSR/HMR caveats)

* fix(runtime): snapshot configuration and make the test reset complete

- configureRuntimeStorage copies the option fields so mutating the
  caller's object after configuring cannot swap the sealed backend or
  identity provider
- resetRuntimeStorageForTests now clears every latched consumer cache
  (store singleton, learner-key in-flight promise) via a reset-hook
  registry, so a reset-then-reconfigure test actually gets the new
  backend

* fix(runtime): reset also clears the default learner-key caches

resetRuntimeStorageForTests left defaultInFlight/defaultKv latched, so
a default-path test could leak its anonymous key or KV store into the
next test despite the documented full-reset promise.

* feat(storage): runtime reference server — auth-derived learnerKey (#939 Part D) (#945)

* wip(storage): reference server — pending deleteAllRuntime route + gates

* feat(storage): runtime reference server — auth-derived learnerKey enforcement (#939 Part D)

- createRuntimeHttpHandler(store, options) wires the documented HTTP
  contract onto any injected RuntimeStore over node http
- authenticate is required; every learner-scoped operation verifies the
  path/body learnerKey against the authenticated principal (403
  FORBIDDEN_LEARNER) — the client-supplied value is never trusted
- mergeLearner requires an explicit authorizeMerge grant and admin
  surfaces (stage cascade, DELETE /runtime) require authorizeAdmin;
  both default-deny
- runnable reference entry demonstrates a pg Pool withTransaction and
  bearer-token authentication, marked as demo-only
- contract suite green through HttpRuntimeStore -> listening reference
  handler -> PgRuntimeStore(pglite); security matrix tested (401/403
  paths, admin default-deny); threat model documented

* fix(storage): cross-review hardening for the reference server (#939 Part D)

- principal learnerKey is optional: admin/merge-only credentials no
  longer fabricate learner identity; learner-scoped routes 403 without
  ownership
- full session/record envelopes pass the JSON-domain gate at the
  handler, so NUL/lone-surrogate identifiers map to 400 instead of 500
- payload validation follows the injected store's validator map
  (options.payloadValidators) instead of imposing DSL defaults
- reference factory accepts authenticate/authorizeMerge/authorizeAdmin/
  payloadValidators overrides; docs no longer claim the factory binds
  to localhost; demo-impersonation warning hardened
- 500 responses carry a generic message; details go to the server log
- ownership checks precede version checks and unowned sessions read as
  404, closing existence/version oracles; cross-learner denial matrix
  tested per route
- merge/delete concurrency documented as linearizable-equivalent with
  the ownership re-check narrowed to the delete call

* fix(storage): align reference-server semantics with the store contract

- future-stamped sessions read and delete through unchanged; 409
  FUTURE_VERSION applies only to guarded writes (status, append, merge)
- check-then-write races reclassify structurally via a post-failure
  re-fetch: missing session 404, non-active session 400, never a
  message-sniffed or generic 500

* fix(storage): close the reference CLI's pg pool on startup failure

ensureSchema opens connections inside createReferenceRuntimeServer, so
a failed schema init or occupied port left the pool holding database
resources until its idle timeout.

* fix(storage): close the records-route existence oracle

cosarah's review point on #946: the records list answered 200 [] for an
absent session but 404 for another learner's, so the 404 leaked that an
id exists. Absent and foreign sessions now answer identically (404
SESSION_NOT_FOUND) and the HTTP client maps that code back to an empty
list, preserving the store contract's absent-lists-as-empty semantics.
Contract doc records the server MAY/SHOULD and the client MUST.
 2026-07-17 16:15:57 +08:00
wyuc 65bf20a84b feat(runtime): cut chat sessions over to RuntimeStore (#926) 
* feat(runtime): persist chat sessions in RuntimeStore

* fix(runtime): harden chat persistence ordering

* fix(runtime): protect cross-tab chat snapshots

* fix(runtime): compare structured chat values safely

* fix(runtime): preserve chat backup and reset semantics

* fix(runtime): restore imported chat snapshots

* fix(runtime): isolate chat load failures

* fix(runtime): make cache clearing lossless

* fix(runtime): normalize empty chat saves

* fix(runtime): serialize legacy chat migration

* fix(chat): serialize backup restoration

* test(chat): cover multi-stage restore locking

* fix(chat): coordinate fallback migrations across tabs

* fix(chat): harden cross-tab migration locking

* fix(database): retire abandoned chat lease schema

* fix(chat): preserve monotonic local edits

* fix(chat): protect unseen runtime sessions

* fix(chat): order lifecycle transitions monotonically

* fix(chat): serialize runtime maintenance

* fix(chat): order queued writes before maintenance

* fix(chat): preserve cross-version lock compatibility

* fix(chat): preserve streamed and backup data

* fix(chat): isolate empty no-lock saves

* fix(chat): isolate partial runtime snapshots

* fix(runtime): close cross-store cutover races

* fix(runtime): close remaining cutover races

* fix(chat): retain caller-visible conflict baseline

* fix(chat): preserve read-only legacy autosaves

* fix(chat): avoid restore lock inversion

* fix(runtime): enroll writes before maintenance

* fix(runtime): retain maintenance ordering

* fix(runtime): close restore and deletion races

* fix(chat): order partition queues within lock epochs

* fix(chat): preserve recovery and deletion intent

* fix(runtime): recover interrupted chat restores

* fix(backup): scope chat deduplication by stage

* fix(backup): stage chats by runtime partition

* fix(backup): clear staged chats with stages
 2026-07-16 18:30:40 +08:00
wyucand杨慎 ebb12b5fc1 feat(agent): add validated JSON Patch element edits (#927) 
* feat(agent): add validated JSON Patch element edits

* fix(agent): preserve JSON Patch replace semantics

* fix(agent): close structured replace contract gaps

* fix(agent): preserve structured add sequencing

* fix(renderer): preserve existing image filter units

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-07-15 14:20:00 +08:00
wyucClaude Fable 5Cursor杨慎
 		cc5a6ab152 feat(agent): edit_elements — NL → EditIntent vertical (#895) (#896) 
* feat(agent): add edit_elements NL → EditIntent vertical (#895)

Give the editing surface its first real consumer: natural-language
per-element style/geometry edits flow through the same EditIntent
vocabulary as pointer gestures, with an all-or-nothing validation gate
and one undo entry per instruction.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(agent): harden edit_elements against CR findings (#895)

Re-validate intents at apply time (atomic refuse on missing/locked ids),
type-check prop values, require an open edit session for undo, clamp line
stroke width correctly, enforce group cohesion, nest shape text chrome,
and surface refuseReason on the tool card.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(agent): close remaining edit_elements CR residuals (#895)

Feed apply outcomes into next-turn history and post-run chat corrections,
phrase tool success as proposed-not-applied, and tighten gradient/filters/
enums/per-type value validation plus label-less shape text refusal.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(agent): derive edit_elements prop validation from the DSL schema (#895)

Three review rounds each found a hand-written validator disagreeing with
the DSL contract (a gradient pos range pinned at [0,1] against the
percent contract, a textType enum matching the DSL in neither direction,
per-type ownership gaps like vertical-on-shape and color-on-everything).
Replace the class instead of the instances:

- The gate now loads the dsl-generated JSON schema
  (@openmaic/dsl/schema/scene.schema.json) and validates every proposed
  prop value against the target element type's subschema via a compact
  schema-subset checker — per-type ownership, enums, and structured
  shapes (outline/shadow/gradient/filters) come from the single source
  of truth and stay correct as the DSL evolves.
- Explicit policy layers stay on top: the content/identity blocklist
  (schema would allow what policy forbids), geometry clamps (coords,
  min sizes, line stroke width), and a numeric-range overlay for what
  plain number types cannot express — gradient stop pos is a percent in
  [0,100] (the previous [0,1] check refused canonical gradients and the
  unit test pinned the wrong contract), fontSize/lineHeight/radius
  bounds, filter value length, stop-count cap.
- Consequences: color is refused on types that do not render it, with
  prompt guidance steering text color to defaultColor and shape body to
  fill; fill on charts is now allowed (the schema has it; the inventory
  already advertised it); textType accepts the full DSL enum.
- Apply-time revalidation also refuses element-type drift via a
  gate-time targetElementTypes map carried in the tool details.
- Refusal reasons render as visible card text, not only a hover title.
- Test pins added for all of the above plus selection scoping, coord
  bounds, non-finite rotate, and empty batch/props.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(agent): harden the schema-derived gate — fail closed, assert schema source, unify numeric clamping (#895)

- A schema construct the subset checker cannot judge (unresolved $ref,
  malformed anyOf, unsupported keywords) now refuses instead of
  accepting — the gate fails closed on the unknown.
- A module-load assertion verifies the loaded dsl schema carries every
  definition the gate maps and a sane TextType enum, so a stale or
  missing build artifact fails loudly with a rebuild hint instead of
  validating an outdated contract silently.
- Numeric-overshoot convention unified and documented: structural,
  enum, or ownership mismatches refuse (teaching the model the
  contract); numeric overshoot on a valid prop clamps (matching gesture
  behavior) — opacity now clamps like geometry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(agent): localize edit element refusals (#895)

* fix(agent): preserve partial element edit semantics (#895)

* fix(agent): prevent stale element edit application (#895)

* fix(agent): preserve live canvas edit invariants (#895)

* fix(agent): close element style contract gaps (#895)

* fix(agent): reject non-persistent element styles (#895)

* fix(agent): preserve rigid group geometry (#895)

* fix(agent): tolerate decimal group translations (#895)

* fix(agent): constrain line height edits (#895)

* fix(agent): reject stale or shadowed element edits (#895)

* fix(agent): close renderer contract gaps (#895)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-07-14 18:24:35 +08:00