Commit Graph
449 Commits
Author SHA1 Message Date
wyucand杨慎 e96e2fbf7a feat(dsl): standardize the asset manifest and converge the export paths (#1007 part 3) (#1117)
* feat(dsl): standardize the document asset manifest

Add asset-manifest.ts to @openmaic/dsl: the canonical AssetManifestEntry
shape (ref, kind, and byteSize/mimeType/duration/voice/prompt metadata
where available) plus enumerateAssetManifest, the pure document-to-manifest
enumeration. An entry's ref is the reference exactly as the document holds
it -- the manifest is the id-based reference enumeration with metadata,
not a content hash and not a resolution result. The traversal walks the
stage whiteboard, each scene's canvas/whiteboards/speech actions, and the
stage video-manifest keys in document order, with logical-owner reference
counts that match the accounting duplication-safe replacement uses.

This settles the media-ref + asset-manifest schema question (#779 open
question 4) on the side the asset-pool RFC already implied: the schema is
a function of the id semantics decided there. The type lives in the dsl
rather than a new @openmaic/exporter package because the enumeration is
pure over document types the dsl already owns (Stage/Scene/Slide/Action),
so a separate package would add a published artifact and release-workflow
surface without adding a capability; the storage contract comment now
points at the module.

Refs #1007

* refactor(export): drive the classroom ZIP from the asset manifest

collectMediaFiles used to scan the whole mediaFiles table for the stage,
so any row the document no longer references -- an orphan left by an edit
or a superseded regeneration -- rode along into the archive. Both ZIP
collectors now take their reference sets from the standardized asset
manifest (buildStageAssetManifest wraps the dsl enumeration with the
compatibility rows' metadata): only referenced assets are archived, and a
referenced asset whose bytes exist only in the pool is still collected via
a synthesized record. Byte resolution is unchanged: pool first through
resolveStoredBytes / resolveAudioBlob, with the compatibility row kept as
the legacy byte fallback and as the metadata source.

mediaIndex is now a serialized view of the manifest, and the missing-audio
report derives from the manifest's audio entries instead of a second
action walk. The audioRef mapping and the legacy audioUrl fetch path
(collectLegacyAudioForExport) are untouched.

Refs #1007

* refactor(video-export): take the timeline's reference sets from the manifest

createVideoTimelineDeps scanned the whole mediaFiles table for the stage
and derived its audio id set from its own action walk -- a third,
independent answer to "which media does this course use?". Both record
loads now key off the standardized asset manifest: media rows are read per
manifest ref by compound key instead of by table scan, and the audio id
set is the manifest's audio entries. Orphan rows were never reachable
through the scene-scoped elementId-to-mediaRef bridge; now they are not
even read.

The bridge itself is untouched: element ids recur across scenes, so the
elementId-to-mediaRef mapping stays scoped per scene, and the legacy
audioUrl fallback keeps its own action walk because a URL is not a
manifest ref. AssetPlan remains the video IR's view of the same
references.

Refs #1007

* refactor(export): resolve PPTX media through the shared resolver only

Each PPTX element branch carried its own resolution chain: a task-state
renderable-URL lookup first, then -- gated on the legacy placeholder
predicate -- a stored-bytes override, with the poster block repeating the
pattern. One helper now owns resolution for backgrounds, images, video /
audio sources, and posters: opaque refs (allocated ids and legacy
placeholders alike, no placeholder-pattern gate) resolve pool-first
through resolveStoredBytes and embed as data URLs, concrete addresses
resolve through the media state machine and keep the caller's fetch path.
exportMediaResolution and the resolveStoredMediaBlob wrapper fold into
the helper; resolvePptxMediaBinding stays as the state-machine entry the
resolution-surface test matrix drives.

Refs #1007

* refactor(export): retire the export-side Dexie byte fallbacks

Export call sites no longer read bytes off compatibility rows directly.
The ZIP collectors and the video timeline's audio load resolve bytes only
through the shared resolvers (resolveStoredBytes / resolveAudioBlob),
which answer pool-first and keep the compatibility row as their internal
legacy fallback level; the row reads that remain at the call sites supply
metadata (format/duration/voice/mime/size/prompt) only. The rows
themselves stay for legacy and regeneration readers -- what goes is the
export paths' own fallback logic.

One observable tightening: a failed media row (error set, empty
placeholder blob) no longer ships a 0-byte file into the classroom ZIP,
and an evicted row no longer ships its empty local blob; referenced-but-
byteless assets are simply absent from the archive, as they already were
when no row existed.

Refs #1007

* test(media): cover the enriched stage asset manifest builder

Pins the join between the pure dsl enumeration and the compatibility
rows: metadata attaches by ref, rows no document reference names never
appear, and a referenced asset with no row keeps a metadata-free entry.

Refs #1007

* fix(video-export): widen the deps stage input for the manifest enumeration

enumerateAssetManifest reads the stage's whiteboard and videoManifest, so
createVideoTimelineDeps declares them on its input instead of the bare id;
callers pass only the id today and the optional fields stay absent. Also
applies the repo prettier formatting to the files this branch touched.

Refs #1007

* fix(dsl): enumerate slide audio elements in the asset manifest

Slide audio elements carry their own src, and the manifest skipped
them, so a manifest-driven collector could never archive their bytes.
The audio slot maps to kind 'audio' alongside narration ids.

Refs #1007

* fix(media): harden ref-keyed lookups against prototype-named asset refs

AssetRef is an unconstrained string alias, so a media reference can legitimately be "__proto__", "constructor", or any other Object.prototype member. Plain objects keyed by such refs silently drop assignments or answer lookups with the prototype object, which rewrite paths then accept as a mapped id.

Convert the remaining ref-keyed lookup tables introduced by the export convergence to prototype-safe structures: the classroom import media/poster alias maps and the legacy-conversion video-manifest reconstruction now use Map / null-prototype containers with explicit membership checks, and every consumed value is validated as a string before it is written into a src / mediaRef / audioId slot. The shared media-task lookup receives the same treatment: one centralized own-property-checked lookupMediaTask now serves the stored-bytes resolver, the PPTX embeddable-src path, the video collection path, and the element/background task resolution, so a prototype-named placeholderRef can no longer hide a re-keyed task from the fallback chain.

Adversarial tests drive "__proto__" and "constructor" refs through the import round trip, the PPTX fallback path, the legacy conversion commit path, and the media-task fallback end to end, including a buildPptxBlob regression with a task re-keyed to an allocated id while retaining a prototype-named placeholderRef.

* fix(export): use safe archive asset paths

* refactor(dsl): centralize slide media slot roles

* refactor(export): derive consumer refs from manifest

* fix(export): sanitize classroom archive extensions

* fix(video-export): preserve narration speech order

* fix(export): enforce kind-coherent archive media

* fix(export): define media coherence boundary

* fix(export): carry task-owned poster binding for PPTX export

A video element with no explicit poster falls back to its media task's
generated poster URL, but resolveVideoMediaForElement left posterTask
undefined for that case, so the PPTX manifest guard saw a foreign URL
with no task-ownership exemption and dropped the video element instead
of using the established runtime poster fallback.

Carry the poster task binding whenever the task poster is the effective
poster: the task-owned URL then satisfies the guard's objectUrl
exemption end to end. A concrete explicit element poster still stays
element-owned and never borrows the binding, and the guard's foreign-ref
rejection is preserved (and exported as a directly testable predicate).

Coverage: an element with no poster plus a task-provided poster embeds
the task poster as the PPTX cover (red at the pre-fix head, green now),
and a genuinely unrelated URL with no task ownership is still rejected
by the guard.

* fix(export): preserve legacy narration source refs in the media index

The explicit sourceRef contract was partial: primary audio and generated
media entries carried it, but legacy URL narration serialized no source
ref. The legacy URL itself is the natural source ref — it is known at
fetch time — so wire it through the collected blob into the mediaIndex
entry. Import already registers serialized sourceRefs as aliases, so the
URL now round-trips as an explicit mapping instead of being reconstructed
only from the action's audioRef.

Poster siblings are deliberately NOT given their own mediaIndex entry:
a sibling poster (media/asset-<n>.poster.<ext>) is a legacy byte copy
written from the video record and is not an independently referenced
document asset — when the poster is a real document asset it already has
its own indexed entry with a sourceRef, and import reconstructs the
sibling by path derivation from its parent video entry, reusing the
poster's own indexed allocation when one exists. The PR description is
narrowed to match; corrected paragraph:

"Archive names never interpolate refs — sequential safe paths
(media/asset-<n>.<ext>, audio/audio-<n>.<ext>) with the original ref
preserved through an explicit sourceRef mapping on every independently
indexed media entry: generated media assets, poster assets, primary
narration, and legacy URL narration (the legacy URL itself is the
entry's sourceRef). Extensions are allowlisted per kind. The one
exception is the legacy sibling poster byte copy
(media/asset-<n>.poster.<ext>, written next to its video when the video
record still carries the pre-pool poster bytes): it is not an
independently referenced document asset, so it has no mediaIndex entry
or sourceRef of its own — its identity is derivable from its parent
video entry (same index), and import reconstructs it by sibling-path
derivation from that video entry, reusing the poster's own indexed
allocation when one exists."

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
@openmaic/dsl@0.10.0
2026-08-19 20:43:14 +08:00
wyuc 086e7626ea ci: cut wall-clock time and bound Playwright browser installs (#1146)
Cache Playwright browsers and Next compile artifacts, retry a timed-out
Chromium download, build the production bundle before Playwright starts,
and run Prettier/ESLint/tsc/i18n in parallel. Unit tests stay sequential.
Skip Playwright apt deps on ubuntu-latest.

Closes #1145
2026-08-19 15:36:00 +08:00
LING DUANandwyuc 0d308ed78d feat(pi): add Native Child Web Search (#1133)
* feat(pi): add native child web search

* fix(pi): align native web search with provider system

* fix(pi): harden native web search configuration and cancellation

* fix(pi): guide Native Child web search selection

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-19 10:35:44 +08:00
xuyuanwei678 4eb435dc40 fix(editor): align editable shape labels with text (#1137)
* fix(editor): align editable shape labels with text

* fix(editor): preserve plain-text line breaks on edit

* fix(editor): preserve entities in plain text editing

* style(editor): format shape editing changes

* test(editor): update manifest version expectation
@openmaic/renderer@0.1.2 @openmaic/editor@0.0.3
2026-08-19 10:22:11 +08:00
0ef3a1d17e fix(editor): prevent class roster cards from shrinking (#1135)
Co-authored-by: tmt <tmt@MacBook-Air-4.lan>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-18 14:25:23 +08:00
wyuc 1a151c4a49 refactor(media): share one stored-bytes resolver across export paths (#1007 part 3a) (#1116)
* refactor(media): share one stored-bytes resolver across export paths

The classroom ZIP, PPTX, and video exports each re-implemented the same
pool-first byte resolution with a Dexie compatibility-row fallback and a
task-URL fallback. resolveStoredBytes in lib/media/resolve-stored-bytes.ts
now owns the chain, with the per-path differences (which levels run, fetch
validation strictness, state-machine gating) as explicit options. The
record-id to ref mapping moves alongside it as mediaRefFromRecordId.

Intended as a behavior-preserving extraction: same bytes, same fallback
order, same error handling. Cross-review found two places where this commit
did not achieve that, both repaired later on this branch -- it moved a
supplied row's `error` verdict past the pool await (restored by the snapshot
fix) and dereferenced `record.blob` unguarded (guarded by the option-surface
fix). See the PR description for the resulting divergence from main, which
is stated rather than claimed away.

Refs #1007

* fix(media): make the resolver's option surface mean what it documents

Cross-review round 1 surfaced the same class of defect from both reviewers:
options and docstrings that promise more than the code does.

- `taskUrlFallback` was inert without `resolutionGating`. The task lookup was
  derived from the gating flag alone, so the final level resolved with no task
  and always returned null. The lookup now runs whenever either concern needs
  it, while the gated levels still read it only when gating is on -- so no
  current caller's behavior changes, and the pre-refactor callers' own
  unconditional lookup is restored.
- `compatRowBytes` dereferenced `record.blob.size` unguarded, which broke the
  module's "never throws" contract and lost the guard the pre-refactor video
  path had (`blob && blob.size > 0`). A row that lost its blob now falls
  through to the next byte source.
- `loadCompatRow` (requires `stageId`) and `compatRowCdnFallback` (needs a row)
  now state their dependencies instead of silently doing nothing.

Six tests pin the behavior; each fix was mutation-checked (reverting the task
decoupling reds one, reverting the blob guard reds two).

* test(media): pin the resolver behavior the round-2 review found unpinned

Round 2 produced no behavior findings from either reviewer; every finding was
a test that would still pass with the behavior broken. Six cases close them.

Mutation-verified -- reverting the code each covers reddens it:

- Gating suppresses stale POOL and row bytes, not only the final level
  (removing either gate reds it).
- With gating off the compatibility row is still accepted while a task is
  generating. This is the discriminator for keeping `gate` separate from
  `task`: leaking the task into the gated levels reds it.
- A zero-length row blob falls through to the CDN source when enabled, and
  never reaches it when not (breaking `compatRowCdnFallback` in either
  direction reds one of the two).
- The effective ref comes from a supplied row's compound id, not the ref the
  caller passed (dropping the derivation reds it).
- The classroom ZIP still ships a non-OK pool response body -- the rebase
  conflict resolution that was correct but untested (tightening `requireOk`
  reds it).

Not mutation-verified, and marked as such in the file: "accepts pool bytes
with gating off" documents the pool level's unconditional branch but survives
every single-edit revert of the task/gate separation, because that level
branches on the option rather than on the gate. It is kept as documentation,
not as a sentinel.

* fix(media): snapshot a supplied row's error verdict before the pool await

Round 4 of cross-review found a divergence three rounds of level-by-level
comparison had missed. The pre-refactor video path computed
`usableRecord = record && !record.error` BEFORE awaiting the pool; the
extraction moved that test after the await. The supplied row is a live
object, so an `error` set (or cleared) while the pool lookup was pending
changed which bytes the export shipped.

The verdict is now taken where the caller took it, and -- the part the first
attempt at this fix got wrong -- it is not re-derived afterwards:
snapshotting only the reference still re-read `error` off the live object at
the later check. A row this function loads itself is still judged when it is
read, after the pool miss, because that read did not exist any earlier.

This was the last value read at a different point in the SEQUENCE than the
code it replaced. What remains is scheduling, not sequence: the row's `blob`
and `ossKey` are still read after the pool as they were on main, only one
microtask later. The PR description states that, and no writer in the repo
mutates a row in place across that window.

Two tests pin both directions (an error landing mid-lookup, and one
clearing); judging the supplied row after the await reds both.

* test(media): commit the frozen-base differential harness (F-1116-001)

The PR description claimed a differential comparison against the verbatim
pre-refactor implementations, but that comparison was run out of tree and
never committed, so no reviewer could re-run it. A claim that cannot be
reproduced is not evidence.

The three implementations are now reproduced verbatim as they stand at the
merge base -- still identical to main, since nothing between the two touched
them -- and driven side by side with the current helper under each call
site's real option set, over a declared matrix of ref shapes, task states,
pool outcomes, fetch outcomes, record shapes, and stage scoping. Every
combination compares the outcome (bytes, null, or the thrown type), blob
identity by reference, and the ordered lease / Dexie / fetch sequence
including arguments.

Absolute microtask offsets are not compared, for the reason the description
gives: the extraction moves later operations one or two ticks while leaving
the sequence, the arguments, and the results alone.

The blob-less PPTX row is the only permitted divergence, and the harness
pins where it may occur rather than merely tolerating it: every divergent
combination must be that row with a stage, and the count must equal the
combinations that actually reach the row -- all of them except the ones
where the frozen path returns pooled bytes first.

The harness bites: dropping the zero-byte rejection reds ZIP and video,
dropping the response.ok check reds PPTX and video, and dropping the
compound-id ref derivation reds video. Its scope is documented in the file --
option combinations no call site uses, and state that mutates mid-await, are
covered by the targeted tests instead.

* test(media): extend equivalence harness for colon refs and PPTX divergence (F-1116-001)
2026-08-18 12:10:52 +08:00
xuyuanwei678 221df3595d fix(renderer): preserve literal text line breaks (#1132)
* fix(renderer): preserve literal text line breaks

* fix(renderer): preserve empty text paragraphs

* fix(renderer): scope line-break preservation to plain text

* fix(renderer): detect non-tag HTML markup

* style(renderer): format rich text tests
@openmaic/renderer@0.1.1
2026-08-17 21:45:46 +08:00
jackefnandwyuc 44e1ca0ca0 feat(rag): add document lexical retrieval foundation (#1078)
* feat(rag): add document lexical retrieval foundation

* fix(rag): align foundation contracts with review

* fix(rag): handle quoted br attributes

* perf(rag): stream grapheme chunk splitting

* fix(rag): preserve exact replacement scope

* perf(rag): avoid repeated grapheme segmentation

* refactor(rag): isolate grapheme chunking

* perf(rag): avoid runtime-dependent grapheme segmentation

* fix(rag): complete review contract corrections

* fix(rag): resolve follow-up correctness review

* fix(rag): address latest review corrections

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-17 12:16:31 +08:00
杨慎 7df8e50054 feat(video-export): add bounded local chunk executor (#1115)
* feat(video-export): add bounded local chunk executor

* fix(video-export): preserve producer plan hash during chunk execution

* fix(video-export): reuse producer chunk sidecars

* style(video-export): format render service entrypoint

* fix(video-export): tighten chunk cancellation and cache validation

* fix(video-export): address chunk lifecycle review findings

* test(video-export): cover chunk cancellation and drain

* fix(video-export): validate observed chunk capture mode

* fix(video-export): enforce chunk plan and resource bounds

* fix(video-export): terminate chunks at job deadline

* fix(video-export): guard chunk worker IPC teardown
2026-08-17 12:11:46 +08:00
Yizuki_Ameandwyuc 19a895e889 fix(video-export): make Cyrillic and Arabic Quiz fonts deterministic (#1114)
* feat(video-export): make Quiz script fonts deterministic

* test(video-export): verify Arabic shaping visually

* fix(video-export): cover Arabic extension characters

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-16 16:22:34 +08:00
5a4a174d72 feat(openmaic-skill): add secondary-development (二开) flow (#1119)
Add a non-invasive 3rd top-level branch to the openmaic skill for users who want to extend / build on / customize OpenMAIC or consume the @openmaic/* SDK — previously no trigger and no SOP path.

- SKILL.md: add 二开 trigger words to description; Phase 0 option 3 + short-circuit; accessCode auto-shortcut now yields to 二开 intent so returning Live Demo users are not mis-routed.
- references/extend.md (new): 二开 SOP router — charter, read-before-modify rules, dev environment (= local-deploy env, reused not duplicated), conditional fork-vs-clone + CI, route to cookbook/sdk, general gotchas.
- references/extend-cookbook.md (new): product-layer tasks with precise file:line entry points (provider / storage / branding / routes / embed renderer-editor).
- references/extend-sdk.md (new): consuming @openmaic/* in a standalone app — 6-package quickref, exact import paths, storage subpath table (incl. Runtime-barrel asymmetry), version pinning.

Existing 5 references and Core Rules are untouched; the use/generate flow is unchanged.

Co-authored-by: Percy <percy@PercydeMacBook-Pro.local>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-16 16:11:24 +08:00
Yizuki_Ameandwyuc 2126f64066 feat(i18n): add German localization (#1128)
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-16 14:07:22 +08:00
LING DUANandwyuc cdf949a017 feat(pi): add native child runtime with scoped Spotlight (#1111)
* feat(pi): add native child runtime with scoped Spotlight

* fix(pi): address native child review feedback

* fix(pi): suppress prefaced structured child output

* fix(pi): restore native child text streaming

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-16 00:05:23 +08:00
wyuc e15780835a feat(media): convert legacy references to allocated asset ids (#1101)
Lands epic #1007 part 2(c): the app-side reference converter, the `audioUrl`
removal, and the `DSL_VERSION` bump as one delivery unit.

`convertDocumentAssetRefs` walks a loaded document (slide canvases, whiteboards,
video-manifest keys, speech actions) and rewrites legacy handles to allocated
pool ids: placeholders and `audioId`s with local bytes are ingested, a co-present
`audioUrl`/`audioId` pair collapses to one asset, a definitively dead URL
converts to an emptied reference, and a transient failure keeps both handles for
a later retry. Unavailable bytes leave the document untouched for a later open,
and a converted document returns by identity with no writes.

Conversion runs lazily on document open and at fetch time for server-generated
classrooms, with conversion and the first document save sharing the per-stage
document lock: once the document commits it owns every allocation, and every
failure exit rolls its pass ledger back. Legacy URL probes are bounded by a
per-probe timeout and a shared 60-second pass budget.

`SpeechAction.audioUrl` is removed from the DSL and every consumer reads the
converted shape; the server classroom generator keeps its derived-id + serving-URL
pair as an explicitly typed pre-conversion transport, consumed before persistence.
`DSL_VERSION` becomes 0.2.0 with the first real ladder entry, and
`@openmaic/dsl` goes to 0.9.0 per the release rule.

Deferred follow-ups are tracked in #1129: production principal derivation with
document/asset ownership aligned, conditional document writes, and hoisting the
aggregate probe budget into the export and video-timeline paths.

Refs #1007
@openmaic/storage@0.4.0 @openmaic/dsl@0.9.0
2026-08-15 17:38:04 +08:00
杨慎 234ea2f540 fix(video-export): constrain GenUI iframe visibility (#1125)
* fix(video-export): constrain interactive iframe visibility

* fix(render-service): allow screenshot fallback in standard profile

* fix(render-service): lower standard memory floor to 8 GiB

* refactor(video-export): address review findings
2026-08-15 01:42:54 +08:00
wyucand杨慎 0a10af11cf feat(storage): opt-in indirect asset byte egress (#1007) (#1100)
* feat(storage): add opt-in indirect asset byte egress

A deployment can now opt an asset byte GET into a 302 to a short-lived
signed URL when the byte layer can sign. It is off by default, and
byte-for-byte unchanged when off. The byte layer gains an optional
signReadUrl capability; S3AssetByteStore implements it through the new
optional @aws-sdk/s3-request-presigner peer, resolved lazily exactly
like the client SDK, with a signer seam mirroring the commands seam so
tests can bind doubles. The signed URL pins the contract's response
headers -- the media type after the renderable allowlist, the fixed
disposition, and the no-store cache posture -- via S3 response-header
overrides. PgAssetStore mints the URL from the same ownership-checked
transactional read a direct resolve performs, so authorization still
runs per read before any URL exists, and a store whose byte layer
cannot sign falls back to direct bytes. The contract documents the
shape and the disclosure tradeoff: a Location into a hash-keyed byte
layer names the content hash, so deployments that need the
no-disclosure property keep direct egress.

Refs #1007

* feat(app): wire ASSET_BYTE_EGRESS into the persistence route

ASSET_BYTE_EGRESS=redirect opts asset byte GETs into the storage
package's indirect egress; unset, direct, or an unrecognized value
keeps the default byte-for-byte behavior, with a warning for the
unrecognized case. The lazy byte-store wrapper now forwards
signReadUrl and answers undefined when the resolved layer has no
signer, so the PostgreSQL byte column degrades to direct bytes and
the S3 layer signs without the wrapper knowing which it holds. The
collector path is untouched: it builds its byte store through
createAssetByteStore as before and only ever calls delete.

Refs #1007

* fix(storage): let the packaged client survive redirect egress

Cross-review round 1 on the indirect byte egress change found two
problems.

A byte response that followed a redirect carries the pinned content
type but no X-Asset-Revision, and HttpAssetStore required both from
the same response, so every cold resolve under redirect egress failed
MALFORMED_RESPONSE. The client now detects the shape once -- a
redirected byte response with no revision -- latches it, and probes
HEAD before each cold GET from then on, taking the label from the
probe. The probe predates the download on purpose: a replacement in
between labels newer bytes with an older revision, which the next
revalidation detects and corrects, while the reverse order could pin
stale bytes under a fresh revision with no signal to repair it. The
probe doubles as the miss check, so a redirect-mode miss costs no
download. The direct path is byte-for-byte unchanged; the one double
download per client lifetime on first contact is documented in the
contract, which gains the client-side paragraph this behavior
implements.

signedUrlTtlSeconds also accepted lifetimes beyond the seven-day
SigV4 presigning maximum, minting redirects the object store would
reject. The option is now capped at construction.

Refs #1007

* fix(storage): cap signed URL lifetime below the reclamation grace

A signed URL whose lifetime exceeds the collector grace period can
outlive its object: the last reference goes, the grace elapses, the
collector deletes the object, and the still-valid URL errors at the
object store. The previous ceiling -- the seven-day SigV4 maximum --
made that window days wide against the one-hour default grace. The
cap is now fifteen minutes, and the contract text states both the
ceiling and the rule for deployments that shorten their grace.

Refs #1007

* fix(storage): decline signing when the optional presigner is absent

An unresolvable @aws-sdk/s3-request-presigner made signReadUrl throw,
which the registry surfaced as a failed asset read -- a 500 for a
deployment whose only fault is a missing optional peer. The contract's
answer for a byte layer that cannot sign is the capability fallback:
return undefined and let the caller serve the bytes directly. Signing
errors from a resolved signer still fail loud; only the missing
capability declines.

Refs #1007

* fix(storage): answer redirect egress as a descriptor to asking clients

Following a 302 is not header-neutral: the platform fetch forwards the
original request's headers to the object store's origin, stripping only
Authorization, so a deployment whose credential travels in a custom
header would hand it to the object store -- and the preflight the
forwarded custom headers provoke commonly fails there besides. This
replaces the latch-and-probe client shape with an explicit one: the
client sends X-Asset-Egress: descriptor on every byte GET, a
redirect-egress server answers 200 with a JSON { url, revision } body
instead of a Location, and the client fetches the signed URL with no
deployment headers at all. The revision comes from the descriptor, so
the probe-first ordering and its one wasted download are gone too. The
302 remains the answer for consumers that did not ask; the descriptor
response is marked by its own header so a JSON-media asset can never
parse as one.

Refs #1007

* fix(storage): negotiate the descriptor through Accept, not a custom header

A custom X-Asset-Egress request header is not CORS-safelisted, so every
byte GET to a cross-origin server became a preflighted request -- a
regression for direct-egress deployments that never opted into
anything, against servers with no reason to allow the header. The
negotiation now rides Accept with a vendor media type, which is
safelisted and costs no preflight, and the descriptor answer is
identified by that media type as its Content-Type rather than by a
marker header.

Refs #1007

* fix(storage): sign outside the registry transaction and reserve the descriptor type

Two review findings on the indirect egress path.

resolveIndirect awaited the signer inside the coordinated read, but a
signer on refreshable credentials can wait on the network, and a
database connection plus the blob row's FOR SHARE lock would be held
across that delay. The read now closes first -- the lock still puts
the hash, label and revision in one snapshot -- and signing runs
afterwards, which cannot observe anything the read did not.

The descriptor media type is also reserved from renderableTypes: the
client identifies a descriptor answer by that exact Content-Type, and
an allowlisted asset served with it would be misread as a descriptor
and fail MALFORMED_RESPONSE. The handler now rejects the configuration
at construction, and the contract states the reservation.

Refs #1007

* fix(storage): ship the signing deps and negotiate without ambiguity

Three review findings on the indirect egress path.

The app now declares @aws-sdk/client-s3 and
@aws-sdk/s3-request-presigner as runtime dependencies, and the
standalone build force-includes them for the persistence route: both
are reached through deliberately untraced dynamic imports, so the
shipped image could not resolve them, and a deployment opting into S3
or redirect egress would have found the capability silently absent.

The descriptor negotiation matches exactly on both sides: the client
compares the Content-Type essence, so a longer media type that merely
begins with the reserved value still serves as bytes, and the server
parses Accept media ranges, so a descriptor range sent with q=0
selects the redirect instead of the descriptor it explicitly rejects.

Refs #1007

* fix(app): refuse redirect egress when the collection grace is shorter

A signed URL must never outlive its object: with the handler's
60-second default lifetime, a deployment that sets
ASSET_COLLECTION_GRACE_MS below ten minutes lets the collector delete
an object while a URL minted against it is still valid. The route now
refuses the combination at handler initialization, naming both
variables -- a deterministic misconfiguration, loud at startup rather
than silent at read time.

Refs #1007

* fix(app): ship the signing SDKs without loading them

Two findings on the standalone deployment of redirect egress.

The outputFileTracingIncludes globs did not follow pnpm's scoped
symlinks, so the standalone image carried neither AWS package and both
S3 mode and redirect egress could not resolve their SDK at runtime.
The packages are now server-external and referenced from literal --
but never called -- dynamic import thunks in the persistence byte-store
wiring, which gets them traced into the image while module resolution
still happens only on first S3 use. Verified against a local
standalone build: node_modules/@aws-sdk now contains both packages,
and the route tests that pin the never-resolve-unless-configured
discipline still pass.

Media type comparisons are also case-insensitive now, as HTTP
requires, on both the Accept parse and the descriptor recognition.

Refs #1007

* fix(app): treat an empty collection grace as unset

Number('') is 0, so a present-but-empty ASSET_COLLECTION_GRACE_MS
failed the redirect-egress coordination check and took persistence
down with it, while the collector's own parsing reads the same value
as the one-hour default. The check now trims and treats empty as
unset, matching durationEnv.

Refs #1007

* fix(storage): accept bytes too, omit signing on the known-PG wrapper, state the full tradeoff

Three review findings.

The client's descriptor request now advertises both representations --
the vendor descriptor type preferred, */* accepted -- so a strict
content-negotiating layer can never answer 406 to a client that does
in fact consume plain bytes.

The lazy byte-store wrapper no longer advertises signReadUrl when no
bucket is configured: the layer is known to be PostgreSQL at
construction, and carrying the method made resolveIndirect take its
ownership query and blob-row lock just to decline, then repeat them
in resolve, on every cold GET. With a bucket configured the lazy
validation is preserved and the wrapper still declines when the
resolved layer cannot sign.

And the contract's disclosure tradeoff now states the second edge:
object-store GETs carry ETag and Last-Modified, response overrides
cannot strip them, and because deduplicated PUTs rewrite the
hash-keyed object, Last-Modified moves when another principal
re-uploads the same bytes -- shared-object write timing, beyond byte
equality. Deployments for whom that signal is sensitive must keep
direct egress.

Refs #1007

* fix(app): degrade on bad grace, share the ttl invariant, cover the real boundary

The latest review round, four findings.

A misconfigured ASSET_COLLECTION_GRACE_MS now degrades redirect
egress to direct with a warning instead of failing the shared
handler's initialization -- the asset backend is optional, and its
misconfiguration must never take document and runtime traffic down.

The package exports assertSignedUrlTtlWithinGrace so a deployment
wiring the handler and the collector separately validates the
invariant once at its own boundary: the handler caps the lifetime,
but only the deployment knows both numbers.

The contract states that nosniff does not survive the redirect --
response-header overrides cannot set it -- and why the client-minted
blob makes that safe for conforming clients.

And the route gains a real-boundary test: the Fetch<->Node adapter,
the real createStorageHttpHandler, and the egress wiring, exercised
with an actual byte GET through the composed stack instead of mocks.

Refs #1007

* fix(storage): make the unsafe egress configuration unrepresentable

Three findings on the indirect egress path, all resolved by moving a
rule into a place where it cannot be bypassed rather than by adding a
check a consumer has to remember.

The grace/TTL invariant now lives in the option shape. byteEgress takes
'direct' or { mode: 'redirect', collectionGraceMs, signedUrlTtlSeconds? },
so enabling indirect egress without declaring the reclamation window it
lives inside no longer type-checks, and the handler validates both
numbers at construction. The flat signedUrlTtlSeconds option, its
cross-field "requires byteEgress redirect" check, and the standalone
900-second cap as the only guard all go away; the ceiling stays, but
alongside the ratio rather than in place of it. This is what the earlier
exported assertSignedUrlTtlWithinGrace could not do -- it protected only
consumers who called it -- and it costs nothing in compatibility because
the option is new in this branch. The app resolves the grace through one
shared parser with the collector, so the number the handler checks is the
number the collector runs with.

A signed URL whose object is gone is now a miss. The client maps a 404
from the byte fetch to ASSET_NOT_FOUND, matching the direct path: the
entry was owned and readable when the URL was minted, so a reclaimed
object is the same physical state the direct read reports as a miss, and
what a read means must not depend on the deployment's egress setting.
Signing still does not probe for the object -- that would restore the
round trip this feature removes and make the mint's price vary with prior
presence -- so the contract instead requires the byte layer to answer 404
for an absent object, and states that S3 needs s3:ListBucket to do so.

Route coverage now spans the whole egress matrix through the real
composed handler rather than only the direct case: 302 for a consumer
that did not ask, descriptor for one that did, direct bytes when the
byte layer declines to sign, and direct bytes when a short grace degraded
the configured mode.

Refs #1007

* feat(storage): forward byte egress options through reference server

* fix(storage): fail closed on redirects and unconfirmed 404s in indirect byte egress

A signed-fetch 404 now maps to a miss only when the object store's XML
body declares NoSuchKey; every other 404 from the signed fetch fails loud,
so a wrong bucket, access point, or endpoint can no longer make a live
asset read as absent. The descriptor byte GET is sent with redirect:
'manual' and any 3xx answer is treated as an error, so a server that
ignores the descriptor negotiation can never forward the deployment's
custom credential headers to a redirect target.

The ASSET_BYTE_EGRESS switch is now documented in .env.example and the
server-persistence deployment section, together with the object-store CORS
and s3:ListBucket prerequisites it requires, and the asset descriptor
media type is exported from the package root and the asset/http subpath.

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
@openmaic/storage@0.3.0
2026-08-14 23:57:21 +08:00
wyucandopencode 673af150c0 release: v0.3.2
Co-Authored-By: opencode <noreply@opencode.ai>
v0.3.2
2026-08-14 02:37:59 -04:00
Yizuki_AmeandCursor 13d94a52a1 feat(ai): add Grok 4.6 to the model registry (#1113)
* feat(ai): add Grok 4.6 to the model registry

The Grok picker still topped out at 4.5, so the new flagship ID was not selectable with its documented xhigh reasoning effort.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(docs): stop the docs Next app from loading app instrumentation

Next 16 discovers the repo-root instrumentation.ts from packages/docs, then fails because that file imports app persistence modules. Give the standalone docs app a no-op register() so the Grok catalog docs can build.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-13 16:59:06 +08:00
杨慎andwyuc 019fe6e6ac feat(video-export): add explicit CPU resource profiles (#1105)
* feat(video-export): add explicit CPU resource profiles

* fix(video-export): preserve failure capture metrics

* fix(video-export): report observed failure capture mode

* docs(video-export): keep benchmark evidence out of service docs

* fix(render-service): build from dated Debian snapshot

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-12 18:10:50 +08:00
Nguyễn Lê TrườngandClaude Opus 5 8db8785506 feat(i18n): add Vietnamese (vi-VN) locale (#1025)
Translate all 1,619 leaf keys with exact parity against en-US.json and
register the locale in supportedLocales.

Rebased onto current main (4ac70cce). Upstream drift resolved here:

- #1023 added 14 edit.agent.* / edit.roster.* keys and #995 added three
  export.video*Cta* keys; all are translated.
- #1060 retired the pbl.* v1 UI, so its 38 keys are dropped from vi-VN.json
  as well. check:i18n-keys compares both directions, so leaving them would
  fail on extras.
- #1068 (fr-FR) registered a locale in the same spot in locales.ts; both
  entries are kept.
- 120 keys landed since the last rebase and are translated in this pass:
  the canvas editor toolbars (edit.line.*, edit.insert.linePresets.*,
  edit.asset.*, edit.latex.*, edit.video.*, edit.audio.*, edit.image.*,
  edit.table.*, edit.contextMenu.*) and classroom folder management
  (classroom.folder*, classroom.deleteFolder*, classroom.moveTo*).
  Vietnamese has no plural inflection, so deleteFolderAndCourses and its
  _other form carry the same text; both keys are present because
  check:i18n-keys requires them.

Two extra files are required and could not be split out: LOCALE_RESOURCES
in lib/video-export-app/cover-config.ts and the expected-labels map in its
test are both typed Record<Locale, ...>. Without these entries tsc --noEmit
fails with TS2741 on '"vi-VN"', exactly as it did for fr-FR in #1068.
Seven added lines outside lib/i18n, no logic change.

Also translates the strings flagged in review that were still English:
the OpenAI/GLM/Qwen voice descriptions, browser-native TTS and ASR labels,
providerUnpdf, voiceNamePlaceholder, lang_auto, the VoxCPM voice-pool
labels, and the protocol/provider names. "(Local)" renders as "(Cục bộ)"
throughout. Left verbatim on purpose: settings.lang_* stay in their native
scripts (matching every existing locale), "Video" is the ordinary
Vietnamese word, and AccessKey ID / AccessKey Secret are the literal field
names in the Alibaba Cloud console.

README language counts updated to 11 and now include French, which #1068
had not backfilled.

pnpm run check:i18n-keys passes across all 11 locale files; prettier
reports the touched files clean; the i18n suite (39 tests) and the
video-export cover-config suite (92 tests) pass; tsc --noEmit reports no
error in any touched file.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 14:24:25 +08:00
LING DUAN 04deb9f2fd fix(agent): harden shared Pi transport contracts (#1108) 2026-08-12 14:22:03 +08:00
Xiao Chengmingandwyuc 859d66ec95 fix(ai): add opt-in streaming chat compatibility (#990)
* fix(ai): add opt-in streaming chat compatibility

* fix(ai): preserve streaming chat metadata

* fix(ai): harden streaming chat compatibility

* fix(ai): isolate primary streamed choice

* fix(ai): propagate streamed chat errors

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-12 14:19:05 +08:00
131af9c25a fix(access-code): refetch server providers after access code is accepted (#1081)
ServerProvidersInit fetches /api/server-providers on mount, which on an
ACCESS_CODE-gated deployment happens before any access cookie exists. The
middleware answers 401 and fetchServerProviders returns early, leaving the
settings store on its blank defaults. AccessCodeModal calls onSuccess without
reloading and nothing else calls fetchServerProviders, so every
server-configured provider reads as unconfigured for the whole session until
the user manually reloads.

Re-run the fetch from AccessCodeGuard's onSuccess, once the cookie makes the
request authorizable.

Co-authored-by: thanos-openclaw <thanos@openclaw.local>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-12 12:16:32 +08:00
杨慎 7d97b9bbd7 docs: align environment variable template (#1107) 2026-08-12 11:56:57 +08:00
Alvin Lamandwyuc 733c853c94 feat: apply 432 reviewed zh-TW translations (#526)
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-12 11:31:06 +08:00
杨慎andwyuc 27b6d082c4 refactor(video-export): extract stable RenderExecutor seam (#1104)
* refactor(video-export): extract render executor seam

* fix(video-export): preserve executor failure semantics

* test(video-export): lock the render HTTP contract

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-12 11:18:21 +08:00
Yizuki_Ame d1eeb95e1c feat(video-export): add deterministic Quiz question-list scrolling (#1102)
* feat(video-export): add deterministic Quiz question-list scrolling

* fix(video-export): bound Quiz layout measurement
2026-08-12 10:31:13 +08:00
杨慎 4ac70cceeb feat(video-export): capture self-contained static interactive HTML (#1086)
* feat(video-export): capture static interactive HTML

* fix(video-export): harden interactive HTML capture diagnostics

* fix(video-export): initialize runtime diagnostics in manifest

* fix(video-export): close interactive capture review findings

* fix(video-export): preserve packaged interactive resources

* fix(video-export): address packaging review feedback

* fix(video-export): harden interactive HTML packaging

* fix(video-export): unify interactive asset packaging

* fix(video-export): close remaining interactive packaging gaps
2026-08-11 22:16:58 +08:00
969574bee4 feat(storage): wire the asset backend into the app (#1007) (#1089)
* fix(persistence): stop corrupting binary response bodies in the route adapter

The embedded persistence route bridges a Node `RequestListener` to the
Fetch API by hand. Its response object is cast with `as unknown as
ServerResponse`, so the compiler checks none of that surface, and two
parts of it were wrong in ways only a byte-carrying handler would hit.

`end` decoded any `Uint8Array` chunk with `Buffer.from(chunk).toString()`,
which is UTF-8. `ServerResponse.end` accepts a `Uint8Array`, and those
bytes are not necessarily valid UTF-8, so every unpaired byte became
U+FFFD -- silent corruption with no error anywhere. The adapter now
buffers chunks as bytes and builds the response from them.

`write` was absent entirely, making any chunked handler a runtime
TypeError rather than a compile error. It is part of the surface this
object claims to implement, so it is now implemented.

Neither reaches a caller today: the document and runtime handlers only
ever end with JSON strings, which took the string branch. Both are traps
laid for the first handler that carries bytes.

Both cases are pinned by tests that fail before this change -- the
binary one on a byte comparison, the chunked one with a 500 from the
TypeError. The existing adapter round-trip test, whose comment already
called this the most bug-prone code in the route, covered only the
string path.

* feat(storage): wire the asset backend into the app (#1007)

The server asset registry shipped with #1007 but was reachable from
nothing: the persistence route never passed an `assetStore`, and no file
under `app/`, `lib/` or `components/` imported the asset server or client
modules. This connects it, additively -- no DSL change, no version bump,
and no behavioural change when server persistence is off.

The route now builds a `PgAssetStore` on the same pool and transaction as
the document store, with `PgAssetByteStore` by default and
`S3AssetByteStore` when `ASSET_S3_BUCKET` opts a deployment into it. The
AWS SDK stays an optional peer, reached only through a dynamic import on
that branch. The offline collector is deliberately not scheduled here,
and the route says so where a reader would look for it, because leaving
it unscheduled grows storage without bound and scheduling it is the
deployment's decision.

On the browser side the asset pool gains a single-shot configuration seam
modelled on the document store's, so that with server persistence enabled
the pool is an `HttpAssetStore` carrying the same auth headers as the
other stores.

Three browser-specific mechanisms needed decisions rather than
translation:

`clearAssetPool` deletes an IndexedDB database today. In server mode
there is nothing local to delete, and a "clear cache" action must not
remove server-side assets -- that would destroy user data from a button
promising to free space. It now revokes local object URLs and closes the
client, and deletes nothing remote.

The cross-tab replacement broadcast is unchanged. Its job is to make
another tab drop a warm object URL when the bytes behind an id change,
and that need is identical in server mode.

The exclusivity proof that decides whether an in-place `replace` is safe
reasons over this browser only: local documents, unflushed state, and a
cross-tab presence probe. In server mode those cover a strict subset of
the holders, since another device can reference the same id and no probe
can see it -- and asking the server who else references an id would be an
existence oracle over other principals. The proof therefore returns not
exclusive unconditionally in server mode, and regeneration takes the
existing fork path. That is the fail-closed direction and a graceful
degradation rather than a break.

The development authenticator had to grow the asset principal's required
`key`; without it every asset request would have been correctly denied.
It derives from the same learner partition, and a principal without one
still gets no asset access. The route now states plainly that this key
comes from a client-supplied header, so the cross-principal isolation the
asset contract describes is not in force under this authenticator -- that
warning previously lived only in the auth module's own header, where
nobody mounting the route would see it.

* fix(storage): close three defects in the asset app wiring (#1007)

**The S3 branch could not load, and isolated nothing.** The route
dynamically imported the storage package's S3 byte store, but that module
statically imported the AWS SDK, so merely resolving it pulled the SDK
into resolution and bundling. The route then separately performed an
ignored native import of the SDK from the app's own scope, which fails
with ERR_MODULE_NOT_FOUND because the SDK lives under the storage
workspace package -- so a deployment that set the bucket got a broken
store rather than an S3 one. There is now one lazy loader in one
resolution scope, the storage module no longer imports the SDK at module
load, and the bucket is validated at initialization rather than treated
as valid because it is non-empty. A test mocks the SDK to throw on
resolution and asserts that importing the byte-store module never
resolves it.

**A cross-tab replacement could leave a mounted lease stale.** The
broadcast handler re-resolves, and `HttpAssetStore.resolve` coalesces
concurrent calls for one id onto a single in-flight request -- correct on
its own, and required by the contract. Together they meant a resolve
triggered by the broadcast could merge into a GET that started before the
peer's replacement, return the old revision, and leave the consumer on
the old URL with nothing to dislodge it. The client gains an invalidation
that retires its cached snapshot and advances the per-id generation
without revoking any URL already issued; `release` would have been wrong
here, since the contract keeps an issued URL valid until its holder
releases it. Removing the invalidation call fails the new race test.

**Clearing a server-backed pool left the closed client installed.** In
server mode `clearAssetPool` closed the client and returned before
clearing the module singleton, so every later `getAssetPool()` handed
back a closed store until a page reload happened to intervene. Only the
IndexedDB deletion is skipped in server mode now; the singleton is always
cleared so the next call rebuilds through the configured factory.

Also adds the tests whose absence let these through: a route round-trip
carrying invalid UTF-8, the SDK-resolution assertion above, the broadcast
race, and reopening the pool after a clear. The earlier tests were not
wrong so much as aimed past the defect -- the route test used JSON text,
and the S3 test injected a loader rather than exercising the imports.

* chore(storage): bump package version to 0.3.0

* fix(persistence): make the route adapter faithful where it carries bytes

The embedded persistence route bridges a Node `RequestListener` to the
Fetch API by hand, and its response object is cast
`as unknown as ServerResponse`, so the compiler checks none of that
surface. Five divergences are closed here; the first two are the ones
that corrupt data, the last three were raised in review.

**`end` decoded bytes as text.** It ran `Buffer.from(chunk).toString()`
on a `Uint8Array` chunk, which is UTF-8, so every byte outside that
encoding became U+FFFD -- silent corruption with no error anywhere. The
adapter now buffers chunks as bytes.

**`write` was absent**, making any chunked handler a runtime `TypeError`
rather than a compile error.

**`write` callbacks ran inline.** Node invokes them after the chunk is
handed off. A handler that writes its next chunk from each callback
therefore recursed synchronously and could overflow the stack, and a
throwing callback threw in the wrong execution phase. They are now
deferred.

**The encoding argument was ignored.** `write(s, 'latin1')` emitted
`0xc3 0xa9` where Node emits `0xe9`, corrupting the body and potentially
invalidating a `Content-Length` the handler had set. Both `write` and
`end` now parse their full overload set and pass the encoding through.
Node's own behaviour for an invalid encoding was established by probing
`ServerResponse` on Node 22 rather than assumed: it throws
`ERR_UNKNOWN_ENCODING` synchronously, so the adapter validates with
`Buffer.isEncoding` and lets `Buffer.from` raise the native error.

**Bodyless statuses were not suppressed.** Node discards writes after
`writeHead(204)` or `writeHead(304)`; this passed the buffered bytes to
the Fetch `Response` constructor, which throws for those statuses, so the
request became a 500 instead of the intended 204 or 304. Buffered data is
now discarded for 204, 205 and 304, and for `HEAD`.

None of the five reaches a caller today -- the document and runtime
handlers only ever end with JSON strings. They are traps laid for the
first handler that carries bytes, which is why they surfaced while
wiring the asset server backend, whose byte routes hit the first two
immediately.

Each is pinned by a test that fails when its fix is reverted.

* fix(storage): keep { client, bucket } working for the S3 byte store

`S3AssetByteStoreOptions.commands` was required and the store's methods
called it directly, so the published `./asset/s3-bytes` export stopped
accepting the `{ client, bucket }` construction it had always taken.
Anyone not going through `loadS3AssetByteStore` either failed to compile
or hit an undefined command factory at runtime.

Make `commands` optional. Omitted, the store resolves the AWS SDK's own
command constructors through a dynamic import on its first `write` /
`read` / `delete` and caches them, so `{ client, bucket }` works again.
Importing the module and constructing a store still never reach the
optional peer dependency: a deployment that does not select S3 never
resolves it. When the SDK cannot be resolved the call rejects naming
`@aws-sdk/client-s3` rather than crashing on an undefined property, and
the resolution is left uncached so installing the dependency fixes an
already-constructed store.

The lazily bound store is run against the byte-store contract, and the
new isolation tests pin both that construction does not resolve the SDK
and that an unresolvable SDK is reported by name.

With the break gone this branch's change to the package is additive, so
the version becomes a patch above main (0.2.4 -> 0.2.5) rather than the
minor it was carrying.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(persistence): run the asset collector in the shipped deployment

`PgAssetStore.remove`, and a `replace` that changes content, only stamp
`asset_blobs.unreferenced_at`. `AssetCollector.collect` is the only path
that deletes anything, and the persistence route deliberately did not
schedule it, on the grounds that scheduling is the deployment's decision.

The deployment this repository ships is docker-compose.yml: the app and
PostgreSQL, and nothing else that could make that decision. "The
deployment decides" therefore meant "it never runs", and ordinary asset
churn retained PostgreSQL bytes or S3 objects forever.

Schedule it from instrumentation.ts, which Next runs once per server
process — a route module can be instantiated more than once and has no
shutdown hook, so a schedule started from one is really started per
instantiation. Next 15 and later pick the file up with no config, so
next.config.ts is unchanged.

Collection is on by default with a 15-minute interval and the package's
one-hour grace period, so the Compose stack is correct with no operator
action. ASSET_COLLECTION_INTERVAL_MS, ASSET_COLLECTION_GRACE_MS, and
ASSET_COLLECTION_ENABLED change or disable it, each documented where it
is read. Without DATABASE_URL nothing is scheduled at all, and a failed
pass is logged and retried on the next tick rather than ending the
schedule or the process.

Several instances may collect concurrently: each candidate blob row is
re-checked and locked FOR UPDATE inside its own transaction, so
collectors serialize on the row rather than racing. The comment says so,
so nobody adds a distributed lock that is not needed.

The byte-store selection moves to lib/persistence/asset-byte-store so the
collector reclaims through the same layer the request path wrote through.
A collector on the PostgreSQL byte store while the route wrote to S3
would drop the blob row and orphan the object permanently.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(storage): bound each collector pass and report whether it filled

An unbounded collect() is sized by however long a deployment ran before
collection was scheduled: one statement selecting every eligible blob,
then a transaction and a byte-layer delete each, in a loop nothing
interrupts. The first pass over the Compose deployment that grew a
backlog before collection was scheduled is exactly that pass.

A pass now takes at most batchSize blobs (default one thousand) and
returns. Ordinary churn between two scheduled passes is far below the
cap, so a healthy deployment behaves as it did when a pass was
unbounded; a capped pass costs the remainder one scheduling interval,
which is what the interval is for.

collect() still answers with the count, and the count alone cannot tell
an empty backlog from a full batch -- a re-referenced or concurrently
taken candidate is skipped, so even a full batch can collect less than
batchSize. collectPass() returns the count together with capped, which
is exactly "this batch was full, run again"; a caller draining a
backlog loops while it is true.

Candidates are taken oldest-unreferenced first, with the content hash
as tiebreaker within one timestamp. Ordering by the hash alone would
starve: digests are uniformly distributed, so a high-sorting blob
waits behind every lower digest stamped after it, and under steady
arrivals those keep coming. The tests pin the ordering against a
planner that happens to answer the right rows without an ORDER BY.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(persistence): keep asset backend faults off the shared handler

Two review findings on the app wiring.

A concrete asset pool instance is single-lifecycle, but
resolveConfiguredAssetPoolStore handed the same object out again after
clearAssetPool() had closed it, reinstalling a dead store as the live
pool. The second handout now refuses loudly and points at the factory
form, which rebuilds on every resolution.

createPersistenceHandler eagerly awaited the asset byte store, so an
invalid ASSET_S3_BUCKET or an unresolvable AWS SDK rejected the shared
persistence handler and took document and runtime traffic down with an
optional backend. Byte-store construction is now deferred to the first
asset byte operation, and a failed construction is not cached, so the
next asset request retries -- the route's own no-poisoned-singleton
rule.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@openmaic/storage@0.2.5
2026-08-11 13:01:16 +08:00
xuyuanwei678 3f210095d5 fix(editor): add npm provenance metadata (#1098) @openmaic/editor@0.0.2 @openmaic/renderer@0.1.0 2026-08-11 11:51:34 +08:00
xuyuanwei678 26d280c9b7 fix(ci): stabilize Hyperframes lint command (#1097) 2026-08-11 11:27:21 +08:00
xuyuanwei678 c38da84ef6 feat(renderer): decouple editor UI from the app (#1072)
* docs: define renderer editor task 1 integration

* docs: plan renderer editor task 1

* feat: gate renderer slide editor

* feat: adapt renderer editor intents

* feat: connect renderer editor to slide surface

* test: complete renderer intent shape fixture

* fix: restore renderer editor action previews

* fix: render editor alignment guides

* perf: smooth renderer element dragging

* feat: align renderer hidden element behavior

* feat: add renderer canvas host commands

* feat: add renderer canvas context interactions

* test: cover renderer context menu interaction

* fix: preserve renderer canvas interaction semantics

* docs: design renderer text editing

* docs: plan renderer text editing

* feat(renderer): add ProseMirror text core

* feat(renderer): add rich text editor controller

* feat(renderer): render editable text in place

* feat(renderer): enter text editing on click

* feat: connect renderer rich text editing

* feat: finalize renderer text edit lifecycle

* fix: stabilize renderer text edit commits

* fix(renderer): move elements from selected borders

* docs: design renderer editing UI text toolbar

* feat(renderer): publish editing UI contracts

* feat(renderer): add controlled text format toolbar

* style(renderer): format editing UI contracts

* fix(renderer): address Task 2 review findings

* fix(renderer): handle zero font sizes in toolbar

* feat(renderer): add editing UI color picker

* fix(renderer): address Task 3 color picker review

* fix(renderer): commit native color input changes

* fix(renderer): coordinate native color picker interaction

* feat(renderer): anchor text toolbar to canvas elements

* feat(renderer): compose editable canvas with text UI

* feat(editor): use renderer text toolbar UI

* fix(renderer): make editing UI tests type-safe

* fix(renderer): harden editing UI integration

* feat(renderer): add editable text UI

* feat(renderer): add image and shape editing

* feat(editor): extend renderer editing controls

* feat(editor): add renderer chart insertion

* docs: design renderer latex editor

* docs: plan renderer latex editor

* feat(renderer): add latex editor contract

* feat(renderer): add latex editor dialog

* feat(renderer): compose latex insert and edit UI

* feat(editor): integrate renderer latex editing

* fix(editor): consolidate latex toolbar actions

* fix(renderer): show toolbar hover labels

* test(renderer): cover localized latex tooltips

* chore(renderer): hide latex sample palette

* feat(renderer): add video editing controls

* docs: define renderer audio editing

* feat(renderer): render audio elements

* feat(renderer): add audio editing UI

* feat(renderer): add preview audio controls

* feat(editor): add renderer element clipboard

* feat(renderer): decouple editor UI from app

* chore(renderer): release 0.0.6

* fix(renderer): clear editor CI gates

* fix(editor): adapt renderer canvas to media resolution API

* fix(i18n): complete French editor labels

* fix(renderer): hide toolbar vertical overflow

* chore(renderer): address review cleanup

* chore(docs): remove planning artifacts from pr

* feat(editor): add transactional core package

* chore(editor): ignore local test cache

* build(editor): publish core entrypoint

* refactor(editor): route renderer changes through transactions

* refactor(editor): split renderer editing into editor package

* ci(editor): register editor package for release

* docs(editor): document package dependencies

* ci: rerun validation after editor release registration

* test(editor): validate deletion against existing element

* fix(editor): address transaction and package review findings

* fix(editor): validate transactions and restore editor styles

* fix(editor): protect required updates and react styles

* fix(editor): validate required transaction fields

* fix(editor): prevent insert popovers shifting canvas

* refactor(editor): own renderer editing capabilities

* fix(editor): harden editing transaction boundaries

* refactor(editor): modularize editor UI integration

* docs(editor): harden third-party integration

* fix(editor): preserve video resize aspect ratio

* fix(editor): discard text draft before deletion

* fix(editor): preserve renderer picking and line presets

* chore(editor): remove unused pick-layer import
2026-08-10 19:56:24 +08:00
PercyandPercy b8bba7653e feat(courses): add folder grouping (#1005)
* feat(courses): add folder grouping behind a feature flag

Group courses into user-created folders on the home page. Folders are
device-local organization metadata kept in the existing IndexedDB
database (new folders + course-to-folder mapping tables, Dexie v16);
the course document aggregate owned by the DocumentStore is untouched.

- Create / rename (inline) / delete folders (keep or remove members)
- Move a course between folders via a hover menu
- Navigate into a folder with a breadcrumb back to all courses
- Search flattens the list, annotating each course with its folder
- Deleting a course cleans up its folder membership
- All UI strings internationalized across 9 locales
- Gated behind NEXT_PUBLIC_ENABLE_COURSE_FOLDERS (default OFF)

* test(courses): update db mocks and version assertion for folder tables

The folder feature adds the `stageFolders` table to the deletion cascade
and bumps the Dexie schema to v16. Update the affected test db mocks to
include the new table and bump the version assertion accordingly.

* fix(courses): stop course click-through when selecting from the move menu

The move-to-folder menu is rendered inside the course card's clickable
container. Selecting a folder item could let the click event reach the
card's onClick (opening the course) because only the trigger button
stopped propagation. Stop propagation on pointerdown for the trigger
and on click for every menu item so selecting a destination never opens
the course.

* fix(courses): open a dialog for the move-menu new-folder entry

The move-to-folder menu's inline "new folder" input never worked: a Radix
DropdownMenu is modal, so a raw <input> inside it cannot keep focus —
Radix closes the menu the instant the input is focused, dropping the
field before anything can be typed. Move the new-folder entry out of the
menu: it now asks the caller to open the existing NewFolderDialog and,
on confirm, moves the requesting course into the freshly created folder.

* feat(courses): drag-to-file folders and stacked cover thumbnails

Per review feedback, make folders feel complete:

- Drag a course card onto a folder tile to file it there. The tile turns
  into a clear drop target (ring + overlay) while a drag is over it. The
  hover 📂 menu remains as the accessible fallback for keyboard/touch.
- Replace the placeholder folder icon with a stable stack of up to three
  member course covers (most recently updated frontmost). Empty folders
  keep the folder icon; the name and course count are always visible.

* feat(courses): refine folder cover stack to a tidy fanned layout

Tune the folder-tile cover stack per review: front cover centered and
upright, rear covers peek out from alternating sides with a slight tilt
and reduced opacity, soft shadow and hairline ring. Reads as a neat pile
of course covers rather than an exaggerated fan.

* fix(courses): address review — dialog mount, empty-list, rename validation, partial-delete refresh, breadcrumb dedup

Per review (wyuc, CHANGES_REQUESTED):

1. Mount NewFolderDialog/DeleteFolderDialog outside the collapsible Recent
   subtree so they are reachable while it is collapsed; the New-folder
   button now expands the section before opening the dialog.
2. Keep the Recent/folder surface alive when the course list is empty so
   folders remain reachable and a first folder can be created.
3. Enforce folder-name validation (width + uniqueness) on rename as well
   as create, at the storage boundary (FolderNameError) and in the UI.
4. Refresh authoritative state on folder-delete both success and failure,
   so a partial "remove" failure does not leave stale cards/counts.
5. Drop the duplicated folder breadcrumb from the top section header;
   keep a single navigation breadcrumb in the content area.

Add focused storage tests covering rename validation, membership writes,
both deletion modes, and partial-failure propagation.

* fix(courses): address 2nd review — folder-view layout, drop highlight, validation parity, a11y, i18n

Per review (wyuc, CHANGES_REQUESTED on 9f77686f):

P1 — folder view layout: opening a folder compacts the hero and surfaces
the library content; the centered header shows a single path
"Recent > Folder name"; the empty-folder state sits directly below it;
clicking Recent returns to the root grid. The duplicate content
breadcrumb is dropped (kept only for search).

P2 — correctness & a11y:
- Clear the folder drop highlight on every drag exit (dragenter counter)
  and gate it on the text/stage-id payload.
- Duplicate-name check in the dialog is now case-insensitive, matching
  the storage boundary; FolderNameError is mapped to specific messages.
- Renames report the actual submitted width, not a hardcoded 0.
- createFolder/renameFolder run read-check-write in one read-write
  transaction so the uniqueness invariant cannot race across tabs.
- The move-menu trigger is focus-visible and pointer-coarse reachable
  (visible on touch / keyboard), not hover-only.

Extra:
- Empty library (no courses, no folders) shows a dedicated hint instead
  of the search-empty string.
- Translate all newly added folder strings across ja/ko/pt/ru/ar, fix
  Simplified-Chinese text in zh-TW, and add the emptyLibraryHint key.

* fix(courses): address 3rd review — ungate folders, fix empty-library hero, stabilize the bar

Per review (wyuc, CHANGES_REQUESTED on 19729038):

P1 — ship folders unconditionally: remove NEXT_PUBLIC_ENABLE_COURSE_FOLDERS,
the isCourseFoldersEnabled helper, the .env.example entry, and every
flag-on/flag-off UI branch. Folder metadata is always loaded; create,
move, drag, and folder navigation are always available. The IndexedDB
v16 schema stays intact.

P1 — full-screen landing hero only when the library is truly empty: the
hero uses min-h-[calc(100dvh-8rem)] only when there are zero courses AND
zero folders. A `hydrated` flag waits for both async loads before
selecting the layout, so folders arriving from storage do not flip the
hero from full-screen to compact.

P2 — geometrically stable centered bar: the Recent bar gets a fixed
height (h-9) so entering/leaving a folder (which toggles the New-folder
action and the folder path) does not shift the search/import controls.

* fix(courses): single stable library action bar across root, folder, and empty states

Per review (wyuc, CHANGES_REQUESTED on 860a1170):

P2 — remove the duplicate floating import controls. The hero section
rendered a second Import Classroom / PPTX cluster whenever the course
list was empty, duplicating the Recent bar's actions and floating above
it as the hero switched layout modes. Import now lives only in the
Recent action bar.

P1 — keep folder creation reachable for a truly empty library. The
Recent section is now always rendered after hydration (not gated on
having courses or folders), so a new user with zero of each can still
create the first folder or import. The empty-library hint renders below
the single stable action bar.

Invariant: one library action bar across root, folder, and empty
states; state changes alter the path and enabled actions only.

* chore: trigger CI after ready-for-review

* fix(courses): inherit folder context when importing from inside a folder

Per review (wyuc, CHANGES_REQUESTED on 37be4716):

[P1] Courses imported from inside a folder were silently placed at the
root. The import contract now carries the new stageId to the success
callback; the page captures the active folder when the file picker opens
(not when the async import resolves) and files the imported course into
that folder before the list refresh, so the card appears immediately and
the folder count increments. Root imports remain ungrouped. A failed
folder assignment surfaces an explicit error toast instead of silently
falling back.

* fix(courses): stable hero on folder delete, lightweight delete menu, breadcrumb count, empty-name validation

Per review (wyuc, CHANGES_REQUESTED on 5e12984f) + QA findings:

P1 — deleting the last folder must not expand the hero. The full-screen
landing hero is a first-visit treatment only: a session-scoped
"librarySeen" flag latches true once the library bar renders, so the
hero stays compact across all subsequent create/delete transitions.

P2 — replace the heavy two-card delete dialog with progressive disclosure.
Empty folder: an inline confirmation overlay on the card tile (matching
the course-delete pattern), with the empty-folder copy. Non-empty
folder: a compact dropdown beside the delete icon — "Delete folder only"
(courses move to unfiled) executes directly; "Delete folder and N
courses" opens a lightweight destructive confirmation. The full modal is
gone.

QA fixes:
- Breadcrumb count is now contextual (total at root, in-folder count
  inside a folder) instead of always showing the global total.
- Renaming a folder to empty/whitespace shows a "name cannot be empty"
  error and shakes, instead of silently exiting edit mode.

* fix(courses): cross-review cleanup — dead i18n keys, missing translations, delete-overlay reset

Post-commit cross-review (leak audit PASS, wyuc 19/19 PASS) found:

- Remove 6 dead i18n keys left over from the removed two-card delete
  dialog (deleteFolderDesc, deleteFolderUngroupTitle, etc.) across all
  9 locales.
- Translate all remaining English folder strings in ja/ko/pt/ru/ar
  (newFolderTitle, folderNameLabel, folderCreate, deleteFolderTitle,
  etc.).
- Fix zh-TW: convert ~17 simplified-Chinese folder strings to
  Traditional (新增資料夾/建立/刪除/etc.).
- Delete the orphaned "feature flag" comment (flag was removed earlier).
- Close the inline delete-confirm overlay before the async delete, so a
  failure leaves the card interactive instead of stuck behind the
  backdrop.

* fix(courses): clear drop highlight on drag end, map limit error in dialog

Two minor findings from cross-review:

- Escape-cancelled drags may not fire dragleave on every folder target,
  leaving a highlight ring. The course card now dispatches a
  'course-drag-end' window event on dragEnd (fires for both normal drop
  and Escape cancel); folder cards listen and clear their drop state.
- FolderNameError kind 'limit' (thrown at the storage boundary when
  FOLDER_COUNT_LIMIT is reached in a cross-tab race) now maps to the
  specific folderCountLimit message in the dialog instead of falling
  through to a generic hint.

Also fixes an SSR hydration mismatch: librarySeen is now initialized
to false and read from sessionStorage in useEffect (not in the
useState initializer).

* fix(courses): atomic folder removal, always-compact hero, no-cover placeholder

Per review (wyuc, CHANGES_REQUESTED on 470f5fea):

P2 — close the orphan-membership race in 'remove' mode. deleteFolder now
captures members, deletes the folder row, and clears all memberships in
ONE transaction BEFORE the course-deletion cascade. The folder is gone
from the moment the cascade starts, so a concurrent setStageFolder
(which checks existence in its own transaction) rejects the assignment.

P2 — remove the first-visit full-screen hero. The librarySeen flag
caused a visible layout jump on refresh (SSR renders full-screen, then
the effect reads sessionStorage and switches to compact). The hero is
now always compact (mt-[10vh]); no sessionStorage, no hydration
mismatch, no geometry regression.

P3 — distinct no-cover fallback for non-empty folders. A folder with
courses but no cached thumbnails now shows a neutral stacked-card
placeholder instead of the empty-folder icon.

Merged with latest main; no conflicts.

---------

Co-authored-by: Percy <percy@PercydeMacBook-Pro.local>
2026-08-10 17:56:56 +08:00
wyucand杨慎 1498ca52b2 fix(persistence): stop corrupting binary response bodies in the route adapter (#1088)
* fix(persistence): stop corrupting binary response bodies in the route adapter

The embedded persistence route bridges a Node `RequestListener` to the
Fetch API by hand. Its response object is cast with `as unknown as
ServerResponse`, so the compiler checks none of that surface, and two
parts of it were wrong in ways only a byte-carrying handler would hit.

`end` decoded any `Uint8Array` chunk with `Buffer.from(chunk).toString()`,
which is UTF-8. `ServerResponse.end` accepts a `Uint8Array`, and those
bytes are not necessarily valid UTF-8, so every unpaired byte became
U+FFFD -- silent corruption with no error anywhere. The adapter now
buffers chunks as bytes and builds the response from them.

`write` was absent entirely, making any chunked handler a runtime
TypeError rather than a compile error. It is part of the surface this
object claims to implement, so it is now implemented.

Neither reaches a caller today: the document and runtime handlers only
ever end with JSON strings, which took the string branch. Both are traps
laid for the first handler that carries bytes.

Both cases are pinned by tests that fail before this change -- the
binary one on a byte comparison, the chunked one with a 500 from the
TypeError. The existing adapter round-trip test, whose comment already
called this the most bug-prone code in the route, covered only the
string path.

* fix(persistence): make the route adapter faithful where it carries bytes

The embedded persistence route bridges a Node `RequestListener` to the
Fetch API by hand, and its response object is cast
`as unknown as ServerResponse`, so the compiler checks none of that
surface. Five divergences are closed here; the first two are the ones
that corrupt data, the last three were raised in review.

**`end` decoded bytes as text.** It ran `Buffer.from(chunk).toString()`
on a `Uint8Array` chunk, which is UTF-8, so every byte outside that
encoding became U+FFFD -- silent corruption with no error anywhere. The
adapter now buffers chunks as bytes.

**`write` was absent**, making any chunked handler a runtime `TypeError`
rather than a compile error.

**`write` callbacks ran inline.** Node invokes them after the chunk is
handed off. A handler that writes its next chunk from each callback
therefore recursed synchronously and could overflow the stack, and a
throwing callback threw in the wrong execution phase. They are now
deferred.

**The encoding argument was ignored.** `write(s, 'latin1')` emitted
`0xc3 0xa9` where Node emits `0xe9`, corrupting the body and potentially
invalidating a `Content-Length` the handler had set. Both `write` and
`end` now parse their full overload set and pass the encoding through.
Node's own behaviour for an invalid encoding was established by probing
`ServerResponse` on Node 22 rather than assumed: it throws
`ERR_UNKNOWN_ENCODING` synchronously, so the adapter validates with
`Buffer.isEncoding` and lets `Buffer.from` raise the native error.

**Bodyless statuses were not suppressed.** Node discards writes after
`writeHead(204)` or `writeHead(304)`; this passed the buffered bytes to
the Fetch `Response` constructor, which throws for those statuses, so the
request became a 500 instead of the intended 204 or 304. Buffered data is
now discarded for 204, 205 and 304, and for `HEAD`.

None of the five reaches a caller today -- the document and runtime
handlers only ever end with JSON strings. They are traps laid for the
first handler that carries bytes, which is why they surfaced while
wiring the asset server backend, whose byte routes hit the first two
immediately.

Each is pinned by a test that fails when its fix is reverted.

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
2026-08-10 16:14:42 +08:00
wyuc 3b4567076f refactor(generation): consume the package everywhere and delete lib/generation (#1090)
Part D of #1057 — the switch. All consumers import @openmaic/generation; lib/generation deleted (net -11,900 lines). PBL wiring preserved (single-call -> classified fallback -> injected loop), streaming outline route deduplicated onto buildOutlinePrompt, serverExternalPackages wired, and the change verified by a real-model four-kind end-to-end run against the production build with a rendered-classroom screenshot check. Closes #1057.
@openmaic/generation@0.3.0
2026-08-10 03:58:33 -04:00
wyuc b64e787749 feat(generation): move scene generation and PBL single-call planning into the package (#1087)
Part C of #1057. Scene content/actions/widget generation, retry, action parsing, interactive post-processing, buildCompleteScene with injectable sceneId, and PBL single-call planning re-seated on AICallFn (pure core + kernel + package-relative planner assets; loop planner stays app-side behind the injected pblLoopFallback with provider/abort classification). Purely additive — app source untouched until Part D. Package 0.2.0.
@openmaic/generation@0.2.0
2026-08-10 00:58:35 -04:00
wyucand杨慎 6825363de1 feat(storage): server asset registry over a pluggable byte layer (#1007) (#1077)
* docs(storage): specify the asset registry HTTP contract (#1007)

Adds the contract document and the server-side store, byte-layer, and
handler types for the asset layer, the only layer with no server backend.
Runtime behaviour is unchanged: the new modules are deliberately
unreferenced until the backends land.

Reclamation moves offline, and that decision carries the design. `remove`
deletes a registry entry and nothing else -- it never reads, counts, or
deletes bytes. A separate collector takes byte rows that have had no
references for a grace period. Three things follow. The hardest race in
the design disappears, because no request can delete bytes another
request is adopting, so no request path needs a lock on the byte row and
the write path never has to reconcile taking one with never branching on
whether the bytes already existed. A side channel closes, because a
`remove` that only deletes a row costs the same whether or not another
principal holds the same bytes. And nothing is given up: under global
deduplication `remove` could never promise the bytes were destroyed, so
deleting them synchronously bought far less than it appeared to. The cost
is that storage grows until the collector runs, which a deployment must
configure rather than inherit.

Offline reclamation is also what makes the byte layer pluggable, so this
keeps #1007's pluggable-backend model rather than narrowing it. With
collection out of every request path, the requirement on a byte layer is
an ordering rule rather than an atomicity one: bytes are written before
the row that references them, deleted after the last row referencing
them, and the reference count is serialized on the blob row that every
implementation keeps regardless of where bytes sit. Two implementations
are planned -- bytes in a column of the transactional store, and an
object store keyed by content hash -- which is why the interface exists;
an interface with one implementation is not a seam, and this package has
removed such a seam before. The browser backend keeps no such seam,
because it reclaims inline and its bytes therefore cannot leave its
transaction; four comments that stated this as a storage difference now
state it as a reclamation one.

Metadata crosses the wire as a multipart part. It is open-ended and
callers already fill it with generated-narration text and image prompts,
so a query parameter would put unbounded user content into the request
target and into every intermediary's logs. That also motivates a rule
the other layers do not need: metadata must never appear in a URL, a
response header, a log line, or an error message. Metadata gets a value
domain, and the document is explicit that this is a capability
reduction -- the browser backend's structured clone carries Date, Map,
Set, and cycles faithfully, so a deployment moving to a server backend
must audit its metadata rather than expect a quietly lossy path to
tighten.

Resolution yields a client-minted object URL, superseding the resolution
that a server deployment resolves to a self-hosted proxy path. A media
element sends only ambient credentials, so a proxy path works for
cookie-session deployments and silently fails for header-authenticated
ones; closing that gap means minting URL tokens, a second credential
form inside a storage library. Fetching bytes over the client's own
authenticated request needs none and removes a class of hash-disclosure
surface. The client obligations that make it preserve the browser
backend's semantics are specified, and split into the two the shared
suite already pins and the four an HTTP backend must pin itself. The
cost, no range requests and no progressive playback, is stated.

Channels that could carry a principal or a content hash are closed
structurally rather than by enumeration: the route table admits no
segment that could carry either, and no route takes a query string at
all. Headers are closed by not reading them -- the contract assigns no
meaning to any header beyond content type and length, and a handler must
not derive identity from one outside its authenticate hook. Rejecting
header names by pattern was tried and removed: no pattern broad enough
to catch X-Remote-User spares User-Agent, and the forwarded-identity
headers such a rule would reject are exactly what an authenticating
reverse proxy injects for that hook to read.

Ids that cannot be a path segment are resolved by the client as misses
rather than raised as errors, a deliberate divergence from the KV
contract. An unknown id is a miss by this layer's id-domain rule and the
shared suite pins the empty string as one, so a client that threw would
fail that suite and reintroduce the shape oracle the rule prevents.

Also specified: response values may not derive from the shared byte row
(Last-Modified off that row is an existence oracle), a collision-
resistant digest is required because writes are unconditional and an
object layer keys on that digest, byte responses carry the revision and
refuse Range, HEAD errors carry the code in a header since they have no
body, quota exhaustion has a status and a code, and relabelling a
non-renderable type is not refusal -- resolve still returns a URL.

* feat(storage): add pluggable server asset store (#1007)

* fix(storage): write asset bytes only after claiming the blob row (#1007)

The write path wrote bytes twice: once before the registry transaction
and once inside it, after the upsert that claims the blob row. Only the
second one carries any safety, and the first is what a large-media
deployment pays for on every upload.

The ordering that matters is claim, write, reference. Claiming the blob
row takes its lock, so a collector already holding that lock finishes
before the write proceeds; writing bytes before the claim instead lets
the collector delete them while the claim waits, leaving a fresh entry
that points at nothing. Writing them before the entry keeps every
surviving entry backed by bytes that were actually stored. Dropping the
first write preserves both ends of that order and removes a crash window
that produced orphans for no reason.

For a byte layer inside the transactional store this also means a failed
write now leaves nothing at all, rather than an orphan the collector had
to sweep later.

The four tests this changed were each pinned to the removed write:

- the statement-sequence test hardcoded the old four-statement shape; it
  now pins claim, write, entry, and still asserts the two existence paths
  are identical, which is the property that matters;
- the quota test asserted two writes per accepted put;
- the crash-window test asserted an orphan that a transactional byte
  layer no longer produces. It is now two tests: a transactional layer
  leaves nothing, and a non-transactional one strands an object with no
  blob row -- which the collector deliberately cannot see, since
  recovering it is deployment housekeeping rather than reference
  counting. It also no longer depends on state left behind by earlier
  tests in the file;
- the collector-race test drove its interleaving off the removed write.
  It was choreography rather than concurrency in any case: PGlite is
  single-connection, and the test serialized transactions, so no row lock
  was ever contended. It is replaced by the two invariants that actually
  make the race safe -- the byte write is unconditional, and bytes the
  collector has already removed are re-stored by an adopting put. The
  contended-lock case belongs to the real-PostgreSQL suite, where it can
  be tested rather than mimed.

Reverting the unconditional write fails two of them, so they pin it.

The contract said a failed write leaves no partially written bytes while
its byte-layer section permits exactly that orphan; the two are
reconciled, and the ordering rule now states the claim step and the
unconditional write it depends on, along with the cost that follows --
the byte write happens with the registry transaction open, so an object
store holds a row lock across a network upload.

* test(storage): fix the real-PostgreSQL asset tests against the new write order

Both were written against the removed pre-transaction byte write, and
both failed once it went away -- one of them by hanging, which also
timed out the next test's TRUNCATE.

The lock-contention test waited for the adopting put to start writing
bytes before releasing the collector. Bytes are now written after the
upsert that claims the blob row, and the collector holds that lock, so
the write could never start: a circular wait. It now waits for a backend
to actually appear in pg_stat_activity blocked on a lock, which is the
condition it meant to wait for, observed rather than signalled. Its final
assertion carries real weight now -- had the bytes been written before
the claim, the collector would have deleted them and the adopting entry
would resolve to nothing.

The orphan test asserted that a failed registry transaction strands
bytes. With this byte layer inside the registry's transaction it strands
nothing, so it now asserts that instead. Object storage does strand an
object, and that case is covered where it belongs.

All of this ran against PostgreSQL 16: 781 tests pass with 3 skipped,
and those 3 pass against a real S3-compatible server. Nothing in the
suite is now unverified for lack of infrastructure.

* feat(storage): add asset HTTP backend (#1007)

* docs(storage): separate trusting a header from reading one (#1007)

The header rule said a server reads exactly Content-Type and
Content-Length, while the size rules require reading Content-Encoding in
order to reject it -- the two cannot both be literal, and the
implementation had to pick one.

What the rule means is narrower: no request header may be trusted to say
who is asking, outside the authenticate hook. Reading a header to decide
how to frame or refuse a request is a different thing, and the transport
headers are read and acted on.

* chore(storage): bump to 0.2.3 for the asset server backend (#1007)

Additive: new asset registry, byte layers, collector, HTTP handler and
client, and their entry points. Nothing existing is removed or changed
incompatibly, so this is a patch under the pre-1.0 rule.

* fix(storage): close four defects found reviewing the asset backend (#1007)

**Concurrent writes could exceed a principal's logical quota.** The check
ran on the pool before the write transaction, so two concurrent puts both
read the old total and both passed; enough concurrency amplified it
arbitrarily. It now runs inside the write transaction behind a
transaction-scoped advisory lock on the principal, taken only when a quota
is configured -- a branch on deployment configuration, never on data.
Pinned by a real-PostgreSQL test: four concurrent six-byte writes against
a ten-byte quota, of which exactly one may be accepted. Removing the lock
accepts three.

**A part name smuggled inside a quoted filename was read as a real part
name.** `Content-Disposition` was scanned with a regex that does not
understand quoted strings, so `filename="x; name=meta; y"` parsed as a
`name` parameter here while an RFC-aware intermediary sees an unnamed
part. That is exactly the parser differential the contract requires be
rejected. Replaced with a tokenizer that honours quoted strings and
escapes and requires exactly one real `name`.

**Any S3 404 was reported as an absent object.** `NoSuchBucket`, a
misdirected endpoint, and a revoked access point all answer 404 while the
bytes still exist, so a storage outage surfaced as `404 ASSET_NOT_FOUND`
and a caller clearing its reference on that would turn it into real data
loss -- the same failure the contract spells out for `401`. Only
key-absent codes map to a miss now; everything else propagates to
`500 INTERNAL_ERROR`.

**Exceeding `maxParts` answered `400` rather than `413`.** It is one of
the declared multipart resource limits and now answers like the other
three.

Verified against PostgreSQL 16 and a real S3-compatible server:
840 tests, none skipped.

* fix(storage): repair two defects the previous fix round introduced (#1007)

Both were created by the fixes themselves rather than surviving them,
which is the failure mode a second review round exists to catch.

**An over-quota replace answered 500.** Moving the quota check inside the
write transaction put it under a catch that re-threw only
AssetNotFoundError, so AssetQuotaExceededError was collapsed into a
generic registry failure and the handler mapped it to INTERNAL_ERROR --
losing the status and code the contract gives that condition. Both typed
errors now survive the catch.

**The hand-written disposition tokenizer accepted malformed input and
trusted it.** The quoted-value loop never checked that it found a closing
quote, so `name="meta` ran to the end of the header and returned `meta`;
`name="meta"junk` returned `meta` as well; and an extended `name*` form
competing with a plain one was silently ignored. Each recreates the
parser differential the tokenizer was written to remove. A quoted value
now requires its closing quote, rejects a dangling escape, and permits
only whitespace and a separator after it, and an extended name form is
refused rather than resolved.

Also widened the quota lock key from `hashtext` to `hashtextextended`.
The 32-bit form collides -- two unrelated principals sharing a key would
block each other for the whole transaction, which spans a byte write that
may be a network upload.

Regression tests cover all three: an over-quota replace raises the quota
error and leaves the original bytes, and each of the three malformed
dispositions is refused.

Verified against PostgreSQL 16 and a real S3-compatible server:
842 tests, none skipped.

* fix(storage): narrow the multipart disposition surface and the error boundary (#1007)

Three review rounds have each found a defect in the part-disposition
parser: a regex that misread quoted strings, then a tokenizer that
accepted an unterminated quote, then RFC 2231 continuation forms
(`name*0*=UTF-8''bytes`) and malformed empty parameter slots. Patching it
a fourth time would have been the wrong move.

This contract needs exactly one parameter, so it now accepts exactly one:
a part disposition is `form-data` plus a single `name` whose value is
`meta` or `bytes`, and everything else is a validation failure. That
removes the continuation forms, the encoded forms, the empty slots, and
`filename` -- the parameter every one of these attacks travelled in --
without having to model RFC 2231 at all.

That rule turned out to bind our own client too: it emitted a fixed
`filename="asset"`. Nothing reads it, and the contract already forbids
deriving a response disposition filename from caller data, so the client
no longer sends one.

Separately, `instanceof` was being used as provenance. The byte layer is
pluggable, so a byte store raising a same-named `AssetQuotaExceededError`
or `AssetNotFoundError` inside the transaction was re-thrown as a logical
registry outcome, carrying the byte layer's own message to a direct
caller. The registry's checks now raise module-private sentinels, and
only those are converted -- at the boundary, into fresh public errors
with this package's fixed messages. Anything else, whatever its name,
collapses to the generic registry failure.

Also pinned the advisory-lock key width. Reverting `hashtextextended` to
the 32-bit `hashtext` left every test green, so the emitted lock
statement is now asserted. A SQL-text assertion is deliberate here: the
behavioural difference is a collision between two particular keys under a
database-internal hash, which is not a stable thing to assert against.

Verified against PostgreSQL 16 and a real S3-compatible server:
845 tests, none skipped.

* fix(storage): define multipart disposition grammar (#1007)

* fix(storage): delegate multipart parsing to Fetch (#1007)

* fix(storage): require both multipart parts to be files (#1007)

Delegating the framing to the platform parser cost two rules for a
metadata part sent without a filename: such a part comes back as a
string with its headers discarded, so its `application/json` type could
no longer be checked, and its bytes had already been replacement-decoded
-- `{"x":"<0xFF>"}" was stored as `{"x":"\uFFFD"}` instead of refused.

The bytes part already had to be a file for binary safety. Metadata is
now symmetric: the client sends a fixed filename on both, the server
requires both to be files, and with the part preserved the media type is
checked and the bytes are decoded fatally before parsing. Neither
filename is read anywhere.

The limits are also described honestly now. `maxParts`, `maxMetaBytes`
and `maxAssetBytes` are validated after the parser has materialized
every part, so they bound what is accepted rather than what is parsed,
and `maxRequestBytes` is what caps memory. The contract said `maxParts`
bounded parser work; it does not. Restoring that would mean adopting a
streaming parser with its own grammar, which is the disagreement with
intermediaries that delegating to the platform exists to remove -- so
the trade is stated rather than reversed.

Also corrected the media-type wording: the two branches are metadata
present versus absent, and retention of an untyped replacement's prior
type is a browser-backend behaviour the HTTP path cannot reproduce,
because a conforming parser supplies a default type for a file part
whose header omits one.

Verified against PostgreSQL 16 and a real S3-compatible server:
853 tests, none skipped.

* docs(storage): the asset server backend ships in this branch (#1007)

The contract still described the server backend as not yet shipped, which
this branch is what changes. The conformance server remains test-only.

* fix(storage): make asset HEAD bytes-free and pin reads (#1007)

* fix(storage): harden asset write boundaries (#1007)

* chore(storage): bump to 0.2.4 for the asset server backend (#1007)

main merged in a 0.2.3 of its own, so this branch's bump no longer
increased the version. Additive relative to the merged base -- new asset
registry, byte layers, collector, HTTP handler and client, with nothing
removed or narrowed -- so a patch bump under the pre-1.0 rule.

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
@openmaic/storage@0.2.4
2026-08-10 12:30:06 +08:00
wyuc 12814079ba refactor(pbl): consume the contract's PBL project types (#1079)
Third and final PR for #1061. lib/pbl/v2/types.ts becomes a barrel over @openmaic/dsl plus a recursive app-side runtime overlay (plain Omit & Overlay after the dsl contract types were made precise again — index signatures removed, schema tolerance moved into generation post-processing). Write validator byte-identical to base; negative type tests and keyof sentinels pin the contract surface. dsl 0.8.0, storage 0.2.3 (comment-only). Closes #1061.
@openmaic/storage@0.2.3 @openmaic/dsl@0.8.0
2026-08-09 23:15:03 -04:00
fc66cc0603 feat(runtime): add learner whiteboard RuntimeStore foundation (#1075)
* feat(runtime): add whiteboard RuntimeStore foundation

* Potential fix for pull request finding 'Comparison between inconvertible types'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* fix(runtime): reject invalid whiteboard appends before commit

* fix(runtime): preserve special JSON keys during canonical clone

* Potential fix for pull request finding 'Invalid prototype value'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* feat(runtime): add minimal learner whiteboard mutation

---------

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-09 12:32:28 -04:00
5d5acd6215 feat: claude search (#393)
* Added search tool support for Claude

* New locales for i18n

* Updated translations

* Prevent default web search base URLs from being prepopulated

* fixed coding style issues

* fixed coding style issues

* fixing lint errors

* fix: add SSRF protection to fetchPageContent in claude.ts

Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/099e53fb-e1db-49f4-af7f-4f17cb3d364a

Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com>

* fix: address all remaining review feedback from PR review thread

Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/e1ab8fe2-fe0d-45a5-a0af-30945b99c268

Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com>

* fix: rename ambiguous variable names in settings store for clarity

Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/e1ab8fe2-fe0d-45a5-a0af-30945b99c268

Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com>

* fixing lint errors

* fix: address 5 items from second PR review thread

Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/09469d80-8e91-449b-8bc2-9c11f83555df

Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com>

* fixing model verification error

* fix: address 3 items from third PR review thread

Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/0aa49d19-df5b-4a08-a7e1-2295ae9cfc38

Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com>

* fixing lint errors

* fix: resolved Copilot review comments

* fix: returning string promise for ssrf-guard

* fix: returning string promise for ssrf-guard

* fix: updated locales, added allowed_callers for claude search tools, handle duplicated claude model keys

* fix: updated locales, added allowed_callers for claude search tools, handle duplicated claude model keys

* updated tests

* using Anthropic SDK for search and connection test in search settings.

* uses @ai-sdk with a custom fetch including allowed_callers in payload instead of adding Anthropic SDK as a direct dependency

* fixed format

* removed @anthropic-ai/sdk as direct dependency

* fix: resolve TypeScript errors in claude web-search tests

Cast mock.calls through unknown to avoid tuple-index type errors on
Vitest mock objects whose parameter types are inferred as empty tuples.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add Claude as a web-search provider on the multi-provider framework

Forward-port of the original Claude web-search work onto main's
searchWeb dispatcher, per review guidance on #393:

- lib/web-search/claude.ts: searchWithClaude via the AI SDK Anthropic
  provider and Anthropic's native web_search server tool, called through
  callLLM for usage accounting. Returns the model's answer plus cited
  sources deduplicated by URL. The page-content enrichment fetch from the
  original PR is removed entirely, resolving the P1 SSRF finding (no
  server-side fetches of attacker-influenced result URLs remain).
- allowed_callers: ["direct"] is still injected at the fetch layer; the
  AI SDK's provider-defined web_search tools do not emit it.
- Tool version picked from the model: web_search_20260209 for 4.6+
  Opus/Sonnet, basic web_search_20250305 for Haiku/older; maxResults maps
  to the tool's max_uses.
- Registry/dispatcher/route wiring: claude provider entry, claudeModelId
  passthrough (web-search route, classroom generation, generation
  preview), WEB_SEARCH_CLAUDE_* env config with server-pinned model via
  WEB_SEARCH_CLAUDE_MODELS, and client base-URL allowlisting for
  api.anthropic.com.
- Settings: per-provider modelId with a model selector in the Claude
  panel; locale keys added to all ten languages.
- Tests: new adapter suite plus claude cases in the dispatcher,
  constants, route, provider-config, web-search-config, and settings
  store suites.

Closes #392

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E9KqAGmKCtBwMC4HtT4JKE

* fix(web-search): trim whitespace around the Claude API key and base URL

Keys are pasted by hand into Settings, and nothing in the client key
path trimmed them: a trailing newline or space was sent verbatim as the
x-api-key header value and came back from Anthropic as 'invalid
x-api-key', with nothing in the message pointing at whitespace.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(web-search): correct Claude adapter tool selection, base URL, and timing

Addresses the four P2 review findings on #393:

- Route dated Claude 4.0 ids (claude-sonnet-4-20250514,
  claude-opus-4-20250514) to the basic web_search_20250305 tool. The
  blocklist regex missed them, so they were sent web_search_20260209,
  which pre-4.6 models reject. Replaced with an allowlist of the models
  that have dynamic filtering, so unrecognized ids also fall back to the
  basic tool -- the safe direction, since 4.6+ models accept both.

- Pin allowed_callers: ["direct"] only on web_search_20250305. Injecting
  it into every request disabled the code-execution caller that provides
  dynamic filtering on web_search_20260209.

- Normalize the bare Anthropic root to /v1 before handing the base URL to
  the SDK, which appends /messages verbatim and would otherwise 404. The
  adapter is the single choke point for the client allowlist, the server
  env var, YAML, and the default; other hosts pass through untouched.

- Return responseTime in seconds, matching every sibling adapter's
  WebSearchResult contract instead of exposing milliseconds for Claude.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-08-09 07:34:12 -04:00
732c36364a fix(release): hand published versions to marker job (#1076)
Co-authored-by: Bryan Nathan <bryan@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-08-09 05:36:43 -04:00
wyuc 0ae21aa559 refactor(pbl): retire v1 write path and dead UI, converge on v2 (#1060)
* refactor(pbl): retire v1 write path and dead UI (#1058)

Remove the legacy generator, renderer branches, chat API, and compatibility mirror writes.

Keep stored v1 scenes readable through documented legacy adapters, and surface ordinary PBL planner double failures instead of degrading.

* fix(pbl): contain planner failure to the scene, support legacy scenes in action regeneration, tighten legacy guards

- Contain typed planner failures per scene while preserving retry and skip behavior.
- Upgrade legacy PBL content before regenerating scene actions.
- Apply video-export import boundaries to legacy readers.
- Harden legacy rendering and empty-state handling.
- Restore legacy upgrade compatibility coverage.
- Remove retired translations and refresh fallback narration and legacy cover documentation.

* fix(pbl): keep tolerant legacy configs renderable, guard invalid chat timestamps

* chore(dsl): bump to 0.6.3 for README accuracy fix

The README no longer references the deleted PBLProjectConfig symbol; the version gate requires a bump for any publishable package input change.

* fix(pbl): harden legacy upgrade and persistence contract

* fix(pbl): align action prompt with v2 workflow and validate project containers

* fix(i18n): remove retired PBL v1 keys from the new fr-FR locale

The fr-FR locale landed on main after this branch removed the v1 UI key families; prune the same 38 keys so all 10 locales align with en-US.

* fix(generation): sync packaged pbl-actions template with the v2 wording

The generation package that landed on main carries a byte-identical copy of the app's pbl-actions template; sync the v2 rewording and bump the package version.

* fix(pbl): normalize legacy content and preserve planner status

* fix(pbl): propagate provider status and ground action prompts

* fix(pbl): classify planner failures for retry, default the project summary prompt input

* fix(pbl): fall back to legacy data when v2 payloads are damaged, harden summary building

* refactor(pbl): resolve hybrid content through a validity-aware resolver

* fix(pbl): keep one v2 acceptance criterion across validator, resolver and persistence

Persistence gates on the same container predicate as the validator and the renderer's resolver: payloads the renderer will not treat as v2 are inert bytes everywhere, avoiding runtime side effects for documents the write barrier would reject. An explicit null proficiencyAssessment stays processable.

* fix(pbl): require runnable projects and harden legacy fallback

* test(pbl): exercise the real save reload and hydration chain

* fix(pbl): require an instructor and usable microtasks before v2 takes precedence

* fix(pbl): fall back to the loop planner unless the provider failed or the user cancelled

* fix(pbl): require role and task identities before v2 takes precedence

Runtime normalization can repair status and create the instructor thread, but it cannot synthesize the role id that binds the thread, the microtask ids used by lookup and update paths, or the labels the workspace renders. The runnable predicate and its video-export mirror now require them, so identity-less payloads fall back to usable legacy data.
@openmaic/generation@0.1.2
2026-08-08 23:26:17 -04:00
jackefn b4834f5c74 feat(document): add transform pipeline foundation (#920)
Add a provider-neutral document transform pipeline with execution history, normalization, conservative noise removal, cancellation, and failure policies.

Keep document structure and outline generation out of the canonical artifact until a concrete consumer and evaluation criteria exist.
2026-08-07 03:00:48 -04:00
wyuc 7aba740a64 refactor(app): consume the contract's interactive and widget types (#1073)
Second of three PRs for #1061 (after #1070). WidgetType re-exports from @openmaic/dsl; the app's rich widget configs extend WidgetConfigBase; InteractiveContent aliases the contract generic; the write validator adopts the contract's html-or-url rule while staying lenient over historical widget shapes (with a primitive-config barrier matching the hydration crash class); widget-config extraction seeds the contract type; @openmaic/generation drops its WidgetType copy and gains the dsl dependency (0.1.1).
@openmaic/generation@0.1.1
2026-08-06 22:53:16 -04:00
wyuc 587deb2bde feat(dsl): promote the interactive and pbl content kinds into the contract (#1070)
First of three PRs for #1061. InteractiveContent (generic widget-config extension, open schema surface) and PBLContent/PBLProject (design-time definition plus planner-seeded skeleton; opaque deprecated legacy payload; historical runtime-overlay fields tolerated) enter @openmaic/dsl with validators, normalizePBLProject, guards, and a four-kind scene schema. DSL_VERSION unchanged (additive); npm 0.6.3 -> 0.7.0.
@openmaic/dsl@0.7.0
2026-08-06 08:04:17 -04:00
ceb05adb41 feat(media): allocate generated assets through the registry (#1007 part 2, step b) (#1039)
* feat(media): establish shared asset ownership primitives

Introduce global browser asset-pool ownership, asset-reference collection, stage reclamation planning, and lease-based URL access. Carry allocated media identity through storage and generation boundaries with regression coverage.

* fix(media): enforce safe resolution across every consumer

Route image, video, thumbnail, presentation, and video-export consumers through one resolution state machine. Prevent opaque allocated or generated references from reaching render and export sinks, with fallback and ownership tests.

* fix(media): protect document-owned assets across mutations

Allocate pool bytes before compatibility writes and document commits, then roll back uncommitted generations safely. Preserve document ownership across edits, retries, imports, speech generation, scene changes, and stage deletion.

* fix(media): scope retries and tighten ownership guard

Scope retries to the target scene and slide and refuse ambiguous shared-reference mutations. Expand retry rendering coverage and keep direct pool URL resolution behind the shared lease owner.

* fix(storage): avoid nested lock during stage cleanup

Execute prepared reclamation plans against an explicitly deleted document so the compatibility cascade cannot re-enter the per-document lock. Cover deletion of stage-owned media rows even when the document has no references.

* refactor(media): confine reclamation to stage deletion

Remove inline pool and compatibility-row cleanup from element, speech, scene, and audio replacement flows. Keep whole-stage reclamation behind explicit stage ownership, preserve stage-less legacy audio rows, and document deferred document-truth sweeping.

* fix(media): close retry and resolution gaps

Restore shared source tasks after successful forks, scope retries across both whiteboard locations, and wait for parallel TTS workers before rollback. Resolve background media through import, rendering, and PPTX export paths while keeping retry controls visible over last-good bytes.

* test(media): execute the consumer safety matrix

Replace source-substring checks with resolver seam execution across all six UI consumers, stage hydration, and both exporters. Pin each rollback layer independently and harden the ownership guard against aliased pool imports.

* chore(packages): publish the additive DSL field

Bump the DSL patch version for the optional speech-action field. Keep the transitional reclamation policy app-owned and document the legitimate transaction rollback removals there.

* fix(media): make retry rollback task-safe

Delay allocation task re-keying until final document reconciliation succeeds, restore shared source tasks on failed forks, and require exact stage-whiteboard targets before falling through from a missed scene.

* fix(media): clear private assets and refresh leases

Delete the asset-pool database during the confirmed local-data wipe. Notify the app lease layer after same-id replacement so mounted consumers re-resolve current bytes without reaching into storage internals.

* test(media): pin closure safety guards

Exercise the CSS allocation boundary, unfiltered legacy-row ownership, delete-and-undo byte survival, and real distinct consumer seams. Restore the video-only manifest overwrite condition and share the direct video resolution hook across both element variants.

* test(media): narrow rollback element assertion

Narrow the reconciled slide element to an image before checking its source so the rollback regression remains type-safe under the full root compiler configuration.

* fix(media): close lease refresh races

Gate the first batch publication by unique resolved refs, then publish every replacement snapshot without mutating the prior React state object. Serialize invalidation behind pending releases, evict rejected refreshes, and register replacement observation at the pool boundary.

* fix(media): reopen pool after clear failures

Always evict the singleton once its store has been closed, including blocked and failed database deletion paths. Report blocked deletion as deferred and prove a later write uses a fresh live store.

* fix(media): isolate shared retry progress

Track forked regeneration under the selected element until it receives a fresh asset identity, leaving the shared source task and bytes untouched. Surface targeted failures through renderer task lookup and skip the redundant fork reconciliation lock.

* fix(media): close final asset retry gaps

Keep blocked asset clears fail-loud until a successful retry, with actionable settings guidance.

Clear failed shared-fork state across durable and live key spaces, and pin renderer lookups, lease publication identity, and committed-ref rollback protection.

* fix: preserve actionable retry failures

Localize the blocked cache-clear recovery hint across every supported locale and pin the deferred-error mapping.

Retain durable fork failure rows while retries run, deleting them only after successful generation so unstructured failures survive reload.

* fix(media): hydrate legacy stored video thumbnails

Home-page recent-video thumbnails regressed for legacy Dexie mediaFiles
rows keyed by gen_vid placeholders: the reworked hydration resolved the
row's bytes through the sealed resolver but never surfaced the stored
blob (and its poster) as object URLs for the preview card, so the CI
recent-video-thumbnail e2e specs found no visible element.

Hydration now materializes legacy stored video rows into blob URLs for
both the element src and poster while keeping the resolver invariants:
opaque refs still never reach a DOM src, and concrete addresses are
never blanked. Unit pins cover the seam so the vitest suite catches this
class without a browser.

* fix(media): resolve sole restored legacy video

Classroom playback restored tasks by exact document media references. Legacy gen_vid references can outlive the key used by the one persisted video row, leaving the player on a placeholder even though bytes were restored.

Select the sole completed stage video only for legacy sequential refs after exact and reconciled matches. Keep exact failures authoritative, refuse ambiguous candidates, and cover success, ambiguity, and failure precedence in unit tests.

* fix(media): scope legacy video recovery

Decide restored legacy video recovery once from the complete document and record it through the shared task lookup consumed by playback, editing, and resolved slides.

Keep ambiguous documents as placeholders, apply the same decision to thumbnail hydration, and preserve exact failure precedence.

* fix(media): exclude claimed video recovery tasks

Model restored legacy recovery as a two-pass match across document video elements and task rows. Remove tasks claimed by exact, targeted, or placeholder lookup before applying the sole-candidate fallback, and cover the ownership/cardinality matrix.

* fix(media): unify video element resolution

Centralize source, task, poster, and legacy recovery decisions for every video consumer. Ensure direct URLs win over opaque refs and play_video waits on element-targeted retry tasks.

* refactor(media): route legacy recovery through resolver

Let document-aware consumers request legacy video recovery through the unified element binding API, keeping thumbnail hydration on a single decision path.

* fix(media): preserve import refs and prefer pool bytes

Recognize unambiguous extensionless relative media addresses during classroom import. Resolve allocated export and thumbnail assets from the shared pool before falling back to lagging compatibility rows.

* fix(export): preserve concrete video sources

Route PPTX video elements through the shared media binding resolver so an unresolved opaque reference cannot replace a playable source. Complete browser and persisted-store cleanup when asset-pool deletion is deferred, while retaining distinct hard-failure behavior.

* fix(media): fork retries without exclusive ownership

Enumerate logical asset owners across every persisted document before allowing global pool replacement. Thread explicit targets through fresh-id rewrites and cover cross-document aliases plus unreadable ownership.

* fix(media): guard global asset reclamation

Share a fail-closed persisted-document liveness check between stage deletion and retry replacement. Preserve cross-document pool aliases while deleting stage-owned compatibility rows and cover enumeration failures.

* fix(media): cover complete slide asset references

Route slide media traversal through a shared mutable slot contract so backgrounds participate in export, thumbnail hydration, collection, and rewrite lifecycles. Snapshot complete surviving-document refs once per reclamation and preserve manifest-only owners while retaining fail-closed behavior.

* fix(media): preserve exclusive retry asset ids

Allow targeted retries to replace exclusively owned pool assets in place. Keep shared and unprovable ownership paths on fresh allocations, and pin production-shaped retries plus compatibility-row cleanup.

* fix(media): revalidate asset bindings at completion

Recheck repository-wide ownership before replacing generated media and fork scoped retries when exclusivity changed. Route video export selection through the unified resolver and keep concrete posters independent of task state.

* fix(media): count unflushed owners and broadcast replacements

The completion-time exclusivity proof read only the persisted document, but
slide duplication updates the Zustand aggregate synchronously and schedules
persistence behind a debounce. A retry finishing inside that window saw a
single persisted owner and replaced the bytes behind a reference the
duplicate also held. The proof now also counts owners in the live stage
snapshot when that snapshot represents the stage being retried, so an
unflushed duplicate forks instead.

Same-id replacement notifications were realm-local, so a second tab showing
the same classroom kept its lease pinned to the superseded blob URL. The
notification now travels over a BroadcastChannel; each receiving realm runs
its own observers against its own pool, so a spoofed message can at most
force a re-resolve. A missing or failing channel never fails the replacement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): bind replacement listeners and spare shared audio rows

A realm that only renders never sends a replacement, so binding the channel
from the sender path left passive tabs deaf to peers. Binding now happens
where the observer is registered, when the asset-pool module loads, and the
receiving realm resolves its own pool lazily so a cleared or unavailable
pool degrades to the next resolve instead of throwing.

Observer notifications ran under Promise.all, so a rejection surfaced after
BrowserAssetStore.replace had already committed and turned a durable success
into a reported failure. They are settled individually now; the callback is
wrapped because a synchronous throw would otherwise escape before allSettled
sees the array.

audioFiles rows are keyed globally by audioId, so deleting a stage removed
the sole row for an id a surviving document still referenced — playback and
both export paths read that table directly and cannot fall back to the
preserved pool blob. Rows are now filtered against surviving references,
while a failed enumeration still withholds only the irreversible pool
removal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): serve replaced bytes from the pool everywhere

Unknown survivor liveness deleted every planned audioFiles row. Those rows
are keyed globally by audioId, and playback plus both export paths read the
table directly, so losing one is as irreversible for them as removing the
pool entry. Unknown liveness now preserves the rows too, leaving bounded
garbage for a later pass that can prove exclusivity.

The earlier pool-first change covered PPTX, video collection and thumbnail
hydration but missed classroom ZIP export, which still serialized the stale
compatibility row after a lagged same-id replacement, shipping media the
classroom no longer renders.

Auditing every direct reader of the media and audio tables surfaced the same
gap in playback: speech regeneration also replaces bytes under a stable id
and does not roll the pool back when the compatibility write fails, so the
player kept serving superseded narration. It now resolves the pool first and
falls back to stored rows for legacy and imported audio.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(audio): replace exclusively owned speech clips in place

regenerateSpeechAudio accepted the action's audioId but always passed
undefined as replaceAssetId, so regenerating an exclusively owned
pool-backed clip allocated a new asset every time, rewrote the action and
orphaned the previous pool entry and compatibility row until stage
reclamation — contradicting the stable-id path generateAndStoreTTS already
implements for media.

Ownership is now established before synthesis, and the rule itself moved to
the shared reference module so media retries, poster replacement and speech
regeneration consume one implementation instead of restating it. An
exclusively owned clip keeps its id and has its bytes replaced; a shared
clip, a legacy id with no pool entry, or unprovable ownership still gets a
fresh allocation so other holders keep their audio.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(media): drop the import left behind by the ownership move

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): resolve audio pool-first and fence peer realms

Stable-id TTS regeneration commits replaced narration to the pool before the
audioFiles mirror write, so a failed mirror leaves the row stale. AudioPlayer
already resolved pool-first, but audioObjectUrl, collectAudioFiles and the
video timeline dependencies still read Dexie directly and would serve the
superseded clip. All allocated-audio readers now share one resolver, with
Dexie kept as the fallback for legacy and imported rows.

The exclusivity proof modelled unflushed owners in the active realm only, so
another tab duplicating the same asset during its save debounce could still
be observed as a single owner and have its bytes replaced globally. A peer's
pending state cannot be read across realms, so presence is probed instead:
any realm holding the stage forces the fork path.

A deferred asset-pool deletion no longer reloads the page. The database is
still on disk and the guidance asks the user to close the other tab and
retry, which the reload discarded. The decision moved into a shared helper so
the rule is pinned rather than living inline in the component.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): fail closed when presence cannot be probed

The presence helper documented that an unanswerable probe must count as a
peer, but every unavailable path — no BroadcastChannel, a constructor that
threw, a send that failed, and the window before the pool's asynchronous
binding completes — returned false, so the ownership proof cleared a single
local owner and replaced globally shared bytes in place.

Probing now returns present, absent or unknown, and only a probe that was
actually sent and went unanswered is absent; the ownership decision treats
unknown exactly like present. The pool declares its binding intent
synchronously so a probe issued during the load-time window waits for the
bind instead of concluding that presence is unavailable, and releases that
gate if the import fails.

Coverage reaches the write boundary: with presence unknown, a
production-shaped targeted retry forks to a fresh id instead of calling
replace, and the original bytes stay intact for a peer's unflushed owner.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(dsl): bump to 0.6.3 after the release dedupe took 0.6.2

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(i18n): add the deferred-clear guidance to fr-FR

The locale landed on main after this branch added the key, so the alignment
check flagged it as the one missing translation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
@openmaic/dsl@0.6.3
2026-08-06 18:49:34 +08:00
wyuc 3ca86aab95 test(pbl-v2): pin single-call prompt goldens and the loop planner's injection seam (#1071)
Follow-up to #1069: golden coverage for all four planner prompt variants (the single-call pair is production's first-priority path), an assertion pinning the loop's injected call contract (toolset, stopWhen, source label, thinking config), plus three review-identified cleanups. No production behavior changes.
2026-08-06 05:49:29 -04:00
wyuc 3f5c51bdba refactor(pbl-v2): split the planner core from the loop; inject the LLM call path (#1069)
Implements #1062. Pure planner-core extracted from the loop's file; both planners receive their LLM entry as a parameter (call sites pass the real callLLM unchanged); operations/ split into kernel vs runtime with a lint-enforced boundary; planner prompts pinned byte-identical via fixtures generated from the pre-refactor implementation.
2026-08-06 05:28:56 -04:00
9d2689439e feat(i18n): add French (fr-FR) locale (#1068)
Translate all 1,533 leaf keys with exact parity against en-US.json and
register the locale in supportedLocales.

Two extra files are required and could not be split out: LOCALE_RESOURCES
in lib/video-export-app/cover-config.ts and the expected-labels map in its
test are both typed Record<Locale, ...>. That file landed on 2026-08-04
via #995, nine days after #942 (es-MX) merged, so no previous locale PR
had to touch it. Without these two edits tsc --noEmit fails with TS2741.
Seven added lines outside lib/i18n, no logic change.

Conventions: formal register (vous); French typography (typographic
apostrophes, ellipsis, U+00A0 before : ; ! ? % and inside guillemets);
all {{var}} and {var} markers preserved verbatim; settings.lang_* left in
their native scripts, matching every existing locale.

Co-authored-by: momo2lajoie <209366801+momo2lajoie@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 04:44:48 -04:00
wyuc 9556a035b1 feat(generation): move outline generation into the package (#1065)
Part B of #1057. Outline primitives, buildOutlinePrompt (single-source prompt construction with golden snapshots), logger injection, JSON repair, and the bare-Node second-consumer smoke script with CI coverage. Shim-first: no app-side behavior changes; duplicated prompt assets guarded by a CI parity check until Part D.
@openmaic/generation@0.1.0
2026-08-05 23:29:36 -04:00