trending/OpenMAIC
1
0
Fork 0
mirror of https://github.com/THU-MAIC/OpenMAIC.git synced 2026-10-02 01:15:18 +08:00
Code Issues Packages Projects Releases Wiki Activity
653 Commits 256 Branches 106 Tags
main
Commit Graph
18 Commits
Author SHA1 Message Date
wyuc e824117c9c feat(storage): give the server the asset entry lifecycle (#1007 amendment, part 1) (#1472) 
@openmaic/storage 0.31.0: a document reference table maintained by the document store, pending -> committed allocations, an entry pass with a one-time bounded backfill and legacy mark, a standing bounded sweep for entries nothing references, live-entry quota, and single-sequence ascending entry locks for every reference-maintaining write.
 2026-09-15 08:49:33 +02:00
Zhenghan Songandwyuc ca3d785221 build(node): enforce direct dependency engine floor (#1337) 
Why:
- The root Node 20.9 contract predates direct runtime dependencies whose declared minimums now reach Node 22.19.
- README, contributor, localized docs, and the OpenMAIC extension skill repeated the stale supported-version claim.

What:
- Raise the root Node minimum to 22.19 and align every operator, contributor, locale, and skill prerequisite.
- Add a check that compares the root minimum with installed direct production dependency engine minimums.
- Run the new contract check in CI after the frozen dependency install.

Risk:
- This changes the declared minimum only; no upper bound is added and Node 24 compatibility remains a separate concern.
- The localized docs build was validated with the independent #1306 boundary fix from PR #1307, which is not included here.

Tests:
- RED on the base: engine check reported pi-agent-core, pi-ai, svg-pathdata, and undici floors
- GREEN: root minimum 22.19 satisfies 35 engine-constrained direct dependencies
- Node 20 lockfile-only install reports the root unsupported-engine warning
- Node 22 frozen install and postinstall
- Docs build with PR #1307 boundary: 34 pages and all locale postexport checks; docs types:check
- Root Prettier, ESLint, TypeScript, i18n, package-version, and internal-dependency gates
- Root pnpm test: 7137 passed, 81 skipped

Live Docs:
- GitHub issue #1304 tracks the Node contract; #1306 / PR #1307 tracks the separate docs-build prerequisite.

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-02 04:29:52 -04:00
wyucandClaude Fable 5 04621578de release: OpenMAIC 1.0.0 — the agent workbench (#1228) 
* feat(storage): add an agent-session store with PG backend and layered contracts (#1163)

* feat(storage): add agent-session store with PG backend and layered contracts

* test(storage): avoid BigInt literals for pre-ES2020 root typecheck

* fix(storage): close agent-session store review findings

* docs(storage): align hook ordering and contention-probe claims with the code

* ci: run on the agent-workbench integration branch

* chore(storage): bump to 0.5.0 for the agent-session store

* fix(storage): carry replay compaction across page boundaries

* feat(agent): add the driver model contract and stage route dialect (#1165)

* feat(agent): add the driver model contract and stage route dialect

* fix(agent): validate route context windows and clarify dialect precedence

* feat(agent): adapt the agent-session store and runtime foundations (#1167)

* feat(agent): adapt the agent-session store and runtime foundations

* feat(agent): resolve request owner identity via an anonymous cookie

* docs(agent): document the opt-in compaction default and harden edge cases

* feat(agent): add the background session runner (#1169)

* feat(agent): add the background session runner

* feat(agent): wire the runner into startup behind feature flags

* fix(agent): stop clean interruptions from consuming the attempt budget

* fix(storage): charge the attempt budget for abandoned leases but not clean parks

* docs(storage): document the attempt-charging contract and decouple its tests

* feat(agent): add agent session and owner event streams (#1170)

* feat(agent): add agent session and owner event streams

* fix(agent): close the session-existence oracle and document the owner seam

* feat(agent): add agent session lifecycle routes (#1171)

* feat(agent): add agent session lifecycle routes

* fix(agent): validate session-create input and preserve the owner cookie on errors

* refactor(storage): drop the unused active-stage API from the agent-session contract (#1174)

* refactor(storage): drop the unused active-stage API from the agent-session contract

Tools address stages explicitly on every call, so the store keeps no
mutable session-level stage pointer. Removes resolveActiveStage and
setActiveStage from the store interface, their PG implementations, the
active_stage_changed lifecycle event, the session_active_stage owner
event variant, and the contract tests pinning them. The active_stage_id
column and the DDL check constraint stay untouched for schema
compatibility.

* chore(storage): bump @openmaic/storage to 0.7.0 for the contract removal

* docs: document the agent runtime configuration surface (#1176)

* fix(agent): repair orphaned and late tool results across interruption boundaries (#1180)

* fix(agent): repair orphaned and late tool results across interruption boundaries

A crash, shutdown, or provider failure can leave the durable transcript with
tool calls that have no result, or with results ordered illegally for the
provider. Three failure modes were fixed:

- Orphaned tool calls: a run that died between an assistant tool-call frame
  and its result left a dangling call in the entry tree. Resume no longer
  synthesizes and persists receipts for it: interrupted results are a
  read-time provider view owned by a shared read-boundary repair, which
  returns the original array for a healthy transcript and never mutates the
  tree.

- Late parallel results: a parallel tool can finish while pi unwinds an
  aborted assistant frame, leaving result(A), assistant(aborted), result(B)
  in durable order. Strict providers reject non-contiguous results, so the
  read-boundary repair moves existing results next to their owning assistant
  frame (in call order), omits incomplete unwind frames, and synthesizes
  receipts only for genuinely missing calls.

- Interrupted calls at the write boundary: a call still in flight when the
  run winds down (shutdown, lease loss, cancellation, provider failure) had
  no receipt at all. The runner now tracks in-flight calls from their
  assistant frames and, before the terminal flush, appends an
  interrupted-result receipt for each still-orphaned call through the same
  attempt-fenced write chain, so a lease-stealing zombie never writes and
  the next claim sees a provider-safe transcript.

* test(agent): pin the runner wiring for interruption-boundary tool repair

* feat(agent): add neutral tool foundation libraries (#1184)

* feat(agent): register a web_search tool on the session runner (#1185)

* feat(storage): add a per-session URL trust gate (#1186)

* feat(agent): add the skills system (#1189)

* feat(agent): add the skills system (builtin directories and durable user skills)

* fix(storage): serialize the user-skill quota check-and-insert per owner

Two concurrent creates at the 50-skill boundary both counted 49 rows and
both inserted (READ COMMITTED, no lock), overshooting the quota contract.
The create transaction now takes a per-owner pg_advisory_xact_lock first,
and the same-name idempotency check runs before the count check so an
at-least-once retry of the create that committed as the owner's 50th row
still returns its durable receipt instead of a quota error. The 23505
backstop is retained for writes that do not take the lock.

* fix(agent): share unstorable-character validation and align skill lookup

* feat(agent): add session materials and a fetch_url tool behind the URL trust gate (#1190)

* feat(agent): add session materials and a fetch_url tool behind the URL trust gate

* fix(agent): harden session material fetching

* feat(storage): add an ownership scope to stage documents (#1191)

* feat(agent): add material read and search tools (#1192)

* feat(agent): add stage read and patch tools (#1194)

* feat(agent): add page generation and deck editing tools (#1198)

* test(storage): keep the PG contract suite order-independent (#1200)

* fix(agent): revoke deleted-session URL authority and reject private ISATAP endpoints (#1199)

* fix(storage): revoke deleted session URL authority

* fix(ssrf): reject private ISATAP endpoints in strict fetches

* chore(storage): bump to 0.11.1 for the session-URL authority fix

* feat(agent): add roster and voice registration tools (#1201)

* feat(agent): add folder organisation tools (#1202)

* feat(api): add stage and material HTTP routes (#1203)

* feat(workbench): add the client data layer (#1204)

* feat(workbench): add the client data layer

* docs(workbench): write the ported comments in English

* chore(edit): remove the in-editor agent panel (#1210)

* chore(edit): remove the in-editor agent panel

* style: apply prettier formatting

* fix(agent): report the runtime as unusable without a database (#1207)

* fix(agent): report the runtime as unusable without a database

* style: apply prettier formatting

* feat(agent): add image, video and pptx import tools (#1211)

* feat(workbench): add the agent chat surface (#1205)

* feat(workbench): add the agent chat surface

* docs(workbench): write the ported comments in English

* fix(workbench): label the folder and rename tools on the timeline

* fix(workbench): label the roster and voice tools on the timeline

The reconciliation test iterates every tool the runner registers and
requires a display label of its own. The roster and voice-clone tools
(list_voices, set_roster, clip_audio, register_voice) reached the
integration base with the roster/voice-registration tools but never
gained presentation rows, so they fell through to the default branch
and rendered their wire names. Port their rows from the reference
implementation (labels and i18n keys verbatim) and extend the
reconciliation allowlist with ROSTER_TOOL_NAMES and
VOICE_CLONE_TOOL_NAMES, so a future tool cannot enter the product
without a label.

* feat(agent): add the material extraction lifecycle (#1212)

* feat(storage): add material extraction lifecycle

* feat(agent): execute queued material extraction

* style: apply prettier formatting

* style: satisfy prefer-const in the extraction runner

* test: give material fixtures the extraction lifecycle fields

The media-tools slice and the extraction lifecycle slice were each green
in isolation but never compiled together: the lifecycle made derivedFrom
and extraction required on AgentSessionMaterial while the media-tool
fixtures predate them.

* chore: remove stray task notes

* fix(workbench): label the extraction lifecycle tools on the timeline

* feat(workbench): add the workspace shell (#1206)

* feat(workbench): add the workspace shell

* docs(workbench): write the ported comments in English

* i18n(workbench): align workspace keys across locales

* fix(workbench): adopt the landed data layer and label the extraction tools

- replace the sibling-slice seam stubs with the real data-layer modules
- drop ambient declarations now shadowed by landed files
- port timeline labels for the extraction lifecycle tools from the reference
- align the new i18n keys across all locales

* ci: retrigger

* feat(api): folder routes, stage-meta viewer surfaces, and the material upload contract (#1215)

* fix(storage): restore capability-based stage access

* fix(api): bind document access to request owner

* fix(agent): restore three-state stage access on the tool layer

Port probeStageAccess and the three-state StageAccess (owned / foreign /
missing / tombstoned) and gate every stageId-bearing stage tool on an owned
probe, mirroring the reference per tool:

- move_to_folder, rename_stage, read_stage_outline refuse a non-owned stage
  with the single not-yours message before touching the store.
- The course/DSL toolset and the roster toolset are wrapped by
  withOwnerStageAuthorization: read_stage, patch_stage, grep_stage and every
  writer refuse a foreign stage with the same message and refusal shape.
- Scene preview keeps its own probe and its own refusal text, and is
  registered beside the course toolset (never double-gated).
- The runner injects one probe factory at the three call sites.

Tests: the dsl cross-owner test premise (a foreign stage is readable by id)
encoded an invented capability-read policy that the reference does not have
at the tool layer; it now asserts foreign read/patch/grep are all refused
while the owner still reads. Curriculum cross-owner assertions were already
the reference's and now pass with the probes in place.

* docs: correct per-file test counts in the fidelity report

* test: fix type errors in stage-access fidelity test

* test: adapt media-tool and gate suites to the owner-scoped store seam

* feat(api): add owner-scoped course-folder HTTP routes

Port the reference implementation's /api/folders family (list, create,
rename, delete with ungroup/remove modes, and folder membership) onto the
owner-bound document store, replacing its provider-based auth with the
existing withRequestOwnerId / owner-scoped store seams.

The storage package's folder store grows the pieces the routes need:
DocumentFolder.order (schema column + max+1 assignment + ordering),
renameFolder, deleteFolder(mode) with captured member ids, and
setStageFolder(stageId, folderId | null) with idempotent un-filing.
FolderNameError moves into folder-name-validation.ts (stage-storage re-exports
it, keeping import sites intact).

Every route gates on the configured agent runtime (plain 404 when off or
unconfigured), keeps the reference's machine codes and envelopes, and is
covered by gate tests plus a behavior suite.

* feat(api): add stage-meta viewer surfaces for the classroom

Port the reference implementation's viewer-facing stage state — can-edit /
collected / published / generation-complete — on top of the stage-access
base (stage_meta + tombstones). stage_meta gains published_at and
generation_complete columns plus a stage_bookmarks table; the reference's
deployment-specific origin/claimed_at columns are stripped.

New gated routes: GET /api/stage-meta/[stageId] (per-viewer facts, 404 for
absent/tombstoned, never returns the owner id), GET /api/stages/[id]/status,
POST generation-complete / publish / unpublish (owner-only), POST
/api/bookmarks. The resolver lives in lib/server/stage-access.ts.

Wiring: a fetchStageMeta client with the reference's three-outcome contract,
stage-store isOwner/isBookmarked/readOnly fields (upstream single-user
defaults, no-op until the sidecar answers) plus setViewerAccess, the
classroom apply path computing readOnly = !(isOwner || isBookmarked), the
Stage editability gate, and a sidecar probe after each classroom load. A
sidecar 'absent' answer keeps the editable default here because the
classroom also serves local-only courses; server writes stay owner-enforced.

* feat(api): port the reference material upload contract

Rewrite POST /api/materials to the reference implementation's upload shape
so the workbench uploader (uploadWorkbenchMaterial, which posts no session
id and expects a flat 201 view) works unchanged: owner-scoped upload with
mime normalization/validation (415), per-class size caps checked on the
declared content-length and the streamed body (413), empty body (400),
quota (429), sha256 reserve->store->finalize lifecycle with abandon on
failure, flat { materialId, originalName, bytes, mime, extraction } 201,
and an x-request-id echo.

Adds the owner-scoped material library (owner_material table + quota +
24h lazy sweep, bytes in the host's asset registry as the neutral
replacement for the reference's object-storage byte path) and the material
cap configuration. The session-scoped GET list is left as-is; the
reference's owner-material extraction worker is not ported (the branch's
session-material extraction lifecycle already covers extraction).

Gate tests now cover all 23 persistence routes across the three runtime
env states; the materials behavior suite pins the new contract.

* feat(media): add an optional local ffmpeg media extractor (#1213)

Adds a local ffmpeg/ffprobe pipeline as a second media extraction
provider behind the extractor registry, ported faithfully from the
reference implementation: duration probing, keyframe-safe chunking,
per-chunk ASR with timeout and deadline budgets, and timestamped
transcript assembly.

- Availability probing feeds the registry's candidate selection: the
  provider simply is not a candidate when ffmpeg/ffprobe are absent.
- With neither ffmpeg nor a cloud provider configured, extraction fails
  with an actionable message naming both enablement paths.
- Media materials route through the same extraction lifecycle and lease
  fence as documents; no parallel queue.
- Tests inject the executable resolver so the missing-ffmpeg path is the
  default-tested one; the real pipeline test is skip-if-unavailable.
- @openmaic/storage 0.13.0 -> 0.14.0 (media routing in the material
  lifecycle surface).

* feat(storage): per-scene monotonic revisions via database triggers (#1214)

* feat(storage): per-scene monotonic revisions via database triggers

Restore the reference implementation's freshness granularity: a
per-scene monotonic revision maintained by database triggers, so every
writer (HTTP routes, agent tools, jobs, manual SQL) bumps it without
application cooperation.

- Companion revision tables + trigger functions in the storage package's
  idempotent schema bootstrap, with the lock-order invariant, pg_notify
  wakeup and the suppression switch for batch writers.
- ensureDocumentSchema gained a dollar-quote-aware statement splitter.
- The freshness and manifest routes serve per-scene revisions.
- Mutation-verified: dropping the triggers turns the revision tests red.
- @openmaic/storage 0.13.0 -> 0.14.0.

* fix: forward the freshness manifest through the owner-bound store

* feat(workbench): add the Pro entry points and preserve the mode-transition semantics (#1208)

* feat(workbench): add the Pro entry points

* feat(workbench): preserve Pro mode transition semantics

* fix(workbench): drop ambient declarations shadowed by landed slices

* fix(workbench): drop ambient declarations shadowed by the landed shell

* feat: port workspace shell sibling modules

Port the 16 leaf modules the Pro workspace shell imports but that were only
ambient-declared, replacing the compile-time bridge with real implementations
adapted from the sibling-slice reference: pure workbench helpers (session
title, rail tab, course-chat bootstrap, created-course tabs, course-tabs
memory, workspace navigation, pane navigation, pro-edit sizing,
existing-course minting, first-message session), the neutral brand context and
course-rename server API, the server-action session delete, the home
discovery hook, the classroom pane host with its load-policy leaf, the theme
toggle and floating-layer owner, plus the floating-layer-owner wiring the
dialog/dropdown/tooltip portals stamp.

Also add the workbench-shell locale copy for all 12 locales, port the
reference tests for the ported modules, and drop
types/workbench-sibling-slices.d.ts now that every declaration has a real
implementation.

* docs: keep ported comments in English and deployment-neutral

* docs: announce 1.0.0 and refresh the feature overview (#1216)

* docs: announce 1.0.0 and refresh the feature overview

* docs: finalize 1.0.0 README after feature merge

* fix(agent): control-plane routes answer 404, not 500, without a database

The agent control-plane routes gated only on the runtime flag, so an
enabled-but-unconfigured deployment (flag on, DATABASE_URL empty)
answered 500 from a store that cannot connect. Gate them on the
configured check instead, matching the stage/material routes: the
whole surface is cleanly absent until both the flag and the database
are present. The status probe keeps reporting both bits.

* test: mock both runtime gate exports in the control-plane route suites

* fix(agent): abort in-flight TTS on cancel and bound each provider request with a timeout (#1217)

The generate_tts / scene-tts path checked the runner's AbortSignal between
actions but never created the provider HTTP requests with it, so a session
cancel left a hung synthesis fetch in flight until a restart repaired the
tool result. Thread the signal end-to-end: TTSModelConfig carries an
optional signal, generateTTS combines it with a per-request timeout
(TTS_REQUEST_TIMEOUT_MS, default 30s, ported from the reference runtime's
TTS bounds) via AbortSignal.any, and every provider fetch (openai, azure,
glm, qwen incl. voice-clone + audio download, voxcpm, minimax, doubao,
elevenlabs, lemonade) is created with that signal.

A timeout now fails the tool call with TTSRequestTimeoutError (a clear
retryable error) instead of wedging the session; a caller cancel propagates
as the interruption so the runner settles the session as cancelled without
a restart.

Tests: hung-provider simulation rejects at the timeout with the retryable
error; abort mid-flight aborts the captured request signal and surfaces the
interrupted shape; removing the signal wiring makes the abort tests fail
(red), restoring them turns green.

* fix(workbench): PG-mode home listing via owner stages; keep the interrupted terminal course card (#1218)

Finding 1: with server persistence on, listStages resolved to the generic
GET /api/persistence/documents listing, which the capability model
deliberately answers 403 FORBIDDEN_DOCUMENTS for (reads by id, listings
owner-only). The home/workspace library now lists through the owner-scoped
GET /api/stages surface (same anonymous-owner cookie the workbench uses)
when server persistence is enabled; the server-side 403 is untouched.

Finding 2: a run interrupted (session_interrupted) and repaired
(session_resumed) that ends cancelled before agent_end stranded its pending
classroom sightings, so the timeline's terminal card lost the course the
answer produced. session_end (cancelled) now flushes the pending sightings
into the same course card set agent_end paints, before the stopped caption.

* chore(workbench): remove the bookmark concept and the saved-courses drawer (#1219)

* chore(classroom): remove the bookmark ('collected') concept entirely

The stage-meta viewer port introduced a bookmark surface (stage_bookmarks
table, POST /api/bookmarks, the isBookmarked sidecar field, and a
readOnly rule that let a saved course stay editable). The product has no
such concept, so remove it as a closure:

- delete the /api/bookmarks route and the stage_bookmarks table plus its
  query helpers from the persistence bootstrap
- drop isBookmarked from GET /api/stage-meta/[stageId]
- simplify the classroom read-only rule to readOnly = !isOwner across the
  sidecar client, ownership signal, classroom load, stage store and the
  classroom page
- keep publish/unpublish, generation-complete, isOwner and isPublic
  exactly as they were
- update the gate and stage-meta route suites and the README mentions

The workspace rail's Bookmark glyphs and comments describe the upstream
saved-courses (favorites) section, which is driven by isOwner and renders
no collect affordance; they are kept as unrelated homonyms.

* chore(workbench): remove the saved-courses drawer UI

The first pass removed the bookmark data model but kept the rail's
"Saved courses" drawer, judging it a separate surface driven by
`isOwner === false`. The home/workspace listing is owner-scoped, so that
flag can never occur: `allSaved` is permanently empty and the drawer
(plus the collapsed-rail Bookmark mini-button) is a dead affordance.
Remove it: the SavedDrawer component and its mount, the savedOpen /
savedSection state, the allSaved / matchedSaved derivations, the 'saved'
variant of the course-list renderers, the mini Bookmark glyph, the
drawer-only CSS, and the drawer's i18n keys from all 12 locales. The
courses tab is now exactly one folders tree.

The authored/favorites split in workspace-tree.ts goes with it; the tree
module no longer reads `isOwner`. The discovery course type keeps the
field — the shell still reads it for read-only gating.

Upstream has no collect concept; the drawer could only ever render empty
here. The reference implementation HAS this drawer (its favorites come
from its account system), so this removal is a deliberate upstream
product decision, not a fidelity bug.

* fix(workbench): restore the attach entry, add the rail settings entry, pin all three entry points (#1221)

* fix(workbench): restore the composer attach entry by gating it on the live runtime

The AttachButton's rollout probe read a `materialsEnabled` field that this
branch's /api/agent/runtime never answers (the materials routes gate on the
runtime itself, like the stages), so the gate could never pass and the attach
button never rendered — the Pro launch and chat composers showed only the
@-mention and enhance glyphs.

Substitute the field with the runtime's `enabled` value, which IS the upload
action's precondition: POST /api/materials answers 404 whenever it is false,
so the render condition now equals the action precondition (no dead button).

The button's label (`proMode.attach`) is a user-visible string that becomes
visible again; port the reference implementation's own translations verbatim
into the 11 locales that still carried the Chinese copy.

* feat(workbench): add the settings entry to the rail's bottom-left cluster

The reference's rail foot carries a cluster of utilities (its saved-courses
drawer, the language switcher, the display toggle). This branch removed the
drawer — it could only ever render empty here — and the product decision is
to fill that freed spot with the settings entry.

Add a settings trigger to the foot cluster (expanded rail, beside the
language and display toggles, and on the collapsed strip) and mount the
model/provider SettingsDialog in the rail, wired to the trigger. It is the
same dialog the classic home opens from its header pill; the workspace had no
settings entry of its own, so nothing is duplicated within a surface.

* test(workbench): pin the restored upload, attach, and settings entry points

Covers the three restored entry points:

- the courses-tab upload control: rendered beside the course name filter,
  wired to the discovery hook's ZIP import trigger, disabled while an import
  runs, and gated by the same condition as its action (the courses tab);
- the composer attach control: an actual render of AttachButton under both
  probe answers (visible when the runtime says the upload path is live,
  hidden otherwise), its mounts in the launch and chat composers, the
  branch's runtime-field substitution in the probe, and the reference's own
  `proMode.attach` copy in all 12 locales;
- the settings entry: the trigger in the rail's foot cluster (expanded and
  collapsed), beside the language and display toggles, opening the
  SettingsDialog the rail mounts.

* chore(config): the Pro workbench flag implies the MAIC Editor gate (#1223)

A workbench build without the editor toggle has no way to edit a course:
enabling NEXT_PUBLIC_PRO_WORKBENCH_ENABLED while forgetting
NEXT_PUBLIC_MAIC_EDITOR_ENABLED produced exactly that split-brain bundle.
The workbench IS Pro mode, so its flag now implies the editor gate; the
standalone flag remains for deployments that want the classroom editor
without the workbench. Documents both flags in .env.example.

* fix(agent): wake SSE tails and the runner on durable deltas (streaming fidelity) (#1222)

The Pro workbench chat did not stream: the session/owner SSE routes polled
the durable event log on a 5s/30s clock with no wakeup, so message_update
deltas (written at 150ms cadence) reached the browser in poll-sized blocks
and the thinking strip only mounted after the whole reasoning text had
accumulated.

Port the reference's LISTEN/NOTIFY delta path:

- storage: add in-transaction wake hooks (onSessionEventAppended,
  onOwnerEventAppended, onCancelRequested) so a host queues pg_notify in
  the same transaction as the durable append; align readEventsAfterForReplay
  to rank the bounded page so the first delta after the cursor is always
  kept (the live tail can never starve). Bump @openmaic/storage to 0.18.0.
- app: port the process-wide event-notify bus (dedicated LISTEN client,
  self-check probe, reconnect backoff; notify through the storage
  transaction surface), wire the store hooks, subscribe both SSE routes
  before the initial read with the reference's initializing gate, and give
  the runner one {kind:'session'} subscription whose wake runs the cancel
  check and the message drain. Polls stay as the lossy-NOTIFY backstop.
- lifecycle: start/stop the bus from instrumentation.

Tests: storage hook + compaction contract; route wakeup latency; runner
wakeup wiring with a fake agent; bus unit tests; PG contracts proving a
real append wakes the routes and a live SSE route forwards a message_update
on the wakeup, and that a rolled-back append never wakes. Also fix the
pre-existing park-attempt-budget PG test TRUNCATE (missing CASCADE against
newer FK tables).

* fix(storage): asset writes self-deadlocked against pooled PostgreSQL (#1225)

* fix(storage): refuse the non-transactional byte-write deadlock configuration

A byte store whose plain write() runs on its own pooled connection cannot be
invoked from inside a registry write transaction: after the transaction has
claimed the blob-row lock, that write blocks on the lock the transaction just
took while the transaction waits on the write - a self-deadlock PostgreSQL
cannot detect (one side is idle in transaction).

There is no lock-safe ordering for such a writer: bytes must be written after
the row claim (writing before it lets the collector delete the bytes while the
upsert waits), and any second-connection write after the claim is the
deadlock. The configuration is therefore detected and refused:

- AssetByteStore gains writesOutsideRegistryDatabase?: true, declaring that the
  layer's plain byte operations cannot contend for the registry's row locks.
- PgAssetStore refuses put()/replace() up front (and defends coordinatedWrite)
  when the byte store has no writeWith and does not declare the flag, throwing
  a clear configuration error before any row is claimed.
- The collector mirrors the guard on its delete path (deleteWith or a declared
  out-of-registry layer, else a configuration error).
- The object store declares the flag (its out-of-transaction write remains
  legitimate); the in-registry PostgreSQL byte column provides writeWith /
  deleteWith instead.
- Write transactions (put/replace/remove) set SET LOCAL lock_timeout = 30s so
  any future lock-contention variant fails loudly instead of hanging.

Bumps @openmaic/storage to 0.18.0.

* fix(persistence): forward the transactional byte methods through the lazy asset byte-store wrapper

The no-bucket case of lazyAssetByteStore returned a bare { write, read,
delete } and dropped writeWith/readWith even though the underlying
PgAssetByteStore has them. The registry's hasTransactionalWriter duck check
then failed and put() fell back to the byte store's own pooled connection,
which blocks forever on the blob-row lock the registry transaction just took
when the bytes live in the same PostgreSQL - the production self-deadlock.

The no-bucket layer is statically PgAssetByteStore, so its transaction-pinned
methods are forwarded eagerly (typed against the real signatures via
PgForwardedByteStore). The bucket case keeps its lazy-probing semantics: no
transactional writer exists there, the signed-URL method stays absent or lazy
exactly as documented, and the wrapper now declares writesOutsideRegistryDatabase
so the registry may run the plain write inside its transaction.

New tests pin the wrapper's transactional capability red-to-green and assert
put()/resolve() route byte traffic through the transaction-pinned queryable.

* fix(home): cap the generate-prep ingest drain at 3s so Generate never waits the full server budget

The classic home flow's Generate click drained in-flight ingests for the full
15s server budget. Cap the wait at GENERATE_DRAIN_CAP_MS (3000ms, documented as
a UX bound) and reuse the existing timeout fallback: sources that miss the cap
proceed on the legacy byte path and each late-resolving id is released.

* chore(storage): bump to 0.19.0 over the concurrently landed 0.18.0

* fix(agent): bound every tool call with a timeout; never resurrect a cancelled session (#1226)

* fix(agent): bound every tool call with a global timeout and settle it on cancel

A tool await that neither resolves nor rejects wedges the session forever:
the lease keeps heartbeating and the driver never reaches its next cancel
checkpoint. Race every tool execution (in buildAgent) against a hard budget
(OPENMAIC_AGENT_TOOL_TIMEOUT_MS, default 10 min, per-tool overrides for
known long runners) and against the caller's AbortSignal, so even a
signal-ignoring await cannot keep a cancelled session running.

On timeout the call rejects with AgentToolTimeoutError; the agent loop turns
the rejection into a structured error tool-result the agent can retry or
proceed from, and the abort signal is delivered to the tool's in-flight work
through a derived controller. Zombie-tool updates after settlement are
dropped.

* fix(storage): never re-lease a cancel-requested session; settle it as cancelled on claim

The claim scan treated a session with cancel_requested_at set as a normal
claim candidate: after a restart it re-leased the same session for attempt
N+1 and resumed generating despite the pending cancel. claimNextSession now
settles such candidates as cancelled under the claim lock (status cancelled,
attempt reset, lease and cancel request cleared, terminal session_end event
and owner projection) instead of leasing them, then keeps scanning.

Bump @openmaic/storage to 0.18.0.

* docs: takeaway-style 1.0.0 announcement with bilingual guide links

The 1.0.0 head is now a short takeaway block — badge links to the
official user guides (English and Chinese), five one-line highlights,
and pointers into Features and the workbench setup section — instead of
six dense paragraphs. The detailed provider-neutrality and freshness
notes move into the Features workbench section, phrased database-
neutrally (the announcement no longer names a specific database).
Release date corrected to August 27.

* fix(workbench): restore editor chrome, mode transition, streaming, materials, mentions, folders (#1229)

* fix(workbench): wire workspace folder routes

* fix(editor): restore reference workbench chrome

* fix(workbench): persist composer materials and course refs

* fix(workbench): preserve live reasoning frames

* fix(persistence): back off failed streaming saves

* chore(workbench): retire stale slice seams

* test(editor): cover element pin layer

* chore(storage): bump to 0.21.0 for the user-message ref/material fields

* chore(editor): translate ported code comments to English

* fix(agent): fence durable tool writes and consume cancel requests atomically (#1230)

* fix(agent): enforce provider force-off in agent tools and scrub vendor identity from tool results (#1231)

* fix(materials): serialize per-owner quota reservations and make crashed uploads reclaimable (#1232)

* fix(editor): resolve dock-bar i18n keys, remove dock height drag, wire element referencing (#1233)

* fix(workbench): send the opening session message exactly once with refs intact (#1234)

* feat(editor): port timeline TTS preview single-flight and voice-all state latching (#1235)

* fix(media): restore the reference classic media chain (#1236)

* fix(import): adapt imported PPTX canvas size so decks render without overflow (#1237)

* fix(editor): complete element referencing — renderer DOM contract and GenUI picking aligned with the reference (#1238)

* test(providers): reconcile the provider-config vendor-token debt count after the main merge

The integration line's AK/SK fallback for the managed document provider adds
occurrences that main's allowlist snapshot predates. Same mixed-composition
debt category the group already documents; no new vendor behavior.

* test(providers): reconcile vendor-token debt counts with the integration line

The main-merge brought main's neutrality-guard snapshot next to integration
features it predates (media-extractor fallback chain, local voice-profile
deletion semantics, the enabled-TTS helper). Same debt categories the guard
already documents; counts updated to the guard's own tally and two grouped
entries added. No new vendor behavior.

* fix(agent): carry reasoning through the completions dialect so the thinking strip renders (#1239)

* feat(skills): add Feynman and spiral curriculum methods (#1240)

* feat(agent): port missing reference tools and skills (parity audit) (#1241)

* feat(media): retire asset-registry wiring; media and materials follow the reference byte model (#1242)

* fix(classroom): center adapted canvases in the stage and send back navigation home during generation (#1243)

* feat(settings): skill management with real list, download, delete, and upload (#1244)

* feat(settings): skill management section with real list, detail, and zip download

* feat(skills): owner skill delete and upload across storage, API, and settings

* fixup! feat(settings): skill management section with real list, detail, and zip download

chore: neutralize a reference note in the settings header comment

* fix(media): persist origin-independent classroom-media references from the agent runtime (#1245)

* feat(editor): float the insert toolbar in the outer frame with collapse (#1246)

The insert strip was bounded to the slide card, so it could only ever sit on
top of slide content: the card's overflow clipped it and it could not be parked
in the padding beside the slide. Move it into the studio frame the element
picker's panel already roams (CanvasOverlayPortal + the frame selector), so both
canvas overlays share one bounding container and their handles behave the same.
While picking, the strip rises over the picker and goes inert, which is the
z-order CANVAS_OVERLAY_Z already documents.

Add a fold beside the grip: the chevron collapses the strip to that grip row and
back, with the buttons unmounted rather than hidden. The fold is session-local
state owned by EditShell, next to the drag offset, so a surface swap keeps it;
nothing is persisted. Expanding a strip parked at the bottom edge re-clamps
through the same bounds rule the keyboard move uses.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* fix(workbench): align the chat timeline's left edge with the composer (#1247)

* fix(agent): fence session claims while an ask_user question is outstanding (#1248)

* fix(agent): settle-time rescue tracks real delivery instead of a count offset (#1249)

* fix(persistence): migrate owner_material to oss_key and drop legacy asset_id (#1250)

* docs(readme): surface the 1.0.0 user guide badges at the top (#1253)

* fix(workbench): show newly created folders in the sidebar without reload (#1254)

* docs(readme): add the release version prefix and drop the opt-in framing

* fix(workbench): single-source the chat gutter so timeline and composer share a left edge (#1255)

The transcript and the composer each established their own column: their own
`px-*` gutter and their own `mx-auto w-full max-w-*` centering wrapper. Equal
padding values were never enough, because the two columns are centered inside
different containing blocks — the transcript's is a scroll container, whose
content box is narrower than the composer footer's by the scrollbar's width:

  transcript text left = pad + (pane - 2*pad - scrollbar - measure) / 2
  composer box  left   = pad + (pane - 2*pad             - measure) / 2

The padding cancels out of the difference and what remains is `-scrollbar/2` at
every padding value, so the transcript sat half a scrollbar to the left of the
composer and tuning the two paddings against each other could not move it.

The column is now established once, by the nearest common ancestor of both
(`chatColumn`), and the scroll viewport and the composer footer are siblings
inside it that add no horizontal inset of their own. The cap carries the gutter
on top of the 760px reading measure, so the text column keeps its width. The
handed-over question row drops the padding that indented it past the agent's
prose; framed rows keep their own inner padding, which is what a card's border
sitting on the column edge means.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* fix(workbench): lock pane-embedded classroom to edit mode (#1256)

The workspace right pane painted the full learning chrome — speed control,
play button, learner avatars, mic bar — for a course the agent had just
created, then flipped to edit once the first scene landed.

resolveStageChromeMode treated playback as the DEFAULT branch for a hosted
classroom, so every shortfall fell into it: a course whose tab opens at
stage_link time has no scenes yet, so currentSceneId is null and
isHostedSceneEditable is false. A folded pane parked the playback root
behind the fold and cross-faded it out over the pane on unfold, and a
failed editor chunk dropped into playback permanently.

Lock it at the pane instead of defaulting per entry path:

- WorkbenchPanelProvider — the single element that mounts a classroom into
  the workspace — publishes editPinned (visible && !playback). Every entry
  path passes through it, so none of them decides.
- The hosted resolution can no longer degrade to playback. Start Learning
  (workbenchLearning, new input, split out from pane visibility) is the one
  door; everything else resolves between the neutral loading shell and edit.
- Stage's chrome dispatch is exhaustive on chromeMode, so the playback root
  is no longer the else-branch of a condition about the current scene.

No flicker: chromeMode is resolved during render, and preloadEditor now
answers synchronously (isEditorPreloaded) so a remount with the chunk
already registered paints edit on the first frame. A failed import is no
longer cached forever, so the lock cannot strand the pane.

Standalone classrooms keep their stored mode unchanged.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-27 19:26:25 +08:00
wyuc 086e7626ea ci: cut wall-clock time and bound Playwright browser installs (#1146) 
Cache Playwright browsers and Next compile artifacts, retry a timed-out
Chromium download, build the production bundle before Playwright starts,
and run Prettier/ESLint/tsc/i18n in parallel. Unit tests stay sequential.
Skip Playwright apt deps on ubuntu-latest.

Closes #1145
 2026-08-19 15:36:00 +08:00
Yizuki_Ameandwyuc 19a895e889 fix(video-export): make Cyrillic and Arabic Quiz fonts deterministic (#1114) 
* feat(video-export): make Quiz script fonts deterministic

* test(video-export): verify Arabic shaping visually

* fix(video-export): cover Arabic extension characters

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-16 16:22:34 +08:00
Yizuki_Ame d1eeb95e1c feat(video-export): add deterministic Quiz question-list scrolling (#1102) 
* feat(video-export): add deterministic Quiz question-list scrolling

* fix(video-export): bound Quiz layout measurement
 2026-08-12 10:31:13 +08:00
xuyuanwei678 c38da84ef6 feat(renderer): decouple editor UI from the app (#1072) 
* docs: define renderer editor task 1 integration

* docs: plan renderer editor task 1

* feat: gate renderer slide editor

* feat: adapt renderer editor intents

* feat: connect renderer editor to slide surface

* test: complete renderer intent shape fixture

* fix: restore renderer editor action previews

* fix: render editor alignment guides

* perf: smooth renderer element dragging

* feat: align renderer hidden element behavior

* feat: add renderer canvas host commands

* feat: add renderer canvas context interactions

* test: cover renderer context menu interaction

* fix: preserve renderer canvas interaction semantics

* docs: design renderer text editing

* docs: plan renderer text editing

* feat(renderer): add ProseMirror text core

* feat(renderer): add rich text editor controller

* feat(renderer): render editable text in place

* feat(renderer): enter text editing on click

* feat: connect renderer rich text editing

* feat: finalize renderer text edit lifecycle

* fix: stabilize renderer text edit commits

* fix(renderer): move elements from selected borders

* docs: design renderer editing UI text toolbar

* feat(renderer): publish editing UI contracts

* feat(renderer): add controlled text format toolbar

* style(renderer): format editing UI contracts

* fix(renderer): address Task 2 review findings

* fix(renderer): handle zero font sizes in toolbar

* feat(renderer): add editing UI color picker

* fix(renderer): address Task 3 color picker review

* fix(renderer): commit native color input changes

* fix(renderer): coordinate native color picker interaction

* feat(renderer): anchor text toolbar to canvas elements

* feat(renderer): compose editable canvas with text UI

* feat(editor): use renderer text toolbar UI

* fix(renderer): make editing UI tests type-safe

* fix(renderer): harden editing UI integration

* feat(renderer): add editable text UI

* feat(renderer): add image and shape editing

* feat(editor): extend renderer editing controls

* feat(editor): add renderer chart insertion

* docs: design renderer latex editor

* docs: plan renderer latex editor

* feat(renderer): add latex editor contract

* feat(renderer): add latex editor dialog

* feat(renderer): compose latex insert and edit UI

* feat(editor): integrate renderer latex editing

* fix(editor): consolidate latex toolbar actions

* fix(renderer): show toolbar hover labels

* test(renderer): cover localized latex tooltips

* chore(renderer): hide latex sample palette

* feat(renderer): add video editing controls

* docs: define renderer audio editing

* feat(renderer): render audio elements

* feat(renderer): add audio editing UI

* feat(renderer): add preview audio controls

* feat(editor): add renderer element clipboard

* feat(renderer): decouple editor UI from app

* chore(renderer): release 0.0.6

* fix(renderer): clear editor CI gates

* fix(editor): adapt renderer canvas to media resolution API

* fix(i18n): complete French editor labels

* fix(renderer): hide toolbar vertical overflow

* chore(renderer): address review cleanup

* chore(docs): remove planning artifacts from pr

* feat(editor): add transactional core package

* chore(editor): ignore local test cache

* build(editor): publish core entrypoint

* refactor(editor): route renderer changes through transactions

* refactor(editor): split renderer editing into editor package

* ci(editor): register editor package for release

* docs(editor): document package dependencies

* ci: rerun validation after editor release registration

* test(editor): validate deletion against existing element

* fix(editor): address transaction and package review findings

* fix(editor): validate transactions and restore editor styles

* fix(editor): protect required updates and react styles

* fix(editor): validate required transaction fields

* fix(editor): prevent insert popovers shifting canvas

* refactor(editor): own renderer editing capabilities

* fix(editor): harden editing transaction boundaries

* refactor(editor): modularize editor UI integration

* docs(editor): harden third-party integration

* fix(editor): preserve video resize aspect ratio

* fix(editor): discard text draft before deletion

* fix(editor): preserve renderer picking and line presets

* chore(editor): remove unused pick-layer import
 2026-08-10 19:56:24 +08:00
wyuc b64e787749 feat(generation): move scene generation and PBL single-call planning into the package (#1087) 
Part C of #1057. Scene content/actions/widget generation, retry, action parsing, interactive post-processing, buildCompleteScene with injectable sceneId, and PBL single-call planning re-seated on AICallFn (pure core + kernel + package-relative planner assets; loop planner stays app-side behind the injected pblLoopFallback with provider/abort classification). Purely additive — app source untouched until Part D. Package 0.2.0.
 2026-08-10 00:58:35 -04:00
wyuc 7aba740a64 refactor(app): consume the contract's interactive and widget types (#1073) 
Second of three PRs for #1061 (after #1070). WidgetType re-exports from @openmaic/dsl; the app's rich widget configs extend WidgetConfigBase; InteractiveContent aliases the contract generic; the write validator adopts the contract's html-or-url rule while staying lenient over historical widget shapes (with a primitive-config barrier matching the hydration crash class); widget-config extraction seeds the contract type; @openmaic/generation drops its WidgetType copy and gains the dsl dependency (0.1.1).
 2026-08-06 22:53:16 -04:00
wyuc 9556a035b1 feat(generation): move outline generation into the package (#1065) 
Part B of #1057. Outline primitives, buildOutlinePrompt (single-source prompt construction with golden snapshots), logger injection, JSON repair, and the bare-Node second-consumer smoke script with CI coverage. Shim-first: no app-side behavior changes; duplicated prompt assets guarded by a CI parity check until Part D.
 2026-08-05 23:29:36 -04:00
wyuc 335180ec9c feat(generation): scaffold @openmaic/generation with pipeline types and packaged prompt assets (#1063) 
Part A of #1057. Package skeleton (ESM, tsc, dsl/storage pattern), pipeline type contracts, prompt loader with package-relative asset resolution, 13 templates + 7 snippets byte-identical with golden tests, allowlist boundary lint, full publish/CI wiring, installed-tarball smoke coverage.
 2026-08-05 22:22:54 -04:00
wyuc 30dd236057 Publish validated package tarballs (#1040) 
* Publish validated package tarballs

* Format artifact verification scripts

* Handle pnpm smoke-test arguments

* Harden validated package publication

* Make digest anchoring shell-portable

* Upload release artifacts before tests

* Anchor release integrity checks to GitHub SHA
 2026-08-03 03:53:42 -04:00
wyuc 90f5f3942b fix(release): dedupe the published dsl, close two release-path blind spots (#1019) 
* fix(release): dedupe the published dsl, close two release-path blind spots

Three independent defects on the release path.

1. `@openmaic/storage`, `@openmaic/renderer` and `@openmaic/importer`
   declared `"@openmaic/dsl": "workspace:*"`, which pnpm publishes as an
   EXACT pin. The tarballs on the registry today say storage@0.1.1 needs
   dsl 0.5.0, renderer@0.0.3 needs dsl 0.4.0 and importer@0.1.1 needs dsl
   0.4.0, so installing storage together with renderer yields two copies
   of the dsl. The dsl carries the schema, the validators and the version
   constants, so a document produced against one copy can be validated by
   the other's schema revision.

   `workspace:^` publishes as `^0.5.0`. Under 0.x that admits 0.5.x, so a
   dsl patch reaches dependents without republishing them while a dsl
   minor still requires a deliberate dependent release. All three
   packages take a patch version increase, both because
   check-package-version-bumps.mjs requires one and because the corrected
   range only reaches consumers through a new release.

   The tarball smoke test now asserts the packed range, so an exact pin
   cannot come back unnoticed.

2. The publish workflow's build-integrity guard ran `git diff --quiet --
   packages/@openmaic`, which compares the worktree to the INDEX. The
   steps before it run third-party package code, which needs no
   credential to rewrite a tracked file and `git add` it, after which the
   guard reported clean and the rewritten bytes were packed and
   published. Both copies of the guard now compare against HEAD, the
   commit actually being released.

3. `@openmaic/storage`'s PostgreSQL contract suites refuse to skip when
   STORAGE_PG_CONTRACT_REQUIRED=1, but that refusal is a `throw` inside
   the test modules, so it never fires if vitest stops collecting them.
   Collection is decided by `packages/@openmaic/storage/vitest.config.ts`,
   which is on the ignore list of publishable inputs, so excluding
   `*.pg.test.ts` there turns both PostgreSQL backends off with a green
   run and no version bump.

   scripts/assert-pg-contract-suites.mjs reads vitest's json results and
   requires both suite files to be present, passed and non-empty, keyed
   on the file names rather than on a count. It runs in the publish
   workflow's validate job and in the storage PostgreSQL contract
   workflow.

Merging this publishes @openmaic/storage@0.1.2, @openmaic/renderer@0.0.4
and @openmaic/importer@0.1.2. That republish is the fix for (1), not a
side effect of it: the corrected dependency range exists only in a
tarball that has not been cut yet.

* fix(release): audit the contract database, pin the format version to a minor

Follow-up on cross-vendor review of the three release-path fixes.

1. The PostgreSQL assertion proved far less than it claimed. Vitest's
   `assertionResults` are test CASES, not `expect()` calls, so one
   `test('x', () => {})` per required filename satisfied it — and the
   whole surface is reachable without a version bump, because `test/` is
   on the same ignore list as `vitest.config.ts`. A `vi.mock('pg', ...)`
   in the already-wired `test/setup.ts`, or a stubbed suite body, made it
   print "ran against a real database" having checked nothing of the
   sort.

   The script now runs in two phases. Phase 1 keeps the file/status
   checks. Phase 2 connects to PG_CONTRACT_URL from outside the vitest
   process and requires all five tables the two backends own to exist and
   to show inserts in pg_stat_user_tables. Nothing inside `test/` can
   forge that. Cumulative insert counters rather than surviving rows,
   because the suites clean up after themselves. The success message now
   states only what was checked.

2. Validation packed with lifecycle scripts while publishing runs
   `--ignore-scripts`, so committing `workspace:*` plus a `prepack` that
   rewrites it to `workspace:^` passed the tarball assertion and then
   published the exact pin. Both the smoke test and the workflow's
   dry-run pack now pass `--config.ignore-scripts=true`, so validation
   and publication have identical lifecycle semantics.

3. `workspace:^` unpins the serialized-format version that the exact pin
   was silently holding. DSL_VERSION and RUNTIME_DSL_VERSION are
   decoupled from the npm version by design, and storage compares them by
   value across the package boundary. Under `^0.5.0` a dsl PATCH could
   therefore hand a new format to an already-published storage: one
   install resolves it and stamps the new version into document_stages,
   another install pinned to the older patch reads that row and hard-
   fails. Two installs of one published storage version, data-
   incompatible.

   check-package-version-bumps.mjs now requires a change to either
   constant to carry at least a MINOR increase of the dsl package
   version, since a caret does not cross a 0.x minor. The rule is
   documented next to both constants. That doc edit is itself a
   publishable change, so dsl takes a patch bump to 0.5.1.

4. The range and build-integrity guards ran only in the publish workflow,
   so a pull request restoring `workspace:*` or leaving a stale generated
   file failed at release time, after a version number had been spent.
   ci.yml gains a cheap source-level range check
   (scripts/check-internal-dependency-ranges.mjs) and the
   `git diff --quiet HEAD` integrity check after its install step. The
   tarball assertion stays where it is as the release-time proof of
   actual published behaviour.

Merging this now publishes @openmaic/dsl@0.5.1 as well as
@openmaic/storage@0.1.2, @openmaic/renderer@0.0.4 and
@openmaic/importer@0.1.2.

* fix(release): fail the gates closed, and state what they actually prove

Confirmation round on the release-path fixes.

1. The format-version rule failed open if version.ts moved. It hard-coded
   one source path and returned successfully when either revision lacked
   it, so renaming the file, changing DSL_VERSION and bumping dsl by a
   patch passed both checks. It now reports an error whenever the file is
   absent at either revision while dsl has any publishable change. The
   constant regex is anchored to the start of a line, so a commented-out
   declaration cannot be read as the value, and a duplicate declaration is
   an error rather than a coin flip.

2. "At least a MINOR" was wrong past 1.0.0: `^1.0.0` admits minors, so a
   format change shipped as a minor would float into published dependents
   exactly as the rule exists to prevent. The rule is now stated as what
   it needs to be — an increase the dependents' caret range will NOT admit
   — computed from the pre-change version: a minor while dsl is 0.x, a
   major once it reaches 1.0.0. The error message and the docs in
   version.ts say the same thing.

3. Internal declarations outside `dependencies` bypassed both range
   checks. Keeping `dependencies` at `workspace:^` while adding an exact
   `peerDependencies` or `optionalDependencies` entry published a second,
   tighter constraint and passed. An owned package must now appear exactly
   once, in `dependencies`; entries in the peer and optional fields are
   rejected outright, in the source check and in the packed-manifest
   assertion. The global count test is replaced by naming the three
   dependents that must be present.

4. The format rule compared against the base tip, so an un-rebased branch
   produced an inverted message. It now uses the merge base, matching the
   `base...HEAD` form the rest of diff mode uses. Deliberately not applied
   to the version comparison, where the tip is the correct reference.

5. Phase 2 had no baseline, so cumulative counters from an earlier run
   satisfied it forever against a non-ephemeral database. It now captures
   the counters before the vitest step and requires a positive delta.
   Both workflows gained the capture step.

6. Phase 2 overclaimed. It proves rows were inserted into those five
   tables during the run, not that the built PgDocumentStore and
   PgRuntimeStore inserted them — `test/` is on the publishable-input
   ignore list, so test code could write directly. The success message now
   says exactly that, and the script documents the threat model: this
   catches accidental silencing, which is the defect it was written for;
   it is not a defence against someone who can merge changes to `test/`,
   who can alter production code just as easily.

Also: scripts/openmaic-packages.mjs is now the single source for the
owned package list, consumed by all three checking scripts, and it
cross-checks itself against both the packages directory and
publish-packages.yml so a package added in one place and forgotten in
another fails rather than going quietly exempt. The workflow's own YAML
lists cannot import it, which is why they are cross-checked instead.

* fix(release): let the format-version source move without opening the gate

Failing closed on a missing version.ts is right, but on its own it made a
rename impossible to merge: the new path does not exist at the base
revision either, so the check refused both before and after the move.

DSL_VERSION_SOURCES is now an ordered candidate list, resolved per
revision against the first path that exists there. A rename lands by
prepending the new path in the same change, and the comparison still
happens across the move. Absent everywhere at either revision, while dsl
has any publishable change, remains an error.

* fix(release): report package-list drift instead of crashing on it

Every check in check-internal-dependency-ranges.mjs reads the shared
package list, so a list naming a package that does not exist threw an
ENOENT out of readManifest before any finding was printed. It failed
closed, but unreadably. The list is now validated first and fatally, with
its own headline.

* fix(release): anchor the caret to the base branch, and check the gates' own inputs

Final confirmation round.

1. The caret escape was computed from the merge base, which is the wrong
   reference for it. With merge base dsl 0.5.1, an ordinary minor landing
   on main taking the tip to 0.6.0, and HEAD at 0.6.1 with a changed
   DSL_VERSION, the rule computed the escape from 0.5.1, got 0.6.0, and
   accepted 0.6.1 — but dependents released against 0.6.0 publish
   `^0.6.0`, which admits 0.6.1, so the new format reached them anyway.
   0.6.1 is also the minimum the ordinary version check allows, making
   that the default outcome rather than an unlucky one.

   The escape is now computed from the HIGHEST dsl version reachable on
   the base branch. Whether the format moved still comes from the merge
   base, because that is a question about the branch. The comment on
   mergeBaseWithHead now records that it is a no-op in both current CI
   invocations (checkout uses the merge ref on pull_request, and the
   before-SHA is an ancestor on push) rather than claiming a case that is
   not exercised there.

2. The workflow cross-check matched raw text, so a commented-out trigger
   path still satisfied it while silently disabling that package's
   release. Full-line comments are now stripped first; the `on.push.paths`
   list and every `for pkg in ...` loop are compared as exact sets, so an
   unexpected entry fails as well as a missing one; and `--filter` usage
   is checked more loosely, because individual steps legitimately filter
   subsets. The limits of a textual check are stated where it lives.

3. `locate()` returned the first candidate format-version source that
   existed, so during a half-finished rename the gate could compare a file
   that is no longer the exported one. It now resolves every candidate per
   revision and fails unless exactly one declares the constants.

4. INTERNAL_DEPENDENTS was checked in one direction only, so deleting an
   entry exempted that package from the source check and from the packed
   assertion that iterates the same map. It is now cross-checked both
   ways, an owned package in another owned package's devDependencies is
   rejected outright, and multiple owned dependencies no longer collapse
   to whichever was declared last.

5. assertPackageListIsComplete() now also runs in release mode, which is
   the gate that decides what gets published; and the PostgreSQL script
   consumes flag values properly instead of treating the first non-flag
   argument as the results path.

Also documents, in openmaic-packages.mjs, the threat model these gates
share: they catch mistakes, and anyone who can merge edits to scripts/
can edit production code directly, so deliberate subversion is out of
scope and cannot be closed by adding further checks.
 2026-08-02 22:44:34 -04:00
wyucandClaude Opus 5 cc95851dc5 chore(packages): publish storage and enforce version bumps (#998) 
* chore(packages): publish storage and enforce version bumps

* fix(packages): cover all publish inputs and tarballs

* fix(packages): scope version guard to release inputs

* fix(packages): fail closed on new package inputs

* docs(packages): clarify version guard boundaries

* fix(packages): harden release guard edge cases

* fix(packages): align smoke peers and git inputs

* docs(packages): make release smoke policy explicit

* fix(packages): anchor version guard at repository root

* fix(packages): report version guard setup errors

* fix(packages): harden release validation

* fix(ci): validate complete main pushes

* test(storage): pin the exported PostgreSQL schemas

DOCUMENT_PG_SCHEMA and RUNTIME_PG_SCHEMA are public API. A deployment that
provisions these tables with its own migration tooling has to reproduce the
DDL exactly for ensureDocumentSchema() / ensureSchema() to stay the intended
no-op against an already-provisioned database.

Nothing guarded that today: every statement is CREATE ... IF NOT EXISTS, so
PostgreSQL silently accepts whatever table already exists under the name. A
column type, a nullability, an index or a FK action can drift apart from a
downstream migration without an error, and the first symptom is a store query
failing in production or succeeding against the wrong types.

Pin both constants verbatim, and assert every statement stays IF NOT EXISTS
guarded so the ensure functions remain idempotent. The pin does not judge the
DDL; it makes changing it impossible to do by accident.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): gate publishing on the version check

The version check ran only in ci.yml. publish-packages.yml is a separate
workflow, so it could reach `pnpm publish` on a main push, a matching tag, or
a manual run with dry_run=false even when CI was red, skipped, or never ran.

Run the validation inside the publish job, before publishing, in a new release
mode that works for every trigger. A push range only exists for branch pushes,
so release mode judges each package against the registry instead: a version
that is not published yet is a release, and a version that is already
published is accepted only when the package source has not moved since that
release. `pnpm publish` skips an already-published version, so without that
second half a drifted package releases as a silent no-op.

The anchor for "since that release" is the @openmaic/<name>@<version> tag the
job now writes after every successful publish, falling back to the push range
for packages released before those tags existed. With neither anchor the
release stops instead of guessing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): make the release gate prove only what it can

Three independent reviews of the previous revision converged on one root
cause: it used git tags and a push range as the record of which tree produced
a published version. Neither is trustworthy. Tags are mutable and can be
created or moved by hand, so "the tag resolves" was being read as "the tag is
authentic"; pnpm records no gitHead; and a push range describes one push, not
the origin of a release. Packages released before the scheme existed had no
anchor at all, which made the tag and manual triggers fail closed from the
first run with no way to self-heal, dry runs included.

Drop the anchor machinery. Drift is prevented where it is provable, in diff
mode at merge time, which sees every commit before it can be released. The
pre-publish gate is now limited to claims a release can establish: every
version it publishes is new and moves forward, an already-published version is
reported and left alone, and any registry answer that is not a definitive
404 stops the release.

Around it, close what the reviews found in the workflow:

- real publishes only from a commit contained in main, so a tag or manual run
  cannot ship an unreviewed ref while holding NPM_TOKEN
- one repository-wide concurrency group, so a main push and a tag push cannot
  each decide the same version is unpublished
- publish and mark each package individually, so a partial failure stays
  retryable instead of leaving published packages unrecorded
- fail if the build rewrites tracked package files, which would otherwise
  publish content that is not in the released commit
- compare against every published version when ordering, and refuse to guess
  when the registry holds versions this check cannot order

The schema pin also asserted the constants without asserting what the ensure
functions execute, so those could diverge. Both are now run against a
recording queryable and their exact statement sequence is asserted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): refuse to release from a tree behind the registry

Simplifying the gate introduced a hole that a mutation check caught: a package
whose local version is already published took the "already published, skip"
path before any ordering check, so a tree rolled back to an older published
version passed silently. That is not a harmless no-op, because pnpm publish
rewrites each workspace dependency to the version in the tree, and a sibling
released alongside it would be published declaring a dependency on the older
package.

Order every package against the registry first, then decide: at the highest
published version it is a legitimate skip, below it the release stops.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): couple publishing to a green CI run for the same commit

Second review round, on the reworked design. The headline defect was a hard
blocker: `git fetch --depth=0` is not valid git (that is an actions/checkout
convention), so under `set -e` every real publish aborted before the gate ever
ran. Remove the depth argument; the checkout already has full history.

The rest closes the gap both reviewers reached independently, that being on
main is not the same as having passed the gate:

- wait for this exact commit's CI conclusion and require success. ci.yml runs
  concurrently with this workflow on a push to main and blocks nothing, so an
  unpublished version whose source drifted could be released while its own
  version check was still running or already red.
- require the commit on main's FIRST-PARENT history. Plain reachability also
  accepts every intermediate commit of every branch merged with a merge commit.
  A pull request that adds a bad tree and reverts it in the next commit leaves
  that tree an ancestor of main forever, and a tag pointing at it published it.
- do not cancel CI runs on main. Each run validates only its own push range, so
  cancelling one drops the range that contained the change, and its replacement
  compares against the cancelled tip and sees nothing.
- run the version check on every main push. A branch creation or force push
  reports an unusable `before`, which used to skip the gate entirely; fall back
  to the first parent instead.
- publish with --ignore-scripts. Every package's prepublishOnly reruns the
  build, deleting and regenerating the dist that was just verified and
  smoke-tested, so npm could receive bytes nothing had checked.
- run storage's PostgreSQL contract suites before publishing it. They skip
  themselves without a database, so the one backend that needs a real
  PostgreSQL was shipping unexercised.
- order registry versions with real semver, prereleases included. Rejecting
  every non-x.y.z version meant one historical prerelease anywhere in one
  package's history would refuse every future release of every package.
- warn when a release marker exists but points at another commit.

Note what is NOT enforceable here: a workflow_dispatch or tag run executes the
workflow definition from the selected ref, so a branch that edits this file can
delete these checks. The boundary has to be a GitHub Environment holding
NPM_TOKEN behind a main-only deployment branch rule. The job now declares that
environment and the header states the required setup; the in-file checks are
defence in depth.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): stop the first-parent check failing on SIGPIPE

Third review round. The blocker this time: `git rev-list --first-parent | grep
-qx` under `set -o pipefail`. grep exits at the first match, rev-list then dies
of SIGPIPE, and the pipeline returns 141, so a commit that IS on main is read
as "not on main" and every real publish is refused. It does not reproduce in a
shallow clone, where main's history fits in the pipe buffer; with the full
history this job checks out it always will. Count matches instead, which reads
the whole stream.

Also from this round:

- identify the CI run by workflow file and triggering event rather than by a
  check-run display name. Names are not unique, so any other workflow or app
  publishing a check called "Lint, Typecheck & Unit Tests" could stand in for a
  red version gate.
- fail the main-push version check when the push range is unusable instead of
  substituting HEAD^. A force push can replace many commits at once, so the
  previous commit is not the range that needs checking, and a green run here is
  what publishing depends on.
- correct the stated limitation. It claimed the package-directory input model
  was exact for dsl and storage; it is not. Their dist is whatever the
  lockfile's TypeScript emits, and dsl's shipped JSON schema comes from the
  lockfile's ts-json-schema-generator, so a toolchain bump can change any of the
  four tarballs with no diff under the package directory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(packages): make a merged version bump the only release input

The tag trigger was incoherent. Its own documentation admitted that any
@openmaic/* tag republishes the whole family and that which packages actually
go out is decided by the manifests, so the tag name never participated in the
decision. It was a second entry point to one decision function, it bypassed
review entirely because a tag is not a diff and can be created by anyone with
write access on any commit, and it cannot be guarded: environment deployment
rules match GITHUB_REF, so refs/tags/* and a main-only rule are mutually
exclusive.

Drop it. A version bump that landed on main is now the only release input, and
`@openmaic/<name>@<version>` tags are an output written after a package
reaches the registry. Nothing is lost: a manual dispatch from main still
republishes without a new commit, and that path is inside the protected
environment.

Split the workflow so the token has a boundary. `validate` holds everything
that does not need NPM_TOKEN and runs from any ref, which keeps dry runs
useful to contributors. `publish` is the only job declaring the `release`
environment, so the token is never attached to a run that is merely
validating. It rebuilds rather than sharing state, which is the right trade
for a release.

Also add the `actions: read` permission the CI-conclusion query needs; it was
switched from the checks API to the Actions API without updating the scope,
which would have failed with 403 on every real publish.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(contributing): explain when a package version must be bumped

CONTRIBUTING said nothing about versioning or releases, while this branch adds
a CI check that fails a contributor's PR with "publishable package inputs
changed but version did not increase". A gate that rejects work without
telling anyone the rule is the kind that ends up disabled.

State the rule, the exact failure message, which files are exempt, and how to
choose the number, with a specific warning for @openmaic/dsl: it is the
contract the other packages validate against, so narrowing what an existing
document may contain is breaking even when the diff is small.

Also state what contributors do NOT do: publishing is automatic once a bump
lands on main, and the release tag is a marker written afterwards rather than
something to push.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): run renderer's tests before publishing renderer

The pre-publish gate ran renderer's typecheck but never its test suite, so the
one package whose behaviour is hardest to typecheck was the one shipping
unexercised. It has 25 test files and 242 tests; they just were not in the
filter list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(packages): keep the git credential away from package code

Review finding: the publish job held `contents: write` while
actions/checkout had persisted that credential, and it then ran
`pnpm install` and the package builds. Any dependency or build script
executed with a credentialed git remote in reach and a write scope to use
it, which is a wider blast radius than publishing needs.

Separate the two capabilities so no job holds both:

- `validate` and `publish` run package code. Both now check out with
  `persist-credentials: false`, `validate` is pinned to `contents: read`,
  and `publish` drops write scope entirely, keeping only the `actions: read`
  it needs for the CI conclusion and `id-token: write` for provenance.
- a new `mark` job is the only holder of `contents: write` and a git
  credential, and it installs nothing and builds nothing.

`mark` also reconciles instead of only recording this run. A marker whose
push failed previously could never be repaired before, because the release
plan excludes versions already on the registry and the marker logic sat
behind that plan. It now marks any registry version whose tag is missing,
and leaves an existing tag alone rather than moving it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-07-29 23:54:23 -04:00
Yanpeng Wang b5e1963e48 feat(document): rename to Document Parsing, show supported formats, extend MinerU support 
Squashed from PR #837 after review approval and green CI.
 2026-07-09 04:29:58 -04:00
wyuc 7cb129183d chore(packages): publish the @openmaic/* SDK family to npm (#778) (#780) 
* chore(packages): publish the @openmaic/* SDK family to npm (#778)

Prepares the @openmaic/{dsl,renderer,importer} family for its first npm
publish, and moves the SDK packages onto the @openmaic scope.

Why the scope move: the @maic org name is unavailable on npm (an unscoped
`maic` package already holds the name), so @maic/* is not claimable. @openmaic
matches the project name, the scope is free, and the repo already ships an
@openmaic/docs package — so the SDK family now lines up with that convention.

- rename @maic/{dsl,renderer,importer} -> @openmaic/* across packages, the
  workspace glob, the package dir, and all import sites; lockfile regenerated
- renderer: add publishConfig (public, registry.npmjs.org) — was missing, so
  a scoped publish would default to the wrong registry / restricted access
- importer: add a files allowlist (dist, README, LICENSE) and drop the
  fragile .npmignore blacklist that shipped src; add an exports map so ESM
  consumers resolve dist/index.js instead of falling back to the .cjs main
- all three: add a prepublishOnly build (+ test/typecheck) guard so a publish
  can never ship a stale or empty dist
- add a tag-triggered publish workflow with npm provenance, pinned by name to
  the three @openmaic packages so the vendored forks (mathml2omml, pptxgenjs)
  are never published

Refs #778, #720 (Phase 1).

* fix(packages): address cross-review on the @openmaic publish prep

Cross-review (Claude /code-review + codex) on this PR surfaced:

- renderer's advertised CJS entry was broken: it keeps @openmaic/dsl external
  and imports a runtime enum from it, but dsl is ESM-only (no `require`
  condition), so `require('@openmaic/renderer')` would throw
  ERR_PACKAGE_PATH_NOT_EXPORTED. Make renderer ESM-only: drop the `.cjs`
  rollup output, `main` now points at the ESM build, and the `require`
  conditions are removed from `exports`. (importer is unaffected — it bundles
  dsl, so its CJS build still works.)
- prepublishOnly re-ran the test suite during `pnpm -r publish`, so a flaky
  test after dsl had already published gave a non-atomic partial release.
  Reduce prepublishOnly to a build-only guard (never ship stale/empty dist)
  and move the real test/typecheck gate into the workflow, before any publish.
- document that an @openmaic/* tag publishes the whole family via `pnpm -r`
  (pnpm skips already-published versions); the tag is a release marker, not a
  per-package gate.

Verified: dsl + renderer + importer build; renderer emits ESM only (0 .cjs),
all exports entries resolve; `npm pack` ships dist + README + LICENSE with no
src leak; frozen-lockfile passes.

Refs #778.

* style: reflow @openmaic/dsl type imports past print-width after rename

The @maic -> @openmaic rename lengthened two single-line type imports past
prettier's 100-col width; prettier --check flagged them. Pure formatting.

Refs #778.

* docs(importer): mark @openmaic/importer browser-only (cr-loop accepted limitation)

codex cross-review flagged that the published @openmaic/importer throws
`XMLHttpRequest is not a constructor` when loaded in a pure Node process —
its rollup build is browser-targeted (`nodeResolve({browser:true})` + a
browser pdf.js build). The app only consumes it client-side ('use client'),
so this is by design. Document it as an accepted limitation: prominent
browser-only note in the README and a `browser` field in the manifest.

Refs #778.
 2026-06-24 15:12:23 +08:00
xuyuanwei678CursorClaude Opus 4.7
 		7bb9e864ae feat(packages): introduce maic-import and maic-renderer workspace packages (#668) 
* feat(import): add pptxtojson-pro workspace package and stage-1 PPTX import

Vendor the TypeScript pptxtojson-pro parser as a workspace package, sync its
bundle to public/vendor at install time (avoids Turbopack dynamic require
issues), and wire a home-page import hook that parses files for inspection.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(import): extend pptxtojson-pro with PPTX → Slide[] pipeline

`importPptx` now turns a .pptx file directly into canvas `Slide[]`,
optionally uploading every base64 image and blob-backed media via a
caller-supplied OSS callback before resolving.

- Vendor OpenMAIC slide types and shapes config into `src/openmaic/`
  so the bundled dist is self-contained and loadable via `/vendor/`
  URL under Turbopack; add stubs for `resolveFont`, `getSvgPathRange`
  and `parseVideoCodec` until the real PPTist helpers are ported.
- Move `import-pipeline/` under `src/` and re-export `importPptx`,
  `parsedToSlides`, `OssUpload`, `ImportPptxOptions` and `CanvasSlide`
  from the package main entry.
- Add `nanoid`, `katex` and `pptxtojson` as runtime deps of the package.
- Update `useImportPptx` to accept `upload` / `onImported` and return
  the converted `Slide[]`.
- Document the new API surface, error semantics and Turbopack workaround
  in the package README.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(slide-renderer): package skeleton (rollup + tsc + empty index)

Workspace package scaffolding: package.json with peer deps (react/react-dom/motion/tailwindcss),
rollup ESM+CJS config, tsconfig (strict, jsx react-jsx), and empty entrypoints under src/.

Verified: pnpm --filter slide-renderer build produces dist/{index,elements/index,types/index}.{js,cjs,d.ts}
(empty chunks are expected at this stage).

* docs(slide-renderer): DESIGN.md + README.md

* feat(slide-renderer): copy Slide/Element types + add effects types

* feat(slide-renderer): add utils (geometry simplified to elements API, element ranges, cn)

* feat(slide-renderer): shared element hooks (fill/outline/shadow/flip) + ElementOutline

* feat(slide-renderer): BaseTextElement

* feat(slide-renderer): BaseShapeElement + GradientDefs + PatternDefs

* feat(slide-renderer): pure <img> BaseImageElement, decouple OpenMAIC media business

The base component drops:
- useMediaGenerationStore / isMediaPlaceholder (AI media generation polling)
- useSettingsStore (imageGenerationEnabled toggle)
- useMediaStageId / retryMediaTask (classroom-scoped retry orchestration)
- useI18n (placeholder/error/disabled copy)
- skeleton / error / disabled render branches

Retains: clip / filter / outline / shadow / colorMask, output <img src={elementInfo.src}>.

Adds a `renderImage` slot so consumers can re-inject business behavior (placeholders,
retry buttons, custom src resolvers) without modifying the package.

* feat(slide-renderer): Line / Table / Latex / Code base elements

- Line + LinePointMarker: copied with stroke-draw animation intact
- Table + StaticTable + tableUtils: copied with getTableSubThemeColor moved to package utils
- Latex: katex HTML + legacy SVG fallback, no business deps
- Code: shiki-based syntax highlighting (dynamic import); shiki added as optional peer
  dependency so consumers without code slides pay zero install cost

* feat(slide-renderer): BaseChartElement + Chart + chartOption (echarts as optional peer)

* feat(slide-renderer): pure <video> BaseVideoElement, decouple OpenMAIC media business

Drops:
- useCanvasStore (playingVideoElementId / pauseVideo)
- useMediaGenerationStore / useMediaStageId / retryMediaTask / getVideoMediaRefForElement
- useSettingsStore (videoGenerationEnabled)
- useI18n, motion/react animate scope
- skeleton / error / disabled / placeholder branches

Retains <video src controls preload=metadata>. Adds a `renderVideo` slot so
consumers can re-inject orchestrator-driven playback or placeholders.

* feat(slide-renderer): canvas hooks (useSlideBackgroundStyle, useViewportSize simplified)

- useSlideBackgroundStyle: copied as-is (no business deps)
- useViewportSize: rewritten to take viewportSize/Ratio/canvasPercentage as args,
  drop useCanvasStore, drop editor-only canvasDragged/dragViewport, return fitScale
  instead of writing to a store

* feat(slide-renderer): 4 effect overlays (Highlight/Spotlight/Laser/Zoom), props-driven

All overlays now receive element/geometry/options through props instead of reading
useCanvasStore. SpotlightOverlay uses a configurable element-id prefix
(default 'slide-element-') to locate DOM nodes for measurement, matching the
convention SlideElement will set in T14.

HighlightEffectOptions extended with color/opacity/borderWidth/animated to surface
the original visual knobs through the public API.

* feat(slide-renderer): SlideElement dispatcher with renderImage/renderVideo slots

* feat(slide-renderer): SlideCanvas v1 main entry (props-driven, auto-fit viewport)

Mirrors ScreenCanvas structure but reads from props: slide.elements, slide.theme,
props.background ?? slide.background, props.scale ?? autoFit, props.effects.

Auto-fits viewport into container by default via useViewportSize+ResizeObserver;
pass scale=1 to opt out and render at slide-native dimensions.

* feat(slide-renderer): SlideRendererProvider + useSlideContext (props or context)

SlideCanvas now reads from context as a fallback when props are omitted. Allows
the wrap-then-compose pattern from spec §3.2: provider holds shared data,
sibling overlays read via useSlideContext.

* feat(slide-renderer): aggregate public exports

Main entry exposes SlideCanvas, SlideElement, Provider/hooks, effects, hooks,
utils. `slide-renderer/elements` re-exports all 9 Base*Element + shared
ElementOutline/useElementXxx for granular composition. `slide-renderer/types`
already lived in src/types/index.ts.

* docs(slide-renderer): complete README (install / quickstart / API / effects / Tailwind setup)

* feat(demo): /slide-renderer-demo route exercising the workspace package

Hand-written single Slide with shapes + text demonstrates the v1 API end-to-end:
- props-driven SlideCanvas
- auto-fit viewport via parent container
- gradient background, shape outline + text fill

Wires slide-renderer as a workspace dep and registers its dist as a Tailwind
@source so utility classes inside the package get picked up by the host PostCSS
pipeline.

* feat(pptxtojson-pro): align imported Slide JSON with renderer contract

- Drive `viewportWidth` from the deck's actual pixel width (json.size.width
  × ratio) instead of the legacy 960 default, so 16:9 widescreen decks no
  longer get text element widths clamped to 4:3 dimensions (e.g. titles
  truncated to 842px and forced to wrap). Per-slide `viewportSize` now
  reports the same value, keeping transform-time and render-time aligned.
- Replace the pass-through font STUB with real resolution: 22-font
  self-hosted whitelist + alias map (微软雅黑/等线/宋体/楷体/仿宋/Arial/etc.)
  → 5-category primary font mapping, so unknown PPT fonts fall back to a
  rendered open-source equivalent and the pipeline can surface the
  replacement via `replacedFonts.fallback`.
- Translate cell `vAlign` from PPT-native (`up`/`mid`/`down`) to CSS-native
  (`top`/`middle`/`bottom`) at the importer boundary, so the renderer can
  transparently pass it through to `vertical-align` / `justify-content`
  without per-component conversion.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(slide-renderer): drop Tailwind defaults, data-drive styles, ship fonts

Three intertwined changes that together make the package self-contained
and align it with the "slide JSON is the single source of visual truth"
contract.

1. Remove Tailwind utility classes from every element renderer.
   - Replace `className="absolute ..."` / `className="rounded ..."` etc.
     with inline `style={{ position: 'absolute', borderRadius: ... }}`.
   - The few rules that can't be expressed inline (descendant selectors,
     keyframes, browser-default <p> margin reset) move to a new
     `src/styles.ts` injected once by `<SlideCanvas>` via `<style>`.
     Named animation classes (`slide-renderer-pulse` / `-ping`) replace
     `animate-pulse` / `animate-ping` so consumers don't need Tailwind.

2. Strip renderer-level positive defaults; surface them as data fields.
   - `BaseTextElement`: drop hardcoded `overflowWrap: 'break-word'` and
     the `paragraphSpace ?? 5` fallback. `--paragraphSpace` is only set
     when defined in data; CSS `var()` fallback is 0.
   - `StaticTable`: drop hardcoded `padding: '5px'`, `verticalAlign:
     'middle'`, `wordBreak: 'break-word'`. Extend `TableCell` with
     `padding?: string` and `vAlign?: 'top' | 'middle' | 'bottom'` so
     PPT-extracted padding / vertical alignment flow through unchanged.
   - `PPTTableElement` gets `rowHeights?: number[]` (per-row min-height;
     content can still grow rows).
   - Reshape `<td>` to match the classroom Vue layout: outer `<td>` carries
     only border + textStyle; inner `slide-renderer-cell-text` <div> is
     a flex column with `min-height = rowHeight - 4`, `padding =
     cell.padding`, `line-height: 1`, `justify-content` from `cell.vAlign`.
     Add `.slide-renderer-cell-text p + p { margin-top: 0.4em }` to give
     adjacent paragraphs breathing room without breaking the reset.

3. Self-contained font shipping.
   - Move the 22 woff2 files into `packages/slide-renderer/fonts/`.
   - Ship a sibling `fonts.css` with @font-face declarations using
     relative `./fonts/<name>.woff2` URLs.
   - Expose both via `package.json` exports (`./fonts.css`, `./fonts/*`)
     and the `files` array.
   - Consumer adds one import (`import 'slide-renderer/fonts.css'`); the
     bundler resolves the relative URLs against the package and emits
     each woff2 as a static asset. Removes the dependency on consumer-
     owned `app/globals.css` @font-face blocks and `public/font/` copies.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(demo): wire slide-renderer fonts and swap demo to a table-heavy slide

- Import `slide-renderer/fonts.css` in the root layout so consumer apps
  pull the package's self-hosted @font-face declarations and woff2 assets
  through Next.js' static-asset pipeline.
- Replace the hardcoded `demoSlides` array (24 mixed-element slides) in
  `/slide-renderer-demo` with a single 5×7 table-heavy slide so the demo
  exercises the new `TableCell.padding` / `TableCell.vAlign` /
  `PPTTableElement.rowHeights` fields and the flex-based cell layout.
  Convert the legacy `vAlign: 'up' | 'mid' | 'down'` values to the
  CSS-native `'top' | 'middle' | 'bottom'` form the renderer now expects.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(slide-renderer): add slideToPng standalone snapshot util

Exposes a new `slide-renderer/snapshot` sub-export with `slideToPng(slide,
options)`, the foundation for the AI-driven visual diff loop (snapshot
the rendered slide, compare against PowerPoint's own PNG export, decide
whether a discrepancy belongs to the import algorithm or the renderer).

Implementation notes
- Mounts `<SlideCanvas>` into an off-screen container sized exactly to
  the slide's native pixels, so `useViewportSize` collapses to a 1:1
  render (no scale offset).
- `flushSync` for the initial commit so the snapshot doesn't fire on an
  empty container; `requestAnimationFrame` × 2 to let the ResizeObserver
  settle; then waits for `document.fonts.ready` and every `<img>` in the
  container (with a configurable `timeoutMs`, default 5s).
- Snapshots the SlideCanvas root (`container.firstElementChild`) rather
  than the outer wrapper — html2canvas-pro's bounding-box math otherwise
  leaves white edges or drops absolutely-positioned children.
- Uses `html2canvas-pro` instead of `html-to-image`: the latter's SVG
  `<foreignObject>` path loads embedded woff2 asynchronously, so the
  drawImage fires before fonts decode and text re-wraps in the fallback
  font. html2canvas-pro walks the DOM and inherits the parent document's
  font registry, keeping wrap positions identical to the on-screen render.
- `onclone` hook injects a small CSS reset (`font-kerning: none`,
  `font-feature-settings: normal`, `font-variant-east-asian: normal`,
  `text-rendering: geometricPrecision`) scoped to the slide tree, to
  prevent html2canvas-pro's CJK text measurement from clipping
  full-width punctuation (e.g. `（` / `）`) past cell boundaries.
- Default `pixelRatio` follows `window.devicePixelRatio` for retina-sharp
  output; consumers can override.
- Optional `debugVisibleMs` briefly shows the off-screen container after
  snapshot so callers can compare what was captured vs the live render.

Wiring
- Sub-export added to `package.json` (`./snapshot`) and Rollup `entries`
  for tree-shaking — apps that don't use snapshot don't pay the
  `html2canvas-pro` bundle cost.
- Demo route adds an "导出 PNG" button that calls `slideToPng` on the
  active slide and triggers a browser download.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: build slide-renderer in root postinstall

Without this, fresh installs leave packages/slide-renderer/dist
missing and Next.js fails to resolve the `slide-renderer` import.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(parser+renderer): iteration-0601-002 fidelity fixes

Parser (pptxtojson-pro):
- shapeSerializer: stop painting parent fill on grpFill children; honor
  explicit <a:ln><a:noFill/></a:ln> over theme lnRef (master logo dots
  were rendering as visible green rings).
- imageSerializer: detect custGeom ellipses and 8-vertex chamfered-rect
  photo frames; resolve to 'ellipse'/'octagon' instead of falling back
  to 'rect'.
- textSerializer: emit text-shadow from rPr effectLst > outerShdw, and
  compensate the half-leading PowerPoint puts above the first line for
  unitless lnSpc > 1 (cover-title gap regression).
- border/tableSerializer: pass rgba()/hsla() colors through ensureHex
  untouched so '#rgba(...)' never reaches the DOM.
- transformParsedToSlides: drop the 1.1 SAFE_PADDING wrap-fudge and
  thread bodyPr@anchor through as vAlign.
- font config: add 思源黑体/思源宋体 + Source Han aliases and a trailing
  region/weight suffix stripper so 'SourceHanSerif CN Light' resolves.

Renderer (slide-renderer):
- SlideCanvas: new chrome prop (default true) to opt out of the
  preview-only card shadow + rounded corners; snapshot pipeline passes
  false so html2canvas output matches PPT edges.
- BaseTextElement: flex-column + justifyContent driven by the new
  vAlign field; width/height stay 100% so the box anchors correctly.
- BaseImageElement: maxWidth/maxHeight 'none' to prevent host CSS from
  shrinking absolutely-positioned <img>.
- snapshot: render with chrome=false.

Docs: SKILL.md expanded with the iteration workflow + OOXML traps.

* fix(parser+renderer): iteration-0605-001 fidelity fixes

slide-renderer:
- PPTImageElement: add `softEdge` field (feather radius that fades image
  alpha to transparent at every edge, per a:softEdge@rad).
- TableCell: add per-side `borders` so cells with partial dividers render
  each side independently instead of falling back to the table-level
  uniform outline.
- BaseTextElement: hoist `fill`/`opacity` to the outer wrapper so the
  background covers the full shape rectangle, matching PowerPoint behavior
  for short text inside tall shapes.
- Misc tweaks across image/latex/shape/video base elements, StaticTable
  and the standalone snapshot util.

pptxtojson-pro:
- Parser/serializer fidelity improvements for text, math, image, shape
  and table (notably ~500 lines in textSerializer and ~127 in
  mathSerializer).
- Refinements in GroupNode, MathNode and ShapeNode models; richer Theme,
  StyleResolver and RenderContext plumbing.
- Expanded shape presets and customGeometry utilities, plus adapter and
  import-pipeline updates to surface the new fields.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: stop tracking local slide-renderer-demo sandbox

`app/slide-renderer-demo/page.tsx` is a personal scratchpad used for
ad-hoc rendering tests; it shouldn't ship with the package. Remove it
from the index (file stays on disk locally) and ignore the directory so
future tweaks don't show up in `git status`.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: rename packages pptxtojson-pro→maic-import and slide-renderer→maic-renderer

Align workspace package names with the MAIC product family:
  packages/pptxtojson-pro  → packages/maic-import
  packages/slide-renderer  → packages/maic-renderer

Functional references updated end to end:
- Root package.json: workspace deps + postinstall/sync scripts
- next.config.ts: transpilePackages
- app/globals.css: Tailwind `@source` for renderer dist
- app/layout.tsx: `maic-renderer/fonts.css` import
- lib/import/use-import-pptx.ts: imports + runtime URL
  (`/vendor/maic-import/index.js`) + script comment
- scripts/sync-maic-import.mjs: renamed and pointed at the new dist
- .gitignore: vendor path and sync-script comment
- Package package.json `name` fields and cross-link in READMEs

Also re-include the gitignored `app/slide-renderer-demo/` in Tailwind's
source detection via an explicit `@source` so its classes (e.g.
`bg-violet-600`) keep getting generated — Tailwind v4 skips gitignored
paths by default, which was making the demo's import button render
white-on-white after the demo was excluded from version control.

CSS class names (`slide-renderer-prose` / `slide-renderer-cell-text` /
`slide-renderer-pulse` / `slide-renderer-ping` / `slide-renderer-code-
cursor-blink`) and internal source comments that still reference the old
package names are intentionally left untouched to keep the rename
diff minimal; they can be updated separately as the codebase evolves.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(dsl): extract @maic/dsl contract package and adopt @maic/* scope

Establish the SDK family's contract keystone and align package naming with
the planned @maic/* scope, per PR #668 review.

1. New @maic/dsl package (packages/maic-dsl)
   - Pure, zero-runtime-dependency spec: the canonical slide object model,
     pure type-guards, and DSL version/migration scaffolding.
   - Seeded from lib/types/slides.ts but reconciled into a *superset* of the
     three previously-drifted copies (app / renderer / importer); merged
     fields are annotated @since-merge. README documents the divergence and
     the acyclic dependency arrows (renderer/importer -> dsl -> nothing).
   - Enums are regular (not const) so consumers compiling with
     isolatedModules can import them across the package boundary (TS2748).

2. Wire @maic/importer to @maic/dsl
   - Import all slide types from @maic/dsl; delete the vendored
     openmaic/types/slides.ts copy.
   - Eliminate the Slide drift at the root: transformParsedToSlides now fills
     viewportSize/viewportRatio/theme at construction and emits a complete
     DSL Slide, removing the partial "draft slide" + post-fill step in
     parsedToSlides (no DraftSlide type needed).
   - Add @maic/dsl to the root postinstall build chain before the importer.

3. Adopt @maic/* scope (agent-noun names)
   - maic-import  -> @maic/importer
   - maic-renderer -> @maic/renderer
   - Update all package-name references (deps, app imports, transpilePackages,
     fonts.css subpath import, Tailwind @source node_modules path, READMEs).
   - Directory names and /vendor paths are intentionally left unchanged.

Note: @maic/renderer still vendors its own slide types; wiring it to
@maic/dsl is the next step.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(renderer): consume slide types from @maic/dsl, drop vendored copy

Wire @maic/renderer to the @maic/dsl contract keystone, completing the
de-duplication of the slide object model across the SDK family.

- Delete packages/maic-renderer/src/types/slides.ts (the vendored copy) and
  repoint all ~30 internal imports at @maic/dsl. SlideElement's runtime
  `ElementTypes` value import now resolves to the DSL enum.
- Re-export the DSL types from src/types/index.ts so the public
  `@maic/renderer/types` entry keeps exporting the full slide contract
  (alongside the renderer-only effect types).
- Add @maic/dsl as a regular dependency and mark it external in the rollup
  build so consumers share a single copy (enum not inlined); the emitted
  .d.ts references @maic/dsl directly.

@maic/dsl is a superset of the renderer's former types, so this is
non-breaking; the renderer additionally gains `script` and the
importer-origin fields it didn't previously declare.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(renderer): serve fonts from OSS, drop the 43.6MB bundled woff2

Move the 22 CJK fonts off the repo and onto object storage to fix the
PR #668 blocking repo-weight concern.

- fonts.css now points every @font-face src at
  https://file.maic.chat/fonts/<name>.woff2 instead of bundled local files.
- Extract the CDN origin + font whitelist into fonts.config.mjs (single
  source of truth) and generate fonts.css from it via
  scripts/generate-fonts-css.mjs. `genfonts` runs first in the build, so the
  CSS always reflects the config; changing the domain is now a one-liner.
- Delete packages/maic-renderer/fonts/*.woff2 (22 files, ~43.6MB) and drop
  the now-unused "./fonts/*" export and "fonts" files entry from package.json
  (fonts.css is kept).

Font names are unchanged, so consumers and the importer font whitelist are
unaffected. Note: this removes the files going forward only; purging them
from git history (LFS / filter-repo) is a separate follow-up.

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(home): gate the unwired PPTX import button behind a feature flag

The PPTX import flow is still scaffolding — `useImportPptx` has no
`onImported` consumer, so clicking it only logs the parsed slides. Hide both
PPTX import entry points (and the hidden file input) behind
NEXT_PUBLIC_ENABLE_PPTX_IMPORT (default off) so the UI doesn't expose a
no-op. The classroom import button is unaffected.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(import): harden the runtime URL-loaded PPTX parser

The PPTX parser is loaded at runtime from /vendor/maic-import/index.js — a
gitignored artifact synced into public/vendor during postinstall. If a deploy
skips that step the URL 404s and import() fails with an opaque SyntaxError
(404 HTML parsed as JS). Add guards on both ends plus docs.

- Build-time assertion: scripts/assert-vendor-maic-import.mjs verifies the
  bundle exists and is non-empty; wired into `build` so a misconfigured deploy
  fails early with an actionable message instead of shipping a broken runtime.
- Runtime guard: use-import-pptx.ts HEAD-probes the URL before import() and,
  on a non-ok response, throws a clear error surfaced to the user via the new
  import.error.parserUnavailable string (added to all 6 locales).
- Docs: document the postinstall deploy dependency and the pull-without-install
  type/runtime drift in @maic/importer's README.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: unify the @maic/* license policy to AGPL-3.0

Pick one license for the SDK family and apply it uniformly, matching the
OpenMAIC root, per PR #668 review.

- @maic/dsl and @maic/importer: relicense MIT -> AGPL-3.0 (package.json +
  full AGPL-3.0 LICENSE text).
- @maic/renderer: already AGPL-3.0; add the LICENSE file its package.json
  `files` already referenced.

@maic/importer is derived from pptxtojson (MIT); MIT permits relicensing a
derivative under AGPL, and the upstream MIT attribution is retained in the
package README/index.html, so this is compliant.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: rename packages/maic-import dir to maic-importer

Align the importer's directory with its @maic/importer package name and the
maic-* sibling dirs (maic-dsl, maic-renderer). Package name is unchanged, so
imports/transpilePackages/node_modules resolution are unaffected.

- git mv packages/maic-import -> packages/maic-importer (history preserved).
- Rename the vendor sync/assert scripts and the runtime vendor path to match:
  scripts/{sync,assert-vendor}-maic-importer.mjs, public/vendor/maic-importer,
  URL /vendor/maic-importer/index.js.
- Update all path references: root postinstall/build/sync scripts, .gitignore,
  use-import-pptx.ts, and doc links in the dsl/renderer/importer READMEs.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: nest the @maic packages under packages/@maic/

Mirror the @maic/* npm scope in the directory layout: move the three family
packages into packages/@maic/{dsl,importer,renderer}. Package names are
unchanged, so imports / transpilePackages / node_modules resolution are
unaffected.

- git mv packages/maic-{dsl,importer,renderer} -> packages/@maic/* (history
  preserved).
- pnpm-workspace.yaml: add packages/@maic/* glob.
- Recompute the root postinstall cd chain for the nested layout.
- Update dir-path references: sync script srcDir, .gitignore comment, the dsl
  source comment, and the renderer/importer README/SKILL cross-links.

Unchanged: the @maic/* package names, the runtime vendor path
(public/vendor/maic-importer) and its sync/assert scripts.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore(renderer): clear font licenses, drop non-redistributable faces, remove importer font remapping

Address the PR #668 blocking font-license-clearance review.

Font licensing (renderer):
- Reduce the self-hosted whitelist to 6 clearly-redistributable faces:
  SourceHanSans/SourceHanSerif/LXGWWenKai/ZhuQueFangSong (SIL OFL 1.1),
  ZcoolHappy (ZCOOL free-use), WenDingPLKaiTi = AR PL KaitiM GB (Arphic PL 1999).
- Drop AlibabaPuHuiTi: free to *use* commercially but its license grants no
  explicit redistribution/re-host right, unlike OFL — dropped to keep public
  CDN serving unambiguous.
- Add FONTS.md (per-face attribution/clearance record, separate from the
  package's AGPL LICENSE) and bundle the verbatim license texts under
  font-licenses/ (OFL.txt, ZcoolHappy-LICENSE.txt, ARPHIC-PL.txt); ship both via
  package.json `files`. Regenerate fonts.css from the trimmed config.

Importer font remapping removed:
- The importer no longer rewrites font-family names (alias -> category ->
  primary). It passes each slide's original font-family through unchanged, so
  it can never target a face the renderer doesn't ship. Delete configs/font.ts,
  the resolveFont/replaceFontFamilyInHtml helpers, and the now-unused
  FontReplacementBucket / ImportContext.replacedFonts plumbing.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(renderer): document the CDN font runtime dependency

Add a "Fonts" section to the renderer README per PR #668: the optional
`@maic/renderer/fonts.css` loads woff2 from an external host
(https://file.maic.chat), which must be reachable + CORS-enabled or the
browser silently falls back to system fonts. Note that the import is optional
and that the origin is configurable via fonts.config.mjs + `pnpm run genfonts`.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(renderer): verify WenDingPLKaiTi == AR PL KaitiM GB from the font name table

Record the embedded-name-table evidence (Family/Copyright/Trademark/Designer =
Arphic, nameID 13 = the full Arphic Public License) so the
WenDingPLKaiTi ⇄ AR PL KaitiM GB correspondence is provable, and correct the
copyright year to 1994–1999 to match the font metadata.

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore: fix `pnpm check` (prettier) after merging main

main added a `prettier . --check` CI step that flagged 180 files. Resolve it
consistently with the existing convention (vendored/workspace packages are
excluded from root Prettier):

- .prettierignore: exclude `packages/@maic/` (own rollup/tsc/eslint toolchains;
  dist is generated, importer/src1 is a vendored oracle), matching how
  pptxgenjs / mathml2omml / docs are already handled.
- Format the three root-level files we added/edited that root Prettier does
  cover: lib/import/use-import-pptx.ts and the two scripts/*-maic-importer.mjs.

`pnpm check` now passes.

Co-authored-by: Cursor <cursoragent@cursor.com>

* style: align @maic/{dsl,renderer,importer} source to root Prettier

Per review preference, the @maic packages are our own code and should follow
the repo style rather than be ignored wholesale. Narrow .prettierignore to only
the generated build output (packages/@maic/*/dist/) and the vendored legacy
reference (packages/@maic/importer/src1/), then `prettier --write` the package
sources. Formatting-only; all three packages still build.

`pnpm check` passes.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(lint): scope ESLint to @maic source and clear all errors

- eslint.config.mjs: replace blanket packages/** ignore with precise
  rules so @maic/{dsl,importer,renderer} source is linted, while skipping
  third-party packages, dist output, importer/src1 and public/vendor
  (the generated bundle was producing thousands of false errors in CI).
- importer: replace `any` with concrete narrowings/types, fix prefer-const.
- renderer: drop manual useMemo that React Compiler could not preserve.
- remove leftover debug console.log + stale eslint-disable in use-import-pptx.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(i18n): add missing PPTX import keys to pt-BR

pt-BR.json was missing import.{pptx,parsingPptx,pptxSuccess} and
import.error.{invalidPptx,parserUnavailable}, failing the i18n key
alignment check against en-US.json.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-09 03:25:05 -04:00
Madoka a47b2d2fb1 Feat/439 i18n key alignment check (#447) 
* feat(ci): add i18n key alignment check

* fix(ci): harden i18n key alignment script
 2026-04-18 13:31:55 +08:00