mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-17 05:29:46 +08:00
@openmaic/generation@0.3.1
16
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 wyuc
|
e15780835a |
feat(media): convert legacy references to allocated asset ids (#1101)
Lands epic #1007 part 2(c): the app-side reference converter, the `audioUrl` removal, and the `DSL_VERSION` bump as one delivery unit. `convertDocumentAssetRefs` walks a loaded document (slide canvases, whiteboards, video-manifest keys, speech actions) and rewrites legacy handles to allocated pool ids: placeholders and `audioId`s with local bytes are ingested, a co-present `audioUrl`/`audioId` pair collapses to one asset, a definitively dead URL converts to an emptied reference, and a transient failure keeps both handles for a later retry. Unavailable bytes leave the document untouched for a later open, and a converted document returns by identity with no writes. Conversion runs lazily on document open and at fetch time for server-generated classrooms, with conversion and the first document save sharing the per-stage document lock: once the document commits it owns every allocation, and every failure exit rolls its pass ledger back. Legacy URL probes are bounded by a per-probe timeout and a shared 60-second pass budget. `SpeechAction.audioUrl` is removed from the DSL and every consumer reads the converted shape; the server classroom generator keeps its derived-id + serving-URL pair as an explicitly typed pre-conversion transport, consumed before persistence. `DSL_VERSION` becomes 0.2.0 with the first real ladder entry, and `@openmaic/dsl` goes to 0.9.0 per the release rule. Deferred follow-ups are tracked in #1129: production principal derivation with document/asset ownership aligned, conditional document writes, and hoisting the aggregate probe budget into the export and video-timeline paths. Refs #1007 |
||
|
|
3eea9dc542 |
feat(runtime): complete the #869 learner-data cutover — quiz + playback onto RuntimeStore (#955)
* feat(runtime): persist quiz attempts in RuntimeStore * fix(runtime): coalesce quiz draft snapshots * fix(runtime): recover concurrent quiz attempts * fix(runtime): handle quiz completion races * fix(runtime): commit quiz review atomically * fix(runtime): dedupe concurrent quiz writes * fix(runtime): drop stale quiz drafts * fix: harden quiz runtime review recovery * fix: serialize quiz attempt identity * feat: read quiz state from runtime store * fix: persist quiz retries before resetting * fix: preserve authoritative quiz outcomes * fix: preserve legacy quiz retries during cutover * fix: reconcile legacy quiz snapshots safely * fix: drain rollover quiz write queues * fix: drain completed quiz retry queues * fix: reuse concurrent quiz retries * fix(quiz): preserve drafts across abrupt reloads * fix(quiz): recover empty retry sessions * fix(chat): abort stalled runtime state reads * fix(quiz): expose queued phases to readers * fix(quiz): retain concurrent writer tails * fix(quiz): canonicalize retry branches * fix(quiz): keep retry rollovers monotonic * fix(quiz): validate skipped retry siblings * fix(quiz): close read cutover races * test(classroom): cover legacy quiz summaries * fix(quiz): reset async consumers on scene changes * fix(quiz): close scene transition windows * test(pbl): cover launch freshness guards * fix(quiz): close cutover concurrency gaps * fix(quiz): harden retry and context freshness * fix(quiz): reject malformed legacy answers * fix(quiz): validate legacy answer values * test(quiz): cover legacy multi-answer migration * fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover Main's three call sites built static storeState blocks that runAgentLoopFn never consumed (it always rebuilds fresh state via getStoreState); the quiz read cutover replaced that callback with the async two-phase RuntimeStore read, leaving the template field with zero consumers. Drop it. * feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError semantics existed only in the browser backend; server-backed deployments would silently accept conflicting appends and leave completed sessions active. Forward the options over the wire, detect conflicts atomically under the PG transaction, map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error client-side so quiz retry logic works across every backend. Co-authored-by: Codex <codex@openai.com> * fix(chat): Pi single requests build storeState via the async runtime quiz read Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the request template straight to /api/chat/pi, which rejects bodies without storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read with the scene-transition guard) and call it on the Pi path too. * ci: whitelist the runtime-data-cutover integration trunk for PR checks * feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956) * feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869) The fourth and last runtime family. Consumed-discussion facts become append-only 'playback' records folded into a set at read (at-least-once appends, no conflict machinery); the resume cursor is device-scoped last-write-wins KV per the amended #779/#869 split. sessionStorage keeps same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie playbackState machinery is retired, with a one-time lazy migration of any legacy row (cursor half + facts half) before deletion, and stage deletion now clears both the KV cursor and any unmigrated legacy row. Co-authored-by: Codex <codex@openai.com> * test(runtime): include playbackState in the stage-delete db mock --------- Co-authored-by: Codex <codex@openai.com> * fix(playback): persist discussion facts on every consumption path (#957) * fix(playback): persist discussion facts on every consumption path (final-review P0+P1s) - The engine now publishes a progress snapshot the moment a discussion is consumed (join / skip / unselected-agent auto-skip). onProgress otherwise fires before the discussion action executes and a discussion is the scene's last action, so the fact never reached persistence. - Reads fold records across ALL playback sessions in the learner partition (mergeLearner deliberately preserves same-kind sessions from both keys). - Legacy migration appends only not-yet-durable facts, so an interrupted migration resumes instead of dropping the tail. - recordConsumedDiscussion reports durability; the component drops failed ids from its observed set so a later progress tick retries (at-least-once). * test(e2e): live verification of the playback persistence chain Seeds a deterministic stage straight into the Dexie DB, starts the lecture via the canvas overlay, and asserts the full chain: discussion auto-skip appends a discussionConsumed record to maic-runtime, the device cursor lands in KV, and both survive a fresh browsing context (empty sessionStorage). * refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959) Product ruling on #869's fourth family: playback learner state is front-end ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined discussions' content already lives in chat runtime records, and no replay export / analytics consumer exists — so durable facts bought nothing over in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half with real UX value), the engine's consumption-time progress snapshot (cursor freshness), and the legacy Dexie retirement (cursor half migrates, row deletes, consumed ids are dropped). * fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966) * fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair) Review findings on #955: didActiveSceneRemainUnchanged compared the active scene by object identity, so a store update reallocating the scene during the async quiz read dropped the learner's graded answers from that turn's request — the scene id is the real boundary. The records route now classifies a malformed sessionTransition as a validation failure instead of letting the store's throw surface as a 500. * fix(playback): superseded-engine cursor guard + migration write-window recheck Second-vendor review of the #959 shrink (requested after the cross-review noted it had single-vendor coverage) found: an engine orphaned by a scene switch during async lecture resume could pass the idle-only recheck, be resurrected, and publish its old scene's progress over the new scene's debounced cursor — the resume continuation now requires identity with the installed engine, and onProgress drops snapshots from superseded engines. The legacy cursor migration also rechecks KV immediately before its write so a concurrent tab's newer cursor cannot be overwritten and orphaned by the legacy-row delete. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> * fix(review): approval follow-up P3 nits (#967) * release: v0.3.1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard - The records route's sessionTransition guard now requires an ISO updatedAt (isIsoTimestamp), matching the sibling PATCH /status route - Dead vi.mock factories for the deleted playback-storage module dropped - A corrupt legacy playback timestamp falls back to 'now' instead of wedging migration into a permanent re-throw that disabled resume --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
bb8d4f30e7 |
feat(chat): add experimental Pi classroom runtime behind a flag (#914)
* feat(chat): add experimental Pi classroom runtime * fix(chat): harden Pi whiteboard action sequencing * style(chat): format session retirement flow * style(playback): format async scene transition * fix(chat): retire actions on session errors * fix(chat): close Pi runtime lifecycle gaps * fix(chat): avoid replaying historical whiteboard actions * fix(chat): prevent Pi structured-output residue from leaking and inflating turns Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(chat): expose whiteboard element/line ids and bound code-line context Surface each element's id (and code line ids) in the Pi whiteboard context so later child agents can target wb_delete / wb_edit_code, and cap the rendered code lines with a shared character budget so a code-heavy board cannot grow the prompt without bound. Return only the current turn's whiteboard ledger from the Pi director loop instead of carrying history forward across requests. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(chat): bound Pi whiteboard code context Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(chat): prioritize newer persisted whiteboard code in budget Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(chat): enforce whiteboard boundaries across Pi sessions * feat(chat): refine soft-closing discussion controls * fix(chat): make whiteboard clearing model-directed --------- Co-authored-by: apple <apple@GigideMacBook-Air.local> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: LING-6150 <duan.lin@northeastern.edu> |
||
|
|
7fffec78f6 |
feat(choreography): shared orchestration spec in lib/choreography (#863) (#890)
* feat(choreography): shared orchestration spec in lib/choreography (#863) Introduce lib/choreography/ as the single source of truth for the orchestration semantics a faithful classroom-video exporter needs from playback, so the app runtime and the exporter interpret one spec instead of each re-implementing (and silently drifting from) the other. Kept in lib/ rather than a package: these semantics co-evolve with the playback engine, and the exporter will also live in the app, so both consumers share them via ordinary imports. Purity is machine-enforced by an eslint boundary on lib/choreography/** (blocks @/ host-app paths + react/react-dom/gsap/framer-motion/motion), so the exporter can interpret the spec in a pure Node environment. - timing.ts — timing constants + the deterministic no-audio speech estimate, moved verbatim from the engines. - cursor.ts — resolvePlaybackCursor + EMPTY_SCENE_DWELL, moved from lib/playback/engine-cursor.ts (typed on dsl SceneCore). - timeline.ts — new pure resolveActionTimeline: index-domain -> time-domain expansion (blocking cursor-advance vs fire-and-forget visual duration), keyed off the DSL fire-and-forget partition. - descriptors/— versioned, zod-schema-validated animation descriptors spotlight.v1 + laser.v1 (declarative: property/from/to/ duration/easing; no implementation), pinned to the current overlay components. Behavior-neutral engine refactor: lib/action/engine.ts and lib/playback/engine.ts import from lib/choreography and the local literals are deleted, so the timing dimension now has exactly one copy. The spotlight/laser overlay components still hardcode their animation values (they do not yet READ the descriptors) — tracked in #889. Closes #863 * fix(choreography): address cross-review findings on resolveActionTimeline + descriptors - P1: model implicit whiteboard auto-open — a wb_* mutation on a closed board now prepends a synthetic IMPLICIT_WB_OPEN (WB_OPEN_MS) beat, mirroring the engine's ensureWhiteboardOpen; open state carries across scenes and toggles on wb_open/wb_close (new `whiteboardOpen` option to seed it). - P2: scale real speech audio duration by playbackSpeed too (live path sets AudioPlayer.setPlaybackRate), keeping it in lockstep with the estimate path. - P2: express the spotlight mask relationship in the descriptor model — LayerSchema gains `role` ('content'|'mask') + `maskedBy` (subtract|intersect); spotlight.v1's cutout is now a mask layer the dim layer subtracts, so a non-React consumer reconstructs the cutout instead of a black rect. - P3: wb_clear on an empty board is 0ms (engine early-returns), not wbClearMs(0). Tests: tests/lib/choreography 45 pass (+6); engine regression 28 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): address final-audit findings on effect lifetime + spotlight descriptor Second cross-review round (fresh-session codex final audit) surfaced three deeper mismatches with live playback: - P1: fire-and-forget effect lifetime is not a flat EFFECT_AUTO_CLEAR_MS. The engine's processNext clears effects at every scene boundary and on completion, and scheduleEffectClear uses one shared timer each new effect resets. Added clampFireAndForgetLifetimes: an effect's visual durationMs is now min(next scene boundary / completion, shared-timer deadline chained through later effects in the same scene). advancesCursorMs (0) is untouched. - P2: spotlight dimness default is 0.5 (executeSpotlight: dimOpacity ?? 0.5; DSL documents 0.5), not the component's unreachable ?? 0.7 fallback. Fixed the descriptor param + test. - P2: model the spotlight wrapper's enter/exit opacity fade (motion.div, no explicit duration → engine default). TrackSchema.durationMs is now optional to express "use the consumer's engine default"; dim layer carries the fade tracks. Tests: tests/lib/choreography 50 pass (+5, incl. boundary-cut / completion-cut / full-lifetime / shared-timer-extension / wrapper-fade); engine regression 28 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): pin spotlight fade duration + laser dot geometry in descriptors Third cross-review round (codex) flagged two descriptor-completeness gaps that would make a non-Motion consumer (the exporter) diverge from the app: - Spotlight wrapper fade: the enter/exit opacity tracks left durationMs implicit (Motion default). A literal consumer treats a missing duration as instant, so the spotlight would pop on/off. Pinned to Motion's default 300ms tween. - Laser dot geometry: the descriptor captured only tracks, not the dot group's center anchor (translate -50%,-50%) or the rounded-full ring/core. A literal renderer would draw an offset 10px square. Added the static geometry (anchor, borderRadius 9999, ring inset/position) so the shape/position match the app. Also refined the effect-lifetime docstring to cite the app's per-scene engine teardown/completion (the actual clearEffects path) rather than an intra-engine boundary gate that is dead in the single-scene-per-engine configuration. The empty-scene "speech dwell → blank chat bubble" observation is pre-existing behavior: EMPTY_SCENE_DWELL is a verbatim move from lib/playback/engine-cursor.ts (unchanged from origin/main), out of scope for this move-only PR; tracked separately. Tests: tests/lib/choreography 54 pass; engine regression 28 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): zero-duration for engine-skipped/no-op actions in timeline Fourth cross-review round (codex) flagged two remaining timeline divergences: - Skipped discussions: the engine skips a discussion outright (no timer) when it's already consumed or its agent isn't selected, but the timeline always charged DISCUSSION_TRIGGER_DELAY_MS. Added `isDiscussionSkipped` resolver (runtime-state-dependent, like getVideoDurationMs) → 0ms when skipped. - No-op whiteboard draws: executeWbDrawText (empty content) and executeWbDrawTable (no rows/cols) return before any delay. The timeline now charges 0ms for these determinable-from-the-action no-ops instead of WB_DRAW_MS. (KaTeX-failure / missing-edit-target no-ops depend on runtime state and remain out of scope, consistent with the resolver pattern.) Tests: tests/lib/choreography 56 pass; engine regression 28 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): exact-5s effect chain break + spotlight dim full-screen geometry Fifth cross-review round (codex), both P2: - Effect chain break at an EXACT 5s boundary: the earlier effect's clear timer is queued before the reading timer that triggers the later effect (same 5000ms delay), so it fires first — the predecessor is cleared at exactly deadlineMs, not extended. Changed the chain guard from `> deadlineMs` to `>= deadlineMs`. - Spotlight dim layer full-screen geometry: the descriptor recorded only fill + mask relation, leaving a literal consumer no way to know the dim rect spans the 0..100 viewport. Added explicit x/y/width/height (100×100 at origin) so the descriptor is self-contained. Tests: tests/lib/choreography 57 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): no-op wb_edit_code resolver in timeline Sixth cross-review round (codex), one P2: executeWbEditCode returns before its delay when the edit can't apply (missing/non-code target, stale line refs). The timeline always charged WB_EDIT_MS. Added `isEditCodeNoop` resolver (runtime- state-dependent, same pattern as getClearElementCount / isDiscussionSkipped) → 0ms when the caller flags a no-op. Tests: tests/lib/choreography 58 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): descriptor layer inheritance for nested effect layers Seventh cross-review round (codex), two P2 with one root cause: the flat layers[] model couldn't express a child layer riding a parent's animation (the source nests some layers inside an animated wrapper). Added an `inheritsFrom: {parentId, props}` relation to LayerSchema: - Laser ring + core inheritsFrom the animated `dot` (left/top/opacity), so a literal consumer flies them in/out with the dot instead of leaving them at a static origin while only the dot moves. - Spotlight border inheritsFrom `dim` (opacity), so the outline fades out with the wrapper instead of lingering after the dimming layer disappears. Tests: tests/lib/choreography 60 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(choreography): address review — discussion auto-skip, explicit video policy, import allowlist Human review (wyuc) raised two blocking timing issues + one non-blocking guard: - Discussion dwell (blocking): a non-skipped discussion in unattended playback/export blocks for the trigger delay AND the ProactiveCard's own auto-skip countdown, not just DISCUSSION_TRIGGER_DELAY_MS. Added DISCUSSION_AUTO_SKIP_MS (5000) to the timing spec and charge DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS. ProactiveCard now reads the same constant (was a hardcoded 5000), so card countdown and timeline can't drift. A `spotlight -> discussion -> speech` timeline now extends the spotlight across the full discussion interval. - play_video (blocking): an unresolved duration no longer silently becomes a 0ms segment (which shifted every later action early). New `onUnresolvedVideoDuration` policy defaults to 'throw' (fail loudly); 'cap' assumes MAX_VIDEO_WAIT_MS, 'zero' opts back into no-dwell explicitly. - Purity guard (non-blocking): turned the lib/choreography boundary into a true import allowlist. Beyond the existing @/… + render-package blocks, it now rejects parent-escape (../…) imports/re-exports, any bare package other than @openmaic/dsl / zod, and dynamic import()/require(). Negative-tested: ../store, a stray bare package, export * from ../playback, and import('react') all fail. Tests: tests/lib/choreography 60 pass; engine regression 28 pass; tsc + lint clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
danishsshaikh
|
d00795094c |
feat(playback): add action-level navigation (#843)
Squashed from #843. Adds Notes click-to-jump action navigation and action-level playback resume/restore, with visible toolbar sentence controls removed after review. |
||
|
wyuc
|
5daf3e5a6e |
fix(editor): keep emptied/zero-action scenes playable, bind outline by stable id, surface incomplete content (#814)
Editor empty-state robustness: dwell on zero-action scenes (engine + PlaybackChromeRoot) so emptied/blank slides stay playable; bind a scene's outline by stable outlineId (reorder/insert/duplicate-safe, allOutlines in scene order); clear the cue preview on glyph unmount; surface incomplete content calmly (amber dashed frame on clips, amber dot on blank outline titles + rail, neutral N/M-ready generate gate). Cross-reviewed (Codex + Claude), 5 rounds to a clean pass. |
||
|
wyuc
|
1d1ce80e04 |
feat(maic-agent): editor-agent line (v0) — Pro-mode "Edit with AI" (#777)
Promote the editor-agent line to main: read-then-act editor agent (read_scene_content, regenerate_scene, regenerate_scene_actions, edit_interactive_html), a routable maic-agent model stage, the agent sidebar UI, interactive-scene runtime-error capture, and the timeline/script editor. Cross-reviewed and smoke-verified; see #777. |
||
|
|
3e1e9f083b |
feat(tts): unify provider-enablement model, default browser-native off (#665) (#666)
A single TTS provider-enablement model honored everywhere (picker,
auto-assignment, fallback, lecture/playback, server).
Model
- lib/audio/provider-enablement.ts: isTTSProviderConfigured /
isTTSProviderEnabled / listEnabledTTSProviderIds. Availability drops the
registry defaultBaseUrl fallback (fixes Lemonade's always-shown voices),
honors the per-provider `enabled` flag, server-disable overrides (server
precedence).
- voice-resolver: getEnabledProvidersWithVoices (enabled server/custom
providers) and getSelectableProvidersWithVoices (the ONE client source of
truth = enabled providers + opt-in browser-native when the browser exposes
voices). resolveAgentVoice picks deterministically among the given providers,
honors a voiceConfig only while its provider is present.
- Picker + discussion build their list from getSelectableProvidersWithVoices, so
the AgentBar and the live discussion agree on selectable voices (fixes
student agents going silent in auto mode). Students resolve via
resolveAgentVoice; the teacher uses the global lecture voice verbatim (same
voice the pre-generated lecture sends), so lecture and discussion never
diverge.
- Voice assignment vs fallback: an agent's voice is bound at generation (the
agent-profile LLM picks from availableVoices; preset uses the user's pick).
The fallback for a missing/invalid/unavailable voice is applied only at the
live TTS calls, not baked into the saved agent — so it reflects current
provider availability.
Defaults / opt-in
- Browser-native: first-class, default OFF, opt-in. Built-in providers default
enabled:true (only surface once configured).
- TTS master toggle defaults OFF; first server-sync auto-enables it when a
provider is configured (mirrors image/video).
- Persist v3->v4 migration normalizes the never-user-editable flags.
Server/admin disable (server precedence): server-providers.yml `enabled:false`
and env `TTS_<P>_ENABLED=false` (incl. browser-native). Surfaced as a per-provider
`disabled` flag on /api/server-providers; merged as serverDisabled; enforced at
the /api/generate/tts trust boundary, classroom media generation, and /api/health.
UI
- Settings TTS panel: per-provider enable toggle, disabled (with a hint) when the
provider is unavailable, locked off (with a note) when server-disabled.
- AgentBar voice pills render disabled with a "No voice" placeholder when no
provider is enabled. The toolbar media-popover TTS tab keeps its existing
on/off toggle (no picker/CTA).
Tests: predicates, resolver determinism + unified selectable list, server
force-disable, store merge + migration + TTS auto-enable, i18n parity. Verified
in a real browser (Playwright): settings toggle states + a full Gemini classroom
generation whose TTS requests used the picked voice.
Found on main @
|
||
|
Shangqing TuandClaude Opus 4.7
|
c02a607377 |
feat: interactive mode clean (#461)
* feat: add interactive mode * feat: add TeacherActions for interactive mode * fix: better prompt for ultra mode * feat: upgrade normal mode's interactive generation with 5 widget type and actions * feat: make interactive mode button clear * fix: remove unused TeacherOverlay * fix: resolve PR #458 review feedback - Revert unrelated infra changes (Dockerfile, docker-compose.yml) - Fix placeholder strings in generateWidgetContent - Rename ultraMode to interactiveMode consistently across codebase - Extract duplicated patchHtmlForIframe to shared utility - Fix widget store race condition by keying callbacks by sceneId - Fix teacher-action ID collisions by using descriptive IDs in prompt - Relax outline prompt constraints for flexibility - Optimize sidebar thumbnail with IntersectionObserver lazy loading - Add trailing newlines to modified files Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: update generateSceneActions call signatures The function signature changed to individual parameters instead of options object pattern after cherry-pick. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * style: fix prettier formatting Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: removed extra language arg * style: fix prettier formatting v2 * fix: add missing i18n keys --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
kaiyuan ZhangandClaude
|
cce7087f67 |
Fix action filtering logic and add safety improvements (#163)
* fix: fix 3 bugs found during code review 1. formatQuestionsForPrompt called .join() on QuizOption objects, producing "[object Object]" in AI prompts instead of readable option text (e.g. "A. Option1, B. Option2"). 2. json-repair Fix 1 regex double-escaped valid JSON escapes (\t, \n, \r, \b, \f) and couldn't handle strings with escaped quotes. Now uses a proper string-aware regex and preserves valid escapes. 3. PlaybackEngine spotlight/laser actions used synchronous recursion via processNext(), risking stack overflow with many consecutive non-speech actions. Now uses queueMicrotask() to break recursion. https://claude.ai/code/session_01JdigqQtiAvomeifvWSQbjm * fix: ensure allowedActions whitelist applies to non-slide scenes The early return in Step 6 (slide-only action filter) caused Step 7 (allowedActions whitelist) to be skipped for non-slide scenes. This meant hallucinated actions from agents in quiz/interactive/pbl scenes were not filtered by the role-based allowedActions whitelist. Refactored to use a mutable result variable instead of early returns so both filters always apply. https://claude.ai/code/session_01JdigqQtiAvomeifvWSQbjm * fix: prevent video playback hang and division by zero in CJK detection 1. executePlayVideo could hang forever if the video element was invalid or the state change was missed between playVideo() and subscribe(). Added a 5-minute safety timeout to prevent indefinite blocking. 2. CJK language detection in browser TTS divided by chunkText.length without checking for empty strings, producing NaN. Now guards against zero-length text. https://claude.ai/code/session_01JdigqQtiAvomeifvWSQbjm * fix: prevent ordered entry with invalid index when partial text has no remaining content When a previously-partial text item completed but all content had already been streamed as deltas (remaining is empty), the code still pushed an ordered entry pointing to textChunks.length - 1, which could be -1 or reference an unrelated chunk. This caused spurious warnings in director-graph and potential text loss. Now only pushes the ordered entry when there is actual remaining content to emit. https://claude.ai/code/session_01JdigqQtiAvomeifvWSQbjm * fix: validate quiz points parameter and fix parse-pdf metadata spread order 1. quiz-grade: `points` from request body was used without validation, causing NaN in score calculation and invalid JSON responses when points was undefined, negative, or non-numeric. Now validates it is a positive finite number. 2. parse-pdf: metadata spread order put the default `pageCount` before `...result.metadata`, so the spread would overwrite the fallback. Fixed by spreading metadata first, then applying defaults after. Also changed `||` to `??` so pageCount=0 is preserved correctly. https://claude.ai/code/session_01JdigqQtiAvomeifvWSQbjm * fix: crash in moveUpElement, dead-code in video options, wrong parens in canCombine 1. use-order-element.ts: moveUpElement crashed with TypeError when a grouped element was already at the top of the z-order. The code accessed copyOfElementList[maxLevel + 1] without bounds checking, then dereferenced .groupId on undefined. Added early return guard matching the pattern used in the non-grouped branch. 2. video-providers.ts: normalizeVideoOptions had a dead-code ternary inside an if-block that already proved the condition was false. Simplified to directly use the fallback value. 3. use-canvas-operations.ts: canCombine check had incorrect parenthesization: (el.groupId && el.groupId) === firstGroupId evaluated the && first, which is equivalent to just el.groupId. Fixed to: el.groupId && el.groupId === firstGroupId. https://claude.ai/code/session_01JdigqQtiAvomeifvWSQbjm * style: fix prettier formatting in 3 files https://claude.ai/code/session_01JdigqQtiAvomeifvWSQbjm * fix: use valid ApiErrorCode in quiz-grade route INVALID_PARAMETER is not a member of API_ERROR_CODES. Use INVALID_REQUEST instead, which is the appropriate code for invalid request parameters. https://claude.ai/code/session_01JdigqQtiAvomeifvWSQbjm --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Yizuki_Ame
|
2841ce6ae4 |
fix: clean up Roundtable UI state after QA/Discussion request failure (#128)
* fix: clean up Roundtable UI state after QA/Discussion request failure (#127) When /api/chat fails mid-stream during QA or Discussion, the catch blocks only appended an error message to the chat panel. The Roundtable live UI (bubble, thinking animation, engine mode) was never cleaned up, leaving the interface stuck until page refresh. Changes: - Add clearLiveSessionAfterError() in use-chat-sessions.ts that notifies Stage via onLiveSessionError callback on request failure - Add handleLiveSessionError() in stage.tsx to reset all live UI state and transition PlaybackEngine back to idle - Add handleDiscussionError() in engine.ts to exit live mode without triggering the normal discussion-end sequence - Add engine.resume() in doResumeTopic() to sync engine mode after soft-pause resume Closes #127 * style: fix prettier formatting |
||
|
wyucandClaude Opus 4.6
|
d0318e1326 |
fix(tts): fix browser-native TTS playback and add provider switching to toolbar (#153)
* fix(tts): fix browser-native TTS playback and add provider switching to toolbar (#145) - Add speechSynthesis.cancel() before speak() in PlaybackEngine to fix garbled/broken audio output (Chrome known bug workaround) - Replace flat voice selector with GroupedSelect in toolbar TTS tab, allowing direct provider switching (Qwen TTS ↔ Browser Native etc.) - Dynamically populate browser-native voices from speechSynthesis.getVoices() instead of a single hardcoded "default" entry - Split browser voices by language for easier navigation - Use i18n for TTS provider names in toolbar Closes #145 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve selectedGroup matching for duplicate groupId sub-groups When browser-native-tts voices are split by language, multiple groups share the same groupId. The GroupedSelect trigger now finds the sub-group that contains the selected item, so it displays the correct language label (e.g. "Browser Native TTS · English" instead of always showing the first sub-group). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
|
fbe8399678 |
feat(generate): server-side media & TTS generation (#75)
* feat(generate): server-side media & TTS generation in classroom pipeline When `enableImageGeneration`, `enableVideoGeneration`, or `enableTTS` flags are passed to /api/generate-classroom, the server now actually generates media files and TTS audio, persists them to disk, and replaces placeholders in the classroom JSON with serving URLs. - Add media serving API route (GET /api/classroom-media/[id]/[...path]) - Add server-side media generation utility (image, video, TTS) - Wire generation phases into classroom pipeline with progress steps - Update AudioPlayer to support URL-based playback (server-generated TTS) - Add audioUrl field to SpeechAction type - Extend /api/health with capabilities detection - Pass through feature flags in generate-classroom route - Update skill docs for optional feature configuration Closes #49 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix prettier formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(generate): add agentMode support for server-side classroom generation Allow callers to choose between built-in default agents ('default') or LLM-generated custom agent profiles ('generate') via a new agentMode field on GenerateClassroomInput. The resolved agents are injected into outline, scene content, and scene action prompts via teacherContext. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(media): harden download, parallelize media gen, fix TTS concatenation - Add timeout (2min) and max size (100MB) guard to downloadToBuffer - Run image and video generation in parallel via Promise.all - Only split TTS text for MP3 (frame-based); skip for WAV/OGG/AAC whose container headers break on naive byte concatenation - Remove unnecessary type assertion for speechAction.audioUrl Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix prettier formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(media): harden media serving and TTS generation - Add realpath check to prevent symlink-based path traversal in media route - Stream files via createReadStream instead of readFile to avoid OOM on large videos - Truncate TTS text for non-concatenable formats (WAV/OGG/AAC) when exceeding provider limit - Add 'researching' step to distinguish web search from outline generation progress Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): split long speech actions instead of concatenating audio bytes Adopt the same strategy as client-side: split a long speech action into multiple shorter actions before TTS generation, so each sub-action gets its own audio file. This avoids the broken byte-concatenation approach that corrupted non-MP3 formats (WAV/OGG/AAC container headers). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(tts): extract shared TTS splitting logic into lib/audio/tts-utils Both client-side (use-scene-generator) and server-side (classroom-media-generation) had duplicated TTS_MAX_TEXT_LENGTH, text splitting, and action splitting logic. Extract into a shared module to keep them in sync. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove unused TTS_MAX_TEXT_LENGTH import Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
db5ac82167 |
fix: use browser speechSynthesis for playback when browser-native-tts is selected (#28)
* fix: use browser speechSynthesis for playback when browser-native-tts is selected Previously, selecting browser-native-tts as the TTS provider would produce sound in the settings test but remain silent during classroom playback. This happened because: 1. The scene generator correctly skipped pre-generation for browser TTS (it runs client-side, not via API) 2. The playback engine fell back to a silent reading timer when no pre-generated audio was found, instead of calling speechSynthesis This commit adds Web Speech API integration directly in the PlaybackEngine: - New playBrowserTTS() method speaks text via speechSynthesis - Properly wires onend/onerror to advance to the next action - pause()/resume() now handle speechSynthesis.pause()/resume() - stop() and handleUserInterrupt() cancel browser TTS Fixes #25, fixes #12, fixes #5 * fix: handle "default" ttsVoice, extract CJK_LANG_THRESHOLD constant - When ttsVoice is "default" (set by Browser Native TTS which has no voice picker), the voiceURI lookup silently fails and no lang is set, causing Chinese text to be spoken with an English voice. - Extract the 0.3 CJK detection threshold as a named constant CJK_LANG_THRESHOLD with JSDoc explaining the rationale. - Fall through to language auto-detection when voice lookup fails, regardless of the reason (missing voice, "default" sentinel, etc.). * fix: support browser-native tts previews * refactor: extract shared TTS preview logic into useTTSPreview hook The browser-native and API-based TTS preview code was duplicated across tts-config-popover, media-popover, and tts-settings. Extract it into a reusable useTTSPreview hook that handles refs, cancellation, audio lifecycle, and staleness checks in one place. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: yangshen <1322568757@qq.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.6
|
843d33e5cc |
chore: enforce Prettier formatting and fix lint issues
- Add .prettierignore to exclude vendor packages, lock files, markdown, and YAML - Update .prettierrc: printWidth 100, singleQuote, trailingComma "all" - Run Prettier across all source files for consistent formatting - Fix unused imports (UserRequirements, setTTSProvider) - Fix eslint-disable comment placement after Prettier reformat Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.6
|
0d20abfed9 |
Initial commit: OpenMAIC AI classroom platform
Next.js 16 App Router application with: - AI-powered interactive classroom generation from PDF/text requirements - Multi-agent discussion system (teacher, students, assistant roles) - Provider abstraction for LLM, TTS, ASR, image, video, web search - Local-first data architecture (IndexedDB/Dexie) - i18n support (zh-CN/en-US) - Structured logger, SSRF guard, lint/build passing (0 errors) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |