3 people 7fffec78f6 feat(choreography): shared orchestration spec in lib/choreography (#863) (#890) ...

* feat(choreography): shared orchestration spec in lib/choreography (#863)

Introduce lib/choreography/ as the single source of truth for the
orchestration semantics a faithful classroom-video exporter needs from
playback, so the app runtime and the exporter interpret one spec instead
of each re-implementing (and silently drifting from) the other.

Kept in lib/ rather than a package: these semantics co-evolve with the
playback engine, and the exporter will also live in the app, so both
consumers share them via ordinary imports. Purity is machine-enforced by
an eslint boundary on lib/choreography/** (blocks @/ host-app paths +
react/react-dom/gsap/framer-motion/motion), so the exporter can interpret
the spec in a pure Node environment.

- timing.ts   — timing constants + the deterministic no-audio speech
                estimate, moved verbatim from the engines.
- cursor.ts   — resolvePlaybackCursor + EMPTY_SCENE_DWELL, moved from
                lib/playback/engine-cursor.ts (typed on dsl SceneCore).
- timeline.ts — new pure resolveActionTimeline: index-domain -> time-domain
                expansion (blocking cursor-advance vs fire-and-forget visual
                duration), keyed off the DSL fire-and-forget partition.
- descriptors/— versioned, zod-schema-validated animation descriptors
                spotlight.v1 + laser.v1 (declarative: property/from/to/
                duration/easing; no implementation), pinned to the current
                overlay components.

Behavior-neutral engine refactor: lib/action/engine.ts and
lib/playback/engine.ts import from lib/choreography and the local literals
are deleted, so the timing dimension now has exactly one copy.

The spotlight/laser overlay components still hardcode their animation
values (they do not yet READ the descriptors) — tracked in #889.

Closes #863

* fix(choreography): address cross-review findings on resolveActionTimeline + descriptors

- P1: model implicit whiteboard auto-open — a wb_* mutation on a closed board
  now prepends a synthetic IMPLICIT_WB_OPEN (WB_OPEN_MS) beat, mirroring the
  engine's ensureWhiteboardOpen; open state carries across scenes and toggles
  on wb_open/wb_close (new `whiteboardOpen` option to seed it).
- P2: scale real speech audio duration by playbackSpeed too (live path sets
  AudioPlayer.setPlaybackRate), keeping it in lockstep with the estimate path.
- P2: express the spotlight mask relationship in the descriptor model —
  LayerSchema gains `role` ('content'|'mask') + `maskedBy` (subtract|intersect);
  spotlight.v1's cutout is now a mask layer the dim layer subtracts, so a
  non-React consumer reconstructs the cutout instead of a black rect.
- P3: wb_clear on an empty board is 0ms (engine early-returns), not wbClearMs(0).

Tests: tests/lib/choreography 45 pass (+6); engine regression 28 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): address final-audit findings on effect lifetime + spotlight descriptor

Second cross-review round (fresh-session codex final audit) surfaced three
deeper mismatches with live playback:

- P1: fire-and-forget effect lifetime is not a flat EFFECT_AUTO_CLEAR_MS. The
  engine's processNext clears effects at every scene boundary and on completion,
  and scheduleEffectClear uses one shared timer each new effect resets. Added
  clampFireAndForgetLifetimes: an effect's visual durationMs is now min(next
  scene boundary / completion, shared-timer deadline chained through later
  effects in the same scene). advancesCursorMs (0) is untouched.
- P2: spotlight dimness default is 0.5 (executeSpotlight: dimOpacity ?? 0.5; DSL
  documents 0.5), not the component's unreachable ?? 0.7 fallback. Fixed the
  descriptor param + test.
- P2: model the spotlight wrapper's enter/exit opacity fade (motion.div, no
  explicit duration → engine default). TrackSchema.durationMs is now optional to
  express "use the consumer's engine default"; dim layer carries the fade tracks.

Tests: tests/lib/choreography 50 pass (+5, incl. boundary-cut / completion-cut /
full-lifetime / shared-timer-extension / wrapper-fade); engine regression 28 pass;
tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): pin spotlight fade duration + laser dot geometry in descriptors

Third cross-review round (codex) flagged two descriptor-completeness gaps that
would make a non-Motion consumer (the exporter) diverge from the app:

- Spotlight wrapper fade: the enter/exit opacity tracks left durationMs implicit
  (Motion default). A literal consumer treats a missing duration as instant, so
  the spotlight would pop on/off. Pinned to Motion's default 300ms tween.
- Laser dot geometry: the descriptor captured only tracks, not the dot group's
  center anchor (translate -50%,-50%) or the rounded-full ring/core. A literal
  renderer would draw an offset 10px square. Added the static geometry (anchor,
  borderRadius 9999, ring inset/position) so the shape/position match the app.

Also refined the effect-lifetime docstring to cite the app's per-scene engine
teardown/completion (the actual clearEffects path) rather than an intra-engine
boundary gate that is dead in the single-scene-per-engine configuration.

The empty-scene "speech dwell → blank chat bubble" observation is pre-existing
behavior: EMPTY_SCENE_DWELL is a verbatim move from lib/playback/engine-cursor.ts
(unchanged from origin/main), out of scope for this move-only PR; tracked separately.

Tests: tests/lib/choreography 54 pass; engine regression 28 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): zero-duration for engine-skipped/no-op actions in timeline

Fourth cross-review round (codex) flagged two remaining timeline divergences:

- Skipped discussions: the engine skips a discussion outright (no timer) when
  it's already consumed or its agent isn't selected, but the timeline always
  charged DISCUSSION_TRIGGER_DELAY_MS. Added `isDiscussionSkipped` resolver
  (runtime-state-dependent, like getVideoDurationMs) → 0ms when skipped.
- No-op whiteboard draws: executeWbDrawText (empty content) and
  executeWbDrawTable (no rows/cols) return before any delay. The timeline now
  charges 0ms for these determinable-from-the-action no-ops instead of
  WB_DRAW_MS. (KaTeX-failure / missing-edit-target no-ops depend on runtime
  state and remain out of scope, consistent with the resolver pattern.)

Tests: tests/lib/choreography 56 pass; engine regression 28 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): exact-5s effect chain break + spotlight dim full-screen geometry

Fifth cross-review round (codex), both P2:

- Effect chain break at an EXACT 5s boundary: the earlier effect's clear timer
  is queued before the reading timer that triggers the later effect (same 5000ms
  delay), so it fires first — the predecessor is cleared at exactly deadlineMs,
  not extended. Changed the chain guard from `> deadlineMs` to `>= deadlineMs`.
- Spotlight dim layer full-screen geometry: the descriptor recorded only fill +
  mask relation, leaving a literal consumer no way to know the dim rect spans the
  0..100 viewport. Added explicit x/y/width/height (100×100 at origin) so the
  descriptor is self-contained.

Tests: tests/lib/choreography 57 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): no-op wb_edit_code resolver in timeline

Sixth cross-review round (codex), one P2: executeWbEditCode returns before its
delay when the edit can't apply (missing/non-code target, stale line refs). The
timeline always charged WB_EDIT_MS. Added `isEditCodeNoop` resolver (runtime-
state-dependent, same pattern as getClearElementCount / isDiscussionSkipped) →
0ms when the caller flags a no-op.

Tests: tests/lib/choreography 58 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): descriptor layer inheritance for nested effect layers

Seventh cross-review round (codex), two P2 with one root cause: the flat
layers[] model couldn't express a child layer riding a parent's animation
(the source nests some layers inside an animated wrapper). Added an
`inheritsFrom: {parentId, props}` relation to LayerSchema:

- Laser ring + core inheritsFrom the animated `dot` (left/top/opacity), so a
  literal consumer flies them in/out with the dot instead of leaving them at a
  static origin while only the dot moves.
- Spotlight border inheritsFrom `dim` (opacity), so the outline fades out with
  the wrapper instead of lingering after the dimming layer disappears.

Tests: tests/lib/choreography 60 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(choreography): address review — discussion auto-skip, explicit video policy, import allowlist

Human review (wyuc) raised two blocking timing issues + one non-blocking guard:

- Discussion dwell (blocking): a non-skipped discussion in unattended
  playback/export blocks for the trigger delay AND the ProactiveCard's own
  auto-skip countdown, not just DISCUSSION_TRIGGER_DELAY_MS. Added
  DISCUSSION_AUTO_SKIP_MS (5000) to the timing spec and charge
  DISCUSSION_TRIGGER_DELAY_MS + DISCUSSION_AUTO_SKIP_MS. ProactiveCard now reads
  the same constant (was a hardcoded 5000), so card countdown and timeline can't
  drift. A `spotlight -> discussion -> speech` timeline now extends the spotlight
  across the full discussion interval.
- play_video (blocking): an unresolved duration no longer silently becomes a
  0ms segment (which shifted every later action early). New
  `onUnresolvedVideoDuration` policy defaults to 'throw' (fail loudly); 'cap'
  assumes MAX_VIDEO_WAIT_MS, 'zero' opts back into no-dwell explicitly.
- Purity guard (non-blocking): turned the lib/choreography boundary into a true
  import allowlist. Beyond the existing @/… + render-package blocks, it now
  rejects parent-escape (../…) imports/re-exports, any bare package other than
  @openmaic/dsl / zod, and dynamic import()/require(). Negative-tested: ../store,
  a stray bare package, export * from ../playback, and import('react') all fail.

Tests: tests/lib/choreography 60 pass; engine regression 28 pass; tsc + lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>

2026-07-11 13:57:06 +08:00

..

.gitignore

feat(choreography): shared orchestration spec in lib/choreography (#863) (#890)

2026-07-11 13:57:06 +08:00