Files
OpenMAIC/components/edit/ActionsBar/ActionsBar.tsx
T
wyucandClaude Fable 5 04621578de release: OpenMAIC 1.0.0 — the agent workbench (#1228)
* feat(storage): add an agent-session store with PG backend and layered contracts (#1163)

* feat(storage): add agent-session store with PG backend and layered contracts

* test(storage): avoid BigInt literals for pre-ES2020 root typecheck

* fix(storage): close agent-session store review findings

* docs(storage): align hook ordering and contention-probe claims with the code

* ci: run on the agent-workbench integration branch

* chore(storage): bump to 0.5.0 for the agent-session store

* fix(storage): carry replay compaction across page boundaries

* feat(agent): add the driver model contract and stage route dialect (#1165)

* feat(agent): add the driver model contract and stage route dialect

* fix(agent): validate route context windows and clarify dialect precedence

* feat(agent): adapt the agent-session store and runtime foundations (#1167)

* feat(agent): adapt the agent-session store and runtime foundations

* feat(agent): resolve request owner identity via an anonymous cookie

* docs(agent): document the opt-in compaction default and harden edge cases

* feat(agent): add the background session runner (#1169)

* feat(agent): add the background session runner

* feat(agent): wire the runner into startup behind feature flags

* fix(agent): stop clean interruptions from consuming the attempt budget

* fix(storage): charge the attempt budget for abandoned leases but not clean parks

* docs(storage): document the attempt-charging contract and decouple its tests

* feat(agent): add agent session and owner event streams (#1170)

* feat(agent): add agent session and owner event streams

* fix(agent): close the session-existence oracle and document the owner seam

* feat(agent): add agent session lifecycle routes (#1171)

* feat(agent): add agent session lifecycle routes

* fix(agent): validate session-create input and preserve the owner cookie on errors

* refactor(storage): drop the unused active-stage API from the agent-session contract (#1174)

* refactor(storage): drop the unused active-stage API from the agent-session contract

Tools address stages explicitly on every call, so the store keeps no
mutable session-level stage pointer. Removes resolveActiveStage and
setActiveStage from the store interface, their PG implementations, the
active_stage_changed lifecycle event, the session_active_stage owner
event variant, and the contract tests pinning them. The active_stage_id
column and the DDL check constraint stay untouched for schema
compatibility.

* chore(storage): bump @openmaic/storage to 0.7.0 for the contract removal

* docs: document the agent runtime configuration surface (#1176)

* fix(agent): repair orphaned and late tool results across interruption boundaries (#1180)

* fix(agent): repair orphaned and late tool results across interruption boundaries

A crash, shutdown, or provider failure can leave the durable transcript with
tool calls that have no result, or with results ordered illegally for the
provider. Three failure modes were fixed:

- Orphaned tool calls: a run that died between an assistant tool-call frame
  and its result left a dangling call in the entry tree. Resume no longer
  synthesizes and persists receipts for it: interrupted results are a
  read-time provider view owned by a shared read-boundary repair, which
  returns the original array for a healthy transcript and never mutates the
  tree.

- Late parallel results: a parallel tool can finish while pi unwinds an
  aborted assistant frame, leaving result(A), assistant(aborted), result(B)
  in durable order. Strict providers reject non-contiguous results, so the
  read-boundary repair moves existing results next to their owning assistant
  frame (in call order), omits incomplete unwind frames, and synthesizes
  receipts only for genuinely missing calls.

- Interrupted calls at the write boundary: a call still in flight when the
  run winds down (shutdown, lease loss, cancellation, provider failure) had
  no receipt at all. The runner now tracks in-flight calls from their
  assistant frames and, before the terminal flush, appends an
  interrupted-result receipt for each still-orphaned call through the same
  attempt-fenced write chain, so a lease-stealing zombie never writes and
  the next claim sees a provider-safe transcript.

* test(agent): pin the runner wiring for interruption-boundary tool repair

* feat(agent): add neutral tool foundation libraries (#1184)

* feat(agent): register a web_search tool on the session runner (#1185)

* feat(storage): add a per-session URL trust gate (#1186)

* feat(agent): add the skills system (#1189)

* feat(agent): add the skills system (builtin directories and durable user skills)

* fix(storage): serialize the user-skill quota check-and-insert per owner

Two concurrent creates at the 50-skill boundary both counted 49 rows and
both inserted (READ COMMITTED, no lock), overshooting the quota contract.
The create transaction now takes a per-owner pg_advisory_xact_lock first,
and the same-name idempotency check runs before the count check so an
at-least-once retry of the create that committed as the owner's 50th row
still returns its durable receipt instead of a quota error. The 23505
backstop is retained for writes that do not take the lock.

* fix(agent): share unstorable-character validation and align skill lookup

* feat(agent): add session materials and a fetch_url tool behind the URL trust gate (#1190)

* feat(agent): add session materials and a fetch_url tool behind the URL trust gate

* fix(agent): harden session material fetching

* feat(storage): add an ownership scope to stage documents (#1191)

* feat(agent): add material read and search tools (#1192)

* feat(agent): add stage read and patch tools (#1194)

* feat(agent): add page generation and deck editing tools (#1198)

* test(storage): keep the PG contract suite order-independent (#1200)

* fix(agent): revoke deleted-session URL authority and reject private ISATAP endpoints (#1199)

* fix(storage): revoke deleted session URL authority

* fix(ssrf): reject private ISATAP endpoints in strict fetches

* chore(storage): bump to 0.11.1 for the session-URL authority fix

* feat(agent): add roster and voice registration tools (#1201)

* feat(agent): add folder organisation tools (#1202)

* feat(api): add stage and material HTTP routes (#1203)

* feat(workbench): add the client data layer (#1204)

* feat(workbench): add the client data layer

* docs(workbench): write the ported comments in English

* chore(edit): remove the in-editor agent panel (#1210)

* chore(edit): remove the in-editor agent panel

* style: apply prettier formatting

* fix(agent): report the runtime as unusable without a database (#1207)

* fix(agent): report the runtime as unusable without a database

* style: apply prettier formatting

* feat(agent): add image, video and pptx import tools (#1211)

* feat(workbench): add the agent chat surface (#1205)

* feat(workbench): add the agent chat surface

* docs(workbench): write the ported comments in English

* fix(workbench): label the folder and rename tools on the timeline

* fix(workbench): label the roster and voice tools on the timeline

The reconciliation test iterates every tool the runner registers and
requires a display label of its own. The roster and voice-clone tools
(list_voices, set_roster, clip_audio, register_voice) reached the
integration base with the roster/voice-registration tools but never
gained presentation rows, so they fell through to the default branch
and rendered their wire names. Port their rows from the reference
implementation (labels and i18n keys verbatim) and extend the
reconciliation allowlist with ROSTER_TOOL_NAMES and
VOICE_CLONE_TOOL_NAMES, so a future tool cannot enter the product
without a label.

* feat(agent): add the material extraction lifecycle (#1212)

* feat(storage): add material extraction lifecycle

* feat(agent): execute queued material extraction

* style: apply prettier formatting

* style: satisfy prefer-const in the extraction runner

* test: give material fixtures the extraction lifecycle fields

The media-tools slice and the extraction lifecycle slice were each green
in isolation but never compiled together: the lifecycle made derivedFrom
and extraction required on AgentSessionMaterial while the media-tool
fixtures predate them.

* chore: remove stray task notes

* fix(workbench): label the extraction lifecycle tools on the timeline

* feat(workbench): add the workspace shell (#1206)

* feat(workbench): add the workspace shell

* docs(workbench): write the ported comments in English

* i18n(workbench): align workspace keys across locales

* fix(workbench): adopt the landed data layer and label the extraction tools

- replace the sibling-slice seam stubs with the real data-layer modules
- drop ambient declarations now shadowed by landed files
- port timeline labels for the extraction lifecycle tools from the reference
- align the new i18n keys across all locales

* ci: retrigger

* feat(api): folder routes, stage-meta viewer surfaces, and the material upload contract (#1215)

* fix(storage): restore capability-based stage access

* fix(api): bind document access to request owner

* fix(agent): restore three-state stage access on the tool layer

Port probeStageAccess and the three-state StageAccess (owned / foreign /
missing / tombstoned) and gate every stageId-bearing stage tool on an owned
probe, mirroring the reference per tool:

- move_to_folder, rename_stage, read_stage_outline refuse a non-owned stage
  with the single not-yours message before touching the store.
- The course/DSL toolset and the roster toolset are wrapped by
  withOwnerStageAuthorization: read_stage, patch_stage, grep_stage and every
  writer refuse a foreign stage with the same message and refusal shape.
- Scene preview keeps its own probe and its own refusal text, and is
  registered beside the course toolset (never double-gated).
- The runner injects one probe factory at the three call sites.

Tests: the dsl cross-owner test premise (a foreign stage is readable by id)
encoded an invented capability-read policy that the reference does not have
at the tool layer; it now asserts foreign read/patch/grep are all refused
while the owner still reads. Curriculum cross-owner assertions were already
the reference's and now pass with the probes in place.

* docs: correct per-file test counts in the fidelity report

* test: fix type errors in stage-access fidelity test

* test: adapt media-tool and gate suites to the owner-scoped store seam

* feat(api): add owner-scoped course-folder HTTP routes

Port the reference implementation's /api/folders family (list, create,
rename, delete with ungroup/remove modes, and folder membership) onto the
owner-bound document store, replacing its provider-based auth with the
existing withRequestOwnerId / owner-scoped store seams.

The storage package's folder store grows the pieces the routes need:
DocumentFolder.order (schema column + max+1 assignment + ordering),
renameFolder, deleteFolder(mode) with captured member ids, and
setStageFolder(stageId, folderId | null) with idempotent un-filing.
FolderNameError moves into folder-name-validation.ts (stage-storage re-exports
it, keeping import sites intact).

Every route gates on the configured agent runtime (plain 404 when off or
unconfigured), keeps the reference's machine codes and envelopes, and is
covered by gate tests plus a behavior suite.

* feat(api): add stage-meta viewer surfaces for the classroom

Port the reference implementation's viewer-facing stage state — can-edit /
collected / published / generation-complete — on top of the stage-access
base (stage_meta + tombstones). stage_meta gains published_at and
generation_complete columns plus a stage_bookmarks table; the reference's
deployment-specific origin/claimed_at columns are stripped.

New gated routes: GET /api/stage-meta/[stageId] (per-viewer facts, 404 for
absent/tombstoned, never returns the owner id), GET /api/stages/[id]/status,
POST generation-complete / publish / unpublish (owner-only), POST
/api/bookmarks. The resolver lives in lib/server/stage-access.ts.

Wiring: a fetchStageMeta client with the reference's three-outcome contract,
stage-store isOwner/isBookmarked/readOnly fields (upstream single-user
defaults, no-op until the sidecar answers) plus setViewerAccess, the
classroom apply path computing readOnly = !(isOwner || isBookmarked), the
Stage editability gate, and a sidecar probe after each classroom load. A
sidecar 'absent' answer keeps the editable default here because the
classroom also serves local-only courses; server writes stay owner-enforced.

* feat(api): port the reference material upload contract

Rewrite POST /api/materials to the reference implementation's upload shape
so the workbench uploader (uploadWorkbenchMaterial, which posts no session
id and expects a flat 201 view) works unchanged: owner-scoped upload with
mime normalization/validation (415), per-class size caps checked on the
declared content-length and the streamed body (413), empty body (400),
quota (429), sha256 reserve->store->finalize lifecycle with abandon on
failure, flat { materialId, originalName, bytes, mime, extraction } 201,
and an x-request-id echo.

Adds the owner-scoped material library (owner_material table + quota +
24h lazy sweep, bytes in the host's asset registry as the neutral
replacement for the reference's object-storage byte path) and the material
cap configuration. The session-scoped GET list is left as-is; the
reference's owner-material extraction worker is not ported (the branch's
session-material extraction lifecycle already covers extraction).

Gate tests now cover all 23 persistence routes across the three runtime
env states; the materials behavior suite pins the new contract.

* feat(media): add an optional local ffmpeg media extractor (#1213)

Adds a local ffmpeg/ffprobe pipeline as a second media extraction
provider behind the extractor registry, ported faithfully from the
reference implementation: duration probing, keyframe-safe chunking,
per-chunk ASR with timeout and deadline budgets, and timestamped
transcript assembly.

- Availability probing feeds the registry's candidate selection: the
  provider simply is not a candidate when ffmpeg/ffprobe are absent.
- With neither ffmpeg nor a cloud provider configured, extraction fails
  with an actionable message naming both enablement paths.
- Media materials route through the same extraction lifecycle and lease
  fence as documents; no parallel queue.
- Tests inject the executable resolver so the missing-ffmpeg path is the
  default-tested one; the real pipeline test is skip-if-unavailable.
- @openmaic/storage 0.13.0 -> 0.14.0 (media routing in the material
  lifecycle surface).

* feat(storage): per-scene monotonic revisions via database triggers (#1214)

* feat(storage): per-scene monotonic revisions via database triggers

Restore the reference implementation's freshness granularity: a
per-scene monotonic revision maintained by database triggers, so every
writer (HTTP routes, agent tools, jobs, manual SQL) bumps it without
application cooperation.

- Companion revision tables + trigger functions in the storage package's
  idempotent schema bootstrap, with the lock-order invariant, pg_notify
  wakeup and the suppression switch for batch writers.
- ensureDocumentSchema gained a dollar-quote-aware statement splitter.
- The freshness and manifest routes serve per-scene revisions.
- Mutation-verified: dropping the triggers turns the revision tests red.
- @openmaic/storage 0.13.0 -> 0.14.0.

* fix: forward the freshness manifest through the owner-bound store

* feat(workbench): add the Pro entry points and preserve the mode-transition semantics (#1208)

* feat(workbench): add the Pro entry points

* feat(workbench): preserve Pro mode transition semantics

* fix(workbench): drop ambient declarations shadowed by landed slices

* fix(workbench): drop ambient declarations shadowed by the landed shell

* feat: port workspace shell sibling modules

Port the 16 leaf modules the Pro workspace shell imports but that were only
ambient-declared, replacing the compile-time bridge with real implementations
adapted from the sibling-slice reference: pure workbench helpers (session
title, rail tab, course-chat bootstrap, created-course tabs, course-tabs
memory, workspace navigation, pane navigation, pro-edit sizing,
existing-course minting, first-message session), the neutral brand context and
course-rename server API, the server-action session delete, the home
discovery hook, the classroom pane host with its load-policy leaf, the theme
toggle and floating-layer owner, plus the floating-layer-owner wiring the
dialog/dropdown/tooltip portals stamp.

Also add the workbench-shell locale copy for all 12 locales, port the
reference tests for the ported modules, and drop
types/workbench-sibling-slices.d.ts now that every declaration has a real
implementation.

* docs: keep ported comments in English and deployment-neutral

* docs: announce 1.0.0 and refresh the feature overview (#1216)

* docs: announce 1.0.0 and refresh the feature overview

* docs: finalize 1.0.0 README after feature merge

* fix(agent): control-plane routes answer 404, not 500, without a database

The agent control-plane routes gated only on the runtime flag, so an
enabled-but-unconfigured deployment (flag on, DATABASE_URL empty)
answered 500 from a store that cannot connect. Gate them on the
configured check instead, matching the stage/material routes: the
whole surface is cleanly absent until both the flag and the database
are present. The status probe keeps reporting both bits.

* test: mock both runtime gate exports in the control-plane route suites

* fix(agent): abort in-flight TTS on cancel and bound each provider request with a timeout (#1217)

The generate_tts / scene-tts path checked the runner's AbortSignal between
actions but never created the provider HTTP requests with it, so a session
cancel left a hung synthesis fetch in flight until a restart repaired the
tool result. Thread the signal end-to-end: TTSModelConfig carries an
optional signal, generateTTS combines it with a per-request timeout
(TTS_REQUEST_TIMEOUT_MS, default 30s, ported from the reference runtime's
TTS bounds) via AbortSignal.any, and every provider fetch (openai, azure,
glm, qwen incl. voice-clone + audio download, voxcpm, minimax, doubao,
elevenlabs, lemonade) is created with that signal.

A timeout now fails the tool call with TTSRequestTimeoutError (a clear
retryable error) instead of wedging the session; a caller cancel propagates
as the interruption so the runner settles the session as cancelled without
a restart.

Tests: hung-provider simulation rejects at the timeout with the retryable
error; abort mid-flight aborts the captured request signal and surfaces the
interrupted shape; removing the signal wiring makes the abort tests fail
(red), restoring them turns green.

* fix(workbench): PG-mode home listing via owner stages; keep the interrupted terminal course card (#1218)

Finding 1: with server persistence on, listStages resolved to the generic
GET /api/persistence/documents listing, which the capability model
deliberately answers 403 FORBIDDEN_DOCUMENTS for (reads by id, listings
owner-only). The home/workspace library now lists through the owner-scoped
GET /api/stages surface (same anonymous-owner cookie the workbench uses)
when server persistence is enabled; the server-side 403 is untouched.

Finding 2: a run interrupted (session_interrupted) and repaired
(session_resumed) that ends cancelled before agent_end stranded its pending
classroom sightings, so the timeline's terminal card lost the course the
answer produced. session_end (cancelled) now flushes the pending sightings
into the same course card set agent_end paints, before the stopped caption.

* chore(workbench): remove the bookmark concept and the saved-courses drawer (#1219)

* chore(classroom): remove the bookmark ('collected') concept entirely

The stage-meta viewer port introduced a bookmark surface (stage_bookmarks
table, POST /api/bookmarks, the isBookmarked sidecar field, and a
readOnly rule that let a saved course stay editable). The product has no
such concept, so remove it as a closure:

- delete the /api/bookmarks route and the stage_bookmarks table plus its
  query helpers from the persistence bootstrap
- drop isBookmarked from GET /api/stage-meta/[stageId]
- simplify the classroom read-only rule to readOnly = !isOwner across the
  sidecar client, ownership signal, classroom load, stage store and the
  classroom page
- keep publish/unpublish, generation-complete, isOwner and isPublic
  exactly as they were
- update the gate and stage-meta route suites and the README mentions

The workspace rail's Bookmark glyphs and comments describe the upstream
saved-courses (favorites) section, which is driven by isOwner and renders
no collect affordance; they are kept as unrelated homonyms.

* chore(workbench): remove the saved-courses drawer UI

The first pass removed the bookmark data model but kept the rail's
"Saved courses" drawer, judging it a separate surface driven by
`isOwner === false`. The home/workspace listing is owner-scoped, so that
flag can never occur: `allSaved` is permanently empty and the drawer
(plus the collapsed-rail Bookmark mini-button) is a dead affordance.
Remove it: the SavedDrawer component and its mount, the savedOpen /
savedSection state, the allSaved / matchedSaved derivations, the 'saved'
variant of the course-list renderers, the mini Bookmark glyph, the
drawer-only CSS, and the drawer's i18n keys from all 12 locales. The
courses tab is now exactly one folders tree.

The authored/favorites split in workspace-tree.ts goes with it; the tree
module no longer reads `isOwner`. The discovery course type keeps the
field — the shell still reads it for read-only gating.

Upstream has no collect concept; the drawer could only ever render empty
here. The reference implementation HAS this drawer (its favorites come
from its account system), so this removal is a deliberate upstream
product decision, not a fidelity bug.

* fix(workbench): restore the attach entry, add the rail settings entry, pin all three entry points (#1221)

* fix(workbench): restore the composer attach entry by gating it on the live runtime

The AttachButton's rollout probe read a `materialsEnabled` field that this
branch's /api/agent/runtime never answers (the materials routes gate on the
runtime itself, like the stages), so the gate could never pass and the attach
button never rendered — the Pro launch and chat composers showed only the
@-mention and enhance glyphs.

Substitute the field with the runtime's `enabled` value, which IS the upload
action's precondition: POST /api/materials answers 404 whenever it is false,
so the render condition now equals the action precondition (no dead button).

The button's label (`proMode.attach`) is a user-visible string that becomes
visible again; port the reference implementation's own translations verbatim
into the 11 locales that still carried the Chinese copy.

* feat(workbench): add the settings entry to the rail's bottom-left cluster

The reference's rail foot carries a cluster of utilities (its saved-courses
drawer, the language switcher, the display toggle). This branch removed the
drawer — it could only ever render empty here — and the product decision is
to fill that freed spot with the settings entry.

Add a settings trigger to the foot cluster (expanded rail, beside the
language and display toggles, and on the collapsed strip) and mount the
model/provider SettingsDialog in the rail, wired to the trigger. It is the
same dialog the classic home opens from its header pill; the workspace had no
settings entry of its own, so nothing is duplicated within a surface.

* test(workbench): pin the restored upload, attach, and settings entry points

Covers the three restored entry points:

- the courses-tab upload control: rendered beside the course name filter,
  wired to the discovery hook's ZIP import trigger, disabled while an import
  runs, and gated by the same condition as its action (the courses tab);
- the composer attach control: an actual render of AttachButton under both
  probe answers (visible when the runtime says the upload path is live,
  hidden otherwise), its mounts in the launch and chat composers, the
  branch's runtime-field substitution in the probe, and the reference's own
  `proMode.attach` copy in all 12 locales;
- the settings entry: the trigger in the rail's foot cluster (expanded and
  collapsed), beside the language and display toggles, opening the
  SettingsDialog the rail mounts.

* chore(config): the Pro workbench flag implies the MAIC Editor gate (#1223)

A workbench build without the editor toggle has no way to edit a course:
enabling NEXT_PUBLIC_PRO_WORKBENCH_ENABLED while forgetting
NEXT_PUBLIC_MAIC_EDITOR_ENABLED produced exactly that split-brain bundle.
The workbench IS Pro mode, so its flag now implies the editor gate; the
standalone flag remains for deployments that want the classroom editor
without the workbench. Documents both flags in .env.example.

* fix(agent): wake SSE tails and the runner on durable deltas (streaming fidelity) (#1222)

The Pro workbench chat did not stream: the session/owner SSE routes polled
the durable event log on a 5s/30s clock with no wakeup, so message_update
deltas (written at 150ms cadence) reached the browser in poll-sized blocks
and the thinking strip only mounted after the whole reasoning text had
accumulated.

Port the reference's LISTEN/NOTIFY delta path:

- storage: add in-transaction wake hooks (onSessionEventAppended,
  onOwnerEventAppended, onCancelRequested) so a host queues pg_notify in
  the same transaction as the durable append; align readEventsAfterForReplay
  to rank the bounded page so the first delta after the cursor is always
  kept (the live tail can never starve). Bump @openmaic/storage to 0.18.0.
- app: port the process-wide event-notify bus (dedicated LISTEN client,
  self-check probe, reconnect backoff; notify through the storage
  transaction surface), wire the store hooks, subscribe both SSE routes
  before the initial read with the reference's initializing gate, and give
  the runner one {kind:'session'} subscription whose wake runs the cancel
  check and the message drain. Polls stay as the lossy-NOTIFY backstop.
- lifecycle: start/stop the bus from instrumentation.

Tests: storage hook + compaction contract; route wakeup latency; runner
wakeup wiring with a fake agent; bus unit tests; PG contracts proving a
real append wakes the routes and a live SSE route forwards a message_update
on the wakeup, and that a rolled-back append never wakes. Also fix the
pre-existing park-attempt-budget PG test TRUNCATE (missing CASCADE against
newer FK tables).

* fix(storage): asset writes self-deadlocked against pooled PostgreSQL (#1225)

* fix(storage): refuse the non-transactional byte-write deadlock configuration

A byte store whose plain write() runs on its own pooled connection cannot be
invoked from inside a registry write transaction: after the transaction has
claimed the blob-row lock, that write blocks on the lock the transaction just
took while the transaction waits on the write - a self-deadlock PostgreSQL
cannot detect (one side is idle in transaction).

There is no lock-safe ordering for such a writer: bytes must be written after
the row claim (writing before it lets the collector delete the bytes while the
upsert waits), and any second-connection write after the claim is the
deadlock. The configuration is therefore detected and refused:

- AssetByteStore gains writesOutsideRegistryDatabase?: true, declaring that the
  layer's plain byte operations cannot contend for the registry's row locks.
- PgAssetStore refuses put()/replace() up front (and defends coordinatedWrite)
  when the byte store has no writeWith and does not declare the flag, throwing
  a clear configuration error before any row is claimed.
- The collector mirrors the guard on its delete path (deleteWith or a declared
  out-of-registry layer, else a configuration error).
- The object store declares the flag (its out-of-transaction write remains
  legitimate); the in-registry PostgreSQL byte column provides writeWith /
  deleteWith instead.
- Write transactions (put/replace/remove) set SET LOCAL lock_timeout = 30s so
  any future lock-contention variant fails loudly instead of hanging.

Bumps @openmaic/storage to 0.18.0.

* fix(persistence): forward the transactional byte methods through the lazy asset byte-store wrapper

The no-bucket case of lazyAssetByteStore returned a bare { write, read,
delete } and dropped writeWith/readWith even though the underlying
PgAssetByteStore has them. The registry's hasTransactionalWriter duck check
then failed and put() fell back to the byte store's own pooled connection,
which blocks forever on the blob-row lock the registry transaction just took
when the bytes live in the same PostgreSQL - the production self-deadlock.

The no-bucket layer is statically PgAssetByteStore, so its transaction-pinned
methods are forwarded eagerly (typed against the real signatures via
PgForwardedByteStore). The bucket case keeps its lazy-probing semantics: no
transactional writer exists there, the signed-URL method stays absent or lazy
exactly as documented, and the wrapper now declares writesOutsideRegistryDatabase
so the registry may run the plain write inside its transaction.

New tests pin the wrapper's transactional capability red-to-green and assert
put()/resolve() route byte traffic through the transaction-pinned queryable.

* fix(home): cap the generate-prep ingest drain at 3s so Generate never waits the full server budget

The classic home flow's Generate click drained in-flight ingests for the full
15s server budget. Cap the wait at GENERATE_DRAIN_CAP_MS (3000ms, documented as
a UX bound) and reuse the existing timeout fallback: sources that miss the cap
proceed on the legacy byte path and each late-resolving id is released.

* chore(storage): bump to 0.19.0 over the concurrently landed 0.18.0

* fix(agent): bound every tool call with a timeout; never resurrect a cancelled session (#1226)

* fix(agent): bound every tool call with a global timeout and settle it on cancel

A tool await that neither resolves nor rejects wedges the session forever:
the lease keeps heartbeating and the driver never reaches its next cancel
checkpoint. Race every tool execution (in buildAgent) against a hard budget
(OPENMAIC_AGENT_TOOL_TIMEOUT_MS, default 10 min, per-tool overrides for
known long runners) and against the caller's AbortSignal, so even a
signal-ignoring await cannot keep a cancelled session running.

On timeout the call rejects with AgentToolTimeoutError; the agent loop turns
the rejection into a structured error tool-result the agent can retry or
proceed from, and the abort signal is delivered to the tool's in-flight work
through a derived controller. Zombie-tool updates after settlement are
dropped.

* fix(storage): never re-lease a cancel-requested session; settle it as cancelled on claim

The claim scan treated a session with cancel_requested_at set as a normal
claim candidate: after a restart it re-leased the same session for attempt
N+1 and resumed generating despite the pending cancel. claimNextSession now
settles such candidates as cancelled under the claim lock (status cancelled,
attempt reset, lease and cancel request cleared, terminal session_end event
and owner projection) instead of leasing them, then keeps scanning.

Bump @openmaic/storage to 0.18.0.

* docs: takeaway-style 1.0.0 announcement with bilingual guide links

The 1.0.0 head is now a short takeaway block — badge links to the
official user guides (English and Chinese), five one-line highlights,
and pointers into Features and the workbench setup section — instead of
six dense paragraphs. The detailed provider-neutrality and freshness
notes move into the Features workbench section, phrased database-
neutrally (the announcement no longer names a specific database).
Release date corrected to August 27.

* fix(workbench): restore editor chrome, mode transition, streaming, materials, mentions, folders (#1229)

* fix(workbench): wire workspace folder routes

* fix(editor): restore reference workbench chrome

* fix(workbench): persist composer materials and course refs

* fix(workbench): preserve live reasoning frames

* fix(persistence): back off failed streaming saves

* chore(workbench): retire stale slice seams

* test(editor): cover element pin layer

* chore(storage): bump to 0.21.0 for the user-message ref/material fields

* chore(editor): translate ported code comments to English

* fix(agent): fence durable tool writes and consume cancel requests atomically (#1230)

* fix(agent): enforce provider force-off in agent tools and scrub vendor identity from tool results (#1231)

* fix(materials): serialize per-owner quota reservations and make crashed uploads reclaimable (#1232)

* fix(editor): resolve dock-bar i18n keys, remove dock height drag, wire element referencing (#1233)

* fix(workbench): send the opening session message exactly once with refs intact (#1234)

* feat(editor): port timeline TTS preview single-flight and voice-all state latching (#1235)

* fix(media): restore the reference classic media chain (#1236)

* fix(import): adapt imported PPTX canvas size so decks render without overflow (#1237)

* fix(editor): complete element referencing — renderer DOM contract and GenUI picking aligned with the reference (#1238)

* test(providers): reconcile the provider-config vendor-token debt count after the main merge

The integration line's AK/SK fallback for the managed document provider adds
occurrences that main's allowlist snapshot predates. Same mixed-composition
debt category the group already documents; no new vendor behavior.

* test(providers): reconcile vendor-token debt counts with the integration line

The main-merge brought main's neutrality-guard snapshot next to integration
features it predates (media-extractor fallback chain, local voice-profile
deletion semantics, the enabled-TTS helper). Same debt categories the guard
already documents; counts updated to the guard's own tally and two grouped
entries added. No new vendor behavior.

* fix(agent): carry reasoning through the completions dialect so the thinking strip renders (#1239)

* feat(skills): add Feynman and spiral curriculum methods (#1240)

* feat(agent): port missing reference tools and skills (parity audit) (#1241)

* feat(media): retire asset-registry wiring; media and materials follow the reference byte model (#1242)

* fix(classroom): center adapted canvases in the stage and send back navigation home during generation (#1243)

* feat(settings): skill management with real list, download, delete, and upload (#1244)

* feat(settings): skill management section with real list, detail, and zip download

* feat(skills): owner skill delete and upload across storage, API, and settings

* fixup! feat(settings): skill management section with real list, detail, and zip download

chore: neutralize a reference note in the settings header comment

* fix(media): persist origin-independent classroom-media references from the agent runtime (#1245)

* feat(editor): float the insert toolbar in the outer frame with collapse (#1246)

The insert strip was bounded to the slide card, so it could only ever sit on
top of slide content: the card's overflow clipped it and it could not be parked
in the padding beside the slide. Move it into the studio frame the element
picker's panel already roams (CanvasOverlayPortal + the frame selector), so both
canvas overlays share one bounding container and their handles behave the same.
While picking, the strip rises over the picker and goes inert, which is the
z-order CANVAS_OVERLAY_Z already documents.

Add a fold beside the grip: the chevron collapses the strip to that grip row and
back, with the buttons unmounted rather than hidden. The fold is session-local
state owned by EditShell, next to the drag offset, so a surface swap keeps it;
nothing is persisted. Expanding a strip parked at the bottom edge re-clamps
through the same bounds rule the keyboard move uses.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* fix(workbench): align the chat timeline's left edge with the composer (#1247)

* fix(agent): fence session claims while an ask_user question is outstanding (#1248)

* fix(agent): settle-time rescue tracks real delivery instead of a count offset (#1249)

* fix(persistence): migrate owner_material to oss_key and drop legacy asset_id (#1250)

* docs(readme): surface the 1.0.0 user guide badges at the top (#1253)

* fix(workbench): show newly created folders in the sidebar without reload (#1254)

* docs(readme): add the release version prefix and drop the opt-in framing

* fix(workbench): single-source the chat gutter so timeline and composer share a left edge (#1255)

The transcript and the composer each established their own column: their own
`px-*` gutter and their own `mx-auto w-full max-w-*` centering wrapper. Equal
padding values were never enough, because the two columns are centered inside
different containing blocks — the transcript's is a scroll container, whose
content box is narrower than the composer footer's by the scrollbar's width:

  transcript text left = pad + (pane - 2*pad - scrollbar - measure) / 2
  composer box  left   = pad + (pane - 2*pad             - measure) / 2

The padding cancels out of the difference and what remains is `-scrollbar/2` at
every padding value, so the transcript sat half a scrollbar to the left of the
composer and tuning the two paddings against each other could not move it.

The column is now established once, by the nearest common ancestor of both
(`chatColumn`), and the scroll viewport and the composer footer are siblings
inside it that add no horizontal inset of their own. The cap carries the gutter
on top of the 760px reading measure, so the text column keeps its width. The
handed-over question row drops the padding that indented it past the agent's
prose; framed rows keep their own inner padding, which is what a card's border
sitting on the column edge means.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* fix(workbench): lock pane-embedded classroom to edit mode (#1256)

The workspace right pane painted the full learning chrome — speed control,
play button, learner avatars, mic bar — for a course the agent had just
created, then flipped to edit once the first scene landed.

resolveStageChromeMode treated playback as the DEFAULT branch for a hosted
classroom, so every shortfall fell into it: a course whose tab opens at
stage_link time has no scenes yet, so currentSceneId is null and
isHostedSceneEditable is false. A folded pane parked the playback root
behind the fold and cross-faded it out over the pane on unfold, and a
failed editor chunk dropped into playback permanently.

Lock it at the pane instead of defaulting per entry path:

- WorkbenchPanelProvider — the single element that mounts a classroom into
  the workspace — publishes editPinned (visible && !playback). Every entry
  path passes through it, so none of them decides.
- The hosted resolution can no longer degrade to playback. Start Learning
  (workbenchLearning, new input, split out from pane visibility) is the one
  door; everything else resolves between the neutral loading shell and edit.
- Stage's chrome dispatch is exhaustive on chromeMode, so the playback root
  is no longer the else-branch of a condition about the current scene.

No flicker: chromeMode is resolved during render, and preloadEditor now
answers synchronously (isEditorPreloaded) so a remount with the chunk
already registered paints edit on the first frame. A failed import is no
longer cached forever, so the lock cannot strand the pane.

Standalone classrooms keep their stored mode unchanged.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 19:26:25 +08:00

1481 lines
58 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
'use client';
/**
* ActionsBar — the "narration script" timeline: a horizontal film-editing strip
* that is also a light editor for the scene's playback `actions`.
*
* This WAS the whole bottom bar. It is now the body of `EditDock`, which owns
* the surface (border, blur, fold) and the global edit bar above it; the
* timeline still renders its own header row — the row's controls and the body
* share one piece of state — and drives the dock's fold through `useEditDock`.
* Nothing about the timeline's own behaviour or geometry changed in the move.
* (The height-drag handle was removed per product decision: only the fold moves
* the dock's height, so the timeline no longer sizes itself.)
*
* The scene's `actions` ARE the timeline: walked left→right, each `speech`
* becomes an editable clip block (one spoken line, numbered) and every non-speech
* cue (spotlight / laser / board) becomes a compact card pinned at its place in
* the flow. Hovering a cue replays the REAL playback effect on its bound element
* (setLaser → LaserPointerOverlay, setSpotlight → SpotlightOverlay).
*
* Editing (persisted via useStageStore.updateScene → actions-edit ops):
* - speech clip text is editable inline (commit on blur);
* - the header "Add action" pill opens ActionPicker to insert a new action;
* - existing items drag to reorder; each card carries a delete button;
* - clicking an element-bound cue arms canvas pick mode (useCanvasStore.pickTarget
* with purpose 'cue'), so the target is chosen by clicking the element directly
* on the slide.
*
* Reactive to the stage store; collapse and height belong to the dock.
*/
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { createPortal } from 'react-dom';
import {
ChevronDown,
ChevronLeft,
ChevronRight,
ChevronsLeft,
ChevronsRight,
Flag,
FoldVertical,
GripVertical,
Loader2,
Play,
Plus,
RefreshCw,
Square,
Trash2,
UnfoldVertical,
Volume2,
} from 'lucide-react';
import { motion, useReducedMotion } from 'motion/react';
import { cn } from '@/lib/utils/cn';
import { useI18n } from '@/lib/hooks/use-i18n';
import { flushStageSave, useStageStore } from '@/lib/store/stage';
import { useCanvasStore } from '@/lib/store/canvas';
import { useSettingsStore } from '@/lib/store/settings';
import { useAgentRegistry } from '@/lib/orchestration/registry/store';
import { AvatarDisplay } from '@/components/ui/avatar-display';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import type { Action, DiscussionAction } from '@/lib/types/action';
import type { SceneType } from '@/lib/types/stage';
import { ELEMENT_BOUND, cueLabel, cueMeta, elementLabel } from './cue-meta';
import { applyCuePreview, clearCuePreview, cuePreviewFor } from './cue-preview';
import {
appendDiscussion,
clampInsertSlot,
hasDiscussion,
insertAt,
makeAction,
moveById,
moveByIdDir,
removeById,
setAudioIdById,
setDiscussionAgentById,
setDiscussionPromptById,
setDiscussionTopicById,
setSpeechTextClearAudioById,
} from './actions-edit';
import { useEditDock } from '@/components/edit/EditDock/dock-context';
import { ActionPicker } from './ActionPicker';
import type { PickerType } from './picker-options';
import {
audioExists,
audioObjectUrl,
discardSpeechAudio,
regenerateSpeechAudio,
resolveLegacySpeechAudioId,
resolveSpeechAudioId,
} from '@/lib/audio/regenerate-speech-tts';
const EMPTY: Action[] = [];
const EMPTY_ELEMENTS: { id?: string; type: string; content?: string }[] = [];
// Stable empty set for the "no lines regenerating" state (avoids re-allocating
// on every reset and keeps a constant identity between batch runs).
const NO_IDS: ReadonlySet<string> = new Set();
/**
* Module-level single-flight controller for TTS preview: at most one
* SpeechTtsBar may be loading or playing at any moment, across every speech
* clip in the timeline. A bar registers its own `stop` handle when it starts a
* preview and clears it in `stopPreview` only when it is still the registered
* owner — so a stale stop (a superseded attempt, or an unmounted non-active
* bar) never kills the currently active preview.
*/
let activePreview: { stop: () => void } | null = null;
/**
* Clear the canvas spotlight/laser preview when a cue glyph unmounts while it is
* being hovered — most importantly when the user deletes the cue. React does not
* fire `onMouseLeave` on unmount, so without this the previewed effect would stay
* stuck on the slide after its cue is gone.
*/
function useClearCuePreviewOnUnmount() {
useEffect(() => () => clearCuePreview(), []);
}
/**
* Soft amber dashed border marking a still-incomplete clip card — an empty
* narration line, a cue bound to no element, a discussion with no topic. A clip
* is a card, so a dashed frame reads as "draft / unfinished" better than a dot;
* the calmer amber stays clear of the blue interactive controls and is dropped
* the moment the clip is filled.
*/
const INCOMPLETE_CLIP = 'border-dashed border-amber-400/70';
const AXIS_FROM_TOP = 20; // px from track top to the axis center (nodes hang below it)
// Radix Select forbids an empty-string item value, so the discussion's
// "unspecified agent" choice rides a sentinel that maps back to '' on change.
const DISCUSSION_AGENT_NONE = '__none__';
type DragPayload = { kind: 'move'; id: string };
interface TooltipState {
action: Action;
anchor: DOMRect;
}
type TFn = (key: string, options?: Record<string, unknown>) => string;
function propsOf(a: Action, t: TFn): Array<[string, string]> {
const rows: Array<[string, string]> = [[t('edit.timeline.fieldAction'), cueLabel(a.type, t)]];
const el = (a as { elementId?: string }).elementId;
if (el) rows.push([t('edit.timeline.fieldElement'), el]);
const content = (a as { content?: string }).content;
if (content)
rows.push([
t('edit.timeline.fieldContent'),
content.length > 48 ? `${content.slice(0, 48)}…` : content,
]);
return rows;
}
function CueTooltip({ tip }: { tip: TooltipState }) {
const { t } = useI18n();
if (typeof document === 'undefined') return null;
return createPortal(
<div
style={{
position: 'fixed',
left: Math.max(8, tip.anchor.left + tip.anchor.width / 2),
top: tip.anchor.top - 8,
transform: 'translate(-50%, -100%)',
maxWidth: 280,
zIndex: 60,
}}
className="pointer-events-none rounded-lg border border-border/80 bg-popover px-2.5 py-1.5 text-popover-foreground shadow-lg shadow-black/5"
>
{propsOf(tip.action, t).map(([k, v]) => (
<div key={k} className="flex gap-2 text-[11px] leading-relaxed">
<span className="shrink-0 text-muted-foreground">{k}</span>
<span className="font-mono [overflow-wrap:anywhere]">{v}</span>
</div>
))}
</div>,
document.body,
);
}
// Native HTML5 drag snapshots the element's square bounding box, so a round
// icon chip drags with white corners ("white border"). Suppress the ghost with a 1×1
// transparent image — the violet drop indicator carries the feedback instead.
let blankDragImg: HTMLImageElement | null = null;
function setBlankDragImage(e: React.DragEvent) {
if (typeof document === 'undefined') return;
if (!blankDragImg) {
blankDragImg = new Image();
blankDragImg.src =
'data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7';
}
try {
e.dataTransfer.setDragImage(blankDragImg, 0, 0);
} catch {
/* not supported — fall back to the default ghost */
}
}
/** Shared delete button — prominent, top-right of a card. */
function DeleteButton({ onDelete }: { onDelete: () => void }) {
const { t } = useI18n();
return (
<button
type="button"
onClick={(e) => {
e.stopPropagation();
onDelete();
}}
className="grid size-5 place-items-center rounded-md text-muted-foreground/55 transition-colors hover:bg-rose-50 hover:text-rose-500 dark:hover:bg-rose-500/15"
aria-label={t('edit.delete')}
>
<Trash2 className="size-3.5" />
</button>
);
}
/** ‹ › buttons to nudge a node left/right along the timeline. */
function MoveButtons({
onLeft,
onRight,
canLeft,
canRight,
}: {
onLeft: () => void;
onRight: () => void;
canLeft: boolean;
canRight: boolean;
}) {
const { t } = useI18n();
const cls =
'grid size-5 place-items-center rounded text-muted-foreground/55 transition-colors hover:bg-muted hover:text-foreground disabled:opacity-25 disabled:hover:bg-transparent';
return (
<>
<button
type="button"
disabled={!canLeft}
onClick={(e) => {
e.stopPropagation();
onLeft();
}}
className={cls}
aria-label={t('edit.timeline.moveLeft')}
title={t('edit.timeline.moveLeft')}
>
<ChevronLeft className="size-3.5" />
</button>
<button
type="button"
disabled={!canRight}
onClick={(e) => {
e.stopPropagation();
onRight();
}}
className={cls}
aria-label={t('edit.timeline.moveRight')}
title={t('edit.timeline.moveRight')}
>
<ChevronRight className="size-3.5" />
</button>
</>
);
}
type TtsStatus = 'none' | 'ready' | 'generating' | 'error';
/** TTS preview lifecycle: idle → loading (awaiting the blob URL) → playing → idle. */
type PreviewPhase = 'idle' | 'loading' | 'playing';
/** Audio status + preview / regenerate row, shown when managed TTS is on. */
export function SpeechTtsBar({
actionId,
audioId,
audioUrl,
audioInvalidated,
sceneOrder,
language,
text,
refreshKey,
regenerating,
onGenerated,
}: {
actionId: string;
audioId?: string;
/** The legacy URL of an unconverted pair: narration exists until conversion removes it. */
audioUrl?: string;
audioInvalidated?: boolean;
sceneOrder: number;
language?: string;
text: string;
refreshKey?: number;
regenerating?: boolean;
/**
* Notification that regeneration succeeded. Carries the freshly allocated
* audioId so the caller can stamp it on the action: this tree allocates pool
* identities (the blob is stored under the returned id), so the id cannot be
* re-derived by the caller like the reference's deterministic key.
*/
onGenerated: (audioId: string) => void;
}) {
const { t } = useI18n();
const [status, setStatus] = useState<TtsStatus>('none');
// Holds this line in the generating state across a batch ("Voice all") run
// and — crucially — until its OWN audio re-check resolves, so it can't
// briefly flash back to not voiced in the window between the batch clearing
// `regenerating` and the async audioExists effect landing. Latched on the
// rising edge of `regenerating`, cleared inside that re-check effect (which
// the batch always re-triggers via `refreshKey`).
const [batchPending, setBatchPending] = useState(false);
const [prevRegenerating, setPrevRegenerating] = useState(regenerating);
if (regenerating !== prevRegenerating) {
// Adjust state during render (per React's "you might not need an effect"),
// not in an effect — avoids a cascading render on the batch's hot path.
setPrevRegenerating(regenerating);
if (regenerating) setBatchPending(true);
}
// TTS preview state machine — local UI state for the preview button, kept
// apart from `status`/`effStatus` (audio availability + regeneration), which
// the playback lifecycle must not disturb.
const [previewPhase, setPreviewPhase] = useState<PreviewPhase>('idle');
// Generation token: every `stopPreview` (and every fresh `preview`) bumps it,
// so an in-flight `audioObjectUrl` await can tell it was superseded and drop
// its result — this is what kills the double-click double-Audio race.
const previewTokenRef = useRef(0);
const audioRef = useRef<HTMLAudioElement | null>(null);
const objUrlRef = useRef<string | null>(null);
const lookupId = resolveSpeechAudioId(sceneOrder, { id: actionId, audioId });
const [readAudioId, setReadAudioId] = useState<string | undefined>(lookupId);
const [seededLookupId, setSeededLookupId] = useState(lookupId);
if (lookupId !== seededLookupId) {
setSeededLookupId(lookupId);
setReadAudioId(lookupId);
}
const stopPreview = useCallback(() => {
// Invalidate every in-flight `preview()` await — a newer click, a takeover
// by another bar, or an unmount all funnel through here.
previewTokenRef.current += 1;
// Only the registered owner clears the module-level handle: a stale stop
// from a superseded or unmounted bar must not kill the current preview.
if (activePreview?.stop === stopPreview) activePreview = null;
audioRef.current?.pause();
audioRef.current = null;
if (objUrlRef.current) {
URL.revokeObjectURL(objUrlRef.current);
objUrlRef.current = null;
}
setPreviewPhase('idle');
}, []);
useEffect(() => {
let alive = true;
(async () => {
try {
// A missing stamped id means "not generated" for new documents. Probe
// the deterministic key only to preserve pre-allocation Dexie rows.
const legacyId = lookupId
? undefined
: await resolveLegacySpeechAudioId(sceneOrder, { id: actionId, audioInvalidated });
const candidateId = lookupId ?? legacyId;
// An unconverted pair's legacy URL is narration that exists: the id
// lookup may find nothing while the URL is still live.
const has = (candidateId ? await audioExists(candidateId) : false) || !!audioUrl;
if (alive) {
setReadAudioId(has ? candidateId : undefined);
setStatus((s) => (s === 'generating' ? s : has ? 'ready' : 'none'));
}
} catch {
/* IndexedDB read failed — leave status as-is (as before this change) */
} finally {
// Clear the batch latch only once the batch itself is over — its
// end-of-batch re-check runs with regenerating=false. A *stale*
// pre-batch check that resolves mid-batch must NOT clear it (adding
// regenerating to the deps also cancels such a check at batch start via
// the cleanup below). Runs even if the read threw, so the row can never
// wedge in the generating state.
if (alive && !regenerating) setBatchPending(false);
}
})();
return () => {
alive = false;
};
}, [lookupId, actionId, sceneOrder, audioInvalidated, audioUrl, refreshKey, regenerating]);
useEffect(() => () => stopPreview(), [stopPreview]);
const preview = async () => {
// Global single-flight: stop whatever is loading/playing in ANY bar first.
// This also bumps the token, so this bar's own previous in-flight attempt
// is already invalidated by the time we capture the fresh token below.
activePreview?.stop();
const token = ++previewTokenRef.current;
setPreviewPhase('loading');
activePreview = { stop: stopPreview };
// The legacy URL of an unconverted pair is the narration when no pool or
// Dexie id resolved -- or when the resolved id turns out to have no local
// bytes, which is exactly the dangling-id case the URL survives for.
const src = (readAudioId ? await audioObjectUrl(readAudioId) : null) ?? audioUrl ?? null;
if (token !== previewTokenRef.current) {
// Superseded while loading (a newer click, a takeover, a stop): drop the
// result and revoke any blob URL we minted — the winner is in charge.
if (src && src.startsWith('blob:')) URL.revokeObjectURL(src);
return;
}
if (!src) {
stopPreview();
return;
}
objUrlRef.current = src;
const a = new Audio(src);
audioRef.current = a;
a.addEventListener('ended', stopPreview);
a.addEventListener('error', stopPreview);
try {
await a.play();
// Stopped while play() was settling (e.g. a takeover in the gap): the
// stop already paused it, nothing more to do.
if (token !== previewTokenRef.current) return;
setPreviewPhase('playing');
} catch {
// Autoplay rejection etc. — treat like any other stop.
stopPreview();
}
};
const regenerate = async () => {
setStatus('generating');
try {
const previousAudioId = audioId;
const id = await regenerateSpeechAudio(
sceneOrder,
{ id: actionId, text, audioId: previousAudioId },
language,
);
if (id) {
setReadAudioId(id);
onGenerated(id);
setStatus('ready');
} else {
setStatus('none');
}
} catch {
setStatus('error');
}
};
const STATUS: Record<TtsStatus, { label: string; cls: string }> = {
ready: { label: t('edit.tts.statusReady'), cls: 'text-emerald-600 dark:text-emerald-400' },
none: { label: t('edit.tts.statusNone'), cls: 'text-muted-foreground' },
generating: {
label: t('edit.tts.statusGenerating'),
cls: 'text-amber-600 dark:text-amber-400',
},
error: { label: t('edit.tts.statusError'), cls: 'text-rose-500' },
};
// A batch "Voice all" run drives this line's loading state from the parent
// (regenerating) — independent of the local single-line status. `batchPending`
// extends the generating state past the prop clearing, until this line's own
// audio re-check resolves to voiced / not voiced, so the batch end shows a
// clean generating → voiced transition with no intermediate flash.
const effStatus: TtsStatus = regenerating || batchPending ? 'generating' : status;
const s = STATUS[effStatus];
const previewLabel =
previewPhase === 'loading'
? t('edit.tts.cancelPreview')
: previewPhase === 'playing'
? t('edit.tts.stopPreview')
: t('edit.tts.preview');
// idle → Play; loading → spinner (click cancels the load); playing → Stop.
const PreviewIcon =
previewPhase === 'loading' ? Loader2 : previewPhase === 'playing' ? Square : Play;
return (
<div className="flex items-center gap-1 border-t border-border/60 px-2 py-1">
<Volume2 className="size-3 shrink-0 text-muted-foreground/40" />
<span className={cn('text-[10px] font-medium', s.cls)}>{s.label}</span>
<span className="ml-auto" />
<button
type="button"
onClick={previewPhase === 'idle' ? () => void preview() : stopPreview}
disabled={effStatus !== 'ready'}
className={cn(
'grid size-5 place-items-center rounded-md transition-colors disabled:opacity-30 disabled:hover:bg-transparent',
previewPhase === 'playing'
? 'text-rose-500 hover:bg-rose-500/10 hover:text-rose-600 dark:hover:text-rose-400'
: 'text-muted-foreground/60 hover:bg-muted hover:text-foreground',
)}
aria-label={previewLabel}
title={previewLabel}
>
<PreviewIcon className={cn('size-3', previewPhase === 'loading' && 'animate-spin')} />
</button>
<button
type="button"
onClick={regenerate}
disabled={effStatus === 'generating' || !text.trim()}
className="grid size-5 place-items-center rounded-md text-muted-foreground/60 transition-colors hover:bg-muted hover:text-foreground disabled:opacity-30 disabled:hover:bg-transparent"
aria-label={t('edit.tts.regenerate')}
title={t('edit.tts.regenerate')}
>
<RefreshCw className={cn('size-3', effStatus === 'generating' && 'animate-spin')} />
</button>
</div>
);
}
/** One spoken line — a numbered, editable clip block. */
function SpeechClip({
text,
index,
actionId,
audioId,
audioUrl,
audioInvalidated,
sceneOrder,
language,
autoFocus,
ttsActive,
ttsRefresh,
regenerating,
onCommit,
onGenerated,
onDelete,
onMoveLeft,
onMoveRight,
canMoveLeft,
canMoveRight,
onDragStart,
onDragEnd,
onFocused,
}: {
text: string;
index: number;
actionId: string;
audioId?: string;
audioUrl?: string;
audioInvalidated?: boolean;
sceneOrder: number;
language?: string;
autoFocus: boolean;
ttsActive: boolean;
ttsRefresh?: number;
regenerating?: boolean;
onCommit: (text: string) => void;
onGenerated: (audioId: string) => void;
onDelete: () => void;
onMoveLeft: () => void;
onMoveRight: () => void;
canMoveLeft: boolean;
canMoveRight: boolean;
onDragStart: (e: React.DragEvent) => void;
onDragEnd: () => void;
onFocused: () => void;
}) {
const { t } = useI18n();
const ref = useRef<HTMLTextAreaElement>(null);
const [val, setVal] = useState(text);
// Has the user typed since the last external sync? If not, external text
// changes (e.g. an agent regeneration mid-edit) are adopted even while
// focused — so a stale draft can't clobber regenerated narration on blur.
const dirtyRef = useRef(false);
useEffect(() => {
if (document.activeElement !== ref.current || !dirtyRef.current) {
// eslint-disable-next-line react-hooks/set-state-in-effect -- sync external text in only when not mid-edit
setVal(text);
dirtyRef.current = false;
}
}, [text]);
useEffect(() => {
if (autoFocus) {
ref.current?.focus();
onFocused();
}
}, [autoFocus, onFocused]);
const commit = () => {
if (dirtyRef.current && val !== text) onCommit(val);
dirtyRef.current = false;
};
const SpeechIcon = cueMeta('speech').icon;
const needsText = !text.trim();
return (
<div
className={cn(
'group/clip relative flex h-full w-[228px] shrink-0 flex-col overflow-hidden rounded-xl border border-border/85 bg-white/75 shadow-sm transition-colors focus-within:border-violet-400 hover:border-violet-300/70 dark:bg-slate-800/50 dark:hover:border-violet-500/40',
needsText && INCOMPLETE_CLIP,
)}
>
<span className="absolute inset-x-0 top-0 h-[3px] bg-primary/35" />
<div className="flex items-center gap-1.5 border-b border-border/60 bg-muted/40 px-2 py-1">
<span
draggable
onDragStart={onDragStart}
onDragEnd={onDragEnd}
className="cursor-grab text-muted-foreground/50 transition-colors hover:text-muted-foreground active:cursor-grabbing"
aria-label={t('edit.timeline.reorder')}
>
<GripVertical className="size-3.5" />
</span>
<span className="font-mono text-[10px] font-semibold tabular-nums text-muted-foreground/55">
{String(index).padStart(2, '0')}
</span>
<SpeechIcon className="size-3 text-primary/45" />
<span className="ml-auto mr-0.5 text-[8.5px] font-medium uppercase tracking-[0.12em] text-muted-foreground/50">
{t('edit.cue.speech')}
</span>
<MoveButtons
onLeft={onMoveLeft}
onRight={onMoveRight}
canLeft={canMoveLeft}
canRight={canMoveRight}
/>
<DeleteButton onDelete={onDelete} />
</div>
<textarea
ref={ref}
value={val}
onChange={(e) => {
dirtyRef.current = true;
setVal(e.target.value);
}}
onBlur={commit}
placeholder={t('edit.timeline.speechPlaceholder')}
className="flex-1 resize-none bg-transparent px-3 py-2 text-[12.5px] leading-[1.7] text-foreground/90 outline-none placeholder:text-muted-foreground/40 [scrollbar-width:thin]"
/>
{ttsActive && (
<SpeechTtsBar
actionId={actionId}
audioId={audioId}
audioUrl={audioUrl}
audioInvalidated={audioInvalidated}
sceneOrder={sceneOrder}
language={language}
text={val}
refreshKey={ttsRefresh}
regenerating={regenerating}
onGenerated={onGenerated}
/>
)}
</div>
);
}
/**
* A discussion node — the scene's terminal roundtable trigger. Unlike the other
* cues it isn't drag-addable or movable: a discussion must be the LAST action and
* there is at most one per scene (mirrors the action-parser invariant), so it's
* appended via the toolbar and pinned at the end. Inline-edits topic (required),
* prompt (optional) and the initiating agent. Topic/prompt commit on blur.
*/
function DiscussionClip({
topic,
prompt,
agentId,
agents,
onTopicChange,
onPromptChange,
onAgentChange,
onDelete,
}: {
topic: string;
prompt: string;
agentId: string;
agents: Array<{ id: string; name: string; avatar?: string }>;
onTopicChange: (v: string) => void;
onPromptChange: (v: string) => void;
onAgentChange: (v: string) => void;
onDelete: () => void;
}) {
const { t } = useI18n();
const m = cueMeta('discussion');
const needsTopic = !topic.trim();
// Local drafts committed on blur; synced from props when not mid-edit so a
// concurrent store update can't clobber an in-flight edit (mirrors SpeechClip).
const [topicVal, setTopicVal] = useState(topic);
const [promptVal, setPromptVal] = useState(prompt);
const topicDirty = useRef(false);
const promptDirty = useRef(false);
useEffect(() => {
// eslint-disable-next-line react-hooks/set-state-in-effect -- adopt external topic only when not mid-edit
if (!topicDirty.current) setTopicVal(topic);
}, [topic]);
useEffect(() => {
// eslint-disable-next-line react-hooks/set-state-in-effect -- adopt external prompt only when not mid-edit
if (!promptDirty.current) setPromptVal(prompt);
}, [prompt]);
return (
<div
className={cn(
'group/disc relative flex h-full w-[228px] shrink-0 flex-col overflow-hidden rounded-xl border border-gray-200/80 bg-white/70 shadow-sm transition-colors focus-within:border-yellow-400 hover:border-yellow-300/70 dark:border-gray-700/60 dark:bg-slate-800/50 dark:hover:border-yellow-500/40',
needsTopic && INCOMPLETE_CLIP,
)}
>
{/* The empty state reads from the amber dashed frame + the "topic
(required)" placeholder; the discussion keeps its Flag glyph identity. */}
<span className={cn('absolute inset-x-0 top-0 h-[3px]', m.accent)} />
<div className="flex items-center gap-1.5 border-b border-yellow-300/50 bg-yellow-400/15 px-2 py-1 dark:border-yellow-500/25 dark:bg-yellow-500/10">
<span className="flex size-4 items-center justify-center rounded-md bg-yellow-400 text-yellow-950 dark:bg-yellow-500 dark:text-slate-900">
<Flag className="size-2.5" />
</span>
<span className="text-[8.5px] font-semibold uppercase tracking-[0.12em] text-yellow-700 dark:text-yellow-400">
{t('edit.cue.discussion')}
</span>
<span
className="ml-auto text-[8.5px] font-medium uppercase tracking-[0.1em] text-yellow-600/70 dark:text-yellow-500/60"
title={t('edit.timeline.discussionTerminalHint')}
>
{t('edit.timeline.discussionTerminal')}
</span>
<DeleteButton onDelete={onDelete} />
</div>
<textarea
value={topicVal}
onChange={(e) => {
topicDirty.current = true;
setTopicVal(e.target.value);
}}
onBlur={() => {
if (topicDirty.current) onTopicChange(topicVal);
topicDirty.current = false;
}}
placeholder={t('edit.timeline.discussionTopicPlaceholder')}
className="h-[46px] shrink-0 resize-none bg-transparent px-3 pt-2 text-[12.5px] font-medium leading-[1.55] text-foreground/85 outline-none placeholder:font-normal placeholder:text-amber-500/60 [scrollbar-width:thin]"
/>
<textarea
value={promptVal}
onChange={(e) => {
promptDirty.current = true;
setPromptVal(e.target.value);
}}
onBlur={() => {
if (promptDirty.current) onPromptChange(promptVal);
promptDirty.current = false;
}}
placeholder={t('edit.timeline.discussionPromptPlaceholder')}
className="min-h-0 flex-1 resize-none bg-transparent px-3 text-[11px] leading-[1.6] text-muted-foreground outline-none placeholder:text-muted-foreground/35 [scrollbar-width:thin]"
/>
<div className="flex items-center gap-1 border-t border-gray-100 px-2 py-1 dark:border-gray-700/50">
<span className="shrink-0 text-[9px] text-muted-foreground/50">
{t('edit.timeline.discussionAgent')}
</span>
<Select
value={agentId || DISCUSSION_AGENT_NONE}
onValueChange={(v) => onAgentChange(v === DISCUSSION_AGENT_NONE ? '' : v)}
>
<SelectTrigger
size="sm"
className="ml-auto h-6 max-w-[150px] gap-1 rounded border-border px-1.5 py-0 text-[10px] shadow-none focus-visible:ring-yellow-400/40 [&_svg]:size-3"
>
<SelectValue placeholder={t('edit.timeline.discussionAgentUnset')} />
</SelectTrigger>
<SelectContent className="max-h-56">
<SelectItem value={DISCUSSION_AGENT_NONE} className="text-[11px]">
<span className="text-muted-foreground">
{t('edit.timeline.discussionAgentUnset')}
</span>
</SelectItem>
{agents.map((a) => (
<SelectItem key={a.id} value={a.id} className="text-[11px]">
<span className="flex items-center gap-1.5">
{a.avatar && (
<span className="size-4 shrink-0 overflow-hidden rounded-full bg-muted">
<AvatarDisplay src={a.avatar} alt={a.name} />
</span>
)}
{a.name}
</span>
</SelectItem>
))}
{/* keep a set agent visible even if it's no longer in the scene roster */}
{agentId && !agents.some((a) => a.id === agentId) && (
<SelectItem value={agentId} className="text-[11px]">
{agentId}
</SelectItem>
)}
</SelectContent>
</Select>
</div>
</div>
);
}
/**
* A non-speech cue — its own compact card on the timeline (the action's implicit
* container, made explicit). Carries a delete button; clicking an element-bound
* cue arms canvas pick mode so the target is chosen on the slide itself.
*/
function CueMarker({
action,
elements,
onTip,
onDelete,
onPick,
onMoveLeft,
onMoveRight,
canMoveLeft,
canMoveRight,
onDragStart,
onDragEnd,
}: {
action: Action;
elements: { id?: string; type: string; content?: string }[];
onTip: (t: TooltipState | null) => void;
onDelete: () => void;
onPick: () => void;
onMoveLeft: () => void;
onMoveRight: () => void;
canMoveLeft: boolean;
canMoveRight: boolean;
onDragStart: (e: React.DragEvent) => void;
onDragEnd: () => void;
}) {
const { t } = useI18n();
useClearCuePreviewOnUnmount();
const m = cueMeta(action.type);
const label = cueLabel(action.type, t);
const Icon = m.icon;
const bound = ELEMENT_BOUND.has(action.type);
const elementId = (action as { elementId?: string }).elementId ?? '';
const needsTarget = bound && !elementId;
// Bound cue → show what it's actually pointing at, not a generic "bound";
// the element may have been deleted since binding, so fall back gracefully.
const boundEl = elementId ? elements.find((e) => e.id === elementId) : undefined;
return (
<div
draggable
onDragStart={onDragStart}
onDragEnd={onDragEnd}
onMouseEnter={(e) => {
onTip({ action, anchor: e.currentTarget.getBoundingClientRect() });
applyCuePreview(cuePreviewFor(action));
}}
onMouseLeave={() => {
onTip(null);
clearCuePreview();
}}
onClick={() => {
if (bound) onPick();
}}
className={cn(
'group/cue relative flex h-full w-[108px] shrink-0 flex-col overflow-hidden rounded-xl border border-gray-200/80 bg-white/65 shadow-sm transition-colors dark:border-gray-700/60 dark:bg-slate-800/40',
bound
? 'cursor-pointer hover:border-violet-300/70 dark:hover:border-violet-500/40'
: 'cursor-grab active:cursor-grabbing',
needsTarget && cn('border-dashed', m.dash),
)}
aria-label={label}
>
<span className={cn('absolute inset-x-0 top-0 h-[3px]', m.accent)} />
<div className="flex items-center gap-0.5 px-1 pt-1">
<span
draggable
onDragStart={onDragStart}
onDragEnd={onDragEnd}
onClick={(e) => e.stopPropagation()}
className="cursor-grab text-muted-foreground/35 transition-colors hover:text-muted-foreground active:cursor-grabbing"
aria-label={t('edit.timeline.reorder')}
>
<GripVertical className="size-3.5" />
</span>
<span className="ml-auto flex items-center">
<MoveButtons
onLeft={onMoveLeft}
onRight={onMoveRight}
canLeft={canMoveLeft}
canRight={canMoveRight}
/>
<DeleteButton onDelete={onDelete} />
</span>
</div>
<div className="flex flex-1 flex-col items-center justify-center gap-1.5 px-1 pb-1">
<span className={cn('flex size-8 items-center justify-center rounded-full', m.glyph)}>
<Icon className="size-4" />
</span>
<span className="text-[10px] font-medium text-foreground/70">{label}</span>
{bound && (
<span
className={cn(
'text-[9px]',
needsTarget
? 'font-medium text-amber-600 dark:text-amber-400'
: 'text-muted-foreground/45',
)}
>
{needsTarget
? t('edit.timeline.pickElement')
: `→ ${boundEl ? elementLabel(boundEl, t) : t('edit.timeline.bound')}`}
</span>
)}
</div>
</div>
);
}
/** A node anchored on the axis — drag handle; for cues, hover preview + click-to-pick. */
function NodeDot({
action,
onTip,
onPick,
onDragStart,
onDragEnd,
canDrag = true,
}: {
action: Action;
onTip: (t: TooltipState | null) => void;
onPick: () => void;
onDragStart: (e: React.DragEvent) => void;
onDragEnd: () => void;
canDrag?: boolean;
}) {
const { t } = useI18n();
useClearCuePreviewOnUnmount();
const isSpeech = action.type === 'speech';
// A discussion is the scene's terminal anchor — give its node a distinct
// marker (square, filled yellow) so it reads as the end stop, not a regular
// cue. Its flag glyph comes from cue-meta like every other type's.
const isDiscussion = action.type === 'discussion';
const bound = ELEMENT_BOUND.has(action.type);
const elementId = (action as { elementId?: string }).elementId ?? '';
const needsTarget = bound && !elementId;
const m = cueMeta(action.type);
const label = cueLabel(action.type, t);
const Icon = m.icon;
return (
<span
draggable={canDrag}
onDragStart={onDragStart}
onDragEnd={onDragEnd}
title={isSpeech ? (action as { text?: string }).text?.slice(0, 60) : label}
onMouseEnter={(e) => {
if (isSpeech) return;
onTip({ action, anchor: e.currentTarget.getBoundingClientRect() });
applyCuePreview(cuePreviewFor(action));
}}
onMouseLeave={() => {
if (isSpeech) return;
onTip(null);
clearCuePreview();
}}
onClick={() => {
if (bound) onPick();
}}
className={cn(
'grid size-6 place-items-center ring-2 ring-white transition-transform hover:scale-110 dark:ring-slate-900',
isDiscussion ? 'rounded-[7px]' : 'rounded-full',
needsTarget
? 'text-amber-600 bg-amber-100 ring-amber-200 animate-pulse dark:bg-amber-500/20 dark:text-amber-400'
: isDiscussion
? 'bg-yellow-400 text-yellow-900 ring-yellow-200 dark:bg-yellow-500 dark:text-slate-900 dark:ring-yellow-500/30'
: m.glyph,
bound
? 'cursor-pointer'
: canDrag
? 'cursor-grab active:cursor-grabbing'
: 'cursor-default',
)}
aria-label={label}
>
<Icon className="size-3.5" />
</span>
);
}
/** Slim insertion slot between items; widens + glows while a drag hovers it. */
function DropZone({
active,
slot,
onEnter,
onDrop,
onInsert,
insertLabel,
flex,
}: {
active: boolean;
slot: number;
onEnter: () => void;
onDrop: () => void;
onInsert: (slot: number, rect: DOMRect) => void;
insertLabel: string;
flex?: boolean;
}) {
return (
<div
onDragOver={(e) => {
e.preventDefault();
onEnter();
}}
onDrop={(e) => {
e.preventDefault();
onDrop();
}}
className={cn(
'group/ins relative flex h-full shrink-0 items-start justify-center pt-2 transition-all',
flex ? 'flex-1' : active ? 'w-10' : 'w-4',
)}
>
<span
className={cn(
'pointer-events-none absolute inset-y-3 left-1/2 w-0.5 -translate-x-1/2 rounded-full transition-colors',
active ? 'bg-primary' : 'bg-transparent',
)}
/>
{!active && (
<button
type="button"
aria-label={insertLabel}
title={insertLabel}
onClick={(e) => onInsert(slot, e.currentTarget.getBoundingClientRect())}
className="relative z-[1] grid size-[22px] scale-90 place-items-center rounded-full border border-dashed border-primary/40 bg-background text-primary/70 opacity-30 transition-all hover:scale-100 hover:border-solid hover:border-primary hover:bg-primary/5 hover:text-primary hover:opacity-100 group-hover/ins:opacity-90"
>
<Plus className="size-3" />
</button>
)}
</div>
);
}
export function ActionsBar({ sceneId }: { sceneId: string }) {
const { t } = useI18n();
const scene = useStageStore((s) => s.scenes.find((x) => x.id === sceneId));
const actions = scene?.actions ?? EMPTY;
const sceneOrder = scene?.order ?? 0;
// Element-bound cues (spotlight / laser) point at slide elements, so they only
// make sense on SLIDE scenes. While the scene hasn't loaded yet, fall back to
// a non-slide type so the picker doesn't briefly offer unsupported cues.
const sceneType: SceneType = scene?.type ?? 'quiz';
// Slide-scene canvas elements — feeds CueMarker's bound-cue label lookup
// ("→ <element name>" instead of a generic "bound"). Non-slide scenes'
// `content` has no `canvas`, so this is always [] there.
const sceneElements =
(
scene?.content as
| { canvas?: { elements?: { id?: string; type: string; content?: string }[] } }
| undefined
)?.canvas?.elements ?? EMPTY_ELEMENTS;
const language = useStageStore((s) => s.stage?.languageDirective);
// Managed TTS on → speech clips show audio status + preview / regenerate.
const ttsActive = useSettingsStore(
(s) => s.ttsEnabled && s.ttsProviderId !== 'browser-native-tts',
);
// Agents a discussion can be initiated by — sourced from the user's currently
// SELECTED agents, the exact set the playback engine gates on: it skips (and
// consumes) any discussion whose `agentId` isn't selected. Offering the same
// set here means whatever the author picks will actually fire at playback;
// anything else (scene/stage roster) could let them save an initiator that the
// engine silently drops. With nothing selected only "unspecified" remains,
// which is correct since an unset `agentId` is never skipped.
const agentsRecord = useAgentRegistry((s) => s.agents);
const selectedAgentIds = useSettingsStore((s) => s.selectedAgentIds);
const discussionAgents = useMemo(
() =>
selectedAgentIds
.map((id) => agentsRecord[id])
.filter(Boolean)
.map((a) => ({ id: a.id, name: a.name, avatar: a.avatar })),
[selectedAgentIds, agentsRecord],
);
// Collapsed = the dock's own fold, which this bar's title and its trailing
// toggle both drive. The timeline's collapsed form is its axis of node icons,
// so it reads `collapsed` rather than being hidden by the shell.
const { collapsed: lineMode, toggleCollapsed } = useEditDock();
const [tip, setTip] = useState<TooltipState | null>(null);
const [dragOver, setDragOver] = useState<number | null>(null);
const [focusId, setFocusId] = useState<string | null>(null);
const [regenAll, setRegenAll] = useState(false);
const [pickerAt, setPickerAt] = useState<{ slot: number; rect: DOMRect } | null>(null);
// Ids of speech lines currently being (re)generated by "Voice all", so each
// line's status row shows the generating state for the duration of the batch.
const [regeneratingIds, setRegeneratingIds] = useState<ReadonlySet<string>>(NO_IDS);
const [ttsRefresh, setTtsRefresh] = useState(0); // bump → speech clips re-check audio status
const reduce = useReducedMotion();
const dragRef = useRef<DragPayload | null>(null);
// Apply an edit to the LATEST actions from the store (not the render-time
// snapshot), so a concurrent agent/TTS update isn't reverted by a later UI
// commit (drag / reorder / blur / delete).
const commit = useCallback(
(updater: (cur: Action[]) => Action[]) => {
const cur = useStageStore.getState().scenes.find((s) => s.id === sceneId)?.actions ?? [];
useStageStore.getState().updateScene(sceneId, { actions: updater(cur) });
},
[sceneId],
);
// Regenerate TTS for every speech line in the scene, then stamp audioIds.
// Reads the latest actions from the store at each step so a concurrent edit
// isn't clobbered, and stamps by id (index-stale-safe).
const regenerateAllAudio = useCallback(async () => {
if (regenAll) return;
const latest = () => useStageStore.getState().scenes.find((s) => s.id === sceneId);
const speeches = (latest()?.actions ?? []).filter(
(a) => a.type === 'speech' && ((a as { text?: string }).text ?? '').trim(),
);
if (!speeches.length) return;
const order = latest()?.order ?? 0;
setRegenAll(true);
// Light up every queued line's status row up front (they're all about to be
// synthesized), cleared together in the finally once the batch settles.
setRegeneratingIds(new Set(speeches.map((a) => a.id).filter(Boolean) as string[]));
try {
for (const queued of speeches) {
if (!queued.id) continue;
try {
const liveAction = latest()?.actions?.find(
(action) => action.type === 'speech' && action.id === queued.id,
);
if (!liveAction || liveAction.type !== 'speech') continue;
const previousAudioId = liveAction.audioId;
const id = await regenerateSpeechAudio(
latest()?.order ?? order,
{
id: liveAction.id,
text: liveAction.text,
audioId: previousAudioId,
},
language,
);
if (id) {
commit((cur) => setAudioIdById(cur, liveAction.id!, id));
await flushStageSave();
}
} catch {
/* skip a failed line, keep going */
}
}
} finally {
setRegenAll(false);
setRegeneratingIds(NO_IDS);
// Always re-check every line's audio at batch end — even when nothing
// synthesized — so each SpeechTtsBar resolves its status (and clears its
// batchPending flag) instead of getting stuck in the generating state.
setTtsRefresh((n) => n + 1);
}
}, [regenAll, sceneId, language, commit]);
const scrollRef = useRef<HTMLDivElement>(null);
const panViewport = (dir: -1 | 1) =>
scrollRef.current?.scrollBy({ left: dir * 280, behavior: 'smooth' });
const newId = () =>
typeof crypto !== 'undefined' && crypto.randomUUID ? crypto.randomUUID() : `a-${Date.now()}`;
// Insert path for the ActionPicker (header pill / inline "+" drop-zone
// buttons): appends a discussion (terminal, at-most-one) or inserts an
// ordinary action at a slot, capped before any existing discussion so it
// always stays last.
const insertActionAt = useCallback(
(type: PickerType, slot: number) => {
const id = newId();
if (type === 'discussion') {
commit((cur) => appendDiscussion(cur, id));
return;
}
const action = makeAction(type, id);
commit((cur) => insertAt(cur, clampInsertSlot(cur, slot), action));
if (type === 'speech') setFocusId(id);
},
[commit],
);
const handleDrop = useCallback(
(slot: number) => {
const p = dragRef.current;
dragRef.current = null;
setDragOver(null);
if (!p) return;
commit((cur) => moveById(cur, p.id, clampInsertSlot(cur, slot)));
},
[commit],
);
const speechCount = actions.filter((a) => a.type === 'speech').length;
const cueCount = actions.length - speechCount;
// A discussion is pinned at the end (at most one), so ordinary actions can move
// right only up to the slot before it; the discussion node itself can't move.
const discussionPresent = hasDiscussion(actions);
const lastMovableIndex = discussionPresent ? actions.length - 2 : actions.length - 1;
let speechIndex = 0;
const items = actions.map((action, index) => {
if (action.type === 'speech') speechIndex += 1;
return { action, index, key: (action.id ?? `a-${index}`) as string, speechIndex };
});
const header = (
<>
{/* The title is also the fold: it was a click target long before there was
a toggle beside it, so the cursor already knows this spot. */}
<button
type="button"
data-testid="edit-timeline-title"
onClick={toggleCollapsed}
className="flex shrink-0 items-center gap-2.5"
>
<span className="size-1.5 rounded-full bg-primary" />
<span className="text-[12px] font-medium tracking-[0.18em] text-foreground/80">
{t('edit.timeline.title')}
</span>
</button>
{!lineMode && (
<button
type="button"
onClick={(e) => {
const slot = discussionPresent ? actions.length - 1 : actions.length;
setPickerAt({ slot, rect: e.currentTarget.getBoundingClientRect() });
}}
className="ml-1.5 inline-flex shrink-0 items-center gap-1 rounded-full border border-primary/25 bg-primary/10 px-2.5 py-0.5 text-[11px] font-medium text-primary transition-colors hover:bg-primary/15"
>
<Plus className="size-3" />
{t('edit.timeline.addAction')}
<ChevronDown className="size-3 opacity-70" />
</button>
)}
{!lineMode && ttsActive && (
<button
type="button"
onClick={regenerateAllAudio}
disabled={regenAll}
title={t('edit.timeline.regenAllTts')}
aria-label={t('edit.timeline.regenAllTts')}
className="inline-flex shrink-0 items-center gap-1 rounded-full border border-border bg-muted/40 px-2 py-0.5 text-[11px] text-muted-foreground transition-colors hover:border-primary/30 hover:text-foreground disabled:opacity-50"
>
<RefreshCw className={cn('size-3', regenAll && 'animate-spin')} />
{t('edit.timeline.voiceAll')}
</button>
)}
<span className="ml-auto shrink-0 font-mono text-[11px] tabular-nums text-muted-foreground/60">
{t('edit.timeline.counts', { speech: speechCount, cue: cueCount })}
</span>
{/* pan the timeline viewport left/right */}
{!lineMode && (
<div className="flex shrink-0 items-center border-l border-gray-200/70 pl-1 dark:border-gray-700/60">
<button
type="button"
onClick={() => panViewport(-1)}
title={t('edit.timeline.panLeft')}
aria-label={t('edit.timeline.panLeft')}
className="grid size-7 place-items-center rounded-md text-muted-foreground/60 transition-colors hover:bg-muted hover:text-foreground"
>
<ChevronsLeft className="size-4" />
</button>
<button
type="button"
onClick={() => panViewport(1)}
title={t('edit.timeline.panRight')}
aria-label={t('edit.timeline.panRight')}
className="grid size-7 place-items-center rounded-md text-muted-foreground/60 transition-colors hover:bg-muted hover:text-foreground"
>
<ChevronsRight className="size-4" />
</button>
</div>
)}
<button
type="button"
onClick={toggleCollapsed}
title={lineMode ? t('edit.timeline.expandTrack') : t('edit.timeline.collapseAxis')}
aria-label={lineMode ? t('edit.timeline.expandTrack') : t('edit.timeline.collapseAxis')}
className="ml-1 grid size-7 shrink-0 place-items-center rounded-md text-muted-foreground/60 transition-colors hover:bg-muted hover:text-foreground"
>
{lineMode ? <UnfoldVertical className="size-4" /> : <FoldVertical className="size-4" />}
</button>
</>
);
return (
<>
{/* The timeline's own header row: what it is, what it can insert, how much
it holds, and its fold. Geometry unchanged from the standalone bar. */}
<div className="flex h-10 shrink-0 items-center gap-2 px-6">{header}</div>
<div ref={scrollRef} className="min-h-0 flex-1 overflow-x-auto overflow-y-hidden">
<div className="relative h-full min-w-max">
{/* the timeline axis (top) — nodes hang below it; hidden when empty
so the placeholder hint doesn't collide with the line */}
{actions.length > 0 && (
<div
className="pointer-events-none absolute inset-x-3 bg-gradient-to-r from-border/30 via-border to-border/30"
style={{ top: AXIS_FROM_TOP - 1, height: 2 }}
/>
)}
<div className="relative flex h-full items-stretch px-3.5">
{actions.length === 0 && (
<span className="pointer-events-none absolute left-1/2 top-1/2 -translate-x-1/2 -translate-y-1/2 text-[12px] text-muted-foreground/60">
{t('edit.timeline.emptyHint')}
</span>
)}
<DropZone
active={dragOver === 0}
slot={0}
flex={actions.length === 0}
onEnter={() => setDragOver(0)}
onDrop={() => handleDrop(0)}
onInsert={(slot, rect) => setPickerAt({ slot, rect })}
insertLabel={t('edit.timeline.addAction')}
/>
{items.map(({ action, index, key, speechIndex: si }) => {
// A discussion is pinned terminal, so it can't be drag-reordered.
const isDiscussion = action.type === 'discussion';
const onDragStart = isDiscussion
? () => {}
: (e: React.DragEvent) => {
dragRef.current = { kind: 'move', id: key };
setBlankDragImage(e);
};
const onDragEnd = () => {
dragRef.current = null;
setDragOver(null);
};
const onPick = () =>
useCanvasStore.getState().setPickTarget(
useStageStore.getState().stage?.id
? {
purpose: 'cue',
stageId: useStageStore.getState().stage!.id,
sceneId,
actionId: key,
cueType: action.type,
}
: null,
);
const dot = (
<NodeDot
action={action}
onTip={setTip}
onPick={onPick}
onDragStart={onDragStart}
onDragEnd={onDragEnd}
canDrag={!isDiscussion}
/>
);
return (
<div key={key} className="relative flex h-full items-stretch">
<motion.div
initial={reduce ? false : { opacity: 0, y: -6 }}
animate={{ opacity: 1, y: 0 }}
transition={{
duration: 0.22,
delay: reduce ? 0 : Math.min(index * 0.02, 0.24),
ease: 'easeOut',
}}
className="flex h-full flex-col items-center"
style={{ paddingTop: AXIS_FROM_TOP - 12 }}
>
{lineMode ? (
<div className="w-9">{dot}</div>
) : (
<>
{dot}
<div className="my-1 h-2.5 w-px bg-border" />
<div className="min-h-0 w-full flex-1">
{action.type === 'speech' ? (
<SpeechClip
text={(action as { text?: string }).text ?? ''}
index={si}
actionId={key}
audioId={(action as { audioId?: string }).audioId}
audioUrl={(action as { audioUrl?: string }).audioUrl}
audioInvalidated={
(action as { audioInvalidated?: boolean }).audioInvalidated
}
sceneOrder={sceneOrder}
language={language}
ttsActive={ttsActive}
ttsRefresh={ttsRefresh}
regenerating={regeneratingIds.has(key)}
autoFocus={key === focusId}
onFocused={() => setFocusId(null)}
onCommit={(text) => {
// Editing the text invalidates any cached audio
// (the blob is keyed by order+id and the stamped
// audioId, not the text), so drop the stamped
// fields and delete the blob — the line then
// reads as un-voiced until regen.
const prevAudioId = (action as { audioId?: string }).audioId;
commit((cur) => setSpeechTextClearAudioById(cur, key, text));
// Re-check status only AFTER the blob is gone, so
// the status row can't race the async delete and
// briefly still read "voiced".
void discardSpeechAudio(sceneOrder, {
id: key,
audioId: prevAudioId,
}).finally(() => setTtsRefresh((n) => n + 1));
}}
onGenerated={(assetId) => {
commit((cur) => setAudioIdById(cur, key, assetId));
// Stage persistence is debounced; flush so the
// stamped reference is durable once the row
// settles (pool bytes persist first, stamp last).
void flushStageSave().catch(() => undefined);
}}
onDelete={() => {
commit((cur) => removeById(cur, key));
}}
onMoveLeft={() => commit((cur) => moveByIdDir(cur, key, -1))}
onMoveRight={() => commit((cur) => moveByIdDir(cur, key, 1))}
canMoveLeft={index > 0}
canMoveRight={index < lastMovableIndex}
onDragStart={onDragStart}
onDragEnd={onDragEnd}
/>
) : isDiscussion ? (
<DiscussionClip
topic={(action as DiscussionAction).topic ?? ''}
prompt={(action as DiscussionAction).prompt ?? ''}
agentId={(action as DiscussionAction).agentId ?? ''}
agents={discussionAgents}
onTopicChange={(v) =>
commit((cur) => setDiscussionTopicById(cur, key, v))
}
onPromptChange={(v) =>
commit((cur) => setDiscussionPromptById(cur, key, v))
}
onAgentChange={(v) =>
commit((cur) => setDiscussionAgentById(cur, key, v))
}
onDelete={() => commit((cur) => removeById(cur, key))}
/>
) : (
<CueMarker
action={action}
elements={sceneElements}
onTip={setTip}
onDelete={() => commit((cur) => removeById(cur, key))}
onPick={onPick}
onMoveLeft={() => commit((cur) => moveByIdDir(cur, key, -1))}
onMoveRight={() => commit((cur) => moveByIdDir(cur, key, 1))}
canMoveLeft={index > 0}
canMoveRight={index < lastMovableIndex}
onDragStart={onDragStart}
onDragEnd={onDragEnd}
/>
)}
</div>
</>
)}
</motion.div>
<DropZone
active={dragOver === index + 1}
slot={index + 1}
onEnter={() => setDragOver(index + 1)}
onDrop={() => handleDrop(index + 1)}
onInsert={(slot, rect) => setPickerAt({ slot, rect })}
insertLabel={t('edit.timeline.addAction')}
/>
</div>
);
})}
</div>
</div>
</div>
{tip && <CueTooltip tip={tip} />}
{pickerAt && (
<ActionPicker
anchor={pickerAt.rect}
sceneType={sceneType}
actions={actions}
onSelect={(type) => insertActionAt(type, pickerAt.slot)}
onClose={() => setPickerAt(null)}
/>
)}
</>
);
}