Files
b4e3145c90 feat(roster): single-source the generated agent roster on the stage document (#994)
* feat(roster): single-source the generated agent roster on the stage document

The generated agent roster used to live in two places at once: the stage
document carried generatedAgentConfigs, while a per-stage IndexedDB mirror
(db.generatedAgents) was the actual read authority — and the only home of
each agent's voice binding. The split caused real losses: voices vanished on
device switches, exports, and shares (the mirror never travels), the load
path trusted a device-local table over the portable document, and roster
edits persisted through a bespoke debounce with no beforeunload flush.

Make the stage document the single source of truth:

- Contract: add optional voiceConfig/voiceDesign to GeneratedAgentConfig in
  @openmaic/dsl (VoiceDesign moves into the contract; the app re-exports it).
  Additive optional fields — not a breaking serialized-shape change, so
  DSL_VERSION stays put per the version policy.
- Writes: remove every mirror writer (registry saveGeneratedAgents, the
  stage store's debouncedSaveAgents bypass, classroom-load hydration, import
  direct writes). setStageAgents now only marks the stage dirty for the
  shared persistence scheduler plus synchronous in-memory registry/selection
  mirrors; generation and import embed the roster on the stage.
- Reads: classroom load hydrates the registry from the loaded stage's
  generatedAgentConfigs; export reads the stage roster directly.
- Lazy migration: the mirror is retained read-only. On load, a roster (or
  voice fields) missing from the document is backfilled from the mirror,
  committed onto the in-memory stage, and persisted by the next flush —
  idempotent, and gated on the current stage id after every await so a
  classroom switch cannot leak a stale roster into the registry.
- Deletion: deleteStageData discards the deleted stage's pending
  persistence work so a queued flush cannot resurrect the document.
- Round trip: classroom ZIP manifests now carry the voice fields, so
  export/import preserves agent voices.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(roster): harden deletion, migration, selection, and ingress paths

Review hardening on top of the roster single-source change:

- Deletion vs in-flight persistence: deleting a stage now registers a
  session-scoped tombstone (lib/utils/deleted-stages.ts) before the cascade
  starts. Every persistence landing point checks it — the scheduled flush and
  the departing-stage retry drop their snapshots, and the aggregate/incremental
  saves re-check under the document lock, including the incremental path's
  full-save fallback that would otherwise rebuild a deleted document from the
  in-memory snapshot. A deletion that fails before the document is removed
  lifts the tombstone so the still-existing stage keeps persisting.
- Lazy migration termination: a fruitless legacy-mirror probe (nothing to
  merge — e.g. server-generated rosters, which are voiceless by design) is
  remembered per stage for the session instead of re-querying the mirror on
  every load forever. Successful merges are not memoized so a failed flush is
  retried on the next load. Docstrings now state the real lifecycle.
- Selection provenance: setStageAgents no longer overwrites a user-set agent
  selection with the full roster. Stage-derived selections still track the
  roster; a user-set auto selection is only narrowed to surviving agents
  (newcomers are not auto-selected, the user-set flag is untouched), and a
  user-set preset selection is left alone — matching restoreAgentSelection.
- Mirror hygiene: deleteStageData clears the deleted stage's rows from the
  legacy roster mirror (best-effort), and the mirror table's docs now describe
  its actual access pattern (migration reads + deletion hygiene).
- Registry persistence: the agent registry's persist snapshot partializes
  generated agents out of localStorage, making the stage document the roster's
  only durable home; the rehydration merge filter remains as defense in depth.
- Ingress validation: imported manifest voice fields are structurally
  validated (malformed bindings dropped per field, the agent survives), and
  applyGeneratedAgentsToRegistry validates providerId against the known TTS
  provider registry instead of casting, treating unknown providers as "no
  bound voice".
- Contract: drop the speculative AgentVoiceConfig.modelId — an audit of every
  roster producer, current and historical, found none that ever emitted it.
  Contract fields are added once a producer exists, not before.

New tests cover the deletion tombstone at both the store and storage layers
(including the missing-destination full-save pin and the lock-race re-check),
the fruitless-probe memo, the selection-provenance matrix, persistence
exclusion of generated agents, and voice-field sanitization on import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(roster): close tombstone lifecycle, probe-memo, and selection-intent gaps

Second hardening pass over the deletion/migration/selection fixes:

- Tombstone lifecycle: a deleted classroom id can legitimately come back
  (deletion is client-side only). Explicit (re)creation points now lift the
  tombstone — server-copy restore in applyClassroomStageAndScenes and backup
  restore in importDatabase — so edits after a same-session restore persist
  again, while in-flight flushes stay fenced.
- Failed deletion now restores the dirt it discarded: deleteStageData
  snapshots the pending map plus the in-flight round before tombstoning, and
  re-marks it when the delete fails with the document still present.
- Legacy-mirror probe memo now distinguishes a FAILED read (null, retried on
  the next load) from a confirmed-empty mirror (memoized), so a transient
  IndexedDB error cannot suppress migration for the whole session.
- Incremental save tail (currentScene KV row + chat sessions) is tombstone-
  fenced like the aggregate path, and both paths re-check the tombstone
  immediately before every write, with comments scoped honestly for the
  lock-free (no Web Locks) best-effort LWW fallback.
- Roster-edit selection mirror: an empty intersection now falls back to the
  full roster and clears the user-set flag (matching restoreAgentSelection's
  length > 0 gate), and a user-set selection that equals the pre-edit full
  roster keeps tracking the roster wholesale so newcomers are not excluded
  forever by the AgentBar auto-toggle snapshot.
- TTS/ASR provider lookups use Object.hasOwn instead of `in`/bare indexing,
  so prototype-chain keys ('toString', 'constructor', ...) no longer pass the
  provider whitelist or resolve to Object.prototype members.
- Documented the per-tab limit of the tombstone set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(roster): replace the boolean deletion tombstone with a deletion-generation model

A boolean tombstone cannot express the deletion lifecycle once a deleted id
legitimately comes back (server-copy restore on revisit, backup import): after
the restore lifts the flag, a pre-delete flush still in flight is
indistinguishable from a post-restore edit and can overwrite the restored
document with pre-delete content. Two rounds of point fixes kept leaking
variants of this race, so this restructures the fence instead.

deleted-stages.ts now keeps a per-stage monotonic deletion epoch plus a
deleted flag. markStageDeleted bumps the epoch; unmarkStageDeleted clears
only the flag and never rewinds the epoch. Every persistence path captures
the epoch at the moment it captures the data it will write (flush-round
snapshot, departing-stage snapshot, aggregate save entry) and re-checks
"captured epoch is current AND not deleted" immediately before each
individual write. Invariants:

- A write captured before a deletion can never land after it, even when a
  same-id restore has lifted the deleted flag (the captured epoch is
  permanently stale).
- A write capturing data after a restore observes the current epoch and
  persists normally.
- A failed deletion lifts only the flag; the restored pending changes are
  re-queued as descriptors, so their flush re-captures current store state
  under the current epoch and is neither dropped nor a stale replay.

Also fixed in the same restructuring:

- Incremental tail writes (currentScene, chats) each re-check independently;
  a delete landing while the first tail write is awaiting now fences the
  chat write instead of riding the earlier check.
- Deleting a classroom evicts it from the warm in-memory store, and
  loadFromStorage treats a warm-but-deleted stage as not loaded (discarding
  the ghost), so navigating Back to a deleted classroom reaches the
  server-restore path instead of rendering an editable ghost whose every
  edit is silently dropped.
- importDatabase records each document's pre-import deletion state and
  reinstates it when a failed import rolls the document back, so an
  outstanding flush cannot recreate the rolled-back document.
- Docstrings on the failed-deletion restore path now state that only dirt
  captured before the deletion is restored; edits refused during the
  deletion window are not part of the snapshot.

New tests cover the epoch invariants at the storage layer (pre-delete rounds
dropped across restores on the incremental, aggregate, and tail paths;
post-restore captures landing; strictly increasing epochs), the scheduler
layer (departing-retry delete+restore straddle; restored dirt flushing under
the current epoch), the warm-ghost restore driven through the real
loadFromStorage, and the import lift/rollback pair.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(deletion): report fenced drops, gate ghost eviction on cascade settlement

Edge hardening around the deletion-epoch fence; the epoch model itself is
unchanged.

- An epoch-stale drop is now reported, not disguised: saveStageData /
  saveStageDataIncremental return a distinct 'stale-dropped' status (entry
  check, every in-mutation re-check, and both tail-write re-checks) instead
  of a shape identical to success. saveToStorage reacts by skipping all
  success bookkeeping - no chatSnapshot rebind to a snapshot that never
  landed, no pending-dirt clearing - and returns false, keeping its
  "true means verified write" contract honest. persistDirtySnapshot keeps
  its existing (correct) semantics explicitly: a drop has nothing to retry.
- Warm-ghost eviction is settlement-gated: the deletion cascade records an
  in-flight bit (begun by deleteStageData, settled in its finally), and the
  deleted-warm branch of loadFromStorage keeps the warm state while the
  cascade is unsettled - a delete can still fail before removing the
  document, in which case the warm state plus the restored pending dirt is
  the only copy of the pre-delete edits. Once settled, the ghost discard
  and server-restore path behave as before.
- Read-side landing re-check: loadFromStorage re-checks isStageDeleted
  after hydration, immediately before set(), mirroring the write-side
  "re-check immediately before landing" discipline - a lock-free
  mid-cascade read can no longer re-materialize a ghost classroom.
- clearStoreForDeletedStage re-checks isStageDeleted at eviction time, so a
  same-id restore completing inside the cascade-tail window is never wiped.
- capturedEpoch is now a required parameter on both storage entry points:
  the default (call-time capture) silently reopened the capture-point/
  validation-point split for future callers; the type system now enforces
  the pairing. All production callers already passed it explicitly.

New tests cover the dropped-status contract, the cascade in-flight
lifecycle (including both failure shapes), keep-warm during an unsettled
delete with end-to-end pending restore after a failed delete, settlement-
then-eviction on success, the eviction guard against a tail-window restore,
and the read-side hydration re-check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(deletion): settle the deleted-warm load by awaiting the cascade outcome

Hardening around the deletion settlement gate introduced previously; the
epoch model is unchanged.

- The deleted-warm branch of loadFromStorage no longer completes the load
  while the deletion cascade is undecided (which exposed a classroom whose
  edits were refused without being restorable, and left a successful
  deletion's eviction with no follow-up reload). It now parks on a per-stage
  settlement promise (stageDeletionSettled) and branches on the outcome:
  a failed delete keeps the warm, restored classroom; a successful delete
  falls through to a full cold load so the server-restore path recovers the
  route; a navigation that moved on during the park is stopped by the load
  token guard. The eviction (clearStoreForDeletedStage) deliberately stops
  claiming a new load token so the parked load stays current - ghost
  re-materialization is fenced by the read-side deleted re-checks instead.
  No deadlock: a parked load holds no document lock, and the cascade never
  waits on a load.
- 'stale-dropped' now propagates through the debounced flush path:
  persistDirtySnapshot returns the sentinel instead of an empty failure set,
  and startFlushRound skips the chatSnapshot rebind on it (mirroring
  saveToStorage) - a falsely rebound baseline would let later chat saves
  no-op-skip chats that never landed and corrupt cross-tab conflict
  detection. Pending-map clearing keeps its empty-failure semantics
  (restored descriptors mint fresh revisions).
- The cascade in-flight state is a counter (begin++/settle--) instead of a
  boolean, and deleteStageData is single-flight per stage (a concurrent
  second call joins the first cascade), so overlapping deletions can never
  expose an undecided cascade as settled.
- The StaleDroppedSave docstring no longer overpromises: a tail
  saveCurrentScene can land after the cascade's clearCurrentScene; the
  orphaned cursor row is ignored by the load path rather than removed by
  the cascade.

Net +8 tests: overlapping-cascade counting, settlement-promise resolution
(success and failure), concurrent double-delete single-flight join, the
parked mid-cascade Back in both outcomes plus the navigation-away guard,
and the fenced flush round keeping an honest chat baseline end-to-end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(deletion): re-verify store identity on keep-warm resume, re-run joined deletes after a mid-cascade restore

Three hardening refinements on the deletion/persistence fencing:

- loadFromStorage's parked keep-warm branch now mirrors the success
  path's store-identity guard: if a tokenless writer (e.g. a database
  import) replaced the stage during the park, a failed deletion falls
  through to the cold load instead of reporting another classroom as
  the kept warm state. The ghost-discard block is correspondingly
  gated on the deleted flag so the fall-through does not mislabel a
  surviving document as a ghost.

- deleteStageData: a joined call carries its own deletion intent. When
  the first cascade fulfills but a same-id restore lifted the deleted
  flag in the meantime, the joined caller now runs one fresh cascade
  against the restored document instead of reporting the pre-restore
  outcome. Exactly one re-check per call (no recursion); a rejected
  first cascade still propagates unchanged to every joined caller.

- Narrowed the failed-delete restore-ownership comments: the restore
  covers only the pending map plus an in-flight flush round's dirt;
  a departing-stage snapshot is outside that capture and is fenced
  and dropped by design. Comment-only, no behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(roster): gate deleted-warm handling on stage identity, keep an explicit empty roster authoritative

Final-audit fixes, three items:

- loadFromStorage keyed its deleted-warm settlement handling (park /
  keep-warm / ghost discard) behind the scenes.length > 0 shortcut, so a
  zero-scene warm ghost of a deleted classroom skipped it entirely: the
  cold load found no document, the ghost stayed in the store, and the
  classroom loader's server-fallback gate (!getCurrentStage()) never ran
  — the silently-uneditable trap this branch exists to close, in its
  scene-less form. Deletion handling now keys on stage identity alone,
  before any scene-count shortcut; the non-deleted warm skip still
  requires scenes.

- rosterNeedsLegacyFallback collapsed "roster field absent" (document
  predates roster persistence) with "explicitly persisted empty roster".
  A future writer persisting [] on a device whose read-only mirror still
  held stale rows would have its emptied roster resurrected on every
  load. The full-lift branch now triggers only when the field is absent
  (undefined); an explicit [] is authoritative and never probes the
  mirror. No current writer produces [], but the read side no longer
  depends on that invariant.

- The GeneratedAgentConfig docstring claimed the voice fields are not a
  breaking change; that only holds for this codebase's tolerant
  structural validators. The generated stage.schema.json sets
  additionalProperties: false, so consumers pinning an older published
  schema artifact reject documents carrying the new fields. The
  docstring now states both sides honestly.

Tests: a zero-scene deleted warm ghost is restored through the real
loadFromStorage (fails pre-fix), an explicit empty roster does not
resurrect stale mirror rows (fails pre-fix), and the
rosterNeedsLegacyFallback unit matrix covers undefined vs [].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(deletion): re-queue aggregate persistence when a failed deletion is recovered

Direct aggregate saves (saveToStorage: generation completion, server-restore
hydration) run outside the pending map and the flush round, so the deletion
path's pre-delete dirt snapshot cannot describe them. When a deletion bumped
the epoch while such a save was in flight, the save was correctly fenced
('stale-dropped') — but if the deletion then failed before removing the
document, the recovery only re-queued the snapshotted scheduler descriptors.
The fenced aggregate-only content stayed memory-only with no retry anywhere,
and a later reload lost it.

The failed-deletion restore now merges a full re-mark of the aggregate
(structure / stage / outline / currentScene / chats plus a descriptor for
every current scene) into the recovery set, still guarded on the store
holding the stage. Because the flush recaptures the CURRENT store state under
the CURRENT epoch, this necessarily carries whatever the fenced aggregate
save held — and, for the same reason, edits refused by the scheduler during
the deletion window, which previously stayed memory-only as well; the scope
docstrings now say so instead of declaring them excluded.

New integration coverage drives the real store scheduler and the real storage
layer end to end: a genuinely fenced in-flight saveToStorage (generation
completion) plus a pre-removal deletion failure ends with the completion flag
durable on the next flush; a deletion-window scene edit survives the same
way; and a successful deletion re-marks nothing (control). Both recovery
tests fail against the previous restore logic.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
2026-07-27 14:39:19 +08:00

86 lines
3.3 KiB
TypeScript

/**
* Provider-neutral per-agent voice design.
*
* A `VoiceDesign` describes an agent's vocal identity (not personality) as a
* 3-layer recipe. It is consumed by any TTS provider: as an inline voice
* prompt where supported, or as the seed for a registered/cloned voice
* (see `voice-registration.ts`). Nothing here is VoxCPM-specific.
*
* The type itself lives in `@openmaic/dsl` (it is part of the persisted
* `GeneratedAgentConfig` contract, so the roster's voice travels with the
* stage document); this module re-exports it and owns the runtime helpers.
*/
import type { VoiceDesign } from '@openmaic/dsl';
export type { VoiceDesign } from '@openmaic/dsl';
const VOICE_DESIGN_PROMPT_MAX_CHARS = 200;
/** Prefix for deterministic auto-voice ids (provider-neutral, backend-name-safe). */
export const AUTO_VOICE_ID_PREFIX = 'auto-' as const;
function sanitizeVoiceDesignPart(value?: string): string {
return (
(value || '')
.replace(/[\p{C}]+/gu, ' ')
// Strip parentheses: VoxCPM uses `(prompt)text` delimiters, so a paren in the
// descriptor/persona would corrupt the bootstrap synthesis prompt.
.replace(/[()()]/gu, ' ')
.replace(/\s+/gu, ' ')
.trim()
.slice(0, VOICE_DESIGN_PROMPT_MAX_CHARS)
.trim()
);
}
/** Compose the 3 layers into one comma-joined prompt, dropping blank layers. */
export function buildVoiceDesignPrompt(design: VoiceDesign): string {
return [design.identity, design.texture, design.delivery]
.map((part) => sanitizeVoiceDesignPart(part))
.filter(Boolean)
.join(', ');
}
/** Coerce an arbitrary (LLM-produced) value into a VoiceDesign, or undefined. */
export function normalizeVoiceDesign(raw: unknown): VoiceDesign | undefined {
if (!raw || typeof raw !== 'object') return undefined;
const record = raw as Record<string, unknown>;
const pick = (value: unknown) => (typeof value === 'string' ? value.trim() : '');
const design = {
identity: pick(record.identity),
texture: pick(record.texture),
delivery: pick(record.delivery),
};
if (!design.identity && !design.texture && !design.delivery) return undefined;
return design;
}
/**
* Deterministic voice id derived from the descriptor (+ provider + model).
* Stable across re-synthesis, recomputable anywhere from the descriptor on the
* agent, and namespaced by provider so a shared registry can't collide.
*
* Note: language is intentionally NOT part of the id — the descriptor text is
* already written in the course language, and language only selects the one-time
* bootstrap sample sentence (which affects neither output language nor timbre).
* Keeping it out means every TTS path (narration passes a directive, discussion
* passes a locale) resolves to the SAME id for the same agent.
*/
export async function getDeterministicVoiceId(
design: VoiceDesign,
opts: { providerId?: string; model?: string } = {},
): Promise<string> {
const seed = [
opts.providerId || '',
design.identity,
design.texture,
design.delivery,
opts.model || '',
].join('|');
const digest = await crypto.subtle.digest('SHA-256', new TextEncoder().encode(seed));
const hex = Array.from(new Uint8Array(digest))
.map((byte) => byte.toString(16).padStart(2, '0'))
.join('');
return `${AUTO_VOICE_ID_PREFIX}${hex.slice(0, 16)}`;
}