mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-16 05:19:50 +08:00
@openmaic/generation@0.3.6
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 wyuc
|
e3fd494658 |
feat(tts): add Qwen TTS voice cloning (#1160)
* feat(tts): add Qwen TTS voice cloning via the registration adapter * feat(settings): add Qwen TTS clone-voice manager * fix(tts): harden Qwen voice cloning integration * fix(tts): enforce voice-bound Qwen synthesis * fix(tts): close Qwen voice clone tail issues * fix(tts): close final voice clone review issues * fix(agents): harden voice token parsing * fix(settings): hide the voice-clone model from manual selection * fix(agents): pin the narrator voice to the user's selected voice * fix(agents): keep narrator fallback alive when the pinned voice is unusable * fix(i18n): correct the narration fallback notice copy * fix(tts): bound the narrator voice retry to a single fallback hop * fix(tts): tolerate transient vendor errors in the voice existence precheck |
||
|
wyuc
|
e83db6a531 |
fix(agents): persist AgentBar voice picks and mode/selection across reloads (#723)
* feat(audio): let resolveAgentVoice honor per-agent voice overrides * feat(settings): persist per-agent voice overrides * fix(agents): persist AgentBar voice picks via settings overrides * fix(classroom): keep user's agent mode and selection across loads * fix(agents): only carry user-set agent selection across classrooms Review follow-ups: stage-derived defaults written by a classroom load were re-read as a user choice on the next load, so visiting a preset classroom permanently downgraded auto classrooms to preset agents. Track the origin with a persisted agentSelectionIsUserSet flag (set by AgentBar interactions, cleared by fallback writes). Also: voice overrides are now clearable (undefined deletes), the override type is single-sourced in voice-resolver, and persist rehydration of the new fields is pinned by tests. * fix(agents): don't mark stage-derived selections as user choices Round-2 review: clicking the already-active mode tab no longer flags the selection as user-set (a visual no-op must not promote stage defaults to a choice), generation-preview clears the flag when it writes the generated roster, and tests now pin that setter writes actually reach storage. * fix(agents): restore generated agent selection when switching back to Auto Review follow-up (#723): handleModeChange('auto') left the preset ids selected, so the UI said Auto while discussion playback kept using preset agents, and the persisted user-set auto pair could never validate on restore. Switching to Auto now selects the loaded classroom's generated agents (empty when none are loaded, falling back to stage defaults on the next classroom load). |
||
|
|
d7c0d80afe |
feat(tts): per-agent auto-voice quality + register-once timbre stability (#670) (#672)
* feat(tts): voxcpm voice-design types + deterministic voice id helpers (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): emit + persist per-agent voiceDesign descriptor (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): voxcpm voice registration backend client (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): voxcpm-voice ensure/register endpoint (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): client auto-voice registration + reference-clip cache (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): reference registered voice id in vLLM-Omni speech (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): thread voiceDesign + backend through tts call sites (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): match vLLM-Omni voice registration contract (#670) Live e2e against the real backend revealed the assumed multipart contract was wrong: POST /v1/audio/voices needs name + consent + audio_sample (not voice_id/file), and there is no per-name GET (405) — existence must list /v1/audio/voices and check membership. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(tts): make auto-voice register-once pattern provider-neutral (#670) The descriptor + register-once/reference-by-id pattern is not VoxCPM-specific. De-couple it into a provider-neutral seam so other registration-capable providers (ElevenLabs/MiniMax/Doubao voice cloning, …) can plug in: - lib/audio/voice-design.ts: VoiceDesign + buildVoiceDesignPrompt / normalizeVoiceDesign / getDeterministicVoiceId (neutral 'auto-<hash>' id, namespaced by providerId). - lib/audio/voice-registration.ts: VoiceRegistrationAdapter interface + providerId->adapter registry + supportsVoiceRegistration. - lib/audio/voice-registration-client.ts: neutral ensureRegisteredVoice() + IndexedDB clip cache. - app/api/generate/voice (replaces .../voxcpm-voice): dispatches by providerId. - voxcpm-registration.ts becomes the VoxCPM adapter (sole registered provider). - AgentConfig.voiceDesign + DB cache table renamed neutral. Behavior-preserving; VoxCPM-specific bits (inline (prompt)text, backend kinds, capability gate) stay in the voxcpm modules. Full suite green (674). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): build the voice clip Blob from an inline Uint8Array (#670) A factored-out helper returning Uint8Array widened to Uint8Array<ArrayBufferLike>, which next build (stricter than bare tsc) rejects as a BlobPart. Inline the buffer like the rest of the repo does. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): managed TTS providers resolve model server-side (#670) A server-managed TTS provider's model was still client-driven, but the managed-provider settings UI hides the model field — so a backend whose model id isn't the client default (e.g. VoxCPM/vLLM-Omni serving the full model path) always 500'd. resolveTTSModel() makes the model authoritative from server config (${PREFIX}_MODELS, first entry) for managed providers, like key/baseUrl; unmanaged/unconfigured providers keep the client model unchanged. Used by the tts and voice routes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): single agent-voice resolver + stable teacher narration voice (#670) Route all TTS voice resolution (narration, discussion, preview) through one resolveAgentVoiceOptions(agent, ...) that reads the agent profile and registers + references the voice by id. Eagerly warm up generated agents' voices on save. Critically fixes the teacher narration drifting (male/female jumps): the registry is always seeded with DEFAULT_AGENTS, so the old narration lookup find(role==='teacher') returned the default teacher (no voiceDesign) instead of the generated one — so narration never registered a voice and fell back to the inline prompt. pickNarratorAgent() now prefers the teacher carrying a voiceDesign. Also: drop language from the deterministic voice id (descriptor already encodes it) so narration (directive) and discussion (locale) resolve the same id; log the effective registeredVoiceId. Regression tests for pickNarratorAgent. Verified e2e: agent-profiles -> 1 voice registration -> 16 narration TTS all referencing the same registeredVoiceId; voice present on the backend. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): fall back to persona as the voice seed for agents without a voiceDesign (#670) Preset/default agents carry no LLM voiceDesign. Rather than add a static field, derive the bootstrap descriptor from the agent's persona when voiceDesign is absent, so they still register one stable reference voice (stable-but-generic; persona is not a vocal spec). Generated agents keep their LLM voiceDesign. Also hardens replay: stage snapshots drop voiceDesign but keep persona. Unit-tested: resolveAgentVoiceOptions uses real voiceDesign when present, persona otherwise. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): match Auto Voice by its localized label in the agent voice picker (#670) The picker filtered on voice.name ('Auto Voice'), but Auto Voice is shown via its localized label (自动音色). Searching '自动' returned '没有匹配音色'. Match the localized label for the Auto Voice option so it's findable in any language. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): address code-review round 1 (#670) - strip parentheses in the voice-design prompt so a paren in the descriptor/ persona can't corrupt the (prompt)text bootstrap delimiter - gate POST /api/generate/voice on isServerTTSProviderDisabled (#665): a force-disabled provider was off for the tts route but not this sibling - check voiceExists before re-registering a client-cached clip, so a cached voice that's still live isn't needlessly re-uploaded every session - dedup concurrent ensureRegisteredVoice calls via an in-flight promise map (eager warm-up + first utterance no longer double bootstrap/register) - warm up only the narrator (teacher), not every generated agent, to avoid synthesizing voices at save time for agents that may never speak - collapse the duplicated vLLM-Omni speech payload into one object Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(tts): prettier formatting (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): key auto-voice memo by (voiceId, backend) so a base-URL change re-registers (#670) Addresses review (cosarah): registeredThisSession/inFlight were keyed by voiceId alone, so switching the VoxCPM base URL mid-session made ensureRegisteredVoice short-circuit and return an id registered only on the old backend — TTS then sent that stale registeredVoiceId and skipped the inline fallback, failing with voice-not-found on the new backend. Memo key now includes the base URL; the IndexedDB clip cache stays keyed by voiceId (the reference clip is backend-independent and reused to re-register elsewhere). Regression test added. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): include API key in the auto-voice memo key (#670) Addresses review (cosarah, round 2): memoKeyFor was keyed by (voiceId, baseUrl) but not the API key. Registration/existence checks and speech calls are auth-scoped, so switching account/key on the same base URL could reuse a registeredVoiceId from the old credentials and skip re-validation. Memo key now includes the API key (in-memory only, never persisted/logged). Regression test covers the key-switch case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
|
3e1e9f083b |
feat(tts): unify provider-enablement model, default browser-native off (#665) (#666)
A single TTS provider-enablement model honored everywhere (picker,
auto-assignment, fallback, lecture/playback, server).
Model
- lib/audio/provider-enablement.ts: isTTSProviderConfigured /
isTTSProviderEnabled / listEnabledTTSProviderIds. Availability drops the
registry defaultBaseUrl fallback (fixes Lemonade's always-shown voices),
honors the per-provider `enabled` flag, server-disable overrides (server
precedence).
- voice-resolver: getEnabledProvidersWithVoices (enabled server/custom
providers) and getSelectableProvidersWithVoices (the ONE client source of
truth = enabled providers + opt-in browser-native when the browser exposes
voices). resolveAgentVoice picks deterministically among the given providers,
honors a voiceConfig only while its provider is present.
- Picker + discussion build their list from getSelectableProvidersWithVoices, so
the AgentBar and the live discussion agree on selectable voices (fixes
student agents going silent in auto mode). Students resolve via
resolveAgentVoice; the teacher uses the global lecture voice verbatim (same
voice the pre-generated lecture sends), so lecture and discussion never
diverge.
- Voice assignment vs fallback: an agent's voice is bound at generation (the
agent-profile LLM picks from availableVoices; preset uses the user's pick).
The fallback for a missing/invalid/unavailable voice is applied only at the
live TTS calls, not baked into the saved agent — so it reflects current
provider availability.
Defaults / opt-in
- Browser-native: first-class, default OFF, opt-in. Built-in providers default
enabled:true (only surface once configured).
- TTS master toggle defaults OFF; first server-sync auto-enables it when a
provider is configured (mirrors image/video).
- Persist v3->v4 migration normalizes the never-user-editable flags.
Server/admin disable (server precedence): server-providers.yml `enabled:false`
and env `TTS_<P>_ENABLED=false` (incl. browser-native). Surfaced as a per-provider
`disabled` flag on /api/server-providers; merged as serverDisabled; enforced at
the /api/generate/tts trust boundary, classroom media generation, and /api/health.
UI
- Settings TTS panel: per-provider enable toggle, disabled (with a hint) when the
provider is unavailable, locked off (with a note) when server-disabled.
- AgentBar voice pills render disabled with a "No voice" placeholder when no
provider is enabled. The toolbar media-popover TTS tab keeps its existing
on/off toggle (no picker/CTA).
Tests: predicates, resolver determinism + unified selectable list, server
force-disable, store merge + migration + TTS auto-enable, i18n parity. Verified
in a real browser (Playwright): settings toggle states + a full Gemini classroom
generation whose TTS requests used the picked voice.
Found on main @
|
||
|
|
35d36905d1 |
Make server-configured providers admin-managed, not client-overridable (#620) (#624)
* refactor(providers): make server-configured providers admin-managed, not client-overridable A server-configured provider is now authoritative: the operator owns its API key and base URL, and the client cannot override either. This collapses the tri-state (server key / client base URL / client key) that was the root of a recurring "missing API key" bug class — when the settings UI echoed a base URL with an empty key, routes that treated "any base URL ⇒ fully client-configured" dropped the server key. #533 patched that per-route via canUseServerApiKeyForBaseUrl; this removes the guard entirely by fixing the design. Server: - resolve*ApiKey/BaseUrl ignore client values when the provider is server- configured; add isServerConfiguredProvider(section, id); drop canUseServerApiKeyForBaseUrl + normalizeProviderBaseUrl. - getServer*Providers() no longer return base URLs — only allowed models (LLM/image) and the managed flag (presence) — so an internal gateway/proxy endpoint is never exposed to the client. - routes (chat/image/video/tts/asr/pdf + verify-*) resolve managed providers purely server-side; SSRF validation now applies only to unmanaged (client) base URLs. Client: - drop serverBaseUrl from ProviderSettings, the settings store, request-building (TTS base URL no longer echoes the server URL) and voice resolution. - settings UI renders managed providers read-only: the key/base-URL override inputs and region toggles are hidden, leaving the "managed by server" notice. Tradeoff: overriding a server provider's key with your own for the same provider is removed; add a separate custom provider instead. Closes #620. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers): refresh managed-provider copy, drop dead audio-settings, add #620 e2e - serverConfiguredNotice now says the provider is managed and not editable here, pointing users to add a separate provider for their own credentials (7 locales). - delete components/settings/audio-settings.tsx — unreferenced dead code. - add e2e/tests/managed-provider-620.spec.ts: managed provider hides the key/base-URL inputs; unmanaged provider keeps them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(providers): lock the model catalog for managed providers with a pinned list When the operator pins an allowed model list (MODELS env/yaml ⇒ serverModels), the model catalog is admin-managed too: the LLM provider panel shows the pinned models read-only with a "Server" badge and no add / reset / edit / delete affordances. Without a pinned list the server manages only credentials and the user keeps curating models. This removes the prior inconsistency where the settings panel let you add models to a pinned managed provider that the generation toolbar then filtered out. e2e: the managed-openai case now also asserts the catalog is locked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers): ignore (not reject) client overrides for managed web-search; clear legacy serverBaseUrl Addresses review feedback on #624: 1. Managed web-search providers now ignore a client-sent key/baseUrl instead of validating and rejecting it. A server-configured provider is admin-owned, so the server config is authoritative and a stale client base URL is dropped rather than 400-ing the request. The allowlist validation still runs for unmanaged providers, where the client base URL is actually used. 2. Drop the removed `serverBaseUrl` field from persisted localStorage on rehydrate (settings version 2 -> 3, stripped in both migrate and merge), so old clients don't keep a stale server URL in client state. Tests: web-search route test split into unmanaged-rejects-allowlist and managed-ignores-client-baseUrl; settings rehydrate test asserts a legacy serverBaseUrl is stripped. Full suite 381 pass; e2e 2 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
wyuc
|
f064590a5e |
test+fix(orchestration): rip maxTurns; restore answer-the-user's-question behavior (#598 follow-up) (#599)
* test(orchestration): add premature-END regression eval (#598 follow-up) Adds eval/orchestration/ following the outline-language pattern: a runner that A/B-tests the director against the same scenario with two prompt variants — current main (post-#554) and a synthesised pre-#554 baseline that strips the role-aware summary labels AND the new system.md rules 10/11/12 together. Pass criterion is now framed as a regression guard rather than a fixture discrimination test: every scenario's post-fix END rate must stay below EVAL_END_THRESHOLD (default 20%). The pre-vs-post Δ is reported as informational data, since #554's reviewer feedback was that earlier fixtures didn't discriminate. Empirical finding across 7 shipped model configs (gpt-4.1-mini, gpt-4o-mini, gpt-5.4-nano, qwen-plus, qwen3.5-flash, deepseek-chat, deepseek-v4-flash, gemini-2.5-flash, claude-haiku-4-5) with 5 scenarios modelled on #511 (incl. the exact Tiananmen-3D objection trace, soft pushback after a long resolved-looking discussion, topic pivot, brief acknowledgement, and explicit teacher-signals-end-then-user-objects): no scenario produced a non-zero END rate in either variant. Useful data for the #554/#598 discussion — the prompt-layer rules don't measurably change behavior on these shipped models with these prompts, but the eval is now wired so future regressions show up. * test(orchestration): add director question-answering eval (#598 follow-up) Adds an A/B eval probing whether the director routes correctly when the conversation contains an unanswered user question — both in the first-turn drift case (agents started answering but drifted onto adjacent topics, no frustration yet) and in the escalated frustration case (user has already complained multiple times). The prod symptom this targets ("答非所问"): agents reply on adjacent topics rather than answering the literal question, and the director keeps picking peer agents for "variety" instead of routing back to the teacher to actually answer. Scenarios: 5 synthesized cases across math, biology, English grammar, physics, and calculus. 3 are first-turn drift (no frustration yet), 2 include frustration signals. Each captures: user asks specific question, agents reply on adjacent topics, director is asked to pick the next move. Decision rule-judged into USER | TEACHER | OTHER_AGENT | END: - USER → ✓ valid (cue user to clarify) - TEACHER → ✓ valid (re-route to teacher to answer) - OTHER_AGENT → ✗ wrong (peer-agent "variety" routing — the bug) - END → ✗ wrong A/B: - baseline : current director template with rule 13 stripped - with_rule : current director template as-shipped Pass = with_rule.correctRate ≥ EVAL_PASS_THRESHOLD (default 0.7). Pre-vs-post Δ is reported as informational only. Note this commit only adds the eval; the rule it strips/restores in the A/B is injected in the next commit. * fix(orchestration): inject director rule for answering user questions Adds rule 13 to lib/prompts/templates/director/system.md so the director recognizes when the conversation contains an unanswered user question and routes appropriately — to the teacher (literal role match) by default, or to USER cue when the question is ambiguous or the user has already complained multiple times. This addresses the "答非所问" symptom: when agents drift onto adjacent topics, the director keeps picking peer agents for "variety" rather than routing back to the teacher to actually answer. The rule is preventive (applies whenever the question is unanswered, including the first turn after drift) rather than reactive-only (waiting for the user to express frustration). Rule wording uses constrained-output framing: literal `role: teacher` match (not "best fit" / "highest priority" / "most knowledgeable"), and an explicit list of forbidden choices (peer agents, END). Earlier softer wordings let the model pick the assistant when it seemed more topically relevant; the literal-match wording prevents that. Validated by the question-answering eval added in the previous commit. On google:gemini-3-flash-preview, 5 samples per variant across 5 synthesized scenarios: | Scenario | baseline | with_rule | |----------------------------------------------------|----------|-----------| | math_quadratic_axis_drift_no_frustration | 100% | 100% | | bio_dark_reaction_drift_no_frustration | 80% | 100% | | english_team_isare_drift_no_frustration | 20% | 100% | | physics_inertial_mass_escalated_frustration | 0% | 100% | | calculus_product_rule_drift_no_frustration | 60% | 100% | | **mean** | **52%** | **100%** | 3 of 5 scenarios reproduce the bug at baseline (correct rate ≤ 60%); the rule lifts every scenario to 100%. The remaining 2 baselines are already high — the rule doesn't regress them. * fix(orchestration): teach agents to answer the user's literal question Adds an "Answering the User's Question" section to the agent system prompt. Before this, the agent SP had no rule for direct-answering — the persona, role guideline, current-state context, and length guidelines all pushed the agent to advance the lesson and inspire thought, with no explicit directive to first answer the user's literal question. The result observed in prod ("答非所问"): when the user asks a specific question (a formula, a yes/no, a term), agents reply with adjacent concepts, examples, or follow-up questions — but never the literal answer. The new section directs the agent, on every response where the user message contains a question, to: - lead with the concrete answer to the literal question, - not pivot to an adjacent topic even when it seems pedagogically richer, - treat "inspire thought" and peer-differentiation guidance as applying only AFTER the literal question is answered, - say "I don't know" directly when uncertain rather than answering a different question, - on frustration signals from prior turns, look back at the original question (before the frustration) and answer THAT specifically. This is the agent-side counterpart to the director rule landed in the previous commit. Director routes to the teacher when the question is unanswered; the teacher now has explicit guidance for what to do when dispatched. * refactor+fix(orchestration): rip maxTurns end-to-end; distinguish errors from completion Two coupled cleanups in one commit since the outcome-handler rewrite naturally falls out of removing the client-side max-turns loop. The director graph used `maxTurns: turnCount + 1` to force "one director→agent cycle per request" — a hack expressing the single-round contract as a turn limit. This commit removes the hack and the surrounding user-facing setting, replacing implicit intent with explicit graph topology. Server: - `director-graph.ts`: drop `maxTurns` from state, drop the hard-cap check at the top of `directorNode`, change `agent_generate → director` edge to `agent_generate → END`. Single-round contract is now expressed by topology. - `lib/types/chat.ts`: drop `maxTurns` and `currentTurn` from `SessionConfig`; drop `maxTurns?` from `CreateSessionRequest.trigger`. Client: - `lib/store/settings.ts`: drop `maxTurns` state, `setMaxTurns` action, the `'10'` default, and the localStorage migration path. - `components/agent/agent-bar.tsx`: drop the entire compact-stepper UI block plus the unused MessageSquare/Minus/Plus lucide imports. - `components/settings/agent-settings.tsx`: drop the AgentSettings prop and the multi-agent-only Input block. - `lib/chat/agent-loop.ts`: drop `maxTurns` parameter, change `while (turnCount < maxTurns)` to `while (true)` controlled by inner exits, drop `'max_turns'` reason from `AgentLoopOutcome`. - `components/chat/use-chat-sessions.ts`: drop the maxTurns lookup, drop the trailing arg to runAgentLoop, drop vestigial `maxTurns: 0` / `currentTurn: 0` from session creation. - `eval/whiteboard-layout/runner.ts`: drop unused MAX_AGENT_TURNS plus the trailing arg. i18n: drop `settings.maxTurns` and `settings.maxTurnsDesc` from six locales (en-US, zh-CN, zh-TW, ja-JP, ar-SA, ru-RU). Motivation for removing the user-facing setting: the LLM director controls round length via cue_user / END decisions, so a numeric "max discussion turns" knob is redundant with rule-driven routing. When the director prompt was buggy, the cap masked the bug by force- ending; once director routing is fixed, the cap just removes user agency. Before, every non-`cue_user` outcome of the agent loop mapped to `status: 'completed'` in `use-chat-sessions.ts`. That conflated four distinct paths: - `end`: director chose END (legitimate completion) - `empty_turns`: two consecutive agents returned no text or actions (parser failure, model timeout, etc.) - `no_done`: SSE stream ended without a `done` event (infrastructure failure) - `aborted`: user pressed stop (handled elsewhere) The new switch routes `end` to completed, `empty_turns` and `no_done` through `clearLiveSessionAfterError` so the user sees an inline System message describing what went wrong instead of a quiet "completed". - `lib/types/chat.ts`: add `'error'` to `SessionStatus`. - `components/chat/use-chat-sessions.ts`: rewrite the outcome handler as a switch; have `clearLiveSessionAfterError` set `status: 'error'` so all entry points (the runAgentLoop outcome AND existing throw paths from fetchChat / SSE error events) converge on the same visible state. - `components/chat/session-list.tsx`: add `AlertCircle` red icon for the new error status. - i18n: add `chat.error.emptyAgentResponses` and `chat.error.streamInterrupted` strings to all six locales. This is the in-product fix that the prod investigation surfaced as an independent root cause of the #511 "discussion ended unexpectedly" symptom — separate from the director and agent prompt fixes in the earlier commits of this PR. * fix(chat): drop dead settingsState binding; wire eval:orchestration:answering script Two minor CR follow-ups: - components/chat/use-chat-sessions.ts: the `settingsState` lookup at the top of runAgentLoopFn was the entry point for the maxTurns lookup ripped in the previous commit; with maxTurns gone, the binding is unused and ESLint flags it. Drop the line. (Other uses of useSettingsStore.getState() elsewhere in this file are unaffected.) - package.json: the answering eval runner had no convenience script, while the premature-end runner did. Add eval:orchestration:answering for symmetry and discoverability. * fix(orchestration): route ambiguous/unaddressed user turns to teacher, never USER cue Addresses reviewer issues 2 & 3 (ambiguous "帮我看一下这个" / "Can you take a look at this?"): the director was cueing USER for vague input, which makes no agent speak — the user faces dead air with nothing to react to ("nobody responds"); and when an agent did speak it guessed a topic instead of asking what the user meant. Root cause was the previous rule 13, which offered `{"next_agent":"USER"}` as a valid response to an ambiguous user turn. A USER cue is the wrong tool here: it returns control to the user without any agent speaking. The right behavior is to dispatch the teacher, who either answers or — if the message is too vague — asks a clarifying question out loud. Director (`director/system.md` rule 13): - When the most recent user turn is unaddressed (clear question, vague request, OR frustration signal), the only acceptable output is the literal `role: teacher` agent id. - USER cue is now explicitly forbidden as a response to a user turn (reserved for rule 7, where an agent has just asked the user something). - Tightened the END prohibition: a high turn count or a thoroughly-discussed topic does NOT mean the specific question was answered. Added rule 5 (don't-drag-on) to the override list, since that pressure was driving the director to END on well-discussed-but-unanswered turns. Agent (`agent-system/system.md`): - Added a clause to "Answering the User's Question": if the user's message is too vague to answer (no clear referent), ask ONE short clarifying question with a concrete option or two — do not guess a topic and lecture, do not stay silent. Eval (`answering-runner.ts` + `scenarios/answering.json`): - Reclassified USER as wrong (dead air); only TEACHER counts as correct now (teacher answers, or asks a clarifying question for vague input). - Added two ambiguity scenarios: a bare "帮我看一下这个" opener, and a vague "再帮我看看这个吧" follow-up after a real Q&A. - Made the rule-13 strip regex match by rule number rather than heading text, so it survives heading rewrites. On google:gemini-3-flash-preview, 10 samples per variant, all 7 scenarios PASS (with_rule mean ~97%, baseline ~57%). The two ambiguity scenarios: ambiguous_look_at_this 100%/100%, ambiguous_vague_followup 20%→100%. Manually verified the agent side: for "帮我看一下这个" the teacher now replies "…你想讨论的是具体的代码逻辑，还是 Agent 的决策流程图呢?" (clarifying question + concrete options) instead of guessing. Note: reviewer issue 1 (follow-up not answered) could not be reproduced on this branch across 7 configurations; it is consistent with the deployed environment running older code and/or provider API flakiness, which the C7 error-vs-completion distinction in this PR already surfaces instead of a silent 已结束. * docs(eval): fix stale physics scenario description after USER→wrong reclassification The physics_inertial_mass scenario description still said "USER-cue signal — but TEACHER re-answer is also valid", which contradicts the new classifier (USER is dead air = wrong; only TEACHER is correct). Cosmetic only — the description field is printed into the report, never used in classification. |
||
|
wyuc
|
5e12ad892c |
[codex] add VoxCPM2 TTS provider (#496)
* add voxcpm2 tts provider * address voxcpm review feedback * translate voxcpm locale strings * narrow toolbar model pill * refine toolbar media controls * address voxcpm review followups |
||
|
杨慎andClaude Opus 4.6
|
470aa5e42e |
feat: inline language inference for outline and PBL generation (#412)
* feat: inline language inference in outline generation Replace manual language selection with automatic inference from user requirement text. The outline generator now produces a languageDirective that propagates through the entire generation pipeline. Key changes: - Rewrite Language Inference section in outline prompt with decision rules for foreign language learning, cross-language PDF, proxy requests, and terminology handling - Reorder pipeline: outlines before agents, so agent profiles can use the inferred languageDirective - Pass languageDirective through scene content/actions generation - Add SSE streaming of languageDirective from outline generation - Remove manual language selection UI Eval test suite: - 50 test cases covering 9 scenario types: single-language, language learning, immersive, explicit instruction, code-switching, minimal input, user profiles, cross-language PDF, locale mismatch - LLM-as-judge evaluation with configurable inference/judge models - 50/50 pass rate with gemini-3-flash-preview + gpt-4o judge - Excluded from CI (requires LLM API keys) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove useless attributes in test case * feat: pass languageDirective to PBL, remove hardcoded zh-CN/en-US - Remove pblConfig.language field, use languageDirective from outline inference instead - Delete all Chinese prompt duplicates in pbl-system-prompt.ts, agent-templates.ts; keep English templates with languageDirective injection - Remove zh-CN/en-US branches in generate-pbl.ts (initial prompt, post-process context, welcome message) - Pipe languageDirective through scene-generator → generatePBLContent → IssueboardMCP → agent templates - Remove i18n template wrapping from PBL welcome messages in pbl-renderer.tsx and use-pbl-chat.ts; use LLM-generated content directly to avoid language mismatch when UI locale ≠ course language - Update pblConfig schema in outline prompt (remove language field) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: clean up legacy language fields, pass languageDirective through buildSceneFromOutline - Add languageDirective param to buildSceneFromOutline, combining with outline.languageNote via buildLanguageText - Remove dead types: AudienceProfile, StylePreferences, LegacyUserRequirements, SceneOutline.language - Remove Stage.language and all its propagation (storage, stageInfo, prompt-builder fallback) - Remove GenerateClassroomInput.language and job store inputSummary.language - Remove unused pdfLanguageSample from outline SSE route Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: persist languageDirective to IndexedDB, fix TTS preview and dead refs - Add languageDirective to StageRecord and stage-storage write path - Add DB v9 migration: convert legacy language locale codes to directives - Use voice.language for TTS preview text instead of dead localStorage key - Omit empty ## Language section in PBL agent prompts - Use JSON.parse for extractLanguageDirective unescape (\uXXXX support) - Remove dead i18n keys: toolbar.languageHint, pbl.chat.welcomeMessage Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.6
|
9a0060e2f0 |
feat(audio): add custom OpenAI-compatible TTS/ASR provider support (#357) (#409)
* feat(audio): extend TTSProviderId/ASRProviderId to support custom providers * feat(audio): update provider registry helpers for custom provider support * feat(audio): route custom providers to OpenAI-compatible implementations Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(audio): add custom TTS/ASR provider CRUD to settings store * feat(audio): add dialog for creating custom audio providers * feat(audio): add custom provider UI with voice management and delete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(i18n): add custom audio provider translation strings * fix(audio): resolve remaining type errors for custom provider support Add `as keyof typeof` casts where TTSProviderId/ASRProviderId (which now include template literal types) index into the built-in-only registries TTS_PROVIDERS, ASR_PROVIDERS, DEFAULT_TTS_VOICES, and DEFAULT_TTS_MODELS. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(audio): allow custom provider IDs to reach switch default branch The early guard `if (!provider) throw` blocked custom provider IDs from reaching the switch-case default branch where they get routed to OpenAI-compatible implementations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(audio): add default model field to dialog and polish voice list UI - Add "Default Model" input to custom audio provider dialog (fixes Kokoro needing model:"kokoro" instead of fallback "gpt-4o-mini-tts") - Redesign voice list as aligned table with column headers, hover reveal delete buttons, and empty state message - Add Enter key support to voice add inputs - Use consistent grid layout for add voice row Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl when testing custom TTS provider The baseUrl field in provider config is empty by default (it's an override field). Custom providers store their default URL in customDefaultBaseUrl, which needs to be used as fallback for test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): auto-select first voice when adding to custom TTS provider When a custom provider's voice is 'default' (initial state before any voices are added), adding the first voice now automatically updates the active voice selection so test TTS works immediately. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl in all TTS call sites Scene generator, discussion TTS, and agent bar voice preview all read providerConfig.baseUrl which is empty for custom providers (the default URL is stored in customDefaultBaseUrl). Add fallback chain to all 4 call sites so custom providers work during actual playback, not just in the settings test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): include custom provider voices in agent voice picker getAvailableProvidersWithVoices only iterated built-in TTS_PROVIDERS, so custom providers never appeared in the agent bar voice dropdown. Now also iterates ttsProvidersConfig for custom-tts-* entries and includes their customVoices in the picker. Also updated getServerVoiceList, findVoiceDisplayName, and resolveAgentVoice to handle custom provider voice lookups. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl in all ASR call sites Same issue as TTS: audio-recorder, asr-settings, and audio-settings all read providerConfig.baseUrl which is empty for custom providers. Add customDefaultBaseUrl fallback to all 3 ASR call sites. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(audio): add model list management for custom ASR providers Custom ASR providers now support adding/removing models with a table UI (same design as TTS voice list). The model selector dropdown shows custom models and auto-selects the first one. Also adds custom ASR providers to the Toolbar's ASR provider dropdown and fixes custom provider handling in audio-settings.tsx (baseUrl fallback, request URL preview, API key section visibility). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): seed customModels from defaultModel and fix ASR model label - When creating custom ASR provider with a defaultModel, auto-add it to customModels so the model list isn't empty on first visit - Fix ASR settings label from "TTS Model" to "Default Model" Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): separate ASR model management from creation dialog ASR models should be managed in provider settings (add/remove from list, select default), not pre-filled at creation time. Unlike TTS where a single default model suffices, ASR users need to manage multiple models (e.g. whisper-small, whisper-large-v3). - Remove Default Model field from ASR creation dialog (keep for TTS) - Remove defaultModel parameter from addCustomASRProvider - ASR models are added via the model list UI in provider settings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): simplify ASR model management — first model is default Remove the separate model selector dropdown for custom ASR providers. The first model in the list is always used as default. Users manage the model list only (add/remove), no separate "default" selection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): add loading state and detailed errors to ASR test - Show spinner with "Processing..." while waiting for transcription - Disable record button during processing - Show specific error messages from API (e.g. "No transcription generated") instead of generic failure text - Handle empty transcription result as an error with helpful message Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): use correct key for processing label in ASR test button * fix(audio): polish custom provider UX and type safety - Add proper type declarations for custom provider fields (customName, customDefaultBaseUrl, customVoices, etc.) eliminating 30+ unsafe casts - Show custom ASR providers in toolbar with default language list - Hide custom ASR providers without models from toolbar selection - Add no-models warning in ASR settings panel - Add delete confirmation dialog for custom TTS/ASR providers - Add duplicate voice/model ID validation with toast error - Add dedicated i18n keys for ASR model management (modelNamePlaceholder, addModel) - Fix i18n key for ASR processing label (settings.asrProcessing) - Add radio-button model selection in custom ASR provider settings - Add visible OpenAI-compatible description in provider creation dialog - Add CUSTOM_ASR_DEFAULT_LANGUAGES constant for toolbar language options Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(test): add isCustomTTSProvider/isCustomASRProvider to audio types mock The settings-server-sync test mocked @/lib/audio/types as an empty object, but settings.ts now imports isCustomASRProvider for provider validation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): add customDefaultBaseUrl fallback in all TTS/ASR call sites When a custom provider is created, the actual URL is stored in customDefaultBaseUrl while baseUrl is initialized empty. Several call sites only checked baseUrl, causing fetch failures during course generation. Add fallback to customDefaultBaseUrl in: - generation-preview/page.tsx (course generation TTS) - tts-providers.ts (getCurrentTTSConfig) - asr-providers.ts (getCurrentASRConfig) - media-popover.tsx (TTS preview) - tts-config-popover.tsx (TTS preview) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: format generation-preview/page.tsx Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused ttsSpeedRange variable in media-popover Flagged by CodeQL code quality check. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
wyucand杨慎
|
3c687dd523 |
fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#326)
* fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#324) - Move model selection from global ttsModelId to per-provider ttsProvidersConfig[providerId].modelId - Remove supportsModelSelection flag, use models.length > 0 instead - Add defaultModelId to TTSProviderConfig and ASRProviderConfig - Add providerOptions field to TTSModelConfig for future extensibility - Fix ElevenLabs hardcoded model_id: 'eleven_multilingual_v2' - Fix MiniMax TTS using config.model instead of config.modelId - Update model lists from provider docs: - Qwen: add qwen3-tts-instruct-flash, qwen-tts - ElevenLabs: add eleven_flash_v2_5, eleven_flash_v2 - MiniMax: remove non-existent speech-2.8 series - Replace custom model CRUD UI with simple Select dropdown - Remove MiniMax-specific hardcoded model selector from audio-settings - Add migration for old ttsModelId/asrModelId/model fields * feat(tts): voice picker groups by provider:model, sets both model and voice Voice selection in agent bar now groups voices by model. Picking a voice from a model group sets both modelId and voiceId on the agent. TTS settings shows available models as display-only chips. - Add compatibleModels field to TTSVoiceInfo (OpenAI marin/cedar restricted to gpt-4o-mini-tts) - Add modelId to agent voiceConfig and ResolvedVoice - Add ModelVoiceGroup to ProviderWithVoices, build groups in getAvailableProvidersWithVoices - Update AgentVoicePill and TeacherVoicePill to iterate modelGroups - Thread modelId from voice selection through discussion TTS pipeline - Replace model Select with display-only chips in tts-settings * fix(tts): show provider name in voice picker group headers * fix(tts): restore MiniMax speech-01 models per official API docs * fix(tts): add MiniMax speech-2.8 models per official docs, default to speech-2.8-hd * feat(tts): expand MiniMax voice list with recommended voices * fix(tts): remove non-official MiniMax recommended voices, keep 10 from official list * fix: address CR findings - align defaults, fix test mocks, ASR modelId fallback * fix(tts): pass modelId in voice preview and classroom generation * fix(tts): voice picker highlights only the exact model:voice match --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
wyucandClaude Opus 4.6
|
ab95a7f99b |
ui(agent-bar): enlarge TTS voice selection pill and popover (#273)
Increase the size of voice pill buttons and popover menu items in the agent bar for better usability and tap targets. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.6
|
f22cdc78fc |
fix: prevent auto-generated agents from bleeding into preset classrooms (#248)
Three bugs combined to cause auto-generated agents from one classroom
to appear when viewing a different preset classroom:
1. agent-bar handleModeChange('preset') only prepended the teacher but
never removed stale generated agent IDs from selectedAgentIds.
→ Filter selectedAgentIds to only keep non-generated agents on switch.
2. loadGeneratedAgentsForStage returned early when no generated agents
were found, skipping registry cleanup of previously loaded agents.
→ Always clear generated agents from registry before the early return.
3. generation-preview saved raw selectedAgentIds (including stale
generated IDs) into stage.agentIds for preset classrooms.
→ Filter out isGenerated agents before persisting to stage.agentIds.
4. classroom/[id]/page.tsx trusted stage.agentIds without filtering,
allowing pre-fix IndexedDB data with mixed IDs to take effect.
→ Filter restored IDs against the registry on load.
Closes #247
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
|
ddb52241fb |
feat: add Discussion TTS with per-agent voice assignment (#211)
* feat(tts): add resolveVoice() and getServerVoiceList() utilities
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tts): add AudioIndicator equalizer bars component
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tts): add onSegmentSealed callback to StreamBuffer
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tts): add voiceOverrides field to AgentConfig and AgentTemplate
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tts): add useDiscussionTTS hook with audio queue and cleanup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): add audio state indicator to Roundtable bubble
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tts): wire onSegmentSealed callback through chat sessions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): add per-agent voice dropdown to AgentBar
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): integrate useDiscussionTTS in Stage and pass state to Roundtable
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(tts): refine voice dropdown to pill-style selector
Replace native select styling with a compact rounded-full pill
that blends into the agent row. Remove border, use muted bg,
smaller text, and a custom chevron icon.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(tts): use shadcn Select for voice dropdown, link with TTS toggle
- Replace native <select> with shadcn Select component for consistent UI
- Hide voice dropdown when TTS is muted (ttsMuted)
- Compact pill-style trigger with rounded-full, no border, muted bg
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(tts): add voice label prefix and always show dropdown
Show "音色: Alloy" instead of plain "Alloy" in the voice pill.
Always show dropdown regardless of mute state (voice config is
independent of playback).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(tts): add volume icon hint in collapsed AgentBar
Show a small Volume2 icon in the collapsed pill when voice config
is available, hinting that voice settings are inside.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): fix voice dropdown layout and click handling
- Move voice pill below agent name (second line) to prevent
horizontal overflow in English
- Wrap Select in div with onPointerDown stopPropagation to fix
Radix click-through to parent row
- Add line-clamp-1 to descriptions for consistent row height
- Use items-start instead of items-center for better multi-line alignment
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(tts): redesign AgentBar voice layout for compactness
- Single-line layout: checkbox · avatar · name · role · voice pill
- Remove descriptions from agent rows (saves vertical space)
- Extract AgentVoicePill component to isolate Select event handling
- Smaller avatars (size-7), tighter row padding (py-1.5)
- Voice pill uses Volume2 icon + voice name (no prefix text)
- Works in both Chinese and English without overflow
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): cross-provider voice selection per agent
- Change voiceConfig from per-provider lookup to explicit
{ providerId, voiceId } per agent
- Each agent can use a different TTS provider's voice
- Voice picker dropdown groups voices by provider
- useDiscussionTTS routes TTS requests per agent's provider
- resolveAgentVoice falls back to global provider if no config
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): align role badge and voice pill across agent rows
Give role badge fixed width (w-14 text-right) so role text
and voice pills align vertically across all rows regardless
of agent name length.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): fix role badge and voice pill alignment
Wrap role badge + voice pill in a fixed-width container
(w-[9.5rem] justify-end) so both align vertically across
all agent rows regardless of name or role text length.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(tts): align role badge and voice pill across agent rows
Add min-w-[52px] text-right to role badge so it starts at a
consistent position regardless of agent name length.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): use fixed w-[60px] for role badge alignment
* fix(tts): use fixed w-[88px] for voice pill alignment
* fix(tts): prevent click-outside from closing AgentBar when Select portal is open
* fix(tts): comprehensive voice picker rewrite
- Replace Radix Select with Popover + button list (fixes click issue)
- Fix getAvailableProvidersWithVoices to always include global provider
- Widen panel from w-80 to w-96 (prevents name truncation)
- Voice pill uses primary color instead of gray (more visible)
- Extract renderAgentRow helper to reduce duplication
- Popover shows voices grouped by provider with active state
- Add findVoiceDisplayName utility
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): align voice provider availability with toolbar logic
* fix(tts): fallback to first available provider when global provider has no voices
* refactor(tts): remove global provider fallback from voice resolution
Voice resolution now only depends on available providers (those with
API keys or server-configured). No more globalProviderId parameter.
Fallback is first available provider, then browser-native-tts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): add browser native TTS voices to agent voice picker
Load speechSynthesis.getVoices() in AgentBar and include as a
"Browser Native" provider group in the voice popover. No API key
needed - always available if browser supports it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): simplify toolbar TTS to on/off toggle, add disabled state
Toolbar:
- Replace volume slider with simple TTS on/off toggle button
- Remove ttsMuted/ttsVolume/onVolumeChange props from CanvasToolbar
- Toggle now controls ttsEnabled (not ttsMuted)
AgentBar:
- Collapsed: show VolumeX icon when TTS disabled
- Voice pills show disabled state (gray, cursor-not-allowed, no popover)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(tts): simplify Settings TTS tab to toggle + provider config
Remove voice selection, speed slider, preview/test, Azure locale
filter from Settings TTS tab. Voice is now per-agent in AgentBar.
Keep: on/off toggle, provider selector, API key + base URL config.
Add hint text pointing to AgentBar for voice configuration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(tts): simplify media popover TTS tab to toggle only
* fix(tts): add voice config hint to media popover TTS tab
* feat(tts): add per-voice preview button in voice picker
Each voice row in the popover has a small speaker icon button.
Click to preview the voice with "欢迎来到AI课堂" / "Welcome to
AI Classroom" (follows i18n). Browser native uses Web Speech API,
server TTS calls /api/generate/tts. Click again or close popover
to stop. Shows spinner while generating.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): preview text follows course language instead of UI language
* refactor(tts): redesign AgentBar expanded panel layout
- Teacher always at top with voice pill (works in both modes)
- Mode tabs moved below teacher
- Auto mode: single compact row with shuffle icon + description
- Max turns: compact inline row with smaller input
- Preset mode: only student agents listed (teacher already above)
- Remove large shuffle animation from auto mode
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(tts): merge max turns into teacher row
* refactor(tts): separate teacher row and max turns, use stepper UI
- Teacher row: avatar + name + voice pill only
- Max turns: bottom row with MessageSquare icon + compact stepper
(minus/number/plus in a rounded pill)
- Remove Input component dependency
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): increase voice pill contrast in dark mode
* fix(tts): make max turns input editable, tighten panel padding
* fix(tts): restore shuffle animation in auto mode (compact version)
* fix(tts): adjust auto mode text spacing and add voice auto-assign hint
* fix(tts): auto-close voice popover after selecting a voice
* fix(tts): increase auto mode vertical padding for better balance
* fix(tts): push auto mode text toward bottom with flex spacer
* fix(tts): reduce auto mode bottom padding
* feat(tts): wait for TTS audio to finish before next agent turn
Add waitForDrain() to useDiscussionTTS that returns a promise
resolving when the audio queue is empty. The agent loop in
useChatSessions now awaits this after buffer drain, so the next
agent's turn doesn't start until the current agent's TTS audio
finishes playing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): keep bubble visible while TTS audio is still playing
When buffer drains (text=null) but audio indicator is still active,
don't clear liveSpeech. Clear it only when audio state goes idle.
This keeps the speech bubble visible until TTS finishes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): hold discussion bubble until TTS audio finishes
When StreamBuffer fires the done signal (onLiveSpeech null), Stage
now checks if TTS is still playing. If so, it defers clearing the
bubble state. The bubble stays visible until onAllAudioEnd fires
from the TTS hook (queue empty + nothing playing), then clears.
This prevents the jarring UX where the bubble disappears while
the agent's voice is still audible.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): fix bubble hold - guard onStopSession instead of onLiveSpeech
Root cause: bubble disappears because doSessionCleanup fires via
onStopSession when the agent loop ends naturally, NOT because of
onLiveSpeech(null, null).
Fix: when onStopSession fires and TTS is still playing, defer
doSessionCleanup to onAllAudioEnd callback. Manual stop (user
presses button) still cleans up immediately via handleStopDiscussion.
Use doSessionCleanupRef to avoid circular dependency between
discussionTTS hook and doSessionCleanup useCallback.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): guard BOTH onLiveSpeech and onStopSession for bubble hold
Two paths clear the bubble:
1. onLiveSpeech(null, null) from StreamBuffer done → clears liveSpeech
2. onStopSession → doSessionCleanup → clears all state
Both fire when agent loop ends. Path 1 fires first (tick loop),
path 2 fires after (waitUntilDrained resolves). Both must be
guarded when TTS is still playing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): hold bubble during TTS playback and respect playback speed
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): LLM picks voice matching agent persona during generation
- Client sends available voices (providerId + voiceId + name) to
/api/generate/agent-profiles
- LLM prompt asks to pick a voice matching each agent's personality
- Parse "providerId::voiceId" from response, save as voiceConfig
- Fallback to index-based assignment if LLM doesn't pick
- Browser native voices hidden when server providers are available
- saveGeneratedAgents accepts and persists voiceConfig
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): restore volume slider in classroom toolbar
Revert the toolbar simplification from
|
||
|
wyucandClaude Opus 4.6
|
843d33e5cc |
chore: enforce Prettier formatting and fix lint issues
- Add .prettierignore to exclude vendor packages, lock files, markdown, and YAML - Update .prettierrc: printWidth 100, singleQuote, trailingComma "all" - Run Prettier across all source files for consistent formatting - Fix unused imports (UserRequirements, setTTSProvider) - Fix eslint-disable comment placement after Prettier reformat Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.6
|
cc44f32ca8 |
Refactor: directory structure cleanup + Google Fonts localization
- Replace next/font/google with local fonts (geist + @fontsource-variable/inter) to eliminate build-time network dependency - Resolve lib/utils.ts vs lib/utils/ naming conflict (cn() → lib/utils/cn.ts) - Move React Context Providers from lib/store/ to lib/contexts/ - Consolidate hooks into lib/hooks/ (use-theme, use-i18n) - Organize loose components into subdirectories: agent-* → components/agent/ canvas-* → components/canvas/ generation-toolbar, media-popover → components/generation/ speech-button, tts-config-popover → components/audio/ proactive-card → components/chat/ 65 files changed, lint 0 errors, build 28 pages passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |