mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-17 05:29:46 +08:00
@openmaic/generation@0.3.7
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 wyuc
|
e3fd494658 |
feat(tts): add Qwen TTS voice cloning (#1160)
* feat(tts): add Qwen TTS voice cloning via the registration adapter * feat(settings): add Qwen TTS clone-voice manager * fix(tts): harden Qwen voice cloning integration * fix(tts): enforce voice-bound Qwen synthesis * fix(tts): close Qwen voice clone tail issues * fix(tts): close final voice clone review issues * fix(agents): harden voice token parsing * fix(settings): hide the voice-clone model from manual selection * fix(agents): pin the narrator voice to the user's selected voice * fix(agents): keep narrator fallback alive when the pinned voice is unusable * fix(i18n): correct the narration fallback notice copy * fix(tts): bound the narrator voice retry to a single fallback hop * fix(tts): tolerate transient vendor errors in the voice existence precheck |
||
|
杨慎andwyuc
|
0f44e9a46f |
Add PBL v2 runtime APIs and classroom UI (#799)
* feat: add PBL v2 runtime UI * fix: ship PBL v2 instructor avatar * fix: ship PBL v2 workspace logo * fix: prevent scenario PBL legacy fallback * chore: restore package lint comments * test: cover ordinary PBL legacy fallback * fix: front-load PBL single-call validation * fix: preserve PBL v2 content when assembling scenes --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
3c30e9397f |
fix(audio): gate SpeechButton on ASR availability (#711)
* fix(audio): gate SpeechButton on ASR availability SpeechButton only honored the `disabled` prop and never read ASR state, so each call site had to wire it and the home page passed nothing — the mic button stayed clickable even when ASR was turned off. Add a `useASRAvailable()` hook as the single source of truth (the `asrEnabled` toggle + whether the selected provider is usable + Web Speech API support for browser-native) and consume it inside SpeechButton, so every call site is gated uniformly. The external `disabled` prop still composes on top, so existing call-site conditions keep working. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(audio): address review — trim API key, require custom-provider models - Treat whitespace-only API keys as unconfigured: the recorder only sends keys after trim(), so an untrimmed check enabled the button and then failed server-side as missing credentials (review P3). - Custom ASR providers now count as unavailable until at least one model is configured, mirroring the media-popover rule that hides model-less custom providers (review P2). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
|
35d36905d1 |
Make server-configured providers admin-managed, not client-overridable (#620) (#624)
* refactor(providers): make server-configured providers admin-managed, not client-overridable A server-configured provider is now authoritative: the operator owns its API key and base URL, and the client cannot override either. This collapses the tri-state (server key / client base URL / client key) that was the root of a recurring "missing API key" bug class — when the settings UI echoed a base URL with an empty key, routes that treated "any base URL ⇒ fully client-configured" dropped the server key. #533 patched that per-route via canUseServerApiKeyForBaseUrl; this removes the guard entirely by fixing the design. Server: - resolve*ApiKey/BaseUrl ignore client values when the provider is server- configured; add isServerConfiguredProvider(section, id); drop canUseServerApiKeyForBaseUrl + normalizeProviderBaseUrl. - getServer*Providers() no longer return base URLs — only allowed models (LLM/image) and the managed flag (presence) — so an internal gateway/proxy endpoint is never exposed to the client. - routes (chat/image/video/tts/asr/pdf + verify-*) resolve managed providers purely server-side; SSRF validation now applies only to unmanaged (client) base URLs. Client: - drop serverBaseUrl from ProviderSettings, the settings store, request-building (TTS base URL no longer echoes the server URL) and voice resolution. - settings UI renders managed providers read-only: the key/base-URL override inputs and region toggles are hidden, leaving the "managed by server" notice. Tradeoff: overriding a server provider's key with your own for the same provider is removed; add a separate custom provider instead. Closes #620. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers): refresh managed-provider copy, drop dead audio-settings, add #620 e2e - serverConfiguredNotice now says the provider is managed and not editable here, pointing users to add a separate provider for their own credentials (7 locales). - delete components/settings/audio-settings.tsx — unreferenced dead code. - add e2e/tests/managed-provider-620.spec.ts: managed provider hides the key/base-URL inputs; unmanaged provider keeps them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(providers): lock the model catalog for managed providers with a pinned list When the operator pins an allowed model list (MODELS env/yaml ⇒ serverModels), the model catalog is admin-managed too: the LLM provider panel shows the pinned models read-only with a "Server" badge and no add / reset / edit / delete affordances. Without a pinned list the server manages only credentials and the user keeps curating models. This removes the prior inconsistency where the settings panel let you add models to a pinned managed provider that the generation toolbar then filtered out. e2e: the managed-openai case now also asserts the catalog is locked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers): ignore (not reject) client overrides for managed web-search; clear legacy serverBaseUrl Addresses review feedback on #624: 1. Managed web-search providers now ignore a client-sent key/baseUrl instead of validating and rejecting it. A server-configured provider is admin-owned, so the server config is authoritative and a stale client base URL is dropped rather than 400-ing the request. The allowlist validation still runs for unmanaged providers, where the client base URL is actually used. 2. Drop the removed `serverBaseUrl` field from persisted localStorage on rehydrate (settings version 2 -> 3, stripped in both migrate and merge), so old clients don't keep a stale server URL in client state. Tests: web-search route test split into unmanaged-rejects-allowlist and managed-ignores-client-baseUrl; settings rehydrate test asserts a legacy serverBaseUrl is stripped. Full suite 381 pass; e2e 2 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
wyuc
|
5e12ad892c |
[codex] add VoxCPM2 TTS provider (#496)
* add voxcpm2 tts provider * address voxcpm review feedback * translate voxcpm locale strings * narrow toolbar model pill * refine toolbar media controls * address voxcpm review followups |
||
|
wyucandClaude Opus 4.6
|
9a0060e2f0 |
feat(audio): add custom OpenAI-compatible TTS/ASR provider support (#357) (#409)
* feat(audio): extend TTSProviderId/ASRProviderId to support custom providers * feat(audio): update provider registry helpers for custom provider support * feat(audio): route custom providers to OpenAI-compatible implementations Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(audio): add custom TTS/ASR provider CRUD to settings store * feat(audio): add dialog for creating custom audio providers * feat(audio): add custom provider UI with voice management and delete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(i18n): add custom audio provider translation strings * fix(audio): resolve remaining type errors for custom provider support Add `as keyof typeof` casts where TTSProviderId/ASRProviderId (which now include template literal types) index into the built-in-only registries TTS_PROVIDERS, ASR_PROVIDERS, DEFAULT_TTS_VOICES, and DEFAULT_TTS_MODELS. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(audio): allow custom provider IDs to reach switch default branch The early guard `if (!provider) throw` blocked custom provider IDs from reaching the switch-case default branch where they get routed to OpenAI-compatible implementations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(audio): add default model field to dialog and polish voice list UI - Add "Default Model" input to custom audio provider dialog (fixes Kokoro needing model:"kokoro" instead of fallback "gpt-4o-mini-tts") - Redesign voice list as aligned table with column headers, hover reveal delete buttons, and empty state message - Add Enter key support to voice add inputs - Use consistent grid layout for add voice row Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl when testing custom TTS provider The baseUrl field in provider config is empty by default (it's an override field). Custom providers store their default URL in customDefaultBaseUrl, which needs to be used as fallback for test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): auto-select first voice when adding to custom TTS provider When a custom provider's voice is 'default' (initial state before any voices are added), adding the first voice now automatically updates the active voice selection so test TTS works immediately. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl in all TTS call sites Scene generator, discussion TTS, and agent bar voice preview all read providerConfig.baseUrl which is empty for custom providers (the default URL is stored in customDefaultBaseUrl). Add fallback chain to all 4 call sites so custom providers work during actual playback, not just in the settings test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): include custom provider voices in agent voice picker getAvailableProvidersWithVoices only iterated built-in TTS_PROVIDERS, so custom providers never appeared in the agent bar voice dropdown. Now also iterates ttsProvidersConfig for custom-tts-* entries and includes their customVoices in the picker. Also updated getServerVoiceList, findVoiceDisplayName, and resolveAgentVoice to handle custom provider voice lookups. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl in all ASR call sites Same issue as TTS: audio-recorder, asr-settings, and audio-settings all read providerConfig.baseUrl which is empty for custom providers. Add customDefaultBaseUrl fallback to all 3 ASR call sites. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(audio): add model list management for custom ASR providers Custom ASR providers now support adding/removing models with a table UI (same design as TTS voice list). The model selector dropdown shows custom models and auto-selects the first one. Also adds custom ASR providers to the Toolbar's ASR provider dropdown and fixes custom provider handling in audio-settings.tsx (baseUrl fallback, request URL preview, API key section visibility). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): seed customModels from defaultModel and fix ASR model label - When creating custom ASR provider with a defaultModel, auto-add it to customModels so the model list isn't empty on first visit - Fix ASR settings label from "TTS Model" to "Default Model" Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): separate ASR model management from creation dialog ASR models should be managed in provider settings (add/remove from list, select default), not pre-filled at creation time. Unlike TTS where a single default model suffices, ASR users need to manage multiple models (e.g. whisper-small, whisper-large-v3). - Remove Default Model field from ASR creation dialog (keep for TTS) - Remove defaultModel parameter from addCustomASRProvider - ASR models are added via the model list UI in provider settings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): simplify ASR model management — first model is default Remove the separate model selector dropdown for custom ASR providers. The first model in the list is always used as default. Users manage the model list only (add/remove), no separate "default" selection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): add loading state and detailed errors to ASR test - Show spinner with "Processing..." while waiting for transcription - Disable record button during processing - Show specific error messages from API (e.g. "No transcription generated") instead of generic failure text - Handle empty transcription result as an error with helpful message Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): use correct key for processing label in ASR test button * fix(audio): polish custom provider UX and type safety - Add proper type declarations for custom provider fields (customName, customDefaultBaseUrl, customVoices, etc.) eliminating 30+ unsafe casts - Show custom ASR providers in toolbar with default language list - Hide custom ASR providers without models from toolbar selection - Add no-models warning in ASR settings panel - Add delete confirmation dialog for custom TTS/ASR providers - Add duplicate voice/model ID validation with toast error - Add dedicated i18n keys for ASR model management (modelNamePlaceholder, addModel) - Fix i18n key for ASR processing label (settings.asrProcessing) - Add radio-button model selection in custom ASR provider settings - Add visible OpenAI-compatible description in provider creation dialog - Add CUSTOM_ASR_DEFAULT_LANGUAGES constant for toolbar language options Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(test): add isCustomTTSProvider/isCustomASRProvider to audio types mock The settings-server-sync test mocked @/lib/audio/types as an empty object, but settings.ts now imports isCustomASRProvider for provider validation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): add customDefaultBaseUrl fallback in all TTS/ASR call sites When a custom provider is created, the actual URL is stored in customDefaultBaseUrl while baseUrl is initialized empty. Several call sites only checked baseUrl, causing fetch failures during course generation. Add fallback to customDefaultBaseUrl in: - generation-preview/page.tsx (course generation TTS) - tts-providers.ts (getCurrentTTSConfig) - asr-providers.ts (getCurrentASRConfig) - media-popover.tsx (TTS preview) - tts-config-popover.tsx (TTS preview) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: format generation-preview/page.tsx Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused ttsSpeedRange variable in media-popover Flagged by CodeQL code quality check. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
wyucand杨慎
|
3c687dd523 |
fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#326)
* fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#324) - Move model selection from global ttsModelId to per-provider ttsProvidersConfig[providerId].modelId - Remove supportsModelSelection flag, use models.length > 0 instead - Add defaultModelId to TTSProviderConfig and ASRProviderConfig - Add providerOptions field to TTSModelConfig for future extensibility - Fix ElevenLabs hardcoded model_id: 'eleven_multilingual_v2' - Fix MiniMax TTS using config.model instead of config.modelId - Update model lists from provider docs: - Qwen: add qwen3-tts-instruct-flash, qwen-tts - ElevenLabs: add eleven_flash_v2_5, eleven_flash_v2 - MiniMax: remove non-existent speech-2.8 series - Replace custom model CRUD UI with simple Select dropdown - Remove MiniMax-specific hardcoded model selector from audio-settings - Add migration for old ttsModelId/asrModelId/model fields * feat(tts): voice picker groups by provider:model, sets both model and voice Voice selection in agent bar now groups voices by model. Picking a voice from a model group sets both modelId and voiceId on the agent. TTS settings shows available models as display-only chips. - Add compatibleModels field to TTSVoiceInfo (OpenAI marin/cedar restricted to gpt-4o-mini-tts) - Add modelId to agent voiceConfig and ResolvedVoice - Add ModelVoiceGroup to ProviderWithVoices, build groups in getAvailableProvidersWithVoices - Update AgentVoicePill and TeacherVoicePill to iterate modelGroups - Thread modelId from voice selection through discussion TTS pipeline - Replace model Select with display-only chips in tts-settings * fix(tts): show provider name in voice picker group headers * fix(tts): restore MiniMax speech-01 models per official API docs * fix(tts): add MiniMax speech-2.8 models per official docs, default to speech-2.8-hd * feat(tts): expand MiniMax voice list with recommended voices * fix(tts): remove non-official MiniMax recommended voices, keep 10 from official list * fix: address CR findings - align defaults, fix test mocks, ASR modelId fallback * fix(tts): pass modelId in voice preview and classroom generation * fix(tts): voice picker highlights only the exact model:voice match --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
wyucand刘少杰
|
5adf91c6f3 |
feat(settings): add configurable model selection for TTS and ASR (#108)
Add ttsModelId/asrModelId to persisted settings, model management UI for providers that support model selection, and propagate selected model through preview, generation, and transcription flows. Co-authored-by: 刘少杰 <liushaojie@cvte.com> |
||
|
|
db5ac82167 |
fix: use browser speechSynthesis for playback when browser-native-tts is selected (#28)
* fix: use browser speechSynthesis for playback when browser-native-tts is selected Previously, selecting browser-native-tts as the TTS provider would produce sound in the settings test but remain silent during classroom playback. This happened because: 1. The scene generator correctly skipped pre-generation for browser TTS (it runs client-side, not via API) 2. The playback engine fell back to a silent reading timer when no pre-generated audio was found, instead of calling speechSynthesis This commit adds Web Speech API integration directly in the PlaybackEngine: - New playBrowserTTS() method speaks text via speechSynthesis - Properly wires onend/onerror to advance to the next action - pause()/resume() now handle speechSynthesis.pause()/resume() - stop() and handleUserInterrupt() cancel browser TTS Fixes #25, fixes #12, fixes #5 * fix: handle "default" ttsVoice, extract CJK_LANG_THRESHOLD constant - When ttsVoice is "default" (set by Browser Native TTS which has no voice picker), the voiceURI lookup silently fails and no lang is set, causing Chinese text to be spoken with an English voice. - Extract the 0.3 CJK detection threshold as a named constant CJK_LANG_THRESHOLD with JSDoc explaining the rationale. - Fall through to language auto-detection when voice lookup fails, regardless of the reason (missing voice, "default" sentinel, etc.). * fix: support browser-native tts previews * refactor: extract shared TTS preview logic into useTTSPreview hook The browser-native and API-based TTS preview code was duplicated across tts-config-popover, media-popover, and tts-settings. Extract it into a reusable useTTSPreview hook that handles refs, cancellation, audio lifecycle, and staleness checks in one place. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: yangshen <1322568757@qq.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.6
|
843d33e5cc |
chore: enforce Prettier formatting and fix lint issues
- Add .prettierignore to exclude vendor packages, lock files, markdown, and YAML - Update .prettierrc: printWidth 100, singleQuote, trailingComma "all" - Run Prettier across all source files for consistent formatting - Fix unused imports (UserRequirements, setTTSProvider) - Fix eslint-disable comment placement after Prettier reformat Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.6
|
cc44f32ca8 |
Refactor: directory structure cleanup + Google Fonts localization
- Replace next/font/google with local fonts (geist + @fontsource-variable/inter) to eliminate build-time network dependency - Resolve lib/utils.ts vs lib/utils/ naming conflict (cn() → lib/utils/cn.ts) - Move React Context Providers from lib/store/ to lib/contexts/ - Consolidate hooks into lib/hooks/ (use-theme, use-i18n) - Organize loose components into subdirectories: agent-* → components/agent/ canvas-* → components/canvas/ generation-toolbar, media-popover → components/generation/ speech-button, tts-config-popover → components/audio/ proactive-card → components/chat/ 65 files changed, lint 0 errors, build 28 pages passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |