50 Commits

Include renames

Author	SHA1	Message	Date
zhifu gaoandLauraGPT	2b3f2eaa37	feat(asr): add local FunASR provider (#1044) ... Signed-off-by: LauraGPT <199134975+LauraGPT@users.noreply.github.com> Co-authored-by: LauraGPT <199134975+LauraGPT@users.noreply.github.com>	2026-08-05 03:03:56 -04:00
Nguyễn Lê TrườngandClaude Opus 5	4f20c5151a	fix(i18n): translate ASR provider names in the generation toolbar (#1028) ... The provider dropdown in the media popover rendered `p.name` straight from `ASR_PROVIDERS`. That field is a registry label, not a UI string, and several entries are written in Chinese: browser-native → 浏览器原生 ASR (Web Speech API) qwen-asr → Qwen ASR (阿里云百炼) So a user on any non-Chinese locale saw Chinese in the toolbar, while the settings dialog showed the same providers correctly — it resolves the name through `t('settings.providerBrowserNative')` and friends. The id→i18n-key tables move to `lib/audio/provider-display.ts` and both surfaces use them, so they cannot drift apart again. The settings dialog keeps its own handling of custom providers, which carry a user-supplied name rather than an i18n key. TTS is included in the shared module for symmetry (`doubao-tts` and `qwen-tts` have the same problem), though the popover's TTS tab is currently a toggle with no provider list. Image and video groups in the same popover also render `p.name`, but those registry labels are already English, so no user-visible string changes; left alone to keep this focused. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-03 06:04:49 -04:00
4 people	f378c25a53	feat(storage): route settings/user-profile persist through KVStore (no legacy migration) (#1001) ... * feat(storage): route zustand persist through KVStore The settings and user-profile stores persisted through zustand's default `localStorage` storage, while the rest of the app's small keyed values had already moved to `@openmaic/storage`'s `KVStore` — two unrelated mechanisms over the same browser API, which is the split-brain the storage RFC set out to remove. `kvPersistStorage` shipped with the package but had no importer. Add `lib/store/kv-persist.ts`, the app seam that adapts a `KVStore` into a zustand `PersistStorage`, and wire both stores to it under an explicit `account` scope. Store logic is unchanged: `version` / `migrate` / `merge` still belong to the stores, only the bytes' destination moves. Existing installs are adopted rather than orphaned. On first read the seam moves the raw `localStorage` entry into the KV scope — write first, then drop the raw key, so an interrupted migration leaves the value readable from one side or the other but never from neither. Once KV holds the entry it is authoritative, so a stale raw entry reappearing cannot roll back newer state, and re-running can neither duplicate nor resurrect anything. An entry that is not a readable persist envelope is left exactly where it is. Hydration is now asynchronous, because a `KVStore` may be remote. The browser backend resolves within a few microtasks of the store module being evaluated, so nothing user-visible changes, but tests that assert on persisted state now await `persist.rehydrate()` instead of relying on a synchronous read. The four pre-persist keys (`llmModel` / `providersConfig` / `ttsModel` / `selectedAgentIds`) are deliberately untouched: they are read-only inputs to a migration that never owned them, and `providersConfig` is still read independently by `lib/ai/providers.ts`. Part of #857. * fix(storage): close the async-storage failure modes in the KV persist seam Cross-review found that routing `persist` through an asynchronous store opens failure modes a synchronous `localStorage` never had. Each one below is now covered by a test that fails without its fix. Data loss on a failed or empty hydration. A KV read that threw was indistinguishable from an empty scope, so the store stayed on defaults while an always-on initializer's `set()` persisted those defaults; the next load then read them as a completed migration and deleted the raw original, leaving nothing. Reads now separate "nothing there" from "the backend could not answer": a failed read serves the raw entry read-only, adopts nothing and deletes nothing, and leaves the key unwritable for the session so no un-hydrated state can be persisted over an original. Raw entries are additionally only dropped against a recorded completion marker, so a KV entry of unknown provenance can never authorize deleting the only copy of the user's data. Lost adoption across tabs. A reader whose KV read answered before another tab's adoption write, and whose localStorage read landed after that tab's delete, saw nothing in either place and hydrated defaults. Against a remote backend that interleaving is ordinary rather than rare, so an apparent "nothing anywhere" is now re-checked before it is believed. Out-of-order writes. Two rapid `setItem`s could complete in either order and let the older value win. Reads and writes are serialized per key. Pre-persist migration resurrecting on every load. `migrateFromOldStorage` was guarded by the existence of the raw `settings-storage` key — the very key the migration removes — so once the blob moved into the KVStore it ran on every load and republished whatever the pre-persist keys still held, including API keys the user had since deleted, into first-frame state. It is now the persist layer's last-resort source instead of a synchronous read in the store initializer: its result is adopted like any other migration, so the adopted entry is the marker that stops it. Initial state is plain defaults, and nothing stored reaches observable state before the storage layer has vouched for it. The source keys are still only read, never deleted, since `lib/ai/providers.ts` remains a reader. Also: KV reads and writes are guarded so a corrupt entry surfaces instead of throwing through zustand's silent catch; clearing the cache now clears the migrated stores through their own storage, which a server-backed scope would need and a blanket `localStorage.clear()` would only appear to do; `removeItem` drops the KV copy before the raw one; and the e2e suite gains post-migration steady-state coverage, which every existing spec missed by seeding the pre-cutover key. Part of #857. * fix(storage): scope migration bookkeeping to the device, gate writes on call Second review round. The findings shared one root cause: two of the three sentinels recorded a fact about *this machine's* files but were stored beside the data, in the store's own scope. Under a syncing `account` backend that is a category error with teeth — one device's history would travel to another and authorize deleting files it says nothing about. Sentinels are now always `device`-scoped and carry a reserved `__` prefix, whatever scope the store uses. The worst combination this closes: device A migrates, its marker syncs, device B — which has never run this build — loads and deletes the raw entry that is still the only copy of its own settings. A test drives exactly that sequence against two devices sharing one account scope and nothing else. The pre-persist fallback gains a device sentinel of its own. Its source keys are device-local and deliberately left in place, so "KV and raw are both empty" was not a safe trigger: clearing the account entry, from the cache button or a server-side wipe, let any device re-read those keys and republish credentials the user had rotated. It is now consulted at most once per device, and `removeItem` deliberately does not clear that sentinel — clearing the entry is precisely the case it exists for. The fallback also no longer writes the adoption marker: that marker is a licence to delete a raw persist entry, and the fallback has adopted no such thing. Were it written, a raw entry later restored by an older bundle would be deleted unread. The write gate is now read when `setItem` is called rather than when its queued turn arrives. Evaluated inside the queue, a write issued during hydration waited behind `getItem`, found the gate opened by the very read it raced, and persisted its pre-hydration snapshot on top of the stored value. Refused writes are no longer a dead end: the first one asks the store to rehydrate, since the backend may have recovered, and if that does not work the user is told through a durable notice rather than losing every change on reload. Each refusal is logged, not just the first — a once-per-session warning hides how long a session has been silently unsaved. `removeItem` now propagates backend failures, so clearing the cache cannot report success for a clear that did not happen, and that flow reloads immediately instead of leaving a window in which a background `set()` re-persists the credentials just deleted. Also: the recovery hook is bound after each store is constructed, because naming the store inside its own definition made its type circular and silently widened every selector to `any`; the `serial` comment no longer claims an unreachable rejection handler; and the pre-persist version stamp documents that it skips `migrate`'s unversioned normalization too, which `merge` redoes anyway. Part of #857. * refactor(storage): make the KV persist seam a per-key state machine Three review rounds kept turning up the same defect class in new places: a backend failure read as "there is nothing there", and the result of an operation that nobody looked at. Patching instances of it was not working, because both are possible whenever a call site is free to decide for itself what a failed operation meant. This makes that decision unavailable to call sites. Every backend call now returns an `Outcome` whose payload is a private field and whose only reader demands a `KeyState`. Feeding the machine is not a convention to remember; it is the only way to get the value out. What comes back is the value or `UNAVAILABLE` — deliberately not `null`, so "the backend could not answer" is not spellable as "nothing is there". The machine then decides, in one place, what failure means: any read failure is unavailability, any write failure closes the key, and either raises the health signal and asks for recovery. The scattered closure flags (a settled set, a warned set, a recovery latch) become one explicit phase per key — unhydrated, settled, unavailable, clearing — with the transition table written out in the docstring and a test per row. What that consumes: - A write failing after the key settled used to be ignored; its outcome now closes the key, signals, and the value is remembered for replay. - `removeItem` enters `clearing` synchronously at call time, so a `set()` racing a clear cannot be admitted, queue behind the delete, and write the just-deleted value back. Nothing is replayed across a clear. - The pre-persist sentinel is recorded by *every* settling path, not only the one that consults the old keys. A device that had only ever loaded an already-populated account entry recorded nothing, and so stayed eligible to republish its own stale credentials the moment that entry was cleared — which the docstring already claimed it would not. - Two places treated a failed read as an answer: the sentinel read that decides whether the fallback runs, and the confirming re-read before adopting. Both now stop. - An adoption whose marker did not land no longer settles, and never deletes the raw entry: no proof, no licence. - Refused and failed writes are remembered and replayed once storage recovers, but only when the store was holding real data at the time — replaying a snapshot taken over defaults would overwrite the stored value, which is the failure this seam exists to prevent. When a snapshot cannot be replayed the user is told, because rehydration is about to discard those edits. - Health notices publish on a later task, so a recovery that lands moments later cancels a warning nobody should have seen, and a subscriber mounting after the failure is caught up once its toast host exists. `StorageHealthNotice` now sits after the `Toaster` for the same reason. Also: adoption re-reads before committing so a concurrent write is not rolled back; a shadowed raw entry is only deleted if it parses, matching the promise never to delete bytes we could not read; the unavailable copy points at the control that resolves it rather than suggesting a reload that cannot; and the version pin, the pre-persist retention rationale, and the cache-clear comment say what is actually true. Part of #857. * fix(storage): separate the contexts settle() was conflating Convergence round on the state machine. The findings share a root: settle() treated every read that concluded as licence to open the key, without asking what else was true at the time. - A read finishing while a clear is outstanding no longer settles. Under a remote backend a recovery rehydrate in flight during a clear is ordinary, and opening the gate there admits a write that queues behind the pending delete and restores the credentials just removed. Only finishClear opens a cleared key. - The raw entry served read-only after a failed read no longer counts as the authoritative value. A shadowed raw entry is by construction the older copy — long-term coexistence with a newer KV value is a designed state — so an edit taken against it must never be replayed, or one transient read failure on one device rolls the whole account back. - A write that lands supersedes an earlier failed one. Both were admitted before the key closed, so the queue ordered them; replaying the older snapshot on recovery would undo the newer value. - Unreachable browser storage is a failure, not an empty store. Privacy modes make localStorage throw; treating that as "nothing stored" hydrated defaults, refused every write and said nothing until the user reloaded and found the work gone. SSR stays a silent no-op. - The recovery latch re-arms when storage answers, so the three recovery transitions are not empty promises from the second failure onward. - A refusal carries the phase it happened in. Cold-start timing — an always-on initializer writing before the first read resolves, on a backend that never failed — is a log line, not a sticky red toast; only a refusal under `unavailable` reports lost changes. The health channel stops delivering stale catch-ups: its timers are cancellable and re-read the current state when they fire, so a recovery cannot be overtaken by a warning with nothing left to dismiss it. And `changes-lost` is now a fait accompli rather than a condition — a later recovery has nothing to retract and leaves it standing. Also: a throwing `localStorage` is unavailability rather than absence wherever the raw entry is read; clearing records the pre-persist sentinel like every other settling path, making that docstring true; the raw entry is dropped before the marker so a failed marker clear leaves something inert; and the pre-persist fallback and its four source reads can no longer throw into zustand's silent catch. Part of #857. * fix(storage): hold a replay until it lands, and prove a raw copy before deleting it Three failure-path corners. A replay was retired the moment its write was attempted. Readable but not writable is a real state — a quota-exhausted backend answers reads perfectly — so a recovering read could succeed, the replay write fail, and the snapshot be gone with nothing owed: the next rehydrate served the older stored value over the user's edit and said nothing. The snapshot is now held until the write is durable; a failed replay goes back to being a refused write, so the next recovery tries again and a permanent failure is eventually reported. The adoption marker was treated as proof that a raw entry is the migration's leftover. It proves only that a migration happened on this device once. A rolled-back deployment, or a tab still running the old bundle, writes the raw key again with settings the migration never saw, and the next load on the new bundle deleted them unread. A raw entry is now dropped only when its content equals the value shadowing it — anything else is data, and keeping a copy beats deleting an original here as everywhere else in this file. `changes-lost` was being cancelled by a later recovery. The two statuses answer different questions: "storage is down" describes the world now and stops being true when storage works, while "your edits were lost" describes something that already happened and no amount of recovery undoes. They are now tracked on independent lines per key and rendered as two toasts with distinct ids, so retracting the fault cannot take the acknowledgement with it; the loss notice goes away only when the user closes it. Part of #857. * fix(storage): bound recovery, keep a live store from walking backwards Three interactions the state machine did not have an answer for. Recovery had no budget. A backend that reads but cannot write — a quota-exhausted one does exactly that — turned it into a treadmill: the recovering read succeeds, the key settles, the replay write fails, and that failure asks for recovery again, forever. Attempts now follow a backoff schedule whose length is the cap, and the budget is re-armed only on real progress: a settle that left nothing owed. A settle reached *by* a recovery with a replay still queued is not progress, and counting it as such is precisely what made every lap look like a fresh start. When the budget runs out the key is left read-only and the loss is reported, rather than retrying where nobody can see it failing. A failed read could walk a live store backwards. Once a session has loaded the authoritative value, a transient read failure was still falling back to the shadowed raw copy — which is legitimately older, and legitimately long-lived now that a differing raw entry is kept rather than deleted — and handing it to zustand overwrote a live store with stale data. Worse, the store had already latched "holding real data", so edits made on top of that stale value counted as replayable. A failed read after an authoritative one now returns nothing, which is exactly "keep what you have", and the raw fallback stays where it belongs: the case where nothing authoritative has been seen at all. Dismissing a lost-changes toast only closed the UI. The key stayed in the lost set, so the same store losing changes a second time was swallowed as a duplicate and any later subscriber resurrected a notice the user had already dealt with. The close button now clears the latch, so a second loss is reported and an acknowledged one stays gone. Part of #857. * fix(storage): close the last two provisional and clear-ordering gaps An abandoned migration was handing back an authoritative value. When the confirming re-read fails, or the adoption write does, `adopt` returns the raw envelope unadopted — the same footing as the raw copy served after any other failed read — but it was returned bare, so the store latched "holding real data" and an edit taken on top of it counted as replayable. Another device publishing to the account scope in that window would have been rolled back. `adopt` now reports provisionality like every other read path, and the shape is a named `LoadedValue` so the next path cannot forget. Clearing swallowed a failed raw-entry deletion. The account entry was already gone by then and the marker was cleared afterwards, so the operation reported success while leaving an empty scope beside a live raw entry — precisely the shape the next hydration re-adopts, handing back the settings the user had just deleted. The raw entry now goes first and its failure propagates, which also fixes the ordering: failing before anything is removed leaves the KV value authoritative and the clear simply unperformed, which the caller can retry. Adoption keeps the tolerant variant, where a leftover raw entry is a duplicate rather than a loss. Part of #857. * fix(storage): make the shadowed-raw delete atomic against a concurrent tab cosarah reproduced a data loss: both raw-entry deletes verified the value, then awaited (reading or writing the adoption marker), then removed the key by name. A tab running an old bundle — which takes no Web Lock, so a lock could not have helped — overwrites the raw key with newer settings during that await, and the unconditional `removeItem` then deletes the newer value. The raw side is synchronous `localStorage`, so the fix needs no lock. Both deletes now finish every await first, then run a single synchronous compare-and-delete: re-read the raw bytes, and remove only if they still equal the exact string that was verified. A competing tab is a separate context whose writes surface here only at event-loop boundaries, so its write lands wholly before the re-read (seen — bytes differ — kept) or after the remove, never in between. A changed value is newer data, not a migration leftover, and is kept. Both sites cosarah named are covered: the shadowed-cleanup path (`dropShadowedLegacy`) and the adoption cleanup in `adopt`. `readLegacyEntry` now carries the exact bytes alongside the parse so the re-check is byte-exact rather than comparing a re-serialized value. The clear path's delete is unchanged: it is a deliberate full delete, not a conditional leftover cleanup. Part of #857. * fix(storage): stop deleting the raw entry — adopt by copy (Plan B) Maintainer decision after cross-vendor + red-team review: `localStorage` has no cross-process atomic compare-and-delete (the storage mutex was dropped from the spec), so a tab on an old bundle can insert a write between our re-read and our removeItem in a genuinely parallel process. The synchronous compare-and-delete narrowed that window but could not close it — as its own comment admitted. Since the raw-cleanup cannot be done safely on localStorage while an old bundle may be live, this stops doing it. Adoption is now a copy: the value is written to KV and the raw entry is left in place. The delete race is therefore impossible — there is no delete. KV stays authoritative (once it holds the value, reads take the KV path and never consume the raw one), so the shadow duplicate is inert, left for a wholesale raw-cleanup at migration exit. Removed with the two deletes: - `dropShadowedLegacy` (its only job was the gated delete) and its three call sites. - `deleteLegacyIfUnchanged` and the `{raw,value}` shape of the legacy read (`readLegacyEntry` → back to `readLegacy` returning the value). - The adoption marker (`__persist-adopted:`), whose only reader was that delete gate. It did not prevent re-adoption — the presence of the KV value does — so removing it breaks no invariant. `dropRaw` / `expectedRaw` on `adopt` and the marker-clear step in the clear path go with it. The clear path (user deletes all data) still deletes the raw entry strictly — that is a deliberate full delete, not a migration cleanup. Tests: the delete/marker cases become "the raw shadow always survives, KV is authoritative"; a two-device and an old-bundle-concurrent-write case guard it, and re-adding a delete on either adoption path turns them red. Part of #857. * fix(storage): add a monotonic migrated marker to stop shadow resurrection Cross-vendor + red-team review independently caught a P1 the previous "adopt-as-copy" revision introduced. Because the raw entry is now kept and writes only ever reach KV, the raw copy becomes a frozen pre-cutover value. If the account KV is later emptied by something other than a local clear — a server-side wipe, an account switch, KV eviction — the next load saw `stored === null` beside a live raw entry and re-adopted that stale copy, refilling the wiped account with ancient settings/credentials and syncing them to every device. Deterministic, not a race; it would fire once a server-backed account backend exists. Root cause: "keep the raw entry" inherently requires a record that it has already been migrated, so an empty-KV load can tell "first migration" from "external wipe". Removing the old delete-authorization marker in the copy revision dropped that record — Plan B was only half done. Adds a monotonic, device-scoped `__migrated:<name>` marker: - Read on every load. `stored === null` + raw present + marker set ⇒ respect the wipe, do not adopt (fall through to the confirm re-read, so a concurrent repopulation is still served). Only a genuine first migration (no marker, raw present) adopts. - Written best-effort after a successful KV write. A failed write degrades to the KV value's own presence check, so resurrection needs both a failed marker write and a later external wipe. - Read failure fails closed (do not adopt), never "assume not migrated". - Monotonic: only ever set true, cleared only by a deliberate full clear — no compare-and-delete race, which is why it is safe where deleting the raw entry was not. Live state with a real reader, not the delete-authorization marker removed earlier. The clear path clears the marker too (back to a genuine first-load state); best-effort, since with the raw entry already gone a leftover marker is inert. Tests: resurrection guard (external wipe does not re-adopt; concurrent post-wipe write is served; first migration records the marker; marker read fails closed) + the clear-path marker clearing. Mutation-checked: removing the gate resurrects the shadow, dropping the record leaves it unguarded. P3: the inert `__persist-adopted:` test rot is retargeted to the real `__migrated:` path. Part of #857. * refactor(storage): remove automatic legacy migration — pure KV wiring Maintainer decision (terminal): after repeated review rounds kept finding new P1s in the legacy-adoption path — its state space is too large to keep correct — the PR is narrowed from "migrate + wire" to "wire only". Settings and user-profile persist through the KVStore; pre-cutover data is not migrated; an upgrading user reconfigures once (the data is reconstructable and the cost is one-time). Removed the entire legacy-adoption subsystem from the adapter: - `adopt()`, `readLegacy()`, the `legacyStorage` dep, and the raw-fallback read on a failed KV read. - The `__migrated:` marker and its resurrection guard, the confirm-re-read, and the `provisional`/`LoadedValue` machinery. - The `prePersistFallback` dep, `recordPrePersistHandled`, the `__pre-persist-consulted:` sentinel, and `readSentinel`/`writeSentinel`. - In settings.ts, `readPrePersistSettings` (the llmModel/providersConfig reader wired as that fallback) — reading legacy keys is migration, which this PR no longer does. `lib/ai/providers.ts` still reads `providersConfig` independently, so those source keys are left in place. The adapter now reads and writes the KV scope and touches no legacy key: an empty scope hydrates the store to defaults (ordinary zustand). `load()` is a single KV read; a failed read stays unsettled so defaults are never persisted over an unread value; the clear path removes only the KV entry. Kept, because they are async-storage *steady state* a remote KV backend needs, not migration scaffolding: the KeyState machine (hydration gate, write gate, recovery, replay), the Outcome discipline, per-key serialization, and the persist-health channel. Added `purgeLegacyPersistKey`: a best-effort, fire-and-forget removal of the old raw persist blob, called once by each store. Never read, so a leftover is pure garbage — and the old settings blob held plaintext provider API keys, so clearing it is a small security win. Net: the adapter drops ~460 lines. Tests are rewritten to steady-state KV coverage plus an explicit "old raw key is ignored, not migrated" proof at the store level; the e2e seed fixtures move from the raw key to the KV scope key. Part of #857. * docs(storage): align README and comments with the no-migration behavior Follow-up requested on the approved PR: the migration/adoption logic was removed, but a few comments and one README roadmap line still referred to those deleted concepts. Documentation only — no behaviour change. - README roadmap: the settings/user-profile line said "adopting their pre-cutover raw keys"; it now states there is no automatic migration (legacy keys ignored and best-effort purged, users reconfigure once). - kv-persist.ts: the per-key serial chain comment no longer claims to keep "adoption" from interleaving; it serializes a key's read/write/clear. - settings.ts: `SETTINGS_PERSIST_VERSION` no longer documents a "pre-persist fallback" that no longer exists. - Test/fixture headers that described seeding "the pre-cutover raw key" for "one-time adoption" now describe seeding the KV `account` scope, with no legacy-key migration. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(storage): bump version to 0.1.1 for README doc alignment The README is a publishable input, and the no-migration doc alignment changed it, so the package version must increase. Patch bump: documentation only, no code or API change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> Co-authored-by: wyuc <zdq1204@gmail.com>	2026-07-30 15:12:51 +08:00
Yanpeng Wangandwyuc	9670100363	feat(extraction): media (audio/video) extraction + AliDocMind provider (#887) ... * feat(extraction): add AliDocMind provider + media extraction abstraction Adds AliDocMind (Aliyun Document Mind LLM version) as a new vendor alongside unpdf/MinerU, and introduces the media (audio/video) extraction layer that mirrors the document extraction one. Document side (file: pdf/docx/pptx/xlsx/images): - PDF_PROVIDERS gains an `alidocmind` entry; parseWithAliDocMind() maps layouts[] -> ParsedPdfContent. Flows through the existing document extractor registry, so AliDocMind is selectable anywhere MinerU is. - PDFParserConfig / DocumentExtractorConfig gain accessKeyId/accessKeySecret (AliDocMind uses AK/SK, not a single apiKey); env fallback via ALIDOCMIND_ACCESS_KEY_ID / ALIDOCMIND_ACCESS_KEY_SECRET. Media side (mp4/mp3/wav/... -> MediaArtifact): - New lib/media-parse/ domain mirroring lib/pdf/ (types/constants/providers). parseMedia() maps AliDocMind segments[]/audio_frames/video_frames -> MediaArtifact (transcript + keyframes). - MediaExtractorProvider interface + media registry + extractMedia() entry, symmetric to DocumentExtractorProvider / extractDocument(). - MediaArtifact and the ExtractionResult/Artifact/Error/Job envelope live in lib/document/types.ts (re-exported from @/lib/document). Shared AliDocMind SDK wrapper (lib/pdf/alidocmind-client.ts) handles the submit -> poll -> get flow for both sides via @alicloud/docmind-api20220711. Tests: env-gated smoke test (tests/document/alidocmind.smoke.test.ts) drives a real PDF and a real video through AliDocMind; media-artifact type test; extractor-registry test updated for the new provider. Part of #621 (MAIC ETL). Media extraction is the sibling to the document extraction landed in #741. * feat(extraction): wire AliDocMind AK/SK through settings UI + routes Surfaces AliDocMind in the (post-#837) Document Parsing settings panel as a peer of unpdf/MinerU — one panel, one credential entry, more supported formats. Threads Aliyun AccessKey ID/Secret from the store through the extraction routes. - Store: pdfProvidersConfig + setPDFProviderConfig gain accessKeyId/accessKeySecret; default alidocmind entry added. - pdf-settings.tsx: AliDocMind branch renders AccessKey ID + Secret inputs (secret masked with show/hide) and a Test Connection button; request-URL preview shows the DocMind endpoint. Provider auto-appears in the panel via PDF_PROVIDERS; supported-format badges come from #837's registry (ALIDOCMIND_MIMES added to lib/document/mime.ts). - verify-pdf-provider route: alidocmind branch verifies AK/SK via a lightweight authenticated probe (verifyAliDocMindCredentials) — auth-level errors fail, anything else passes. - extract-document route + generation flow (app/page.tsx, generation-preview): accessKeyId/accessKeySecret carried through session → FormData → config. - i18n: alidocmindAccessKeyId / alidocmindAccessKeySecret in 8 locales. - Icon: reuse /logos/bailian.svg (Aliyun family) instead of a missing asset. Verified end-to-end against the running app: AliDocMind panel renders, Test Connection returns "连接成功" through the real Aliyun API. Part of #886. * feat(extraction): route audio/video uploads through extractMedia() (reuse document path) Media uploads now flow through the same upload picker, /api/extract-document route, and generation pipeline as documents — no separate upload area. Only the extraction differs: media mimes dispatch to extractMedia() -> MediaArtifact, which is flattened to the text shape the generation pipeline already consumes. - mime.ts: register audio/video formats in DOCUMENT_FORMATS (accept string, extension map, badges resolve for them); add MEDIA_PROVIDER_SUPPORTED_MIME_TYPES + SUPPORTED_MEDIA_MIME_TYPES, kept separate from PROVIDER_SUPPORTED_MIME_TYPES so the document drift-guard stays document-only. mimesForProviders() folds in a provider's media mimes, so the existing upload helpers (getAcceptStringForProviders / isMimeSupportedByProviders / format badges) cover media automatically when the provider supports it. - extract-document route: media mimes dispatch to extractMedia(); MediaArtifact flattened to timestamped text (synopsis + transcript + keyframes). - Tests: 6 media cases in mime.test.ts (accept/validation/badges/normalization). Verified: uploading a real video through the route returns synopsis + timestamped transcript/keyframes as text (curl, real AliDocMind key). Part of #886. * feat(extraction): use Alibaba Cloud icon for AliDocMind provider Replace the placeholder bailian.svg with a dedicated Alibaba Cloud icon mark (the square symbol from the official wordmark, text removed) so the provider reads as Aliyun rather than Bailian. Square aspect matches the other provider icons. Source: Alibaba Cloud / Alibaba Group brand assets. * style: prettier format AliDocMind + media extraction files * fix(extraction): harden AliDocMind — SSRF guard, cred verify, env gate, table text Addresses review findings on the AliDocMind provider: - SSRF (high): the media branch of /api/extract-document now runs the same validateUrlForSSRF check on a client-supplied baseUrl as the document branch, so an audio/video upload can't point the server's Aliyun SDK at an internal host. - Credential verify (high): verifyAliDocMindCredentials now whitelists success signals instead of blacklisting auth errors. Probed against the real API: valid creds + bogus job returns a no-throw "BizIdNotExistOrResultExpired" body; invalid creds throw InvalidAccessKeyId.NotFound. Only a no-throw response or a "biz-not-found" business error counts as valid — an unreachable endpoint, a localized error, or throttling now correctly reports failure instead of a false "connection successful". - Env-fallback gate (med): resolveCredentials no longer reads ALIDOCMIND_ACCESS_KEY_ID/SECRET unconditionally. Env fallback is opt-in via allowEnvFallback, which the route sets only for a server-managed provider, so an unauthenticated client request can't silently run on the server account. - getDocParserResult error body (med): fetchResult throws on a non-200 result envelope instead of returning {} (empty text presented as success). - Media pagination (med): stop after the first page when segments[] are present — layoutNum/layoutStepSize address layout blocks, not media segments, so re-requesting would loop over the same segments to the safety cap. - Table content (med): tables/charts carry content in llmResult, not markdownContent; the layouts→text mapping now prefers llmResult for those types so table content isn't dropped (tables: true was advertised). - Dead code: remove getCurrentMediaParseConfig (referenced non-existent store fields; the single-panel UI reuses pdfProvidersConfig). - Dedup: media MIME list lives only in lib/document/mime.ts (ALIDOCMIND_MEDIA_MIMES); the media registry imports it. - Tests: smoke test asserts durationMs is in ms (>1000 for a ~52s clip) to guard a ms/s unit mismatch; uses allowEnvFallback for the env-cred path. Verified with the real key: PDF + video smoke tests pass; verify classifies valid vs invalid creds correctly. Part of #886. * feat(extraction): extract AliDocMind images to base64 (parity with unpdf/MinerU) AliDocMind embeds figure/picture image URLs inside each layout's markdownContent (markdown ``), not a dedicated field, and the URLs are short-lived OSS signed links. Previously we emitted `images: []` and left the expiring URLs inside the extracted text. Now, for figure/picture layouts we: - parse the OSS image URL out of markdownContent, - fetch it at extraction time (before the signature expires) and re-encode to PNG base64 via sharp — the same base64 `images[]` contract unpdf/MinerU produce, so downstream storeImages → IndexedDB → slide works unchanged, - populate metadata.pdfImages + imageMapping (the generation flow prefers pdfImages), - strip the remote-URL markdown from the emitted text so expiring links don't leak into the prompt. Downloads run concurrently; a failed/`sharp` image is dropped, never failing the whole parse. Also fixes the prior over-broad table handling: only `table` blocks read llmResult; `figure` is treated as an image (chart-figure llmResult still kept in text). Verified with the real key: a sample PDF yields 24 base64 images in both images[] and metadata.pdfImages, and no oss-cn-hangzhou URLs remain in text. Part of #886. * docs(test): note AliDocMind video smoke test is non-deterministic server-side * fix(extraction): correct AliDocMind pageCount (pageNum is 0-based) Verified against a real response: AliDocMind reports pageNum 0..13 and pageCountEstimate 13 for a 14-page document — both are 0-based. The metadata pageCount previously used pageCountEstimate directly, undercounting by one. Use the already-1-based maxPage, falling back to pageCountEstimate+1 only when no blocks were seen. Document the 0-based convention at the normalization site. * fix(extraction): address AliDocMind review — verify/SSRF/config/media (P1+P2) Resolves all P1/P2 findings from the cross-review on #887. P1 (blocking): 1. verifyAliDocMindCredentials now inspects the no-throw response body.code. An OSS-only key returns NoPermission without throwing; previously that was green-lit as "connection successful" then failed at extraction. Only a success/200 or the job-not-found probe code is accepted. Deterministic mocked test added (tests/document/alidocmind-verify.test.ts). 2. verify route trust boundary: managed → server-owned AK/SK + default endpoint only (ignore client values); unmanaged → client creds only, never env fallback, and the client endpoint is SSRF-validated before signing. 3. image fetch hardened: restricted to Aliyun OSS hosts, redirects disallowed, per-image byte cap, image-count cap, bounded concurrency (was unbounded Promise.all over provider-returned URLs). 4. AliDocMind is now selectable in the generation toolbar — availability recognizes the AK/SK pair, not just apiKey. P2 (correctness): 5. Explicit server-config for the AK/SK pair (applyAliDocMindFallback + resolveManagedAliDocMindCredentials); verify and extract now resolve managed/env identically instead of verify-uses-env / extract-rejects. 6. Poll loop checks body.code before status, so a body-level error (e.g. NoPermission) fails fast instead of retrying for the full 15 min. 7. Image page numbers preserved through fetch/filter — no longer hard-coded to page 1, so multi-page image→page association is correct. 8. Empty media extraction (no synopsis/transcript/keyframes) returns 422 PARSE_FAILED instead of HTTP 200 with empty text. 9. Format matrix trimmed to the official contract: images JPG/JPEG/PNG/BMP/GIF (dropped WebP/JP2), media MP4/MKV/AVI/MOV/WMV/MP3/WAV/AAC (dropped M4A). 10. A document-only provider (unpdf/mineru) uploaded with a media file now returns a clear 4xx instead of an opaque 500. P3: credential-verify failures return INVALID_CREDENTIALS 4xx (not INTERNAL_ERROR 500); formatTimestamp emits HH:MM:SS past one hour. Verified with the real key: verify classifies valid→ok and invalid→fail; PDF (24 images, correct page numbers) and MP4 extraction pass end-to-end. * fix(extraction): make AliDocMind selectable + correct analysis label for media Two UI/UX fixes found while manually testing the AliDocMind flow end-to-end: - Persisted-state backfill: add ensureBuiltInPDFProviders so a PDF/document provider added after a user's settings were persisted (AliDocMind) is backfilled into pdfProvidersConfig on rehydrate. Without it the provider never appeared in the store, so it couldn't be selected and never picked up its server-configured flag. Wired into both persist migrate() and merge(), mirroring the existing image/video/web-search backfills. - Analysis step label: getGenerationStepText showed "解析 document 文件" for audio/video (the type map fell through to the literal "document"). Documents keep their precise token (PDF/DOCX/PPTX/XLSX/images); audio/video now use a dedicated, locale-correct string (generation.analyzingMediaMaterial, added to all 8 locales) instead of forcing a format token into the "{{type}} 文件" template. Manually verified end-to-end (real key): PDF, MP4, and MP3 (audio extracted from the sample video) each extract and drive full course generation; a DocMind-restricted AK/SK (OSS-only) now correctly fails verification with NoPermission (400) instead of a false "connection successful". * fix(extraction): address 2nd-round AliDocMind review (managed creds, stream cap, empty verify) Resolves the three follow-up findings on #887: 1. [P1] YAML-managed AliDocMind creds now reach extraction. Both extract paths (document + media) previously cleared the managed AK/SK and relied on an env-only fallback, so a YAML-only deployment verified but failed to extract. They now resolve server-owned creds via the shared resolveManagedAliDocMindCredentials() (env OR YAML), matching the verifier. Regression test added for YAML-only creds with no ALIDOCMIND_* env. 2. [P1] Image download is now size-capped while streaming. Instead of buffering the whole response and checking length afterward, the body is read chunk by chunk with a cumulative byte count that aborts the moment it exceeds the cap — a missing/false Content-Length can no longer exhaust memory. Host allowlist tightened to oss-*.aliyuncs.com. Tests: non-OSS host refusal, no-Content-Length overflow abort, declared-oversize rejection. 3. [P2] Empty verification body no longer counts as success. Removed the codeStr === '' branch from the positive-signal whitelist (a working key always returns the job-not-found business code for the bogus probe id). Regression test added for empty and absent bodies. Rebased onto main (package.json: kept @openmaic/storage + AliCloud deps). Verified with the real key: PDF + MP4 extraction still pass end-to-end. * fix(extraction): merge AliDocMind AK/SK into a baseUrl-configured YAML entry Follow-up to the YAML-managed credential fix. When a YAML `pdf.alidocmind` entry also specifies `baseUrl`, the generic loadEnvSection() (pdf requires a baseUrl) creates the pdf.alidocmind entry copying only apiKey/baseUrl/models/ proxy — never AK/SK. applyAliDocMindFallback() then returned early because the entry already existed, so the provider was "managed" but had no usable credentials, and resolveManagedAliDocMindCredentials() returned undefined (verify + extract both silently lost the creds). Merge the AK/SK into the existing entry instead of returning early. Added a regression test with baseUrl + accessKeyId + accessKeySecret together (the previous test omitted baseUrl, so the generic loader skipped the entry and the bug was masked). * fix(extraction): don't mark AliDocMind managed without AK/SK; align poll code check Two edge cases from a follow-up adversarial pass: - [MED] A YAML `pdf.alidocmind` entry with `baseUrl` but no AK/SK (and no env AK/SK) made the generic loader create the entry → isServerConfigured=true (managed) → but resolveManagedAliDocMindCredentials() returned undefined, so the provider was locked out AND client-entered AK/SK were silently dropped. applyAliDocMindFallback now deletes a credential-less entry so the provider stays UNMANAGED (clients supply their own creds). Regression test added. - [LOW] verify accepted a `code: "success"` body but the extraction poll loop threw on any non-"200" code — a success-shaped status would pass verification then fail extraction. The poll loop now treats "200"/"success" as benign, matching verifyAliDocMindCredentials. --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-07-14 18:12:31 +08:00
3 people	c56929510c	feat(web-search): add SearXNG provider support (#842) ... * feat(web-search): add SearXNG provider support Enable self-hosted SearXNG via SEARXNG_BASE_URL, with server-side provider fallback, settings sync, and tests. Co-authored-by: Cursor <cursoragent@cursor.com> * style: fix Prettier formatting in web-search files Restore pnpm check by applying Prettier to imports, headers, and test assertions. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(web-search): keep SearXNG base URLs server-managed only Close the SSRF primitive by ignoring client-supplied SearXNG URLs at the route boundary and rejecting them in resolveSafeClientWebSearchBaseUrl. Add negative route/config tests and an opt-in live smoke test for the JSON API. Co-authored-by: Cursor <cursoragent@cursor.com> * style: fix Prettier formatting for SearXNG web-search changes Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: 徐松 <song.xu@aieshanghai.com> Co-authored-by: Cursor <cursoragent@cursor.com>	2026-07-13 23:12:31 +08:00
3 people	3b63710376	feat(ai): add Azure OpenAI provider (#916) ... * feat(ai): add Azure OpenAI provider Add deployment-based Azure OpenAI configuration for both client and server-managed setups, normalize Azure portal endpoints, and expose the provider in settings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b0d3a94f-089a-470e-9987-608f1506ca6f * style: format Azure provider type Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b0d3a94f-089a-470e-9987-608f1506ca6f --------- Co-authored-by: MarshellOnMoon <121332425+MarshellOnMoon@users.noreply.github.com>	2026-07-13 23:12:23 +08:00
PhillLittlewoodandwyuc	a2886af92a	Added ComfyUi to has Image Provider (#850) ... * feat: integrate comfyui workflows into image provider selection * style: format code with prettier * fix: align i18n keys for russian locale * feat: implement requested changes and format code * feat: implement requested changes including renaming default workflow --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-07-10 18:49:28 +08:00
Yanpeng Wang	b5e1963e48	feat(document): rename to Document Parsing, show supported formats, extend MinerU support ... Squashed from PR #837 after review approval and green CI.	2026-07-09 04:29:58 -04:00
Yanpeng Wang	d54e62f5c4	feat(token-plan): one-click token-plan setup + deployment usage dashboard (#784) ... * feat(usage): add usage normalization, pricing, model-fetch, balance, storage Foundational layer for token-plan usage tracking (cc-switch-modeled): - lib/usage/normalize.ts: AI SDK v6 usage → four-class token shape - lib/usage/pricing.ts + defaults: per-class USD pricing table - lib/server/model-fetch.ts: /models candidate-URL multi-fallback (ported) - lib/usage/balance-providers.ts: built-in balance queries + detection - lib/server/usage-storage.ts: fire-and-forget jsonl logging to data/usage All pure/storage logic covered by vitest (32 tests). * feat(usage): capture token usage at the callLLM/streamLLM chokepoint - callLLM records result.usage before returning - streamLLM wraps onFinish to record totalUsage on stream completion, preserving any caller-supplied onFinish - provider/model derived from the model instance (no route changes) - fs-backed storage imported dynamically; fire-and-forget, never throws - include_usage is already sent by @ai-sdk/openai, so streaming is covered * feat(usage): add probe-models, balance, and usage API routes - POST /api/provider/probe-models: discover chat models via /models with candidate fallback; SSRF-guarded; filters non-chat ids; 401/404 typed - POST /api/provider/balance: built-in balance detection + billing fallback - GET /api/usage: aggregate jsonl by model/day/source with costIncomplete flag Verified e2e against the live MAIC gateway: 16 chat models (6 filtered), balance detected, and a real callLLM writes a costed usage row. * feat(settings): add token-plan preset picker to provider dialog - lib/config/token-plan-presets.ts: data-driven vendor presets (Huawei/MiniMax/ Xiaomi token plans, OpenRouter/SiliconFlow gateways, DeepSeek/GLM/Qwen/Hunyuan/ Doubao direct) with baseURL, protocol, optional modelsUrl, category - add-provider-dialog: '选择厂商/自定义' tabs; picking a preset auto-fills baseURL+protocol+modelsUrl, custom tab unchanged - ProviderSettings.modelsUrl carries the optional /models override - i18n keys added across all 8 locales Verified in-browser: preset picker renders grouped by category. * feat(settings): add Fetch Models button and balance bar to provider panel - 拉取模型: probes /models, merges discovered ids into the model list (dedupe, keeps manual additions), with success/no-endpoint/auth messages - 查询余额: queries /api/provider/balance, renders a balance bar or a 'check console' hint when unsupported - index.tsx: handleModelsFetched merges probe results into provider config - i18n keys across all 8 locales; removed a now-unused eslint-disable Verified in-browser against MAIC gateway: 16 models fetched, balance shown. * feat(settings): add usage dashboard to System Settings - usage-dashboard.tsx: echarts dual-axis daily trend (tokens + cost), totals cards, by-model table, refresh; reads GET /api/usage - mounted at the top of GeneralSettings (系统设置) - honest disclaimer + costIncomplete marker when a model lacks pricing - i18n keys across all 8 locales Verified in-browser: shows 1 request / 31 tokens / $0.0003 from a prior call. * feat(token-plan): multi-modal one-click setup in System Settings - token-plan-presets.ts: presets now declare per-modality targets (llm/image/video/tts/webSearch); MiniMax is the full-set template, others LLM-only — extend by adding entries (at-our-best adaptation) - apply-token-plan.ts: fills one key into every declared modality via injected store setters, isolating per-modality failures (TDD, 4 tests) - token-plan-settings.tsx: new sidebar page — pick plan, enter key, one-click apply lights up adapted modalities (+LLM model probe), shows 'not adapted yet' for the rest, balance bar reused - reverted add-provider dialog to plain custom form (preset picker moved here) - i18n across all 8 locales Verified in-browser: Token Plan page renders, MiniMax shows all 5 modalities, apply/balance UI wired. 859 tests pass, build clean. * feat(token-plan): add custom token plan entry to Token Plan page - Custom card at the bottom of the provider list: pick it to manually enter name + protocol + baseURL (LLM-only), then key + one-click apply via the same applyTokenPlan flow (mirrors cc-switch's custom provider) - effectivePreset unifies preset vs custom for apply/balance/probe - i18n keys (customGroup/customName/customHint) across all 8 locales Verified in-browser: custom card expands the manual form; preset flow intact. * feat(token-plan): reflect persisted config on the Token Plan page Other settings panels read the store directly, so they survive section switches; the Token Plan page only wrote the store, so it looked blank on return. Now it also reads providersConfig: - selecting a preset prefills its saved API key - configured presets show a '已配置/Configured' badge State persistence was never broken (apply writes zustand→localStorage); this fixes the missing read-back so the page reflects it. * fix(settings): isLLMProviderConfigured crashed on providers without models Root cause of 'token plan config disappears': applyTokenPlan writes a new LLM provider with no models yet (probe fills them later). isLLMProviderConfigured did config.models.length unguarded → threw inside setProviderConfig's resolver → the whole set() aborted → key/baseUrl/type never persisted. - guard models in isLLMProviderConfigured (shared validator; affects any provider written without a models array) - seed models:[] in applyTokenPlan's LLM write for a valid initial shape - regression tests: validator no longer throws; apply+probe write keeps apiKey Verified in-browser: DeepSeek persists key and shows 已配置 after section switch. * feat(token-plan): add remove/teardown for a configured token plan Apply had no inverse. Add removeTokenPlan: clears the API key and disables (enabled:false) every modality the plan declared; a custom LLM provider is deleted entirely (removeProvider), built-ins keep the cleared shape. - removeTokenPlan in apply-token-plan.ts (mirrors applyTokenPlan, isolated per-modality, injected setters; 3 tests) - trash button on configured preset cards in the Token Plan page; resets page state if the removed plan was selected - i18n 'remove' across all 8 locales Verified in-browser: removing DeepSeek empties its key and drops the 已配置 badge. * feat(usage): track multimodal usage, drop cost/pricing Reframe usage stats as pure usage (no cost), per user decision: - usage-storage: drop all cost fields; add kind (llm/image/video/tts/asr) + quantity + unit; LLM keeps token counts, others store quantity - delete lib/usage/pricing.ts + pricing-defaults.json + its test - instrument image/video/tts at the server-only API routes (not in the provider dispatch — those files are in the client graph and importing fs-backed usage-storage broke the client bundle) - /api/usage: aggregate by model/day/modality, no cost - dashboard: per-modality usage (tokens/images/seconds/chars), token-only daily trend, no '$'; i18n cost keys replaced with modality/unit keys - backward compatible: legacy rows (no kind, stray cost fields) read as llm 912 tests pass, build clean. * feat(usage): per-modality dashboard layout + softer dark-mode chart - group usage into per-modality sections (LLM/image/video/tts/asr), each table's usage column uses one consistent unit (token/image/sec/char) — no more mixed units in a single column - summary chips per modality with their own unit - trend chart now plots daily REQUESTS (unit-agnostic, works for any modality) instead of LLM-only tokens - theme-aware chart: faint thin line + soft gradient area, muted axis/grid colors via useTheme — fixes the harsh solid stroke in dark mode Verified in-browser (dark): TTS shows '字符', LLM shows 'Token', separate sections; chart no longer has a hard line. * refactor(usage): dedupe model/fetch/usage helpers, parallel balance probe Review cleanups on the token-plan/usage branch: - extract modelInfoFromId() (shared vision heuristic + ModelInfo shape) - extract fetchWithTimeout() shared by model-fetch and balance-providers - extract recordGenerationUsage() to dedupe the image/tts/video routes - queryBalance: fetch billing subscription + usage in parallel - parseOneApiBilling: report quota without remaining when usage endpoint is unavailable, instead of implying zero spend (full balance) - split the two jammed imports in settings/index.tsx * feat(token-plan): drop custom token-plan support Custom token plans (manual baseURL/protocol entry) added complexity for little gain — a one-off provider is better configured directly on the Providers page. Token Plan is now preset-only: - remove custom mode, manual fields, and the custom card from the UI - collapse effectivePreset back to the selected preset - drop the now-dead removeProvider action + custom-id branch in removeTokenPlan - remove orphaned customGroup/customName/customHint i18n keys (8 locales) * feat(token-plan): add Volcengine/Tencent/Bailian plans, drop balance feature Add three vendor token-plan presets (all map to existing built-in LLM providers, so it's data-only — no new adapters): - 火山方舟 Volcengine Ark → doubao, OpenAI /api/v3 - 腾讯 TokenHub Token Plan → tencent-hunyuan, OpenAI /plan/v3 (the plan-specific base; /v1 is the pay-as-you-go gateway) - 阿里百炼 Token Plan → qwen, cross-model plan (Qwen + DeepSeek/Kimi/ GLM/MiniMax) on one key; model list is probed/entered Remove the balance/quota feature entirely — we now track usage, not cost, and every vendor's balance query needs its own cloud AK/SK + signature (Volcengine SigV4 / Tencent TC3 / Aliyun BSS), which the Bearer-key billing-endpoint probe never supported anyway: - delete lib/usage/balance-providers.ts, /api/provider/balance, its test - strip the Check Balance button + balance bar from the token-plan page and the provider config panel - remove the 4 balance i18n keys across all 8 locales - restore an eslint-disable the branch had dropped in provider-config-panel * feat(token-plan): use cloud-brand logos for vendor token plans The three vendor plans are cloud offerings, not single-model products, so icon them with the cloud brand rather than a model logo: - 火山方舟 → volcengine.svg (was doubao.svg) - 腾讯 TokenHub → tencentcloud.svg (was hunyuan.svg) - 阿里百炼 → alibabacloud.svg (was bailian.svg) Logos are the colored brand variants from lobehub/lobe-icons, matching the existing colored-logo style (plain <img>, no dark:invert needed). * feat(token-plan): keep only MiniMax and Volcengine presets Trim the token-plan list to the two we want to ship: MiniMax (full-set template) and 火山方舟 Volcengine Ark. Drop the Tencent/Bailian plans and the OpenRouter/SiliconFlow/DeepSeek/GLM/Qwen entries. - remove the now-unused tencentcloud.svg / alibabacloud.svg logos (volcengine.svg stays; the other logos are still used by the provider registry) - retarget the LLM-only apply test from the deleted deepseek preset to volcengine-ark * feat(token-plan): restore aggregator/third-party presets Previous commit over-trimmed: the intent was to drop only the Tencent and Bailian token plans, not the OpenRouter/SiliconFlow/DeepSeek/GLM/Qwen entries. Bring those back; keep only Tencent/Bailian removed. - token_plan: MiniMax, 火山方舟 Volcengine Ark - aggregator: OpenRouter, SiliconFlow - third_party: DeepSeek, GLM, Qwen Revert the apply test back to the deepseek fixture (restored). tencentcloud.svg / alibabacloud.svg stay deleted (their plans are gone). * fix(token-plan): point Volcengine plan at the Coding Plan endpoint The plan's ark--prefixed API keys authenticate only against /api/coding/v3, not the general /api/v3 endpoint — the latter rejects them with "The API key format is incorrect", so model probing returned nothing. Switch the base URL to https://ark.cn-beijing.volces.com/api/coding/v3. * fix(token-plan): Volcengine is an Agent Plan (Anthropic /api/plan) Per the Ark Agent Plan docs, the ark--prefixed keys authenticate ONLY against the dedicated Anthropic-compatible base https://ark.cn-beijing. volces.com/api/plan ("其他 Base URL 无法在 Agent Plan 中使用"). The general /api/v3 and the Coding Plan /api/coding endpoints both reject the key as "API key format is incorrect", which is why model probing kept returning 0. - baseUrl → https://ark.cn-beijing.volces.com/api/plan/v1 (the /v1 lets the Anthropic SDK land on /api/plan/v1/messages) - apiFormat → anthropic - rename to 火山方舟 Agent Plan Probe still targets /api/plan/v1/models (the path exists); if the Anthropic gateway doesn't return an OpenAI-shaped list, users fall back to typing a model id like ark-code-latest. * fix(token-plan): Volcengine Agent Plan = OpenAI /api/plan/v3 + ark-code-latest Settled after probing the real key and reading cc-switch's approach: - The ark- plan key works on the OpenAI-compatible /api/plan/v3 endpoint (chat/completions returns 200); switch apiFormat back to openai. - The plan exposes NO /models list (every /api/plan/*/models is 404), which is why probing kept returning 0. cc-switch handles this by hardcoding a single ark-code-latest (an auto-routing alias valid on any tier) and does NOT use AK/SK for model listing — so we do the same. - Seed defaultModels: ['ark-code-latest'] only; users add specific ids by hand. Supporting machinery (kept, general-purpose): - applyTokenPlan seeds models from defaultModels instead of wiping to [] - handleApply uses defaultModels and skips the doomed probe when present - drop stray .playwright-mcp/ debug artifacts and gitignore them * style: fix prettier formatting in usage files CI runs prettier on the whole repo (prettier . --check); these four files predate this branch's formatting pass and tripped the check. * feat(token-plan): verify Volcengine Agent Plan's published model set The Agent Plan publishes a fixed model set but exposes no /models endpoint, so carry the documented models as CANDIDATES and verify each on apply: - add verifyModels flag to TokenPlanModalityTarget - new /api/provider/probe-chat-models route: sends a minimal chat request per candidate (OpenAI /chat/completions or Anthropic /messages) in parallel, returns the subset that succeeds; SSRF-guarded, auth-failure short-circuits - handleApply gains a verify branch (before the fixed-defaultModels fast path), falling back to the seeded list if verification fails - Volcengine preset now carries the 12 published Agent Plan text models (doubao-seed-2.0-*/deepseek-v4-*/minimax-m*/glm-5.2/kimi-k2.*) as candidates This auto-prunes retired (docs flag deepseek-v3.2/glm-5.1 as 即将下线) and tier-gated models without code changes. Verified all 12 resolve against a real plan key. * feat(token-plan): wire Volcengine Agent Plan image + video modalities Make the Ark seedream/seedance adapters path-configurable and light up the image/video modalities on the Volcengine plan: - seedream/seedance adapters: resolveArkRoot() uses baseUrl verbatim when it already carries an /api/... path (token plan's /api/plan/v3), else appends the standard /api/v3 — no regression for the pay-as-you-go default host. - applyTokenPlan: image/video branches inject a modality's defaultModels as customModels and set them as the active provider+model, so generation works out of the box. New optional setImageProvider/ModelId + setVideoProvider/ ModelId actions (UI passes the store setters; tests omit them). - Volcengine preset declares image (doubao-seedream-5.0-lite, verified 200 on /api/plan/v3/images/generations) and video (doubao-seedance-2.0/1.5-pro — Medium+ tiers only; lower tiers reject at call time, no code change needed to upgrade). Applying the plan overwrites the shared seedream/seedance slot with the plan config (same overwrite model as LLM); switching back to pay-as-you-go is a manual edit or plan removal. Verified image end-to-end with a real plan key. * feat(token-plan): verify image/video models on apply, disable unsupported tiers The Volcengine plan lit up video optimistically, but lower tiers (Small) don't include video — so using it 404'd with UnsupportedModel. Probe media models on apply and only keep what the tier actually supports: - generalize /api/provider/probe-chat-models with a `kind` (chat|image|video): image hits /images/generations, video hits /contents/generations/tasks with empty content. The model-support check (404 UnsupportedModel) runs before any billable work, so probing never starts a real image/video job; for media, "supported" = any non-404 response. - handleApply: after lighting up image/video, probe each verifyModels modality; prune to the verified model set + re-select a working model, or disable the modality entirely if none pass (no false "available"). - Volcengine preset: image/video targets gain verifyModels: true. - add settings.tokenPlan.tierUnsupported across 8 locales. Verified with a real Small-tier key: image (seedream-5.0-lite) passes and is kept; video (seedance-2.0/1.5-pro) 404s and is disabled. * feat(web-search): add Doubao (豆包搜索) provider Doubao Search (Custom 版) over its REST endpoint POST open.feedcoopapi.com/search_api/web_search with Bearer auth — the same endpoint the askecho-search-infinity MCP server wraps, so the Volcengine Agent Plan key authenticates directly. Mirrors the MiniMax adapter: maps Result.WebResults to WebSearchSource (prefers Summary, the query-relevant excerpt, over Snippet for LLM use) and surfaces errors from ResponseMetadata.Error. - register 'doubao' in WebSearchProviderId + WEB_SEARCH_PROVIDERS - searchWithDoubao adapter, searchWeb dispatch, store default config - SSRF allowlist entry for the search host * feat(audio): support Agent Plan single-key auth for Doubao TTS generateDoubaoTTS now picks auth + endpoint from the key shape, since Volcengine exposes Seed-TTS as two products with separate credentials (verified: a plan key 401s on the normal endpoint, and the plan endpoint rejects appId-style auth): - single key (no colon) -> X-Api-Key, for the Agent Plan /plan endpoint - appId:accessKey -> X-Api-App-Id + X-Api-Access-Key (unchanged) A malformed pair (empty half) fails clearly instead of sending an empty header. Reuses the existing NDJSON/base64-mp3 parsing and voice list. * fix(media): map MiniMax video 720p to its real 768P tier normalizeVideoOptions defaults minimax-video to '720p' (the first supported resolution), but Hailuo 2.3 only accepts 768P/1080P and rejects 720P with '2013 ... does not support resolution 720P'. MiniMax's mid tier is 768P, not 720P (the adapter already falls back to 768P, as does the connectivity test), so map the shared enum's '720p' to 768P. Regression tests lock the mapping. * feat(token-plan): add web search + TTS to Volcengine Agent Plan, widen image tiers Extend the volcengine-ark preset now that the adapters exist: - webSearch -> doubao (own host open.feedcoopapi.com, not the ark endpoint) - tts -> doubao-tts on the /api/plan/tts endpoint (single-key auth) - image defaultModels widened to a best-first Seedream 5.0/4.5/4.0 list so a higher tier keeps the strongest model while verifyModels prunes the rest; video keeps the 2.0 + 1.5-pro candidates Comments record the verified host/auth quirks of each modality. * feat(token-plan): show result panel only after probing, with two clear states Addresses review feedback that a green check implied generation works when it only meant 'configured'. The panel now renders after probing finishes (gated on results && !applying) so it reflects the final set, and uses two states: green when the modality is configured/usable, muted when a live probe proved it unavailable (e.g. video on a tier without it). * feat(token-plan): scope presets to true multi-modal token plans Drop the single-modality LLM presets (OpenRouter, SiliconFlow, DeepSeek, GLM, Qwen) from Token Plan. A token plan's defining trait is one key spanning many modalities; those entries are ordinary LLM API providers already covered by the add-provider flow, and listing them here muddied the 'one key, every modality' promise. Only MiniMax and the Volcengine Ark Agent Plan remain. The UI already hides categories with no entries. apply-token-plan's LLM-only test now uses a local fixture instead of the removed deepseek preset. * feat(token-plan): progressive reveal of probe results on apply The result panel previously rendered all at once after probing finished, reading as dead air during the model probe. Now rows appear immediately on Apply: modalities with a live probe in flight show a spinner ('pending') and resolve to lit/failed independently as each probe returns, while non-probe modalities show lit right away. Probes run in parallel (Promise.all) instead of sequentially. A row only turns green once its own probe confirms, so this reveals structure + live progress without a premature green — complementing the earlier 'render only after probing' intent rather than reverting it. Per review feedback from @wyuc on #784. * fix(token-plan): enrich seeded models with built-in thinking capability Token Plan built ModelInfo objects from probed ids via modelInfoFromId(), filling only streaming/tools/vision — so a model that supports configurable thinking lost capabilities.thinking and InlineThinkingControl was hidden. modelInfoFromId now takes an optional providerId and overlays the catalog thinking capability for that (provider, model) pair; applyTokenPlan does the same for its synchronously-seeded list. Added the Ark Agent Plan's dotted aliases to the metadata table: - native Doubao Seed 2.0 family (doubao-seed-2.0-pro/code/lite/mini) - cross-vendor models the plan serves through its OpenAI-compatible endpoint (deepseek-v4-pro/flash, glm-5.2, kimi-k2.7-code/k2.6, minimax-m3/m2.7, ark-code-latest) All verified against a live plan key: each accepts the gateway's unified reasoning_effort field (low/medium/high) and actually reasons. They share the doubao effort adapter, which disables via 'minimal' (not 'none') — matching what the plan endpoint accepts (it rejects reasoning_effort:'none'). Addresses review point #1 from @wyuc on #784. * Improve token plan capability setup UI * fix token plan setup flow * chore: prettier format tts-providers.ts	2026-07-03 11:03:12 +08:00
wyuc	1d1ce80e04	feat(maic-agent): editor-agent line (v0) — Pro-mode "Edit with AI" (#777) ... Promote the editor-agent line to main: read-then-act editor agent (read_scene_content, regenerate_scene, regenerate_scene_actions, edit_interactive_html), a routable maic-agent model stage, the agent sidebar UI, interactive-scene runtime-error capture, and the timeline/script editor. Cross-reviewed and smoke-verified; see #777.	2026-06-23 16:56:37 +08:00
3 people	3e1e9f083b	feat(tts): unify provider-enablement model, default browser-native off (#665) (#666) ... A single TTS provider-enablement model honored everywhere (picker, auto-assignment, fallback, lecture/playback, server). Model - lib/audio/provider-enablement.ts: isTTSProviderConfigured / isTTSProviderEnabled / listEnabledTTSProviderIds. Availability drops the registry defaultBaseUrl fallback (fixes Lemonade's always-shown voices), honors the per-provider `enabled` flag, server-disable overrides (server precedence). - voice-resolver: getEnabledProvidersWithVoices (enabled server/custom providers) and getSelectableProvidersWithVoices (the ONE client source of truth = enabled providers + opt-in browser-native when the browser exposes voices). resolveAgentVoice picks deterministically among the given providers, honors a voiceConfig only while its provider is present. - Picker + discussion build their list from getSelectableProvidersWithVoices, so the AgentBar and the live discussion agree on selectable voices (fixes student agents going silent in auto mode). Students resolve via resolveAgentVoice; the teacher uses the global lecture voice verbatim (same voice the pre-generated lecture sends), so lecture and discussion never diverge. - Voice assignment vs fallback: an agent's voice is bound at generation (the agent-profile LLM picks from availableVoices; preset uses the user's pick). The fallback for a missing/invalid/unavailable voice is applied only at the live TTS calls, not baked into the saved agent — so it reflects current provider availability. Defaults / opt-in - Browser-native: first-class, default OFF, opt-in. Built-in providers default enabled:true (only surface once configured). - TTS master toggle defaults OFF; first server-sync auto-enables it when a provider is configured (mirrors image/video). - Persist v3->v4 migration normalizes the never-user-editable flags. Server/admin disable (server precedence): server-providers.yml `enabled:false` and env `TTS_<P>_ENABLED=false` (incl. browser-native). Surfaced as a per-provider `disabled` flag on /api/server-providers; merged as serverDisabled; enforced at the /api/generate/tts trust boundary, classroom media generation, and /api/health. UI - Settings TTS panel: per-provider enable toggle, disabled (with a hint) when the provider is unavailable, locked off (with a note) when server-disabled. - AgentBar voice pills render disabled with a "No voice" placeholder when no provider is enabled. The toolbar media-popover TTS tab keeps its existing on/off toggle (no picker/CTA). Tests: predicates, resolver determinism + unified selectable list, server force-disable, store merge + migration + TTS auto-enable, i18n parity. Verified in a real browser (Playwright): settings toggle states + a full Gemini classroom generation whose TTS requests used the picked voice. Found on main @ 2586cb5. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>	2026-06-07 17:25:57 +08:00
3 people	35d36905d1	Make server-configured providers admin-managed, not client-overridable (#620) (#624) ... * refactor(providers): make server-configured providers admin-managed, not client-overridable A server-configured provider is now authoritative: the operator owns its API key and base URL, and the client cannot override either. This collapses the tri-state (server key / client base URL / client key) that was the root of a recurring "missing API key" bug class — when the settings UI echoed a base URL with an empty key, routes that treated "any base URL ⇒ fully client-configured" dropped the server key. #533 patched that per-route via canUseServerApiKeyForBaseUrl; this removes the guard entirely by fixing the design. Server: - resolve*ApiKey/BaseUrl ignore client values when the provider is server- configured; add isServerConfiguredProvider(section, id); drop canUseServerApiKeyForBaseUrl + normalizeProviderBaseUrl. - getServer*Providers() no longer return base URLs — only allowed models (LLM/image) and the managed flag (presence) — so an internal gateway/proxy endpoint is never exposed to the client. - routes (chat/image/video/tts/asr/pdf + verify-*) resolve managed providers purely server-side; SSRF validation now applies only to unmanaged (client) base URLs. Client: - drop serverBaseUrl from ProviderSettings, the settings store, request-building (TTS base URL no longer echoes the server URL) and voice resolution. - settings UI renders managed providers read-only: the key/base-URL override inputs and region toggles are hidden, leaving the "managed by server" notice. Tradeoff: overriding a server provider's key with your own for the same provider is removed; add a separate custom provider instead. Closes #620. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers): refresh managed-provider copy, drop dead audio-settings, add #620 e2e - serverConfiguredNotice now says the provider is managed and not editable here, pointing users to add a separate provider for their own credentials (7 locales). - delete components/settings/audio-settings.tsx — unreferenced dead code. - add e2e/tests/managed-provider-620.spec.ts: managed provider hides the key/base-URL inputs; unmanaged provider keeps them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(providers): lock the model catalog for managed providers with a pinned list When the operator pins an allowed model list (MODELS env/yaml ⇒ serverModels), the model catalog is admin-managed too: the LLM provider panel shows the pinned models read-only with a "Server" badge and no add / reset / edit / delete affordances. Without a pinned list the server manages only credentials and the user keeps curating models. This removes the prior inconsistency where the settings panel let you add models to a pinned managed provider that the generation toolbar then filtered out. e2e: the managed-openai case now also asserts the catalog is locked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers): ignore (not reject) client overrides for managed web-search; clear legacy serverBaseUrl Addresses review feedback on #624: 1. Managed web-search providers now ignore a client-sent key/baseUrl instead of validating and rejecting it. A server-configured provider is admin-owned, so the server config is authoritative and a stale client base URL is dropped rather than 400-ing the request. The allowlist validation still runs for unmanaged providers, where the client base URL is actually used. 2. Drop the removed `serverBaseUrl` field from persisted localStorage on rehydrate (settings version 2 -> 3, stripped in both migrate and merge), so old clients don't keep a stale server URL in client state. Tests: web-search route test split into unmanaged-rejects-allowlist and managed-ignores-client-baseUrl; settings rehydrate test asserts a legacy serverBaseUrl is stripped. Full suite 381 pass; e2e 2 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>	2026-05-31 21:28:22 +08:00
wyuc	f064590a5e	test+fix(orchestration): rip maxTurns; restore answer-the-user's-question behavior (#598 follow-up) (#599) ... * test(orchestration): add premature-END regression eval (#598 follow-up) Adds eval/orchestration/ following the outline-language pattern: a runner that A/B-tests the director against the same scenario with two prompt variants — current main (post-#554) and a synthesised pre-#554 baseline that strips the role-aware summary labels AND the new system.md rules 10/11/12 together. Pass criterion is now framed as a regression guard rather than a fixture discrimination test: every scenario's post-fix END rate must stay below EVAL_END_THRESHOLD (default 20%). The pre-vs-post Δ is reported as informational data, since #554's reviewer feedback was that earlier fixtures didn't discriminate. Empirical finding across 7 shipped model configs (gpt-4.1-mini, gpt-4o-mini, gpt-5.4-nano, qwen-plus, qwen3.5-flash, deepseek-chat, deepseek-v4-flash, gemini-2.5-flash, claude-haiku-4-5) with 5 scenarios modelled on #511 (incl. the exact Tiananmen-3D objection trace, soft pushback after a long resolved-looking discussion, topic pivot, brief acknowledgement, and explicit teacher-signals-end-then-user-objects): no scenario produced a non-zero END rate in either variant. Useful data for the #554/#598 discussion — the prompt-layer rules don't measurably change behavior on these shipped models with these prompts, but the eval is now wired so future regressions show up. * test(orchestration): add director question-answering eval (#598 follow-up) Adds an A/B eval probing whether the director routes correctly when the conversation contains an unanswered user question — both in the first-turn drift case (agents started answering but drifted onto adjacent topics, no frustration yet) and in the escalated frustration case (user has already complained multiple times). The prod symptom this targets ("答非所问"): agents reply on adjacent topics rather than answering the literal question, and the director keeps picking peer agents for "variety" instead of routing back to the teacher to actually answer. Scenarios: 5 synthesized cases across math, biology, English grammar, physics, and calculus. 3 are first-turn drift (no frustration yet), 2 include frustration signals. Each captures: user asks specific question, agents reply on adjacent topics, director is asked to pick the next move. Decision rule-judged into USER | TEACHER | OTHER_AGENT | END: - USER → ✓ valid (cue user to clarify) - TEACHER → ✓ valid (re-route to teacher to answer) - OTHER_AGENT → ✗ wrong (peer-agent "variety" routing — the bug) - END → ✗ wrong A/B: - baseline : current director template with rule 13 stripped - with_rule : current director template as-shipped Pass = with_rule.correctRate ≥ EVAL_PASS_THRESHOLD (default 0.7). Pre-vs-post Δ is reported as informational only. Note this commit only adds the eval; the rule it strips/restores in the A/B is injected in the next commit. * fix(orchestration): inject director rule for answering user questions Adds rule 13 to lib/prompts/templates/director/system.md so the director recognizes when the conversation contains an unanswered user question and routes appropriately — to the teacher (literal role match) by default, or to USER cue when the question is ambiguous or the user has already complained multiple times. This addresses the "答非所问" symptom: when agents drift onto adjacent topics, the director keeps picking peer agents for "variety" rather than routing back to the teacher to actually answer. The rule is preventive (applies whenever the question is unanswered, including the first turn after drift) rather than reactive-only (waiting for the user to express frustration). Rule wording uses constrained-output framing: literal `role: teacher` match (not "best fit" / "highest priority" / "most knowledgeable"), and an explicit list of forbidden choices (peer agents, END). Earlier softer wordings let the model pick the assistant when it seemed more topically relevant; the literal-match wording prevents that. Validated by the question-answering eval added in the previous commit. On google:gemini-3-flash-preview, 5 samples per variant across 5 synthesized scenarios: | Scenario | baseline | with_rule | |----------------------------------------------------|----------|-----------| | math_quadratic_axis_drift_no_frustration | 100% | 100% | | bio_dark_reaction_drift_no_frustration | 80% | 100% | | english_team_isare_drift_no_frustration | 20% | 100% | | physics_inertial_mass_escalated_frustration | 0% | 100% | | calculus_product_rule_drift_no_frustration | 60% | 100% | | **mean** | **52%** | **100%** | 3 of 5 scenarios reproduce the bug at baseline (correct rate ≤ 60%); the rule lifts every scenario to 100%. The remaining 2 baselines are already high — the rule doesn't regress them. * fix(orchestration): teach agents to answer the user's literal question Adds an "Answering the User's Question" section to the agent system prompt. Before this, the agent SP had no rule for direct-answering — the persona, role guideline, current-state context, and length guidelines all pushed the agent to advance the lesson and inspire thought, with no explicit directive to first answer the user's literal question. The result observed in prod ("答非所问"): when the user asks a specific question (a formula, a yes/no, a term), agents reply with adjacent concepts, examples, or follow-up questions — but never the literal answer. The new section directs the agent, on every response where the user message contains a question, to: - lead with the concrete answer to the literal question, - not pivot to an adjacent topic even when it seems pedagogically richer, - treat "inspire thought" and peer-differentiation guidance as applying only AFTER the literal question is answered, - say "I don't know" directly when uncertain rather than answering a different question, - on frustration signals from prior turns, look back at the original question (before the frustration) and answer THAT specifically. This is the agent-side counterpart to the director rule landed in the previous commit. Director routes to the teacher when the question is unanswered; the teacher now has explicit guidance for what to do when dispatched. * refactor+fix(orchestration): rip maxTurns end-to-end; distinguish errors from completion Two coupled cleanups in one commit since the outcome-handler rewrite naturally falls out of removing the client-side max-turns loop. The director graph used `maxTurns: turnCount + 1` to force "one director→agent cycle per request" — a hack expressing the single-round contract as a turn limit. This commit removes the hack and the surrounding user-facing setting, replacing implicit intent with explicit graph topology. Server: - `director-graph.ts`: drop `maxTurns` from state, drop the hard-cap check at the top of `directorNode`, change `agent_generate → director` edge to `agent_generate → END`. Single-round contract is now expressed by topology. - `lib/types/chat.ts`: drop `maxTurns` and `currentTurn` from `SessionConfig`; drop `maxTurns?` from `CreateSessionRequest.trigger`. Client: - `lib/store/settings.ts`: drop `maxTurns` state, `setMaxTurns` action, the `'10'` default, and the localStorage migration path. - `components/agent/agent-bar.tsx`: drop the entire compact-stepper UI block plus the unused MessageSquare/Minus/Plus lucide imports. - `components/settings/agent-settings.tsx`: drop the AgentSettings prop and the multi-agent-only Input block. - `lib/chat/agent-loop.ts`: drop `maxTurns` parameter, change `while (turnCount < maxTurns)` to `while (true)` controlled by inner exits, drop `'max_turns'` reason from `AgentLoopOutcome`. - `components/chat/use-chat-sessions.ts`: drop the maxTurns lookup, drop the trailing arg to runAgentLoop, drop vestigial `maxTurns: 0` / `currentTurn: 0` from session creation. - `eval/whiteboard-layout/runner.ts`: drop unused MAX_AGENT_TURNS plus the trailing arg. i18n: drop `settings.maxTurns` and `settings.maxTurnsDesc` from six locales (en-US, zh-CN, zh-TW, ja-JP, ar-SA, ru-RU). Motivation for removing the user-facing setting: the LLM director controls round length via cue_user / END decisions, so a numeric "max discussion turns" knob is redundant with rule-driven routing. When the director prompt was buggy, the cap masked the bug by force- ending; once director routing is fixed, the cap just removes user agency. Before, every non-`cue_user` outcome of the agent loop mapped to `status: 'completed'` in `use-chat-sessions.ts`. That conflated four distinct paths: - `end`: director chose END (legitimate completion) - `empty_turns`: two consecutive agents returned no text or actions (parser failure, model timeout, etc.) - `no_done`: SSE stream ended without a `done` event (infrastructure failure) - `aborted`: user pressed stop (handled elsewhere) The new switch routes `end` to completed, `empty_turns` and `no_done` through `clearLiveSessionAfterError` so the user sees an inline System message describing what went wrong instead of a quiet "completed". - `lib/types/chat.ts`: add `'error'` to `SessionStatus`. - `components/chat/use-chat-sessions.ts`: rewrite the outcome handler as a switch; have `clearLiveSessionAfterError` set `status: 'error'` so all entry points (the runAgentLoop outcome AND existing throw paths from fetchChat / SSE error events) converge on the same visible state. - `components/chat/session-list.tsx`: add `AlertCircle` red icon for the new error status. - i18n: add `chat.error.emptyAgentResponses` and `chat.error.streamInterrupted` strings to all six locales. This is the in-product fix that the prod investigation surfaced as an independent root cause of the #511 "discussion ended unexpectedly" symptom — separate from the director and agent prompt fixes in the earlier commits of this PR. * fix(chat): drop dead settingsState binding; wire eval:orchestration:answering script Two minor CR follow-ups: - components/chat/use-chat-sessions.ts: the `settingsState` lookup at the top of runAgentLoopFn was the entry point for the maxTurns lookup ripped in the previous commit; with maxTurns gone, the binding is unused and ESLint flags it. Drop the line. (Other uses of useSettingsStore.getState() elsewhere in this file are unaffected.) - package.json: the answering eval runner had no convenience script, while the premature-end runner did. Add eval:orchestration:answering for symmetry and discoverability. * fix(orchestration): route ambiguous/unaddressed user turns to teacher, never USER cue Addresses reviewer issues 2 & 3 (ambiguous "帮我看一下这个" / "Can you take a look at this?"): the director was cueing USER for vague input, which makes no agent speak — the user faces dead air with nothing to react to ("nobody responds"); and when an agent did speak it guessed a topic instead of asking what the user meant. Root cause was the previous rule 13, which offered `{"next_agent":"USER"}` as a valid response to an ambiguous user turn. A USER cue is the wrong tool here: it returns control to the user without any agent speaking. The right behavior is to dispatch the teacher, who either answers or — if the message is too vague — asks a clarifying question out loud. Director (`director/system.md` rule 13): - When the most recent user turn is unaddressed (clear question, vague request, OR frustration signal), the only acceptable output is the literal `role: teacher` agent id. - USER cue is now explicitly forbidden as a response to a user turn (reserved for rule 7, where an agent has just asked the user something). - Tightened the END prohibition: a high turn count or a thoroughly-discussed topic does NOT mean the specific question was answered. Added rule 5 (don't-drag-on) to the override list, since that pressure was driving the director to END on well-discussed-but-unanswered turns. Agent (`agent-system/system.md`): - Added a clause to "Answering the User's Question": if the user's message is too vague to answer (no clear referent), ask ONE short clarifying question with a concrete option or two — do not guess a topic and lecture, do not stay silent. Eval (`answering-runner.ts` + `scenarios/answering.json`): - Reclassified USER as wrong (dead air); only TEACHER counts as correct now (teacher answers, or asks a clarifying question for vague input). - Added two ambiguity scenarios: a bare "帮我看一下这个" opener, and a vague "再帮我看看这个吧" follow-up after a real Q&A. - Made the rule-13 strip regex match by rule number rather than heading text, so it survives heading rewrites. On google:gemini-3-flash-preview, 10 samples per variant, all 7 scenarios PASS (with_rule mean ~97%, baseline ~57%). The two ambiguity scenarios: ambiguous_look_at_this 100%/100%, ambiguous_vague_followup 20%→100%. Manually verified the agent side: for "帮我看一下这个" the teacher now replies "…你想讨论的是具体的代码逻辑，还是 Agent 的决策流程图呢?" (clarifying question + concrete options) instead of guessing. Note: reviewer issue 1 (follow-up not answered) could not be reproduced on this branch across 7 configurations; it is consistent with the deployed environment running older code and/or provider API flakiness, which the C7 error-vs-completion distinction in this PR already surfaces instead of a silent 已结束. * docs(eval): fix stale physics scenario description after USER→wrong reclassification The physics_inertial_mass scenario description still said "USER-cue signal — but TEACHER re-answer is also valid", which contradicts the new classifier (USER is dead air = wrong; only TEACHER is correct). Cosmetic only — the description field is printed into the report, never used in classification.	2026-05-29 10:52:30 +08:00
Ismail Ariyan	07115dff17	feat(asr): add Azure STT (Fast Transcription) provider (#175) ... * add Azure ASR config and localization * add transcribeAzureASR func * fix(settings): prevent Azure STT crash in ASR modal * feat(i18n): add Azure STT provider support in language locales	2026-05-27 22:38:49 +08:00
wyucandClaude Opus 4.7	d9aecf8052	fix(settings): enforce "usable provider ⇒ concrete model" invariant (#580) (#581) ... * fix(settings): enforce "usable provider ⇒ concrete model" invariant (#580) Layer 1 — resolve provider AND model atomically at every source: - add resolveSelectedModel(): like validateModel but no empty-input short-circuit, so it returns '' only when the provider has no models - fetchServerProviders: symmetric LLM/image/video provider recovery + resolveSelectedModel; remove the asymmetric #577 isServerConfigured-gated LLM tail and the separate first-load auto-select block (now subsumed, and covers client-API-key providers too) - setProviderConfig: entering an API key for the active/unset LLM provider resolves a concrete model in the same set() - setVideoProvider: resolve a model on switch (was id-only); setImageProvider routed through the shared resolver Layer 2 — remove states unreachable under the invariant: - add shared isLLMProviderConfigured / hasUsableLLMProvider; gate the landing-page generate button on a usable provider (state A vs B) and drop the modelNotConfigured toast + forced SettingsDialog - generation-toolbar: use the shared predicate; drop the dead "Select Model" label/tooltip fallback - delete dead components/settings/model-selector.tsx (zero refs) and the now-orphan settings.selectModel i18n key (6 locales) Note: generation-toolbar's `providerEntries.length === 0` branch is KEPT — it is reachable via search (search-no-results), not the dead no-provider state the issue assumed. modelNotConfigured i18n key kept (still used by the chat path in use-chat-sessions). Tests: resolveSelectedModel + hasUsableLLMProvider unit tests; server-sync invariant tests (client-key resolution, atomic API-key entry, provider switch, serverModels-preference + first-load regression guards). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(settings): address code review — symmetric image/video config resolution Code review Important #1: fold atomic model resolution into setImageProviderConfig / setVideoProviderConfig (design doc Layer 1 step 2 scoped these). Deleting the selected custom image/video model on the active provider now resolves to a valid model in the same set() instead of leaving it stale until the next server sync — restores LLM/image/video symmetry. Minor #2/#3: document the deliberate store-level (credential path) vs UX-level (incl. ≥1 model) "usable" split on isProviderUsable; clarify that setProviderConfig also adopts providerId on the first-load API-key path. Tests: +2 invariant tests (delete selected custom image/video model → falls back to a valid model). 89/89 store tests pass, tsc clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(settings): keyless providers need explicit baseUrl to count as usable (#580) Code-review/e2e follow-up. ollama/lemonade ship requiresApiKey:false + a registry defaultBaseUrl, so the old isLLMProviderConfigured counted them as configured on a fresh install — but buildFallback/reconcile never auto-selects keyless providers, so hasUsableProvider could be true while modelId stayed '', re-enabling generation with no model (the exact #498/#580 class of bug, just via the keyless path). Narrow isLLMProviderConfigured to be consistent with isProviderUsable: keyless ⇒ usable only once the user sets an explicit baseUrl (registry defaultBaseUrl alone is not user intent); server-config and key-requiring paths unchanged. Now gate + toolbar + reconcile agree, and genuine State A (no usable provider) is reachable again. Add e2e/tests/model-invariant-580.spec.ts: State A (no provider → generate disabled + single "Set up model" affordance, no toast/forced dialog) and State B (server-configured → concrete model auto-resolved, generation enabled). +3 keyless unit tests. 92/92 store tests + 2 e2e pass, tsc clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(settings): re-resolve selection on bulk provider-config change (#580) Self-test found: deleting the selected provider in Settings left the model selector pointing at the deleted/invalid provider (or, via the component's hand-rolled fallback, at an unusable built-in like openai with no key). Root cause: setProvidersConfig (the bulk setter behind delete-provider / delete-model) did ZERO re-resolution — Layer 1 covered setProviderConfig (singular) + image/video setters + the reconcile, but not this path. The component's confirmDeleteProvider then picked Object.keys()[0] (always a built-in, ignoring usability), the exact per-call-site hand-rolled shape #580 condemns. Fix at the source: setProvidersConfig now re-resolves (providerId, modelId) via the canonical isLLMProviderConfigured + resolveSelectedModel — keep the current provider if still usable, else first usable, else State A. Removed the brittle confirmDeleteProvider override (kept only its local Settings-tab selection). Now every bulk config mutation (delete, import, reset) holds the invariant. Tests: +2 store tests (delete selected/only-usable provider → State A, not the deleted/invalid one; delete non-selected → keep usable selection). 94/94 store tests, full suite green (only pre-existing ssrf DNS), tsc/eslint clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(settings): one shared LLM-selection resolver for all config edits (#580) Clarified repro: the realistic case isn't deleting a custom provider — it's clearing the SELECTED provider's API key so it becomes invalid. setProvider Config only re-resolved when the edit made a provider *usable*; the "became unusable" transition fell through (return base) → the now-invalid provider stayed selected. The previous commit fixed only the bulk path. Root-cause fix: extract one shared `resolveLLMSelection(config, providerId, modelId)` — keep current provider if still usable, else first usable, else State A; then resolve the model. Use it in BOTH setProviderConfig (single edit: key add/clear, model-list edit) and setProvidersConfig (bulk: delete, import, reset). The two paths can no longer diverge — exactly the per-call- site asymmetry #580 set out to eliminate. Drops the now-unused isProviderUsable import. Tests: +2 store tests (clear selected provider key → drops stale selection; clear non-selected key → keeps usable selection). 96/96 store tests, full suite green except pre-existing ssrf DNS; tsc + eslint clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * style: prettier-format #580 test specs CI 'Lint, Typecheck & Unit Tests' ran prettier . --check and failed on the two added test specs (only tsc/eslint/vitest were run locally). Pure whitespace reformat, no logic change — 96/96 store tests still green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-18 16:04:23 +08:00
Yizuki_Ame	47cc2a510c	feat: add Brave Search and Baidu Search integration ... Add Brave Search and Baidu Search providers with Baidu sub-source support, provider configuration updates, tests, and docs links.	2026-05-13 20:28:56 +08:00
Helckandxuruiray	46b61dedfd	feat: add HappyHorse video adapter (#509) ... Add Alibaba Cloud DashScope HappyHorse text-to-video support as an OpenMAIC video generation provider. Co-authored-by: xuruiray <15962843+xuruiray@users.noreply.github.com>	2026-05-09 10:07:31 +08:00
3 people	be759a8252	feat: add lemonade to provider (#508) ... * feat: add lemonade to provider * fix(lemonade): correct TTS/ASR model IDs and expand Kokoro voices - TTS model: kokoro → kokoro-v1 - ASR model: single 'whisper' → 6 Whisper variants (Base/Large-v3/Large-v3-Turbo/Medium/Small/Tiny), default Whisper-Base - Expand lemonade-tts voices to full Kokoro v1 set (54 voices across en-US/en-GB/zh-CN/ja-JP/es/fr-FR/hi/it-IT/pt-BR) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lemonade): zh-TW i18n parity, real Kokoro default voice, language code consistency - zh-TW.json: add missing providerLemonadeTTS/ASR/Image keys - TTS default voice: 'default' → 'af_heart' (Kokoro expects real voice ID, not placeholder) - Voice language codes: 'es'/'hi' → 'es-ES'/'hi-IN' for consistency Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(lemonade): add unit tests for image/TTS/ASR adapters and wav-utils Covers: - lib/media/adapters/lemonade-image-adapter: request shape, base URL fallback, custom model, optional Bearer auth, error paths, connectivity probe - lib/audio/tts-providers (lemonade-tts case): request body (model/voice/format), af_heart fallback, optional auth, error path - lib/audio/asr-providers (lemonade-asr case): WAV-only validation, model/language form fields, "audio is empty" graceful empty result, error propagation, default model fallback - lib/audio/wav-utils: isWavBlob detection by MIME and filename, normalizeASRUploadAudio pass-through and wav handling Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lemonade): remove text path fork and harden fallbacks * style: format lemonade tts files * i18n: add lemonade voice language labels * fix(lemonade): align image default model and harden wav detection * test(lemonade): fix File BlobPart typing * fix(lemonade): simplify tts voice selection and clear asr test state * docs: highlight Lemonade support --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-05-08 13:53:37 +08:00
wyuc	dbe1d2ec76	[codex] Add Bocha web search provider (#524) ... * Add Bocha web search provider * Address Bocha web search review feedback	2026-05-04 12:06:05 +08:00
wyuc	5e12ad892c	[codex] add VoxCPM2 TTS provider (#496) ... * add voxcpm2 tts provider * address voxcpm review feedback * translate voxcpm locale strings * narrow toolbar model pill * refine toolbar media controls * address voxcpm review followups	2026-04-26 23:45:23 +08:00
wyucand杨慎	e613b7578f	[codex] add per-model thinking config (#494) ... * add per-model thinking config * Refine thinking model controls --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>	2026-04-26 19:41:10 +08:00
wyuc	4754bbae6d	support gpt-5.5 api (#487)	2026-04-25 14:37:22 +08:00
wyuc	a95aa87bf1	support recent model providers (#481)	2026-04-24 15:29:12 +08:00
3 people	2f542cf33f	feat(providers): add international base URL chips for GLM/Kimi/MiniMax (#449) ... * feat(providers): add international base URL chips for GLM/Kimi/MiniMax - Add optional alternateBaseUrls field to ProviderConfig - Populate GLM, Kimi, MiniMax with China/International endpoints - Settings UI renders chips below base URL input; clicking fills and auto-saves - i18n: settings.baseUrlRegion.{china,international} in all 5 locales - README / README-zh: list GLM and add quick-examples block Closes #448 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: use glm-5.1 as the GLM default model example Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>	2026-04-18 14:00:14 +08:00
wyucandClaude Opus 4.6	91c4015732	feat(pdf): add MinerU Cloud API as PDF parsing provider (#438) ... * feat(pdf): add MinerU Cloud API as PDF parsing provider Add MinerU Cloud (v4 API) as a new PDF provider alongside unpdf (built-in) and MinerU (self-hosted). Users can now parse PDFs via the cloud API without deploying a self-hosted MinerU instance. - New provider `mineru-cloud` with API key auth and optional base URL - Cloud flow: batch create → presigned upload → poll → ZIP download → parse - Extract shared `mineru-parser.ts` from inline code (used by both paths) - Settings UI adapted for cloud (API key required) vs self-hosted (base URL required) - Server-side env var support: PDF_MINERU_CLOUD_API_KEY / PDF_MINERU_CLOUD_BASE_URL - SSRF protection on cloud verification endpoint - i18n translations for en-US, zh-CN, ja-JP, ru-RU Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(provider-config): revert PDF activation to requiresBaseUrl Reverts the PDF section from keylessProviders back to requiresBaseUrl: true to fix failing test: mineru with only apiKey (no baseUrl) should not activate. mineru-cloud API key can be configured via browser settings UI instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(pdf-settings): enable test button when API key is server-configured The test connection button was disabled when the API key was only configured server-side (not entered by user in browser). Now checks isServerConfigured in addition to user-entered values. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-16 15:15:55 +08:00
Yizuki_Ame	396171101d	fix(settings): persist custom provider baseUrl on creation (#414) (#417)	2026-04-14 13:05:09 +08:00
杨慎andClaude Opus 4.6	082fcbcbbe	fix(#419): hide Ollama from model selector when not configured (#420) ... Keyless providers (e.g. Ollama) with a hardcoded defaultBaseUrl were unconditionally passing the provider filter because `!requiresApiKey` short-circuited to true. Now keyless providers require an explicit user-provided baseUrl or server configuration to appear. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-13 22:33:06 +08:00
wyucandClaude Opus 4.6	9a0060e2f0	feat(audio): add custom OpenAI-compatible TTS/ASR provider support (#357) (#409) ... * feat(audio): extend TTSProviderId/ASRProviderId to support custom providers * feat(audio): update provider registry helpers for custom provider support * feat(audio): route custom providers to OpenAI-compatible implementations Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(audio): add custom TTS/ASR provider CRUD to settings store * feat(audio): add dialog for creating custom audio providers * feat(audio): add custom provider UI with voice management and delete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(i18n): add custom audio provider translation strings * fix(audio): resolve remaining type errors for custom provider support Add `as keyof typeof` casts where TTSProviderId/ASRProviderId (which now include template literal types) index into the built-in-only registries TTS_PROVIDERS, ASR_PROVIDERS, DEFAULT_TTS_VOICES, and DEFAULT_TTS_MODELS. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(audio): allow custom provider IDs to reach switch default branch The early guard `if (!provider) throw` blocked custom provider IDs from reaching the switch-case default branch where they get routed to OpenAI-compatible implementations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(audio): add default model field to dialog and polish voice list UI - Add "Default Model" input to custom audio provider dialog (fixes Kokoro needing model:"kokoro" instead of fallback "gpt-4o-mini-tts") - Redesign voice list as aligned table with column headers, hover reveal delete buttons, and empty state message - Add Enter key support to voice add inputs - Use consistent grid layout for add voice row Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl when testing custom TTS provider The baseUrl field in provider config is empty by default (it's an override field). Custom providers store their default URL in customDefaultBaseUrl, which needs to be used as fallback for test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): auto-select first voice when adding to custom TTS provider When a custom provider's voice is 'default' (initial state before any voices are added), adding the first voice now automatically updates the active voice selection so test TTS works immediately. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl in all TTS call sites Scene generator, discussion TTS, and agent bar voice preview all read providerConfig.baseUrl which is empty for custom providers (the default URL is stored in customDefaultBaseUrl). Add fallback chain to all 4 call sites so custom providers work during actual playback, not just in the settings test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): include custom provider voices in agent voice picker getAvailableProvidersWithVoices only iterated built-in TTS_PROVIDERS, so custom providers never appeared in the agent bar voice dropdown. Now also iterates ttsProvidersConfig for custom-tts-* entries and includes their customVoices in the picker. Also updated getServerVoiceList, findVoiceDisplayName, and resolveAgentVoice to handle custom provider voice lookups. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl in all ASR call sites Same issue as TTS: audio-recorder, asr-settings, and audio-settings all read providerConfig.baseUrl which is empty for custom providers. Add customDefaultBaseUrl fallback to all 3 ASR call sites. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(audio): add model list management for custom ASR providers Custom ASR providers now support adding/removing models with a table UI (same design as TTS voice list). The model selector dropdown shows custom models and auto-selects the first one. Also adds custom ASR providers to the Toolbar's ASR provider dropdown and fixes custom provider handling in audio-settings.tsx (baseUrl fallback, request URL preview, API key section visibility). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): seed customModels from defaultModel and fix ASR model label - When creating custom ASR provider with a defaultModel, auto-add it to customModels so the model list isn't empty on first visit - Fix ASR settings label from "TTS Model" to "Default Model" Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): separate ASR model management from creation dialog ASR models should be managed in provider settings (add/remove from list, select default), not pre-filled at creation time. Unlike TTS where a single default model suffices, ASR users need to manage multiple models (e.g. whisper-small, whisper-large-v3). - Remove Default Model field from ASR creation dialog (keep for TTS) - Remove defaultModel parameter from addCustomASRProvider - ASR models are added via the model list UI in provider settings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): simplify ASR model management — first model is default Remove the separate model selector dropdown for custom ASR providers. The first model in the list is always used as default. Users manage the model list only (add/remove), no separate "default" selection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): add loading state and detailed errors to ASR test - Show spinner with "Processing..." while waiting for transcription - Disable record button during processing - Show specific error messages from API (e.g. "No transcription generated") instead of generic failure text - Handle empty transcription result as an error with helpful message Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): use correct key for processing label in ASR test button * fix(audio): polish custom provider UX and type safety - Add proper type declarations for custom provider fields (customName, customDefaultBaseUrl, customVoices, etc.) eliminating 30+ unsafe casts - Show custom ASR providers in toolbar with default language list - Hide custom ASR providers without models from toolbar selection - Add no-models warning in ASR settings panel - Add delete confirmation dialog for custom TTS/ASR providers - Add duplicate voice/model ID validation with toast error - Add dedicated i18n keys for ASR model management (modelNamePlaceholder, addModel) - Fix i18n key for ASR processing label (settings.asrProcessing) - Add radio-button model selection in custom ASR provider settings - Add visible OpenAI-compatible description in provider creation dialog - Add CUSTOM_ASR_DEFAULT_LANGUAGES constant for toolbar language options Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(test): add isCustomTTSProvider/isCustomASRProvider to audio types mock The settings-server-sync test mocked @/lib/audio/types as an empty object, but settings.ts now imports isCustomASRProvider for provider validation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): add customDefaultBaseUrl fallback in all TTS/ASR call sites When a custom provider is created, the actual URL is stored in customDefaultBaseUrl while baseUrl is initialized empty. Several call sites only checked baseUrl, causing fetch failures during course generation. Add fallback to customDefaultBaseUrl in: - generation-preview/page.tsx (course generation TTS) - tts-providers.ts (getCurrentTTSConfig) - asr-providers.ts (getCurrentASRConfig) - media-popover.tsx (TTS preview) - tts-config-popover.tsx (TTS preview) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: format generation-preview/page.tsx Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused ttsSpeedRange variable in media-popover Flagged by CodeQL code quality check. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-04-12 21:49:09 +08:00
杨慎andClaude Opus 4.6	00152890c6	chore: update Ollama logo to official version (#400) ... * chore: update Ollama logo to official version Closes #399 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: improve OpenAI and Ollama logo visibility across themes OpenAI logo (white fill) was invisible on light backgrounds, and Ollama logo (black fill) was invisible on dark backgrounds. Normalize OpenAI SVG to black fill and apply dark:invert via a central MONO_LOGO_PROVIDERS set. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-11 17:41:37 +08:00
杨慎andClaude Opus 4.6	9007445db0	fix(i18n): add missing Ollama and Doubao provider names for ru-RU (#389) ... * fix(i18n): add missing Ollama and Doubao provider names for ru-RU Closes #388 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): use translated provider names in settings detail panel The provider detail header was showing hardcoded names from the provider registry (e.g. "豆包", "硅基流动") instead of i18n-translated names. Apply the same providerNames lookup used in the provider list. Also adds missing Ollama and Doubao keys for ru-RU. Closes #388 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-09 21:41:45 +08:00
杨慎andClaude Opus 4.6	ad9e0ee7c6	refactor(i18n): migrate to i18next framework (#331) ... * refactor(i18n): migrate to i18next framework Replace hand-rolled i18n with i18next + react-i18next so that adding a new language only requires dropping a JSON file in lib/i18n/locales/. - Add i18next, react-i18next, i18next-browser-languagedetector deps - Generate zh-CN.json / en-US.json from existing TS translation modules - Rewrite lib/i18n/index.ts as a thin wrapper around i18n.t() - Rewrite use-i18n hook to delegate to useTranslation(); external API (locale, setLocale, t) is unchanged so consumers need no changes - SSR-safe: LanguageDetector only loaded on client side Closes #327 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): use interpolation for greeting to support natural phrasing Replace string concatenation (greeting + displayName) with two i18next keys: greetingWithName (with {{name}} interpolation) and greetingDefault (standalone, no name). This lets each locale choose natural phrasing independently: - zh-CN: "嗨，同学" / "嗨，Alice" - en-US: "Hi there" / "Hi, Alice" - Future locales can avoid gender issues by choosing genderless defaults Also widen the t() type signature to accept interpolation options. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(i18n): auto-discover locale files via dynamic import Replace hardcoded zh-CN/en-US imports with i18next-resources-to-backend and dynamic import(`./locales/${language}.json`). Bundler scans the locales/ directory at build time, so adding a new language now requires only dropping a JSON file — zero changes to existing code. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): resolve hydration mismatch by deferring language detection LanguageDetector ran during i18next init(), detecting browser language before React hydrated — server rendered zh-CN while client switched to en-US immediately, causing a hydration mismatch. Fix: remove i18next-browser-languagedetector; init with a fixed lng (zh-CN) so server and client agree on the first render. Language detection is now done in I18nProvider's useEffect after hydration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(i18n): remove hardcoded locale list from language detection Replace manual locale validation and startsWith('zh') prefix matching with i18next's built-in fallback mechanism. Now changeLanguage() is called with navigator.language directly — if the exact locale has no JSON file, i18next automatically falls back to fallbackLng. Also widen Locale type from union to string so adding new languages doesn't require modifying types.ts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(i18n): add course language config with 11 languages and UI linkage - Add lib/i18n/course-languages.ts with curated language list (zh-CN, zh-TW, en-US, ja, ko, fr, de, es, pt, ru, ar) including native labels and English prompt names - Course language defaults to UI locale on first visit; once user explicitly picks a language, that choice persists across sessions - Replace toggle button with dropdown selector showing native labels - Widen language types from 'zh-CN'|'en-US' to string throughout - Fix hardcoded language ternaries in LLM prompt injection: - prompt-builder.ts: use getCourseLanguagePromptName() - classroom-generation.ts: remove normalizeLanguage() that forced all non-English to zh-CN - PBL system prompt, agent templates, generate-pbl: append language instruction for non-zh/en languages - quiz-grade API: add language suffix for grading feedback Closes #327 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(quiz): use course language instead of UI locale for grading quiz-view was passing the UI locale to the grading API, causing AI feedback to follow the student's answer language instead of the course language. Now reads stage.language from the store. Also strengthen the grading prompt: explicitly instruct the LLM to write comments in the course language regardless of the student's input language. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * revert: remove course language config and quiz fix from i18next branch Reverts f7bd6bd and 5bb5133 — these are independent features that should go into a separate branch/PR. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(i18n): regenerate locale JSON after merging latest main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(i18n): remove obsolete TS translation files These files are replaced by lib/i18n/locales/*.json and are no longer imported anywhere in the codebase. Removed: chat.ts, common.ts, generation.ts, settings.ts, stage.ts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style(i18n): fix prettier formatting in locale JSON files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(i18n): replace hardcoded language options with central locale registry Add lib/i18n/locales.ts as a single source of truth for supported languages. Language selectors in header and homepage now render dynamically from this registry. Also adds supportedLngs to i18next config so unsupported languages fall back correctly. Adding a new language now only requires a JSON file and one line in the registry. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): add missing rename keys to JSON locale files Add classroom.rename, classroom.renamePlaceholder, and classroom.renameFailed that were added to generation.ts on main but missing from the JSON locale files. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(i18n): extract LanguageSwitcher, restore type safety, fix locale resolution - Extract shared <LanguageSwitcher /> component from page.tsx and header.tsx - Derive Locale type from supportedLocales registry (as const satisfies) - Add resolveLocale() to match browser language prefixes (e.g. 'en' → 'en-US') - Remove config.ts changes (nonExplicitSupportedLngs + inline resources) that broke translation loading when combined with resourcesToBackend Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): use i18next double-brace interpolation instead of manual .replace() - Convert all {var} placeholders in locale JSONs to {{var}} (i18next default) - Replace .replace('{var}', value) calls with t(key, {var: value}) in all consuming components: lecture-notes-view, pbl-renderer, use-pbl-chat, generation-preview, whiteboard-history, agent-settings - Widen t() type signature in handleIssueComplete to accept interpolation options Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(i18n): unify greeting with default nickname, add translation guide - Remove greetingDefault key; greeting always uses greetingWithName with displayName (falls back to profile.defaultNickname) - Change en-US defaultNickname from "Student" to "Learner" - Add TRANSLATION_GUIDE.md documenting how to add languages and explaining keys with non-obvious UX design intent Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-04 12:04:42 +08:00
wyucand杨慎	3c687dd523	fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#326) ... * fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#324) - Move model selection from global ttsModelId to per-provider ttsProvidersConfig[providerId].modelId - Remove supportsModelSelection flag, use models.length > 0 instead - Add defaultModelId to TTSProviderConfig and ASRProviderConfig - Add providerOptions field to TTSModelConfig for future extensibility - Fix ElevenLabs hardcoded model_id: 'eleven_multilingual_v2' - Fix MiniMax TTS using config.model instead of config.modelId - Update model lists from provider docs: - Qwen: add qwen3-tts-instruct-flash, qwen-tts - ElevenLabs: add eleven_flash_v2_5, eleven_flash_v2 - MiniMax: remove non-existent speech-2.8 series - Replace custom model CRUD UI with simple Select dropdown - Remove MiniMax-specific hardcoded model selector from audio-settings - Add migration for old ttsModelId/asrModelId/model fields * feat(tts): voice picker groups by provider:model, sets both model and voice Voice selection in agent bar now groups voices by model. Picking a voice from a model group sets both modelId and voiceId on the agent. TTS settings shows available models as display-only chips. - Add compatibleModels field to TTSVoiceInfo (OpenAI marin/cedar restricted to gpt-4o-mini-tts) - Add modelId to agent voiceConfig and ResolvedVoice - Add ModelVoiceGroup to ProviderWithVoices, build groups in getAvailableProvidersWithVoices - Update AgentVoicePill and TeacherVoicePill to iterate modelGroups - Thread modelId from voice selection through discussion TTS pipeline - Replace model Select with display-only chips in tts-settings * fix(tts): show provider name in voice picker group headers * fix(tts): restore MiniMax speech-01 models per official API docs * fix(tts): add MiniMax speech-2.8 models per official docs, default to speech-2.8-hd * feat(tts): expand MiniMax voice list with recommended voices * fix(tts): remove non-official MiniMax recommended voices, keep 10 from official list * fix: address CR findings - align defaults, fix test mocks, ASR modelId fallback * fix(tts): pass modelId in voice preview and classroom generation * fix(tts): voice picker highlights only the exact model:voice match --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>	2026-03-30 17:21:19 +08:00
杨慎andClaude Opus 4.6	1d23a6c2b3	fix(settings): allow model-level test connection without client-side API key (#309) ... * fix(settings): allow model-level test connection without client-side API key and fix i18n keys Remove the client-side apiKey guard in model edit dialog so test connection delegates to the server (matching provider-level behavior). Also replace non-existent common.save/common.cancel i18n keys with settings.saveModel/ settings.cancelEdit in image and video model dialogs. Closes #308 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(settings): disable model-level test button when no API key is available Match provider-level behavior: disable the test connection button in the model edit dialog when requiresApiKey && !apiKey && !isServerConfigured. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-30 15:27:55 +08:00
wyucandHi-Jiajun	9badd59856	feat: add and expand MiniMax provider support (#182) ... Add MiniMax LLM models (M2–M2.7), TTS provider (speech-2.8/2.6/02/01), image generation (image-01), and video generation (Hailuo 2.3/02, T2V-01). Unify base URLs to api.minimaxi.com and add MiniMax-specific base URL normalization for the Anthropic-compatible endpoint. Co-authored-by: Hi-Jiajun <3138947285@qq.com>	2026-03-30 12:41:43 +08:00
wyucand刘少杰	5adf91c6f3	feat(settings): add configurable model selection for TTS and ASR (#108) ... Add ttsModelId/asrModelId to persisted settings, model management UI for providers that support model selection, and propagate selected model through preview, generation, and transcription flows. Co-authored-by: 刘少杰 <liushaojie@cvte.com>	2026-03-30 11:29:54 +08:00
wyucandClaude Opus 4.6	694570783a	feat(tts): add Doubao TTS 2.0 (Volcengine Seed-TTS 2.0) provider (#283) ... * feat(tts): add Doubao TTS 2.0 (Volcengine Seed-TTS 2.0) provider Add Doubao TTS as a new TTS provider with 17 voices (14 Chinese, 3 English), streaming response parsing for Volcengine's chunked JSON format, rate-limit error handling, and full integration across types, constants, provider implementation, server config, settings store, i18n, and UI components. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ui(tts): split Doubao API key into separate App ID and Access Key fields Show two input fields (App ID + Access Key) for Doubao TTS in settings, instead of requiring the compound "appId:accessKey" format. The values are combined internally so no changes are needed to the API layer. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): address code review feedback for Doubao TTS provider - Add TODO note explaining TTSRateLimitError's future use for retry/backoff - Replace atob() + manual byte loop with Buffer.from() for server-side code - Trim supportedFormats to ['mp3'] since only mp3 is used in requests - Add i18n keys for Doubao App ID / Access Key labels (zh-CN + en-US) - Add show/hide toggle to Access Key input field Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-27 16:43:18 +08:00
3 people	ddb52241fb	feat: add Discussion TTS with per-agent voice assignment (#211) ... * feat(tts): add resolveVoice() and getServerVoiceList() utilities Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tts): add AudioIndicator equalizer bars component Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tts): add onSegmentSealed callback to StreamBuffer Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tts): add voiceOverrides field to AgentConfig and AgentTemplate Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tts): add useDiscussionTTS hook with audio queue and cleanup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tts): add audio state indicator to Roundtable bubble Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(tts): wire onSegmentSealed callback through chat sessions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tts): add per-agent voice dropdown to AgentBar Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tts): integrate useDiscussionTTS in Stage and pass state to Roundtable Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style(tts): refine voice dropdown to pill-style selector Replace native select styling with a compact rounded-full pill that blends into the agent row. Remove border, use muted bg, smaller text, and a custom chevron icon. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style(tts): use shadcn Select for voice dropdown, link with TTS toggle - Replace native <select> with shadcn Select component for consistent UI - Hide voice dropdown when TTS is muted (ttsMuted) - Compact pill-style trigger with rounded-full, no border, muted bg Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style(tts): add voice label prefix and always show dropdown Show "音色: Alloy" instead of plain "Alloy" in the voice pill. Always show dropdown regardless of mute state (voice config is independent of playback). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style(tts): add volume icon hint in collapsed AgentBar Show a small Volume2 icon in the collapsed pill when voice config is available, hinting that voice settings are inside. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): fix voice dropdown layout and click handling - Move voice pill below agent name (second line) to prevent horizontal overflow in English - Wrap Select in div with onPointerDown stopPropagation to fix Radix click-through to parent row - Add line-clamp-1 to descriptions for consistent row height - Use items-start instead of items-center for better multi-line alignment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(tts): redesign AgentBar voice layout for compactness - Single-line layout: checkbox · avatar · name · role · voice pill - Remove descriptions from agent rows (saves vertical space) - Extract AgentVoicePill component to isolate Select event handling - Smaller avatars (size-7), tighter row padding (py-1.5) - Voice pill uses Volume2 icon + voice name (no prefix text) - Works in both Chinese and English without overflow Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tts): cross-provider voice selection per agent - Change voiceConfig from per-provider lookup to explicit { providerId, voiceId } per agent - Each agent can use a different TTS provider's voice - Voice picker dropdown groups voices by provider - useDiscussionTTS routes TTS requests per agent's provider - resolveAgentVoice falls back to global provider if no config Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): align role badge and voice pill across agent rows Give role badge fixed width (w-14 text-right) so role text and voice pills align vertically across all rows regardless of agent name length. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): fix role badge and voice pill alignment Wrap role badge + voice pill in a fixed-width container (w-[9.5rem] justify-end) so both align vertically across all agent rows regardless of name or role text length. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style(tts): align role badge and voice pill across agent rows Add min-w-[52px] text-right to role badge so it starts at a consistent position regardless of agent name length. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): use fixed w-[60px] for role badge alignment * fix(tts): use fixed w-[88px] for voice pill alignment * fix(tts): prevent click-outside from closing AgentBar when Select portal is open * fix(tts): comprehensive voice picker rewrite - Replace Radix Select with Popover + button list (fixes click issue) - Fix getAvailableProvidersWithVoices to always include global provider - Widen panel from w-80 to w-96 (prevents name truncation) - Voice pill uses primary color instead of gray (more visible) - Extract renderAgentRow helper to reduce duplication - Popover shows voices grouped by provider with active state - Add findVoiceDisplayName utility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): align voice provider availability with toolbar logic * fix(tts): fallback to first available provider when global provider has no voices * refactor(tts): remove global provider fallback from voice resolution Voice resolution now only depends on available providers (those with API keys or server-configured). No more globalProviderId parameter. Fallback is first available provider, then browser-native-tts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tts): add browser native TTS voices to agent voice picker Load speechSynthesis.getVoices() in AgentBar and include as a "Browser Native" provider group in the voice popover. No API key needed - always available if browser supports it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tts): simplify toolbar TTS to on/off toggle, add disabled state Toolbar: - Replace volume slider with simple TTS on/off toggle button - Remove ttsMuted/ttsVolume/onVolumeChange props from CanvasToolbar - Toggle now controls ttsEnabled (not ttsMuted) AgentBar: - Collapsed: show VolumeX icon when TTS disabled - Voice pills show disabled state (gray, cursor-not-allowed, no popover) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(tts): simplify Settings TTS tab to toggle + provider config Remove voice selection, speed slider, preview/test, Azure locale filter from Settings TTS tab. Voice is now per-agent in AgentBar. Keep: on/off toggle, provider selector, API key + base URL config. Add hint text pointing to AgentBar for voice configuration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(tts): simplify media popover TTS tab to toggle only * fix(tts): add voice config hint to media popover TTS tab * feat(tts): add per-voice preview button in voice picker Each voice row in the popover has a small speaker icon button. Click to preview the voice with "欢迎来到AI课堂" / "Welcome to AI Classroom" (follows i18n). Browser native uses Web Speech API, server TTS calls /api/generate/tts. Click again or close popover to stop. Shows spinner while generating. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): preview text follows course language instead of UI language * refactor(tts): redesign AgentBar expanded panel layout - Teacher always at top with voice pill (works in both modes) - Mode tabs moved below teacher - Auto mode: single compact row with shuffle icon + description - Max turns: compact inline row with smaller input - Preset mode: only student agents listed (teacher already above) - Remove large shuffle animation from auto mode Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(tts): merge max turns into teacher row * refactor(tts): separate teacher row and max turns, use stepper UI - Teacher row: avatar + name + voice pill only - Max turns: bottom row with MessageSquare icon + compact stepper (minus/number/plus in a rounded pill) - Remove Input component dependency Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): increase voice pill contrast in dark mode * fix(tts): make max turns input editable, tighten panel padding * fix(tts): restore shuffle animation in auto mode (compact version) * fix(tts): adjust auto mode text spacing and add voice auto-assign hint * fix(tts): auto-close voice popover after selecting a voice * fix(tts): increase auto mode vertical padding for better balance * fix(tts): push auto mode text toward bottom with flex spacer * fix(tts): reduce auto mode bottom padding * feat(tts): wait for TTS audio to finish before next agent turn Add waitForDrain() to useDiscussionTTS that returns a promise resolving when the audio queue is empty. The agent loop in useChatSessions now awaits this after buffer drain, so the next agent's turn doesn't start until the current agent's TTS audio finishes playing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): keep bubble visible while TTS audio is still playing When buffer drains (text=null) but audio indicator is still active, don't clear liveSpeech. Clear it only when audio state goes idle. This keeps the speech bubble visible until TTS finishes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tts): hold discussion bubble until TTS audio finishes When StreamBuffer fires the done signal (onLiveSpeech null), Stage now checks if TTS is still playing. If so, it defers clearing the bubble state. The bubble stays visible until onAllAudioEnd fires from the TTS hook (queue empty + nothing playing), then clears. This prevents the jarring UX where the bubble disappears while the agent's voice is still audible. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): fix bubble hold - guard onStopSession instead of onLiveSpeech Root cause: bubble disappears because doSessionCleanup fires via onStopSession when the agent loop ends naturally, NOT because of onLiveSpeech(null, null). Fix: when onStopSession fires and TTS is still playing, defer doSessionCleanup to onAllAudioEnd callback. Manual stop (user presses button) still cleans up immediately via handleStopDiscussion. Use doSessionCleanupRef to avoid circular dependency between discussionTTS hook and doSessionCleanup useCallback. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): guard BOTH onLiveSpeech and onStopSession for bubble hold Two paths clear the bubble: 1. onLiveSpeech(null, null) from StreamBuffer done → clears liveSpeech 2. onStopSession → doSessionCleanup → clears all state Both fire when agent loop ends. Path 1 fires first (tick loop), path 2 fires after (waitUntilDrained resolves). Both must be guarded when TTS is still playing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tts): hold bubble during TTS playback and respect playback speed Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(tts): LLM picks voice matching agent persona during generation - Client sends available voices (providerId + voiceId + name) to /api/generate/agent-profiles - LLM prompt asks to pick a voice matching each agent's personality - Parse "providerId::voiceId" from response, save as voiceConfig - Fallback to index-based assignment if LLM doesn't pick - Browser native voices hidden when server providers are available - saveGeneratedAgents accepts and persists voiceConfig Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): restore volume slider in classroom toolbar Revert the toolbar simplification from 36e3997 that replaced the volume slider with a TTS on/off toggle. The volume control with hover slider is a core classroom UX. TTS on/off is controlled via Settings and Media popover instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): teacher uses global lecture voice in discussion when no voiceConfig override * fix(tts): teacher always uses global lecture voice, no overrides * fix(tts): sync playback speed to currently playing audio in real-time * fix(tts): address code review issues - Issue 2: enabled flag now checks ttsEnabled && !ttsMuted in stage.tsx - Issue 4: remove unused browserAvailableVoices from useDiscussionTTS - Issue 5: remove dead code in audio-settings.tsx (Slider, Loader2, handleTTSVoiceChange, handleTTSSpeedChange, handleTestTTS, testingTTS, ttsTestStatus, ttsTestMessage, testText, ttsSpeed, setTTSSpeed, and unused browser-tts-preview imports) - Issue 6: shouldHold now checks queue length in addition to isPlayingRef - Issue 8: hide AgentVoicePill for teacher row in agent-bar.tsx (teacher voice is controlled in Settings) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(tts): address PR review — abort preview fetch, defer error recovery 1. Add AbortController to voice preview server TTS fetch, abort on stopPreview to prevent stale responses on rapid switching 2. Use queueMicrotask for processQueue calls in error/ended handlers to prevent synchronous recursion if multiple items fail consecutively 3. Add ordering invariant comment on sealLastText's onSegmentSealed Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tts): restore teacher voice pill, respect voiceConfig override * fix(tts): sync volume and mute to discussion TTS audio in real-time * fix(tts): allow browser-native TTS alongside server providers * fix(tts): remove top padding from voice popover content * fix(tts): make selectedAgents reactive to voiceConfig changes * fix(tts): use agents record instead of listAgents() to avoid infinite loop * fix(tts): single source of truth for teacher voice Teacher voice pill now reads/writes global ttsProviderId + ttsVoice (same settings used by lecture TTS). This ensures lecture and discussion always use the same teacher voice. Student agents still use per-agent voiceConfig. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add avatar descriptions for smarter LLM avatar selection Each avatar now has a one-line description (appearance, vibe) sent to the agent-profiles generation API. LLM picks avatars matching agent personality instead of guessing from file paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>	2026-03-23 17:35:09 +08:00
3 people	3a3e27bac0	feat: add ElevenLabs TTS provider (#134) ... * feat: add ElevenLabs TTS provider * fix(tts): wire ElevenLabs provider config Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> --------- Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-03-22 23:43:08 +08:00
Madoka	e92d1958a6	feat: add Grok (xAI) support as an LLM, image generation, and video generation provider (#113) ... * feat: add Grok (xAI) provider support for LLM, Image and Video generation - Add Grok as LLM provider (OpenAI-compatible, 11 models including grok-4.20, grok-4, grok-3) - Add Grok Image generation provider (grok-imagine-image, grok-imagine-image-pro) - Add Grok Video generation provider with async polling (grok-imagine-video) - Add env vars: GROK_*, IMAGE_GROK_*, VIDEO_GROK_* - Add web search note: Grok search is via chat completions, not standalone API - Update README/README-zh with Grok in supported providers Closes #87 * fix: ensure new image/video providers are auto-added to persisted settings Add ensureBuiltInImageProviders and ensureBuiltInVideoProviders functions that run on every store rehydrate, mirroring the existing LLM provider sync logic. Without this, Grok Image/Video providers would not appear for users with existing localStorage settings. * fix: add missing English translation for providerGrokVideo * style: fix prettier formatting for grok video provider * fix: remove unsupported grok multi-agent model and improve video error	2026-03-22 21:42:34 +08:00
Yizuki_Ame	ddfbc93bfd	fix: gracefully handle stale provider IDs in localStorage (#137) ... When localStorage contains a provider ID that no longer exists in the code (e.g. after a provider is removed or renamed), the Settings dialog crashes with 'Cannot read properties of undefined'. - Add ensureValidProviderSelections() to validate all 6 provider IDs (PDF, web search, image, video, TTS, ASR) during store rehydration - Hook validation into both migrate and merge paths - Add null guards in settings UI header for PDF and web search providers Closes #135	2026-03-21 17:47:02 +08:00
3 people	db5ac82167	fix: use browser speechSynthesis for playback when browser-native-tts is selected (#28) ... * fix: use browser speechSynthesis for playback when browser-native-tts is selected Previously, selecting browser-native-tts as the TTS provider would produce sound in the settings test but remain silent during classroom playback. This happened because: 1. The scene generator correctly skipped pre-generation for browser TTS (it runs client-side, not via API) 2. The playback engine fell back to a silent reading timer when no pre-generated audio was found, instead of calling speechSynthesis This commit adds Web Speech API integration directly in the PlaybackEngine: - New playBrowserTTS() method speaks text via speechSynthesis - Properly wires onend/onerror to advance to the next action - pause()/resume() now handle speechSynthesis.pause()/resume() - stop() and handleUserInterrupt() cancel browser TTS Fixes #25, fixes #12, fixes #5 * fix: handle "default" ttsVoice, extract CJK_LANG_THRESHOLD constant - When ttsVoice is "default" (set by Browser Native TTS which has no voice picker), the voiceURI lookup silently fails and no lang is set, causing Chinese text to be spoken with an English voice. - Extract the 0.3 CJK detection threshold as a named constant CJK_LANG_THRESHOLD with JSDoc explaining the rationale. - Fall through to language auto-detection when voice lookup fails, regardless of the reason (missing voice, "default" sentinel, etc.). * fix: support browser-native tts previews * refactor: extract shared TTS preview logic into useTTSPreview hook The browser-native and API-based TTS preview code was duplicated across tts-config-popover, media-popover, and tts-settings. Extract it into a reusable useTTSPreview hook that handles refs, cancellation, audio lifecycle, and staleness checks in one place. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: yangshen <1322568757@qq.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-03-18 22:25:29 +08:00
刘少杰	2ef6b6eac7	fix: isolate settings API key autofill fields (#48)	2026-03-18 15:18:36 +08:00
QianweiandJokerQianwei	477f976cbf	fix(tts): fix GLM TTS silent failure due to text length limit and error swallowing (#16) ... * fix: make GLM TTS generation reliable * fix: address review feedback — remove mutation, restore logging, drop redundant state --------- Co-authored-by: JokerQianwei <117466534@qq.com>	2026-03-16 20:41:56 +08:00
yangshenandClaude Opus 4.6	1f2ad6fbff	fix(tts): use provider's default voice when testing non-active TTS provider ... When testing a TTS provider different from the currently active one, the voice from the active provider was sent (e.g. OpenAI's 'alloy' to GLM API), causing error 1214. Now falls back to the selected provider's default voice. Also extracts duplicated default voice mappings into a shared constant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-16 15:24:28 +08:00
yangshenandClaude Opus 4.6	09bc93e7d9	fix: remove useCallback to satisfy React Compiler memoization rules ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 19:19:37 +08:00
yangshenandClaude Opus 4.6	5302804ec4	Merge branch 'main' into fix/mineru-deploy ... Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 19:16:10 +08:00
yangshenandClaude Opus 4.6	697bc547e0	fix: correct MinerU deployment docs and add PDF provider test button ... - Remove bundled MinerU Docker service and Dockerfile; link to official API and self-host docs instead - Set MinerU requiresApiKey to false (self-hosted doesn't need it) - Add test connection button to PDF settings panel with backend proxy to avoid CORS - Add verify-pdf-provider API route for server-side connection testing - Set Vercel API body size limit to 50MB for document uploads Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 19:06:45 +08:00
wyucandClaude Opus 4.6	843d33e5cc	chore: enforce Prettier formatting and fix lint issues ... - Add .prettierignore to exclude vendor packages, lock files, markdown, and YAML - Update .prettierrc: printWidth 100, singleQuote, trailingComma "all" - Run Prettier across all source files for consistent formatting - Fix unused imports (UserRequirements, setTTSProvider) - Fix eslint-disable comment placement after Prettier reformat Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-13 16:09:45 +08:00
wyucandClaude Opus 4.6	cc44f32ca8	Refactor: directory structure cleanup + Google Fonts localization ... - Replace next/font/google with local fonts (geist + @fontsource-variable/inter) to eliminate build-time network dependency - Resolve lib/utils.ts vs lib/utils/ naming conflict (cn() → lib/utils/cn.ts) - Move React Context Providers from lib/store/ to lib/contexts/ - Consolidate hooks into lib/hooks/ (use-theme, use-i18n) - Organize loose components into subdirectories: agent-* → components/agent/ canvas-* → components/canvas/ generation-toolbar, media-popover → components/generation/ speech-button, tts-config-popover → components/audio/ proactive-card → components/chat/ 65 files changed, lint 0 errors, build 28 pages passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 19:35:06 +08:00
wyucandClaude Opus 4.6	0d20abfed9	Initial commit: OpenMAIC AI classroom platform ... Next.js 16 App Router application with: - AI-powered interactive classroom generation from PDF/text requirements - Multi-agent discussion system (teacher, students, assistant roles) - Provider abstraction for LLM, TTS, ASR, image, video, web search - Local-first data architecture (IndexedDB/Dexie) - i18n support (zh-CN/en-US) - Structured logger, SSRF guard, lint/build passing (0 errors) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-03-12 16:41:07 +08:00