Commit Graph
44 Commits
Author SHA1 Message Date
wyuc 98904c1c5d fix(providers): three silent-behavior fixes from the provider audit (#1196)
* fix(extract): fail loudly instead of silently falling back to MinerU Cloud

A request that selects the self-hosted MinerU extractor without a
configured base URL used to fall back to MinerU Cloud whenever a cloud
API key was available, sending documents to a third party without the
operator's knowledge. The cloud fallback now requires an explicit
operator opt-in (ALLOW_MINERU_CLOUD_FALLBACK, default off, documented
in .env.example); otherwise the route answers a 422 that names what was
configured (self-hosted MinerU) and what was unavailable (its base
URL), and points at both remedies.

* fix(media): remove the selectable video provider that has no adapter

The Sora entry in the video provider catalog could be selected in the
settings UI and server config, but had no connectivity or generation
dispatch case, so choosing it failed only at execution time. A catalog
entry that cannot execute is worse than an absent one: the entry, its
type-union member, UI names/icons, store defaults, and the VIDEO_SORA
env mapping are removed. A new catalog test pins that every selectable
video provider dispatches to its own adapter in both switches.

* fix(storage): delete the obsolete no-op storage provider abstraction

getStorageProvider() unconditionally returned a NoopStorageProvider and
the module swallowed every operation into silence, so any caller
believing it had storage got neither bytes nor an error. No real caller
exists anywhere in the repo (only lib/storage/client.ts, a separate
module with a documented null contract and a live caller, remains). The
dead entry point, its type file, and the no-op provider are deleted; a
test pins the removal and that the real client upload helper is kept.
2026-08-27 11:49:23 +08:00
wyuc b8fbac1a54 feat(providers): uniform capability force-off and consistent missing-key contract (#1181)
* feat(providers): uniform capability force-off and consistent missing-key contract

* fix(providers): close force-off bypasses found in review
2026-08-26 10:59:43 +08:00
wyucandClaude Fable 5 5a3e81c4d9 feat(document): ingest uploaded sources into the asset pool and extract by id (#1153 part 0) (#1154)
* feat(document): declare a version on every extractor provider (#1153 part 0)

* feat(api): accept asset-id input in the extract-document route (#1153 part 0)

* feat(upload): ingest selected course materials into the asset pool (#1153 part 0)

* fix(document): harden asset-id extraction inputs and close upload ingest leaks (#1153 part 0)

* fix(document): scope extraction fallback and harden the asset-id path (#1153 part 0)

- G1: byte fallback only on pre-extraction failures; PARSE_FAILED
  (422/500) is surfaced without re-running the paid extractor
- G2: freeze the course-material set for the duration of generate-prep
  (guarded add/remove + disabled toolbar affordances, session built
  from the click-time snapshot, belt-and-braces drain)
- G3: bound the pre-generation ingest await (~15s); timed-out ingests
  go byte-path and their late-resolving ids are released
- G4: reject non-object JSON bodies (null/array/string/number) as 400
- G5: JSON-path provider pre-validation with generic messages, generic
  JSON-path PARSE_FAILED/422 responses, control-char-free log values
- G6: size cap enforced via identify() before resolve() materializes
  bytes; too_large maps to the existing 413

* fix(document): mirror provider-hint semantics on the asset-id form and freeze prep-time controls (#1153 part 0)

* fix(document): keep the asset-id form's responses free of caller input (#1153 part 0)

Address review findings: answer the extractor-selection failure with a
generic static message on the asset-id form (the registry's interpolated
message carries the caller's MIME type; multipart keeps it verbatim, both
sides now pinned by tests), dedupe same-batch material additions by content
fingerprint inside the pure updater, document the network-error retry
exception in shouldRetryWithByteUpload's docstring, and fix a test name
asserting 500 but named 400.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 11:45:31 +08:00
wyuc 3b4567076f refactor(generation): consume the package everywhere and delete lib/generation (#1090)
Part D of #1057 — the switch. All consumers import @openmaic/generation; lib/generation deleted (net -11,900 lines). PBL wiring preserved (single-call -> classified fallback -> injected loop), streaming outline route deduplicated onto buildOutlinePrompt, serverExternalPackages wired, and the change verified by a real-model four-kind end-to-end run against the production build with a rendered-classroom screenshot check. Closes #1057.
2026-08-10 03:58:33 -04:00
Nguyễn Lê TrườngandClaude Opus 5 4f20c5151a fix(i18n): translate ASR provider names in the generation toolbar (#1028)
The provider dropdown in the media popover rendered `p.name` straight from
`ASR_PROVIDERS`. That field is a registry label, not a UI string, and
several entries are written in Chinese:

    browser-native  →  浏览器原生 ASR (Web Speech API)
    qwen-asr        →  Qwen ASR (阿里云百炼)

So a user on any non-Chinese locale saw Chinese in the toolbar, while the
settings dialog showed the same providers correctly — it resolves the name
through `t('settings.providerBrowserNative')` and friends.

The id→i18n-key tables move to `lib/audio/provider-display.ts` and both
surfaces use them, so they cannot drift apart again. The settings dialog
keeps its own handling of custom providers, which carry a user-supplied
name rather than an i18n key.

TTS is included in the shared module for symmetry (`doubao-tts` and
`qwen-tts` have the same problem), though the popover's TTS tab is
currently a toggle with no provider list.

Image and video groups in the same popover also render `p.name`, but those
registry labels are already English, so no user-visible string changes;
left alone to keep this focused.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 06:04:49 -04:00
Yanpeng Wangandwyuc 9670100363 feat(extraction): media (audio/video) extraction + AliDocMind provider (#887)
* feat(extraction): add AliDocMind provider + media extraction abstraction

Adds AliDocMind (Aliyun Document Mind LLM version) as a new vendor
alongside unpdf/MinerU, and introduces the media (audio/video) extraction
layer that mirrors the document extraction one.

Document side (file: pdf/docx/pptx/xlsx/images):
- PDF_PROVIDERS gains an `alidocmind` entry; parseWithAliDocMind() maps
  layouts[] -> ParsedPdfContent. Flows through the existing document
  extractor registry, so AliDocMind is selectable anywhere MinerU is.
- PDFParserConfig / DocumentExtractorConfig gain accessKeyId/accessKeySecret
  (AliDocMind uses AK/SK, not a single apiKey); env fallback via
  ALIDOCMIND_ACCESS_KEY_ID / ALIDOCMIND_ACCESS_KEY_SECRET.

Media side (mp4/mp3/wav/... -> MediaArtifact):
- New lib/media-parse/ domain mirroring lib/pdf/ (types/constants/providers).
  parseMedia() maps AliDocMind segments[]/audio_frames/video_frames ->
  MediaArtifact (transcript + keyframes).
- MediaExtractorProvider interface + media registry + extractMedia() entry,
  symmetric to DocumentExtractorProvider / extractDocument().
- MediaArtifact and the ExtractionResult/Artifact/Error/Job envelope live in
  lib/document/types.ts (re-exported from @/lib/document).

Shared AliDocMind SDK wrapper (lib/pdf/alidocmind-client.ts) handles the
submit -> poll -> get flow for both sides via @alicloud/docmind-api20220711.

Tests: env-gated smoke test (tests/document/alidocmind.smoke.test.ts) drives
a real PDF and a real video through AliDocMind; media-artifact type test;
extractor-registry test updated for the new provider.

Part of #621 (MAIC ETL). Media extraction is the sibling to the document
extraction landed in #741.

* feat(extraction): wire AliDocMind AK/SK through settings UI + routes

Surfaces AliDocMind in the (post-#837) Document Parsing settings panel as a
peer of unpdf/MinerU — one panel, one credential entry, more supported
formats. Threads Aliyun AccessKey ID/Secret from the store through the
extraction routes.

- Store: pdfProvidersConfig + setPDFProviderConfig gain accessKeyId/accessKeySecret;
  default alidocmind entry added.
- pdf-settings.tsx: AliDocMind branch renders AccessKey ID + Secret inputs
  (secret masked with show/hide) and a Test Connection button; request-URL
  preview shows the DocMind endpoint. Provider auto-appears in the panel via
  PDF_PROVIDERS; supported-format badges come from #837's registry
  (ALIDOCMIND_MIMES added to lib/document/mime.ts).
- verify-pdf-provider route: alidocmind branch verifies AK/SK via a lightweight
  authenticated probe (verifyAliDocMindCredentials) — auth-level errors fail,
  anything else passes.
- extract-document route + generation flow (app/page.tsx, generation-preview):
  accessKeyId/accessKeySecret carried through session → FormData → config.
- i18n: alidocmindAccessKeyId / alidocmindAccessKeySecret in 8 locales.
- Icon: reuse /logos/bailian.svg (Aliyun family) instead of a missing asset.

Verified end-to-end against the running app: AliDocMind panel renders,
Test Connection returns "连接成功" through the real Aliyun API.

Part of #886.

* feat(extraction): route audio/video uploads through extractMedia() (reuse document path)

Media uploads now flow through the same upload picker, /api/extract-document
route, and generation pipeline as documents — no separate upload area. Only
the extraction differs: media mimes dispatch to extractMedia() -> MediaArtifact,
which is flattened to the text shape the generation pipeline already consumes.

- mime.ts: register audio/video formats in DOCUMENT_FORMATS (accept string,
  extension map, badges resolve for them); add MEDIA_PROVIDER_SUPPORTED_MIME_TYPES
  + SUPPORTED_MEDIA_MIME_TYPES, kept separate from PROVIDER_SUPPORTED_MIME_TYPES
  so the document drift-guard stays document-only. mimesForProviders() folds in
  a provider's media mimes, so the existing upload helpers
  (getAcceptStringForProviders / isMimeSupportedByProviders / format badges)
  cover media automatically when the provider supports it.
- extract-document route: media mimes dispatch to extractMedia(); MediaArtifact
  flattened to timestamped text (synopsis + transcript + keyframes).
- Tests: 6 media cases in mime.test.ts (accept/validation/badges/normalization).

Verified: uploading a real video through the route returns synopsis +
timestamped transcript/keyframes as text (curl, real AliDocMind key).

Part of #886.

* feat(extraction): use Alibaba Cloud icon for AliDocMind provider

Replace the placeholder bailian.svg with a dedicated Alibaba Cloud icon mark
(the square symbol from the official wordmark, text removed) so the provider
reads as Aliyun rather than Bailian. Square aspect matches the other provider
icons. Source: Alibaba Cloud / Alibaba Group brand assets.

* style: prettier format AliDocMind + media extraction files

* fix(extraction): harden AliDocMind — SSRF guard, cred verify, env gate, table text

Addresses review findings on the AliDocMind provider:

- SSRF (high): the media branch of /api/extract-document now runs the same
  validateUrlForSSRF check on a client-supplied baseUrl as the document branch,
  so an audio/video upload can't point the server's Aliyun SDK at an internal
  host.
- Credential verify (high): verifyAliDocMindCredentials now whitelists success
  signals instead of blacklisting auth errors. Probed against the real API:
  valid creds + bogus job returns a no-throw "BizIdNotExistOrResultExpired"
  body; invalid creds throw InvalidAccessKeyId.NotFound. Only a no-throw
  response or a "biz-not-found" business error counts as valid — an unreachable
  endpoint, a localized error, or throttling now correctly reports failure
  instead of a false "connection successful".
- Env-fallback gate (med): resolveCredentials no longer reads
  ALIDOCMIND_ACCESS_KEY_ID/SECRET unconditionally. Env fallback is opt-in via
  allowEnvFallback, which the route sets only for a server-managed provider, so
  an unauthenticated client request can't silently run on the server account.
- getDocParserResult error body (med): fetchResult throws on a non-200 result
  envelope instead of returning {} (empty text presented as success).
- Media pagination (med): stop after the first page when segments[] are present
  — layoutNum/layoutStepSize address layout blocks, not media segments, so
  re-requesting would loop over the same segments to the safety cap.
- Table content (med): tables/charts carry content in llmResult, not
  markdownContent; the layouts→text mapping now prefers llmResult for those
  types so table content isn't dropped (tables: true was advertised).
- Dead code: remove getCurrentMediaParseConfig (referenced non-existent store
  fields; the single-panel UI reuses pdfProvidersConfig).
- Dedup: media MIME list lives only in lib/document/mime.ts
  (ALIDOCMIND_MEDIA_MIMES); the media registry imports it.
- Tests: smoke test asserts durationMs is in ms (>1000 for a ~52s clip) to
  guard a ms/s unit mismatch; uses allowEnvFallback for the env-cred path.

Verified with the real key: PDF + video smoke tests pass; verify classifies
valid vs invalid creds correctly.

Part of #886.

* feat(extraction): extract AliDocMind images to base64 (parity with unpdf/MinerU)

AliDocMind embeds figure/picture image URLs inside each layout's
markdownContent (markdown `![](oss-url)`), not a dedicated field, and the URLs
are short-lived OSS signed links. Previously we emitted `images: []` and left
the expiring URLs inside the extracted text.

Now, for figure/picture layouts we:
- parse the OSS image URL out of markdownContent,
- fetch it at extraction time (before the signature expires) and re-encode to
  PNG base64 via sharp — the same base64 `images[]` contract unpdf/MinerU
  produce, so downstream storeImages → IndexedDB → slide works unchanged,
- populate metadata.pdfImages + imageMapping (the generation flow prefers
  pdfImages),
- strip the remote-URL markdown from the emitted text so expiring links don't
  leak into the prompt.

Downloads run concurrently; a failed/`sharp` image is dropped, never failing
the whole parse. Also fixes the prior over-broad table handling: only `table`
blocks read llmResult; `figure` is treated as an image (chart-figure llmResult
still kept in text).

Verified with the real key: a sample PDF yields 24 base64 images in both
images[] and metadata.pdfImages, and no oss-cn-hangzhou URLs remain in text.

Part of #886.

* docs(test): note AliDocMind video smoke test is non-deterministic server-side

* fix(extraction): correct AliDocMind pageCount (pageNum is 0-based)

Verified against a real response: AliDocMind reports pageNum 0..13 and
pageCountEstimate 13 for a 14-page document — both are 0-based. The metadata
pageCount previously used pageCountEstimate directly, undercounting by one.
Use the already-1-based maxPage, falling back to pageCountEstimate+1 only when
no blocks were seen. Document the 0-based convention at the normalization site.

* fix(extraction): address AliDocMind review — verify/SSRF/config/media (P1+P2)

Resolves all P1/P2 findings from the cross-review on #887.

P1 (blocking):
1. verifyAliDocMindCredentials now inspects the no-throw response body.code.
   An OSS-only key returns NoPermission without throwing; previously that was
   green-lit as "connection successful" then failed at extraction. Only a
   success/200 or the job-not-found probe code is accepted. Deterministic
   mocked test added (tests/document/alidocmind-verify.test.ts).
2. verify route trust boundary: managed → server-owned AK/SK + default endpoint
   only (ignore client values); unmanaged → client creds only, never env
   fallback, and the client endpoint is SSRF-validated before signing.
3. image fetch hardened: restricted to Aliyun OSS hosts, redirects disallowed,
   per-image byte cap, image-count cap, bounded concurrency (was unbounded
   Promise.all over provider-returned URLs).
4. AliDocMind is now selectable in the generation toolbar — availability
   recognizes the AK/SK pair, not just apiKey.

P2 (correctness):
5. Explicit server-config for the AK/SK pair (applyAliDocMindFallback +
   resolveManagedAliDocMindCredentials); verify and extract now resolve
   managed/env identically instead of verify-uses-env / extract-rejects.
6. Poll loop checks body.code before status, so a body-level error (e.g.
   NoPermission) fails fast instead of retrying for the full 15 min.
7. Image page numbers preserved through fetch/filter — no longer hard-coded to
   page 1, so multi-page image→page association is correct.
8. Empty media extraction (no synopsis/transcript/keyframes) returns 422
   PARSE_FAILED instead of HTTP 200 with empty text.
9. Format matrix trimmed to the official contract: images JPG/JPEG/PNG/BMP/GIF
   (dropped WebP/JP2), media MP4/MKV/AVI/MOV/WMV/MP3/WAV/AAC (dropped M4A).
10. A document-only provider (unpdf/mineru) uploaded with a media file now
    returns a clear 4xx instead of an opaque 500.

P3: credential-verify failures return INVALID_CREDENTIALS 4xx (not
INTERNAL_ERROR 500); formatTimestamp emits HH:MM:SS past one hour.

Verified with the real key: verify classifies valid→ok and invalid→fail; PDF
(24 images, correct page numbers) and MP4 extraction pass end-to-end.

* fix(extraction): make AliDocMind selectable + correct analysis label for media

Two UI/UX fixes found while manually testing the AliDocMind flow end-to-end:

- Persisted-state backfill: add ensureBuiltInPDFProviders so a PDF/document
  provider added after a user's settings were persisted (AliDocMind) is
  backfilled into pdfProvidersConfig on rehydrate. Without it the provider
  never appeared in the store, so it couldn't be selected and never picked up
  its server-configured flag. Wired into both persist migrate() and merge(),
  mirroring the existing image/video/web-search backfills.

- Analysis step label: getGenerationStepText showed "解析 document 文件" for
  audio/video (the type map fell through to the literal "document"). Documents
  keep their precise token (PDF/DOCX/PPTX/XLSX/images); audio/video now use a
  dedicated, locale-correct string (generation.analyzingMediaMaterial, added to
  all 8 locales) instead of forcing a format token into the "{{type}} 文件"
  template.

Manually verified end-to-end (real key): PDF, MP4, and MP3 (audio extracted
from the sample video) each extract and drive full course generation; a
DocMind-restricted AK/SK (OSS-only) now correctly fails verification with
NoPermission (400) instead of a false "connection successful".

* fix(extraction): address 2nd-round AliDocMind review (managed creds, stream cap, empty verify)

Resolves the three follow-up findings on #887:

1. [P1] YAML-managed AliDocMind creds now reach extraction. Both extract paths
   (document + media) previously cleared the managed AK/SK and relied on an
   env-only fallback, so a YAML-only deployment verified but failed to extract.
   They now resolve server-owned creds via the shared
   resolveManagedAliDocMindCredentials() (env OR YAML), matching the verifier.
   Regression test added for YAML-only creds with no ALIDOCMIND_* env.

2. [P1] Image download is now size-capped while streaming. Instead of buffering
   the whole response and checking length afterward, the body is read chunk by
   chunk with a cumulative byte count that aborts the moment it exceeds the cap
   — a missing/false Content-Length can no longer exhaust memory. Host allowlist
   tightened to oss-*.aliyuncs.com. Tests: non-OSS host refusal, no-Content-Length
   overflow abort, declared-oversize rejection.

3. [P2] Empty verification body no longer counts as success. Removed the
   codeStr === '' branch from the positive-signal whitelist (a working key
   always returns the job-not-found business code for the bogus probe id).
   Regression test added for empty and absent bodies.

Rebased onto main (package.json: kept @openmaic/storage + AliCloud deps).
Verified with the real key: PDF + MP4 extraction still pass end-to-end.

* fix(extraction): merge AliDocMind AK/SK into a baseUrl-configured YAML entry

Follow-up to the YAML-managed credential fix. When a YAML `pdf.alidocmind`
entry also specifies `baseUrl`, the generic loadEnvSection() (pdf requires a
baseUrl) creates the pdf.alidocmind entry copying only apiKey/baseUrl/models/
proxy — never AK/SK. applyAliDocMindFallback() then returned early because the
entry already existed, so the provider was "managed" but had no usable
credentials, and resolveManagedAliDocMindCredentials() returned undefined
(verify + extract both silently lost the creds).

Merge the AK/SK into the existing entry instead of returning early. Added a
regression test with baseUrl + accessKeyId + accessKeySecret together (the
previous test omitted baseUrl, so the generic loader skipped the entry and the
bug was masked).

* fix(extraction): don't mark AliDocMind managed without AK/SK; align poll code check

Two edge cases from a follow-up adversarial pass:

- [MED] A YAML `pdf.alidocmind` entry with `baseUrl` but no AK/SK (and no env
  AK/SK) made the generic loader create the entry → isServerConfigured=true
  (managed) → but resolveManagedAliDocMindCredentials() returned undefined, so
  the provider was locked out AND client-entered AK/SK were silently dropped.
  applyAliDocMindFallback now deletes a credential-less entry so the provider
  stays UNMANAGED (clients supply their own creds). Regression test added.

- [LOW] verify accepted a `code: "success"` body but the extraction poll loop
  threw on any non-"200" code — a success-shaped status would pass verification
  then fail extraction. The poll loop now treats "200"/"success" as benign,
  matching verifyAliDocMindCredentials.

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-07-14 18:12:31 +08:00
c56929510c feat(web-search): add SearXNG provider support (#842)
* feat(web-search): add SearXNG provider support

Enable self-hosted SearXNG via SEARXNG_BASE_URL, with server-side provider
fallback, settings sync, and tests.

Co-authored-by: Cursor <cursoragent@cursor.com>

* style: fix Prettier formatting in web-search files

Restore pnpm check by applying Prettier to imports, headers, and test assertions.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(web-search): keep SearXNG base URLs server-managed only

Close the SSRF primitive by ignoring client-supplied SearXNG URLs at the
route boundary and rejecting them in resolveSafeClientWebSearchBaseUrl.
Add negative route/config tests and an opt-in live smoke test for the
JSON API.

Co-authored-by: Cursor <cursoragent@cursor.com>

* style: fix Prettier formatting for SearXNG web-search changes

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: 徐松 <song.xu@aieshanghai.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-13 23:12:31 +08:00
jackefnandRowan_lxb baff0998bb Feat/document bundles milestone 3 (#844)
* feat(document): support document bundles

* fix(document): avoid server extractor imports in client bundle

* fix(pdf): default mineru backend to pipeline

* fix(document): address bundle review findings

* fix(document): harden extraction limits and storage errors

* fix(document): make analysis step format agnostic

---------

Co-authored-by: Rowan_lxb <Lxb_savior@163.com>
2026-07-13 23:01:40 +08:00
wyucand杨慎 c8a638a101 feat: support GPT-5.6 model family (#907)
* feat: support GPT-5.6 model family

* test: cover GPT-5.6 SDK validation

* fix: canonicalize GPT-5.6 Sol alias

* fix: complete GPT-5.6 alias lookups

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
2026-07-13 18:19:40 +08:00
Yizuki_AmeandClaude Fable 5 e136c81c03 chore(generation): remove dead legacy pipeline chain (#904) (#905)
Remove the generation-pipeline leftovers orphaned since the old
/api/generate route was deleted in 0610053:

- pipeline-runner.ts (runGenerationPipeline / createGenerationSession)
- generateFullScenes + generateSingleScene in scene-generator.ts
- unreachable callbacks param of generateSceneOutlinesFromRequirements,
  including its five hardcoded Chinese statusMessage strings
- generating-progress.tsx (never imported since the initial commit)
- orphaned GenerationCallbacks / GenerationProgress / GenerationSession
  types, 5 unused i18n keys across all 8 locales, and stale
  generateFullScenes references in DESIGN-regenerate-scene.md

Found while investigating #884. Verified against a pre-change baseline:
tsc --noEmit diff clean, vitest failure set unchanged (1956 passed),
check:i18n-keys passes, zero residual references repo-wide.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 21:56:35 +08:00
wyucand杨慎 ce89b4c314 fix(home): clarify Interactive Mode selected state (#901)
* fix(home): clarify interactive mode selected state

* fix(home): soften light interactive mode state

* fix(home): honor reduced motion in dark mode

* test(home): assert current interactive mode styles

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
2026-07-11 20:53:24 +08:00
PhillLittlewoodandwyuc a2886af92a Added ComfyUi to has Image Provider (#850)
* feat: integrate comfyui workflows into image provider selection

* style: format code with prettier

* fix: align i18n keys for russian locale

* feat: implement requested changes and format code

* feat: implement requested changes including renaming default workflow

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-07-10 18:49:28 +08:00
Yanpeng Wang b5e1963e48 feat(document): rename to Document Parsing, show supported formats, extend MinerU support
Squashed from PR #837 after review approval and green CI.
2026-07-09 04:29:58 -04:00
Yanpeng Wang d54e62f5c4 feat(token-plan): one-click token-plan setup + deployment usage dashboard (#784)
* feat(usage): add usage normalization, pricing, model-fetch, balance, storage

Foundational layer for token-plan usage tracking (cc-switch-modeled):
- lib/usage/normalize.ts: AI SDK v6 usage → four-class token shape
- lib/usage/pricing.ts + defaults: per-class USD pricing table
- lib/server/model-fetch.ts: /models candidate-URL multi-fallback (ported)
- lib/usage/balance-providers.ts: built-in balance queries + detection
- lib/server/usage-storage.ts: fire-and-forget jsonl logging to data/usage

All pure/storage logic covered by vitest (32 tests).

* feat(usage): capture token usage at the callLLM/streamLLM chokepoint

- callLLM records result.usage before returning
- streamLLM wraps onFinish to record totalUsage on stream completion,
  preserving any caller-supplied onFinish
- provider/model derived from the model instance (no route changes)
- fs-backed storage imported dynamically; fire-and-forget, never throws
- include_usage is already sent by @ai-sdk/openai, so streaming is covered

* feat(usage): add probe-models, balance, and usage API routes

- POST /api/provider/probe-models: discover chat models via /models with
  candidate fallback; SSRF-guarded; filters non-chat ids; 401/404 typed
- POST /api/provider/balance: built-in balance detection + billing fallback
- GET /api/usage: aggregate jsonl by model/day/source with costIncomplete flag

Verified e2e against the live MAIC gateway: 16 chat models (6 filtered),
balance detected, and a real callLLM writes a costed usage row.

* feat(settings): add token-plan preset picker to provider dialog

- lib/config/token-plan-presets.ts: data-driven vendor presets (Huawei/MiniMax/
  Xiaomi token plans, OpenRouter/SiliconFlow gateways, DeepSeek/GLM/Qwen/Hunyuan/
  Doubao direct) with baseURL, protocol, optional modelsUrl, category
- add-provider-dialog: '选择厂商/自定义' tabs; picking a preset auto-fills
  baseURL+protocol+modelsUrl, custom tab unchanged
- ProviderSettings.modelsUrl carries the optional /models override
- i18n keys added across all 8 locales

Verified in-browser: preset picker renders grouped by category.

* feat(settings): add Fetch Models button and balance bar to provider panel

- 拉取模型: probes /models, merges discovered ids into the model list
  (dedupe, keeps manual additions), with success/no-endpoint/auth messages
- 查询余额: queries /api/provider/balance, renders a balance bar or a
  'check console' hint when unsupported
- index.tsx: handleModelsFetched merges probe results into provider config
- i18n keys across all 8 locales; removed a now-unused eslint-disable

Verified in-browser against MAIC gateway: 16 models fetched, balance shown.

* feat(settings): add usage dashboard to System Settings

- usage-dashboard.tsx: echarts dual-axis daily trend (tokens + cost),
  totals cards, by-model table, refresh; reads GET /api/usage
- mounted at the top of GeneralSettings (系统设置)
- honest disclaimer + costIncomplete marker when a model lacks pricing
- i18n keys across all 8 locales

Verified in-browser: shows 1 request / 31 tokens / $0.0003 from a prior call.

* feat(token-plan): multi-modal one-click setup in System Settings

- token-plan-presets.ts: presets now declare per-modality targets
  (llm/image/video/tts/webSearch); MiniMax is the full-set template,
  others LLM-only — extend by adding entries (at-our-best adaptation)
- apply-token-plan.ts: fills one key into every declared modality via
  injected store setters, isolating per-modality failures (TDD, 4 tests)
- token-plan-settings.tsx: new sidebar page — pick plan, enter key,
  one-click apply lights up adapted modalities (+LLM model probe),
  shows 'not adapted yet' for the rest, balance bar reused
- reverted add-provider dialog to plain custom form (preset picker moved here)
- i18n across all 8 locales

Verified in-browser: Token Plan page renders, MiniMax shows all 5
modalities, apply/balance UI wired. 859 tests pass, build clean.

* feat(token-plan): add custom token plan entry to Token Plan page

- Custom card at the bottom of the provider list: pick it to manually enter
  name + protocol + baseURL (LLM-only), then key + one-click apply via the
  same applyTokenPlan flow (mirrors cc-switch's custom provider)
- effectivePreset unifies preset vs custom for apply/balance/probe
- i18n keys (customGroup/customName/customHint) across all 8 locales

Verified in-browser: custom card expands the manual form; preset flow intact.

* feat(token-plan): reflect persisted config on the Token Plan page

Other settings panels read the store directly, so they survive section
switches; the Token Plan page only wrote the store, so it looked blank on
return. Now it also reads providersConfig:
- selecting a preset prefills its saved API key
- configured presets show a '已配置/Configured' badge

State persistence was never broken (apply writes zustand→localStorage);
this fixes the missing read-back so the page reflects it.

* fix(settings): isLLMProviderConfigured crashed on providers without models

Root cause of 'token plan config disappears': applyTokenPlan writes a new
LLM provider with no models yet (probe fills them later). isLLMProviderConfigured
did config.models.length unguarded → threw inside setProviderConfig's resolver
→ the whole set() aborted → key/baseUrl/type never persisted.

- guard models in isLLMProviderConfigured (shared validator; affects any
  provider written without a models array)
- seed models:[] in applyTokenPlan's LLM write for a valid initial shape
- regression tests: validator no longer throws; apply+probe write keeps apiKey

Verified in-browser: DeepSeek persists key and shows 已配置 after section switch.

* feat(token-plan): add remove/teardown for a configured token plan

Apply had no inverse. Add removeTokenPlan: clears the API key and disables
(enabled:false) every modality the plan declared; a custom LLM provider is
deleted entirely (removeProvider), built-ins keep the cleared shape.

- removeTokenPlan in apply-token-plan.ts (mirrors applyTokenPlan, isolated
  per-modality, injected setters; 3 tests)
- trash button on configured preset cards in the Token Plan page; resets
  page state if the removed plan was selected
- i18n 'remove' across all 8 locales

Verified in-browser: removing DeepSeek empties its key and drops the
已配置 badge.

* feat(usage): track multimodal usage, drop cost/pricing

Reframe usage stats as pure usage (no cost), per user decision:
- usage-storage: drop all cost fields; add kind (llm/image/video/tts/asr)
  + quantity + unit; LLM keeps token counts, others store quantity
- delete lib/usage/pricing.ts + pricing-defaults.json + its test
- instrument image/video/tts at the server-only API routes (not in the
  provider dispatch — those files are in the client graph and importing
  fs-backed usage-storage broke the client bundle)
- /api/usage: aggregate by model/day/modality, no cost
- dashboard: per-modality usage (tokens/images/seconds/chars), token-only
  daily trend, no '$'; i18n cost keys replaced with modality/unit keys
- backward compatible: legacy rows (no kind, stray cost fields) read as llm

912 tests pass, build clean.

* feat(usage): per-modality dashboard layout + softer dark-mode chart

- group usage into per-modality sections (LLM/image/video/tts/asr), each
  table's usage column uses one consistent unit (token/image/sec/char) —
  no more mixed units in a single column
- summary chips per modality with their own unit
- trend chart now plots daily REQUESTS (unit-agnostic, works for any
  modality) instead of LLM-only tokens
- theme-aware chart: faint thin line + soft gradient area, muted axis/grid
  colors via useTheme — fixes the harsh solid stroke in dark mode

Verified in-browser (dark): TTS shows '字符', LLM shows 'Token', separate
sections; chart no longer has a hard line.

* refactor(usage): dedupe model/fetch/usage helpers, parallel balance probe

Review cleanups on the token-plan/usage branch:
- extract modelInfoFromId() (shared vision heuristic + ModelInfo shape)
- extract fetchWithTimeout() shared by model-fetch and balance-providers
- extract recordGenerationUsage() to dedupe the image/tts/video routes
- queryBalance: fetch billing subscription + usage in parallel
- parseOneApiBilling: report quota without remaining when usage endpoint
  is unavailable, instead of implying zero spend (full balance)
- split the two jammed imports in settings/index.tsx

* feat(token-plan): drop custom token-plan support

Custom token plans (manual baseURL/protocol entry) added complexity for
little gain — a one-off provider is better configured directly on the
Providers page. Token Plan is now preset-only:
- remove custom mode, manual fields, and the custom card from the UI
- collapse effectivePreset back to the selected preset
- drop the now-dead removeProvider action + custom-id branch in
  removeTokenPlan
- remove orphaned customGroup/customName/customHint i18n keys (8 locales)

* feat(token-plan): add Volcengine/Tencent/Bailian plans, drop balance feature

Add three vendor token-plan presets (all map to existing built-in LLM
providers, so it's data-only — no new adapters):
- 火山方舟 Volcengine Ark → doubao, OpenAI /api/v3
- 腾讯 TokenHub Token Plan → tencent-hunyuan, OpenAI /plan/v3 (the
  plan-specific base; /v1 is the pay-as-you-go gateway)
- 阿里百炼 Token Plan → qwen, cross-model plan (Qwen + DeepSeek/Kimi/
  GLM/MiniMax) on one key; model list is probed/entered

Remove the balance/quota feature entirely — we now track usage, not cost,
and every vendor's balance query needs its own cloud AK/SK + signature
(Volcengine SigV4 / Tencent TC3 / Aliyun BSS), which the Bearer-key
billing-endpoint probe never supported anyway:
- delete lib/usage/balance-providers.ts, /api/provider/balance, its test
- strip the Check Balance button + balance bar from the token-plan page
  and the provider config panel
- remove the 4 balance i18n keys across all 8 locales
- restore an eslint-disable the branch had dropped in provider-config-panel

* feat(token-plan): use cloud-brand logos for vendor token plans

The three vendor plans are cloud offerings, not single-model products, so
icon them with the cloud brand rather than a model logo:
- 火山方舟 → volcengine.svg (was doubao.svg)
- 腾讯 TokenHub → tencentcloud.svg (was hunyuan.svg)
- 阿里百炼 → alibabacloud.svg (was bailian.svg)

Logos are the colored brand variants from lobehub/lobe-icons, matching the
existing colored-logo style (plain <img>, no dark:invert needed).

* feat(token-plan): keep only MiniMax and Volcengine presets

Trim the token-plan list to the two we want to ship: MiniMax (full-set
template) and 火山方舟 Volcengine Ark. Drop the Tencent/Bailian plans and
the OpenRouter/SiliconFlow/DeepSeek/GLM/Qwen entries.

- remove the now-unused tencentcloud.svg / alibabacloud.svg logos
  (volcengine.svg stays; the other logos are still used by the provider
  registry)
- retarget the LLM-only apply test from the deleted deepseek preset to
  volcengine-ark

* feat(token-plan): restore aggregator/third-party presets

Previous commit over-trimmed: the intent was to drop only the Tencent and
Bailian token plans, not the OpenRouter/SiliconFlow/DeepSeek/GLM/Qwen
entries. Bring those back; keep only Tencent/Bailian removed.

- token_plan: MiniMax, 火山方舟 Volcengine Ark
- aggregator: OpenRouter, SiliconFlow
- third_party: DeepSeek, GLM, Qwen

Revert the apply test back to the deepseek fixture (restored).
tencentcloud.svg / alibabacloud.svg stay deleted (their plans are gone).

* fix(token-plan): point Volcengine plan at the Coding Plan endpoint

The plan's ark--prefixed API keys authenticate only against
/api/coding/v3, not the general /api/v3 endpoint — the latter rejects them
with "The API key format is incorrect", so model probing returned nothing.
Switch the base URL to https://ark.cn-beijing.volces.com/api/coding/v3.

* fix(token-plan): Volcengine is an Agent Plan (Anthropic /api/plan)

Per the Ark Agent Plan docs, the ark--prefixed keys authenticate ONLY
against the dedicated Anthropic-compatible base https://ark.cn-beijing.
volces.com/api/plan ("其他 Base URL 无法在 Agent Plan 中使用"). The general
/api/v3 and the Coding Plan /api/coding endpoints both reject the key as
"API key format is incorrect", which is why model probing kept returning 0.

- baseUrl → https://ark.cn-beijing.volces.com/api/plan/v1 (the /v1 lets the
  Anthropic SDK land on /api/plan/v1/messages)
- apiFormat → anthropic
- rename to 火山方舟 Agent Plan

Probe still targets /api/plan/v1/models (the path exists); if the Anthropic
gateway doesn't return an OpenAI-shaped list, users fall back to typing a
model id like ark-code-latest.

* fix(token-plan): Volcengine Agent Plan = OpenAI /api/plan/v3 + ark-code-latest

Settled after probing the real key and reading cc-switch's approach:
- The ark- plan key works on the OpenAI-compatible /api/plan/v3 endpoint
  (chat/completions returns 200); switch apiFormat back to openai.
- The plan exposes NO /models list (every /api/plan/*/models is 404), which
  is why probing kept returning 0. cc-switch handles this by hardcoding a
  single ark-code-latest (an auto-routing alias valid on any tier) and does
  NOT use AK/SK for model listing — so we do the same.
- Seed defaultModels: ['ark-code-latest'] only; users add specific ids by hand.

Supporting machinery (kept, general-purpose):
- applyTokenPlan seeds models from defaultModels instead of wiping to []
- handleApply uses defaultModels and skips the doomed probe when present
- drop stray .playwright-mcp/ debug artifacts and gitignore them

* style: fix prettier formatting in usage files

CI runs prettier on the whole repo (prettier . --check); these four files
predate this branch's formatting pass and tripped the check.

* feat(token-plan): verify Volcengine Agent Plan's published model set

The Agent Plan publishes a fixed model set but exposes no /models endpoint,
so carry the documented models as CANDIDATES and verify each on apply:

- add verifyModels flag to TokenPlanModalityTarget
- new /api/provider/probe-chat-models route: sends a minimal chat request per
  candidate (OpenAI /chat/completions or Anthropic /messages) in parallel,
  returns the subset that succeeds; SSRF-guarded, auth-failure short-circuits
- handleApply gains a verify branch (before the fixed-defaultModels fast path),
  falling back to the seeded list if verification fails
- Volcengine preset now carries the 12 published Agent Plan text models
  (doubao-seed-2.0-*/deepseek-v4-*/minimax-m*/glm-5.2/kimi-k2.*) as candidates

This auto-prunes retired (docs flag deepseek-v3.2/glm-5.1 as 即将下线) and
tier-gated models without code changes. Verified all 12 resolve against a real
plan key.

* feat(token-plan): wire Volcengine Agent Plan image + video modalities

Make the Ark seedream/seedance adapters path-configurable and light up the
image/video modalities on the Volcengine plan:

- seedream/seedance adapters: resolveArkRoot() uses baseUrl verbatim when it
  already carries an /api/... path (token plan's /api/plan/v3), else appends
  the standard /api/v3 — no regression for the pay-as-you-go default host.
- applyTokenPlan: image/video branches inject a modality's defaultModels as
  customModels and set them as the active provider+model, so generation works
  out of the box. New optional setImageProvider/ModelId + setVideoProvider/
  ModelId actions (UI passes the store setters; tests omit them).
- Volcengine preset declares image (doubao-seedream-5.0-lite, verified 200 on
  /api/plan/v3/images/generations) and video (doubao-seedance-2.0/1.5-pro —
  Medium+ tiers only; lower tiers reject at call time, no code change needed
  to upgrade).

Applying the plan overwrites the shared seedream/seedance slot with the plan
config (same overwrite model as LLM); switching back to pay-as-you-go is a
manual edit or plan removal. Verified image end-to-end with a real plan key.

* feat(token-plan): verify image/video models on apply, disable unsupported tiers

The Volcengine plan lit up video optimistically, but lower tiers (Small) don't
include video — so using it 404'd with UnsupportedModel. Probe media models on
apply and only keep what the tier actually supports:

- generalize /api/provider/probe-chat-models with a `kind` (chat|image|video):
  image hits /images/generations, video hits /contents/generations/tasks with
  empty content. The model-support check (404 UnsupportedModel) runs before any
  billable work, so probing never starts a real image/video job; for media,
  "supported" = any non-404 response.
- handleApply: after lighting up image/video, probe each verifyModels modality;
  prune to the verified model set + re-select a working model, or disable the
  modality entirely if none pass (no false "available").
- Volcengine preset: image/video targets gain verifyModels: true.
- add settings.tokenPlan.tierUnsupported across 8 locales.

Verified with a real Small-tier key: image (seedream-5.0-lite) passes and is
kept; video (seedance-2.0/1.5-pro) 404s and is disabled.

* feat(web-search): add Doubao (豆包搜索) provider

Doubao Search (Custom 版) over its REST endpoint
POST open.feedcoopapi.com/search_api/web_search with Bearer auth — the
same endpoint the askecho-search-infinity MCP server wraps, so the
Volcengine Agent Plan key authenticates directly. Mirrors the MiniMax
adapter: maps Result.WebResults to WebSearchSource (prefers Summary, the
query-relevant excerpt, over Snippet for LLM use) and surfaces errors
from ResponseMetadata.Error.

- register 'doubao' in WebSearchProviderId + WEB_SEARCH_PROVIDERS
- searchWithDoubao adapter, searchWeb dispatch, store default config
- SSRF allowlist entry for the search host

* feat(audio): support Agent Plan single-key auth for Doubao TTS

generateDoubaoTTS now picks auth + endpoint from the key shape, since
Volcengine exposes Seed-TTS as two products with separate credentials
(verified: a plan key 401s on the normal endpoint, and the plan endpoint
rejects appId-style auth):
  - single key (no colon) -> X-Api-Key, for the Agent Plan /plan endpoint
  - appId:accessKey        -> X-Api-App-Id + X-Api-Access-Key (unchanged)
A malformed pair (empty half) fails clearly instead of sending an empty
header. Reuses the existing NDJSON/base64-mp3 parsing and voice list.

* fix(media): map MiniMax video 720p to its real 768P tier

normalizeVideoOptions defaults minimax-video to '720p' (the first
supported resolution), but Hailuo 2.3 only accepts 768P/1080P and rejects
720P with '2013 ... does not support resolution 720P'. MiniMax's mid tier
is 768P, not 720P (the adapter already falls back to 768P, as does the
connectivity test), so map the shared enum's '720p' to 768P. Regression
tests lock the mapping.

* feat(token-plan): add web search + TTS to Volcengine Agent Plan, widen image tiers

Extend the volcengine-ark preset now that the adapters exist:
  - webSearch -> doubao (own host open.feedcoopapi.com, not the ark endpoint)
  - tts -> doubao-tts on the /api/plan/tts endpoint (single-key auth)
  - image defaultModels widened to a best-first Seedream 5.0/4.5/4.0 list so
    a higher tier keeps the strongest model while verifyModels prunes the
    rest; video keeps the 2.0 + 1.5-pro candidates

Comments record the verified host/auth quirks of each modality.

* feat(token-plan): show result panel only after probing, with two clear states

Addresses review feedback that a green check implied generation works
when it only meant 'configured'. The panel now renders after probing
finishes (gated on results && !applying) so it reflects the final set,
and uses two states: green when the modality is configured/usable, muted
when a live probe proved it unavailable (e.g. video on a tier without it).

* feat(token-plan): scope presets to true multi-modal token plans

Drop the single-modality LLM presets (OpenRouter, SiliconFlow, DeepSeek,
GLM, Qwen) from Token Plan. A token plan's defining trait is one key
spanning many modalities; those entries are ordinary LLM API providers
already covered by the add-provider flow, and listing them here muddied
the 'one key, every modality' promise. Only MiniMax and the Volcengine
Ark Agent Plan remain. The UI already hides categories with no entries.

apply-token-plan's LLM-only test now uses a local fixture instead of the
removed deepseek preset.

* feat(token-plan): progressive reveal of probe results on apply

The result panel previously rendered all at once after probing finished,
reading as dead air during the model probe. Now rows appear immediately on
Apply: modalities with a live probe in flight show a spinner ('pending')
and resolve to lit/failed independently as each probe returns, while
non-probe modalities show lit right away. Probes run in parallel
(Promise.all) instead of sequentially.

A row only turns green once its own probe confirms, so this reveals
structure + live progress without a premature green — complementing the
earlier 'render only after probing' intent rather than reverting it.

Per review feedback from @wyuc on #784.

* fix(token-plan): enrich seeded models with built-in thinking capability

Token Plan built ModelInfo objects from probed ids via modelInfoFromId(),
filling only streaming/tools/vision — so a model that supports configurable
thinking lost capabilities.thinking and InlineThinkingControl was hidden.

modelInfoFromId now takes an optional providerId and overlays the catalog
thinking capability for that (provider, model) pair; applyTokenPlan does the
same for its synchronously-seeded list. Added the Ark Agent Plan's dotted
aliases to the metadata table:
  - native Doubao Seed 2.0 family (doubao-seed-2.0-pro/code/lite/mini)
  - cross-vendor models the plan serves through its OpenAI-compatible endpoint
    (deepseek-v4-pro/flash, glm-5.2, kimi-k2.7-code/k2.6, minimax-m3/m2.7,
    ark-code-latest)

All verified against a live plan key: each accepts the gateway's unified
reasoning_effort field (low/medium/high) and actually reasons. They share the
doubao effort adapter, which disables via 'minimal' (not 'none') — matching
what the plan endpoint accepts (it rejects reasoning_effort:'none').

Addresses review point #1 from @wyuc on #784.

* Improve token plan capability setup UI

* fix token plan setup flow

* chore: prettier format tts-providers.ts
2026-07-03 11:03:12 +08:00
jackefnandYanpeng Wang 3b7a0ca0c0 feat(document): add multi-format course material upload (#741)
* feat(document): add multi-format course material upload

* fix(document): support office uploads with mineru cloud

* fix(document): handle mineru cloud office edge cases

* fix(document): align mineru cloud artifact metadata

---------

Co-authored-by: Yanpeng Wang <yanpg.wang@gmail.com>
2026-07-02 17:58:58 +08:00
wyuc 5daf3e5a6e fix(editor): keep emptied/zero-action scenes playable, bind outline by stable id, surface incomplete content (#814)
Editor empty-state robustness: dwell on zero-action scenes (engine + PlaybackChromeRoot) so emptied/blank slides stay playable; bind a scene's outline by stable outlineId (reorder/insert/duplicate-safe, allOutlines in scene order); clear the cue preview on glyph unmount; surface incomplete content calmly (amber dashed frame on clips, amber dot on blank outline titles + rail, neutral N/M-ready generate gate). Cross-reviewed (Codex + Claude), 5 rounds to a clean pass.
2026-06-30 05:57:26 -04:00
杨慎 09863af421 Add PBL v2 core schema and generation path (#795)
* feat: add PBL v2 core planner

* chore: restore unrelated lint comments

* fix: document PBL v2 scenario outline flags

* fix: expose PBL scenario subtype in outlines

* chore: clear PBL v2 completion stats lint warning

* chore: address PBL instructor code quality comments

* fix: address PBL v2 review findings

* chore: restore unrelated lint comments
2026-06-25 18:44:46 +08:00
wyucandClaude Opus 4.8 8d74639c19 fix(generation): show full key-point text in outline editor (#782)
Key-point chips in the pre-generation outline editor used a single-line
`truncate`, so longer points were cut off with an ellipsis and users could
not read the full text while reviewing the outline. Let the text wrap within
the chip's max width and add a native title tooltip as a fallback.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 04:43:05 -04:00
wyucandClaude Opus 4.8 dcf8a63333 fix(generation): make outline type changes take effect (interactive/PBL no longer downgraded to slide) (#772)
* feat(generation): changeOutlineType total constructor for valid type switches

Switching a scene outline's type in the editor only flipped `type` and left
the new type without its required config, so applyOutlineFallbacks silently
downgraded interactive/pbl back to slide. changeOutlineType returns a new
outline that strips foreign per-type config and seeds the target type's
required config from the shared fields, valid by construction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* i18n(generation): keys for interactive widget kind and PBL project config

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(editor): make outline type changes take effect with config forms

TypePill now builds a valid outline via changeOutlineType (replace, not
partial-merge) so a type switch drops stale config and seeds the new type's
required config — no longer silently downgraded to slide by applyOutlineFallbacks.
Adds InteractiveConfigDisclosure (widget kind + concept) and PblConfigDisclosure
(topic / description / target skills), mirroring QuizConfigDisclosure, so users
can configure interactive and PBL scenes that previously had no input UI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(editor): move type-specific config under the type pill

The quiz/interactive/PBL config disclosure rendered as a detached chip at the
bottom-left of the card, reading like a stray key point. Pin it to the
bottom-right of the key-points row instead, so it sits directly under the
TypePill it configures (type top-right -> its config bottom-right, same edge),
with key points flowing on the left.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(editor): cascade type config as a connected segment left of the type pill

Replace the standalone config chip with a cascading control in the header: the
type-specific config (quiz count / interactive widget kind / PBL project) renders
as a themed segment joined to the left of the TypePill, the two clipped into one
rounded pill split by a hairline divider. Reads as a single 'config › type'
cascade and groups the config with the selector it refines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(editor): address cross-review — preserve widget config, drop stale fields

- changeOutlineType: preserve an existing procedural-skill widget config
  instead of replacing it with a seeded simulation (codex P2 — task-engine
  fields were silently dropped on Interactive re-select).
- InteractiveConfigDisclosure.setWidgetType: reset widgetOutline to the shared
  concept only, so switching widget kind drops the previous kind's fields
  (claude P2 — language/gameType/etc. leaked across kinds).
- i18n locale-coverage test now also checks ko-KR and pt-BR.
- Drop a redundant WidgetType cast; add tests for procedural-skill preservation
  and pbl targetSkills dedupe/cap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(editor): round 2 cross-review nits — procedural-skill label + skill a11y

- InteractiveConfigDisclosure: when an outline carries a preserved
  procedural-skill widget, surface it as the current (selected) kind with its
  own label instead of mislabeling it 'Simulation', and make a same-kind
  re-select a no-op so its task-engine fields are never silently clobbered.
- PBL target-skill remove button uses a dedicated removeSkill aria-label.
- Add widgetProceduralSkill / removeSkill i18n keys (8 locales) + test coverage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(generation): make re-selecting the current outline type a no-op

changeOutlineType rebuilt the outline on every type-menu click, so re-selecting
the already-current type dropped fields it doesn't re-seed — legacy
interactiveConfig, or a partial pblConfig whose projectTopic is empty — turning a
harmless click into data loss. Short-circuit same-type selections to return the
outline untouched (codex P2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(format): prettier --write files left unformatted on main

These 14 agent/edit files (not part of this PR's feature) fail the repo-wide
`prettier . --check` in the required Lint job — they landed on main unformatted,
turning main's CI (and therefore this PR after a branch update) red. Formatting
them here unblocks the required check; pure formatting, no logic changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(lint): suppress intentional set-state-in-effect in reasoning timer

The reasoning timer does a deliberate one-shot setState in its effect to show
the right elapsed value before the first 1s interval tick. main's CI fails the
required Lint job on the new react-hooks/set-state-in-effect rule here; suppress
it with a justification (no behavior change). Unblocks the required check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 17:48:07 +08:00
wyucandClaude Opus 4.7 d9aecf8052 fix(settings): enforce "usable provider ⇒ concrete model" invariant (#580) (#581)
* fix(settings): enforce "usable provider ⇒ concrete model" invariant (#580)

Layer 1 — resolve provider AND model atomically at every source:
- add resolveSelectedModel(): like validateModel but no empty-input
  short-circuit, so it returns '' only when the provider has no models
- fetchServerProviders: symmetric LLM/image/video provider recovery +
  resolveSelectedModel; remove the asymmetric #577 isServerConfigured-gated
  LLM tail and the separate first-load auto-select block (now subsumed,
  and covers client-API-key providers too)
- setProviderConfig: entering an API key for the active/unset LLM provider
  resolves a concrete model in the same set()
- setVideoProvider: resolve a model on switch (was id-only); setImageProvider
  routed through the shared resolver

Layer 2 — remove states unreachable under the invariant:
- add shared isLLMProviderConfigured / hasUsableLLMProvider; gate the
  landing-page generate button on a usable provider (state A vs B) and drop
  the modelNotConfigured toast + forced SettingsDialog
- generation-toolbar: use the shared predicate; drop the dead "Select Model"
  label/tooltip fallback
- delete dead components/settings/model-selector.tsx (zero refs) and the
  now-orphan settings.selectModel i18n key (6 locales)

Note: generation-toolbar's `providerEntries.length === 0` branch is KEPT —
it is reachable via search (search-no-results), not the dead no-provider
state the issue assumed. modelNotConfigured i18n key kept (still used by
the chat path in use-chat-sessions).

Tests: resolveSelectedModel + hasUsableLLMProvider unit tests; server-sync
invariant tests (client-key resolution, atomic API-key entry, provider
switch, serverModels-preference + first-load regression guards).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(settings): address code review — symmetric image/video config resolution

Code review Important #1: fold atomic model resolution into
setImageProviderConfig / setVideoProviderConfig (design doc Layer 1 step 2
scoped these). Deleting the selected custom image/video model on the active
provider now resolves to a valid model in the same set() instead of leaving
it stale until the next server sync — restores LLM/image/video symmetry.

Minor #2/#3: document the deliberate store-level (credential path) vs
UX-level (incl. ≥1 model) "usable" split on isProviderUsable; clarify that
setProviderConfig also adopts providerId on the first-load API-key path.

Tests: +2 invariant tests (delete selected custom image/video model →
falls back to a valid model). 89/89 store tests pass, tsc clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(settings): keyless providers need explicit baseUrl to count as usable (#580)

Code-review/e2e follow-up. ollama/lemonade ship requiresApiKey:false +
a registry defaultBaseUrl, so the old isLLMProviderConfigured counted them
as configured on a fresh install — but buildFallback/reconcile never
auto-selects keyless providers, so hasUsableProvider could be true while
modelId stayed '', re-enabling generation with no model (the exact #498/#580
class of bug, just via the keyless path).

Narrow isLLMProviderConfigured to be consistent with isProviderUsable:
keyless ⇒ usable only once the user sets an explicit baseUrl (registry
defaultBaseUrl alone is not user intent); server-config and key-requiring
paths unchanged. Now gate + toolbar + reconcile agree, and genuine State A
(no usable provider) is reachable again.

Add e2e/tests/model-invariant-580.spec.ts: State A (no provider → generate
disabled + single "Set up model" affordance, no toast/forced dialog) and
State B (server-configured → concrete model auto-resolved, generation
enabled). +3 keyless unit tests. 92/92 store tests + 2 e2e pass, tsc clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(settings): re-resolve selection on bulk provider-config change (#580)

Self-test found: deleting the selected provider in Settings left the model
selector pointing at the deleted/invalid provider (or, via the component's
hand-rolled fallback, at an unusable built-in like openai with no key).

Root cause: setProvidersConfig (the bulk setter behind delete-provider /
delete-model) did ZERO re-resolution — Layer 1 covered setProviderConfig
(singular) + image/video setters + the reconcile, but not this path. The
component's confirmDeleteProvider then picked Object.keys()[0] (always a
built-in, ignoring usability), the exact per-call-site hand-rolled shape
#580 condemns.

Fix at the source: setProvidersConfig now re-resolves (providerId, modelId)
via the canonical isLLMProviderConfigured + resolveSelectedModel — keep the
current provider if still usable, else first usable, else State A. Removed
the brittle confirmDeleteProvider override (kept only its local Settings-tab
selection). Now every bulk config mutation (delete, import, reset) holds the
invariant.

Tests: +2 store tests (delete selected/only-usable provider → State A, not
the deleted/invalid one; delete non-selected → keep usable selection).
94/94 store tests, full suite green (only pre-existing ssrf DNS), tsc/eslint clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(settings): one shared LLM-selection resolver for all config edits (#580)

Clarified repro: the realistic case isn't deleting a custom provider — it's
clearing the SELECTED provider's API key so it becomes invalid. setProvider
Config only re-resolved when the edit made a provider *usable*; the
"became unusable" transition fell through (return base) → the now-invalid
provider stayed selected. The previous commit fixed only the bulk path.

Root-cause fix: extract one shared `resolveLLMSelection(config, providerId,
modelId)` — keep current provider if still usable, else first usable, else
State A; then resolve the model. Use it in BOTH setProviderConfig (single
edit: key add/clear, model-list edit) and setProvidersConfig (bulk: delete,
import, reset). The two paths can no longer diverge — exactly the per-call-
site asymmetry #580 set out to eliminate. Drops the now-unused
isProviderUsable import.

Tests: +2 store tests (clear selected provider key → drops stale selection;
clear non-selected key → keeps usable selection). 96/96 store tests, full
suite green except pre-existing ssrf DNS; tsc + eslint clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* style: prettier-format #580 test specs

CI 'Lint, Typecheck & Unit Tests' ran prettier . --check and failed on the
two added test specs (only tsc/eslint/vitest were run locally). Pure
whitespace reformat, no logic change — 96/96 store tests still green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 16:04:23 +08:00
Yizuki_Ame 47cc2a510c feat: add Brave Search and Baidu Search integration
Add Brave Search and Baidu Search providers with Baidu sub-source support, provider configuration updates, tests, and docs links.
2026-05-13 20:28:56 +08:00
wyuc a5209d747e feat(outline-review): clickable streaming card morphs into editor (#558)
* feat: add outline review preview flow

* style: refine outline review editor

* fix(home): make draft cache survive round-trips through generation flows

The `useDraftCache` hook initialised state via `useState`'s lazy
initialiser. Under React 18+ SSR, the initialiser runs on the server
(returns `undefined` because `window` is missing), the SSR'd state is
sent to the client, and React reuses it on hydration without re-running
the initialiser. As a result `cachedValue` was always `undefined` and
nothing ever rehydrated from `localStorage`.

The home page's `prevCachedRequirement` derived-state pattern then
masked the symptom: it initialised `prev` to the same (always
undefined) `cachedRequirement`, so the comparison never fired and the
restore branch was unreachable.

Switch to an effect-based read so consumers actually receive a state
update after mount, and replace the home page's derived-state restore
with a `useEffect` + `useRef` guard that only fills the input when it
is empty. Side benefit: PBL chat draft and quiz answer caches start
working too — they used the same broken pattern.

* feat(outline-review): clickable streaming card morphs into editor + UX overhaul

Major refresh of the outline review surface introduced in #198, applying
the placement design from docs/superpowers/specs. Key shifts:

Streaming card → editor morph
- The streaming preview card in the generation-preview page is now an
  interactive surface (tilt + hover-straighten + always-visible "点击审阅"
  affordance) that morphs into the full editor via shared `layoutId`.
- SSE keeps streaming uninterrupted into the expanded editor; the editor
  consumes either `session.sceneOutlines` or live `streamingOutlines` so
  scenes appear as they arrive and stay in sync after edits.
- A header collapse button shrinks back to the preview card. Mid-stream
  collapse keeps SSE running; post-stream collapse re-arms the same 2.5s
  auto-continue timer the no-review path uses, so the card doesn't strand.

Document-style editor
- Replaces the bordered card + accordion + numbered rail with a Notion-
  feeling stack: drag handle and number share a flex left rail aligned to
  the title baseline; title/description are inline-editable borderless
  textareas with auto-resize.
- ⋮ overflow menu and explicit move-up/move-down are removed; reordering
  is drag-only, with hover-revealed delete (popover-confirmed) next to the
  type pill.
- Hover-revealed `+` insert dividers between scenes; faint `+` at the
  list edges replaces the "Add scene" bar so insertion is uniform.
- Quiz config popover redesigned: stepper for question count, segmented
  control for difficulty, multi-select pills for question types so mixed
  types from the LLM round-trip correctly (the previous single-select
  silently truncated `questionTypes` to one element).
- Streaming placeholder row pulses below the latest scene during SSE.
- Scene types use a colour-coded pill that doubles as the type switcher.

State + flow plumbing
- Adds `previewPhase: 'review'` entry from the streaming visualizer via
  `outlineReviewIntentRef`; SSE-completion ORs in this intent when
  deciding whether to wait for confirmation.
- `localStorage.removeItem('requirementDraft')` moves to after the review
  promise resolves so "Back to requirements" actually restores the draft.
- Auto-start condition broadened so refreshes during early-review
  reattach to a fresh SSE rather than stalling on an empty editor.
- Six new locales (en-US, zh-CN, zh-TW, ja-JP, ru-RU, ar-SA) for the
  expand hint, streaming progress, collapse, delete confirmation,
  insert hint, and quiz summary; locale coverage test asserts the new
  keys plus the `{{count}}` interpolation contracts.

Removed: the redundant "Review outline" button + auto-continue caption
under the preview card, since the card itself is now the entry point.
Removed: the home toolbar "Outline" tab from #198 — placement design
explicitly rejects adding another always-visible toolbar control.

* fix(outline-review): address code review feedback (PR #558)

Single follow-up commit batching the issues raised in code review:

Critical
- E2E page object's reviewOutlineButton was hunting for the removed
  "Review outline" button. Re-target it to the streaming card's
  outlineExpandHint aria-label across all six locales.

Important
- waitForOutlineReviewChoice now accepts an AbortSignal and rejects on
  abort, so goBackToHome / unmount settle the parked promise instead of
  leaking the awaiting startGeneration closure. The existing AbortError
  catch swallows the rejection cleanly.
- Add CLDR plural variants for the count-bearing strings: en gets _one;
  ru gets _one and _few for quizConfigSummary; ar gets _one and _two
  for the three count-bearing strings. Bare key remains as _other
  fallback. (zh / zh-TW / ja have a single plural form.)
- Document the mid-stream collapse intent: dropping outlineReviewIntentRef
  is deliberate so SSE-completion takes the no-review path.
- Add keyboard reorder on the drag handle (Ctrl/⌘ + ArrowUp/ArrowDown)
  with aria-keyshortcuts; replace dragScene copy with dragSceneHint that
  surfaces the shortcut. Restores WCAG 2.1.1 reachability for reorder.
- KeyPointInput no longer commits on blur — only Enter / comma. Avoids
  surprise chips when users click away mid-typing.

Minor
- Drop unused i18n keys: reviewOutlineAction, moveSceneUp/Down, dragScene,
  sceneDescriptionLabel, keyPointsLabel/Placeholder, quizConfigLabel,
  addScene. Locale coverage test pruned to match actual usage.
- requestAnimationFrame-batch useAutoResize so a burst of edits doesn't
  thrash layout (forced reflow per keystroke).
- Reset isConfirmingOutlines in the no-parked-promise fallback path of
  handleConfirmOutlines so state doesn't linger.

* chore(outline-review): prettier format + drop redundant session check

- Run prettier on outlines-editor.tsx (CI Lint job was red).
- Remove `&& session` from the StepVisualizer onExpandOutline conditional;
  `session` is guaranteed non-null at that point by the early-return guard
  above (flagged by github-code-quality bot).

* fix(home): silence react-hooks/set-state-in-effect on draft restore

Match the existing eslint-disable convention already used in
app/page.tsx for the other localStorage-hydration effects. The setState
calls are deliberate: client-side state hydration from localStorage
must happen in an effect to remain SSR-safe (the lazy useState
initialiser runs on the server with `window` undefined).

* revert(i18n): drop CLDR plural variants until key-alignment check supports them

scripts/check-i18n-keys.mjs enforces strict key equality across all
locales, which is incompatible with i18next's per-locale plural-suffix
keys (a Russian locale needs `_one`/`_few`/`_many`/`_other`, an
Arabic one adds `_two`/`_zero`, while zh / ja need none). The check
script doesn't strip plural suffixes before comparing, so any partial
pluralization breaks alignment.

Roll back the partial fix from PR review #2 and revert to the bare
keys. The original "ungrammatical for n=1" wording stays, to be
addressed in a follow-up that also teaches check-i18n-keys.mjs to
ignore CLDR plural suffixes when comparing key sets.

* fix(outline-review): make streaming card expand-hint pill actually visible

The "Tap to review" pill was rendering but invisible in practice because:
- Both the bottom white gradient (z-auto) and the pill (z-10) sit inside the
  same parent stacking context, but motion.div applies a transform that
  creates per-element stacking contexts; the implicit ordering put the
  gradient above the pill in some browsers.
- text-[9px] + bg-blue-500/95 was too thin/translucent to read against the
  white gradient even when on top.

Make stacking explicit (gradient z-0, pill z-20), bump the pill to text-[10px]
font-semibold with solid bg-blue-500, larger Maximize icon, and lift it
slightly to bottom-2 so it sits above the gradient haze.
2026-05-11 19:44:31 +08:00
wyuc dbe1d2ec76 [codex] Add Bocha web search provider (#524)
* Add Bocha web search provider

* Address Bocha web search review feedback
2026-05-04 12:06:05 +08:00
wyuc 5e12ad892c [codex] add VoxCPM2 TTS provider (#496)
* add voxcpm2 tts provider

* address voxcpm review feedback

* translate voxcpm locale strings

* narrow toolbar model pill

* refine toolbar media controls

* address voxcpm review followups
2026-04-26 23:45:23 +08:00
wyucand杨慎 e613b7578f [codex] add per-model thinking config (#494)
* add per-model thinking config

* Refine thinking model controls

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
2026-04-26 19:41:10 +08:00
wyuc a95aa87bf1 support recent model providers (#481) 2026-04-24 15:29:12 +08:00
Shangqing TuandClaude Opus 4.7 c02a607377 feat: interactive mode clean (#461)
* feat: add interactive mode

* feat: add TeacherActions for interactive mode

* fix: better prompt for ultra mode

* feat: upgrade normal mode's interactive generation with 5 widget type and actions

* feat: make interactive mode button clear

* fix: remove unused TeacherOverlay

* fix: resolve PR #458 review feedback

- Revert unrelated infra changes (Dockerfile, docker-compose.yml)
- Fix placeholder strings in generateWidgetContent
- Rename ultraMode to interactiveMode consistently across codebase
- Extract duplicated patchHtmlForIframe to shared utility
- Fix widget store race condition by keying callbacks by sceneId
- Fix teacher-action ID collisions by using descriptive IDs in prompt
- Relax outline prompt constraints for flexibility
- Optimize sidebar thumbnail with IntersectionObserver lazy loading
- Add trailing newlines to modified files

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: update generateSceneActions call signatures

The function signature changed to individual parameters instead of
options object pattern after cherry-pick.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* style: fix prettier formatting

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: removed extra language arg

* style: fix prettier formatting v2

* fix:  add missing i18n keys

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 21:35:30 +08:00
杨慎andClaude Opus 4.6 082fcbcbbe fix(#419): hide Ollama from model selector when not configured (#420)
Keyless providers (e.g. Ollama) with a hardcoded defaultBaseUrl were
unconditionally passing the provider filter because `!requiresApiKey`
short-circuited to true. Now keyless providers require an explicit
user-provided baseUrl or server configuration to appear.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:33:06 +08:00
杨慎andClaude Opus 4.6 470aa5e42e feat: inline language inference for outline and PBL generation (#412)
* feat: inline language inference in outline generation

Replace manual language selection with automatic inference from user
requirement text. The outline generator now produces a languageDirective
that propagates through the entire generation pipeline.

Key changes:
- Rewrite Language Inference section in outline prompt with decision
  rules for foreign language learning, cross-language PDF, proxy
  requests, and terminology handling
- Reorder pipeline: outlines before agents, so agent profiles can
  use the inferred languageDirective
- Pass languageDirective through scene content/actions generation
- Add SSE streaming of languageDirective from outline generation
- Remove manual language selection UI

Eval test suite:
- 50 test cases covering 9 scenario types: single-language, language
  learning, immersive, explicit instruction, code-switching, minimal
  input, user profiles, cross-language PDF, locale mismatch
- LLM-as-judge evaluation with configurable inference/judge models
- 50/50 pass rate with gemini-3-flash-preview + gpt-4o judge
- Excluded from CI (requires LLM API keys)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove useless attributes in test case

* feat: pass languageDirective to PBL, remove hardcoded zh-CN/en-US

- Remove pblConfig.language field, use languageDirective from outline
  inference instead
- Delete all Chinese prompt duplicates in pbl-system-prompt.ts,
  agent-templates.ts; keep English templates with languageDirective
  injection
- Remove zh-CN/en-US branches in generate-pbl.ts (initial prompt,
  post-process context, welcome message)
- Pipe languageDirective through scene-generator → generatePBLContent
  → IssueboardMCP → agent templates
- Remove i18n template wrapping from PBL welcome messages in
  pbl-renderer.tsx and use-pbl-chat.ts; use LLM-generated content
  directly to avoid language mismatch when UI locale ≠ course language
- Update pblConfig schema in outline prompt (remove language field)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: clean up legacy language fields, pass languageDirective through buildSceneFromOutline

- Add languageDirective param to buildSceneFromOutline, combining with
  outline.languageNote via buildLanguageText
- Remove dead types: AudienceProfile, StylePreferences, LegacyUserRequirements,
  SceneOutline.language
- Remove Stage.language and all its propagation (storage, stageInfo, prompt-builder fallback)
- Remove GenerateClassroomInput.language and job store inputSummary.language
- Remove unused pdfLanguageSample from outline SSE route

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: persist languageDirective to IndexedDB, fix TTS preview and dead refs

- Add languageDirective to StageRecord and stage-storage write path
- Add DB v9 migration: convert legacy language locale codes to directives
- Use voice.language for TTS preview text instead of dead localStorage key
- Omit empty ## Language section in PBL agent prompts
- Use JSON.parse for extractLanguageDirective unescape (\uXXXX support)
- Remove dead i18n keys: toolbar.languageHint, pbl.chat.welcomeMessage

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:49:56 +08:00
wyucandClaude Opus 4.6 9a0060e2f0 feat(audio): add custom OpenAI-compatible TTS/ASR provider support (#357) (#409)
* feat(audio): extend TTSProviderId/ASRProviderId to support custom providers

* feat(audio): update provider registry helpers for custom provider support

* feat(audio): route custom providers to OpenAI-compatible implementations

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(audio): add custom TTS/ASR provider CRUD to settings store

* feat(audio): add dialog for creating custom audio providers

* feat(audio): add custom provider UI with voice management and delete

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(i18n): add custom audio provider translation strings

* fix(audio): resolve remaining type errors for custom provider support

Add `as keyof typeof` casts where TTSProviderId/ASRProviderId (which now
include template literal types) index into the built-in-only registries
TTS_PROVIDERS, ASR_PROVIDERS, DEFAULT_TTS_VOICES, and DEFAULT_TTS_MODELS.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(audio): allow custom provider IDs to reach switch default branch

The early guard `if (!provider) throw` blocked custom provider IDs from
reaching the switch-case default branch where they get routed to
OpenAI-compatible implementations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(audio): add default model field to dialog and polish voice list UI

- Add "Default Model" input to custom audio provider dialog (fixes
  Kokoro needing model:"kokoro" instead of fallback "gpt-4o-mini-tts")
- Redesign voice list as aligned table with column headers, hover
  reveal delete buttons, and empty state message
- Add Enter key support to voice add inputs
- Use consistent grid layout for add voice row

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): fallback to customDefaultBaseUrl when testing custom TTS provider

The baseUrl field in provider config is empty by default (it's an
override field). Custom providers store their default URL in
customDefaultBaseUrl, which needs to be used as fallback for test.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): auto-select first voice when adding to custom TTS provider

When a custom provider's voice is 'default' (initial state before any
voices are added), adding the first voice now automatically updates the
active voice selection so test TTS works immediately.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): fallback to customDefaultBaseUrl in all TTS call sites

Scene generator, discussion TTS, and agent bar voice preview all read
providerConfig.baseUrl which is empty for custom providers (the default
URL is stored in customDefaultBaseUrl). Add fallback chain to all 4
call sites so custom providers work during actual playback, not just
in the settings test.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): include custom provider voices in agent voice picker

getAvailableProvidersWithVoices only iterated built-in TTS_PROVIDERS,
so custom providers never appeared in the agent bar voice dropdown.
Now also iterates ttsProvidersConfig for custom-tts-* entries and
includes their customVoices in the picker.

Also updated getServerVoiceList, findVoiceDisplayName, and
resolveAgentVoice to handle custom provider voice lookups.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): fallback to customDefaultBaseUrl in all ASR call sites

Same issue as TTS: audio-recorder, asr-settings, and audio-settings
all read providerConfig.baseUrl which is empty for custom providers.
Add customDefaultBaseUrl fallback to all 3 ASR call sites.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(audio): add model list management for custom ASR providers

Custom ASR providers now support adding/removing models with a table
UI (same design as TTS voice list). The model selector dropdown shows
custom models and auto-selects the first one.

Also adds custom ASR providers to the Toolbar's ASR provider dropdown
and fixes custom provider handling in audio-settings.tsx (baseUrl
fallback, request URL preview, API key section visibility).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): seed customModels from defaultModel and fix ASR model label

- When creating custom ASR provider with a defaultModel, auto-add it
  to customModels so the model list isn't empty on first visit
- Fix ASR settings label from "TTS Model" to "Default Model"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): separate ASR model management from creation dialog

ASR models should be managed in provider settings (add/remove from
list, select default), not pre-filled at creation time. Unlike TTS
where a single default model suffices, ASR users need to manage
multiple models (e.g. whisper-small, whisper-large-v3).

- Remove Default Model field from ASR creation dialog (keep for TTS)
- Remove defaultModel parameter from addCustomASRProvider
- ASR models are added via the model list UI in provider settings

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): simplify ASR model management — first model is default

Remove the separate model selector dropdown for custom ASR providers.
The first model in the list is always used as default. Users manage
the model list only (add/remove), no separate "default" selection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): add loading state and detailed errors to ASR test

- Show spinner with "Processing..." while waiting for transcription
- Disable record button during processing
- Show specific error messages from API (e.g. "No transcription
  generated") instead of generic failure text
- Handle empty transcription result as an error with helpful message

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(i18n): use correct key for processing label in ASR test button

* fix(audio): polish custom provider UX and type safety

- Add proper type declarations for custom provider fields (customName,
  customDefaultBaseUrl, customVoices, etc.) eliminating 30+ unsafe casts
- Show custom ASR providers in toolbar with default language list
- Hide custom ASR providers without models from toolbar selection
- Add no-models warning in ASR settings panel
- Add delete confirmation dialog for custom TTS/ASR providers
- Add duplicate voice/model ID validation with toast error
- Add dedicated i18n keys for ASR model management (modelNamePlaceholder, addModel)
- Fix i18n key for ASR processing label (settings.asrProcessing)
- Add radio-button model selection in custom ASR provider settings
- Add visible OpenAI-compatible description in provider creation dialog
- Add CUSTOM_ASR_DEFAULT_LANGUAGES constant for toolbar language options

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): add isCustomTTSProvider/isCustomASRProvider to audio types mock

The settings-server-sync test mocked @/lib/audio/types as an empty object,
but settings.ts now imports isCustomASRProvider for provider validation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(audio): add customDefaultBaseUrl fallback in all TTS/ASR call sites

When a custom provider is created, the actual URL is stored in
customDefaultBaseUrl while baseUrl is initialized empty. Several call
sites only checked baseUrl, causing fetch failures during course
generation. Add fallback to customDefaultBaseUrl in:
- generation-preview/page.tsx (course generation TTS)
- tts-providers.ts (getCurrentTTSConfig)
- asr-providers.ts (getCurrentASRConfig)
- media-popover.tsx (TTS preview)
- tts-config-popover.tsx (TTS preview)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format generation-preview/page.tsx

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused ttsSpeedRange variable in media-popover

Flagged by CodeQL code quality check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-12 21:49:09 +08:00
杨慎andClaude Opus 4.6 00152890c6 chore: update Ollama logo to official version (#400)
* chore: update Ollama logo to official version

Closes #399

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: improve OpenAI and Ollama logo visibility across themes

OpenAI logo (white fill) was invisible on light backgrounds, and Ollama
logo (black fill) was invisible on dark backgrounds. Normalize OpenAI SVG
to black fill and apply dark:invert via a central MONO_LOGO_PROVIDERS set.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 17:41:37 +08:00
wyucand杨慎 3c687dd523 fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#326)
* fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#324)

- Move model selection from global ttsModelId to per-provider
  ttsProvidersConfig[providerId].modelId
- Remove supportsModelSelection flag, use models.length > 0 instead
- Add defaultModelId to TTSProviderConfig and ASRProviderConfig
- Add providerOptions field to TTSModelConfig for future extensibility
- Fix ElevenLabs hardcoded model_id: 'eleven_multilingual_v2'
- Fix MiniMax TTS using config.model instead of config.modelId
- Update model lists from provider docs:
  - Qwen: add qwen3-tts-instruct-flash, qwen-tts
  - ElevenLabs: add eleven_flash_v2_5, eleven_flash_v2
  - MiniMax: remove non-existent speech-2.8 series
- Replace custom model CRUD UI with simple Select dropdown
- Remove MiniMax-specific hardcoded model selector from audio-settings
- Add migration for old ttsModelId/asrModelId/model fields

* feat(tts): voice picker groups by provider:model, sets both model and voice

Voice selection in agent bar now groups voices by model. Picking a
voice from a model group sets both modelId and voiceId on the agent.
TTS settings shows available models as display-only chips.

- Add compatibleModels field to TTSVoiceInfo (OpenAI marin/cedar restricted to gpt-4o-mini-tts)
- Add modelId to agent voiceConfig and ResolvedVoice
- Add ModelVoiceGroup to ProviderWithVoices, build groups in getAvailableProvidersWithVoices
- Update AgentVoicePill and TeacherVoicePill to iterate modelGroups
- Thread modelId from voice selection through discussion TTS pipeline
- Replace model Select with display-only chips in tts-settings

* fix(tts): show provider name in voice picker group headers

* fix(tts): restore MiniMax speech-01 models per official API docs

* fix(tts): add MiniMax speech-2.8 models per official docs, default to speech-2.8-hd

* feat(tts): expand MiniMax voice list with recommended voices

* fix(tts): remove non-official MiniMax recommended voices, keep 10 from official list

* fix: address CR findings - align defaults, fix test mocks, ASR modelId fallback

* fix(tts): pass modelId in voice preview and classroom generation

* fix(tts): voice picker highlights only the exact model:voice match

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
2026-03-30 17:21:19 +08:00
wyucandHi-Jiajun 9badd59856 feat: add and expand MiniMax provider support (#182)
Add MiniMax LLM models (M2–M2.7), TTS provider (speech-2.8/2.6/02/01),
image generation (image-01), and video generation (Hailuo 2.3/02,
T2V-01). Unify base URLs to api.minimaxi.com and add MiniMax-specific
base URL normalization for the Anthropic-compatible endpoint.

Co-authored-by: Hi-Jiajun <3138947285@qq.com>
2026-03-30 12:41:43 +08:00
wyucand刘少杰 5adf91c6f3 feat(settings): add configurable model selection for TTS and ASR (#108)
Add ttsModelId/asrModelId to persisted settings, model management UI
for providers that support model selection, and propagate selected
model through preview, generation, and transcription flows.

Co-authored-by: 刘少杰 <liushaojie@cvte.com>
2026-03-30 11:29:54 +08:00
wyucandClaude Opus 4.6 694570783a feat(tts): add Doubao TTS 2.0 (Volcengine Seed-TTS 2.0) provider (#283)
* feat(tts): add Doubao TTS 2.0 (Volcengine Seed-TTS 2.0) provider

Add Doubao TTS as a new TTS provider with 17 voices (14 Chinese, 3 English),
streaming response parsing for Volcengine's chunked JSON format, rate-limit
error handling, and full integration across types, constants, provider
implementation, server config, settings store, i18n, and UI components.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ui(tts): split Doubao API key into separate App ID and Access Key fields

Show two input fields (App ID + Access Key) for Doubao TTS in settings,
instead of requiring the compound "appId:accessKey" format. The values
are combined internally so no changes are needed to the API layer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): address code review feedback for Doubao TTS provider

- Add TODO note explaining TTSRateLimitError's future use for retry/backoff
- Replace atob() + manual byte loop with Buffer.from() for server-side code
- Trim supportedFormats to ['mp3'] since only mp3 is used in requests
- Add i18n keys for Doubao App ID / Access Key labels (zh-CN + en-US)
- Add show/hide toggle to Access Key input field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 16:43:18 +08:00
ddb52241fb feat: add Discussion TTS with per-agent voice assignment (#211)
* feat(tts): add resolveVoice() and getServerVoiceList() utilities

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tts): add AudioIndicator equalizer bars component

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tts): add onSegmentSealed callback to StreamBuffer

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tts): add voiceOverrides field to AgentConfig and AgentTemplate

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tts): add useDiscussionTTS hook with audio queue and cleanup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tts): add audio state indicator to Roundtable bubble

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(tts): wire onSegmentSealed callback through chat sessions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tts): add per-agent voice dropdown to AgentBar

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tts): integrate useDiscussionTTS in Stage and pass state to Roundtable

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style(tts): refine voice dropdown to pill-style selector

Replace native select styling with a compact rounded-full pill
that blends into the agent row. Remove border, use muted bg,
smaller text, and a custom chevron icon.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style(tts): use shadcn Select for voice dropdown, link with TTS toggle

- Replace native <select> with shadcn Select component for consistent UI
- Hide voice dropdown when TTS is muted (ttsMuted)
- Compact pill-style trigger with rounded-full, no border, muted bg

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style(tts): add voice label prefix and always show dropdown

Show "音色: Alloy" instead of plain "Alloy" in the voice pill.
Always show dropdown regardless of mute state (voice config is
independent of playback).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style(tts): add volume icon hint in collapsed AgentBar

Show a small Volume2 icon in the collapsed pill when voice config
is available, hinting that voice settings are inside.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): fix voice dropdown layout and click handling

- Move voice pill below agent name (second line) to prevent
  horizontal overflow in English
- Wrap Select in div with onPointerDown stopPropagation to fix
  Radix click-through to parent row
- Add line-clamp-1 to descriptions for consistent row height
- Use items-start instead of items-center for better multi-line alignment

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(tts): redesign AgentBar voice layout for compactness

- Single-line layout: checkbox · avatar · name · role · voice pill
- Remove descriptions from agent rows (saves vertical space)
- Extract AgentVoicePill component to isolate Select event handling
- Smaller avatars (size-7), tighter row padding (py-1.5)
- Voice pill uses Volume2 icon + voice name (no prefix text)
- Works in both Chinese and English without overflow

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tts): cross-provider voice selection per agent

- Change voiceConfig from per-provider lookup to explicit
  { providerId, voiceId } per agent
- Each agent can use a different TTS provider's voice
- Voice picker dropdown groups voices by provider
- useDiscussionTTS routes TTS requests per agent's provider
- resolveAgentVoice falls back to global provider if no config

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): align role badge and voice pill across agent rows

Give role badge fixed width (w-14 text-right) so role text
and voice pills align vertically across all rows regardless
of agent name length.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): fix role badge and voice pill alignment

Wrap role badge + voice pill in a fixed-width container
(w-[9.5rem] justify-end) so both align vertically across
all agent rows regardless of name or role text length.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style(tts): align role badge and voice pill across agent rows

Add min-w-[52px] text-right to role badge so it starts at a
consistent position regardless of agent name length.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): use fixed w-[60px] for role badge alignment

* fix(tts): use fixed w-[88px] for voice pill alignment

* fix(tts): prevent click-outside from closing AgentBar when Select portal is open

* fix(tts): comprehensive voice picker rewrite

- Replace Radix Select with Popover + button list (fixes click issue)
- Fix getAvailableProvidersWithVoices to always include global provider
- Widen panel from w-80 to w-96 (prevents name truncation)
- Voice pill uses primary color instead of gray (more visible)
- Extract renderAgentRow helper to reduce duplication
- Popover shows voices grouped by provider with active state
- Add findVoiceDisplayName utility

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): align voice provider availability with toolbar logic

* fix(tts): fallback to first available provider when global provider has no voices

* refactor(tts): remove global provider fallback from voice resolution

Voice resolution now only depends on available providers (those with
API keys or server-configured). No more globalProviderId parameter.
Fallback is first available provider, then browser-native-tts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tts): add browser native TTS voices to agent voice picker

Load speechSynthesis.getVoices() in AgentBar and include as a
"Browser Native" provider group in the voice popover. No API key
needed - always available if browser supports it.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tts): simplify toolbar TTS to on/off toggle, add disabled state

Toolbar:
- Replace volume slider with simple TTS on/off toggle button
- Remove ttsMuted/ttsVolume/onVolumeChange props from CanvasToolbar
- Toggle now controls ttsEnabled (not ttsMuted)

AgentBar:
- Collapsed: show VolumeX icon when TTS disabled
- Voice pills show disabled state (gray, cursor-not-allowed, no popover)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(tts): simplify Settings TTS tab to toggle + provider config

Remove voice selection, speed slider, preview/test, Azure locale
filter from Settings TTS tab. Voice is now per-agent in AgentBar.
Keep: on/off toggle, provider selector, API key + base URL config.
Add hint text pointing to AgentBar for voice configuration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(tts): simplify media popover TTS tab to toggle only

* fix(tts): add voice config hint to media popover TTS tab

* feat(tts): add per-voice preview button in voice picker

Each voice row in the popover has a small speaker icon button.
Click to preview the voice with "欢迎来到AI课堂" / "Welcome to
AI Classroom" (follows i18n). Browser native uses Web Speech API,
server TTS calls /api/generate/tts. Click again or close popover
to stop. Shows spinner while generating.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): preview text follows course language instead of UI language

* refactor(tts): redesign AgentBar expanded panel layout

- Teacher always at top with voice pill (works in both modes)
- Mode tabs moved below teacher
- Auto mode: single compact row with shuffle icon + description
- Max turns: compact inline row with smaller input
- Preset mode: only student agents listed (teacher already above)
- Remove large shuffle animation from auto mode

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(tts): merge max turns into teacher row

* refactor(tts): separate teacher row and max turns, use stepper UI

- Teacher row: avatar + name + voice pill only
- Max turns: bottom row with MessageSquare icon + compact stepper
  (minus/number/plus in a rounded pill)
- Remove Input component dependency

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): increase voice pill contrast in dark mode

* fix(tts): make max turns input editable, tighten panel padding

* fix(tts): restore shuffle animation in auto mode (compact version)

* fix(tts): adjust auto mode text spacing and add voice auto-assign hint

* fix(tts): auto-close voice popover after selecting a voice

* fix(tts): increase auto mode vertical padding for better balance

* fix(tts): push auto mode text toward bottom with flex spacer

* fix(tts): reduce auto mode bottom padding

* feat(tts): wait for TTS audio to finish before next agent turn

Add waitForDrain() to useDiscussionTTS that returns a promise
resolving when the audio queue is empty. The agent loop in
useChatSessions now awaits this after buffer drain, so the next
agent's turn doesn't start until the current agent's TTS audio
finishes playing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): keep bubble visible while TTS audio is still playing

When buffer drains (text=null) but audio indicator is still active,
don't clear liveSpeech. Clear it only when audio state goes idle.
This keeps the speech bubble visible until TTS finishes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tts): hold discussion bubble until TTS audio finishes

When StreamBuffer fires the done signal (onLiveSpeech null), Stage
now checks if TTS is still playing. If so, it defers clearing the
bubble state. The bubble stays visible until onAllAudioEnd fires
from the TTS hook (queue empty + nothing playing), then clears.

This prevents the jarring UX where the bubble disappears while
the agent's voice is still audible.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): fix bubble hold - guard onStopSession instead of onLiveSpeech

Root cause: bubble disappears because doSessionCleanup fires via
onStopSession when the agent loop ends naturally, NOT because of
onLiveSpeech(null, null).

Fix: when onStopSession fires and TTS is still playing, defer
doSessionCleanup to onAllAudioEnd callback. Manual stop (user
presses button) still cleans up immediately via handleStopDiscussion.

Use doSessionCleanupRef to avoid circular dependency between
discussionTTS hook and doSessionCleanup useCallback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): guard BOTH onLiveSpeech and onStopSession for bubble hold

Two paths clear the bubble:
1. onLiveSpeech(null, null) from StreamBuffer done → clears liveSpeech
2. onStopSession → doSessionCleanup → clears all state

Both fire when agent loop ends. Path 1 fires first (tick loop),
path 2 fires after (waitUntilDrained resolves). Both must be
guarded when TTS is still playing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tts): hold bubble during TTS playback and respect playback speed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(tts): LLM picks voice matching agent persona during generation

- Client sends available voices (providerId + voiceId + name) to
  /api/generate/agent-profiles
- LLM prompt asks to pick a voice matching each agent's personality
- Parse "providerId::voiceId" from response, save as voiceConfig
- Fallback to index-based assignment if LLM doesn't pick
- Browser native voices hidden when server providers are available
- saveGeneratedAgents accepts and persists voiceConfig

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): restore volume slider in classroom toolbar

Revert the toolbar simplification from 36e3997 that replaced the
volume slider with a TTS on/off toggle. The volume control with
hover slider is a core classroom UX. TTS on/off is controlled via
Settings and Media popover instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): teacher uses global lecture voice in discussion when no voiceConfig override

* fix(tts): teacher always uses global lecture voice, no overrides

* fix(tts): sync playback speed to currently playing audio in real-time

* fix(tts): address code review issues

- Issue 2: enabled flag now checks ttsEnabled && !ttsMuted in stage.tsx
- Issue 4: remove unused browserAvailableVoices from useDiscussionTTS
- Issue 5: remove dead code in audio-settings.tsx (Slider, Loader2, handleTTSVoiceChange, handleTTSSpeedChange, handleTestTTS, testingTTS, ttsTestStatus, ttsTestMessage, testText, ttsSpeed, setTTSSpeed, and unused browser-tts-preview imports)
- Issue 6: shouldHold now checks queue length in addition to isPlayingRef
- Issue 8: hide AgentVoicePill for teacher row in agent-bar.tsx (teacher voice is controlled in Settings)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(tts): address PR review — abort preview fetch, defer error recovery

1. Add AbortController to voice preview server TTS fetch, abort on
   stopPreview to prevent stale responses on rapid switching
2. Use queueMicrotask for processQueue calls in error/ended handlers
   to prevent synchronous recursion if multiple items fail consecutively
3. Add ordering invariant comment on sealLastText's onSegmentSealed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tts): restore teacher voice pill, respect voiceConfig override

* fix(tts): sync volume and mute to discussion TTS audio in real-time

* fix(tts): allow browser-native TTS alongside server providers

* fix(tts): remove top padding from voice popover content

* fix(tts): make selectedAgents reactive to voiceConfig changes

* fix(tts): use agents record instead of listAgents() to avoid infinite loop

* fix(tts): single source of truth for teacher voice

Teacher voice pill now reads/writes global ttsProviderId + ttsVoice
(same settings used by lecture TTS). This ensures lecture and
discussion always use the same teacher voice. Student agents still
use per-agent voiceConfig.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add avatar descriptions for smarter LLM avatar selection

Each avatar now has a one-line description (appearance, vibe) sent
to the agent-profiles generation API. LLM picks avatars matching
agent personality instead of guessing from file paths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
2026-03-23 17:35:09 +08:00
3a3e27bac0 feat: add ElevenLabs TTS provider (#134)
* feat: add ElevenLabs TTS provider

* fix(tts): wire ElevenLabs provider config

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
2026-03-22 23:43:08 +08:00
Madoka e92d1958a6 feat: add Grok (xAI) support as an LLM, image generation, and video generation provider (#113)
* feat: add Grok (xAI) provider support for LLM, Image and Video generation

- Add Grok as LLM provider (OpenAI-compatible, 11 models including grok-4.20, grok-4, grok-3)
- Add Grok Image generation provider (grok-imagine-image, grok-imagine-image-pro)
- Add Grok Video generation provider with async polling (grok-imagine-video)
- Add env vars: GROK_*, IMAGE_GROK_*, VIDEO_GROK_*
- Add web search note: Grok search is via chat completions, not standalone API
- Update README/README-zh with Grok in supported providers

Closes #87

* fix: ensure new image/video providers are auto-added to persisted settings

Add ensureBuiltInImageProviders and ensureBuiltInVideoProviders functions
that run on every store rehydrate, mirroring the existing LLM provider
sync logic. Without this, Grok Image/Video providers would not appear
for users with existing localStorage settings.

* fix: add missing English translation for providerGrokVideo

* style: fix prettier formatting for grok video provider

* fix: remove unsupported grok multi-agent model and improve video error
2026-03-22 21:42:34 +08:00
wyucandClaude Opus 4.6 d0318e1326 fix(tts): fix browser-native TTS playback and add provider switching to toolbar (#153)
* fix(tts): fix browser-native TTS playback and add provider switching to toolbar (#145)

- Add speechSynthesis.cancel() before speak() in PlaybackEngine to fix
  garbled/broken audio output (Chrome known bug workaround)
- Replace flat voice selector with GroupedSelect in toolbar TTS tab,
  allowing direct provider switching (Qwen TTS ↔ Browser Native etc.)
- Dynamically populate browser-native voices from speechSynthesis.getVoices()
  instead of a single hardcoded "default" entry
- Split browser voices by language for easier navigation
- Use i18n for TTS provider names in toolbar

Closes #145

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve selectedGroup matching for duplicate groupId sub-groups

When browser-native-tts voices are split by language, multiple groups
share the same groupId. The GroupedSelect trigger now finds the
sub-group that contains the selected item, so it displays the correct
language label (e.g. "Browser Native TTS · English" instead of always
showing the first sub-group).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 16:05:06 +08:00
db5ac82167 fix: use browser speechSynthesis for playback when browser-native-tts is selected (#28)
* fix: use browser speechSynthesis for playback when browser-native-tts is selected

Previously, selecting browser-native-tts as the TTS provider would
produce sound in the settings test but remain silent during classroom
playback. This happened because:

1. The scene generator correctly skipped pre-generation for browser TTS
   (it runs client-side, not via API)
2. The playback engine fell back to a silent reading timer when no
   pre-generated audio was found, instead of calling speechSynthesis

This commit adds Web Speech API integration directly in the
PlaybackEngine:
- New playBrowserTTS() method speaks text via speechSynthesis
- Properly wires onend/onerror to advance to the next action
- pause()/resume() now handle speechSynthesis.pause()/resume()
- stop() and handleUserInterrupt() cancel browser TTS

Fixes #25, fixes #12, fixes #5

* fix: handle "default" ttsVoice, extract CJK_LANG_THRESHOLD constant

- When ttsVoice is "default" (set by Browser Native TTS which has no
  voice picker), the voiceURI lookup silently fails and no lang is set,
  causing Chinese text to be spoken with an English voice.

- Extract the 0.3 CJK detection threshold as a named constant
  CJK_LANG_THRESHOLD with JSDoc explaining the rationale.

- Fall through to language auto-detection when voice lookup fails,
  regardless of the reason (missing voice, "default" sentinel, etc.).

* fix: support browser-native tts previews

* refactor: extract shared TTS preview logic into useTTSPreview hook

The browser-native and API-based TTS preview code was duplicated across
tts-config-popover, media-popover, and tts-settings. Extract it into a
reusable useTTSPreview hook that handles refs, cancellation, audio
lifecycle, and staleness checks in one place.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: yangshen <1322568757@qq.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 22:25:29 +08:00
wyucandClaude Opus 4.6 843d33e5cc chore: enforce Prettier formatting and fix lint issues
- Add .prettierignore to exclude vendor packages, lock files, markdown, and YAML
- Update .prettierrc: printWidth 100, singleQuote, trailingComma "all"
- Run Prettier across all source files for consistent formatting
- Fix unused imports (UserRequirements, setTTSProvider)
- Fix eslint-disable comment placement after Prettier reformat

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 16:09:45 +08:00
wyucandClaude Opus 4.6 71559690e4 feat: redesign MediaPopover with tab layout, grouped selects, and TTS preview
Replace the card-based media popover with a modern segmented-control tab UI.
Add GroupedSelect for provider+model selection (filtered to available providers),
TTS voice preview with audio playback, voice name localization, and remove
unsupported speedRange from qwen-tts provider.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:16:33 +08:00
wyucandClaude Opus 4.6 cc44f32ca8 Refactor: directory structure cleanup + Google Fonts localization
- Replace next/font/google with local fonts (geist + @fontsource-variable/inter)
  to eliminate build-time network dependency
- Resolve lib/utils.ts vs lib/utils/ naming conflict (cn() → lib/utils/cn.ts)
- Move React Context Providers from lib/store/ to lib/contexts/
- Consolidate hooks into lib/hooks/ (use-theme, use-i18n)
- Organize loose components into subdirectories:
  agent-* → components/agent/
  canvas-* → components/canvas/
  generation-toolbar, media-popover → components/generation/
  speech-button, tts-config-popover → components/audio/
  proactive-card → components/chat/

65 files changed, lint 0 errors, build 28 pages passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:35:06 +08:00
wyucandClaude Opus 4.6 0d20abfed9 Initial commit: OpenMAIC AI classroom platform
Next.js 16 App Router application with:
- AI-powered interactive classroom generation from PDF/text requirements
- Multi-agent discussion system (teacher, students, assistant roles)
- Provider abstraction for LLM, TTS, ASR, image, video, web search
- Local-first data architecture (IndexedDB/Dexie)
- i18n support (zh-CN/en-US)
- Structured logger, SSRF guard, lint/build passing (0 errors)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 16:41:07 +08:00