mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-19 05:49:27 +08:00
@openmaic/generation@0.2.0
41
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
5d5acd6215 |
feat: claude search (#393)
* Added search tool support for Claude * New locales for i18n * Updated translations * Prevent default web search base URLs from being prepopulated * fixed coding style issues * fixed coding style issues * fixing lint errors * fix: add SSRF protection to fetchPageContent in claude.ts Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/099e53fb-e1db-49f4-af7f-4f17cb3d364a Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fix: address all remaining review feedback from PR review thread Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/e1ab8fe2-fe0d-45a5-a0af-30945b99c268 Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fix: rename ambiguous variable names in settings store for clarity Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/e1ab8fe2-fe0d-45a5-a0af-30945b99c268 Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fixing lint errors * fix: address 5 items from second PR review thread Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/09469d80-8e91-449b-8bc2-9c11f83555df Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fixing model verification error * fix: address 3 items from third PR review thread Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/0aa49d19-df5b-4a08-a7e1-2295ae9cfc38 Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fixing lint errors * fix: resolved Copilot review comments * fix: returning string promise for ssrf-guard * fix: returning string promise for ssrf-guard * fix: updated locales, added allowed_callers for claude search tools, handle duplicated claude model keys * fix: updated locales, added allowed_callers for claude search tools, handle duplicated claude model keys * updated tests * using Anthropic SDK for search and connection test in search settings. * uses @ai-sdk with a custom fetch including allowed_callers in payload instead of adding Anthropic SDK as a direct dependency * fixed format * removed @anthropic-ai/sdk as direct dependency * fix: resolve TypeScript errors in claude web-search tests Cast mock.calls through unknown to avoid tuple-index type errors on Vitest mock objects whose parameter types are inferred as empty tuples. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add Claude as a web-search provider on the multi-provider framework Forward-port of the original Claude web-search work onto main's searchWeb dispatcher, per review guidance on #393: - lib/web-search/claude.ts: searchWithClaude via the AI SDK Anthropic provider and Anthropic's native web_search server tool, called through callLLM for usage accounting. Returns the model's answer plus cited sources deduplicated by URL. The page-content enrichment fetch from the original PR is removed entirely, resolving the P1 SSRF finding (no server-side fetches of attacker-influenced result URLs remain). - allowed_callers: ["direct"] is still injected at the fetch layer; the AI SDK's provider-defined web_search tools do not emit it. - Tool version picked from the model: web_search_20260209 for 4.6+ Opus/Sonnet, basic web_search_20250305 for Haiku/older; maxResults maps to the tool's max_uses. - Registry/dispatcher/route wiring: claude provider entry, claudeModelId passthrough (web-search route, classroom generation, generation preview), WEB_SEARCH_CLAUDE_* env config with server-pinned model via WEB_SEARCH_CLAUDE_MODELS, and client base-URL allowlisting for api.anthropic.com. - Settings: per-provider modelId with a model selector in the Claude panel; locale keys added to all ten languages. - Tests: new adapter suite plus claude cases in the dispatcher, constants, route, provider-config, web-search-config, and settings store suites. Closes #392 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E9KqAGmKCtBwMC4HtT4JKE * fix(web-search): trim whitespace around the Claude API key and base URL Keys are pasted by hand into Settings, and nothing in the client key path trimmed them: a trailing newline or space was sent verbatim as the x-api-key header value and came back from Anthropic as 'invalid x-api-key', with nothing in the message pointing at whitespace. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(web-search): correct Claude adapter tool selection, base URL, and timing Addresses the four P2 review findings on #393: - Route dated Claude 4.0 ids (claude-sonnet-4-20250514, claude-opus-4-20250514) to the basic web_search_20250305 tool. The blocklist regex missed them, so they were sent web_search_20260209, which pre-4.6 models reject. Replaced with an allowlist of the models that have dynamic filtering, so unrecognized ids also fall back to the basic tool -- the safe direction, since 4.6+ models accept both. - Pin allowed_callers: ["direct"] only on web_search_20250305. Injecting it into every request disabled the code-execution caller that provides dynamic filtering on web_search_20260209. - Normalize the bare Anthropic root to /v1 before handing the base URL to the SDK, which appends /messages verbatim and would otherwise 404. The adapter is the single choke point for the client allowlist, the server env var, YAML, and the default; other hosts pass through untouched. - Return responseTime in seconds, matching every sibling adapter's WebSearchResult contract instead of exposing milliseconds for Claude. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
ceb05adb41 |
feat(media): allocate generated assets through the registry (#1007 part 2, step b) (#1039)
* feat(media): establish shared asset ownership primitives Introduce global browser asset-pool ownership, asset-reference collection, stage reclamation planning, and lease-based URL access. Carry allocated media identity through storage and generation boundaries with regression coverage. * fix(media): enforce safe resolution across every consumer Route image, video, thumbnail, presentation, and video-export consumers through one resolution state machine. Prevent opaque allocated or generated references from reaching render and export sinks, with fallback and ownership tests. * fix(media): protect document-owned assets across mutations Allocate pool bytes before compatibility writes and document commits, then roll back uncommitted generations safely. Preserve document ownership across edits, retries, imports, speech generation, scene changes, and stage deletion. * fix(media): scope retries and tighten ownership guard Scope retries to the target scene and slide and refuse ambiguous shared-reference mutations. Expand retry rendering coverage and keep direct pool URL resolution behind the shared lease owner. * fix(storage): avoid nested lock during stage cleanup Execute prepared reclamation plans against an explicitly deleted document so the compatibility cascade cannot re-enter the per-document lock. Cover deletion of stage-owned media rows even when the document has no references. * refactor(media): confine reclamation to stage deletion Remove inline pool and compatibility-row cleanup from element, speech, scene, and audio replacement flows. Keep whole-stage reclamation behind explicit stage ownership, preserve stage-less legacy audio rows, and document deferred document-truth sweeping. * fix(media): close retry and resolution gaps Restore shared source tasks after successful forks, scope retries across both whiteboard locations, and wait for parallel TTS workers before rollback. Resolve background media through import, rendering, and PPTX export paths while keeping retry controls visible over last-good bytes. * test(media): execute the consumer safety matrix Replace source-substring checks with resolver seam execution across all six UI consumers, stage hydration, and both exporters. Pin each rollback layer independently and harden the ownership guard against aliased pool imports. * chore(packages): publish the additive DSL field Bump the DSL patch version for the optional speech-action field. Keep the transitional reclamation policy app-owned and document the legitimate transaction rollback removals there. * fix(media): make retry rollback task-safe Delay allocation task re-keying until final document reconciliation succeeds, restore shared source tasks on failed forks, and require exact stage-whiteboard targets before falling through from a missed scene. * fix(media): clear private assets and refresh leases Delete the asset-pool database during the confirmed local-data wipe. Notify the app lease layer after same-id replacement so mounted consumers re-resolve current bytes without reaching into storage internals. * test(media): pin closure safety guards Exercise the CSS allocation boundary, unfiltered legacy-row ownership, delete-and-undo byte survival, and real distinct consumer seams. Restore the video-only manifest overwrite condition and share the direct video resolution hook across both element variants. * test(media): narrow rollback element assertion Narrow the reconciled slide element to an image before checking its source so the rollback regression remains type-safe under the full root compiler configuration. * fix(media): close lease refresh races Gate the first batch publication by unique resolved refs, then publish every replacement snapshot without mutating the prior React state object. Serialize invalidation behind pending releases, evict rejected refreshes, and register replacement observation at the pool boundary. * fix(media): reopen pool after clear failures Always evict the singleton once its store has been closed, including blocked and failed database deletion paths. Report blocked deletion as deferred and prove a later write uses a fresh live store. * fix(media): isolate shared retry progress Track forked regeneration under the selected element until it receives a fresh asset identity, leaving the shared source task and bytes untouched. Surface targeted failures through renderer task lookup and skip the redundant fork reconciliation lock. * fix(media): close final asset retry gaps Keep blocked asset clears fail-loud until a successful retry, with actionable settings guidance. Clear failed shared-fork state across durable and live key spaces, and pin renderer lookups, lease publication identity, and committed-ref rollback protection. * fix: preserve actionable retry failures Localize the blocked cache-clear recovery hint across every supported locale and pin the deferred-error mapping. Retain durable fork failure rows while retries run, deleting them only after successful generation so unstructured failures survive reload. * fix(media): hydrate legacy stored video thumbnails Home-page recent-video thumbnails regressed for legacy Dexie mediaFiles rows keyed by gen_vid placeholders: the reworked hydration resolved the row's bytes through the sealed resolver but never surfaced the stored blob (and its poster) as object URLs for the preview card, so the CI recent-video-thumbnail e2e specs found no visible element. Hydration now materializes legacy stored video rows into blob URLs for both the element src and poster while keeping the resolver invariants: opaque refs still never reach a DOM src, and concrete addresses are never blanked. Unit pins cover the seam so the vitest suite catches this class without a browser. * fix(media): resolve sole restored legacy video Classroom playback restored tasks by exact document media references. Legacy gen_vid references can outlive the key used by the one persisted video row, leaving the player on a placeholder even though bytes were restored. Select the sole completed stage video only for legacy sequential refs after exact and reconciled matches. Keep exact failures authoritative, refuse ambiguous candidates, and cover success, ambiguity, and failure precedence in unit tests. * fix(media): scope legacy video recovery Decide restored legacy video recovery once from the complete document and record it through the shared task lookup consumed by playback, editing, and resolved slides. Keep ambiguous documents as placeholders, apply the same decision to thumbnail hydration, and preserve exact failure precedence. * fix(media): exclude claimed video recovery tasks Model restored legacy recovery as a two-pass match across document video elements and task rows. Remove tasks claimed by exact, targeted, or placeholder lookup before applying the sole-candidate fallback, and cover the ownership/cardinality matrix. * fix(media): unify video element resolution Centralize source, task, poster, and legacy recovery decisions for every video consumer. Ensure direct URLs win over opaque refs and play_video waits on element-targeted retry tasks. * refactor(media): route legacy recovery through resolver Let document-aware consumers request legacy video recovery through the unified element binding API, keeping thumbnail hydration on a single decision path. * fix(media): preserve import refs and prefer pool bytes Recognize unambiguous extensionless relative media addresses during classroom import. Resolve allocated export and thumbnail assets from the shared pool before falling back to lagging compatibility rows. * fix(export): preserve concrete video sources Route PPTX video elements through the shared media binding resolver so an unresolved opaque reference cannot replace a playable source. Complete browser and persisted-store cleanup when asset-pool deletion is deferred, while retaining distinct hard-failure behavior. * fix(media): fork retries without exclusive ownership Enumerate logical asset owners across every persisted document before allowing global pool replacement. Thread explicit targets through fresh-id rewrites and cover cross-document aliases plus unreadable ownership. * fix(media): guard global asset reclamation Share a fail-closed persisted-document liveness check between stage deletion and retry replacement. Preserve cross-document pool aliases while deleting stage-owned compatibility rows and cover enumeration failures. * fix(media): cover complete slide asset references Route slide media traversal through a shared mutable slot contract so backgrounds participate in export, thumbnail hydration, collection, and rewrite lifecycles. Snapshot complete surviving-document refs once per reclamation and preserve manifest-only owners while retaining fail-closed behavior. * fix(media): preserve exclusive retry asset ids Allow targeted retries to replace exclusively owned pool assets in place. Keep shared and unprovable ownership paths on fresh allocations, and pin production-shaped retries plus compatibility-row cleanup. * fix(media): revalidate asset bindings at completion Recheck repository-wide ownership before replacing generated media and fork scoped retries when exclusivity changed. Route video export selection through the unified resolver and keep concrete posters independent of task state. * fix(media): count unflushed owners and broadcast replacements The completion-time exclusivity proof read only the persisted document, but slide duplication updates the Zustand aggregate synchronously and schedules persistence behind a debounce. A retry finishing inside that window saw a single persisted owner and replaced the bytes behind a reference the duplicate also held. The proof now also counts owners in the live stage snapshot when that snapshot represents the stage being retried, so an unflushed duplicate forks instead. Same-id replacement notifications were realm-local, so a second tab showing the same classroom kept its lease pinned to the superseded blob URL. The notification now travels over a BroadcastChannel; each receiving realm runs its own observers against its own pool, so a spoofed message can at most force a re-resolve. A missing or failing channel never fails the replacement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): bind replacement listeners and spare shared audio rows A realm that only renders never sends a replacement, so binding the channel from the sender path left passive tabs deaf to peers. Binding now happens where the observer is registered, when the asset-pool module loads, and the receiving realm resolves its own pool lazily so a cleared or unavailable pool degrades to the next resolve instead of throwing. Observer notifications ran under Promise.all, so a rejection surfaced after BrowserAssetStore.replace had already committed and turned a durable success into a reported failure. They are settled individually now; the callback is wrapped because a synchronous throw would otherwise escape before allSettled sees the array. audioFiles rows are keyed globally by audioId, so deleting a stage removed the sole row for an id a surviving document still referenced — playback and both export paths read that table directly and cannot fall back to the preserved pool blob. Rows are now filtered against surviving references, while a failed enumeration still withholds only the irreversible pool removal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): serve replaced bytes from the pool everywhere Unknown survivor liveness deleted every planned audioFiles row. Those rows are keyed globally by audioId, and playback plus both export paths read the table directly, so losing one is as irreversible for them as removing the pool entry. Unknown liveness now preserves the rows too, leaving bounded garbage for a later pass that can prove exclusivity. The earlier pool-first change covered PPTX, video collection and thumbnail hydration but missed classroom ZIP export, which still serialized the stale compatibility row after a lagged same-id replacement, shipping media the classroom no longer renders. Auditing every direct reader of the media and audio tables surfaced the same gap in playback: speech regeneration also replaces bytes under a stable id and does not roll the pool back when the compatibility write fails, so the player kept serving superseded narration. It now resolves the pool first and falls back to stored rows for legacy and imported audio. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(audio): replace exclusively owned speech clips in place regenerateSpeechAudio accepted the action's audioId but always passed undefined as replaceAssetId, so regenerating an exclusively owned pool-backed clip allocated a new asset every time, rewrote the action and orphaned the previous pool entry and compatibility row until stage reclamation — contradicting the stable-id path generateAndStoreTTS already implements for media. Ownership is now established before synthesis, and the rule itself moved to the shared reference module so media retries, poster replacement and speech regeneration consume one implementation instead of restating it. An exclusively owned clip keeps its id and has its bytes replaced; a shared clip, a legacy id with no pool entry, or unprovable ownership still gets a fresh allocation so other holders keep their audio. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(media): drop the import left behind by the ownership move Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): resolve audio pool-first and fence peer realms Stable-id TTS regeneration commits replaced narration to the pool before the audioFiles mirror write, so a failed mirror leaves the row stale. AudioPlayer already resolved pool-first, but audioObjectUrl, collectAudioFiles and the video timeline dependencies still read Dexie directly and would serve the superseded clip. All allocated-audio readers now share one resolver, with Dexie kept as the fallback for legacy and imported rows. The exclusivity proof modelled unflushed owners in the active realm only, so another tab duplicating the same asset during its save debounce could still be observed as a single owner and have its bytes replaced globally. A peer's pending state cannot be read across realms, so presence is probed instead: any realm holding the stage forces the fork path. A deferred asset-pool deletion no longer reloads the page. The database is still on disk and the guidance asks the user to close the other tab and retry, which the reload discarded. The decision moved into a shared helper so the rule is pinned rather than living inline in the component. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): fail closed when presence cannot be probed The presence helper documented that an unanswerable probe must count as a peer, but every unavailable path — no BroadcastChannel, a constructor that threw, a send that failed, and the window before the pool's asynchronous binding completes — returned false, so the ownership proof cleared a single local owner and replaced globally shared bytes in place. Probing now returns present, absent or unknown, and only a probe that was actually sent and went unanswered is absent; the ownership decision treats unknown exactly like present. The pool declares its binding intent synchronously so a probe issued during the load-time window waits for the bind instead of concluding that presence is unavailable, and releases that gate if the import fails. Coverage reaches the write boundary: with presence unknown, a production-shaped targeted retry forks to a fresh id instead of calling replace, and the original bytes stay intact for a peer's unflushed owner. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(dsl): bump to 0.6.3 after the release dedupe took 0.6.2 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(i18n): add the deferred-clear guidance to fr-FR The locale landed on main after this branch added the key, so the alignment check flagged it as the one missing translation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
|
b4e3145c90 |
feat(roster): single-source the generated agent roster on the stage document (#994)
* feat(roster): single-source the generated agent roster on the stage document The generated agent roster used to live in two places at once: the stage document carried generatedAgentConfigs, while a per-stage IndexedDB mirror (db.generatedAgents) was the actual read authority — and the only home of each agent's voice binding. The split caused real losses: voices vanished on device switches, exports, and shares (the mirror never travels), the load path trusted a device-local table over the portable document, and roster edits persisted through a bespoke debounce with no beforeunload flush. Make the stage document the single source of truth: - Contract: add optional voiceConfig/voiceDesign to GeneratedAgentConfig in @openmaic/dsl (VoiceDesign moves into the contract; the app re-exports it). Additive optional fields — not a breaking serialized-shape change, so DSL_VERSION stays put per the version policy. - Writes: remove every mirror writer (registry saveGeneratedAgents, the stage store's debouncedSaveAgents bypass, classroom-load hydration, import direct writes). setStageAgents now only marks the stage dirty for the shared persistence scheduler plus synchronous in-memory registry/selection mirrors; generation and import embed the roster on the stage. - Reads: classroom load hydrates the registry from the loaded stage's generatedAgentConfigs; export reads the stage roster directly. - Lazy migration: the mirror is retained read-only. On load, a roster (or voice fields) missing from the document is backfilled from the mirror, committed onto the in-memory stage, and persisted by the next flush — idempotent, and gated on the current stage id after every await so a classroom switch cannot leak a stale roster into the registry. - Deletion: deleteStageData discards the deleted stage's pending persistence work so a queued flush cannot resurrect the document. - Round trip: classroom ZIP manifests now carry the voice fields, so export/import preserves agent voices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): harden deletion, migration, selection, and ingress paths Review hardening on top of the roster single-source change: - Deletion vs in-flight persistence: deleting a stage now registers a session-scoped tombstone (lib/utils/deleted-stages.ts) before the cascade starts. Every persistence landing point checks it — the scheduled flush and the departing-stage retry drop their snapshots, and the aggregate/incremental saves re-check under the document lock, including the incremental path's full-save fallback that would otherwise rebuild a deleted document from the in-memory snapshot. A deletion that fails before the document is removed lifts the tombstone so the still-existing stage keeps persisting. - Lazy migration termination: a fruitless legacy-mirror probe (nothing to merge — e.g. server-generated rosters, which are voiceless by design) is remembered per stage for the session instead of re-querying the mirror on every load forever. Successful merges are not memoized so a failed flush is retried on the next load. Docstrings now state the real lifecycle. - Selection provenance: setStageAgents no longer overwrites a user-set agent selection with the full roster. Stage-derived selections still track the roster; a user-set auto selection is only narrowed to surviving agents (newcomers are not auto-selected, the user-set flag is untouched), and a user-set preset selection is left alone — matching restoreAgentSelection. - Mirror hygiene: deleteStageData clears the deleted stage's rows from the legacy roster mirror (best-effort), and the mirror table's docs now describe its actual access pattern (migration reads + deletion hygiene). - Registry persistence: the agent registry's persist snapshot partializes generated agents out of localStorage, making the stage document the roster's only durable home; the rehydration merge filter remains as defense in depth. - Ingress validation: imported manifest voice fields are structurally validated (malformed bindings dropped per field, the agent survives), and applyGeneratedAgentsToRegistry validates providerId against the known TTS provider registry instead of casting, treating unknown providers as "no bound voice". - Contract: drop the speculative AgentVoiceConfig.modelId — an audit of every roster producer, current and historical, found none that ever emitted it. Contract fields are added once a producer exists, not before. New tests cover the deletion tombstone at both the store and storage layers (including the missing-destination full-save pin and the lock-race re-check), the fruitless-probe memo, the selection-provenance matrix, persistence exclusion of generated agents, and voice-field sanitization on import. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): close tombstone lifecycle, probe-memo, and selection-intent gaps Second hardening pass over the deletion/migration/selection fixes: - Tombstone lifecycle: a deleted classroom id can legitimately come back (deletion is client-side only). Explicit (re)creation points now lift the tombstone — server-copy restore in applyClassroomStageAndScenes and backup restore in importDatabase — so edits after a same-session restore persist again, while in-flight flushes stay fenced. - Failed deletion now restores the dirt it discarded: deleteStageData snapshots the pending map plus the in-flight round before tombstoning, and re-marks it when the delete fails with the document still present. - Legacy-mirror probe memo now distinguishes a FAILED read (null, retried on the next load) from a confirmed-empty mirror (memoized), so a transient IndexedDB error cannot suppress migration for the whole session. - Incremental save tail (currentScene KV row + chat sessions) is tombstone- fenced like the aggregate path, and both paths re-check the tombstone immediately before every write, with comments scoped honestly for the lock-free (no Web Locks) best-effort LWW fallback. - Roster-edit selection mirror: an empty intersection now falls back to the full roster and clears the user-set flag (matching restoreAgentSelection's length > 0 gate), and a user-set selection that equals the pre-edit full roster keeps tracking the roster wholesale so newcomers are not excluded forever by the AgentBar auto-toggle snapshot. - TTS/ASR provider lookups use Object.hasOwn instead of `in`/bare indexing, so prototype-chain keys ('toString', 'constructor', ...) no longer pass the provider whitelist or resolve to Object.prototype members. - Documented the per-tab limit of the tombstone set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): replace the boolean deletion tombstone with a deletion-generation model A boolean tombstone cannot express the deletion lifecycle once a deleted id legitimately comes back (server-copy restore on revisit, backup import): after the restore lifts the flag, a pre-delete flush still in flight is indistinguishable from a post-restore edit and can overwrite the restored document with pre-delete content. Two rounds of point fixes kept leaking variants of this race, so this restructures the fence instead. deleted-stages.ts now keeps a per-stage monotonic deletion epoch plus a deleted flag. markStageDeleted bumps the epoch; unmarkStageDeleted clears only the flag and never rewinds the epoch. Every persistence path captures the epoch at the moment it captures the data it will write (flush-round snapshot, departing-stage snapshot, aggregate save entry) and re-checks "captured epoch is current AND not deleted" immediately before each individual write. Invariants: - A write captured before a deletion can never land after it, even when a same-id restore has lifted the deleted flag (the captured epoch is permanently stale). - A write capturing data after a restore observes the current epoch and persists normally. - A failed deletion lifts only the flag; the restored pending changes are re-queued as descriptors, so their flush re-captures current store state under the current epoch and is neither dropped nor a stale replay. Also fixed in the same restructuring: - Incremental tail writes (currentScene, chats) each re-check independently; a delete landing while the first tail write is awaiting now fences the chat write instead of riding the earlier check. - Deleting a classroom evicts it from the warm in-memory store, and loadFromStorage treats a warm-but-deleted stage as not loaded (discarding the ghost), so navigating Back to a deleted classroom reaches the server-restore path instead of rendering an editable ghost whose every edit is silently dropped. - importDatabase records each document's pre-import deletion state and reinstates it when a failed import rolls the document back, so an outstanding flush cannot recreate the rolled-back document. - Docstrings on the failed-deletion restore path now state that only dirt captured before the deletion is restored; edits refused during the deletion window are not part of the snapshot. New tests cover the epoch invariants at the storage layer (pre-delete rounds dropped across restores on the incremental, aggregate, and tail paths; post-restore captures landing; strictly increasing epochs), the scheduler layer (departing-retry delete+restore straddle; restored dirt flushing under the current epoch), the warm-ghost restore driven through the real loadFromStorage, and the import lift/rollback pair. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): report fenced drops, gate ghost eviction on cascade settlement Edge hardening around the deletion-epoch fence; the epoch model itself is unchanged. - An epoch-stale drop is now reported, not disguised: saveStageData / saveStageDataIncremental return a distinct 'stale-dropped' status (entry check, every in-mutation re-check, and both tail-write re-checks) instead of a shape identical to success. saveToStorage reacts by skipping all success bookkeeping - no chatSnapshot rebind to a snapshot that never landed, no pending-dirt clearing - and returns false, keeping its "true means verified write" contract honest. persistDirtySnapshot keeps its existing (correct) semantics explicitly: a drop has nothing to retry. - Warm-ghost eviction is settlement-gated: the deletion cascade records an in-flight bit (begun by deleteStageData, settled in its finally), and the deleted-warm branch of loadFromStorage keeps the warm state while the cascade is unsettled - a delete can still fail before removing the document, in which case the warm state plus the restored pending dirt is the only copy of the pre-delete edits. Once settled, the ghost discard and server-restore path behave as before. - Read-side landing re-check: loadFromStorage re-checks isStageDeleted after hydration, immediately before set(), mirroring the write-side "re-check immediately before landing" discipline - a lock-free mid-cascade read can no longer re-materialize a ghost classroom. - clearStoreForDeletedStage re-checks isStageDeleted at eviction time, so a same-id restore completing inside the cascade-tail window is never wiped. - capturedEpoch is now a required parameter on both storage entry points: the default (call-time capture) silently reopened the capture-point/ validation-point split for future callers; the type system now enforces the pairing. All production callers already passed it explicitly. New tests cover the dropped-status contract, the cascade in-flight lifecycle (including both failure shapes), keep-warm during an unsettled delete with end-to-end pending restore after a failed delete, settlement- then-eviction on success, the eviction guard against a tail-window restore, and the read-side hydration re-check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): settle the deleted-warm load by awaiting the cascade outcome Hardening around the deletion settlement gate introduced previously; the epoch model is unchanged. - The deleted-warm branch of loadFromStorage no longer completes the load while the deletion cascade is undecided (which exposed a classroom whose edits were refused without being restorable, and left a successful deletion's eviction with no follow-up reload). It now parks on a per-stage settlement promise (stageDeletionSettled) and branches on the outcome: a failed delete keeps the warm, restored classroom; a successful delete falls through to a full cold load so the server-restore path recovers the route; a navigation that moved on during the park is stopped by the load token guard. The eviction (clearStoreForDeletedStage) deliberately stops claiming a new load token so the parked load stays current - ghost re-materialization is fenced by the read-side deleted re-checks instead. No deadlock: a parked load holds no document lock, and the cascade never waits on a load. - 'stale-dropped' now propagates through the debounced flush path: persistDirtySnapshot returns the sentinel instead of an empty failure set, and startFlushRound skips the chatSnapshot rebind on it (mirroring saveToStorage) - a falsely rebound baseline would let later chat saves no-op-skip chats that never landed and corrupt cross-tab conflict detection. Pending-map clearing keeps its empty-failure semantics (restored descriptors mint fresh revisions). - The cascade in-flight state is a counter (begin++/settle--) instead of a boolean, and deleteStageData is single-flight per stage (a concurrent second call joins the first cascade), so overlapping deletions can never expose an undecided cascade as settled. - The StaleDroppedSave docstring no longer overpromises: a tail saveCurrentScene can land after the cascade's clearCurrentScene; the orphaned cursor row is ignored by the load path rather than removed by the cascade. Net +8 tests: overlapping-cascade counting, settlement-promise resolution (success and failure), concurrent double-delete single-flight join, the parked mid-cascade Back in both outcomes plus the navigation-away guard, and the fenced flush round keeping an honest chat baseline end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): re-verify store identity on keep-warm resume, re-run joined deletes after a mid-cascade restore Three hardening refinements on the deletion/persistence fencing: - loadFromStorage's parked keep-warm branch now mirrors the success path's store-identity guard: if a tokenless writer (e.g. a database import) replaced the stage during the park, a failed deletion falls through to the cold load instead of reporting another classroom as the kept warm state. The ghost-discard block is correspondingly gated on the deleted flag so the fall-through does not mislabel a surviving document as a ghost. - deleteStageData: a joined call carries its own deletion intent. When the first cascade fulfills but a same-id restore lifted the deleted flag in the meantime, the joined caller now runs one fresh cascade against the restored document instead of reporting the pre-restore outcome. Exactly one re-check per call (no recursion); a rejected first cascade still propagates unchanged to every joined caller. - Narrowed the failed-delete restore-ownership comments: the restore covers only the pending map plus an in-flight flush round's dirt; a departing-stage snapshot is outside that capture and is fenced and dropped by design. Comment-only, no behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): gate deleted-warm handling on stage identity, keep an explicit empty roster authoritative Final-audit fixes, three items: - loadFromStorage keyed its deleted-warm settlement handling (park / keep-warm / ghost discard) behind the scenes.length > 0 shortcut, so a zero-scene warm ghost of a deleted classroom skipped it entirely: the cold load found no document, the ghost stayed in the store, and the classroom loader's server-fallback gate (!getCurrentStage()) never ran — the silently-uneditable trap this branch exists to close, in its scene-less form. Deletion handling now keys on stage identity alone, before any scene-count shortcut; the non-deleted warm skip still requires scenes. - rosterNeedsLegacyFallback collapsed "roster field absent" (document predates roster persistence) with "explicitly persisted empty roster". A future writer persisting [] on a device whose read-only mirror still held stale rows would have its emptied roster resurrected on every load. The full-lift branch now triggers only when the field is absent (undefined); an explicit [] is authoritative and never probes the mirror. No current writer produces [], but the read side no longer depends on that invariant. - The GeneratedAgentConfig docstring claimed the voice fields are not a breaking change; that only holds for this codebase's tolerant structural validators. The generated stage.schema.json sets additionalProperties: false, so consumers pinning an older published schema artifact reject documents carrying the new fields. The docstring now states both sides honestly. Tests: a zero-scene deleted warm ghost is restored through the real loadFromStorage (fails pre-fix), an explicit empty roster does not resurrect stale mirror rows (fails pre-fix), and the rosterNeedsLegacyFallback unit matrix covers undefined vs []. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): re-queue aggregate persistence when a failed deletion is recovered Direct aggregate saves (saveToStorage: generation completion, server-restore hydration) run outside the pending map and the flush round, so the deletion path's pre-delete dirt snapshot cannot describe them. When a deletion bumped the epoch while such a save was in flight, the save was correctly fenced ('stale-dropped') — but if the deletion then failed before removing the document, the recovery only re-queued the snapshotted scheduler descriptors. The fenced aggregate-only content stayed memory-only with no retry anywhere, and a later reload lost it. The failed-deletion restore now merges a full re-mark of the aggregate (structure / stage / outline / currentScene / chats plus a descriptor for every current scene) into the recovery set, still guarded on the store holding the stage. Because the flush recaptures the CURRENT store state under the CURRENT epoch, this necessarily carries whatever the fenced aggregate save held — and, for the same reason, edits refused by the scheduler during the deletion window, which previously stayed memory-only as well; the scope docstrings now say so instead of declaring them excluded. New integration coverage drives the real store scheduler and the real storage layer end to end: a genuinely fenced in-flight saveToStorage (generation completion) plus a pre-removal deletion failure ends with the completion flag durable on the next flush; a deletion-window scene edit survives the same way; and a successful deletion re-marks nothing (control). Both recovery tests fail against the previous restore logic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
Yanpeng Wangandwyuc
|
9670100363 |
feat(extraction): media (audio/video) extraction + AliDocMind provider (#887)
* feat(extraction): add AliDocMind provider + media extraction abstraction Adds AliDocMind (Aliyun Document Mind LLM version) as a new vendor alongside unpdf/MinerU, and introduces the media (audio/video) extraction layer that mirrors the document extraction one. Document side (file: pdf/docx/pptx/xlsx/images): - PDF_PROVIDERS gains an `alidocmind` entry; parseWithAliDocMind() maps layouts[] -> ParsedPdfContent. Flows through the existing document extractor registry, so AliDocMind is selectable anywhere MinerU is. - PDFParserConfig / DocumentExtractorConfig gain accessKeyId/accessKeySecret (AliDocMind uses AK/SK, not a single apiKey); env fallback via ALIDOCMIND_ACCESS_KEY_ID / ALIDOCMIND_ACCESS_KEY_SECRET. Media side (mp4/mp3/wav/... -> MediaArtifact): - New lib/media-parse/ domain mirroring lib/pdf/ (types/constants/providers). parseMedia() maps AliDocMind segments[]/audio_frames/video_frames -> MediaArtifact (transcript + keyframes). - MediaExtractorProvider interface + media registry + extractMedia() entry, symmetric to DocumentExtractorProvider / extractDocument(). - MediaArtifact and the ExtractionResult/Artifact/Error/Job envelope live in lib/document/types.ts (re-exported from @/lib/document). Shared AliDocMind SDK wrapper (lib/pdf/alidocmind-client.ts) handles the submit -> poll -> get flow for both sides via @alicloud/docmind-api20220711. Tests: env-gated smoke test (tests/document/alidocmind.smoke.test.ts) drives a real PDF and a real video through AliDocMind; media-artifact type test; extractor-registry test updated for the new provider. Part of #621 (MAIC ETL). Media extraction is the sibling to the document extraction landed in #741. * feat(extraction): wire AliDocMind AK/SK through settings UI + routes Surfaces AliDocMind in the (post-#837) Document Parsing settings panel as a peer of unpdf/MinerU — one panel, one credential entry, more supported formats. Threads Aliyun AccessKey ID/Secret from the store through the extraction routes. - Store: pdfProvidersConfig + setPDFProviderConfig gain accessKeyId/accessKeySecret; default alidocmind entry added. - pdf-settings.tsx: AliDocMind branch renders AccessKey ID + Secret inputs (secret masked with show/hide) and a Test Connection button; request-URL preview shows the DocMind endpoint. Provider auto-appears in the panel via PDF_PROVIDERS; supported-format badges come from #837's registry (ALIDOCMIND_MIMES added to lib/document/mime.ts). - verify-pdf-provider route: alidocmind branch verifies AK/SK via a lightweight authenticated probe (verifyAliDocMindCredentials) — auth-level errors fail, anything else passes. - extract-document route + generation flow (app/page.tsx, generation-preview): accessKeyId/accessKeySecret carried through session → FormData → config. - i18n: alidocmindAccessKeyId / alidocmindAccessKeySecret in 8 locales. - Icon: reuse /logos/bailian.svg (Aliyun family) instead of a missing asset. Verified end-to-end against the running app: AliDocMind panel renders, Test Connection returns "连接成功" through the real Aliyun API. Part of #886. * feat(extraction): route audio/video uploads through extractMedia() (reuse document path) Media uploads now flow through the same upload picker, /api/extract-document route, and generation pipeline as documents — no separate upload area. Only the extraction differs: media mimes dispatch to extractMedia() -> MediaArtifact, which is flattened to the text shape the generation pipeline already consumes. - mime.ts: register audio/video formats in DOCUMENT_FORMATS (accept string, extension map, badges resolve for them); add MEDIA_PROVIDER_SUPPORTED_MIME_TYPES + SUPPORTED_MEDIA_MIME_TYPES, kept separate from PROVIDER_SUPPORTED_MIME_TYPES so the document drift-guard stays document-only. mimesForProviders() folds in a provider's media mimes, so the existing upload helpers (getAcceptStringForProviders / isMimeSupportedByProviders / format badges) cover media automatically when the provider supports it. - extract-document route: media mimes dispatch to extractMedia(); MediaArtifact flattened to timestamped text (synopsis + transcript + keyframes). - Tests: 6 media cases in mime.test.ts (accept/validation/badges/normalization). Verified: uploading a real video through the route returns synopsis + timestamped transcript/keyframes as text (curl, real AliDocMind key). Part of #886. * feat(extraction): use Alibaba Cloud icon for AliDocMind provider Replace the placeholder bailian.svg with a dedicated Alibaba Cloud icon mark (the square symbol from the official wordmark, text removed) so the provider reads as Aliyun rather than Bailian. Square aspect matches the other provider icons. Source: Alibaba Cloud / Alibaba Group brand assets. * style: prettier format AliDocMind + media extraction files * fix(extraction): harden AliDocMind — SSRF guard, cred verify, env gate, table text Addresses review findings on the AliDocMind provider: - SSRF (high): the media branch of /api/extract-document now runs the same validateUrlForSSRF check on a client-supplied baseUrl as the document branch, so an audio/video upload can't point the server's Aliyun SDK at an internal host. - Credential verify (high): verifyAliDocMindCredentials now whitelists success signals instead of blacklisting auth errors. Probed against the real API: valid creds + bogus job returns a no-throw "BizIdNotExistOrResultExpired" body; invalid creds throw InvalidAccessKeyId.NotFound. Only a no-throw response or a "biz-not-found" business error counts as valid — an unreachable endpoint, a localized error, or throttling now correctly reports failure instead of a false "connection successful". - Env-fallback gate (med): resolveCredentials no longer reads ALIDOCMIND_ACCESS_KEY_ID/SECRET unconditionally. Env fallback is opt-in via allowEnvFallback, which the route sets only for a server-managed provider, so an unauthenticated client request can't silently run on the server account. - getDocParserResult error body (med): fetchResult throws on a non-200 result envelope instead of returning {} (empty text presented as success). - Media pagination (med): stop after the first page when segments[] are present — layoutNum/layoutStepSize address layout blocks, not media segments, so re-requesting would loop over the same segments to the safety cap. - Table content (med): tables/charts carry content in llmResult, not markdownContent; the layouts→text mapping now prefers llmResult for those types so table content isn't dropped (tables: true was advertised). - Dead code: remove getCurrentMediaParseConfig (referenced non-existent store fields; the single-panel UI reuses pdfProvidersConfig). - Dedup: media MIME list lives only in lib/document/mime.ts (ALIDOCMIND_MEDIA_MIMES); the media registry imports it. - Tests: smoke test asserts durationMs is in ms (>1000 for a ~52s clip) to guard a ms/s unit mismatch; uses allowEnvFallback for the env-cred path. Verified with the real key: PDF + video smoke tests pass; verify classifies valid vs invalid creds correctly. Part of #886. * feat(extraction): extract AliDocMind images to base64 (parity with unpdf/MinerU) AliDocMind embeds figure/picture image URLs inside each layout's markdownContent (markdown ``), not a dedicated field, and the URLs are short-lived OSS signed links. Previously we emitted `images: []` and left the expiring URLs inside the extracted text. Now, for figure/picture layouts we: - parse the OSS image URL out of markdownContent, - fetch it at extraction time (before the signature expires) and re-encode to PNG base64 via sharp — the same base64 `images[]` contract unpdf/MinerU produce, so downstream storeImages → IndexedDB → slide works unchanged, - populate metadata.pdfImages + imageMapping (the generation flow prefers pdfImages), - strip the remote-URL markdown from the emitted text so expiring links don't leak into the prompt. Downloads run concurrently; a failed/`sharp` image is dropped, never failing the whole parse. Also fixes the prior over-broad table handling: only `table` blocks read llmResult; `figure` is treated as an image (chart-figure llmResult still kept in text). Verified with the real key: a sample PDF yields 24 base64 images in both images[] and metadata.pdfImages, and no oss-cn-hangzhou URLs remain in text. Part of #886. * docs(test): note AliDocMind video smoke test is non-deterministic server-side * fix(extraction): correct AliDocMind pageCount (pageNum is 0-based) Verified against a real response: AliDocMind reports pageNum 0..13 and pageCountEstimate 13 for a 14-page document — both are 0-based. The metadata pageCount previously used pageCountEstimate directly, undercounting by one. Use the already-1-based maxPage, falling back to pageCountEstimate+1 only when no blocks were seen. Document the 0-based convention at the normalization site. * fix(extraction): address AliDocMind review — verify/SSRF/config/media (P1+P2) Resolves all P1/P2 findings from the cross-review on #887. P1 (blocking): 1. verifyAliDocMindCredentials now inspects the no-throw response body.code. An OSS-only key returns NoPermission without throwing; previously that was green-lit as "connection successful" then failed at extraction. Only a success/200 or the job-not-found probe code is accepted. Deterministic mocked test added (tests/document/alidocmind-verify.test.ts). 2. verify route trust boundary: managed → server-owned AK/SK + default endpoint only (ignore client values); unmanaged → client creds only, never env fallback, and the client endpoint is SSRF-validated before signing. 3. image fetch hardened: restricted to Aliyun OSS hosts, redirects disallowed, per-image byte cap, image-count cap, bounded concurrency (was unbounded Promise.all over provider-returned URLs). 4. AliDocMind is now selectable in the generation toolbar — availability recognizes the AK/SK pair, not just apiKey. P2 (correctness): 5. Explicit server-config for the AK/SK pair (applyAliDocMindFallback + resolveManagedAliDocMindCredentials); verify and extract now resolve managed/env identically instead of verify-uses-env / extract-rejects. 6. Poll loop checks body.code before status, so a body-level error (e.g. NoPermission) fails fast instead of retrying for the full 15 min. 7. Image page numbers preserved through fetch/filter — no longer hard-coded to page 1, so multi-page image→page association is correct. 8. Empty media extraction (no synopsis/transcript/keyframes) returns 422 PARSE_FAILED instead of HTTP 200 with empty text. 9. Format matrix trimmed to the official contract: images JPG/JPEG/PNG/BMP/GIF (dropped WebP/JP2), media MP4/MKV/AVI/MOV/WMV/MP3/WAV/AAC (dropped M4A). 10. A document-only provider (unpdf/mineru) uploaded with a media file now returns a clear 4xx instead of an opaque 500. P3: credential-verify failures return INVALID_CREDENTIALS 4xx (not INTERNAL_ERROR 500); formatTimestamp emits HH:MM:SS past one hour. Verified with the real key: verify classifies valid→ok and invalid→fail; PDF (24 images, correct page numbers) and MP4 extraction pass end-to-end. * fix(extraction): make AliDocMind selectable + correct analysis label for media Two UI/UX fixes found while manually testing the AliDocMind flow end-to-end: - Persisted-state backfill: add ensureBuiltInPDFProviders so a PDF/document provider added after a user's settings were persisted (AliDocMind) is backfilled into pdfProvidersConfig on rehydrate. Without it the provider never appeared in the store, so it couldn't be selected and never picked up its server-configured flag. Wired into both persist migrate() and merge(), mirroring the existing image/video/web-search backfills. - Analysis step label: getGenerationStepText showed "解析 document 文件" for audio/video (the type map fell through to the literal "document"). Documents keep their precise token (PDF/DOCX/PPTX/XLSX/images); audio/video now use a dedicated, locale-correct string (generation.analyzingMediaMaterial, added to all 8 locales) instead of forcing a format token into the "{{type}} 文件" template. Manually verified end-to-end (real key): PDF, MP4, and MP3 (audio extracted from the sample video) each extract and drive full course generation; a DocMind-restricted AK/SK (OSS-only) now correctly fails verification with NoPermission (400) instead of a false "connection successful". * fix(extraction): address 2nd-round AliDocMind review (managed creds, stream cap, empty verify) Resolves the three follow-up findings on #887: 1. [P1] YAML-managed AliDocMind creds now reach extraction. Both extract paths (document + media) previously cleared the managed AK/SK and relied on an env-only fallback, so a YAML-only deployment verified but failed to extract. They now resolve server-owned creds via the shared resolveManagedAliDocMindCredentials() (env OR YAML), matching the verifier. Regression test added for YAML-only creds with no ALIDOCMIND_* env. 2. [P1] Image download is now size-capped while streaming. Instead of buffering the whole response and checking length afterward, the body is read chunk by chunk with a cumulative byte count that aborts the moment it exceeds the cap — a missing/false Content-Length can no longer exhaust memory. Host allowlist tightened to oss-*.aliyuncs.com. Tests: non-OSS host refusal, no-Content-Length overflow abort, declared-oversize rejection. 3. [P2] Empty verification body no longer counts as success. Removed the codeStr === '' branch from the positive-signal whitelist (a working key always returns the job-not-found business code for the bogus probe id). Regression test added for empty and absent bodies. Rebased onto main (package.json: kept @openmaic/storage + AliCloud deps). Verified with the real key: PDF + MP4 extraction still pass end-to-end. * fix(extraction): merge AliDocMind AK/SK into a baseUrl-configured YAML entry Follow-up to the YAML-managed credential fix. When a YAML `pdf.alidocmind` entry also specifies `baseUrl`, the generic loadEnvSection() (pdf requires a baseUrl) creates the pdf.alidocmind entry copying only apiKey/baseUrl/models/ proxy — never AK/SK. applyAliDocMindFallback() then returned early because the entry already existed, so the provider was "managed" but had no usable credentials, and resolveManagedAliDocMindCredentials() returned undefined (verify + extract both silently lost the creds). Merge the AK/SK into the existing entry instead of returning early. Added a regression test with baseUrl + accessKeyId + accessKeySecret together (the previous test omitted baseUrl, so the generic loader skipped the entry and the bug was masked). * fix(extraction): don't mark AliDocMind managed without AK/SK; align poll code check Two edge cases from a follow-up adversarial pass: - [MED] A YAML `pdf.alidocmind` entry with `baseUrl` but no AK/SK (and no env AK/SK) made the generic loader create the entry → isServerConfigured=true (managed) → but resolveManagedAliDocMindCredentials() returned undefined, so the provider was locked out AND client-entered AK/SK were silently dropped. applyAliDocMindFallback now deletes a credential-less entry so the provider stays UNMANAGED (clients supply their own creds). Regression test added. - [LOW] verify accepted a `code: "success"` body but the extraction poll loop threw on any non-"200" code — a success-shaped status would pass verification then fail extraction. The poll loop now treats "200"/"success" as benign, matching verifyAliDocMindCredentials. --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
c56929510c |
feat(web-search): add SearXNG provider support (#842)
* feat(web-search): add SearXNG provider support Enable self-hosted SearXNG via SEARXNG_BASE_URL, with server-side provider fallback, settings sync, and tests. Co-authored-by: Cursor <cursoragent@cursor.com> * style: fix Prettier formatting in web-search files Restore pnpm check by applying Prettier to imports, headers, and test assertions. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(web-search): keep SearXNG base URLs server-managed only Close the SSRF primitive by ignoring client-supplied SearXNG URLs at the route boundary and rejecting them in resolveSafeClientWebSearchBaseUrl. Add negative route/config tests and an opt-in live smoke test for the JSON API. Co-authored-by: Cursor <cursoragent@cursor.com> * style: fix Prettier formatting for SearXNG web-search changes Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: 徐松 <song.xu@aieshanghai.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
jackefnandRowan_lxb
|
baff0998bb |
Feat/document bundles milestone 3 (#844)
* feat(document): support document bundles * fix(document): avoid server extractor imports in client bundle * fix(pdf): default mineru backend to pipeline * fix(document): address bundle review findings * fix(document): harden extraction limits and storage errors * fix(document): make analysis step format agnostic --------- Co-authored-by: Rowan_lxb <Lxb_savior@163.com> |
||
|
|
b4c1c580f8 |
fix(generation): localize scene generation errors (#894)
Co-authored-by: wsun1 <wsun1@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
jackefnandYanpeng Wang
|
3b7a0ca0c0 |
feat(document): add multi-format course material upload (#741)
* feat(document): add multi-format course material upload * fix(document): support office uploads with mineru cloud * fix(document): handle mineru cloud office edge cases * fix(document): align mineru cloud artifact metadata --------- Co-authored-by: Yanpeng Wang <yanpg.wang@gmail.com> |
||
|
Yizuki_Ame
|
243e9f3a48 |
feat: auto-retry transient failures during scene generation (#788)
Adds automatic retry with exponential backoff for scene generation (content/actions/TTS): shared withGenerationRetry helper at the fetch boundary, smart error classification (retries 429/5xx/timeouts/network, fails fast on 401/403/404), AI SDK RetryError unwrapping, abort-aware backoff, per-route thinking config preserved, and upstream HTTP status surfaced for client retry classification. Fixes #755. |
||
|
wyucandwyuc
|
267965d6ed |
feat(generation): infer concise courseTitle from outlines for readable course names (#756)
* feat(generation): infer concise courseTitle from outlines for readable course names
Generated courses were named after the raw user prompt (truncated to 500 chars
client-side, or 50 chars / first outline title server-side), which is long and
unreadable. The outline-generation step already understands the full course
semantics, so have it emit a concise `courseTitle` alongside `languageDirective`
and `outlines`, and use it as the stage name.
- Prompts (requirements-to-outlines, interactive-outlines): top-level JSON gains
a required `courseTitle` (≤30 chars, in the teaching language, noun phrase).
- Streaming route: extract `courseTitle` head-bound like `languageDirective`,
emit a `courseTitle` SSE event, and include it in the `done` event.
- Non-streaming outline-generator: parse and return optional `courseTitle`
(defensive trim + 120-char cap).
- Naming: stage.name = courseTitle || outlines[0]?.title || requirement fallback
(client preview path backfills stage.name after outlines resolve; server
classroom-generation uses it directly).
- Session: persist `courseTitle` on GenerationSessionState for reload safety.
- Tests: add courseTitle parsing coverage; e2e mock done event carries it.
Fully optional in the pipeline — every consumer falls back to the previous
behavior when the field is absent, so legacy/LLM-skipped cases are unaffected.
* fix(generation): make courseTitle propagation robust across all outline paths
Address review feedback on the courseTitle threading.
Prompts — align every outline-prompt schema statement to the 3-key shape.
The user-prompt "final reminder" / top-level shape blocks for
requirements-to-outlines, interactive-outlines, and task-engine-outlines still
demanded the old two-key {languageDirective, outlines} object; as the last
instruction the model reads, that caused it to omit courseTitle and the stage
name to fall back to the raw requirement. Task Engine prompts now emit
courseTitle as well.
Streaming route — normalize the streamed courseTitle (trim, ignore
whitespace-only, cap length) to match the non-streaming parser, and add a
full-buffer fallback so a title emitted after the outlines array or beyond the
head-scan window is still recovered before the done event.
Client — reset latched languageDirective/title on outline retry so a
succeeding attempt that omits them falls back instead of inheriting a failed
attempt's stale value; also carry courseTitle in the stream-end resolve path
that fires when the stream closes without an explicit done event.
courseTitle remains optional on every consumer path.
---------
Co-authored-by: wyuc <zdq1204@gmail.com>
|
||
|
wyuc
|
e83db6a531 |
fix(agents): persist AgentBar voice picks and mode/selection across reloads (#723)
* feat(audio): let resolveAgentVoice honor per-agent voice overrides * feat(settings): persist per-agent voice overrides * fix(agents): persist AgentBar voice picks via settings overrides * fix(classroom): keep user's agent mode and selection across loads * fix(agents): only carry user-set agent selection across classrooms Review follow-ups: stage-derived defaults written by a classroom load were re-read as a user choice on the next load, so visiting a preset classroom permanently downgraded auto classrooms to preset agents. Track the origin with a persisted agentSelectionIsUserSet flag (set by AgentBar interactions, cleared by fallback writes). Also: voice overrides are now clearable (undefined deletes), the override type is single-sourced in voice-resolver, and persist rehydration of the new fields is pinned by tests. * fix(agents): don't mark stage-derived selections as user choices Round-2 review: clicking the already-active mode tab no longer flags the selection as user-set (a visual no-op must not promote stage defaults to a choice), generation-preview clears the flag when it writes the generated roster, and tests now pin that setter writes actually reach storage. * fix(agents): restore generated agent selection when switching back to Auto Review follow-up (#723): handleModeChange('auto') left the preset ids selected, so the UI said Auto while discussion playback kept using preset agents, and the persisted user-set auto pair could never validate on restore. Switching to Auto now selects the loaded classroom's generated agents (empty when none are loaded, falling back to stage defaults on the next classroom load). |
||
|
RongHuiQiang
|
50c0aefd13 | fix(vocational): gate procedural content generation | ||
|
RongHuiQiang
|
170b7df98f | merge upstream/main into vocational task engine branch | ||
|
RongHuiQiang
|
e6debb093b | fix(vocational): address review feedback | ||
|
|
d7c0d80afe |
feat(tts): per-agent auto-voice quality + register-once timbre stability (#670) (#672)
* feat(tts): voxcpm voice-design types + deterministic voice id helpers (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): emit + persist per-agent voiceDesign descriptor (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): voxcpm voice registration backend client (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): voxcpm-voice ensure/register endpoint (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): client auto-voice registration + reference-clip cache (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): reference registered voice id in vLLM-Omni speech (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): thread voiceDesign + backend through tts call sites (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): match vLLM-Omni voice registration contract (#670) Live e2e against the real backend revealed the assumed multipart contract was wrong: POST /v1/audio/voices needs name + consent + audio_sample (not voice_id/file), and there is no per-name GET (405) — existence must list /v1/audio/voices and check membership. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(tts): make auto-voice register-once pattern provider-neutral (#670) The descriptor + register-once/reference-by-id pattern is not VoxCPM-specific. De-couple it into a provider-neutral seam so other registration-capable providers (ElevenLabs/MiniMax/Doubao voice cloning, …) can plug in: - lib/audio/voice-design.ts: VoiceDesign + buildVoiceDesignPrompt / normalizeVoiceDesign / getDeterministicVoiceId (neutral 'auto-<hash>' id, namespaced by providerId). - lib/audio/voice-registration.ts: VoiceRegistrationAdapter interface + providerId->adapter registry + supportsVoiceRegistration. - lib/audio/voice-registration-client.ts: neutral ensureRegisteredVoice() + IndexedDB clip cache. - app/api/generate/voice (replaces .../voxcpm-voice): dispatches by providerId. - voxcpm-registration.ts becomes the VoxCPM adapter (sole registered provider). - AgentConfig.voiceDesign + DB cache table renamed neutral. Behavior-preserving; VoxCPM-specific bits (inline (prompt)text, backend kinds, capability gate) stay in the voxcpm modules. Full suite green (674). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): build the voice clip Blob from an inline Uint8Array (#670) A factored-out helper returning Uint8Array widened to Uint8Array<ArrayBufferLike>, which next build (stricter than bare tsc) rejects as a BlobPart. Inline the buffer like the rest of the repo does. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): managed TTS providers resolve model server-side (#670) A server-managed TTS provider's model was still client-driven, but the managed-provider settings UI hides the model field — so a backend whose model id isn't the client default (e.g. VoxCPM/vLLM-Omni serving the full model path) always 500'd. resolveTTSModel() makes the model authoritative from server config (${PREFIX}_MODELS, first entry) for managed providers, like key/baseUrl; unmanaged/unconfigured providers keep the client model unchanged. Used by the tts and voice routes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): single agent-voice resolver + stable teacher narration voice (#670) Route all TTS voice resolution (narration, discussion, preview) through one resolveAgentVoiceOptions(agent, ...) that reads the agent profile and registers + references the voice by id. Eagerly warm up generated agents' voices on save. Critically fixes the teacher narration drifting (male/female jumps): the registry is always seeded with DEFAULT_AGENTS, so the old narration lookup find(role==='teacher') returned the default teacher (no voiceDesign) instead of the generated one — so narration never registered a voice and fell back to the inline prompt. pickNarratorAgent() now prefers the teacher carrying a voiceDesign. Also: drop language from the deterministic voice id (descriptor already encodes it) so narration (directive) and discussion (locale) resolve the same id; log the effective registeredVoiceId. Regression tests for pickNarratorAgent. Verified e2e: agent-profiles -> 1 voice registration -> 16 narration TTS all referencing the same registeredVoiceId; voice present on the backend. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): fall back to persona as the voice seed for agents without a voiceDesign (#670) Preset/default agents carry no LLM voiceDesign. Rather than add a static field, derive the bootstrap descriptor from the agent's persona when voiceDesign is absent, so they still register one stable reference voice (stable-but-generic; persona is not a vocal spec). Generated agents keep their LLM voiceDesign. Also hardens replay: stage snapshots drop voiceDesign but keep persona. Unit-tested: resolveAgentVoiceOptions uses real voiceDesign when present, persona otherwise. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): match Auto Voice by its localized label in the agent voice picker (#670) The picker filtered on voice.name ('Auto Voice'), but Auto Voice is shown via its localized label (自动音色). Searching '自动' returned '没有匹配音色'. Match the localized label for the Auto Voice option so it's findable in any language. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): address code-review round 1 (#670) - strip parentheses in the voice-design prompt so a paren in the descriptor/ persona can't corrupt the (prompt)text bootstrap delimiter - gate POST /api/generate/voice on isServerTTSProviderDisabled (#665): a force-disabled provider was off for the tts route but not this sibling - check voiceExists before re-registering a client-cached clip, so a cached voice that's still live isn't needlessly re-uploaded every session - dedup concurrent ensureRegisteredVoice calls via an in-flight promise map (eager warm-up + first utterance no longer double bootstrap/register) - warm up only the narrator (teacher), not every generated agent, to avoid synthesizing voices at save time for agents that may never speak - collapse the duplicated vLLM-Omni speech payload into one object Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(tts): prettier formatting (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): key auto-voice memo by (voiceId, backend) so a base-URL change re-registers (#670) Addresses review (cosarah): registeredThisSession/inFlight were keyed by voiceId alone, so switching the VoxCPM base URL mid-session made ensureRegisteredVoice short-circuit and return an id registered only on the old backend — TTS then sent that stale registeredVoiceId and skipped the inline fallback, failing with voice-not-found on the new backend. Memo key now includes the base URL; the IndexedDB clip cache stays keyed by voiceId (the reference clip is backend-independent and reused to re-register elsewhere). Regression test added. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): include API key in the auto-voice memo key (#670) Addresses review (cosarah, round 2): memoKeyFor was keyed by (voiceId, baseUrl) but not the API key. Registration/existence checks and speech calls are auth-scoped, so switching account/key on the same base URL could reuse a registeredVoiceId from the old credentials and skip re-validation. Memo key now includes the API key (in-memory only, never persisted/logged). Regression test covers the key-switch case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
|
3e1e9f083b |
feat(tts): unify provider-enablement model, default browser-native off (#665) (#666)
A single TTS provider-enablement model honored everywhere (picker,
auto-assignment, fallback, lecture/playback, server).
Model
- lib/audio/provider-enablement.ts: isTTSProviderConfigured /
isTTSProviderEnabled / listEnabledTTSProviderIds. Availability drops the
registry defaultBaseUrl fallback (fixes Lemonade's always-shown voices),
honors the per-provider `enabled` flag, server-disable overrides (server
precedence).
- voice-resolver: getEnabledProvidersWithVoices (enabled server/custom
providers) and getSelectableProvidersWithVoices (the ONE client source of
truth = enabled providers + opt-in browser-native when the browser exposes
voices). resolveAgentVoice picks deterministically among the given providers,
honors a voiceConfig only while its provider is present.
- Picker + discussion build their list from getSelectableProvidersWithVoices, so
the AgentBar and the live discussion agree on selectable voices (fixes
student agents going silent in auto mode). Students resolve via
resolveAgentVoice; the teacher uses the global lecture voice verbatim (same
voice the pre-generated lecture sends), so lecture and discussion never
diverge.
- Voice assignment vs fallback: an agent's voice is bound at generation (the
agent-profile LLM picks from availableVoices; preset uses the user's pick).
The fallback for a missing/invalid/unavailable voice is applied only at the
live TTS calls, not baked into the saved agent — so it reflects current
provider availability.
Defaults / opt-in
- Browser-native: first-class, default OFF, opt-in. Built-in providers default
enabled:true (only surface once configured).
- TTS master toggle defaults OFF; first server-sync auto-enables it when a
provider is configured (mirrors image/video).
- Persist v3->v4 migration normalizes the never-user-editable flags.
Server/admin disable (server precedence): server-providers.yml `enabled:false`
and env `TTS_<P>_ENABLED=false` (incl. browser-native). Surfaced as a per-provider
`disabled` flag on /api/server-providers; merged as serverDisabled; enforced at
the /api/generate/tts trust boundary, classroom media generation, and /api/health.
UI
- Settings TTS panel: per-provider enable toggle, disabled (with a hint) when the
provider is unavailable, locked off (with a note) when server-disabled.
- AgentBar voice pills render disabled with a "No voice" placeholder when no
provider is enabled. The toolbar media-popover TTS tab keeps its existing
on/off toggle (no picker/CTA).
Tests: predicates, resolver determinism + unified selectable list, server
force-disable, store merge + migration + TTS auto-enable, i18n parity. Verified
in a real browser (Playwright): settings toggle states + a full Gemini classroom
generation whose TTS requests used the picked voice.
Found on main @
|
||
|
|
35d36905d1 |
Make server-configured providers admin-managed, not client-overridable (#620) (#624)
* refactor(providers): make server-configured providers admin-managed, not client-overridable A server-configured provider is now authoritative: the operator owns its API key and base URL, and the client cannot override either. This collapses the tri-state (server key / client base URL / client key) that was the root of a recurring "missing API key" bug class — when the settings UI echoed a base URL with an empty key, routes that treated "any base URL ⇒ fully client-configured" dropped the server key. #533 patched that per-route via canUseServerApiKeyForBaseUrl; this removes the guard entirely by fixing the design. Server: - resolve*ApiKey/BaseUrl ignore client values when the provider is server- configured; add isServerConfiguredProvider(section, id); drop canUseServerApiKeyForBaseUrl + normalizeProviderBaseUrl. - getServer*Providers() no longer return base URLs — only allowed models (LLM/image) and the managed flag (presence) — so an internal gateway/proxy endpoint is never exposed to the client. - routes (chat/image/video/tts/asr/pdf + verify-*) resolve managed providers purely server-side; SSRF validation now applies only to unmanaged (client) base URLs. Client: - drop serverBaseUrl from ProviderSettings, the settings store, request-building (TTS base URL no longer echoes the server URL) and voice resolution. - settings UI renders managed providers read-only: the key/base-URL override inputs and region toggles are hidden, leaving the "managed by server" notice. Tradeoff: overriding a server provider's key with your own for the same provider is removed; add a separate custom provider instead. Closes #620. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers): refresh managed-provider copy, drop dead audio-settings, add #620 e2e - serverConfiguredNotice now says the provider is managed and not editable here, pointing users to add a separate provider for their own credentials (7 locales). - delete components/settings/audio-settings.tsx — unreferenced dead code. - add e2e/tests/managed-provider-620.spec.ts: managed provider hides the key/base-URL inputs; unmanaged provider keeps them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(providers): lock the model catalog for managed providers with a pinned list When the operator pins an allowed model list (MODELS env/yaml ⇒ serverModels), the model catalog is admin-managed too: the LLM provider panel shows the pinned models read-only with a "Server" badge and no add / reset / edit / delete affordances. Without a pinned list the server manages only credentials and the user keeps curating models. This removes the prior inconsistency where the settings panel let you add models to a pinned managed provider that the generation toolbar then filtered out. e2e: the managed-openai case now also asserts the catalog is locked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers): ignore (not reject) client overrides for managed web-search; clear legacy serverBaseUrl Addresses review feedback on #624: 1. Managed web-search providers now ignore a client-sent key/baseUrl instead of validating and rejecting it. A server-configured provider is admin-owned, so the server config is authoritative and a stale client base URL is dropped rather than 400-ing the request. The allowlist validation still runs for unmanaged providers, where the client base URL is actually used. 2. Drop the removed `serverBaseUrl` field from persisted localStorage on rehydrate (settings version 2 -> 3, stripped in both migrate and merge), so old clients don't keep a stale server URL in client state. Tests: web-search route test split into unmanaged-rejects-allowlist and managed-ignores-client-baseUrl; settings rehydrate test asserts a legacy serverBaseUrl is stripped. Full suite 381 pass; e2e 2 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
Yizuki_Ame
|
47cc2a510c |
feat: add Brave Search and Baidu Search integration
Add Brave Search and Baidu Search providers with Baidu sub-source support, provider configuration updates, tests, and docs links. |
||
|
wyuc
|
a5209d747e |
feat(outline-review): clickable streaming card morphs into editor (#558)
* feat: add outline review preview flow * style: refine outline review editor * fix(home): make draft cache survive round-trips through generation flows The `useDraftCache` hook initialised state via `useState`'s lazy initialiser. Under React 18+ SSR, the initialiser runs on the server (returns `undefined` because `window` is missing), the SSR'd state is sent to the client, and React reuses it on hydration without re-running the initialiser. As a result `cachedValue` was always `undefined` and nothing ever rehydrated from `localStorage`. The home page's `prevCachedRequirement` derived-state pattern then masked the symptom: it initialised `prev` to the same (always undefined) `cachedRequirement`, so the comparison never fired and the restore branch was unreachable. Switch to an effect-based read so consumers actually receive a state update after mount, and replace the home page's derived-state restore with a `useEffect` + `useRef` guard that only fills the input when it is empty. Side benefit: PBL chat draft and quiz answer caches start working too — they used the same broken pattern. * feat(outline-review): clickable streaming card morphs into editor + UX overhaul Major refresh of the outline review surface introduced in #198, applying the placement design from docs/superpowers/specs. Key shifts: Streaming card → editor morph - The streaming preview card in the generation-preview page is now an interactive surface (tilt + hover-straighten + always-visible "点击审阅" affordance) that morphs into the full editor via shared `layoutId`. - SSE keeps streaming uninterrupted into the expanded editor; the editor consumes either `session.sceneOutlines` or live `streamingOutlines` so scenes appear as they arrive and stay in sync after edits. - A header collapse button shrinks back to the preview card. Mid-stream collapse keeps SSE running; post-stream collapse re-arms the same 2.5s auto-continue timer the no-review path uses, so the card doesn't strand. Document-style editor - Replaces the bordered card + accordion + numbered rail with a Notion- feeling stack: drag handle and number share a flex left rail aligned to the title baseline; title/description are inline-editable borderless textareas with auto-resize. - ⋮ overflow menu and explicit move-up/move-down are removed; reordering is drag-only, with hover-revealed delete (popover-confirmed) next to the type pill. - Hover-revealed `+` insert dividers between scenes; faint `+` at the list edges replaces the "Add scene" bar so insertion is uniform. - Quiz config popover redesigned: stepper for question count, segmented control for difficulty, multi-select pills for question types so mixed types from the LLM round-trip correctly (the previous single-select silently truncated `questionTypes` to one element). - Streaming placeholder row pulses below the latest scene during SSE. - Scene types use a colour-coded pill that doubles as the type switcher. State + flow plumbing - Adds `previewPhase: 'review'` entry from the streaming visualizer via `outlineReviewIntentRef`; SSE-completion ORs in this intent when deciding whether to wait for confirmation. - `localStorage.removeItem('requirementDraft')` moves to after the review promise resolves so "Back to requirements" actually restores the draft. - Auto-start condition broadened so refreshes during early-review reattach to a fresh SSE rather than stalling on an empty editor. - Six new locales (en-US, zh-CN, zh-TW, ja-JP, ru-RU, ar-SA) for the expand hint, streaming progress, collapse, delete confirmation, insert hint, and quiz summary; locale coverage test asserts the new keys plus the `{{count}}` interpolation contracts. Removed: the redundant "Review outline" button + auto-continue caption under the preview card, since the card itself is now the entry point. Removed: the home toolbar "Outline" tab from #198 — placement design explicitly rejects adding another always-visible toolbar control. * fix(outline-review): address code review feedback (PR #558) Single follow-up commit batching the issues raised in code review: Critical - E2E page object's reviewOutlineButton was hunting for the removed "Review outline" button. Re-target it to the streaming card's outlineExpandHint aria-label across all six locales. Important - waitForOutlineReviewChoice now accepts an AbortSignal and rejects on abort, so goBackToHome / unmount settle the parked promise instead of leaking the awaiting startGeneration closure. The existing AbortError catch swallows the rejection cleanly. - Add CLDR plural variants for the count-bearing strings: en gets _one; ru gets _one and _few for quizConfigSummary; ar gets _one and _two for the three count-bearing strings. Bare key remains as _other fallback. (zh / zh-TW / ja have a single plural form.) - Document the mid-stream collapse intent: dropping outlineReviewIntentRef is deliberate so SSE-completion takes the no-review path. - Add keyboard reorder on the drag handle (Ctrl/⌘ + ArrowUp/ArrowDown) with aria-keyshortcuts; replace dragScene copy with dragSceneHint that surfaces the shortcut. Restores WCAG 2.1.1 reachability for reorder. - KeyPointInput no longer commits on blur — only Enter / comma. Avoids surprise chips when users click away mid-typing. Minor - Drop unused i18n keys: reviewOutlineAction, moveSceneUp/Down, dragScene, sceneDescriptionLabel, keyPointsLabel/Placeholder, quizConfigLabel, addScene. Locale coverage test pruned to match actual usage. - requestAnimationFrame-batch useAutoResize so a burst of edits doesn't thrash layout (forced reflow per keystroke). - Reset isConfirmingOutlines in the no-parked-promise fallback path of handleConfirmOutlines so state doesn't linger. * chore(outline-review): prettier format + drop redundant session check - Run prettier on outlines-editor.tsx (CI Lint job was red). - Remove `&& session` from the StepVisualizer onExpandOutline conditional; `session` is guaranteed non-null at that point by the early-return guard above (flagged by github-code-quality bot). * fix(home): silence react-hooks/set-state-in-effect on draft restore Match the existing eslint-disable convention already used in app/page.tsx for the other localStorage-hydration effects. The setState calls are deliberate: client-side state hydration from localStorage must happen in an effect to remain SSR-safe (the lazy useState initialiser runs on the server with `window` undefined). * revert(i18n): drop CLDR plural variants until key-alignment check supports them scripts/check-i18n-keys.mjs enforces strict key equality across all locales, which is incompatible with i18next's per-locale plural-suffix keys (a Russian locale needs `_one`/`_few`/`_many`/`_other`, an Arabic one adds `_two`/`_zero`, while zh / ja need none). The check script doesn't strip plural suffixes before comparing, so any partial pluralization breaks alignment. Roll back the partial fix from PR review #2 and revert to the bare keys. The original "ungrammatical for n=1" wording stays, to be addressed in a follow-up that also teaches check-i18n-keys.mjs to ignore CLDR plural suffixes when comparing key sets. * fix(outline-review): make streaming card expand-hint pill actually visible The "Tap to review" pill was rendering but invisible in practice because: - Both the bottom white gradient (z-auto) and the pill (z-10) sit inside the same parent stacking context, but motion.div applies a transform that creates per-element stacking contexts; the implicit ordering put the gradient above the pill in some browsers. - text-[9px] + bg-blue-500/95 was too thin/translucent to read against the white gradient even when on top. Make stacking explicit (gradient z-0, pill z-20), bump the pill to text-[10px] font-semibold with solid bg-blue-500, larger Maximize icon, and lift it slightly to bottom-2 so it sits above the gradient haze. |
||
|
wyuc
|
2dff6d10be |
feat: add generated video manifest refs (#540)
* feat: add generated video manifest refs * style: format video manifest changes * fix: address video manifest review feedback |
||
|
|
be759a8252 |
feat: add lemonade to provider (#508)
* feat: add lemonade to provider * fix(lemonade): correct TTS/ASR model IDs and expand Kokoro voices - TTS model: kokoro → kokoro-v1 - ASR model: single 'whisper' → 6 Whisper variants (Base/Large-v3/Large-v3-Turbo/Medium/Small/Tiny), default Whisper-Base - Expand lemonade-tts voices to full Kokoro v1 set (54 voices across en-US/en-GB/zh-CN/ja-JP/es/fr-FR/hi/it-IT/pt-BR) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lemonade): zh-TW i18n parity, real Kokoro default voice, language code consistency - zh-TW.json: add missing providerLemonadeTTS/ASR/Image keys - TTS default voice: 'default' → 'af_heart' (Kokoro expects real voice ID, not placeholder) - Voice language codes: 'es'/'hi' → 'es-ES'/'hi-IN' for consistency Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(lemonade): add unit tests for image/TTS/ASR adapters and wav-utils Covers: - lib/media/adapters/lemonade-image-adapter: request shape, base URL fallback, custom model, optional Bearer auth, error paths, connectivity probe - lib/audio/tts-providers (lemonade-tts case): request body (model/voice/format), af_heart fallback, optional auth, error path - lib/audio/asr-providers (lemonade-asr case): WAV-only validation, model/language form fields, "audio is empty" graceful empty result, error propagation, default model fallback - lib/audio/wav-utils: isWavBlob detection by MIME and filename, normalizeASRUploadAudio pass-through and wav handling Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lemonade): remove text path fork and harden fallbacks * style: format lemonade tts files * i18n: add lemonade voice language labels * fix(lemonade): align image default model and harden wav detection * test(lemonade): fix File BlobPart typing * fix(lemonade): simplify tts voice selection and clear asr test state * docs: highlight Lemonade support --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyuc
|
dbe1d2ec76 |
[codex] Add Bocha web search provider (#524)
* Add Bocha web search provider * Address Bocha web search review feedback |
||
|
wyuc
|
5e12ad892c |
[codex] add VoxCPM2 TTS provider (#496)
* add voxcpm2 tts provider * address voxcpm review feedback * translate voxcpm locale strings * narrow toolbar model pill * refine toolbar media controls * address voxcpm review followups |
||
|
wyucand杨慎
|
e613b7578f |
[codex] add per-model thinking config (#494)
* add per-model thinking config * Refine thinking model controls --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
wyucandClaude Opus 4.7
|
e76d510009 |
feat(home): badge Deep-Interactive classrooms on card thumbnails (#478)
* feat(stage): add optional Stage.interactiveMode flag Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(db): add optional interactiveMode column on StageRecord Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(stage-storage): pass interactiveMode through save/list Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(generation): persist interactiveMode when creating a classroom Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * i18n: add classroom.deepInteractive across all locales Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(home): show Deep Interactive badge on classroom cards Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(home): move Deep Interactive badge into thumbnail corner Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(home): slim Deep Interactive badge to icon-only chip Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(home): use Tooltip component for Deep Interactive badge Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(home): move Deep Interactive badge to bottom-left corner Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(home): tighten Deep Interactive tooltip placement Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(home): close gap between Deep Interactive tooltip and badge Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(home): edge-align Deep Interactive tooltip without overlap Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: address code review - Reuse toolbar.interactiveModeLabel instead of adding a duplicate classroom.deepInteractive i18n key (was identical to the existing label in zh-CN and semantically the same in the other 4 locales). - Drop the v0.2.0 version tag from comments (will go stale at release). - Document on ManifestStage that omitting interactiveMode is intentional — imports can't faithfully reproduce the original generation branch. - Add a brief comment on the Tooltip's negative sideOffset explaining why the standard positive offset does not visually close the gap here. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
杨慎andClaude Opus 4.6
|
470aa5e42e |
feat: inline language inference for outline and PBL generation (#412)
* feat: inline language inference in outline generation Replace manual language selection with automatic inference from user requirement text. The outline generator now produces a languageDirective that propagates through the entire generation pipeline. Key changes: - Rewrite Language Inference section in outline prompt with decision rules for foreign language learning, cross-language PDF, proxy requests, and terminology handling - Reorder pipeline: outlines before agents, so agent profiles can use the inferred languageDirective - Pass languageDirective through scene content/actions generation - Add SSE streaming of languageDirective from outline generation - Remove manual language selection UI Eval test suite: - 50 test cases covering 9 scenario types: single-language, language learning, immersive, explicit instruction, code-switching, minimal input, user profiles, cross-language PDF, locale mismatch - LLM-as-judge evaluation with configurable inference/judge models - 50/50 pass rate with gemini-3-flash-preview + gpt-4o judge - Excluded from CI (requires LLM API keys) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove useless attributes in test case * feat: pass languageDirective to PBL, remove hardcoded zh-CN/en-US - Remove pblConfig.language field, use languageDirective from outline inference instead - Delete all Chinese prompt duplicates in pbl-system-prompt.ts, agent-templates.ts; keep English templates with languageDirective injection - Remove zh-CN/en-US branches in generate-pbl.ts (initial prompt, post-process context, welcome message) - Pipe languageDirective through scene-generator → generatePBLContent → IssueboardMCP → agent templates - Remove i18n template wrapping from PBL welcome messages in pbl-renderer.tsx and use-pbl-chat.ts; use LLM-generated content directly to avoid language mismatch when UI locale ≠ course language - Update pblConfig schema in outline prompt (remove language field) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: clean up legacy language fields, pass languageDirective through buildSceneFromOutline - Add languageDirective param to buildSceneFromOutline, combining with outline.languageNote via buildLanguageText - Remove dead types: AudienceProfile, StylePreferences, LegacyUserRequirements, SceneOutline.language - Remove Stage.language and all its propagation (storage, stageInfo, prompt-builder fallback) - Remove GenerateClassroomInput.language and job store inputSummary.language - Remove unused pdfLanguageSample from outline SSE route Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: persist languageDirective to IndexedDB, fix TTS preview and dead refs - Add languageDirective to StageRecord and stage-storage write path - Add DB v9 migration: convert legacy language locale codes to directives - Use voice.language for TTS preview text instead of dead localStorage key - Omit empty ## Language section in PBL agent prompts - Use JSON.parse for extractLanguageDirective unescape (\uXXXX support) - Remove dead i18n keys: toolbar.languageHint, pbl.chat.welcomeMessage Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.6
|
9a0060e2f0 |
feat(audio): add custom OpenAI-compatible TTS/ASR provider support (#357) (#409)
* feat(audio): extend TTSProviderId/ASRProviderId to support custom providers * feat(audio): update provider registry helpers for custom provider support * feat(audio): route custom providers to OpenAI-compatible implementations Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(audio): add custom TTS/ASR provider CRUD to settings store * feat(audio): add dialog for creating custom audio providers * feat(audio): add custom provider UI with voice management and delete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(i18n): add custom audio provider translation strings * fix(audio): resolve remaining type errors for custom provider support Add `as keyof typeof` casts where TTSProviderId/ASRProviderId (which now include template literal types) index into the built-in-only registries TTS_PROVIDERS, ASR_PROVIDERS, DEFAULT_TTS_VOICES, and DEFAULT_TTS_MODELS. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(audio): allow custom provider IDs to reach switch default branch The early guard `if (!provider) throw` blocked custom provider IDs from reaching the switch-case default branch where they get routed to OpenAI-compatible implementations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(audio): add default model field to dialog and polish voice list UI - Add "Default Model" input to custom audio provider dialog (fixes Kokoro needing model:"kokoro" instead of fallback "gpt-4o-mini-tts") - Redesign voice list as aligned table with column headers, hover reveal delete buttons, and empty state message - Add Enter key support to voice add inputs - Use consistent grid layout for add voice row Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl when testing custom TTS provider The baseUrl field in provider config is empty by default (it's an override field). Custom providers store their default URL in customDefaultBaseUrl, which needs to be used as fallback for test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): auto-select first voice when adding to custom TTS provider When a custom provider's voice is 'default' (initial state before any voices are added), adding the first voice now automatically updates the active voice selection so test TTS works immediately. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl in all TTS call sites Scene generator, discussion TTS, and agent bar voice preview all read providerConfig.baseUrl which is empty for custom providers (the default URL is stored in customDefaultBaseUrl). Add fallback chain to all 4 call sites so custom providers work during actual playback, not just in the settings test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): include custom provider voices in agent voice picker getAvailableProvidersWithVoices only iterated built-in TTS_PROVIDERS, so custom providers never appeared in the agent bar voice dropdown. Now also iterates ttsProvidersConfig for custom-tts-* entries and includes their customVoices in the picker. Also updated getServerVoiceList, findVoiceDisplayName, and resolveAgentVoice to handle custom provider voice lookups. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): fallback to customDefaultBaseUrl in all ASR call sites Same issue as TTS: audio-recorder, asr-settings, and audio-settings all read providerConfig.baseUrl which is empty for custom providers. Add customDefaultBaseUrl fallback to all 3 ASR call sites. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(audio): add model list management for custom ASR providers Custom ASR providers now support adding/removing models with a table UI (same design as TTS voice list). The model selector dropdown shows custom models and auto-selects the first one. Also adds custom ASR providers to the Toolbar's ASR provider dropdown and fixes custom provider handling in audio-settings.tsx (baseUrl fallback, request URL preview, API key section visibility). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): seed customModels from defaultModel and fix ASR model label - When creating custom ASR provider with a defaultModel, auto-add it to customModels so the model list isn't empty on first visit - Fix ASR settings label from "TTS Model" to "Default Model" Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): separate ASR model management from creation dialog ASR models should be managed in provider settings (add/remove from list, select default), not pre-filled at creation time. Unlike TTS where a single default model suffices, ASR users need to manage multiple models (e.g. whisper-small, whisper-large-v3). - Remove Default Model field from ASR creation dialog (keep for TTS) - Remove defaultModel parameter from addCustomASRProvider - ASR models are added via the model list UI in provider settings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): simplify ASR model management — first model is default Remove the separate model selector dropdown for custom ASR providers. The first model in the list is always used as default. Users manage the model list only (add/remove), no separate "default" selection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): add loading state and detailed errors to ASR test - Show spinner with "Processing..." while waiting for transcription - Disable record button during processing - Show specific error messages from API (e.g. "No transcription generated") instead of generic failure text - Handle empty transcription result as an error with helpful message Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): use correct key for processing label in ASR test button * fix(audio): polish custom provider UX and type safety - Add proper type declarations for custom provider fields (customName, customDefaultBaseUrl, customVoices, etc.) eliminating 30+ unsafe casts - Show custom ASR providers in toolbar with default language list - Hide custom ASR providers without models from toolbar selection - Add no-models warning in ASR settings panel - Add delete confirmation dialog for custom TTS/ASR providers - Add duplicate voice/model ID validation with toast error - Add dedicated i18n keys for ASR model management (modelNamePlaceholder, addModel) - Fix i18n key for ASR processing label (settings.asrProcessing) - Add radio-button model selection in custom ASR provider settings - Add visible OpenAI-compatible description in provider creation dialog - Add CUSTOM_ASR_DEFAULT_LANGUAGES constant for toolbar language options Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(test): add isCustomTTSProvider/isCustomASRProvider to audio types mock The settings-server-sync test mocked @/lib/audio/types as an empty object, but settings.ts now imports isCustomASRProvider for provider validation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(audio): add customDefaultBaseUrl fallback in all TTS/ASR call sites When a custom provider is created, the actual URL is stored in customDefaultBaseUrl while baseUrl is initialized empty. Several call sites only checked baseUrl, causing fetch failures during course generation. Add fallback to customDefaultBaseUrl in: - generation-preview/page.tsx (course generation TTS) - tts-providers.ts (getCurrentTTSConfig) - asr-providers.ts (getCurrentASRConfig) - media-popover.tsx (TTS preview) - tts-config-popover.tsx (TTS preview) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: format generation-preview/page.tsx Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused ttsSpeedRange variable in media-popover Flagged by CodeQL code quality check. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
2c26e3a461 |
feat: Ollama local LLM support (#94)
* feat: Ollama local LLM support - Ollama as a built-in provider (OpenAI-compatible, no API key needed) - 8 default models: llama3.3, llama3.2, qwen2.5, qwen2.5:32b, mistral, gemma3, deepseek-r1, phi4 - Keyless provider activation: providers with only a baseUrl (no API key) now activate correctly - isProviderKeyRequired() helper for consistent key validation across routes - i18n strings for Ollama (zh-CN + en-US) and missing Doubao strings - SSRF guard documentation clarifying server-configured URLs bypass safely Previously, clients could send `x-requires-api-key: false` (header) or `requiresApiKey: false` (body) to skip API key validation for ANY provider, including ones like OpenAI that require keys. This meant a client could consume server-configured credentials without authorization. Fix: the server now derives requiresApiKey exclusively from its own provider registry. The client header and body field are removed from all server-side parsing. For Ollama, the registry says `requiresApiKey: false` so it works keyless. For OpenAI, the registry says `true` so the check is enforced regardless of what a client sends. * fix: address reviewer feedback on Ollama integration - Fix env-var activation so OLLAMA_BASE_URL alone activates the provider - Move SSRF bypass comment from ssrf-guard.ts to resolve-model.ts - Add ollama and doubao provider names to i18n locale files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fixed issues from reviewer notes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: format provider-config.ts with Prettier Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
杨慎andClaude Opus 4.6
|
ad9e0ee7c6 |
refactor(i18n): migrate to i18next framework (#331)
* refactor(i18n): migrate to i18next framework Replace hand-rolled i18n with i18next + react-i18next so that adding a new language only requires dropping a JSON file in lib/i18n/locales/. - Add i18next, react-i18next, i18next-browser-languagedetector deps - Generate zh-CN.json / en-US.json from existing TS translation modules - Rewrite lib/i18n/index.ts as a thin wrapper around i18n.t() - Rewrite use-i18n hook to delegate to useTranslation(); external API (locale, setLocale, t) is unchanged so consumers need no changes - SSR-safe: LanguageDetector only loaded on client side Closes #327 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): use interpolation for greeting to support natural phrasing Replace string concatenation (greeting + displayName) with two i18next keys: greetingWithName (with {{name}} interpolation) and greetingDefault (standalone, no name). This lets each locale choose natural phrasing independently: - zh-CN: "嗨，同学" / "嗨，Alice" - en-US: "Hi there" / "Hi, Alice" - Future locales can avoid gender issues by choosing genderless defaults Also widen the t() type signature to accept interpolation options. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(i18n): auto-discover locale files via dynamic import Replace hardcoded zh-CN/en-US imports with i18next-resources-to-backend and dynamic import(`./locales/${language}.json`). Bundler scans the locales/ directory at build time, so adding a new language now requires only dropping a JSON file — zero changes to existing code. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(i18n): resolve hydration mismatch by deferring language detection LanguageDetector ran during i18next init(), detecting browser language before React hydrated — server rendered zh-CN while client switched to en-US immediately, causing a hydration mismatch. Fix: remove i18next-browser-languagedetector; init with a fixed lng (zh-CN) so server and client agree on the first render. Language detection is now done in I18nProvider's useEffect after hydration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(i18n): remove hardcoded locale list from language detection Replace manual locale validation and startsWith('zh') prefix matching with i18next's built-in fallback mechanism. Now changeLanguage() is called with navigator.language directly — if the exact locale has no JSON file, i18next automatically falls back to fallbackLng. Also widen Locale type from union to string so adding new languages doesn't require modifying types.ts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(i18n): add course language config with 11 languages and UI linkage - Add lib/i18n/course-languages.ts with curated language list (zh-CN, zh-TW, en-US, ja, ko, fr, de, es, pt, ru, ar) including native labels and English prompt names - Course language defaults to UI locale on first visit; once user explicitly picks a language, that choice persists across sessions - Replace toggle button with dropdown selector showing native labels - Widen language types from 'zh-CN'|'en-US' to string throughout - Fix hardcoded language ternaries in LLM prompt injection: - prompt-builder.ts: use getCourseLanguagePromptName() - classroom-generation.ts: remove normalizeLanguage() that forced all non-English to zh-CN - PBL system prompt, agent templates, generate-pbl: append language instruction for non-zh/en languages - quiz-grade API: add language suffix for grading feedback Closes #327 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(quiz): use course language instead of UI locale for grading quiz-view was passing the UI locale to the grading API, causing AI feedback to follow the student's answer language instead of the course language. Now reads stage.language from the store. Also strengthen the grading prompt: explicitly instruct the LLM to write comments in the course language regardless of the student's input language. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * revert: remove course language config and quiz fix from i18next branch Reverts |
||
|
Mohit Neerukonda
|
531b3c101f |
feat: make Tavily web search query context-aware when PDF is uploaded (#258)
* fix(web-search): rewrite search query for pdf and long requirements * fix(web-search): address query rewrite review feedback |
||
|
wyucand杨慎
|
3c687dd523 |
fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#326)
* fix(tts): unify model selection to per-provider and fix ElevenLabs model_id (#324) - Move model selection from global ttsModelId to per-provider ttsProvidersConfig[providerId].modelId - Remove supportsModelSelection flag, use models.length > 0 instead - Add defaultModelId to TTSProviderConfig and ASRProviderConfig - Add providerOptions field to TTSModelConfig for future extensibility - Fix ElevenLabs hardcoded model_id: 'eleven_multilingual_v2' - Fix MiniMax TTS using config.model instead of config.modelId - Update model lists from provider docs: - Qwen: add qwen3-tts-instruct-flash, qwen-tts - ElevenLabs: add eleven_flash_v2_5, eleven_flash_v2 - MiniMax: remove non-existent speech-2.8 series - Replace custom model CRUD UI with simple Select dropdown - Remove MiniMax-specific hardcoded model selector from audio-settings - Add migration for old ttsModelId/asrModelId/model fields * feat(tts): voice picker groups by provider:model, sets both model and voice Voice selection in agent bar now groups voices by model. Picking a voice from a model group sets both modelId and voiceId on the agent. TTS settings shows available models as display-only chips. - Add compatibleModels field to TTSVoiceInfo (OpenAI marin/cedar restricted to gpt-4o-mini-tts) - Add modelId to agent voiceConfig and ResolvedVoice - Add ModelVoiceGroup to ProviderWithVoices, build groups in getAvailableProvidersWithVoices - Update AgentVoicePill and TeacherVoicePill to iterate modelGroups - Thread modelId from voice selection through discussion TTS pipeline - Replace model Select with display-only chips in tts-settings * fix(tts): show provider name in voice picker group headers * fix(tts): restore MiniMax speech-01 models per official API docs * fix(tts): add MiniMax speech-2.8 models per official docs, default to speech-2.8-hd * feat(tts): expand MiniMax voice list with recommended voices * fix(tts): remove non-official MiniMax recommended voices, keep 10 from official list * fix: address CR findings - align defaults, fix test mocks, ASR modelId fallback * fix(tts): pass modelId in voice preview and classroom generation * fix(tts): voice picker highlights only the exact model:voice match --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
wyucand刘少杰
|
5adf91c6f3 |
feat(settings): add configurable model selection for TTS and ASR (#108)
Add ttsModelId/asrModelId to persisted settings, model management UI for providers that support model selection, and propagate selected model through preview, generation, and transcription flows. Co-authored-by: 刘少杰 <liushaojie@cvte.com> |
||
|
wyucandClaude Opus 4.6
|
f22cdc78fc |
fix: prevent auto-generated agents from bleeding into preset classrooms (#248)
Three bugs combined to cause auto-generated agents from one classroom
to appear when viewing a different preset classroom:
1. agent-bar handleModeChange('preset') only prepended the teacher but
never removed stale generated agent IDs from selectedAgentIds.
→ Filter selectedAgentIds to only keep non-generated agents on switch.
2. loadGeneratedAgentsForStage returned early when no generated agents
were found, skipping registry cleanup of previously loaded agents.
→ Always clear generated agents from registry before the early return.
3. generation-preview saved raw selectedAgentIds (including stale
generated IDs) into stage.agentIds for preset classrooms.
→ Filter out isGenerated agents before persisting to stage.agentIds.
4. classroom/[id]/page.tsx trusted stage.agentIds without filtering,
allowing pre-fix IndexedDB data with mixed IDs to take effect.
→ Filter restored IDs against the registry on load.
Closes #247
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
杨慎andClaude Opus 4.6
|
6a3bd06dec |
fix: preserve preset agent selection when switching classrooms (#241)
* fix: preserve preset agent selection when switching classrooms When switching from an auto-agent classroom to a preset classroom, the stale generated agent IDs remained in selectedAgentIds because the load path skipped setSelectedAgentIds when no generated agents were found. Add a dedicated presetSelectedAgentIds field to persist the user's preset preference separately from the runtime selection, so auto classrooms never corrupt the user's saved preset choice. Closes #239 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: format settings.ts with prettier Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: persist agent IDs in stage instead of global settings Replace the global presetSelectedAgentIds approach with per-stage agent storage. Each classroom now saves its selected agent IDs into stage.agentIds at creation time, and restores them when the classroom is loaded. This makes agent selection a per-classroom property rather than a global setting that can be corrupted by auto classrooms. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: retrigger CodeQL analysis Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Yizuki_Ame
|
519ea72eb5 |
fix: clean up stale generationSession on failure (#194)
Closes #185 When generation fails, the session was left in sessionStorage, causing stale settings (e.g. webSearch: true) to replay on page refresh. Now sessionStorage.removeItem('generationSession') is called in the catch block so failed sessions are cleaned up. |
||
|
|
ddb52241fb |
feat: add Discussion TTS with per-agent voice assignment (#211)
* feat(tts): add resolveVoice() and getServerVoiceList() utilities
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tts): add AudioIndicator equalizer bars component
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tts): add onSegmentSealed callback to StreamBuffer
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tts): add voiceOverrides field to AgentConfig and AgentTemplate
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tts): add useDiscussionTTS hook with audio queue and cleanup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): add audio state indicator to Roundtable bubble
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(tts): wire onSegmentSealed callback through chat sessions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): add per-agent voice dropdown to AgentBar
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): integrate useDiscussionTTS in Stage and pass state to Roundtable
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(tts): refine voice dropdown to pill-style selector
Replace native select styling with a compact rounded-full pill
that blends into the agent row. Remove border, use muted bg,
smaller text, and a custom chevron icon.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(tts): use shadcn Select for voice dropdown, link with TTS toggle
- Replace native <select> with shadcn Select component for consistent UI
- Hide voice dropdown when TTS is muted (ttsMuted)
- Compact pill-style trigger with rounded-full, no border, muted bg
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(tts): add voice label prefix and always show dropdown
Show "音色: Alloy" instead of plain "Alloy" in the voice pill.
Always show dropdown regardless of mute state (voice config is
independent of playback).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(tts): add volume icon hint in collapsed AgentBar
Show a small Volume2 icon in the collapsed pill when voice config
is available, hinting that voice settings are inside.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): fix voice dropdown layout and click handling
- Move voice pill below agent name (second line) to prevent
horizontal overflow in English
- Wrap Select in div with onPointerDown stopPropagation to fix
Radix click-through to parent row
- Add line-clamp-1 to descriptions for consistent row height
- Use items-start instead of items-center for better multi-line alignment
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(tts): redesign AgentBar voice layout for compactness
- Single-line layout: checkbox · avatar · name · role · voice pill
- Remove descriptions from agent rows (saves vertical space)
- Extract AgentVoicePill component to isolate Select event handling
- Smaller avatars (size-7), tighter row padding (py-1.5)
- Voice pill uses Volume2 icon + voice name (no prefix text)
- Works in both Chinese and English without overflow
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): cross-provider voice selection per agent
- Change voiceConfig from per-provider lookup to explicit
{ providerId, voiceId } per agent
- Each agent can use a different TTS provider's voice
- Voice picker dropdown groups voices by provider
- useDiscussionTTS routes TTS requests per agent's provider
- resolveAgentVoice falls back to global provider if no config
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): align role badge and voice pill across agent rows
Give role badge fixed width (w-14 text-right) so role text
and voice pills align vertically across all rows regardless
of agent name length.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): fix role badge and voice pill alignment
Wrap role badge + voice pill in a fixed-width container
(w-[9.5rem] justify-end) so both align vertically across
all agent rows regardless of name or role text length.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style(tts): align role badge and voice pill across agent rows
Add min-w-[52px] text-right to role badge so it starts at a
consistent position regardless of agent name length.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): use fixed w-[60px] for role badge alignment
* fix(tts): use fixed w-[88px] for voice pill alignment
* fix(tts): prevent click-outside from closing AgentBar when Select portal is open
* fix(tts): comprehensive voice picker rewrite
- Replace Radix Select with Popover + button list (fixes click issue)
- Fix getAvailableProvidersWithVoices to always include global provider
- Widen panel from w-80 to w-96 (prevents name truncation)
- Voice pill uses primary color instead of gray (more visible)
- Extract renderAgentRow helper to reduce duplication
- Popover shows voices grouped by provider with active state
- Add findVoiceDisplayName utility
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): align voice provider availability with toolbar logic
* fix(tts): fallback to first available provider when global provider has no voices
* refactor(tts): remove global provider fallback from voice resolution
Voice resolution now only depends on available providers (those with
API keys or server-configured). No more globalProviderId parameter.
Fallback is first available provider, then browser-native-tts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): add browser native TTS voices to agent voice picker
Load speechSynthesis.getVoices() in AgentBar and include as a
"Browser Native" provider group in the voice popover. No API key
needed - always available if browser supports it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): simplify toolbar TTS to on/off toggle, add disabled state
Toolbar:
- Replace volume slider with simple TTS on/off toggle button
- Remove ttsMuted/ttsVolume/onVolumeChange props from CanvasToolbar
- Toggle now controls ttsEnabled (not ttsMuted)
AgentBar:
- Collapsed: show VolumeX icon when TTS disabled
- Voice pills show disabled state (gray, cursor-not-allowed, no popover)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(tts): simplify Settings TTS tab to toggle + provider config
Remove voice selection, speed slider, preview/test, Azure locale
filter from Settings TTS tab. Voice is now per-agent in AgentBar.
Keep: on/off toggle, provider selector, API key + base URL config.
Add hint text pointing to AgentBar for voice configuration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(tts): simplify media popover TTS tab to toggle only
* fix(tts): add voice config hint to media popover TTS tab
* feat(tts): add per-voice preview button in voice picker
Each voice row in the popover has a small speaker icon button.
Click to preview the voice with "欢迎来到AI课堂" / "Welcome to
AI Classroom" (follows i18n). Browser native uses Web Speech API,
server TTS calls /api/generate/tts. Click again or close popover
to stop. Shows spinner while generating.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): preview text follows course language instead of UI language
* refactor(tts): redesign AgentBar expanded panel layout
- Teacher always at top with voice pill (works in both modes)
- Mode tabs moved below teacher
- Auto mode: single compact row with shuffle icon + description
- Max turns: compact inline row with smaller input
- Preset mode: only student agents listed (teacher already above)
- Remove large shuffle animation from auto mode
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor(tts): merge max turns into teacher row
* refactor(tts): separate teacher row and max turns, use stepper UI
- Teacher row: avatar + name + voice pill only
- Max turns: bottom row with MessageSquare icon + compact stepper
(minus/number/plus in a rounded pill)
- Remove Input component dependency
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): increase voice pill contrast in dark mode
* fix(tts): make max turns input editable, tighten panel padding
* fix(tts): restore shuffle animation in auto mode (compact version)
* fix(tts): adjust auto mode text spacing and add voice auto-assign hint
* fix(tts): auto-close voice popover after selecting a voice
* fix(tts): increase auto mode vertical padding for better balance
* fix(tts): push auto mode text toward bottom with flex spacer
* fix(tts): reduce auto mode bottom padding
* feat(tts): wait for TTS audio to finish before next agent turn
Add waitForDrain() to useDiscussionTTS that returns a promise
resolving when the audio queue is empty. The agent loop in
useChatSessions now awaits this after buffer drain, so the next
agent's turn doesn't start until the current agent's TTS audio
finishes playing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): keep bubble visible while TTS audio is still playing
When buffer drains (text=null) but audio indicator is still active,
don't clear liveSpeech. Clear it only when audio state goes idle.
This keeps the speech bubble visible until TTS finishes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): hold discussion bubble until TTS audio finishes
When StreamBuffer fires the done signal (onLiveSpeech null), Stage
now checks if TTS is still playing. If so, it defers clearing the
bubble state. The bubble stays visible until onAllAudioEnd fires
from the TTS hook (queue empty + nothing playing), then clears.
This prevents the jarring UX where the bubble disappears while
the agent's voice is still audible.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): fix bubble hold - guard onStopSession instead of onLiveSpeech
Root cause: bubble disappears because doSessionCleanup fires via
onStopSession when the agent loop ends naturally, NOT because of
onLiveSpeech(null, null).
Fix: when onStopSession fires and TTS is still playing, defer
doSessionCleanup to onAllAudioEnd callback. Manual stop (user
presses button) still cleans up immediately via handleStopDiscussion.
Use doSessionCleanupRef to avoid circular dependency between
discussionTTS hook and doSessionCleanup useCallback.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): guard BOTH onLiveSpeech and onStopSession for bubble hold
Two paths clear the bubble:
1. onLiveSpeech(null, null) from StreamBuffer done → clears liveSpeech
2. onStopSession → doSessionCleanup → clears all state
Both fire when agent loop ends. Path 1 fires first (tick loop),
path 2 fires after (waitUntilDrained resolves). Both must be
guarded when TTS is still playing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): hold bubble during TTS playback and respect playback speed
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(tts): LLM picks voice matching agent persona during generation
- Client sends available voices (providerId + voiceId + name) to
/api/generate/agent-profiles
- LLM prompt asks to pick a voice matching each agent's personality
- Parse "providerId::voiceId" from response, save as voiceConfig
- Fallback to index-based assignment if LLM doesn't pick
- Browser native voices hidden when server providers are available
- saveGeneratedAgents accepts and persists voiceConfig
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(tts): restore volume slider in classroom toolbar
Revert the toolbar simplification from
|
||
|
wyucandClaude Opus 4.6
|
843d33e5cc |
chore: enforce Prettier formatting and fix lint issues
- Add .prettierignore to exclude vendor packages, lock files, markdown, and YAML - Update .prettierrc: printWidth 100, singleQuote, trailingComma "all" - Run Prettier across all source files for consistent formatting - Fix unused imports (UserRequirements, setTTSProvider) - Fix eslint-disable comment placement after Prettier reformat Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
wyuc
|
7cd47f83ae |
Merge branch 'worktree-refactor-structure'
# Conflicts: # app/generation-preview/page.tsx |
||
|
wyucandClaude Opus 4.6
|
cc44f32ca8 |
Refactor: directory structure cleanup + Google Fonts localization
- Replace next/font/google with local fonts (geist + @fontsource-variable/inter) to eliminate build-time network dependency - Resolve lib/utils.ts vs lib/utils/ naming conflict (cn() → lib/utils/cn.ts) - Move React Context Providers from lib/store/ to lib/contexts/ - Consolidate hooks into lib/hooks/ (use-theme, use-i18n) - Organize loose components into subdirectories: agent-* → components/agent/ canvas-* → components/canvas/ generation-toolbar, media-popover → components/generation/ speech-button, tts-config-popover → components/audio/ proactive-card → components/chat/ 65 files changed, lint 0 errors, build 28 pages passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.6
|
b58c509c33 |
Split app/generation-preview/page.tsx (1653 lines) into 3 modules
Extract types/constants to types.ts and all 7 visualizer components to components/visualizers.tsx. Page reduced from 1653 to 918 lines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.6
|
0d20abfed9 |
Initial commit: OpenMAIC AI classroom platform
Next.js 16 App Router application with: - AI-powered interactive classroom generation from PDF/text requirements - Multi-agent discussion system (teacher, students, assistant roles) - Provider abstraction for LLM, TTS, ASR, image, video, web search - Local-first data architecture (IndexedDB/Dexie) - i18n support (zh-CN/en-US) - Structured logger, SSRF guard, lint/build passing (0 errors) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |