mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-16 05:19:50 +08:00
@openmaic/renderer@0.1.9
57
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 wyucandClaude Opus 5
|
2cbd011c1f |
feat(token-plan): add TokenDance one-key preset for every modality (#1525)
* feat(token-plan): add TokenDance one-key preset for every modality TokenDance is a model gateway: chat and images are OpenAI-compatible at /gateway/v1, and the same key authenticates vendor-protocol routes on the same host (Ark, MiniMax, Bocha). The preset reuses the existing adapters with those route prefixes as base URLs, so one key lights up LLM, image, video, TTS and web search from Settings -> Token Plan. - providers: add a built-in `tokendance` OpenAI-compatible provider (TOKENDANCE_* env prefix, logo, provider name in all locales) - token-plan: add the TokenDance preset (Seedream image, MiniMax H3 video, MiniMax speech TTS, Bocha web search) - seedream: use a base URL that already ends in a version segment verbatim, so gateway routes like `/ark/v3` do not get `/api/v3` appended - minimax-video: route H3-family models through the v2 task API (content array submit, task-envelope poll); connectivity checks for H3 probe auth on the v2 query route instead of submitting a billable task - README: add a one-key quick example and replace the Gemini-specific model recommendation with a provider-agnostic setup recommendation Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qrrq9CPwb718mpouz8Y2KL * fix(token-plan): accept preset web-search base URLs and report H3 dimensions per ratio - web-search: the client base URL allowlist also accepts the exact base URL a built-in token plan preset writes for that provider, derived from TOKEN_PLAN_PRESETS. Applying a plan whose web-search route is not an official vendor host previously stored a URL that the route rejected with 400. Any other client URL is still rejected. - minimax-video: report H3 v2 clip dimensions for 16:9, 9:16, 4:3 and 1:1 instead of assuming landscape for every non-portrait ratio. - tests: pin the allowlist for every preset, the 1:1 H3 dimensions, and clear TOKENDANCE_* in the provider-config env isolation list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qrrq9CPwb718mpouz8Y2KL --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.8
|
87c4524b4d |
fix(audio): validate redirects and pin connections on provider requests (#1514)
Audio provider requests (TTS, ASR, voice registration and voice cloning) validated a client-supplied base URL once and then issued a plain fetch with default redirect-follow and no pinned dispatcher. A base URL that resolved to a public address but answered with a redirect to an internal one was followed, and a DNS answer that changed between the guard's lookup and the connect reached an internal host — both readable in-band. - Route every lib/audio provider request through a new lib/server/audio-provider-fetch.ts that combines per-hop redirect re-validation with a pinned undici dispatcher, so the socket can only reach an address the guard validated, on every hop. - Select the public-vs-local policy server-side from isServerConfiguredProvider; a client-supplied base URL is always strict public and can never reach a private, loopback or cloud-metadata address, even with ALLOW_LOCAL_NETWORKS set. - Pin the result-audio download hop as well, keeping its host allowlist and redirect:'error'. - Add a coverage-matrix test that fails if any lib/audio module regains a raw provider fetch. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
wyucandClaude Opus 4.8
|
ee79762ef9 |
fix(access-code): expire tokens server-side and throttle verification behind a trusted proxy (#1513)
When ACCESS_CODE is set, verification tokens (timestamp.HMAC) never expired because neither verifier checked the timestamp, and POST /api/access-code/verify had no attempt throttling. - Enforce a 7-day token lifetime in a shared, Edge-safe module used by both the Node verifier and the middleware Web-Crypto verifier, and reject non-canonical signatures in both. - Rate-limit verification only when the client identity is trusted (TRUST_PROXY_HEADERS=true): a per-client sliding window (10 failures / 60s) whose attempt is reserved atomically at check time, returning 429 with Retry-After. Without a trusted proxy the app cannot attribute requests to a client, so no shared throttle is applied — a long random ACCESS_CODE is the protection, and a warning is logged when it is short. - Store bounded, copied identity keys so a large forwarding header cannot retain memory. - Document the behavior in .env.example, README, and configuration docs. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Zhenghan Songandwyuc
|
1da720377c |
test(security): isolate image URL guard environment (#1476)
Why: - Keep the image URL-guard regression deterministic when contributors export generic OpenAI credentials. - Prevent this test file from deleting inherited environment values for later tests in the same worker. What: - Stub image-provider and generic OpenAI fallback variables as absent with restorable Vitest environment stubs. - Restore environment stubs after each case. - Assert the allowed-local-network case reaches generation with the client API key. Risk: - Test-only change; production provider selection and SSRF behavior are unchanged. Tests: - 120 focused server security and provider tests with OPENAI_API_KEY=openmaic-test-sentinel - pnpm test with OPENAI_API_KEY=openmaic-test-sentinel (7894 passed, 81 skipped) - pnpm check; pnpm lint; tsc --noEmit; pnpm check:i18n-keys - pnpm build - package version, internal dependency range, and Node engine checks Live Docs: - Not applicable; test isolation defect tracked in #1475. Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyuc
|
68aed2932a |
fix(proxy-media): connect only to addresses the SSRF guard validated (#1488)
* fix(proxy-media): connect only to addresses the SSRF guard validated The route validated a URL with validateUrlForSSRF and then handed the hostname to fetch(), which resolved it again at connect time. An attacker-controlled name could answer a public address to the guard and an internal address (loopback, RFC1918, link-local, cloud metadata) to the socket, and every redirect hop repeated the same validate-then-refetch gap. Add a shared policy-aware pinned dispatcher (lib/server/pinned-dispatcher.ts) whose connect.lookup resolves all addresses, runs the same local-network policy as validateUrlForSSRF over every answer (cloud metadata is always blocked, even with ALLOW_LOCAL_NETWORKS=true), and passes only the vetted list to the connection. proxy-media creates one dispatcher per request and uses it on every hop, mapping a connect-time refusal to 403 INVALID_URL instead of a generic 500. The agent-runtime pinned agent now reuses the same builder with its strict assertSafeIp policy unchanged. The other routes that validate with validateUrlForSSRF and then hand the URL to provider SDKs (parse-pdf, transcription, generate/*, verify-*, probe-models, azure-voices) are not covered; that is a follow-up. Tests: new tests/server/proxy-media-rebinding.test.ts covers the two-answer rebind, the ALLOW_LOCAL_NETWORKS opt-in through the pinned path, metadata refusal under the opt-in, and a redirect-hop rebind. * fix(proxy-media): destroy the per-request dispatcher and align non-unicast policy across layers Follow-up hardening for the proxy-media SSRF pin. 1. Handler hang: the route awaited `dispatcher.close()` in `finally`. undici's `close()` drains in-flight requests, and the response body is intentionally left unread on the 30x (`redirect: 'manual'`), non-2xx, oversize and connect-refusal paths, so an upstream that keeps its body open never completes and the POST never settles. Tear the pool down with a non-awaited `destroy()` instead; the response blob is already in memory by the time `finally` runs. Regression tests drive a loopback server that answers 404, a 30x, and an oversize 200 while trickling its body forever and assert the route settles in well under a second. 2. Headline rebinding test sensitivity: install an attacker-controlled global dispatcher (`setGlobalDispatcher` plus a lookup that always answers loopback) in `beforeEach` so the unpinned fetch path really reaches the loopback server. Removing the route's dispatcher now fails on `internal.requests() === 0` rather than on an unrelated DNS resolution error. The positive control also asserts the pinned connect lookup was invoked, so a runtime that silently ignores `init.dispatcher` goes red. 3. Non-unicast policy parity: `validateUrlForSSRF` only classified IP-literal URLs through `isPrivateIP`, so a literal in CGNAT 100.64/10 or an IANA reserved/special-use range (240/4, 198.18/15, the TEST-NET blocks, 192.0.0.0/24, multicast, broadcast) passed the URL layer while the same address reached through a hostname was refused at connect time. Both layers now share `isNeverAllowedRange`, applied to IP-literal URLs and to every resolved answer, with or without `ALLOW_LOCAL_NETWORKS`. `connectionAddressBlockReason` drops its blunt `range() !== 'unicast'` test so the two layers agree exactly, while private, loopback and link-local answers keep following the opt-in. Behaviour change: a hostname resolving into CGNAT 100.64/10 or an IANA reserved/special-use range is now refused (403 INVALID_URL) where it was previously proxied, including under `ALLOW_LOCAL_NETWORKS=true`, since these ranges are never legitimate proxy targets. 4. Do not echo an unparseable connect-time answer in the refusal. Return the generic block message instead of `Unable to classify network address: <value>`; the route relays that message to the client in a 403 body. Tests: new trickle-body regression tests and CGNAT/reserved cases for both literals and resolved answers, plus a `connectionAddressBlockReason` parity suite. One existing fixture is updated from the IANA documentation prefix 2001:db8::/32 (reserved) to a real public prefix 2001:4860::/32. * fix(ssrf-guard): let the local-network opt-in cover CGNAT and decode local-use NAT64 prefixes |
||
|
wyuc
|
1fd4348f21 |
fix(classroom): generate classroom ids server-side and create files exclusively (#1489)
* fix(classroom): generate classroom ids server-side and create files exclusively
The legacy file-backed classroom store accepted a caller-chosen stage.id and
renamed a temp file over the target, so anyone who knew a public share URL
could POST the same id and replace that classroom's content. POST
/api/classroom now mints the id itself (nanoid, 10 characters, matching the
generation pipeline), ignores any client-supplied stage.id, and rebinds the
stage and every scene to the generated id. Both the route and
classroom-generation persist through a new exclusive create that hard-links a
temp file into place, so an existing classroom is never replaced; EEXIST is
retried with a fresh id a bounded number of times and then reported as 409.
Unchanged: the server-backed persistence path (app/api/persistence,
lib/persistence), the read/GET path, middleware, and the access-code gate;
writeJsonFileAtomic keeps its overwrite semantics for the other callers.
Adds regression tests pinning non-overwrite, the typed EEXIST error, collision
retry/exhaustion, and generation-path exclusivity.
* fix(classroom): fall back to exclusive open without hard links and reserve ids before media generation
writeJsonFileExclusive kept `fs.link` as its only route to the destination, so
classroom creation failed hard on mounts without hard links (gcsfuse, s3fs and
some FUSE gateways return ENOSYS/ENOTSUP/EOPNOTSUPP/EPERM/EXDEV). Keep `link`
as the fast path and, for exactly those codes, fall back to an exclusive `wx`
open: still never replaces an existing file and still maps EEXIST to
ClassroomAlreadyExistsError. Any other code keeps propagating, and the temp
file is always removed.
The generation pipeline wrote media and TTS into
<CLASSROOMS_DIR>/<id>/{media,audio} before the exclusive create, so a collision
would have landed the new classroom's files in an existing classroom's
directory and the retried document's URLs would still point at that other id.
Reserve the id first by exclusively creating the classroom file with a
placeholder document (`reserved: true`, empty scenes). The file is the only
token that atomically covers the whole collision namespace: a classroom created
through POST /api/classroom has a JSON file but no directory, so reserving a
directory would miss it. readClassroom treats a reserved document as absent, so
an in-flight or crashed reservation is never served as an empty classroom. The
retry on EEXIST now happens at reservation time (bounded, before any media),
and the final persist is an ordinary overwrite of the id the process owns, so
the post-media id retry is gone.
Document OPENMAIC_CLASSROOMS_DIR in .env.example, including that
CLASSROOM_JOBS_DIR does not move with it.
* fix(classroom): release an unused reservation when generation fails
|
||
|
Yizuki_Ameandwyuc
|
c460bceb60 |
feat(export): preflight render queue availability (#1455)
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyucandClaude Fable 5.1
|
29735f10d0 |
fix(ssrf): keep cloud metadata endpoints blocked under ALLOW_LOCAL_NETWORKS (#1419)
* fix(ssrf): keep cloud metadata endpoints blocked under ALLOW_LOCAL_NETWORKS ALLOW_LOCAL_NETWORKS=true exists so self-hosted deployments can point providers at loopback, RFC1918 and split-horizon targets. It also returned early from validateUrlForSSRF before any classification, so a client-supplied base URL of 169.254.169.254 (or metadata.google.internal, 100.100.100.200, fd00:ec2::254) was accepted on such deployments. Cloud instance-metadata endpoints are now rejected regardless of the flag: literal hosts and IPv4-mapped forms are classified directly, and non-IP hostnames are resolved so an answer that lands on a metadata address is rejected too. DNS failure under the flag still fails open, as before, because split-horizon DNS is an explicit use case of the flag. Without the flag the DNS path now also rejects answers on metadata addresses that are not RFC1918 (100.100.100.200). The route tests that used the metadata address as their example of a target the flag allows now use a private-network address instead. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ * fix(ssrf): widen the metadata set and bound the flagged DNS lookup Review follow-up. Add the AWS ECS task credential and EKS Pod Identity endpoints (169.254.170.2, 169.254.170.23, fd00:ec2::23), the Azure WireServer address (168.63.129.16) and the legacy OCI IMDS address (192.0.0.192) to the blocked set; the last two are neither RFC1918 nor link-local, so they were reachable even without the flag. Recognise the metadata addresses when carried inside 6to4, Teredo, ISATAP and NAT64 literals. Bound the DNS lookup done under the flag to three seconds and fail open on expiry, matching the existing fail-open on error. Say in .env.example that the set is a fixed list and that DNS failure is allowed through. Use the same private-network fixture for the reject and allow cases of the four route guard tests so a partial NODE_ENV re-gate of the guard goes red again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ * fix(ssrf): share the tunnel decoder between the private and metadata checks Review follow-up. assertSafeIp now classifies metadata addresses through isCloudMetadataAddress, so an ISATAP identifier under a globally routable prefix that carries 168.63.129.16, 192.0.0.192 or 100.100.100.200 is rejected on the strict-fetch path too. isPrivateIP uses the same tunnelEmbeddedIPv4 helper instead of its own copies of the 6to4, Teredo and ISATAP decoders, which also gives it NAT64. Tests cover the globally-routable ISATAP forms, the NAT64 private case, and the false-positive direction (tunnel literals carrying public or RFC1918 addresses stay allowed under the flag). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ * fix(ssrf): reject metadata hostnames before the flag branch Review follow-up. The metadata hostname and address check now runs before the ALLOW_LOCAL_NETWORKS branch, so metadata.google.internal is rejected by name in both flag states (previously only under the flag), and metadata literals get the metadata message rather than the one that suggests setting the flag. Pin the tunnel decoder boundaries in tests (2001:db8 is not Teredo, 64:ff9b:1 is not NAT64, 2003 is not 6to4). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
wyucandwyuc
|
0bab621e09 |
harden input validation and outbound URL handling (#1387)
* harden input validation and outbound URL handling - Validate the classroom id on the write path with the same allowlist the read path already applies, and assert in the storage layer that a resolved classroom file stays inside CLASSROOMS_DIR. - Apply validateUrlForSSRF at every call site regardless of NODE_ENV; the guard already owns the ALLOW_LOCAL_NETWORKS escape hatch, so the extra environment condition only disabled the check outside production builds. Add a repository-scanning test so a gated call site cannot reappear. - Re-validate every redirect hop of an outbound provider request through a shared transport, mirroring the per-hop loop proxy-media already uses. - Restrict stored slide HTML to the formatting vocabulary the renderer produces, sanitizing at the classroom persistence boundary on both write and read, with a separate policy for KaTeX snapshots. - Refuse the development persistence authenticator under NODE_ENV=production unless an explicit opt-in is set, and document it in .env.example. * drop credential headers on a cross-origin redirect hop The manual redirect loop reused the caller's init on every hop, so provider credentials were re-sent to the redirect target even on a different origin. The platform fetch drops Authorization itself when it follows redirects, so mirror that: strip authorization, api-key, x-api-key and x-goog-api-key before a cross-origin hop, matching case-insensitively and preserving the shape of the caller's headers. Same-origin hops are untouched. A streaming request body cannot be replayed onto the next hop, so fail with a clear message rather than sending an empty body. --------- Co-authored-by: wyuc <dhq1024@proton.me> |
||
|
Yizuki_Ame
|
eb2cbc701a |
feat(pro): generate concise conversation titles (#1275)
* feat(storage): add automatic conversation title lifecycle * feat(agent): generate concise conversation titles * feat(agent): schedule automatic conversation titles * fix(agent): harden automatic title lifecycle * fix(agent): refine conversation title prompt * fix(agent): harden automatic title boundaries |
||
|
杨慎andClaude Code
|
3a0612e962 |
fix(server): bound skill zip inflation instead of trusting declared sizes (#1374)
parseUserSkillZip guarded the SKILL.md size against the zip central
directory's self-declared uncompressedSize and then handed the entry to
JSZip's async('string'), which inflates the real deflate stream no
matter what the header claims. A malicious zip declaring a tiny size
slipped past the guard, and with the upload route's 1 MiB archive cap
and deflate's ~1032:1 ratio, one request could expand ~1 GiB in the
server process — repeatable per request for memory-exhaustion DoS.
Inflate the entry's compressed bytes with node:zlib
inflateRawSync({ maxOutputLength: 70_000 }) so zlib aborts the moment
output crosses the cap, whatever the headers say. The declared-size
check stays as a fast path that refuses honest oversized zips without
inflating; STORE entries are byte-length checked directly since their
"compressed" content is the content. Over-limit output raises the same
UserSkillUploadError('The archive SKILL.md is too large.'), so callers,
route responses, and status codes are unchanged.
Tests: a forged-header bomb (a real zip whose central directory is
patched to declare a small uncompressedSize, with the forgery asserted
to have taken on reload) is refused; an honest oversized zip is refused
by the fast path; a forged STORE entry is refused by the byte-length
check; a STORE zip round-trips; a corrupt deflate stream still maps to
invalid-zip rather than a size refusal.
Co-authored-by: Claude Code <noreply@anthropic.com>
|
||
|
|
dd74cfe23b |
feat(web-search): add Exa provider (#1342)
Co-authored-by: white638 <212083322+white638@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
宙见·星野andwyuc
|
9b7b42c4c4 |
fix: normalize GPT Image 2 generation sizes (#1346)
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
mehmet turacandwyuc
|
a7e9dc8337 |
fix(providers): apply server-pinned models for image and video providers (#1298)
* fix(providers): apply server-pinned models for image and video providers The server-providers API already exposes image models, and the server resolves them correctly, but getServerVideoProviders() omitted the models field entirely, and fetchServerProviders() on the client side discarded the models payload for both image and video — it only set isServerConfigured and serverDisabled. This caused the client to fall back to built-in model IDs, which fail on endpoints that require different IDs (e.g. Volcengine Ark Agent Plan uses dotted aliases like doubao-seedream-5.0-lite). Server: getServerVideoProviders() now exposes models, mirroring the image listing. Client: fetchServerProviders() stores the server model list as customModels with replaceBuiltInModels: true for both image and video, matching the existing LLM provider model-filtering pattern. Fixes #1251 * style(tests): wrap long assertion to satisfy prettier printWidth --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
Zhenghan Songandwyuc
|
3d0d74065e |
test(media): isolate force-off test from OpenAI fallback (#1303)
Why: - A shell-exported OPENAI_API_KEY enables the generic image fallback inside a test that expects every configured media provider to be disabled. - The resulting fetch-mock failure includes the Authorization header in local test output. What: - Mask the generic OpenAI key only for the force-disabled media test before provider configuration is imported. - Reuse the existing Vitest environment cleanup to restore the caller's shell environment after the test. Risk: - Test-only change; production provider fallback behavior and the global shell-environment policy remain unchanged. - Dedicated provider-config coverage continues to exercise the generic OpenAI image fallback. Tests: - Focused force-disabled test with OPENAI_API_KEY=openmaic-test-sentinel - Full classroom-media-generation test with and without the sentinel: 8 passed - tests/server/provider-config.test.ts: 107 passed - Full root suite with the sentinel: 7137 passed, 81 skipped - Prettier, ESLint, TypeScript, and i18n quality gates Live Docs: - GitHub issue #1302; test-environment policy remains documented by PR #1162. Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyuc
|
dfebbcf33f |
perf(classroom): speed up classroom loading (media hydration, sidebar thumbnails, media range requests) (#1276)
* perf(classroom): defer non-priority media blob hydration off the load path Entering a classroom awaited the full mediaFiles restore — one object URL per restored image/video blob — before the loading gate opened. For video-heavy courses that is hundreds of MB of IndexedDB materialization blocking first paint. Split the restore into two phases: - The awaited phase now builds metadata-complete task entries but only creates object URLs for failed rows (none needed) and for media referenced by the scene the classroom opens on (persisted cursor, else the first scene). buildRestoredMediaTasks gains an optional shouldHydrateBlob predicate, defaulting to eager hydration so existing callers are unchanged. - Remaining blob-backed records hydrate in the background, chunked over requestIdleCallback (setTimeout fallback). Each deferred task stays 'done' — so generation resume never re-runs it — but carries no objectUrl yet, which the media resolution state machine already renders as a pending skeleton until the URL lands. Background hydration is guarded per record: a task that was replaced (classroom switch, regeneration, retry) or that belongs to another stage is skipped and its freshly minted URLs are revoked immediately. * perf(classroom): lazy-render slide thumbnails in the playback sidebar The playback scene sidebar mounted a full SlideCanvas for every slide scene the moment the classroom opened (and every off-screen video element opened a preload="metadata" fetch), because SlideThumbnail's existing visible prop was never passed. Extract the editor nav rail's near-viewport IntersectionObserver hook to lib/hooks/use-near-viewport.ts and gate the sidebar's slide thumbnails through it: only scenes within 200px of the viewport render the live canvas; the rest show SlideThumbnail's existing placeholder until scrolled near. The placeholder keeps the same box size, so gating never shifts layout. The shared hook now starts hidden instead of eager: with the previous eager-initial state, opening a long deck still mounted every canvas for a frame before the observer could flip off-screen items off. The observer's guaranteed initial delivery flips near-viewport items within a frame; the no-IntersectionObserver fallback (e.g. jsdom) defers to a microtask so the effect never synchronously re-renders. * perf(media): lazy image decoding and HTTP Range support for classroom media Renderer: BaseImageElement's <img> now carries loading="lazy" and decoding="async", so thumbnail-heavy surfaces (playback sidebar, editor nav rail, course cards) no longer fetch and decode every slide image up front. In-viewport images are unaffected — the browser fetches them immediately. slideToPng forces eager loading inside its permanently off-screen snapshot tree, where lazy images would otherwise never fetch and exports would capture blank slides. Server: GET /api/classroom-media/[classroomId]/[...path] now answers single byte-range requests with 206 Partial Content (Content-Range, Accept-Ranges, correct Content-Length), enabling progressive playback and seeking for hosted video/audio instead of downloading whole files. Suffix ranges are supported; unsatisfiable ranges get 416 with the full size; unsupported units or multi-range sets fall back to the plain 200 full-body response, which is always a legal answer. Existing caching headers are kept on every response shape. * chore(renderer): bump version to 0.1.4 for image lazy-loading change * fix(classroom): guard deferred hydration against restarted tasks and bound idle waits A deferred record is only ever 'done' without an objectUrl; a task that regeneration or retry restarted passes through pending/generating, so skip those instead of attaching stale persisted bytes. Also give the idle scheduling a timeout so a busy main thread cannot starve hydration. * fix(classroom): stop superseded deferred hydration before minting URLs A restore epoch captured at apply time now gates each idle chunk: loading another classroom or reloading the same one invalidates any older hydration loop, so it neither keeps scheduling work for an abandoned classroom nor attaches bytes read by an older load to a newer load's tasks. * fix(classroom): keep 416s uncached and classify element-keyed media as priority A 416 with public immutable caching can poison the media URL for later valid requests, so range errors now send Cache-Control: no-store. Priority classification also collects the opening scene's media element ids, since task lookup binds records keyed stage:<elementId> even when the slide slot carries a different opaque ref. * fix(classroom): tie deferred hydration liveness to the classroom load token The restore epoch only advanced when the next load reached apply, so an abandoned classroom kept hydrating during the next load's storage/network phase. Compose the epoch with the load's isCurrent (load token plus effect cleanup) so navigation stops the loop at the next idle boundary. * fix(classroom): classify legacy-recovered media before deferring it Legacy singleton video recovery assigns a placeholderRef only at the end of the task build, so a record keyed by an allocated id without placeholderRef was deferred even when it backs the opening scene's gen_vid_* element. Run a metadata-only build first to learn each record's effective ref and classify against it, keeping the first visible page's legacy video eager. * fix(action): wait for deferred video bytes before starting play_video executePlayVideo treated status done as immediately playable, but a deferred restore is done without an objectUrl: the renderer shows a skeleton, no <video> exists, and the later hydration never retriggers play, leaving the action stuck until the safety timeout. Readiness now follows the renderer's contract (only done-with-bytes is playable) at the initial check, the subscription exit, and the post-subscription recheck; the failed skip applies whether or not a wait happened. * fix(classroom): include the stage whiteboard in priority media refs The stage-level whiteboard stays open across standalone classroom switches, so its media can be visible before any scene is. Classifying it as deferred left visible whiteboard media pending behind idle hydration chunks. |
||
|
wyucandClaude Fable 5
|
04621578de |
release: OpenMAIC 1.0.0 — the agent workbench (#1228)
* feat(storage): add an agent-session store with PG backend and layered contracts (#1163) * feat(storage): add agent-session store with PG backend and layered contracts * test(storage): avoid BigInt literals for pre-ES2020 root typecheck * fix(storage): close agent-session store review findings * docs(storage): align hook ordering and contention-probe claims with the code * ci: run on the agent-workbench integration branch * chore(storage): bump to 0.5.0 for the agent-session store * fix(storage): carry replay compaction across page boundaries * feat(agent): add the driver model contract and stage route dialect (#1165) * feat(agent): add the driver model contract and stage route dialect * fix(agent): validate route context windows and clarify dialect precedence * feat(agent): adapt the agent-session store and runtime foundations (#1167) * feat(agent): adapt the agent-session store and runtime foundations * feat(agent): resolve request owner identity via an anonymous cookie * docs(agent): document the opt-in compaction default and harden edge cases * feat(agent): add the background session runner (#1169) * feat(agent): add the background session runner * feat(agent): wire the runner into startup behind feature flags * fix(agent): stop clean interruptions from consuming the attempt budget * fix(storage): charge the attempt budget for abandoned leases but not clean parks * docs(storage): document the attempt-charging contract and decouple its tests * feat(agent): add agent session and owner event streams (#1170) * feat(agent): add agent session and owner event streams * fix(agent): close the session-existence oracle and document the owner seam * feat(agent): add agent session lifecycle routes (#1171) * feat(agent): add agent session lifecycle routes * fix(agent): validate session-create input and preserve the owner cookie on errors * refactor(storage): drop the unused active-stage API from the agent-session contract (#1174) * refactor(storage): drop the unused active-stage API from the agent-session contract Tools address stages explicitly on every call, so the store keeps no mutable session-level stage pointer. Removes resolveActiveStage and setActiveStage from the store interface, their PG implementations, the active_stage_changed lifecycle event, the session_active_stage owner event variant, and the contract tests pinning them. The active_stage_id column and the DDL check constraint stay untouched for schema compatibility. * chore(storage): bump @openmaic/storage to 0.7.0 for the contract removal * docs: document the agent runtime configuration surface (#1176) * fix(agent): repair orphaned and late tool results across interruption boundaries (#1180) * fix(agent): repair orphaned and late tool results across interruption boundaries A crash, shutdown, or provider failure can leave the durable transcript with tool calls that have no result, or with results ordered illegally for the provider. Three failure modes were fixed: - Orphaned tool calls: a run that died between an assistant tool-call frame and its result left a dangling call in the entry tree. Resume no longer synthesizes and persists receipts for it: interrupted results are a read-time provider view owned by a shared read-boundary repair, which returns the original array for a healthy transcript and never mutates the tree. - Late parallel results: a parallel tool can finish while pi unwinds an aborted assistant frame, leaving result(A), assistant(aborted), result(B) in durable order. Strict providers reject non-contiguous results, so the read-boundary repair moves existing results next to their owning assistant frame (in call order), omits incomplete unwind frames, and synthesizes receipts only for genuinely missing calls. - Interrupted calls at the write boundary: a call still in flight when the run winds down (shutdown, lease loss, cancellation, provider failure) had no receipt at all. The runner now tracks in-flight calls from their assistant frames and, before the terminal flush, appends an interrupted-result receipt for each still-orphaned call through the same attempt-fenced write chain, so a lease-stealing zombie never writes and the next claim sees a provider-safe transcript. * test(agent): pin the runner wiring for interruption-boundary tool repair * feat(agent): add neutral tool foundation libraries (#1184) * feat(agent): register a web_search tool on the session runner (#1185) * feat(storage): add a per-session URL trust gate (#1186) * feat(agent): add the skills system (#1189) * feat(agent): add the skills system (builtin directories and durable user skills) * fix(storage): serialize the user-skill quota check-and-insert per owner Two concurrent creates at the 50-skill boundary both counted 49 rows and both inserted (READ COMMITTED, no lock), overshooting the quota contract. The create transaction now takes a per-owner pg_advisory_xact_lock first, and the same-name idempotency check runs before the count check so an at-least-once retry of the create that committed as the owner's 50th row still returns its durable receipt instead of a quota error. The 23505 backstop is retained for writes that do not take the lock. * fix(agent): share unstorable-character validation and align skill lookup * feat(agent): add session materials and a fetch_url tool behind the URL trust gate (#1190) * feat(agent): add session materials and a fetch_url tool behind the URL trust gate * fix(agent): harden session material fetching * feat(storage): add an ownership scope to stage documents (#1191) * feat(agent): add material read and search tools (#1192) * feat(agent): add stage read and patch tools (#1194) * feat(agent): add page generation and deck editing tools (#1198) * test(storage): keep the PG contract suite order-independent (#1200) * fix(agent): revoke deleted-session URL authority and reject private ISATAP endpoints (#1199) * fix(storage): revoke deleted session URL authority * fix(ssrf): reject private ISATAP endpoints in strict fetches * chore(storage): bump to 0.11.1 for the session-URL authority fix * feat(agent): add roster and voice registration tools (#1201) * feat(agent): add folder organisation tools (#1202) * feat(api): add stage and material HTTP routes (#1203) * feat(workbench): add the client data layer (#1204) * feat(workbench): add the client data layer * docs(workbench): write the ported comments in English * chore(edit): remove the in-editor agent panel (#1210) * chore(edit): remove the in-editor agent panel * style: apply prettier formatting * fix(agent): report the runtime as unusable without a database (#1207) * fix(agent): report the runtime as unusable without a database * style: apply prettier formatting * feat(agent): add image, video and pptx import tools (#1211) * feat(workbench): add the agent chat surface (#1205) * feat(workbench): add the agent chat surface * docs(workbench): write the ported comments in English * fix(workbench): label the folder and rename tools on the timeline * fix(workbench): label the roster and voice tools on the timeline The reconciliation test iterates every tool the runner registers and requires a display label of its own. The roster and voice-clone tools (list_voices, set_roster, clip_audio, register_voice) reached the integration base with the roster/voice-registration tools but never gained presentation rows, so they fell through to the default branch and rendered their wire names. Port their rows from the reference implementation (labels and i18n keys verbatim) and extend the reconciliation allowlist with ROSTER_TOOL_NAMES and VOICE_CLONE_TOOL_NAMES, so a future tool cannot enter the product without a label. * feat(agent): add the material extraction lifecycle (#1212) * feat(storage): add material extraction lifecycle * feat(agent): execute queued material extraction * style: apply prettier formatting * style: satisfy prefer-const in the extraction runner * test: give material fixtures the extraction lifecycle fields The media-tools slice and the extraction lifecycle slice were each green in isolation but never compiled together: the lifecycle made derivedFrom and extraction required on AgentSessionMaterial while the media-tool fixtures predate them. * chore: remove stray task notes * fix(workbench): label the extraction lifecycle tools on the timeline * feat(workbench): add the workspace shell (#1206) * feat(workbench): add the workspace shell * docs(workbench): write the ported comments in English * i18n(workbench): align workspace keys across locales * fix(workbench): adopt the landed data layer and label the extraction tools - replace the sibling-slice seam stubs with the real data-layer modules - drop ambient declarations now shadowed by landed files - port timeline labels for the extraction lifecycle tools from the reference - align the new i18n keys across all locales * ci: retrigger * feat(api): folder routes, stage-meta viewer surfaces, and the material upload contract (#1215) * fix(storage): restore capability-based stage access * fix(api): bind document access to request owner * fix(agent): restore three-state stage access on the tool layer Port probeStageAccess and the three-state StageAccess (owned / foreign / missing / tombstoned) and gate every stageId-bearing stage tool on an owned probe, mirroring the reference per tool: - move_to_folder, rename_stage, read_stage_outline refuse a non-owned stage with the single not-yours message before touching the store. - The course/DSL toolset and the roster toolset are wrapped by withOwnerStageAuthorization: read_stage, patch_stage, grep_stage and every writer refuse a foreign stage with the same message and refusal shape. - Scene preview keeps its own probe and its own refusal text, and is registered beside the course toolset (never double-gated). - The runner injects one probe factory at the three call sites. Tests: the dsl cross-owner test premise (a foreign stage is readable by id) encoded an invented capability-read policy that the reference does not have at the tool layer; it now asserts foreign read/patch/grep are all refused while the owner still reads. Curriculum cross-owner assertions were already the reference's and now pass with the probes in place. * docs: correct per-file test counts in the fidelity report * test: fix type errors in stage-access fidelity test * test: adapt media-tool and gate suites to the owner-scoped store seam * feat(api): add owner-scoped course-folder HTTP routes Port the reference implementation's /api/folders family (list, create, rename, delete with ungroup/remove modes, and folder membership) onto the owner-bound document store, replacing its provider-based auth with the existing withRequestOwnerId / owner-scoped store seams. The storage package's folder store grows the pieces the routes need: DocumentFolder.order (schema column + max+1 assignment + ordering), renameFolder, deleteFolder(mode) with captured member ids, and setStageFolder(stageId, folderId | null) with idempotent un-filing. FolderNameError moves into folder-name-validation.ts (stage-storage re-exports it, keeping import sites intact). Every route gates on the configured agent runtime (plain 404 when off or unconfigured), keeps the reference's machine codes and envelopes, and is covered by gate tests plus a behavior suite. * feat(api): add stage-meta viewer surfaces for the classroom Port the reference implementation's viewer-facing stage state — can-edit / collected / published / generation-complete — on top of the stage-access base (stage_meta + tombstones). stage_meta gains published_at and generation_complete columns plus a stage_bookmarks table; the reference's deployment-specific origin/claimed_at columns are stripped. New gated routes: GET /api/stage-meta/[stageId] (per-viewer facts, 404 for absent/tombstoned, never returns the owner id), GET /api/stages/[id]/status, POST generation-complete / publish / unpublish (owner-only), POST /api/bookmarks. The resolver lives in lib/server/stage-access.ts. Wiring: a fetchStageMeta client with the reference's three-outcome contract, stage-store isOwner/isBookmarked/readOnly fields (upstream single-user defaults, no-op until the sidecar answers) plus setViewerAccess, the classroom apply path computing readOnly = !(isOwner || isBookmarked), the Stage editability gate, and a sidecar probe after each classroom load. A sidecar 'absent' answer keeps the editable default here because the classroom also serves local-only courses; server writes stay owner-enforced. * feat(api): port the reference material upload contract Rewrite POST /api/materials to the reference implementation's upload shape so the workbench uploader (uploadWorkbenchMaterial, which posts no session id and expects a flat 201 view) works unchanged: owner-scoped upload with mime normalization/validation (415), per-class size caps checked on the declared content-length and the streamed body (413), empty body (400), quota (429), sha256 reserve->store->finalize lifecycle with abandon on failure, flat { materialId, originalName, bytes, mime, extraction } 201, and an x-request-id echo. Adds the owner-scoped material library (owner_material table + quota + 24h lazy sweep, bytes in the host's asset registry as the neutral replacement for the reference's object-storage byte path) and the material cap configuration. The session-scoped GET list is left as-is; the reference's owner-material extraction worker is not ported (the branch's session-material extraction lifecycle already covers extraction). Gate tests now cover all 23 persistence routes across the three runtime env states; the materials behavior suite pins the new contract. * feat(media): add an optional local ffmpeg media extractor (#1213) Adds a local ffmpeg/ffprobe pipeline as a second media extraction provider behind the extractor registry, ported faithfully from the reference implementation: duration probing, keyframe-safe chunking, per-chunk ASR with timeout and deadline budgets, and timestamped transcript assembly. - Availability probing feeds the registry's candidate selection: the provider simply is not a candidate when ffmpeg/ffprobe are absent. - With neither ffmpeg nor a cloud provider configured, extraction fails with an actionable message naming both enablement paths. - Media materials route through the same extraction lifecycle and lease fence as documents; no parallel queue. - Tests inject the executable resolver so the missing-ffmpeg path is the default-tested one; the real pipeline test is skip-if-unavailable. - @openmaic/storage 0.13.0 -> 0.14.0 (media routing in the material lifecycle surface). * feat(storage): per-scene monotonic revisions via database triggers (#1214) * feat(storage): per-scene monotonic revisions via database triggers Restore the reference implementation's freshness granularity: a per-scene monotonic revision maintained by database triggers, so every writer (HTTP routes, agent tools, jobs, manual SQL) bumps it without application cooperation. - Companion revision tables + trigger functions in the storage package's idempotent schema bootstrap, with the lock-order invariant, pg_notify wakeup and the suppression switch for batch writers. - ensureDocumentSchema gained a dollar-quote-aware statement splitter. - The freshness and manifest routes serve per-scene revisions. - Mutation-verified: dropping the triggers turns the revision tests red. - @openmaic/storage 0.13.0 -> 0.14.0. * fix: forward the freshness manifest through the owner-bound store * feat(workbench): add the Pro entry points and preserve the mode-transition semantics (#1208) * feat(workbench): add the Pro entry points * feat(workbench): preserve Pro mode transition semantics * fix(workbench): drop ambient declarations shadowed by landed slices * fix(workbench): drop ambient declarations shadowed by the landed shell * feat: port workspace shell sibling modules Port the 16 leaf modules the Pro workspace shell imports but that were only ambient-declared, replacing the compile-time bridge with real implementations adapted from the sibling-slice reference: pure workbench helpers (session title, rail tab, course-chat bootstrap, created-course tabs, course-tabs memory, workspace navigation, pane navigation, pro-edit sizing, existing-course minting, first-message session), the neutral brand context and course-rename server API, the server-action session delete, the home discovery hook, the classroom pane host with its load-policy leaf, the theme toggle and floating-layer owner, plus the floating-layer-owner wiring the dialog/dropdown/tooltip portals stamp. Also add the workbench-shell locale copy for all 12 locales, port the reference tests for the ported modules, and drop types/workbench-sibling-slices.d.ts now that every declaration has a real implementation. * docs: keep ported comments in English and deployment-neutral * docs: announce 1.0.0 and refresh the feature overview (#1216) * docs: announce 1.0.0 and refresh the feature overview * docs: finalize 1.0.0 README after feature merge * fix(agent): control-plane routes answer 404, not 500, without a database The agent control-plane routes gated only on the runtime flag, so an enabled-but-unconfigured deployment (flag on, DATABASE_URL empty) answered 500 from a store that cannot connect. Gate them on the configured check instead, matching the stage/material routes: the whole surface is cleanly absent until both the flag and the database are present. The status probe keeps reporting both bits. * test: mock both runtime gate exports in the control-plane route suites * fix(agent): abort in-flight TTS on cancel and bound each provider request with a timeout (#1217) The generate_tts / scene-tts path checked the runner's AbortSignal between actions but never created the provider HTTP requests with it, so a session cancel left a hung synthesis fetch in flight until a restart repaired the tool result. Thread the signal end-to-end: TTSModelConfig carries an optional signal, generateTTS combines it with a per-request timeout (TTS_REQUEST_TIMEOUT_MS, default 30s, ported from the reference runtime's TTS bounds) via AbortSignal.any, and every provider fetch (openai, azure, glm, qwen incl. voice-clone + audio download, voxcpm, minimax, doubao, elevenlabs, lemonade) is created with that signal. A timeout now fails the tool call with TTSRequestTimeoutError (a clear retryable error) instead of wedging the session; a caller cancel propagates as the interruption so the runner settles the session as cancelled without a restart. Tests: hung-provider simulation rejects at the timeout with the retryable error; abort mid-flight aborts the captured request signal and surfaces the interrupted shape; removing the signal wiring makes the abort tests fail (red), restoring them turns green. * fix(workbench): PG-mode home listing via owner stages; keep the interrupted terminal course card (#1218) Finding 1: with server persistence on, listStages resolved to the generic GET /api/persistence/documents listing, which the capability model deliberately answers 403 FORBIDDEN_DOCUMENTS for (reads by id, listings owner-only). The home/workspace library now lists through the owner-scoped GET /api/stages surface (same anonymous-owner cookie the workbench uses) when server persistence is enabled; the server-side 403 is untouched. Finding 2: a run interrupted (session_interrupted) and repaired (session_resumed) that ends cancelled before agent_end stranded its pending classroom sightings, so the timeline's terminal card lost the course the answer produced. session_end (cancelled) now flushes the pending sightings into the same course card set agent_end paints, before the stopped caption. * chore(workbench): remove the bookmark concept and the saved-courses drawer (#1219) * chore(classroom): remove the bookmark ('collected') concept entirely The stage-meta viewer port introduced a bookmark surface (stage_bookmarks table, POST /api/bookmarks, the isBookmarked sidecar field, and a readOnly rule that let a saved course stay editable). The product has no such concept, so remove it as a closure: - delete the /api/bookmarks route and the stage_bookmarks table plus its query helpers from the persistence bootstrap - drop isBookmarked from GET /api/stage-meta/[stageId] - simplify the classroom read-only rule to readOnly = !isOwner across the sidecar client, ownership signal, classroom load, stage store and the classroom page - keep publish/unpublish, generation-complete, isOwner and isPublic exactly as they were - update the gate and stage-meta route suites and the README mentions The workspace rail's Bookmark glyphs and comments describe the upstream saved-courses (favorites) section, which is driven by isOwner and renders no collect affordance; they are kept as unrelated homonyms. * chore(workbench): remove the saved-courses drawer UI The first pass removed the bookmark data model but kept the rail's "Saved courses" drawer, judging it a separate surface driven by `isOwner === false`. The home/workspace listing is owner-scoped, so that flag can never occur: `allSaved` is permanently empty and the drawer (plus the collapsed-rail Bookmark mini-button) is a dead affordance. Remove it: the SavedDrawer component and its mount, the savedOpen / savedSection state, the allSaved / matchedSaved derivations, the 'saved' variant of the course-list renderers, the mini Bookmark glyph, the drawer-only CSS, and the drawer's i18n keys from all 12 locales. The courses tab is now exactly one folders tree. The authored/favorites split in workspace-tree.ts goes with it; the tree module no longer reads `isOwner`. The discovery course type keeps the field — the shell still reads it for read-only gating. Upstream has no collect concept; the drawer could only ever render empty here. The reference implementation HAS this drawer (its favorites come from its account system), so this removal is a deliberate upstream product decision, not a fidelity bug. * fix(workbench): restore the attach entry, add the rail settings entry, pin all three entry points (#1221) * fix(workbench): restore the composer attach entry by gating it on the live runtime The AttachButton's rollout probe read a `materialsEnabled` field that this branch's /api/agent/runtime never answers (the materials routes gate on the runtime itself, like the stages), so the gate could never pass and the attach button never rendered — the Pro launch and chat composers showed only the @-mention and enhance glyphs. Substitute the field with the runtime's `enabled` value, which IS the upload action's precondition: POST /api/materials answers 404 whenever it is false, so the render condition now equals the action precondition (no dead button). The button's label (`proMode.attach`) is a user-visible string that becomes visible again; port the reference implementation's own translations verbatim into the 11 locales that still carried the Chinese copy. * feat(workbench): add the settings entry to the rail's bottom-left cluster The reference's rail foot carries a cluster of utilities (its saved-courses drawer, the language switcher, the display toggle). This branch removed the drawer — it could only ever render empty here — and the product decision is to fill that freed spot with the settings entry. Add a settings trigger to the foot cluster (expanded rail, beside the language and display toggles, and on the collapsed strip) and mount the model/provider SettingsDialog in the rail, wired to the trigger. It is the same dialog the classic home opens from its header pill; the workspace had no settings entry of its own, so nothing is duplicated within a surface. * test(workbench): pin the restored upload, attach, and settings entry points Covers the three restored entry points: - the courses-tab upload control: rendered beside the course name filter, wired to the discovery hook's ZIP import trigger, disabled while an import runs, and gated by the same condition as its action (the courses tab); - the composer attach control: an actual render of AttachButton under both probe answers (visible when the runtime says the upload path is live, hidden otherwise), its mounts in the launch and chat composers, the branch's runtime-field substitution in the probe, and the reference's own `proMode.attach` copy in all 12 locales; - the settings entry: the trigger in the rail's foot cluster (expanded and collapsed), beside the language and display toggles, opening the SettingsDialog the rail mounts. * chore(config): the Pro workbench flag implies the MAIC Editor gate (#1223) A workbench build without the editor toggle has no way to edit a course: enabling NEXT_PUBLIC_PRO_WORKBENCH_ENABLED while forgetting NEXT_PUBLIC_MAIC_EDITOR_ENABLED produced exactly that split-brain bundle. The workbench IS Pro mode, so its flag now implies the editor gate; the standalone flag remains for deployments that want the classroom editor without the workbench. Documents both flags in .env.example. * fix(agent): wake SSE tails and the runner on durable deltas (streaming fidelity) (#1222) The Pro workbench chat did not stream: the session/owner SSE routes polled the durable event log on a 5s/30s clock with no wakeup, so message_update deltas (written at 150ms cadence) reached the browser in poll-sized blocks and the thinking strip only mounted after the whole reasoning text had accumulated. Port the reference's LISTEN/NOTIFY delta path: - storage: add in-transaction wake hooks (onSessionEventAppended, onOwnerEventAppended, onCancelRequested) so a host queues pg_notify in the same transaction as the durable append; align readEventsAfterForReplay to rank the bounded page so the first delta after the cursor is always kept (the live tail can never starve). Bump @openmaic/storage to 0.18.0. - app: port the process-wide event-notify bus (dedicated LISTEN client, self-check probe, reconnect backoff; notify through the storage transaction surface), wire the store hooks, subscribe both SSE routes before the initial read with the reference's initializing gate, and give the runner one {kind:'session'} subscription whose wake runs the cancel check and the message drain. Polls stay as the lossy-NOTIFY backstop. - lifecycle: start/stop the bus from instrumentation. Tests: storage hook + compaction contract; route wakeup latency; runner wakeup wiring with a fake agent; bus unit tests; PG contracts proving a real append wakes the routes and a live SSE route forwards a message_update on the wakeup, and that a rolled-back append never wakes. Also fix the pre-existing park-attempt-budget PG test TRUNCATE (missing CASCADE against newer FK tables). * fix(storage): asset writes self-deadlocked against pooled PostgreSQL (#1225) * fix(storage): refuse the non-transactional byte-write deadlock configuration A byte store whose plain write() runs on its own pooled connection cannot be invoked from inside a registry write transaction: after the transaction has claimed the blob-row lock, that write blocks on the lock the transaction just took while the transaction waits on the write - a self-deadlock PostgreSQL cannot detect (one side is idle in transaction). There is no lock-safe ordering for such a writer: bytes must be written after the row claim (writing before it lets the collector delete the bytes while the upsert waits), and any second-connection write after the claim is the deadlock. The configuration is therefore detected and refused: - AssetByteStore gains writesOutsideRegistryDatabase?: true, declaring that the layer's plain byte operations cannot contend for the registry's row locks. - PgAssetStore refuses put()/replace() up front (and defends coordinatedWrite) when the byte store has no writeWith and does not declare the flag, throwing a clear configuration error before any row is claimed. - The collector mirrors the guard on its delete path (deleteWith or a declared out-of-registry layer, else a configuration error). - The object store declares the flag (its out-of-transaction write remains legitimate); the in-registry PostgreSQL byte column provides writeWith / deleteWith instead. - Write transactions (put/replace/remove) set SET LOCAL lock_timeout = 30s so any future lock-contention variant fails loudly instead of hanging. Bumps @openmaic/storage to 0.18.0. * fix(persistence): forward the transactional byte methods through the lazy asset byte-store wrapper The no-bucket case of lazyAssetByteStore returned a bare { write, read, delete } and dropped writeWith/readWith even though the underlying PgAssetByteStore has them. The registry's hasTransactionalWriter duck check then failed and put() fell back to the byte store's own pooled connection, which blocks forever on the blob-row lock the registry transaction just took when the bytes live in the same PostgreSQL - the production self-deadlock. The no-bucket layer is statically PgAssetByteStore, so its transaction-pinned methods are forwarded eagerly (typed against the real signatures via PgForwardedByteStore). The bucket case keeps its lazy-probing semantics: no transactional writer exists there, the signed-URL method stays absent or lazy exactly as documented, and the wrapper now declares writesOutsideRegistryDatabase so the registry may run the plain write inside its transaction. New tests pin the wrapper's transactional capability red-to-green and assert put()/resolve() route byte traffic through the transaction-pinned queryable. * fix(home): cap the generate-prep ingest drain at 3s so Generate never waits the full server budget The classic home flow's Generate click drained in-flight ingests for the full 15s server budget. Cap the wait at GENERATE_DRAIN_CAP_MS (3000ms, documented as a UX bound) and reuse the existing timeout fallback: sources that miss the cap proceed on the legacy byte path and each late-resolving id is released. * chore(storage): bump to 0.19.0 over the concurrently landed 0.18.0 * fix(agent): bound every tool call with a timeout; never resurrect a cancelled session (#1226) * fix(agent): bound every tool call with a global timeout and settle it on cancel A tool await that neither resolves nor rejects wedges the session forever: the lease keeps heartbeating and the driver never reaches its next cancel checkpoint. Race every tool execution (in buildAgent) against a hard budget (OPENMAIC_AGENT_TOOL_TIMEOUT_MS, default 10 min, per-tool overrides for known long runners) and against the caller's AbortSignal, so even a signal-ignoring await cannot keep a cancelled session running. On timeout the call rejects with AgentToolTimeoutError; the agent loop turns the rejection into a structured error tool-result the agent can retry or proceed from, and the abort signal is delivered to the tool's in-flight work through a derived controller. Zombie-tool updates after settlement are dropped. * fix(storage): never re-lease a cancel-requested session; settle it as cancelled on claim The claim scan treated a session with cancel_requested_at set as a normal claim candidate: after a restart it re-leased the same session for attempt N+1 and resumed generating despite the pending cancel. claimNextSession now settles such candidates as cancelled under the claim lock (status cancelled, attempt reset, lease and cancel request cleared, terminal session_end event and owner projection) instead of leasing them, then keeps scanning. Bump @openmaic/storage to 0.18.0. * docs: takeaway-style 1.0.0 announcement with bilingual guide links The 1.0.0 head is now a short takeaway block — badge links to the official user guides (English and Chinese), five one-line highlights, and pointers into Features and the workbench setup section — instead of six dense paragraphs. The detailed provider-neutrality and freshness notes move into the Features workbench section, phrased database- neutrally (the announcement no longer names a specific database). Release date corrected to August 27. * fix(workbench): restore editor chrome, mode transition, streaming, materials, mentions, folders (#1229) * fix(workbench): wire workspace folder routes * fix(editor): restore reference workbench chrome * fix(workbench): persist composer materials and course refs * fix(workbench): preserve live reasoning frames * fix(persistence): back off failed streaming saves * chore(workbench): retire stale slice seams * test(editor): cover element pin layer * chore(storage): bump to 0.21.0 for the user-message ref/material fields * chore(editor): translate ported code comments to English * fix(agent): fence durable tool writes and consume cancel requests atomically (#1230) * fix(agent): enforce provider force-off in agent tools and scrub vendor identity from tool results (#1231) * fix(materials): serialize per-owner quota reservations and make crashed uploads reclaimable (#1232) * fix(editor): resolve dock-bar i18n keys, remove dock height drag, wire element referencing (#1233) * fix(workbench): send the opening session message exactly once with refs intact (#1234) * feat(editor): port timeline TTS preview single-flight and voice-all state latching (#1235) * fix(media): restore the reference classic media chain (#1236) * fix(import): adapt imported PPTX canvas size so decks render without overflow (#1237) * fix(editor): complete element referencing — renderer DOM contract and GenUI picking aligned with the reference (#1238) * test(providers): reconcile the provider-config vendor-token debt count after the main merge The integration line's AK/SK fallback for the managed document provider adds occurrences that main's allowlist snapshot predates. Same mixed-composition debt category the group already documents; no new vendor behavior. * test(providers): reconcile vendor-token debt counts with the integration line The main-merge brought main's neutrality-guard snapshot next to integration features it predates (media-extractor fallback chain, local voice-profile deletion semantics, the enabled-TTS helper). Same debt categories the guard already documents; counts updated to the guard's own tally and two grouped entries added. No new vendor behavior. * fix(agent): carry reasoning through the completions dialect so the thinking strip renders (#1239) * feat(skills): add Feynman and spiral curriculum methods (#1240) * feat(agent): port missing reference tools and skills (parity audit) (#1241) * feat(media): retire asset-registry wiring; media and materials follow the reference byte model (#1242) * fix(classroom): center adapted canvases in the stage and send back navigation home during generation (#1243) * feat(settings): skill management with real list, download, delete, and upload (#1244) * feat(settings): skill management section with real list, detail, and zip download * feat(skills): owner skill delete and upload across storage, API, and settings * fixup! feat(settings): skill management section with real list, detail, and zip download chore: neutralize a reference note in the settings header comment * fix(media): persist origin-independent classroom-media references from the agent runtime (#1245) * feat(editor): float the insert toolbar in the outer frame with collapse (#1246) The insert strip was bounded to the slide card, so it could only ever sit on top of slide content: the card's overflow clipped it and it could not be parked in the padding beside the slide. Move it into the studio frame the element picker's panel already roams (CanvasOverlayPortal + the frame selector), so both canvas overlays share one bounding container and their handles behave the same. While picking, the strip rises over the picker and goes inert, which is the z-order CANVAS_OVERLAY_Z already documents. Add a fold beside the grip: the chevron collapses the strip to that grip row and back, with the buttons unmounted rather than hidden. The fold is session-local state owned by EditShell, next to the drag offset, so a surface swap keeps it; nothing is persisted. Expanding a strip parked at the bottom edge re-clamps through the same bounds rule the keyboard move uses. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(workbench): align the chat timeline's left edge with the composer (#1247) * fix(agent): fence session claims while an ask_user question is outstanding (#1248) * fix(agent): settle-time rescue tracks real delivery instead of a count offset (#1249) * fix(persistence): migrate owner_material to oss_key and drop legacy asset_id (#1250) * docs(readme): surface the 1.0.0 user guide badges at the top (#1253) * fix(workbench): show newly created folders in the sidebar without reload (#1254) * docs(readme): add the release version prefix and drop the opt-in framing * fix(workbench): single-source the chat gutter so timeline and composer share a left edge (#1255) The transcript and the composer each established their own column: their own `px-*` gutter and their own `mx-auto w-full max-w-*` centering wrapper. Equal padding values were never enough, because the two columns are centered inside different containing blocks — the transcript's is a scroll container, whose content box is narrower than the composer footer's by the scrollbar's width: transcript text left = pad + (pane - 2*pad - scrollbar - measure) / 2 composer box left = pad + (pane - 2*pad - measure) / 2 The padding cancels out of the difference and what remains is `-scrollbar/2` at every padding value, so the transcript sat half a scrollbar to the left of the composer and tuning the two paddings against each other could not move it. The column is now established once, by the nearest common ancestor of both (`chatColumn`), and the scroll viewport and the composer footer are siblings inside it that add no horizontal inset of their own. The cap carries the gutter on top of the 760px reading measure, so the text column keeps its width. The handed-over question row drops the padding that indented it past the agent's prose; framed rows keep their own inner padding, which is what a card's border sitting on the column edge means. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(workbench): lock pane-embedded classroom to edit mode (#1256) The workspace right pane painted the full learning chrome — speed control, play button, learner avatars, mic bar — for a course the agent had just created, then flipped to edit once the first scene landed. resolveStageChromeMode treated playback as the DEFAULT branch for a hosted classroom, so every shortfall fell into it: a course whose tab opens at stage_link time has no scenes yet, so currentSceneId is null and isHostedSceneEditable is false. A folded pane parked the playback root behind the fold and cross-faded it out over the pane on unfold, and a failed editor chunk dropped into playback permanently. Lock it at the pane instead of defaulting per entry path: - WorkbenchPanelProvider — the single element that mounts a classroom into the workspace — publishes editPinned (visible && !playback). Every entry path passes through it, so none of them decides. - The hosted resolution can no longer degrade to playback. Start Learning (workbenchLearning, new input, split out from pane visibility) is the one door; everything else resolves between the neutral loading shell and edit. - Stage's chrome dispatch is exhaustive on chromeMode, so the playback root is no longer the else-branch of a condition about the current scene. No flicker: chromeMode is resolved during render, and preloadEditor now answers synchronously (isEditorPreloaded) so a remount with the chunk already registered paints edit on the first frame. A failed import is no longer cached forever, so the lock cannot strand the pane. Standalone classrooms keep their stored mode unchanged. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
wyuc
|
06f105ed4d |
feat(server): validate model routing config at startup (#1182)
* feat(server): validate model routing config at startup and deprecate bare model ids * fix(server): keep the bare-id deprecation warning out of request paths parseModelString is reachable with request-controlled strings, so warning there lets clients drive log volume and grow the dedupe set without bound; the warning now fires only from boot-time validation of config-derived sites, whose surface is finite. DEFAULT_MODEL missing-key wording no longer claims requests will fail (unrouted resolution honors client keys), and a keyless provider with a default base URL is not flagged as unconfigured. |
||
|
wyuc
|
b8fbac1a54 |
feat(providers): uniform capability force-off and consistent missing-key contract (#1181)
* feat(providers): uniform capability force-off and consistent missing-key contract * fix(providers): close force-off bypasses found in review |
||
|
wyuc
|
4a9f906a13 |
feat(media): resolve image, video, and ASR models from server config (#1175)
* feat(media): resolve image, video, and ASR models from server config The image generation chain never consulted the server-side IMAGE_<PREFIX>_MODELS config: the route only forwarded the client x-image-model header and each adapter silently fell back to a hardcoded vendor default, so a managed provider could generate with a model the operator never chose. The provider itself also defaulted to a hardcoded vendor id when the client sent no preference. - Add resolveImageModel / resolveVideoModel / resolveASRModel and resolveServerImageProviderId / resolveServerVideoProviderId to the server provider config, mirroring how TTS already resolves server-managed settings. - Routes resolve provider and model server-side when the client sends no preference and fail loud with a clear 400 when nothing resolves; no more hardcoded vendor defaults. - Adapters require an explicit model via requireModel; connectivity probes keep a fixed probe payload since they are auth handshakes, not user-facing generation. - Tests cover the resolver precedence and the fail-loud paths. * fix(media): align model resolution semantics across capabilities resolveImageModel and resolveASRModel let the server pin win unconditionally, while resolveVideoModel implements allowlist semantics (client choice wins when it is in the pinned list, otherwise the first pinned entry). Align image and ASR to the video allowlist semantics so a client picking any server-pinned model is honored, and update the resolver docstrings and tests to pin the aligned behavior for all three capabilities. The classroom media path has no client model and no HTTP response to fail loud with: it now falls back to the first catalog model when the operator pins no _MODELS list, so key-only deployments keep generating media instead of silently skipping every element via the adapter's requireModel backstop. * fix(media): normalize configured model lists and client model ids |
||
|
wyuc
|
e3fd494658 |
feat(tts): add Qwen TTS voice cloning (#1160)
* feat(tts): add Qwen TTS voice cloning via the registration adapter * feat(settings): add Qwen TTS clone-voice manager * fix(tts): harden Qwen voice cloning integration * fix(tts): enforce voice-bound Qwen synthesis * fix(tts): close Qwen voice clone tail issues * fix(tts): close final voice clone review issues * fix(agents): harden voice token parsing * fix(settings): hide the voice-clone model from manual selection * fix(agents): pin the narrator voice to the user's selected voice * fix(agents): keep narrator fallback alive when the pinned voice is unusable * fix(i18n): correct the narration fallback notice copy * fix(tts): bound the narrator voice retry to a single fallback hop * fix(tts): tolerate transient vendor errors in the voice existence precheck |
||
|
LING DUANandwyuc
|
0d308ed78d |
feat(pi): add Native Child Web Search (#1133)
* feat(pi): add native child web search * fix(pi): align native web search with provider system * fix(pi): harden native web search configuration and cancellation * fix(pi): guide Native Child web search selection --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyuc
|
3b4567076f |
refactor(generation): consume the package everywhere and delete lib/generation (#1090)
Part D of #1057 — the switch. All consumers import @openmaic/generation; lib/generation deleted (net -11,900 lines). PBL wiring preserved (single-call -> classified fallback -> injected loop), streaming outline route deduplicated onto buildOutlinePrompt, serverExternalPackages wired, and the change verified by a real-model four-kind end-to-end run against the production build with a rendered-classroom screenshot check. Closes #1057. |
||
|
|
5d5acd6215 |
feat: claude search (#393)
* Added search tool support for Claude * New locales for i18n * Updated translations * Prevent default web search base URLs from being prepopulated * fixed coding style issues * fixed coding style issues * fixing lint errors * fix: add SSRF protection to fetchPageContent in claude.ts Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/099e53fb-e1db-49f4-af7f-4f17cb3d364a Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fix: address all remaining review feedback from PR review thread Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/e1ab8fe2-fe0d-45a5-a0af-30945b99c268 Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fix: rename ambiguous variable names in settings store for clarity Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/e1ab8fe2-fe0d-45a5-a0af-30945b99c268 Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fixing lint errors * fix: address 5 items from second PR review thread Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/09469d80-8e91-449b-8bc2-9c11f83555df Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fixing model verification error * fix: address 3 items from third PR review thread Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/0aa49d19-df5b-4a08-a7e1-2295ae9cfc38 Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fixing lint errors * fix: resolved Copilot review comments * fix: returning string promise for ssrf-guard * fix: returning string promise for ssrf-guard * fix: updated locales, added allowed_callers for claude search tools, handle duplicated claude model keys * fix: updated locales, added allowed_callers for claude search tools, handle duplicated claude model keys * updated tests * using Anthropic SDK for search and connection test in search settings. * uses @ai-sdk with a custom fetch including allowed_callers in payload instead of adding Anthropic SDK as a direct dependency * fixed format * removed @anthropic-ai/sdk as direct dependency * fix: resolve TypeScript errors in claude web-search tests Cast mock.calls through unknown to avoid tuple-index type errors on Vitest mock objects whose parameter types are inferred as empty tuples. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add Claude as a web-search provider on the multi-provider framework Forward-port of the original Claude web-search work onto main's searchWeb dispatcher, per review guidance on #393: - lib/web-search/claude.ts: searchWithClaude via the AI SDK Anthropic provider and Anthropic's native web_search server tool, called through callLLM for usage accounting. Returns the model's answer plus cited sources deduplicated by URL. The page-content enrichment fetch from the original PR is removed entirely, resolving the P1 SSRF finding (no server-side fetches of attacker-influenced result URLs remain). - allowed_callers: ["direct"] is still injected at the fetch layer; the AI SDK's provider-defined web_search tools do not emit it. - Tool version picked from the model: web_search_20260209 for 4.6+ Opus/Sonnet, basic web_search_20250305 for Haiku/older; maxResults maps to the tool's max_uses. - Registry/dispatcher/route wiring: claude provider entry, claudeModelId passthrough (web-search route, classroom generation, generation preview), WEB_SEARCH_CLAUDE_* env config with server-pinned model via WEB_SEARCH_CLAUDE_MODELS, and client base-URL allowlisting for api.anthropic.com. - Settings: per-provider modelId with a model selector in the Claude panel; locale keys added to all ten languages. - Tests: new adapter suite plus claude cases in the dispatcher, constants, route, provider-config, web-search-config, and settings store suites. Closes #392 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E9KqAGmKCtBwMC4HtT4JKE * fix(web-search): trim whitespace around the Claude API key and base URL Keys are pasted by hand into Settings, and nothing in the client key path trimmed them: a trailing newline or space was sent verbatim as the x-api-key header value and came back from Anthropic as 'invalid x-api-key', with nothing in the message pointing at whitespace. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(web-search): correct Claude adapter tool selection, base URL, and timing Addresses the four P2 review findings on #393: - Route dated Claude 4.0 ids (claude-sonnet-4-20250514, claude-opus-4-20250514) to the basic web_search_20250305 tool. The blocklist regex missed them, so they were sent web_search_20260209, which pre-4.6 models reject. Replaced with an allowlist of the models that have dynamic filtering, so unrecognized ids also fall back to the basic tool -- the safe direction, since 4.6+ models accept both. - Pin allowed_callers: ["direct"] only on web_search_20250305. Injecting it into every request disabled the code-execution caller that provides dynamic filtering on web_search_20260209. - Normalize the bare Anthropic root to /v1 before handing the base URL to the SDK, which appends /messages verbatim and would otherwise 404. The adapter is the single choke point for the client allowlist, the server env var, YAML, and the default; other hosts pass through untouched. - Return responseTime in seconds, matching every sibling adapter's WebSearchResult contract instead of exposing milliseconds for Claude. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
wyuc
|
0ae21aa559 |
refactor(pbl): retire v1 write path and dead UI, converge on v2 (#1060)
* refactor(pbl): retire v1 write path and dead UI (#1058) Remove the legacy generator, renderer branches, chat API, and compatibility mirror writes. Keep stored v1 scenes readable through documented legacy adapters, and surface ordinary PBL planner double failures instead of degrading. * fix(pbl): contain planner failure to the scene, support legacy scenes in action regeneration, tighten legacy guards - Contain typed planner failures per scene while preserving retry and skip behavior. - Upgrade legacy PBL content before regenerating scene actions. - Apply video-export import boundaries to legacy readers. - Harden legacy rendering and empty-state handling. - Restore legacy upgrade compatibility coverage. - Remove retired translations and refresh fallback narration and legacy cover documentation. * fix(pbl): keep tolerant legacy configs renderable, guard invalid chat timestamps * chore(dsl): bump to 0.6.3 for README accuracy fix The README no longer references the deleted PBLProjectConfig symbol; the version gate requires a bump for any publishable package input change. * fix(pbl): harden legacy upgrade and persistence contract * fix(pbl): align action prompt with v2 workflow and validate project containers * fix(i18n): remove retired PBL v1 keys from the new fr-FR locale The fr-FR locale landed on main after this branch removed the v1 UI key families; prune the same 38 keys so all 10 locales align with en-US. * fix(generation): sync packaged pbl-actions template with the v2 wording The generation package that landed on main carries a byte-identical copy of the app's pbl-actions template; sync the v2 rewording and bump the package version. * fix(pbl): normalize legacy content and preserve planner status * fix(pbl): propagate provider status and ground action prompts * fix(pbl): classify planner failures for retry, default the project summary prompt input * fix(pbl): fall back to legacy data when v2 payloads are damaged, harden summary building * refactor(pbl): resolve hybrid content through a validity-aware resolver * fix(pbl): keep one v2 acceptance criterion across validator, resolver and persistence Persistence gates on the same container predicate as the validator and the renderer's resolver: payloads the renderer will not treat as v2 are inert bytes everywhere, avoiding runtime side effects for documents the write barrier would reject. An explicit null proficiencyAssessment stays processable. * fix(pbl): require runnable projects and harden legacy fallback * test(pbl): exercise the real save reload and hydration chain * fix(pbl): require an instructor and usable microtasks before v2 takes precedence * fix(pbl): fall back to the loop planner unless the provider failed or the user cancelled * fix(pbl): require role and task identities before v2 takes precedence Runtime normalization can repair status and create the instructor thread, but it cannot synthesize the role id that binds the thread, the microtask ids used by lookup and update paths, or the labels the workspace renders. The runnable predicate and its video-export mirror now require them, so identity-less payloads fall back to usable legacy data. |
||
|
|
ceb05adb41 |
feat(media): allocate generated assets through the registry (#1007 part 2, step b) (#1039)
* feat(media): establish shared asset ownership primitives Introduce global browser asset-pool ownership, asset-reference collection, stage reclamation planning, and lease-based URL access. Carry allocated media identity through storage and generation boundaries with regression coverage. * fix(media): enforce safe resolution across every consumer Route image, video, thumbnail, presentation, and video-export consumers through one resolution state machine. Prevent opaque allocated or generated references from reaching render and export sinks, with fallback and ownership tests. * fix(media): protect document-owned assets across mutations Allocate pool bytes before compatibility writes and document commits, then roll back uncommitted generations safely. Preserve document ownership across edits, retries, imports, speech generation, scene changes, and stage deletion. * fix(media): scope retries and tighten ownership guard Scope retries to the target scene and slide and refuse ambiguous shared-reference mutations. Expand retry rendering coverage and keep direct pool URL resolution behind the shared lease owner. * fix(storage): avoid nested lock during stage cleanup Execute prepared reclamation plans against an explicitly deleted document so the compatibility cascade cannot re-enter the per-document lock. Cover deletion of stage-owned media rows even when the document has no references. * refactor(media): confine reclamation to stage deletion Remove inline pool and compatibility-row cleanup from element, speech, scene, and audio replacement flows. Keep whole-stage reclamation behind explicit stage ownership, preserve stage-less legacy audio rows, and document deferred document-truth sweeping. * fix(media): close retry and resolution gaps Restore shared source tasks after successful forks, scope retries across both whiteboard locations, and wait for parallel TTS workers before rollback. Resolve background media through import, rendering, and PPTX export paths while keeping retry controls visible over last-good bytes. * test(media): execute the consumer safety matrix Replace source-substring checks with resolver seam execution across all six UI consumers, stage hydration, and both exporters. Pin each rollback layer independently and harden the ownership guard against aliased pool imports. * chore(packages): publish the additive DSL field Bump the DSL patch version for the optional speech-action field. Keep the transitional reclamation policy app-owned and document the legitimate transaction rollback removals there. * fix(media): make retry rollback task-safe Delay allocation task re-keying until final document reconciliation succeeds, restore shared source tasks on failed forks, and require exact stage-whiteboard targets before falling through from a missed scene. * fix(media): clear private assets and refresh leases Delete the asset-pool database during the confirmed local-data wipe. Notify the app lease layer after same-id replacement so mounted consumers re-resolve current bytes without reaching into storage internals. * test(media): pin closure safety guards Exercise the CSS allocation boundary, unfiltered legacy-row ownership, delete-and-undo byte survival, and real distinct consumer seams. Restore the video-only manifest overwrite condition and share the direct video resolution hook across both element variants. * test(media): narrow rollback element assertion Narrow the reconciled slide element to an image before checking its source so the rollback regression remains type-safe under the full root compiler configuration. * fix(media): close lease refresh races Gate the first batch publication by unique resolved refs, then publish every replacement snapshot without mutating the prior React state object. Serialize invalidation behind pending releases, evict rejected refreshes, and register replacement observation at the pool boundary. * fix(media): reopen pool after clear failures Always evict the singleton once its store has been closed, including blocked and failed database deletion paths. Report blocked deletion as deferred and prove a later write uses a fresh live store. * fix(media): isolate shared retry progress Track forked regeneration under the selected element until it receives a fresh asset identity, leaving the shared source task and bytes untouched. Surface targeted failures through renderer task lookup and skip the redundant fork reconciliation lock. * fix(media): close final asset retry gaps Keep blocked asset clears fail-loud until a successful retry, with actionable settings guidance. Clear failed shared-fork state across durable and live key spaces, and pin renderer lookups, lease publication identity, and committed-ref rollback protection. * fix: preserve actionable retry failures Localize the blocked cache-clear recovery hint across every supported locale and pin the deferred-error mapping. Retain durable fork failure rows while retries run, deleting them only after successful generation so unstructured failures survive reload. * fix(media): hydrate legacy stored video thumbnails Home-page recent-video thumbnails regressed for legacy Dexie mediaFiles rows keyed by gen_vid placeholders: the reworked hydration resolved the row's bytes through the sealed resolver but never surfaced the stored blob (and its poster) as object URLs for the preview card, so the CI recent-video-thumbnail e2e specs found no visible element. Hydration now materializes legacy stored video rows into blob URLs for both the element src and poster while keeping the resolver invariants: opaque refs still never reach a DOM src, and concrete addresses are never blanked. Unit pins cover the seam so the vitest suite catches this class without a browser. * fix(media): resolve sole restored legacy video Classroom playback restored tasks by exact document media references. Legacy gen_vid references can outlive the key used by the one persisted video row, leaving the player on a placeholder even though bytes were restored. Select the sole completed stage video only for legacy sequential refs after exact and reconciled matches. Keep exact failures authoritative, refuse ambiguous candidates, and cover success, ambiguity, and failure precedence in unit tests. * fix(media): scope legacy video recovery Decide restored legacy video recovery once from the complete document and record it through the shared task lookup consumed by playback, editing, and resolved slides. Keep ambiguous documents as placeholders, apply the same decision to thumbnail hydration, and preserve exact failure precedence. * fix(media): exclude claimed video recovery tasks Model restored legacy recovery as a two-pass match across document video elements and task rows. Remove tasks claimed by exact, targeted, or placeholder lookup before applying the sole-candidate fallback, and cover the ownership/cardinality matrix. * fix(media): unify video element resolution Centralize source, task, poster, and legacy recovery decisions for every video consumer. Ensure direct URLs win over opaque refs and play_video waits on element-targeted retry tasks. * refactor(media): route legacy recovery through resolver Let document-aware consumers request legacy video recovery through the unified element binding API, keeping thumbnail hydration on a single decision path. * fix(media): preserve import refs and prefer pool bytes Recognize unambiguous extensionless relative media addresses during classroom import. Resolve allocated export and thumbnail assets from the shared pool before falling back to lagging compatibility rows. * fix(export): preserve concrete video sources Route PPTX video elements through the shared media binding resolver so an unresolved opaque reference cannot replace a playable source. Complete browser and persisted-store cleanup when asset-pool deletion is deferred, while retaining distinct hard-failure behavior. * fix(media): fork retries without exclusive ownership Enumerate logical asset owners across every persisted document before allowing global pool replacement. Thread explicit targets through fresh-id rewrites and cover cross-document aliases plus unreadable ownership. * fix(media): guard global asset reclamation Share a fail-closed persisted-document liveness check between stage deletion and retry replacement. Preserve cross-document pool aliases while deleting stage-owned compatibility rows and cover enumeration failures. * fix(media): cover complete slide asset references Route slide media traversal through a shared mutable slot contract so backgrounds participate in export, thumbnail hydration, collection, and rewrite lifecycles. Snapshot complete surviving-document refs once per reclamation and preserve manifest-only owners while retaining fail-closed behavior. * fix(media): preserve exclusive retry asset ids Allow targeted retries to replace exclusively owned pool assets in place. Keep shared and unprovable ownership paths on fresh allocations, and pin production-shaped retries plus compatibility-row cleanup. * fix(media): revalidate asset bindings at completion Recheck repository-wide ownership before replacing generated media and fork scoped retries when exclusivity changed. Route video export selection through the unified resolver and keep concrete posters independent of task state. * fix(media): count unflushed owners and broadcast replacements The completion-time exclusivity proof read only the persisted document, but slide duplication updates the Zustand aggregate synchronously and schedules persistence behind a debounce. A retry finishing inside that window saw a single persisted owner and replaced the bytes behind a reference the duplicate also held. The proof now also counts owners in the live stage snapshot when that snapshot represents the stage being retried, so an unflushed duplicate forks instead. Same-id replacement notifications were realm-local, so a second tab showing the same classroom kept its lease pinned to the superseded blob URL. The notification now travels over a BroadcastChannel; each receiving realm runs its own observers against its own pool, so a spoofed message can at most force a re-resolve. A missing or failing channel never fails the replacement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): bind replacement listeners and spare shared audio rows A realm that only renders never sends a replacement, so binding the channel from the sender path left passive tabs deaf to peers. Binding now happens where the observer is registered, when the asset-pool module loads, and the receiving realm resolves its own pool lazily so a cleared or unavailable pool degrades to the next resolve instead of throwing. Observer notifications ran under Promise.all, so a rejection surfaced after BrowserAssetStore.replace had already committed and turned a durable success into a reported failure. They are settled individually now; the callback is wrapped because a synchronous throw would otherwise escape before allSettled sees the array. audioFiles rows are keyed globally by audioId, so deleting a stage removed the sole row for an id a surviving document still referenced — playback and both export paths read that table directly and cannot fall back to the preserved pool blob. Rows are now filtered against surviving references, while a failed enumeration still withholds only the irreversible pool removal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): serve replaced bytes from the pool everywhere Unknown survivor liveness deleted every planned audioFiles row. Those rows are keyed globally by audioId, and playback plus both export paths read the table directly, so losing one is as irreversible for them as removing the pool entry. Unknown liveness now preserves the rows too, leaving bounded garbage for a later pass that can prove exclusivity. The earlier pool-first change covered PPTX, video collection and thumbnail hydration but missed classroom ZIP export, which still serialized the stale compatibility row after a lagged same-id replacement, shipping media the classroom no longer renders. Auditing every direct reader of the media and audio tables surfaced the same gap in playback: speech regeneration also replaces bytes under a stable id and does not roll the pool back when the compatibility write fails, so the player kept serving superseded narration. It now resolves the pool first and falls back to stored rows for legacy and imported audio. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(audio): replace exclusively owned speech clips in place regenerateSpeechAudio accepted the action's audioId but always passed undefined as replaceAssetId, so regenerating an exclusively owned pool-backed clip allocated a new asset every time, rewrote the action and orphaned the previous pool entry and compatibility row until stage reclamation — contradicting the stable-id path generateAndStoreTTS already implements for media. Ownership is now established before synthesis, and the rule itself moved to the shared reference module so media retries, poster replacement and speech regeneration consume one implementation instead of restating it. An exclusively owned clip keeps its id and has its bytes replaced; a shared clip, a legacy id with no pool entry, or unprovable ownership still gets a fresh allocation so other holders keep their audio. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(media): drop the import left behind by the ownership move Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): resolve audio pool-first and fence peer realms Stable-id TTS regeneration commits replaced narration to the pool before the audioFiles mirror write, so a failed mirror leaves the row stale. AudioPlayer already resolved pool-first, but audioObjectUrl, collectAudioFiles and the video timeline dependencies still read Dexie directly and would serve the superseded clip. All allocated-audio readers now share one resolver, with Dexie kept as the fallback for legacy and imported rows. The exclusivity proof modelled unflushed owners in the active realm only, so another tab duplicating the same asset during its save debounce could still be observed as a single owner and have its bytes replaced globally. A peer's pending state cannot be read across realms, so presence is probed instead: any realm holding the stage forces the fork path. A deferred asset-pool deletion no longer reloads the page. The database is still on disk and the guidance asks the user to close the other tab and retry, which the reload discarded. The decision moved into a shared helper so the rule is pinned rather than living inline in the component. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(media): fail closed when presence cannot be probed The presence helper documented that an unanswerable probe must count as a peer, but every unavailable path — no BroadcastChannel, a constructor that threw, a send that failed, and the window before the pool's asynchronous binding completes — returned false, so the ownership proof cleared a single local owner and replaced globally shared bytes in place. Probing now returns present, absent or unknown, and only a probe that was actually sent and went unanswered is absent; the ownership decision treats unknown exactly like present. The pool declares its binding intent synchronously so a probe issued during the load-time window waits for the bind instead of concluding that presence is unavailable, and releases that gate if the import fails. Coverage reaches the write boundary: with presence unknown, a production-shaped targeted retry forks to a fresh id instead of calling replace, and the original bytes stay intact for a peer's unflushed owner. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(dsl): bump to 0.6.3 after the release dedupe took 0.6.2 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(i18n): add the deferred-clear guidance to fr-FR The locale landed on main after this branch added the key, so the alignment check flagged it as the one missing translation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
zhifu gaoandLauraGPT
|
2b3f2eaa37 |
feat(asr): add local FunASR provider (#1044)
Signed-off-by: LauraGPT <199134975+LauraGPT@users.noreply.github.com> Co-authored-by: LauraGPT <199134975+LauraGPT@users.noreply.github.com> |
||
|
littlebullGitandwyuc
|
3d85525665 |
[codex] Add Amazon Bedrock LLM provider support (#538)
* Enable AWS-hosted text models without exposing ambient credentials Rebase the Bedrock provider onto current main and close the review-requested security, credential lifecycle, usage attribution, configuration, and metadata gaps. Constraint: Bedrock may use ambient AWS credentials only when explicitly enabled by the server operator Rejected: Trust client-supplied provider types | permits built-in keyless IDs to reach Bedrock credentials Confidence: high Scope-risk: moderate Directive: Keep provider ID/type validation at both request resolution and model construction boundaries Tested: targeted Bedrock/resolver/config/usage tests; TypeScript; ESLint; i18n alignment; production build Not-tested: Full suite has 27 failures in unchanged quiz/runtime and runtime/chat-storage tests * Give Bedrock a recognizable provider identity Use the official AWS architecture service icon so Bedrock no longer falls back to the generic provider cube in settings and model selection. Constraint: Preserve the AWS-provided artwork without redesigning the service mark Confidence: high Scope-risk: narrow Tested: SVG XML validation; provider unit test; TypeScript; ESLint; production build --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
12ccd315ea |
Add Atlas Cloud LLM provider (#948)
* Add Atlas Cloud provider * Account for Atlas Cloud non-thinking model metadata * test: verify Atlas Cloud thinking payloads * style: format Atlas Cloud payload test * test: type Atlas Cloud fetch mock --------- Co-authored-by: binyangzhu000-sudo <224954946+binyangzhu000-sudo@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
b4e3145c90 |
feat(roster): single-source the generated agent roster on the stage document (#994)
* feat(roster): single-source the generated agent roster on the stage document The generated agent roster used to live in two places at once: the stage document carried generatedAgentConfigs, while a per-stage IndexedDB mirror (db.generatedAgents) was the actual read authority — and the only home of each agent's voice binding. The split caused real losses: voices vanished on device switches, exports, and shares (the mirror never travels), the load path trusted a device-local table over the portable document, and roster edits persisted through a bespoke debounce with no beforeunload flush. Make the stage document the single source of truth: - Contract: add optional voiceConfig/voiceDesign to GeneratedAgentConfig in @openmaic/dsl (VoiceDesign moves into the contract; the app re-exports it). Additive optional fields — not a breaking serialized-shape change, so DSL_VERSION stays put per the version policy. - Writes: remove every mirror writer (registry saveGeneratedAgents, the stage store's debouncedSaveAgents bypass, classroom-load hydration, import direct writes). setStageAgents now only marks the stage dirty for the shared persistence scheduler plus synchronous in-memory registry/selection mirrors; generation and import embed the roster on the stage. - Reads: classroom load hydrates the registry from the loaded stage's generatedAgentConfigs; export reads the stage roster directly. - Lazy migration: the mirror is retained read-only. On load, a roster (or voice fields) missing from the document is backfilled from the mirror, committed onto the in-memory stage, and persisted by the next flush — idempotent, and gated on the current stage id after every await so a classroom switch cannot leak a stale roster into the registry. - Deletion: deleteStageData discards the deleted stage's pending persistence work so a queued flush cannot resurrect the document. - Round trip: classroom ZIP manifests now carry the voice fields, so export/import preserves agent voices. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): harden deletion, migration, selection, and ingress paths Review hardening on top of the roster single-source change: - Deletion vs in-flight persistence: deleting a stage now registers a session-scoped tombstone (lib/utils/deleted-stages.ts) before the cascade starts. Every persistence landing point checks it — the scheduled flush and the departing-stage retry drop their snapshots, and the aggregate/incremental saves re-check under the document lock, including the incremental path's full-save fallback that would otherwise rebuild a deleted document from the in-memory snapshot. A deletion that fails before the document is removed lifts the tombstone so the still-existing stage keeps persisting. - Lazy migration termination: a fruitless legacy-mirror probe (nothing to merge — e.g. server-generated rosters, which are voiceless by design) is remembered per stage for the session instead of re-querying the mirror on every load forever. Successful merges are not memoized so a failed flush is retried on the next load. Docstrings now state the real lifecycle. - Selection provenance: setStageAgents no longer overwrites a user-set agent selection with the full roster. Stage-derived selections still track the roster; a user-set auto selection is only narrowed to surviving agents (newcomers are not auto-selected, the user-set flag is untouched), and a user-set preset selection is left alone — matching restoreAgentSelection. - Mirror hygiene: deleteStageData clears the deleted stage's rows from the legacy roster mirror (best-effort), and the mirror table's docs now describe its actual access pattern (migration reads + deletion hygiene). - Registry persistence: the agent registry's persist snapshot partializes generated agents out of localStorage, making the stage document the roster's only durable home; the rehydration merge filter remains as defense in depth. - Ingress validation: imported manifest voice fields are structurally validated (malformed bindings dropped per field, the agent survives), and applyGeneratedAgentsToRegistry validates providerId against the known TTS provider registry instead of casting, treating unknown providers as "no bound voice". - Contract: drop the speculative AgentVoiceConfig.modelId — an audit of every roster producer, current and historical, found none that ever emitted it. Contract fields are added once a producer exists, not before. New tests cover the deletion tombstone at both the store and storage layers (including the missing-destination full-save pin and the lock-race re-check), the fruitless-probe memo, the selection-provenance matrix, persistence exclusion of generated agents, and voice-field sanitization on import. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): close tombstone lifecycle, probe-memo, and selection-intent gaps Second hardening pass over the deletion/migration/selection fixes: - Tombstone lifecycle: a deleted classroom id can legitimately come back (deletion is client-side only). Explicit (re)creation points now lift the tombstone — server-copy restore in applyClassroomStageAndScenes and backup restore in importDatabase — so edits after a same-session restore persist again, while in-flight flushes stay fenced. - Failed deletion now restores the dirt it discarded: deleteStageData snapshots the pending map plus the in-flight round before tombstoning, and re-marks it when the delete fails with the document still present. - Legacy-mirror probe memo now distinguishes a FAILED read (null, retried on the next load) from a confirmed-empty mirror (memoized), so a transient IndexedDB error cannot suppress migration for the whole session. - Incremental save tail (currentScene KV row + chat sessions) is tombstone- fenced like the aggregate path, and both paths re-check the tombstone immediately before every write, with comments scoped honestly for the lock-free (no Web Locks) best-effort LWW fallback. - Roster-edit selection mirror: an empty intersection now falls back to the full roster and clears the user-set flag (matching restoreAgentSelection's length > 0 gate), and a user-set selection that equals the pre-edit full roster keeps tracking the roster wholesale so newcomers are not excluded forever by the AgentBar auto-toggle snapshot. - TTS/ASR provider lookups use Object.hasOwn instead of `in`/bare indexing, so prototype-chain keys ('toString', 'constructor', ...) no longer pass the provider whitelist or resolve to Object.prototype members. - Documented the per-tab limit of the tombstone set. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): replace the boolean deletion tombstone with a deletion-generation model A boolean tombstone cannot express the deletion lifecycle once a deleted id legitimately comes back (server-copy restore on revisit, backup import): after the restore lifts the flag, a pre-delete flush still in flight is indistinguishable from a post-restore edit and can overwrite the restored document with pre-delete content. Two rounds of point fixes kept leaking variants of this race, so this restructures the fence instead. deleted-stages.ts now keeps a per-stage monotonic deletion epoch plus a deleted flag. markStageDeleted bumps the epoch; unmarkStageDeleted clears only the flag and never rewinds the epoch. Every persistence path captures the epoch at the moment it captures the data it will write (flush-round snapshot, departing-stage snapshot, aggregate save entry) and re-checks "captured epoch is current AND not deleted" immediately before each individual write. Invariants: - A write captured before a deletion can never land after it, even when a same-id restore has lifted the deleted flag (the captured epoch is permanently stale). - A write capturing data after a restore observes the current epoch and persists normally. - A failed deletion lifts only the flag; the restored pending changes are re-queued as descriptors, so their flush re-captures current store state under the current epoch and is neither dropped nor a stale replay. Also fixed in the same restructuring: - Incremental tail writes (currentScene, chats) each re-check independently; a delete landing while the first tail write is awaiting now fences the chat write instead of riding the earlier check. - Deleting a classroom evicts it from the warm in-memory store, and loadFromStorage treats a warm-but-deleted stage as not loaded (discarding the ghost), so navigating Back to a deleted classroom reaches the server-restore path instead of rendering an editable ghost whose every edit is silently dropped. - importDatabase records each document's pre-import deletion state and reinstates it when a failed import rolls the document back, so an outstanding flush cannot recreate the rolled-back document. - Docstrings on the failed-deletion restore path now state that only dirt captured before the deletion is restored; edits refused during the deletion window are not part of the snapshot. New tests cover the epoch invariants at the storage layer (pre-delete rounds dropped across restores on the incremental, aggregate, and tail paths; post-restore captures landing; strictly increasing epochs), the scheduler layer (departing-retry delete+restore straddle; restored dirt flushing under the current epoch), the warm-ghost restore driven through the real loadFromStorage, and the import lift/rollback pair. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): report fenced drops, gate ghost eviction on cascade settlement Edge hardening around the deletion-epoch fence; the epoch model itself is unchanged. - An epoch-stale drop is now reported, not disguised: saveStageData / saveStageDataIncremental return a distinct 'stale-dropped' status (entry check, every in-mutation re-check, and both tail-write re-checks) instead of a shape identical to success. saveToStorage reacts by skipping all success bookkeeping - no chatSnapshot rebind to a snapshot that never landed, no pending-dirt clearing - and returns false, keeping its "true means verified write" contract honest. persistDirtySnapshot keeps its existing (correct) semantics explicitly: a drop has nothing to retry. - Warm-ghost eviction is settlement-gated: the deletion cascade records an in-flight bit (begun by deleteStageData, settled in its finally), and the deleted-warm branch of loadFromStorage keeps the warm state while the cascade is unsettled - a delete can still fail before removing the document, in which case the warm state plus the restored pending dirt is the only copy of the pre-delete edits. Once settled, the ghost discard and server-restore path behave as before. - Read-side landing re-check: loadFromStorage re-checks isStageDeleted after hydration, immediately before set(), mirroring the write-side "re-check immediately before landing" discipline - a lock-free mid-cascade read can no longer re-materialize a ghost classroom. - clearStoreForDeletedStage re-checks isStageDeleted at eviction time, so a same-id restore completing inside the cascade-tail window is never wiped. - capturedEpoch is now a required parameter on both storage entry points: the default (call-time capture) silently reopened the capture-point/ validation-point split for future callers; the type system now enforces the pairing. All production callers already passed it explicitly. New tests cover the dropped-status contract, the cascade in-flight lifecycle (including both failure shapes), keep-warm during an unsettled delete with end-to-end pending restore after a failed delete, settlement- then-eviction on success, the eviction guard against a tail-window restore, and the read-side hydration re-check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): settle the deleted-warm load by awaiting the cascade outcome Hardening around the deletion settlement gate introduced previously; the epoch model is unchanged. - The deleted-warm branch of loadFromStorage no longer completes the load while the deletion cascade is undecided (which exposed a classroom whose edits were refused without being restorable, and left a successful deletion's eviction with no follow-up reload). It now parks on a per-stage settlement promise (stageDeletionSettled) and branches on the outcome: a failed delete keeps the warm, restored classroom; a successful delete falls through to a full cold load so the server-restore path recovers the route; a navigation that moved on during the park is stopped by the load token guard. The eviction (clearStoreForDeletedStage) deliberately stops claiming a new load token so the parked load stays current - ghost re-materialization is fenced by the read-side deleted re-checks instead. No deadlock: a parked load holds no document lock, and the cascade never waits on a load. - 'stale-dropped' now propagates through the debounced flush path: persistDirtySnapshot returns the sentinel instead of an empty failure set, and startFlushRound skips the chatSnapshot rebind on it (mirroring saveToStorage) - a falsely rebound baseline would let later chat saves no-op-skip chats that never landed and corrupt cross-tab conflict detection. Pending-map clearing keeps its empty-failure semantics (restored descriptors mint fresh revisions). - The cascade in-flight state is a counter (begin++/settle--) instead of a boolean, and deleteStageData is single-flight per stage (a concurrent second call joins the first cascade), so overlapping deletions can never expose an undecided cascade as settled. - The StaleDroppedSave docstring no longer overpromises: a tail saveCurrentScene can land after the cascade's clearCurrentScene; the orphaned cursor row is ignored by the load path rather than removed by the cascade. Net +8 tests: overlapping-cascade counting, settlement-promise resolution (success and failure), concurrent double-delete single-flight join, the parked mid-cascade Back in both outcomes plus the navigation-away guard, and the fenced flush round keeping an honest chat baseline end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): re-verify store identity on keep-warm resume, re-run joined deletes after a mid-cascade restore Three hardening refinements on the deletion/persistence fencing: - loadFromStorage's parked keep-warm branch now mirrors the success path's store-identity guard: if a tokenless writer (e.g. a database import) replaced the stage during the park, a failed deletion falls through to the cold load instead of reporting another classroom as the kept warm state. The ghost-discard block is correspondingly gated on the deleted flag so the fall-through does not mislabel a surviving document as a ghost. - deleteStageData: a joined call carries its own deletion intent. When the first cascade fulfills but a same-id restore lifted the deleted flag in the meantime, the joined caller now runs one fresh cascade against the restored document instead of reporting the pre-restore outcome. Exactly one re-check per call (no recursion); a rejected first cascade still propagates unchanged to every joined caller. - Narrowed the failed-delete restore-ownership comments: the restore covers only the pending map plus an in-flight flush round's dirt; a departing-stage snapshot is outside that capture and is fenced and dropped by design. Comment-only, no behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(roster): gate deleted-warm handling on stage identity, keep an explicit empty roster authoritative Final-audit fixes, three items: - loadFromStorage keyed its deleted-warm settlement handling (park / keep-warm / ghost discard) behind the scenes.length > 0 shortcut, so a zero-scene warm ghost of a deleted classroom skipped it entirely: the cold load found no document, the ghost stayed in the store, and the classroom loader's server-fallback gate (!getCurrentStage()) never ran — the silently-uneditable trap this branch exists to close, in its scene-less form. Deletion handling now keys on stage identity alone, before any scene-count shortcut; the non-deleted warm skip still requires scenes. - rosterNeedsLegacyFallback collapsed "roster field absent" (document predates roster persistence) with "explicitly persisted empty roster". A future writer persisting [] on a device whose read-only mirror still held stale rows would have its emptied roster resurrected on every load. The full-lift branch now triggers only when the field is absent (undefined); an explicit [] is authoritative and never probes the mirror. No current writer produces [], but the read side no longer depends on that invariant. - The GeneratedAgentConfig docstring claimed the voice fields are not a breaking change; that only holds for this codebase's tolerant structural validators. The generated stage.schema.json sets additionalProperties: false, so consumers pinning an older published schema artifact reject documents carrying the new fields. The docstring now states both sides honestly. Tests: a zero-scene deleted warm ghost is restored through the real loadFromStorage (fails pre-fix), an explicit empty roster does not resurrect stale mirror rows (fails pre-fix), and the rosterNeedsLegacyFallback unit matrix covers undefined vs []. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(deletion): re-queue aggregate persistence when a failed deletion is recovered Direct aggregate saves (saveToStorage: generation completion, server-restore hydration) run outside the pending map and the flush round, so the deletion path's pre-delete dirt snapshot cannot describe them. When a deletion bumped the epoch while such a save was in flight, the save was correctly fenced ('stale-dropped') — but if the deletion then failed before removing the document, the recovery only re-queued the snapshotted scheduler descriptors. The fenced aggregate-only content stayed memory-only with no retry anywhere, and a later reload lost it. The failed-deletion restore now merges a full re-mark of the aggregate (structure / stage / outline / currentScene / chats plus a descriptor for every current scene) into the recovery set, still guarded on the store holding the stage. Because the flush recaptures the CURRENT store state under the CURRENT epoch, this necessarily carries whatever the fenced aggregate save held — and, for the same reason, edits refused by the scheduler during the deletion window, which previously stayed memory-only as well; the scope docstrings now say so instead of declaring them excluded. New integration coverage drives the real store scheduler and the real storage layer end to end: a genuinely fenced in-flight saveToStorage (generation completion) plus a pre-removal deletion failure ends with the completion flag durable on the next flush; a deletion-window scene edit survives the same way; and a successful deletion re-marks nothing (control). Both recovery tests fail against the previous restore logic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
Yizuki_Ameandwyuc
|
7c2aaafc91 |
fix(ssrf): harden provider redirects and ISATAP detection (#928)
* fix(ssrf): detect private IPv4 in ISATAP addresses * fix(ssrf): reject redirects in provider probes --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
Yanpeng Wangandwyuc
|
9670100363 |
feat(extraction): media (audio/video) extraction + AliDocMind provider (#887)
* feat(extraction): add AliDocMind provider + media extraction abstraction Adds AliDocMind (Aliyun Document Mind LLM version) as a new vendor alongside unpdf/MinerU, and introduces the media (audio/video) extraction layer that mirrors the document extraction one. Document side (file: pdf/docx/pptx/xlsx/images): - PDF_PROVIDERS gains an `alidocmind` entry; parseWithAliDocMind() maps layouts[] -> ParsedPdfContent. Flows through the existing document extractor registry, so AliDocMind is selectable anywhere MinerU is. - PDFParserConfig / DocumentExtractorConfig gain accessKeyId/accessKeySecret (AliDocMind uses AK/SK, not a single apiKey); env fallback via ALIDOCMIND_ACCESS_KEY_ID / ALIDOCMIND_ACCESS_KEY_SECRET. Media side (mp4/mp3/wav/... -> MediaArtifact): - New lib/media-parse/ domain mirroring lib/pdf/ (types/constants/providers). parseMedia() maps AliDocMind segments[]/audio_frames/video_frames -> MediaArtifact (transcript + keyframes). - MediaExtractorProvider interface + media registry + extractMedia() entry, symmetric to DocumentExtractorProvider / extractDocument(). - MediaArtifact and the ExtractionResult/Artifact/Error/Job envelope live in lib/document/types.ts (re-exported from @/lib/document). Shared AliDocMind SDK wrapper (lib/pdf/alidocmind-client.ts) handles the submit -> poll -> get flow for both sides via @alicloud/docmind-api20220711. Tests: env-gated smoke test (tests/document/alidocmind.smoke.test.ts) drives a real PDF and a real video through AliDocMind; media-artifact type test; extractor-registry test updated for the new provider. Part of #621 (MAIC ETL). Media extraction is the sibling to the document extraction landed in #741. * feat(extraction): wire AliDocMind AK/SK through settings UI + routes Surfaces AliDocMind in the (post-#837) Document Parsing settings panel as a peer of unpdf/MinerU — one panel, one credential entry, more supported formats. Threads Aliyun AccessKey ID/Secret from the store through the extraction routes. - Store: pdfProvidersConfig + setPDFProviderConfig gain accessKeyId/accessKeySecret; default alidocmind entry added. - pdf-settings.tsx: AliDocMind branch renders AccessKey ID + Secret inputs (secret masked with show/hide) and a Test Connection button; request-URL preview shows the DocMind endpoint. Provider auto-appears in the panel via PDF_PROVIDERS; supported-format badges come from #837's registry (ALIDOCMIND_MIMES added to lib/document/mime.ts). - verify-pdf-provider route: alidocmind branch verifies AK/SK via a lightweight authenticated probe (verifyAliDocMindCredentials) — auth-level errors fail, anything else passes. - extract-document route + generation flow (app/page.tsx, generation-preview): accessKeyId/accessKeySecret carried through session → FormData → config. - i18n: alidocmindAccessKeyId / alidocmindAccessKeySecret in 8 locales. - Icon: reuse /logos/bailian.svg (Aliyun family) instead of a missing asset. Verified end-to-end against the running app: AliDocMind panel renders, Test Connection returns "连接成功" through the real Aliyun API. Part of #886. * feat(extraction): route audio/video uploads through extractMedia() (reuse document path) Media uploads now flow through the same upload picker, /api/extract-document route, and generation pipeline as documents — no separate upload area. Only the extraction differs: media mimes dispatch to extractMedia() -> MediaArtifact, which is flattened to the text shape the generation pipeline already consumes. - mime.ts: register audio/video formats in DOCUMENT_FORMATS (accept string, extension map, badges resolve for them); add MEDIA_PROVIDER_SUPPORTED_MIME_TYPES + SUPPORTED_MEDIA_MIME_TYPES, kept separate from PROVIDER_SUPPORTED_MIME_TYPES so the document drift-guard stays document-only. mimesForProviders() folds in a provider's media mimes, so the existing upload helpers (getAcceptStringForProviders / isMimeSupportedByProviders / format badges) cover media automatically when the provider supports it. - extract-document route: media mimes dispatch to extractMedia(); MediaArtifact flattened to timestamped text (synopsis + transcript + keyframes). - Tests: 6 media cases in mime.test.ts (accept/validation/badges/normalization). Verified: uploading a real video through the route returns synopsis + timestamped transcript/keyframes as text (curl, real AliDocMind key). Part of #886. * feat(extraction): use Alibaba Cloud icon for AliDocMind provider Replace the placeholder bailian.svg with a dedicated Alibaba Cloud icon mark (the square symbol from the official wordmark, text removed) so the provider reads as Aliyun rather than Bailian. Square aspect matches the other provider icons. Source: Alibaba Cloud / Alibaba Group brand assets. * style: prettier format AliDocMind + media extraction files * fix(extraction): harden AliDocMind — SSRF guard, cred verify, env gate, table text Addresses review findings on the AliDocMind provider: - SSRF (high): the media branch of /api/extract-document now runs the same validateUrlForSSRF check on a client-supplied baseUrl as the document branch, so an audio/video upload can't point the server's Aliyun SDK at an internal host. - Credential verify (high): verifyAliDocMindCredentials now whitelists success signals instead of blacklisting auth errors. Probed against the real API: valid creds + bogus job returns a no-throw "BizIdNotExistOrResultExpired" body; invalid creds throw InvalidAccessKeyId.NotFound. Only a no-throw response or a "biz-not-found" business error counts as valid — an unreachable endpoint, a localized error, or throttling now correctly reports failure instead of a false "connection successful". - Env-fallback gate (med): resolveCredentials no longer reads ALIDOCMIND_ACCESS_KEY_ID/SECRET unconditionally. Env fallback is opt-in via allowEnvFallback, which the route sets only for a server-managed provider, so an unauthenticated client request can't silently run on the server account. - getDocParserResult error body (med): fetchResult throws on a non-200 result envelope instead of returning {} (empty text presented as success). - Media pagination (med): stop after the first page when segments[] are present — layoutNum/layoutStepSize address layout blocks, not media segments, so re-requesting would loop over the same segments to the safety cap. - Table content (med): tables/charts carry content in llmResult, not markdownContent; the layouts→text mapping now prefers llmResult for those types so table content isn't dropped (tables: true was advertised). - Dead code: remove getCurrentMediaParseConfig (referenced non-existent store fields; the single-panel UI reuses pdfProvidersConfig). - Dedup: media MIME list lives only in lib/document/mime.ts (ALIDOCMIND_MEDIA_MIMES); the media registry imports it. - Tests: smoke test asserts durationMs is in ms (>1000 for a ~52s clip) to guard a ms/s unit mismatch; uses allowEnvFallback for the env-cred path. Verified with the real key: PDF + video smoke tests pass; verify classifies valid vs invalid creds correctly. Part of #886. * feat(extraction): extract AliDocMind images to base64 (parity with unpdf/MinerU) AliDocMind embeds figure/picture image URLs inside each layout's markdownContent (markdown ``), not a dedicated field, and the URLs are short-lived OSS signed links. Previously we emitted `images: []` and left the expiring URLs inside the extracted text. Now, for figure/picture layouts we: - parse the OSS image URL out of markdownContent, - fetch it at extraction time (before the signature expires) and re-encode to PNG base64 via sharp — the same base64 `images[]` contract unpdf/MinerU produce, so downstream storeImages → IndexedDB → slide works unchanged, - populate metadata.pdfImages + imageMapping (the generation flow prefers pdfImages), - strip the remote-URL markdown from the emitted text so expiring links don't leak into the prompt. Downloads run concurrently; a failed/`sharp` image is dropped, never failing the whole parse. Also fixes the prior over-broad table handling: only `table` blocks read llmResult; `figure` is treated as an image (chart-figure llmResult still kept in text). Verified with the real key: a sample PDF yields 24 base64 images in both images[] and metadata.pdfImages, and no oss-cn-hangzhou URLs remain in text. Part of #886. * docs(test): note AliDocMind video smoke test is non-deterministic server-side * fix(extraction): correct AliDocMind pageCount (pageNum is 0-based) Verified against a real response: AliDocMind reports pageNum 0..13 and pageCountEstimate 13 for a 14-page document — both are 0-based. The metadata pageCount previously used pageCountEstimate directly, undercounting by one. Use the already-1-based maxPage, falling back to pageCountEstimate+1 only when no blocks were seen. Document the 0-based convention at the normalization site. * fix(extraction): address AliDocMind review — verify/SSRF/config/media (P1+P2) Resolves all P1/P2 findings from the cross-review on #887. P1 (blocking): 1. verifyAliDocMindCredentials now inspects the no-throw response body.code. An OSS-only key returns NoPermission without throwing; previously that was green-lit as "connection successful" then failed at extraction. Only a success/200 or the job-not-found probe code is accepted. Deterministic mocked test added (tests/document/alidocmind-verify.test.ts). 2. verify route trust boundary: managed → server-owned AK/SK + default endpoint only (ignore client values); unmanaged → client creds only, never env fallback, and the client endpoint is SSRF-validated before signing. 3. image fetch hardened: restricted to Aliyun OSS hosts, redirects disallowed, per-image byte cap, image-count cap, bounded concurrency (was unbounded Promise.all over provider-returned URLs). 4. AliDocMind is now selectable in the generation toolbar — availability recognizes the AK/SK pair, not just apiKey. P2 (correctness): 5. Explicit server-config for the AK/SK pair (applyAliDocMindFallback + resolveManagedAliDocMindCredentials); verify and extract now resolve managed/env identically instead of verify-uses-env / extract-rejects. 6. Poll loop checks body.code before status, so a body-level error (e.g. NoPermission) fails fast instead of retrying for the full 15 min. 7. Image page numbers preserved through fetch/filter — no longer hard-coded to page 1, so multi-page image→page association is correct. 8. Empty media extraction (no synopsis/transcript/keyframes) returns 422 PARSE_FAILED instead of HTTP 200 with empty text. 9. Format matrix trimmed to the official contract: images JPG/JPEG/PNG/BMP/GIF (dropped WebP/JP2), media MP4/MKV/AVI/MOV/WMV/MP3/WAV/AAC (dropped M4A). 10. A document-only provider (unpdf/mineru) uploaded with a media file now returns a clear 4xx instead of an opaque 500. P3: credential-verify failures return INVALID_CREDENTIALS 4xx (not INTERNAL_ERROR 500); formatTimestamp emits HH:MM:SS past one hour. Verified with the real key: verify classifies valid→ok and invalid→fail; PDF (24 images, correct page numbers) and MP4 extraction pass end-to-end. * fix(extraction): make AliDocMind selectable + correct analysis label for media Two UI/UX fixes found while manually testing the AliDocMind flow end-to-end: - Persisted-state backfill: add ensureBuiltInPDFProviders so a PDF/document provider added after a user's settings were persisted (AliDocMind) is backfilled into pdfProvidersConfig on rehydrate. Without it the provider never appeared in the store, so it couldn't be selected and never picked up its server-configured flag. Wired into both persist migrate() and merge(), mirroring the existing image/video/web-search backfills. - Analysis step label: getGenerationStepText showed "解析 document 文件" for audio/video (the type map fell through to the literal "document"). Documents keep their precise token (PDF/DOCX/PPTX/XLSX/images); audio/video now use a dedicated, locale-correct string (generation.analyzingMediaMaterial, added to all 8 locales) instead of forcing a format token into the "{{type}} 文件" template. Manually verified end-to-end (real key): PDF, MP4, and MP3 (audio extracted from the sample video) each extract and drive full course generation; a DocMind-restricted AK/SK (OSS-only) now correctly fails verification with NoPermission (400) instead of a false "connection successful". * fix(extraction): address 2nd-round AliDocMind review (managed creds, stream cap, empty verify) Resolves the three follow-up findings on #887: 1. [P1] YAML-managed AliDocMind creds now reach extraction. Both extract paths (document + media) previously cleared the managed AK/SK and relied on an env-only fallback, so a YAML-only deployment verified but failed to extract. They now resolve server-owned creds via the shared resolveManagedAliDocMindCredentials() (env OR YAML), matching the verifier. Regression test added for YAML-only creds with no ALIDOCMIND_* env. 2. [P1] Image download is now size-capped while streaming. Instead of buffering the whole response and checking length afterward, the body is read chunk by chunk with a cumulative byte count that aborts the moment it exceeds the cap — a missing/false Content-Length can no longer exhaust memory. Host allowlist tightened to oss-*.aliyuncs.com. Tests: non-OSS host refusal, no-Content-Length overflow abort, declared-oversize rejection. 3. [P2] Empty verification body no longer counts as success. Removed the codeStr === '' branch from the positive-signal whitelist (a working key always returns the job-not-found business code for the bogus probe id). Regression test added for empty and absent bodies. Rebased onto main (package.json: kept @openmaic/storage + AliCloud deps). Verified with the real key: PDF + MP4 extraction still pass end-to-end. * fix(extraction): merge AliDocMind AK/SK into a baseUrl-configured YAML entry Follow-up to the YAML-managed credential fix. When a YAML `pdf.alidocmind` entry also specifies `baseUrl`, the generic loadEnvSection() (pdf requires a baseUrl) creates the pdf.alidocmind entry copying only apiKey/baseUrl/models/ proxy — never AK/SK. applyAliDocMindFallback() then returned early because the entry already existed, so the provider was "managed" but had no usable credentials, and resolveManagedAliDocMindCredentials() returned undefined (verify + extract both silently lost the creds). Merge the AK/SK into the existing entry instead of returning early. Added a regression test with baseUrl + accessKeyId + accessKeySecret together (the previous test omitted baseUrl, so the generic loader skipped the entry and the bug was masked). * fix(extraction): don't mark AliDocMind managed without AK/SK; align poll code check Two edge cases from a follow-up adversarial pass: - [MED] A YAML `pdf.alidocmind` entry with `baseUrl` but no AK/SK (and no env AK/SK) made the generic loader create the entry → isServerConfigured=true (managed) → but resolveManagedAliDocMindCredentials() returned undefined, so the provider was locked out AND client-entered AK/SK were silently dropped. applyAliDocMindFallback now deletes a credential-less entry so the provider stays UNMANAGED (clients supply their own creds). Regression test added. - [LOW] verify accepted a `code: "success"` body but the extraction poll loop threw on any non-"200" code — a success-shaped status would pass verification then fail extraction. The poll loop now treats "200"/"success" as benign, matching verifyAliDocMindCredentials. --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
c56929510c |
feat(web-search): add SearXNG provider support (#842)
* feat(web-search): add SearXNG provider support Enable self-hosted SearXNG via SEARXNG_BASE_URL, with server-side provider fallback, settings sync, and tests. Co-authored-by: Cursor <cursoragent@cursor.com> * style: fix Prettier formatting in web-search files Restore pnpm check by applying Prettier to imports, headers, and test assertions. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(web-search): keep SearXNG base URLs server-managed only Close the SSRF primitive by ignoring client-supplied SearXNG URLs at the route boundary and rejecting them in resolveSafeClientWebSearchBaseUrl. Add negative route/config tests and an opt-in live smoke test for the JSON API. Co-authored-by: Cursor <cursoragent@cursor.com> * style: fix Prettier formatting for SearXNG web-search changes Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: 徐松 <song.xu@aieshanghai.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
3b63710376 |
feat(ai): add Azure OpenAI provider (#916)
* feat(ai): add Azure OpenAI provider Add deployment-based Azure OpenAI configuration for both client and server-managed setups, normalize Azure portal endpoints, and expose the provider in settings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b0d3a94f-089a-470e-9987-608f1506ca6f * style: format Azure provider type Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b0d3a94f-089a-470e-9987-608f1506ca6f --------- Co-authored-by: MarshellOnMoon <121332425+MarshellOnMoon@users.noreply.github.com> |
||
|
Yanpeng Wang
|
d54e62f5c4 |
feat(token-plan): one-click token-plan setup + deployment usage dashboard (#784)
* feat(usage): add usage normalization, pricing, model-fetch, balance, storage Foundational layer for token-plan usage tracking (cc-switch-modeled): - lib/usage/normalize.ts: AI SDK v6 usage → four-class token shape - lib/usage/pricing.ts + defaults: per-class USD pricing table - lib/server/model-fetch.ts: /models candidate-URL multi-fallback (ported) - lib/usage/balance-providers.ts: built-in balance queries + detection - lib/server/usage-storage.ts: fire-and-forget jsonl logging to data/usage All pure/storage logic covered by vitest (32 tests). * feat(usage): capture token usage at the callLLM/streamLLM chokepoint - callLLM records result.usage before returning - streamLLM wraps onFinish to record totalUsage on stream completion, preserving any caller-supplied onFinish - provider/model derived from the model instance (no route changes) - fs-backed storage imported dynamically; fire-and-forget, never throws - include_usage is already sent by @ai-sdk/openai, so streaming is covered * feat(usage): add probe-models, balance, and usage API routes - POST /api/provider/probe-models: discover chat models via /models with candidate fallback; SSRF-guarded; filters non-chat ids; 401/404 typed - POST /api/provider/balance: built-in balance detection + billing fallback - GET /api/usage: aggregate jsonl by model/day/source with costIncomplete flag Verified e2e against the live MAIC gateway: 16 chat models (6 filtered), balance detected, and a real callLLM writes a costed usage row. * feat(settings): add token-plan preset picker to provider dialog - lib/config/token-plan-presets.ts: data-driven vendor presets (Huawei/MiniMax/ Xiaomi token plans, OpenRouter/SiliconFlow gateways, DeepSeek/GLM/Qwen/Hunyuan/ Doubao direct) with baseURL, protocol, optional modelsUrl, category - add-provider-dialog: '选择厂商/自定义' tabs; picking a preset auto-fills baseURL+protocol+modelsUrl, custom tab unchanged - ProviderSettings.modelsUrl carries the optional /models override - i18n keys added across all 8 locales Verified in-browser: preset picker renders grouped by category. * feat(settings): add Fetch Models button and balance bar to provider panel - 拉取模型: probes /models, merges discovered ids into the model list (dedupe, keeps manual additions), with success/no-endpoint/auth messages - 查询余额: queries /api/provider/balance, renders a balance bar or a 'check console' hint when unsupported - index.tsx: handleModelsFetched merges probe results into provider config - i18n keys across all 8 locales; removed a now-unused eslint-disable Verified in-browser against MAIC gateway: 16 models fetched, balance shown. * feat(settings): add usage dashboard to System Settings - usage-dashboard.tsx: echarts dual-axis daily trend (tokens + cost), totals cards, by-model table, refresh; reads GET /api/usage - mounted at the top of GeneralSettings (系统设置) - honest disclaimer + costIncomplete marker when a model lacks pricing - i18n keys across all 8 locales Verified in-browser: shows 1 request / 31 tokens / $0.0003 from a prior call. * feat(token-plan): multi-modal one-click setup in System Settings - token-plan-presets.ts: presets now declare per-modality targets (llm/image/video/tts/webSearch); MiniMax is the full-set template, others LLM-only — extend by adding entries (at-our-best adaptation) - apply-token-plan.ts: fills one key into every declared modality via injected store setters, isolating per-modality failures (TDD, 4 tests) - token-plan-settings.tsx: new sidebar page — pick plan, enter key, one-click apply lights up adapted modalities (+LLM model probe), shows 'not adapted yet' for the rest, balance bar reused - reverted add-provider dialog to plain custom form (preset picker moved here) - i18n across all 8 locales Verified in-browser: Token Plan page renders, MiniMax shows all 5 modalities, apply/balance UI wired. 859 tests pass, build clean. * feat(token-plan): add custom token plan entry to Token Plan page - Custom card at the bottom of the provider list: pick it to manually enter name + protocol + baseURL (LLM-only), then key + one-click apply via the same applyTokenPlan flow (mirrors cc-switch's custom provider) - effectivePreset unifies preset vs custom for apply/balance/probe - i18n keys (customGroup/customName/customHint) across all 8 locales Verified in-browser: custom card expands the manual form; preset flow intact. * feat(token-plan): reflect persisted config on the Token Plan page Other settings panels read the store directly, so they survive section switches; the Token Plan page only wrote the store, so it looked blank on return. Now it also reads providersConfig: - selecting a preset prefills its saved API key - configured presets show a '已配置/Configured' badge State persistence was never broken (apply writes zustand→localStorage); this fixes the missing read-back so the page reflects it. * fix(settings): isLLMProviderConfigured crashed on providers without models Root cause of 'token plan config disappears': applyTokenPlan writes a new LLM provider with no models yet (probe fills them later). isLLMProviderConfigured did config.models.length unguarded → threw inside setProviderConfig's resolver → the whole set() aborted → key/baseUrl/type never persisted. - guard models in isLLMProviderConfigured (shared validator; affects any provider written without a models array) - seed models:[] in applyTokenPlan's LLM write for a valid initial shape - regression tests: validator no longer throws; apply+probe write keeps apiKey Verified in-browser: DeepSeek persists key and shows 已配置 after section switch. * feat(token-plan): add remove/teardown for a configured token plan Apply had no inverse. Add removeTokenPlan: clears the API key and disables (enabled:false) every modality the plan declared; a custom LLM provider is deleted entirely (removeProvider), built-ins keep the cleared shape. - removeTokenPlan in apply-token-plan.ts (mirrors applyTokenPlan, isolated per-modality, injected setters; 3 tests) - trash button on configured preset cards in the Token Plan page; resets page state if the removed plan was selected - i18n 'remove' across all 8 locales Verified in-browser: removing DeepSeek empties its key and drops the 已配置 badge. * feat(usage): track multimodal usage, drop cost/pricing Reframe usage stats as pure usage (no cost), per user decision: - usage-storage: drop all cost fields; add kind (llm/image/video/tts/asr) + quantity + unit; LLM keeps token counts, others store quantity - delete lib/usage/pricing.ts + pricing-defaults.json + its test - instrument image/video/tts at the server-only API routes (not in the provider dispatch — those files are in the client graph and importing fs-backed usage-storage broke the client bundle) - /api/usage: aggregate by model/day/modality, no cost - dashboard: per-modality usage (tokens/images/seconds/chars), token-only daily trend, no '$'; i18n cost keys replaced with modality/unit keys - backward compatible: legacy rows (no kind, stray cost fields) read as llm 912 tests pass, build clean. * feat(usage): per-modality dashboard layout + softer dark-mode chart - group usage into per-modality sections (LLM/image/video/tts/asr), each table's usage column uses one consistent unit (token/image/sec/char) — no more mixed units in a single column - summary chips per modality with their own unit - trend chart now plots daily REQUESTS (unit-agnostic, works for any modality) instead of LLM-only tokens - theme-aware chart: faint thin line + soft gradient area, muted axis/grid colors via useTheme — fixes the harsh solid stroke in dark mode Verified in-browser (dark): TTS shows '字符', LLM shows 'Token', separate sections; chart no longer has a hard line. * refactor(usage): dedupe model/fetch/usage helpers, parallel balance probe Review cleanups on the token-plan/usage branch: - extract modelInfoFromId() (shared vision heuristic + ModelInfo shape) - extract fetchWithTimeout() shared by model-fetch and balance-providers - extract recordGenerationUsage() to dedupe the image/tts/video routes - queryBalance: fetch billing subscription + usage in parallel - parseOneApiBilling: report quota without remaining when usage endpoint is unavailable, instead of implying zero spend (full balance) - split the two jammed imports in settings/index.tsx * feat(token-plan): drop custom token-plan support Custom token plans (manual baseURL/protocol entry) added complexity for little gain — a one-off provider is better configured directly on the Providers page. Token Plan is now preset-only: - remove custom mode, manual fields, and the custom card from the UI - collapse effectivePreset back to the selected preset - drop the now-dead removeProvider action + custom-id branch in removeTokenPlan - remove orphaned customGroup/customName/customHint i18n keys (8 locales) * feat(token-plan): add Volcengine/Tencent/Bailian plans, drop balance feature Add three vendor token-plan presets (all map to existing built-in LLM providers, so it's data-only — no new adapters): - 火山方舟 Volcengine Ark → doubao, OpenAI /api/v3 - 腾讯 TokenHub Token Plan → tencent-hunyuan, OpenAI /plan/v3 (the plan-specific base; /v1 is the pay-as-you-go gateway) - 阿里百炼 Token Plan → qwen, cross-model plan (Qwen + DeepSeek/Kimi/ GLM/MiniMax) on one key; model list is probed/entered Remove the balance/quota feature entirely — we now track usage, not cost, and every vendor's balance query needs its own cloud AK/SK + signature (Volcengine SigV4 / Tencent TC3 / Aliyun BSS), which the Bearer-key billing-endpoint probe never supported anyway: - delete lib/usage/balance-providers.ts, /api/provider/balance, its test - strip the Check Balance button + balance bar from the token-plan page and the provider config panel - remove the 4 balance i18n keys across all 8 locales - restore an eslint-disable the branch had dropped in provider-config-panel * feat(token-plan): use cloud-brand logos for vendor token plans The three vendor plans are cloud offerings, not single-model products, so icon them with the cloud brand rather than a model logo: - 火山方舟 → volcengine.svg (was doubao.svg) - 腾讯 TokenHub → tencentcloud.svg (was hunyuan.svg) - 阿里百炼 → alibabacloud.svg (was bailian.svg) Logos are the colored brand variants from lobehub/lobe-icons, matching the existing colored-logo style (plain <img>, no dark:invert needed). * feat(token-plan): keep only MiniMax and Volcengine presets Trim the token-plan list to the two we want to ship: MiniMax (full-set template) and 火山方舟 Volcengine Ark. Drop the Tencent/Bailian plans and the OpenRouter/SiliconFlow/DeepSeek/GLM/Qwen entries. - remove the now-unused tencentcloud.svg / alibabacloud.svg logos (volcengine.svg stays; the other logos are still used by the provider registry) - retarget the LLM-only apply test from the deleted deepseek preset to volcengine-ark * feat(token-plan): restore aggregator/third-party presets Previous commit over-trimmed: the intent was to drop only the Tencent and Bailian token plans, not the OpenRouter/SiliconFlow/DeepSeek/GLM/Qwen entries. Bring those back; keep only Tencent/Bailian removed. - token_plan: MiniMax, 火山方舟 Volcengine Ark - aggregator: OpenRouter, SiliconFlow - third_party: DeepSeek, GLM, Qwen Revert the apply test back to the deepseek fixture (restored). tencentcloud.svg / alibabacloud.svg stay deleted (their plans are gone). * fix(token-plan): point Volcengine plan at the Coding Plan endpoint The plan's ark--prefixed API keys authenticate only against /api/coding/v3, not the general /api/v3 endpoint — the latter rejects them with "The API key format is incorrect", so model probing returned nothing. Switch the base URL to https://ark.cn-beijing.volces.com/api/coding/v3. * fix(token-plan): Volcengine is an Agent Plan (Anthropic /api/plan) Per the Ark Agent Plan docs, the ark--prefixed keys authenticate ONLY against the dedicated Anthropic-compatible base https://ark.cn-beijing. volces.com/api/plan ("其他 Base URL 无法在 Agent Plan 中使用"). The general /api/v3 and the Coding Plan /api/coding endpoints both reject the key as "API key format is incorrect", which is why model probing kept returning 0. - baseUrl → https://ark.cn-beijing.volces.com/api/plan/v1 (the /v1 lets the Anthropic SDK land on /api/plan/v1/messages) - apiFormat → anthropic - rename to 火山方舟 Agent Plan Probe still targets /api/plan/v1/models (the path exists); if the Anthropic gateway doesn't return an OpenAI-shaped list, users fall back to typing a model id like ark-code-latest. * fix(token-plan): Volcengine Agent Plan = OpenAI /api/plan/v3 + ark-code-latest Settled after probing the real key and reading cc-switch's approach: - The ark- plan key works on the OpenAI-compatible /api/plan/v3 endpoint (chat/completions returns 200); switch apiFormat back to openai. - The plan exposes NO /models list (every /api/plan/*/models is 404), which is why probing kept returning 0. cc-switch handles this by hardcoding a single ark-code-latest (an auto-routing alias valid on any tier) and does NOT use AK/SK for model listing — so we do the same. - Seed defaultModels: ['ark-code-latest'] only; users add specific ids by hand. Supporting machinery (kept, general-purpose): - applyTokenPlan seeds models from defaultModels instead of wiping to [] - handleApply uses defaultModels and skips the doomed probe when present - drop stray .playwright-mcp/ debug artifacts and gitignore them * style: fix prettier formatting in usage files CI runs prettier on the whole repo (prettier . --check); these four files predate this branch's formatting pass and tripped the check. * feat(token-plan): verify Volcengine Agent Plan's published model set The Agent Plan publishes a fixed model set but exposes no /models endpoint, so carry the documented models as CANDIDATES and verify each on apply: - add verifyModels flag to TokenPlanModalityTarget - new /api/provider/probe-chat-models route: sends a minimal chat request per candidate (OpenAI /chat/completions or Anthropic /messages) in parallel, returns the subset that succeeds; SSRF-guarded, auth-failure short-circuits - handleApply gains a verify branch (before the fixed-defaultModels fast path), falling back to the seeded list if verification fails - Volcengine preset now carries the 12 published Agent Plan text models (doubao-seed-2.0-*/deepseek-v4-*/minimax-m*/glm-5.2/kimi-k2.*) as candidates This auto-prunes retired (docs flag deepseek-v3.2/glm-5.1 as 即将下线) and tier-gated models without code changes. Verified all 12 resolve against a real plan key. * feat(token-plan): wire Volcengine Agent Plan image + video modalities Make the Ark seedream/seedance adapters path-configurable and light up the image/video modalities on the Volcengine plan: - seedream/seedance adapters: resolveArkRoot() uses baseUrl verbatim when it already carries an /api/... path (token plan's /api/plan/v3), else appends the standard /api/v3 — no regression for the pay-as-you-go default host. - applyTokenPlan: image/video branches inject a modality's defaultModels as customModels and set them as the active provider+model, so generation works out of the box. New optional setImageProvider/ModelId + setVideoProvider/ ModelId actions (UI passes the store setters; tests omit them). - Volcengine preset declares image (doubao-seedream-5.0-lite, verified 200 on /api/plan/v3/images/generations) and video (doubao-seedance-2.0/1.5-pro — Medium+ tiers only; lower tiers reject at call time, no code change needed to upgrade). Applying the plan overwrites the shared seedream/seedance slot with the plan config (same overwrite model as LLM); switching back to pay-as-you-go is a manual edit or plan removal. Verified image end-to-end with a real plan key. * feat(token-plan): verify image/video models on apply, disable unsupported tiers The Volcengine plan lit up video optimistically, but lower tiers (Small) don't include video — so using it 404'd with UnsupportedModel. Probe media models on apply and only keep what the tier actually supports: - generalize /api/provider/probe-chat-models with a `kind` (chat|image|video): image hits /images/generations, video hits /contents/generations/tasks with empty content. The model-support check (404 UnsupportedModel) runs before any billable work, so probing never starts a real image/video job; for media, "supported" = any non-404 response. - handleApply: after lighting up image/video, probe each verifyModels modality; prune to the verified model set + re-select a working model, or disable the modality entirely if none pass (no false "available"). - Volcengine preset: image/video targets gain verifyModels: true. - add settings.tokenPlan.tierUnsupported across 8 locales. Verified with a real Small-tier key: image (seedream-5.0-lite) passes and is kept; video (seedance-2.0/1.5-pro) 404s and is disabled. * feat(web-search): add Doubao (豆包搜索) provider Doubao Search (Custom 版) over its REST endpoint POST open.feedcoopapi.com/search_api/web_search with Bearer auth — the same endpoint the askecho-search-infinity MCP server wraps, so the Volcengine Agent Plan key authenticates directly. Mirrors the MiniMax adapter: maps Result.WebResults to WebSearchSource (prefers Summary, the query-relevant excerpt, over Snippet for LLM use) and surfaces errors from ResponseMetadata.Error. - register 'doubao' in WebSearchProviderId + WEB_SEARCH_PROVIDERS - searchWithDoubao adapter, searchWeb dispatch, store default config - SSRF allowlist entry for the search host * feat(audio): support Agent Plan single-key auth for Doubao TTS generateDoubaoTTS now picks auth + endpoint from the key shape, since Volcengine exposes Seed-TTS as two products with separate credentials (verified: a plan key 401s on the normal endpoint, and the plan endpoint rejects appId-style auth): - single key (no colon) -> X-Api-Key, for the Agent Plan /plan endpoint - appId:accessKey -> X-Api-App-Id + X-Api-Access-Key (unchanged) A malformed pair (empty half) fails clearly instead of sending an empty header. Reuses the existing NDJSON/base64-mp3 parsing and voice list. * fix(media): map MiniMax video 720p to its real 768P tier normalizeVideoOptions defaults minimax-video to '720p' (the first supported resolution), but Hailuo 2.3 only accepts 768P/1080P and rejects 720P with '2013 ... does not support resolution 720P'. MiniMax's mid tier is 768P, not 720P (the adapter already falls back to 768P, as does the connectivity test), so map the shared enum's '720p' to 768P. Regression tests lock the mapping. * feat(token-plan): add web search + TTS to Volcengine Agent Plan, widen image tiers Extend the volcengine-ark preset now that the adapters exist: - webSearch -> doubao (own host open.feedcoopapi.com, not the ark endpoint) - tts -> doubao-tts on the /api/plan/tts endpoint (single-key auth) - image defaultModels widened to a best-first Seedream 5.0/4.5/4.0 list so a higher tier keeps the strongest model while verifyModels prunes the rest; video keeps the 2.0 + 1.5-pro candidates Comments record the verified host/auth quirks of each modality. * feat(token-plan): show result panel only after probing, with two clear states Addresses review feedback that a green check implied generation works when it only meant 'configured'. The panel now renders after probing finishes (gated on results && !applying) so it reflects the final set, and uses two states: green when the modality is configured/usable, muted when a live probe proved it unavailable (e.g. video on a tier without it). * feat(token-plan): scope presets to true multi-modal token plans Drop the single-modality LLM presets (OpenRouter, SiliconFlow, DeepSeek, GLM, Qwen) from Token Plan. A token plan's defining trait is one key spanning many modalities; those entries are ordinary LLM API providers already covered by the add-provider flow, and listing them here muddied the 'one key, every modality' promise. Only MiniMax and the Volcengine Ark Agent Plan remain. The UI already hides categories with no entries. apply-token-plan's LLM-only test now uses a local fixture instead of the removed deepseek preset. * feat(token-plan): progressive reveal of probe results on apply The result panel previously rendered all at once after probing finished, reading as dead air during the model probe. Now rows appear immediately on Apply: modalities with a live probe in flight show a spinner ('pending') and resolve to lit/failed independently as each probe returns, while non-probe modalities show lit right away. Probes run in parallel (Promise.all) instead of sequentially. A row only turns green once its own probe confirms, so this reveals structure + live progress without a premature green — complementing the earlier 'render only after probing' intent rather than reverting it. Per review feedback from @wyuc on #784. * fix(token-plan): enrich seeded models with built-in thinking capability Token Plan built ModelInfo objects from probed ids via modelInfoFromId(), filling only streaming/tools/vision — so a model that supports configurable thinking lost capabilities.thinking and InlineThinkingControl was hidden. modelInfoFromId now takes an optional providerId and overlays the catalog thinking capability for that (provider, model) pair; applyTokenPlan does the same for its synchronously-seeded list. Added the Ark Agent Plan's dotted aliases to the metadata table: - native Doubao Seed 2.0 family (doubao-seed-2.0-pro/code/lite/mini) - cross-vendor models the plan serves through its OpenAI-compatible endpoint (deepseek-v4-pro/flash, glm-5.2, kimi-k2.7-code/k2.6, minimax-m3/m2.7, ark-code-latest) All verified against a live plan key: each accepts the gateway's unified reasoning_effort field (low/medium/high) and actually reasons. They share the doubao effort adapter, which disables via 'minimal' (not 'none') — matching what the plan endpoint accepts (it rejects reasoning_effort:'none'). Addresses review point #1 from @wyuc on #784. * Improve token plan capability setup UI * fix token plan setup flow * chore: prettier format tts-providers.ts |
||
|
jackefnandYanpeng Wang
|
3b7a0ca0c0 |
feat(document): add multi-format course material upload (#741)
* feat(document): add multi-format course material upload * fix(document): support office uploads with mineru cloud * fix(document): handle mineru cloud office edge cases * fix(document): align mineru cloud artifact metadata --------- Co-authored-by: Yanpeng Wang <yanpg.wang@gmail.com> |
||
|
|
cec3c6aaea |
Refactor widget actions into the scene action pipeline (#796)
* refactor: align widget actions with scene pipeline * feat(edit): migrate legacy teacherActions off interactive content on load The widget-actions pipeline no longer persists a teacherActions authoring layer; playback reads only scene.actions. Add migrateInteractiveContent to strip the dead field from existing documents as they load through migrateScene (wired into all store/classroom load paths). Satisfies the #794 forward-migration acceptance item. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(generation): default widget_setState.state to {} when the LLM omits it The removed TeacherAction->Action converter always defaulted state to {}. Restore that guard in the structured-output parser so an action missing state can't forward state: undefined into the widget iframe, where SET_WIDGET_STATE handlers dereference it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(edit): also strip nested widgetConfig.teacherActions in migration Legacy WidgetConfig variants each declared their own teacherActions, so existing documents can carry the dead field nested inside content.widgetConfig as well as at the top level. Extend migrateInteractiveContent to drop both, preserving all other widgetConfig fields. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(prompts): document stable widget-action selectors for diagram/game/viz3d The interactive-actions prompt documented stable target selectors only for procedural-skill and simulation widgets. Add the canonical selectors the diagram / game / visualization3d content prompts actually emit (node IDs via revealOrder; #zoom-in-btn / #canvas-container / #speed-slider / ...; #start-btn), plus a rule to use widget_setState or a speech-only beat instead of guessing a target when no stable selector is known (e.g. free-form code widgets). Hardens against widget actions silently no-oping on non-existent selectors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
Yizuki_Ame
|
243e9f3a48 |
feat: auto-retry transient failures during scene generation (#788)
Adds automatic retry with exponential backoff for scene generation (content/actions/TTS): shared withGenerationRetry helper at the fetch boundary, smart error classification (retries 429/5xx/timeouts/network, fails fast on 401/403/404), AI SDK RetryError unwrapping, abort-aware backoff, per-route thinking config preserved, and upstream HTTP status surfaced for client retry classification. Fixes #755. |
||
|
wyuc
|
1d1ce80e04 |
feat(maic-agent): editor-agent line (v0) — Pro-mode "Edit with AI" (#777)
Promote the editor-agent line to main: read-then-act editor agent (read_scene_content, regenerate_scene, regenerate_scene_actions, edit_interactive_html), a routable maic-agent model stage, the agent sidebar UI, interactive-scene runtime-error capture, and the timeline/script editor. Cross-reviewed and smoke-verified; see #777. |
||
|
wyucandClaude Opus 4.8
|
25cf58d19f |
feat(model): optional per-stage LLM model routing (#745) (#747)
* feat(model): optional per-stage LLM model routing (#745) Add a config-only stage -> model map consulted during model resolution, falling back to DEFAULT_MODEL when unset (zero behavior change unless opted in). - lib/server/model-routes.ts: parse/validate/cache MODEL_ROUTES env JSON; LLM_STAGES registry of routable stages; getStageModel(stage). - resolveModel: resolution order x-model > stage route > DEFAULT_MODEL > builtin; thread optional `stage` through resolveModelFromHeaders/FromRequest. - Wire each route's resolveModel call site to its stage; classroom-generation resolves generate-classroom and web-search-query-rewrite independently. - .env.example: document MODEL_ROUTES; unit tests for both modules. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): address cross-review findings for per-stage routing (#745) - classroom-generation: resolve the web-search-query-rewrite model lazily, only when explicitly routed and inside the web-search branch, with try/catch fallback to the classroom model. Fixes a regression where a misconfigured optional route (keyless/invalid provider) threw and aborted ALL classroom generation even with web search disabled; also avoids wasted resolution on the common no-search path and preserves classroom-model inheritance when unrouted. - resolve-model: type the stage param as LlmStage across resolveModel/ resolveModelFromHeaders/resolveModelFromRequest so a mistyped stage literal is a compile error instead of silently falling through to DEFAULT_MODEL. - model-routes: simplify the cache to a process singleton (matching provider-config), dropping the raw-string-keyed cache and redundant STAGE_SET. - .env.example: clarify x-model precedence and that a route to an unconfigured provider fails at request time (no startup validation), like a bad DEFAULT_MODEL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): stage route takes precedence over client x-model (#745) Cross-review (codex P2 + Claude tracer) found that the browser UI always sends its saved model as x-model, so the previous x-model > stage order made MODEL_ROUTES inert for normal UI traffic on exactly the heavy stages the RFC targets (scene-content, quiz, pbl-chat, chat). Flip resolution to stage route > x-model > DEFAULT_MODEL: a configured route is the operator's deliberate per-stage choice and wins, while unrouted stages still honor the client's x-model. Update docs and tests accordingly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(model): prettier-format changed files for #745 CI Run the repo formatter (pnpm check / prettier) on the changed routes, the new model-routes module, and the new tests so `prettier . --check` (CI) passes; also correct a stale precedence comment in the resolve-model test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): isolate routed model from client connection params (#745) When a stage route overrides the client's x-model and points at a different provider, the client-sent apiKey/baseUrl/providerType (for the client's model) must not bleed onto the routed provider — otherwise a routed Anthropic model would be constructed with the client's OpenAI providerType/key and fail. A routed model now resolves its connection params purely from server config, as if no x-model was sent. Unrouted stages still honor the client params. Adds tests covering both the drop (routed) and keep (unrouted) cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(model): correct graceful-degradation comment in classroom (#745) resolveModel does not throw on a missing key; clarify that a keyless route surfaces later in callLLM (degraded by the outer try/catch), while only a resolution-time failure is caught at the rewrite re-resolve. Comment-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(model): route scene-content per scene type (#745) Extend MODEL_ROUTES with composite keys scene-content:<type> for the four core scene types (slide/quiz/interactive/pbl). getStageModel now resolves composite `a:b` stages most-specific-first, trimming `:` segments, so scene-content:<type> falls back to the base scene-content route, then x-model, then DEFAULT_MODEL. The scene-content route derives its stage from outline.type. Backward compatible: with no composite key configured, behavior is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(model): per-stage thinking effort + drop hardcoded gpt fallback (#745) MODEL_ROUTES values may now be {model, effort} (not just a model string) to pin the thinking effort per stage. Arbitration mirrors model routing: a routed stage with effort set wins over the client's thinking (effort "none" disables); routed with no effort uses the model's default and drops client thinking; unrouted stages keep the client thinking. resolveModel is the single arbiter — body thinking is threaded in via resolveModelFromRequest. classroom-generation passes the resolved thinking into its callLLM calls so generate-classroom and web-search-query-rewrite honor route effort too. Also: removed the hardcoded `|| 'gpt-5.4-mini'` fallback in resolveModel — if no model resolves (no route / x-model / DEFAULT_MODEL) it now throws instead of silently picking a vendor default. model-routes: getStageRoute returns {model, effort?}; getStageModel delegates to it. Docs + tests updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(model): route value carries full ThinkingConfig (#745) Per-stage routing accepts the full unified ThinkingConfig in the route value ({model, thinking:{mode,effort,level,enabled,budgetTokens,excludeReasoningOutput}}) instead of just an effort string. The route's thinking is passed through resolveModel and normalized per the model's capability by callLLM, so budgetTokens (qwen), level (Gemini), enabled/mode, etc. all work. (qwen3.7-plus/max thinking capability now comes from upstream #753, so this no longer registers them.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): honor routed thinking for chat-adapter; doc fixes (#745) Cross-review (codex P2 + Claude) found chat-adapter was the only routable stage that ignored the route's thinking: it took the routed model but built its own thinkingConfig from the request body, so operator-pinned thinking never applied and client thinking still leaked onto a routed model. Now pass the client thinking into resolveModel and use the resolved thinkingConfig (route-pinned for a routed stage, client otherwise; defaults to disabled for low-latency chat). Also: clarify DEFAULT_MODEL is now required for server-side stages (the hardcoded gpt-5.4-mini fallback was removed → resolveModel throws if nothing resolves), and fix a stale {model,effort}→{model,thinking} doc comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
0989bd5d15 |
fix(proxy): bypass proxy for loopback hosts and honor NO_PROXY (#718)
proxyFetch routed every request through the configured HTTP(S) proxy, including loopback targets. A deployment with HTTP_PROXY set would send a http://localhost:8888 SearXNG (or any self-hosted service) request to the external proxy, where localhost resolves to the proxy machine and the request fails. - Always bypass loopback hosts (localhost, *.localhost, 127.0.0.0/8, ::1). - Honor the standard no_proxy / NO_PROXY env var: comma-separated hosts, `*` wildcard, optional `:port` (with protocol-default ports), and curl-style domain-suffix matching (`example.com` matches `api.example.com`; a leading dot is equivalent). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
3ccd5da6da |
feat(generation): opt-in parallel scene-content generation (#660)
* feat(generation): opt-in parallel scene-content generation Scene generation was strictly serial — content -> actions -> TTS, one outline at a time — so an N-scene classroom paid N x (content + actions + TTS) of wall-clock latency, which dominates the post-outline wait. Add an opt-in hybrid two-phase path (#572): - Phase 1 fetches scene *content* concurrently (bounded). Content is the only per-scene step independent of cross-scene state, so it is safe to parallelise; a content failure marks just that outline and does not pause the batch. - Phase 2 keeps the existing in-order actions + TTS loop, so previousSpeeches threading, ordered addScene, the abort/epoch guards, and the pause-on-failure UX are all unchanged. Gated by a server-side PARALLEL_SCENE_CONCURRENCY (default 0 = off, clamped to 10), surfaced to the client through the existing /api/server-providers response. With it unset the content map is null and the loop is byte-for-byte the original serial path, so out-of-box behaviour is unchanged. The bounded worker pool is extracted to lib/utils/concurrency.ts (mapWithConcurrency, with an early-stop hook for abort/epoch) and unit-tested; getParallelSceneConcurrency env parsing/clamping is tested too. Closes #572 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(generation): pipeline content fetches instead of a barrier Address @wyuc's review on #660. The previous version awaited the whole bounded content pool before Phase 2, so for N > C scenes the user saw the first scene, then a stall on a single page for the entire content phase, then a burst — perceived as worse than serial despite the wall-clock win (the gap scales with ceil((N-1)/C)). Pipeline instead: lazyBoundedMap starts the content fetches (<= C in flight) and returns one promise per outline immediately; the serial loop awaits them in order, so each resolves as soon as its own content is ready (usually already done, hidden behind the previous scene's actions/TTS). The first scene now paints after content(1)+actions(1)+TTS(1) — same as serial — with the full concurrency benefit and no stall. Also from the review: - a content fetch can no longer take its siblings down: an unexpected throw is caught and returned as a failure result, routed through the same mark-failed path as the serial loop (symmetric); - drop the cosmetic Phase-1 setCurrentGeneratingOrder — the loop already sets it per outline; - note the intentional belt-and-suspenders clamp; - concurrency.test.ts asserts <= limit (not == limit), and gains lazyBoundedMap tests including the no-barrier property. mapWithConcurrency is kept as a thin await-all wrapper over lazyBoundedMap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
d7c0d80afe |
feat(tts): per-agent auto-voice quality + register-once timbre stability (#670) (#672)
* feat(tts): voxcpm voice-design types + deterministic voice id helpers (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): emit + persist per-agent voiceDesign descriptor (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): voxcpm voice registration backend client (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): voxcpm-voice ensure/register endpoint (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): client auto-voice registration + reference-clip cache (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): reference registered voice id in vLLM-Omni speech (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): thread voiceDesign + backend through tts call sites (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): match vLLM-Omni voice registration contract (#670) Live e2e against the real backend revealed the assumed multipart contract was wrong: POST /v1/audio/voices needs name + consent + audio_sample (not voice_id/file), and there is no per-name GET (405) — existence must list /v1/audio/voices and check membership. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(tts): make auto-voice register-once pattern provider-neutral (#670) The descriptor + register-once/reference-by-id pattern is not VoxCPM-specific. De-couple it into a provider-neutral seam so other registration-capable providers (ElevenLabs/MiniMax/Doubao voice cloning, …) can plug in: - lib/audio/voice-design.ts: VoiceDesign + buildVoiceDesignPrompt / normalizeVoiceDesign / getDeterministicVoiceId (neutral 'auto-<hash>' id, namespaced by providerId). - lib/audio/voice-registration.ts: VoiceRegistrationAdapter interface + providerId->adapter registry + supportsVoiceRegistration. - lib/audio/voice-registration-client.ts: neutral ensureRegisteredVoice() + IndexedDB clip cache. - app/api/generate/voice (replaces .../voxcpm-voice): dispatches by providerId. - voxcpm-registration.ts becomes the VoxCPM adapter (sole registered provider). - AgentConfig.voiceDesign + DB cache table renamed neutral. Behavior-preserving; VoxCPM-specific bits (inline (prompt)text, backend kinds, capability gate) stay in the voxcpm modules. Full suite green (674). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): build the voice clip Blob from an inline Uint8Array (#670) A factored-out helper returning Uint8Array widened to Uint8Array<ArrayBufferLike>, which next build (stricter than bare tsc) rejects as a BlobPart. Inline the buffer like the rest of the repo does. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): managed TTS providers resolve model server-side (#670) A server-managed TTS provider's model was still client-driven, but the managed-provider settings UI hides the model field — so a backend whose model id isn't the client default (e.g. VoxCPM/vLLM-Omni serving the full model path) always 500'd. resolveTTSModel() makes the model authoritative from server config (${PREFIX}_MODELS, first entry) for managed providers, like key/baseUrl; unmanaged/unconfigured providers keep the client model unchanged. Used by the tts and voice routes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): single agent-voice resolver + stable teacher narration voice (#670) Route all TTS voice resolution (narration, discussion, preview) through one resolveAgentVoiceOptions(agent, ...) that reads the agent profile and registers + references the voice by id. Eagerly warm up generated agents' voices on save. Critically fixes the teacher narration drifting (male/female jumps): the registry is always seeded with DEFAULT_AGENTS, so the old narration lookup find(role==='teacher') returned the default teacher (no voiceDesign) instead of the generated one — so narration never registered a voice and fell back to the inline prompt. pickNarratorAgent() now prefers the teacher carrying a voiceDesign. Also: drop language from the deterministic voice id (descriptor already encodes it) so narration (directive) and discussion (locale) resolve the same id; log the effective registeredVoiceId. Regression tests for pickNarratorAgent. Verified e2e: agent-profiles -> 1 voice registration -> 16 narration TTS all referencing the same registeredVoiceId; voice present on the backend. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): fall back to persona as the voice seed for agents without a voiceDesign (#670) Preset/default agents carry no LLM voiceDesign. Rather than add a static field, derive the bootstrap descriptor from the agent's persona when voiceDesign is absent, so they still register one stable reference voice (stable-but-generic; persona is not a vocal spec). Generated agents keep their LLM voiceDesign. Also hardens replay: stage snapshots drop voiceDesign but keep persona. Unit-tested: resolveAgentVoiceOptions uses real voiceDesign when present, persona otherwise. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): match Auto Voice by its localized label in the agent voice picker (#670) The picker filtered on voice.name ('Auto Voice'), but Auto Voice is shown via its localized label (自动音色). Searching '自动' returned '没有匹配音色'. Match the localized label for the Auto Voice option so it's findable in any language. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): address code-review round 1 (#670) - strip parentheses in the voice-design prompt so a paren in the descriptor/ persona can't corrupt the (prompt)text bootstrap delimiter - gate POST /api/generate/voice on isServerTTSProviderDisabled (#665): a force-disabled provider was off for the tts route but not this sibling - check voiceExists before re-registering a client-cached clip, so a cached voice that's still live isn't needlessly re-uploaded every session - dedup concurrent ensureRegisteredVoice calls via an in-flight promise map (eager warm-up + first utterance no longer double bootstrap/register) - warm up only the narrator (teacher), not every generated agent, to avoid synthesizing voices at save time for agents that may never speak - collapse the duplicated vLLM-Omni speech payload into one object Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(tts): prettier formatting (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): key auto-voice memo by (voiceId, backend) so a base-URL change re-registers (#670) Addresses review (cosarah): registeredThisSession/inFlight were keyed by voiceId alone, so switching the VoxCPM base URL mid-session made ensureRegisteredVoice short-circuit and return an id registered only on the old backend — TTS then sent that stale registeredVoiceId and skipped the inline fallback, failing with voice-not-found on the new backend. Memo key now includes the base URL; the IndexedDB clip cache stays keyed by voiceId (the reference clip is backend-independent and reused to re-register elsewhere). Regression test added. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): include API key in the auto-voice memo key (#670) Addresses review (cosarah, round 2): memoKeyFor was keyed by (voiceId, baseUrl) but not the API key. Registration/existence checks and speech calls are auth-scoped, so switching account/key on the same base URL could reuse a registeredVoiceId from the old credentials and skip re-validation. Memo key now includes the API key (in-memory only, never persisted/logged). Regression test covers the key-switch case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
|
3e1e9f083b |
feat(tts): unify provider-enablement model, default browser-native off (#665) (#666)
A single TTS provider-enablement model honored everywhere (picker,
auto-assignment, fallback, lecture/playback, server).
Model
- lib/audio/provider-enablement.ts: isTTSProviderConfigured /
isTTSProviderEnabled / listEnabledTTSProviderIds. Availability drops the
registry defaultBaseUrl fallback (fixes Lemonade's always-shown voices),
honors the per-provider `enabled` flag, server-disable overrides (server
precedence).
- voice-resolver: getEnabledProvidersWithVoices (enabled server/custom
providers) and getSelectableProvidersWithVoices (the ONE client source of
truth = enabled providers + opt-in browser-native when the browser exposes
voices). resolveAgentVoice picks deterministically among the given providers,
honors a voiceConfig only while its provider is present.
- Picker + discussion build their list from getSelectableProvidersWithVoices, so
the AgentBar and the live discussion agree on selectable voices (fixes
student agents going silent in auto mode). Students resolve via
resolveAgentVoice; the teacher uses the global lecture voice verbatim (same
voice the pre-generated lecture sends), so lecture and discussion never
diverge.
- Voice assignment vs fallback: an agent's voice is bound at generation (the
agent-profile LLM picks from availableVoices; preset uses the user's pick).
The fallback for a missing/invalid/unavailable voice is applied only at the
live TTS calls, not baked into the saved agent — so it reflects current
provider availability.
Defaults / opt-in
- Browser-native: first-class, default OFF, opt-in. Built-in providers default
enabled:true (only surface once configured).
- TTS master toggle defaults OFF; first server-sync auto-enables it when a
provider is configured (mirrors image/video).
- Persist v3->v4 migration normalizes the never-user-editable flags.
Server/admin disable (server precedence): server-providers.yml `enabled:false`
and env `TTS_<P>_ENABLED=false` (incl. browser-native). Surfaced as a per-provider
`disabled` flag on /api/server-providers; merged as serverDisabled; enforced at
the /api/generate/tts trust boundary, classroom media generation, and /api/health.
UI
- Settings TTS panel: per-provider enable toggle, disabled (with a hint) when the
provider is unavailable, locked off (with a note) when server-disabled.
- AgentBar voice pills render disabled with a "No voice" placeholder when no
provider is enabled. The toolbar media-popover TTS tab keeps its existing
on/off toggle (no picker/CTA).
Tests: predicates, resolver determinism + unified selectable list, server
force-disable, store merge + migration + TTS auto-enable, i18n parity. Verified
in a real browser (Playwright): settings toggle states + a full Gemini classroom
generation whose TTS requests used the picked voice.
Found on main @
|
||
|
|
d2b6db60a4 |
fix: point SSRF local-network rejection at ALLOW_LOCAL_NETWORKS escape hatch (#667)
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyuc
|
2165e55bac | feat: add MiniMax web search provider (#634) | ||
|
|
35d36905d1 |
Make server-configured providers admin-managed, not client-overridable (#620) (#624)
* refactor(providers): make server-configured providers admin-managed, not client-overridable A server-configured provider is now authoritative: the operator owns its API key and base URL, and the client cannot override either. This collapses the tri-state (server key / client base URL / client key) that was the root of a recurring "missing API key" bug class — when the settings UI echoed a base URL with an empty key, routes that treated "any base URL ⇒ fully client-configured" dropped the server key. #533 patched that per-route via canUseServerApiKeyForBaseUrl; this removes the guard entirely by fixing the design. Server: - resolve*ApiKey/BaseUrl ignore client values when the provider is server- configured; add isServerConfiguredProvider(section, id); drop canUseServerApiKeyForBaseUrl + normalizeProviderBaseUrl. - getServer*Providers() no longer return base URLs — only allowed models (LLM/image) and the managed flag (presence) — so an internal gateway/proxy endpoint is never exposed to the client. - routes (chat/image/video/tts/asr/pdf + verify-*) resolve managed providers purely server-side; SSRF validation now applies only to unmanaged (client) base URLs. Client: - drop serverBaseUrl from ProviderSettings, the settings store, request-building (TTS base URL no longer echoes the server URL) and voice resolution. - settings UI renders managed providers read-only: the key/base-URL override inputs and region toggles are hidden, leaving the "managed by server" notice. Tradeoff: overriding a server provider's key with your own for the same provider is removed; add a separate custom provider instead. Closes #620. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers): refresh managed-provider copy, drop dead audio-settings, add #620 e2e - serverConfiguredNotice now says the provider is managed and not editable here, pointing users to add a separate provider for their own credentials (7 locales). - delete components/settings/audio-settings.tsx — unreferenced dead code. - add e2e/tests/managed-provider-620.spec.ts: managed provider hides the key/base-URL inputs; unmanaged provider keeps them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(providers): lock the model catalog for managed providers with a pinned list When the operator pins an allowed model list (MODELS env/yaml ⇒ serverModels), the model catalog is admin-managed too: the LLM provider panel shows the pinned models read-only with a "Server" badge and no add / reset / edit / delete affordances. Without a pinned list the server manages only credentials and the user keeps curating models. This removes the prior inconsistency where the settings panel let you add models to a pinned managed provider that the generation toolbar then filtered out. e2e: the managed-openai case now also asserts the catalog is locked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers): ignore (not reject) client overrides for managed web-search; clear legacy serverBaseUrl Addresses review feedback on #624: 1. Managed web-search providers now ignore a client-sent key/baseUrl instead of validating and rejecting it. A server-configured provider is admin-owned, so the server config is authoritative and a stale client base URL is dropped rather than 400-ing the request. The allowlist validation still runs for unmanaged providers, where the client base URL is actually used. 2. Drop the removed `serverBaseUrl` field from persisted localStorage on rehydrate (settings version 2 -> 3, stripped in both migrate and merge), so old clients don't keep a stale server URL in client state. Tests: web-search route test split into unmanaged-rejects-allowlist and managed-ignores-client-baseUrl; settings rehydrate test asserts a legacy serverBaseUrl is stripped. Full suite 381 pass; e2e 2 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
Yizuki_Ame
|
47cc2a510c |
feat: add Brave Search and Baidu Search integration
Add Brave Search and Baidu Search providers with Baidu sub-source support, provider configuration updates, tests, and docs links. |
||
|
wyuc
|
2dff6d10be |
feat: add generated video manifest refs (#540)
* feat: add generated video manifest refs * style: format video manifest changes * fix: address video manifest review feedback |
||
|
HelckandCodex
|
58acad55d8 |
feat: support OpenAI image env fallback (#510)
Summary: - Enable the OpenAI image provider from standard OpenAI server env vars when image-specific env vars are absent. - Add provider-config coverage for resolving the fallback API key and base URL. Rationale: - Allows gpt-image-2 image generation to work with common OpenAI proxy configuration. Tests: - pnpm vitest run tests/media/openai-image-adapter.test.ts tests/server/provider-config.test.ts - pnpm prettier lib/server/provider-config.ts tests/server/provider-config.test.ts --check - pnpm lint -- lib/server/provider-config.ts tests/server/provider-config.test.ts Co-authored-by: Codex <codex@openai.com> |