mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-18 05:40:25 +08:00
@openmaic/generation@0.3.1
88
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 wyucandClaude Fable 5
|
4f00cf7ad6 |
feat(materials): material library manifest and id-addressed generation images (#1153 part 2) (#1168)
* feat(materials): material library manifest and id-addressed generation images (#1153 part 2) * fix(materials): library-owned references and failure-path parity (#1153 part 2) * fix(generation): resolve-then-slice vision images and close allocation leaks (#1153 part 2) * fix(generation): bound vision resolution and make library writes leak-safe (#1153 part 2) Bound the resolve-with-refill phase with an aggregate 15s budget raced against every probe plus a 3-consecutive-failure fuse — either stop strips unresolved and unprobed ids from the mapping and proceeds (text-only at worst), never failing the request. Restore the original interleaved text ordering for non-vision runs with a mapping present. Guard the library's failed-entry-write release with an ambiguous-commit re-read: release only when the entry provably does not point at the new id, keep it when the write actually landed, and prefer a recoverable leak over a dangling entry when the re-read fails. Drop the unused package export. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(generation): bump @openmaic/generation to 0.3.1 (#1153 part 2) The package's inputs changed (partitionImagesForVision, resolvedVisionImages option, non-vision ordering fix); the merge-time gate requires a version bump. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
wyuc
|
e3fd494658 |
feat(tts): add Qwen TTS voice cloning (#1160)
* feat(tts): add Qwen TTS voice cloning via the registration adapter * feat(settings): add Qwen TTS clone-voice manager * fix(tts): harden Qwen voice cloning integration * fix(tts): enforce voice-bound Qwen synthesis * fix(tts): close Qwen voice clone tail issues * fix(tts): close final voice clone review issues * fix(agents): harden voice token parsing * fix(settings): hide the voice-clone model from manual selection * fix(agents): pin the narrator voice to the user's selected voice * fix(agents): keep narrator fallback alive when the pinned voice is unusable * fix(i18n): correct the narration fallback notice copy * fix(tts): bound the narrator voice retry to a single fallback hop * fix(tts): tolerate transient vendor errors in the voice existence precheck |
||
|
LING DUANandwyuc
|
a0dccbe78d |
feat(pi): add Native Child additive whiteboard tools (#1156)
* feat(pi): add Native Child additive whiteboard tools * chore(renderer): bump package version to 0.1.3 * fix(pi): reject invisible whiteboard mutations * fix(pi): close native whiteboard review blockers --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyucandClaude Fable 5
|
5a3e81c4d9 |
feat(document): ingest uploaded sources into the asset pool and extract by id (#1153 part 0) (#1154)
* feat(document): declare a version on every extractor provider (#1153 part 0) * feat(api): accept asset-id input in the extract-document route (#1153 part 0) * feat(upload): ingest selected course materials into the asset pool (#1153 part 0) * fix(document): harden asset-id extraction inputs and close upload ingest leaks (#1153 part 0) * fix(document): scope extraction fallback and harden the asset-id path (#1153 part 0) - G1: byte fallback only on pre-extraction failures; PARSE_FAILED (422/500) is surfaced without re-running the paid extractor - G2: freeze the course-material set for the duration of generate-prep (guarded add/remove + disabled toolbar affordances, session built from the click-time snapshot, belt-and-braces drain) - G3: bound the pre-generation ingest await (~15s); timed-out ingests go byte-path and their late-resolving ids are released - G4: reject non-object JSON bodies (null/array/string/number) as 400 - G5: JSON-path provider pre-validation with generic messages, generic JSON-path PARSE_FAILED/422 responses, control-char-free log values - G6: size cap enforced via identify() before resolve() materializes bytes; too_large maps to the existing 413 * fix(document): mirror provider-hint semantics on the asset-id form and freeze prep-time controls (#1153 part 0) * fix(document): keep the asset-id form's responses free of caller input (#1153 part 0) Address review findings: answer the extractor-selection failure with a generic static message on the asset-id form (the registry's interpolated message carries the caller's MIME type; multipart keeps it verbatim, both sides now pinned by tests), dedupe same-batch material additions by content fingerprint inside the pure updater, document the network-error retry exception in shouldRetryWithByteUpload's docstring, and fix a test name asserting 500 but named 400. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
LING DUAN
|
2cbe4ae293 |
feat(pi): add Native Child whiteboard tools (#1152)
* feat(pi): add native whiteboard tools * style(pi): format native whiteboard tests * fix(pi): clarify whiteboard CAS handoff * fix(whiteboard): reconcile runtime projection on open |
||
|
LING DUANandwyuc
|
59adbef03b |
Simplify Pi agent tool ownership and completion (#1148)
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
LING DUANandwyuc
|
0d308ed78d |
feat(pi): add Native Child Web Search (#1133)
* feat(pi): add native child web search * fix(pi): align native web search with provider system * fix(pi): harden native web search configuration and cancellation * fix(pi): guide Native Child web search selection --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
LING DUANandwyuc
|
cdf949a017 |
feat(pi): add native child runtime with scoped Spotlight (#1111)
* feat(pi): add native child runtime with scoped Spotlight * fix(pi): address native child review feedback * fix(pi): suppress prefaced structured child output * fix(pi): restore native child text streaming --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyucand杨慎
|
0a10af11cf |
feat(storage): opt-in indirect asset byte egress (#1007) (#1100)
* feat(storage): add opt-in indirect asset byte egress A deployment can now opt an asset byte GET into a 302 to a short-lived signed URL when the byte layer can sign. It is off by default, and byte-for-byte unchanged when off. The byte layer gains an optional signReadUrl capability; S3AssetByteStore implements it through the new optional @aws-sdk/s3-request-presigner peer, resolved lazily exactly like the client SDK, with a signer seam mirroring the commands seam so tests can bind doubles. The signed URL pins the contract's response headers -- the media type after the renderable allowlist, the fixed disposition, and the no-store cache posture -- via S3 response-header overrides. PgAssetStore mints the URL from the same ownership-checked transactional read a direct resolve performs, so authorization still runs per read before any URL exists, and a store whose byte layer cannot sign falls back to direct bytes. The contract documents the shape and the disclosure tradeoff: a Location into a hash-keyed byte layer names the content hash, so deployments that need the no-disclosure property keep direct egress. Refs #1007 * feat(app): wire ASSET_BYTE_EGRESS into the persistence route ASSET_BYTE_EGRESS=redirect opts asset byte GETs into the storage package's indirect egress; unset, direct, or an unrecognized value keeps the default byte-for-byte behavior, with a warning for the unrecognized case. The lazy byte-store wrapper now forwards signReadUrl and answers undefined when the resolved layer has no signer, so the PostgreSQL byte column degrades to direct bytes and the S3 layer signs without the wrapper knowing which it holds. The collector path is untouched: it builds its byte store through createAssetByteStore as before and only ever calls delete. Refs #1007 * fix(storage): let the packaged client survive redirect egress Cross-review round 1 on the indirect byte egress change found two problems. A byte response that followed a redirect carries the pinned content type but no X-Asset-Revision, and HttpAssetStore required both from the same response, so every cold resolve under redirect egress failed MALFORMED_RESPONSE. The client now detects the shape once -- a redirected byte response with no revision -- latches it, and probes HEAD before each cold GET from then on, taking the label from the probe. The probe predates the download on purpose: a replacement in between labels newer bytes with an older revision, which the next revalidation detects and corrects, while the reverse order could pin stale bytes under a fresh revision with no signal to repair it. The probe doubles as the miss check, so a redirect-mode miss costs no download. The direct path is byte-for-byte unchanged; the one double download per client lifetime on first contact is documented in the contract, which gains the client-side paragraph this behavior implements. signedUrlTtlSeconds also accepted lifetimes beyond the seven-day SigV4 presigning maximum, minting redirects the object store would reject. The option is now capped at construction. Refs #1007 * fix(storage): cap signed URL lifetime below the reclamation grace A signed URL whose lifetime exceeds the collector grace period can outlive its object: the last reference goes, the grace elapses, the collector deletes the object, and the still-valid URL errors at the object store. The previous ceiling -- the seven-day SigV4 maximum -- made that window days wide against the one-hour default grace. The cap is now fifteen minutes, and the contract text states both the ceiling and the rule for deployments that shorten their grace. Refs #1007 * fix(storage): decline signing when the optional presigner is absent An unresolvable @aws-sdk/s3-request-presigner made signReadUrl throw, which the registry surfaced as a failed asset read -- a 500 for a deployment whose only fault is a missing optional peer. The contract's answer for a byte layer that cannot sign is the capability fallback: return undefined and let the caller serve the bytes directly. Signing errors from a resolved signer still fail loud; only the missing capability declines. Refs #1007 * fix(storage): answer redirect egress as a descriptor to asking clients Following a 302 is not header-neutral: the platform fetch forwards the original request's headers to the object store's origin, stripping only Authorization, so a deployment whose credential travels in a custom header would hand it to the object store -- and the preflight the forwarded custom headers provoke commonly fails there besides. This replaces the latch-and-probe client shape with an explicit one: the client sends X-Asset-Egress: descriptor on every byte GET, a redirect-egress server answers 200 with a JSON { url, revision } body instead of a Location, and the client fetches the signed URL with no deployment headers at all. The revision comes from the descriptor, so the probe-first ordering and its one wasted download are gone too. The 302 remains the answer for consumers that did not ask; the descriptor response is marked by its own header so a JSON-media asset can never parse as one. Refs #1007 * fix(storage): negotiate the descriptor through Accept, not a custom header A custom X-Asset-Egress request header is not CORS-safelisted, so every byte GET to a cross-origin server became a preflighted request -- a regression for direct-egress deployments that never opted into anything, against servers with no reason to allow the header. The negotiation now rides Accept with a vendor media type, which is safelisted and costs no preflight, and the descriptor answer is identified by that media type as its Content-Type rather than by a marker header. Refs #1007 * fix(storage): sign outside the registry transaction and reserve the descriptor type Two review findings on the indirect egress path. resolveIndirect awaited the signer inside the coordinated read, but a signer on refreshable credentials can wait on the network, and a database connection plus the blob row's FOR SHARE lock would be held across that delay. The read now closes first -- the lock still puts the hash, label and revision in one snapshot -- and signing runs afterwards, which cannot observe anything the read did not. The descriptor media type is also reserved from renderableTypes: the client identifies a descriptor answer by that exact Content-Type, and an allowlisted asset served with it would be misread as a descriptor and fail MALFORMED_RESPONSE. The handler now rejects the configuration at construction, and the contract states the reservation. Refs #1007 * fix(storage): ship the signing deps and negotiate without ambiguity Three review findings on the indirect egress path. The app now declares @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner as runtime dependencies, and the standalone build force-includes them for the persistence route: both are reached through deliberately untraced dynamic imports, so the shipped image could not resolve them, and a deployment opting into S3 or redirect egress would have found the capability silently absent. The descriptor negotiation matches exactly on both sides: the client compares the Content-Type essence, so a longer media type that merely begins with the reserved value still serves as bytes, and the server parses Accept media ranges, so a descriptor range sent with q=0 selects the redirect instead of the descriptor it explicitly rejects. Refs #1007 * fix(app): refuse redirect egress when the collection grace is shorter A signed URL must never outlive its object: with the handler's 60-second default lifetime, a deployment that sets ASSET_COLLECTION_GRACE_MS below ten minutes lets the collector delete an object while a URL minted against it is still valid. The route now refuses the combination at handler initialization, naming both variables -- a deterministic misconfiguration, loud at startup rather than silent at read time. Refs #1007 * fix(app): ship the signing SDKs without loading them Two findings on the standalone deployment of redirect egress. The outputFileTracingIncludes globs did not follow pnpm's scoped symlinks, so the standalone image carried neither AWS package and both S3 mode and redirect egress could not resolve their SDK at runtime. The packages are now server-external and referenced from literal -- but never called -- dynamic import thunks in the persistence byte-store wiring, which gets them traced into the image while module resolution still happens only on first S3 use. Verified against a local standalone build: node_modules/@aws-sdk now contains both packages, and the route tests that pin the never-resolve-unless-configured discipline still pass. Media type comparisons are also case-insensitive now, as HTTP requires, on both the Accept parse and the descriptor recognition. Refs #1007 * fix(app): treat an empty collection grace as unset Number('') is 0, so a present-but-empty ASSET_COLLECTION_GRACE_MS failed the redirect-egress coordination check and took persistence down with it, while the collector's own parsing reads the same value as the one-hour default. The check now trims and treats empty as unset, matching durationEnv. Refs #1007 * fix(storage): accept bytes too, omit signing on the known-PG wrapper, state the full tradeoff Three review findings. The client's descriptor request now advertises both representations -- the vendor descriptor type preferred, */* accepted -- so a strict content-negotiating layer can never answer 406 to a client that does in fact consume plain bytes. The lazy byte-store wrapper no longer advertises signReadUrl when no bucket is configured: the layer is known to be PostgreSQL at construction, and carrying the method made resolveIndirect take its ownership query and blob-row lock just to decline, then repeat them in resolve, on every cold GET. With a bucket configured the lazy validation is preserved and the wrapper still declines when the resolved layer cannot sign. And the contract's disclosure tradeoff now states the second edge: object-store GETs carry ETag and Last-Modified, response overrides cannot strip them, and because deduplicated PUTs rewrite the hash-keyed object, Last-Modified moves when another principal re-uploads the same bytes -- shared-object write timing, beyond byte equality. Deployments for whom that signal is sensitive must keep direct egress. Refs #1007 * fix(app): degrade on bad grace, share the ttl invariant, cover the real boundary The latest review round, four findings. A misconfigured ASSET_COLLECTION_GRACE_MS now degrades redirect egress to direct with a warning instead of failing the shared handler's initialization -- the asset backend is optional, and its misconfiguration must never take document and runtime traffic down. The package exports assertSignedUrlTtlWithinGrace so a deployment wiring the handler and the collector separately validates the invariant once at its own boundary: the handler caps the lifetime, but only the deployment knows both numbers. The contract states that nosniff does not survive the redirect -- response-header overrides cannot set it -- and why the client-minted blob makes that safe for conforming clients. And the route gains a real-boundary test: the Fetch<->Node adapter, the real createStorageHttpHandler, and the egress wiring, exercised with an actual byte GET through the composed stack instead of mocks. Refs #1007 * fix(storage): make the unsafe egress configuration unrepresentable Three findings on the indirect egress path, all resolved by moving a rule into a place where it cannot be bypassed rather than by adding a check a consumer has to remember. The grace/TTL invariant now lives in the option shape. byteEgress takes 'direct' or { mode: 'redirect', collectionGraceMs, signedUrlTtlSeconds? }, so enabling indirect egress without declaring the reclamation window it lives inside no longer type-checks, and the handler validates both numbers at construction. The flat signedUrlTtlSeconds option, its cross-field "requires byteEgress redirect" check, and the standalone 900-second cap as the only guard all go away; the ceiling stays, but alongside the ratio rather than in place of it. This is what the earlier exported assertSignedUrlTtlWithinGrace could not do -- it protected only consumers who called it -- and it costs nothing in compatibility because the option is new in this branch. The app resolves the grace through one shared parser with the collector, so the number the handler checks is the number the collector runs with. A signed URL whose object is gone is now a miss. The client maps a 404 from the byte fetch to ASSET_NOT_FOUND, matching the direct path: the entry was owned and readable when the URL was minted, so a reclaimed object is the same physical state the direct read reports as a miss, and what a read means must not depend on the deployment's egress setting. Signing still does not probe for the object -- that would restore the round trip this feature removes and make the mint's price vary with prior presence -- so the contract instead requires the byte layer to answer 404 for an absent object, and states that S3 needs s3:ListBucket to do so. Route coverage now spans the whole egress matrix through the real composed handler rather than only the direct case: 302 for a consumer that did not ask, descriptor for one that did, direct bytes when the byte layer declines to sign, and direct bytes when a short grace degraded the configured mode. Refs #1007 * feat(storage): forward byte egress options through reference server * fix(storage): fail closed on redirects and unconfirmed 404s in indirect byte egress A signed-fetch 404 now maps to a miss only when the object store's XML body declares NoSuchKey; every other 404 from the signed fetch fails loud, so a wrong bucket, access point, or endpoint can no longer make a live asset read as absent. The descriptor byte GET is sent with redirect: 'manual' and any 3xx answer is treated as an error, so a server that ignores the descriptor negotiation can never forward the deployment's custom credential headers to a redirect target. The ASSET_BYTE_EGRESS switch is now documented in .env.example and the server-persistence deployment section, together with the object-store CORS and s3:ListBucket prerequisites it requires, and the asset descriptor media type is exported from the package root and the asset/http subpath. --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
|
969574bee4 |
feat(storage): wire the asset backend into the app (#1007) (#1089)
* fix(persistence): stop corrupting binary response bodies in the route adapter The embedded persistence route bridges a Node `RequestListener` to the Fetch API by hand. Its response object is cast with `as unknown as ServerResponse`, so the compiler checks none of that surface, and two parts of it were wrong in ways only a byte-carrying handler would hit. `end` decoded any `Uint8Array` chunk with `Buffer.from(chunk).toString()`, which is UTF-8. `ServerResponse.end` accepts a `Uint8Array`, and those bytes are not necessarily valid UTF-8, so every unpaired byte became U+FFFD -- silent corruption with no error anywhere. The adapter now buffers chunks as bytes and builds the response from them. `write` was absent entirely, making any chunked handler a runtime TypeError rather than a compile error. It is part of the surface this object claims to implement, so it is now implemented. Neither reaches a caller today: the document and runtime handlers only ever end with JSON strings, which took the string branch. Both are traps laid for the first handler that carries bytes. Both cases are pinned by tests that fail before this change -- the binary one on a byte comparison, the chunked one with a 500 from the TypeError. The existing adapter round-trip test, whose comment already called this the most bug-prone code in the route, covered only the string path. * feat(storage): wire the asset backend into the app (#1007) The server asset registry shipped with #1007 but was reachable from nothing: the persistence route never passed an `assetStore`, and no file under `app/`, `lib/` or `components/` imported the asset server or client modules. This connects it, additively -- no DSL change, no version bump, and no behavioural change when server persistence is off. The route now builds a `PgAssetStore` on the same pool and transaction as the document store, with `PgAssetByteStore` by default and `S3AssetByteStore` when `ASSET_S3_BUCKET` opts a deployment into it. The AWS SDK stays an optional peer, reached only through a dynamic import on that branch. The offline collector is deliberately not scheduled here, and the route says so where a reader would look for it, because leaving it unscheduled grows storage without bound and scheduling it is the deployment's decision. On the browser side the asset pool gains a single-shot configuration seam modelled on the document store's, so that with server persistence enabled the pool is an `HttpAssetStore` carrying the same auth headers as the other stores. Three browser-specific mechanisms needed decisions rather than translation: `clearAssetPool` deletes an IndexedDB database today. In server mode there is nothing local to delete, and a "clear cache" action must not remove server-side assets -- that would destroy user data from a button promising to free space. It now revokes local object URLs and closes the client, and deletes nothing remote. The cross-tab replacement broadcast is unchanged. Its job is to make another tab drop a warm object URL when the bytes behind an id change, and that need is identical in server mode. The exclusivity proof that decides whether an in-place `replace` is safe reasons over this browser only: local documents, unflushed state, and a cross-tab presence probe. In server mode those cover a strict subset of the holders, since another device can reference the same id and no probe can see it -- and asking the server who else references an id would be an existence oracle over other principals. The proof therefore returns not exclusive unconditionally in server mode, and regeneration takes the existing fork path. That is the fail-closed direction and a graceful degradation rather than a break. The development authenticator had to grow the asset principal's required `key`; without it every asset request would have been correctly denied. It derives from the same learner partition, and a principal without one still gets no asset access. The route now states plainly that this key comes from a client-supplied header, so the cross-principal isolation the asset contract describes is not in force under this authenticator -- that warning previously lived only in the auth module's own header, where nobody mounting the route would see it. * fix(storage): close three defects in the asset app wiring (#1007) **The S3 branch could not load, and isolated nothing.** The route dynamically imported the storage package's S3 byte store, but that module statically imported the AWS SDK, so merely resolving it pulled the SDK into resolution and bundling. The route then separately performed an ignored native import of the SDK from the app's own scope, which fails with ERR_MODULE_NOT_FOUND because the SDK lives under the storage workspace package -- so a deployment that set the bucket got a broken store rather than an S3 one. There is now one lazy loader in one resolution scope, the storage module no longer imports the SDK at module load, and the bucket is validated at initialization rather than treated as valid because it is non-empty. A test mocks the SDK to throw on resolution and asserts that importing the byte-store module never resolves it. **A cross-tab replacement could leave a mounted lease stale.** The broadcast handler re-resolves, and `HttpAssetStore.resolve` coalesces concurrent calls for one id onto a single in-flight request -- correct on its own, and required by the contract. Together they meant a resolve triggered by the broadcast could merge into a GET that started before the peer's replacement, return the old revision, and leave the consumer on the old URL with nothing to dislodge it. The client gains an invalidation that retires its cached snapshot and advances the per-id generation without revoking any URL already issued; `release` would have been wrong here, since the contract keeps an issued URL valid until its holder releases it. Removing the invalidation call fails the new race test. **Clearing a server-backed pool left the closed client installed.** In server mode `clearAssetPool` closed the client and returned before clearing the module singleton, so every later `getAssetPool()` handed back a closed store until a page reload happened to intervene. Only the IndexedDB deletion is skipped in server mode now; the singleton is always cleared so the next call rebuilds through the configured factory. Also adds the tests whose absence let these through: a route round-trip carrying invalid UTF-8, the SDK-resolution assertion above, the broadcast race, and reopening the pool after a clear. The earlier tests were not wrong so much as aimed past the defect -- the route test used JSON text, and the S3 test injected a loader rather than exercising the imports. * chore(storage): bump package version to 0.3.0 * fix(persistence): make the route adapter faithful where it carries bytes The embedded persistence route bridges a Node `RequestListener` to the Fetch API by hand, and its response object is cast `as unknown as ServerResponse`, so the compiler checks none of that surface. Five divergences are closed here; the first two are the ones that corrupt data, the last three were raised in review. **`end` decoded bytes as text.** It ran `Buffer.from(chunk).toString()` on a `Uint8Array` chunk, which is UTF-8, so every byte outside that encoding became U+FFFD -- silent corruption with no error anywhere. The adapter now buffers chunks as bytes. **`write` was absent**, making any chunked handler a runtime `TypeError` rather than a compile error. **`write` callbacks ran inline.** Node invokes them after the chunk is handed off. A handler that writes its next chunk from each callback therefore recursed synchronously and could overflow the stack, and a throwing callback threw in the wrong execution phase. They are now deferred. **The encoding argument was ignored.** `write(s, 'latin1')` emitted `0xc3 0xa9` where Node emits `0xe9`, corrupting the body and potentially invalidating a `Content-Length` the handler had set. Both `write` and `end` now parse their full overload set and pass the encoding through. Node's own behaviour for an invalid encoding was established by probing `ServerResponse` on Node 22 rather than assumed: it throws `ERR_UNKNOWN_ENCODING` synchronously, so the adapter validates with `Buffer.isEncoding` and lets `Buffer.from` raise the native error. **Bodyless statuses were not suppressed.** Node discards writes after `writeHead(204)` or `writeHead(304)`; this passed the buffered bytes to the Fetch `Response` constructor, which throws for those statuses, so the request became a 500 instead of the intended 204 or 304. Buffered data is now discarded for 204, 205 and 304, and for `HEAD`. None of the five reaches a caller today -- the document and runtime handlers only ever end with JSON strings. They are traps laid for the first handler that carries bytes, which is why they surfaced while wiring the asset server backend, whose byte routes hit the first two immediately. Each is pinned by a test that fails when its fix is reverted. * fix(storage): keep { client, bucket } working for the S3 byte store `S3AssetByteStoreOptions.commands` was required and the store's methods called it directly, so the published `./asset/s3-bytes` export stopped accepting the `{ client, bucket }` construction it had always taken. Anyone not going through `loadS3AssetByteStore` either failed to compile or hit an undefined command factory at runtime. Make `commands` optional. Omitted, the store resolves the AWS SDK's own command constructors through a dynamic import on its first `write` / `read` / `delete` and caches them, so `{ client, bucket }` works again. Importing the module and constructing a store still never reach the optional peer dependency: a deployment that does not select S3 never resolves it. When the SDK cannot be resolved the call rejects naming `@aws-sdk/client-s3` rather than crashing on an undefined property, and the resolution is left uncached so installing the dependency fixes an already-constructed store. The lazily bound store is run against the byte-store contract, and the new isolation tests pin both that construction does not resolve the SDK and that an unresolvable SDK is reported by name. With the break gone this branch's change to the package is additive, so the version becomes a patch above main (0.2.4 -> 0.2.5) rather than the minor it was carrying. Refs #1007 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(persistence): run the asset collector in the shipped deployment `PgAssetStore.remove`, and a `replace` that changes content, only stamp `asset_blobs.unreferenced_at`. `AssetCollector.collect` is the only path that deletes anything, and the persistence route deliberately did not schedule it, on the grounds that scheduling is the deployment's decision. The deployment this repository ships is docker-compose.yml: the app and PostgreSQL, and nothing else that could make that decision. "The deployment decides" therefore meant "it never runs", and ordinary asset churn retained PostgreSQL bytes or S3 objects forever. Schedule it from instrumentation.ts, which Next runs once per server process — a route module can be instantiated more than once and has no shutdown hook, so a schedule started from one is really started per instantiation. Next 15 and later pick the file up with no config, so next.config.ts is unchanged. Collection is on by default with a 15-minute interval and the package's one-hour grace period, so the Compose stack is correct with no operator action. ASSET_COLLECTION_INTERVAL_MS, ASSET_COLLECTION_GRACE_MS, and ASSET_COLLECTION_ENABLED change or disable it, each documented where it is read. Without DATABASE_URL nothing is scheduled at all, and a failed pass is logged and retried on the next tick rather than ending the schedule or the process. Several instances may collect concurrently: each candidate blob row is re-checked and locked FOR UPDATE inside its own transaction, so collectors serialize on the row rather than racing. The comment says so, so nobody adds a distributed lock that is not needed. The byte-store selection moves to lib/persistence/asset-byte-store so the collector reclaims through the same layer the request path wrote through. A collector on the PostgreSQL byte store while the route wrote to S3 would drop the blob row and orphan the object permanently. Refs #1007 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(storage): bound each collector pass and report whether it filled An unbounded collect() is sized by however long a deployment ran before collection was scheduled: one statement selecting every eligible blob, then a transaction and a byte-layer delete each, in a loop nothing interrupts. The first pass over the Compose deployment that grew a backlog before collection was scheduled is exactly that pass. A pass now takes at most batchSize blobs (default one thousand) and returns. Ordinary churn between two scheduled passes is far below the cap, so a healthy deployment behaves as it did when a pass was unbounded; a capped pass costs the remainder one scheduling interval, which is what the interval is for. collect() still answers with the count, and the count alone cannot tell an empty backlog from a full batch -- a re-referenced or concurrently taken candidate is skipped, so even a full batch can collect less than batchSize. collectPass() returns the count together with capped, which is exactly "this batch was full, run again"; a caller draining a backlog loops while it is true. Candidates are taken oldest-unreferenced first, with the content hash as tiebreaker within one timestamp. Ordering by the hash alone would starve: digests are uniformly distributed, so a high-sorting blob waits behind every lower digest stamped after it, and under steady arrivals those keep coming. The tests pin the ordering against a planner that happens to answer the right rows without an ORDER BY. Refs #1007 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(persistence): keep asset backend faults off the shared handler Two review findings on the app wiring. A concrete asset pool instance is single-lifecycle, but resolveConfiguredAssetPoolStore handed the same object out again after clearAssetPool() had closed it, reinstalling a dead store as the live pool. The second handout now refuses loudly and points at the factory form, which rebuilds on every resolution. createPersistenceHandler eagerly awaited the asset byte store, so an invalid ASSET_S3_BUCKET or an unresolvable AWS SDK rejected the shared persistence handler and took document and runtime traffic down with an optional backend. Byte-store construction is now deferred to the first asset byte operation, and a failed construction is not cached, so the next asset request retries -- the route's own no-poisoned-singleton rule. Refs #1007 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
wyucand杨慎
|
1498ca52b2 |
fix(persistence): stop corrupting binary response bodies in the route adapter (#1088)
* fix(persistence): stop corrupting binary response bodies in the route adapter The embedded persistence route bridges a Node `RequestListener` to the Fetch API by hand. Its response object is cast with `as unknown as ServerResponse`, so the compiler checks none of that surface, and two parts of it were wrong in ways only a byte-carrying handler would hit. `end` decoded any `Uint8Array` chunk with `Buffer.from(chunk).toString()`, which is UTF-8. `ServerResponse.end` accepts a `Uint8Array`, and those bytes are not necessarily valid UTF-8, so every unpaired byte became U+FFFD -- silent corruption with no error anywhere. The adapter now buffers chunks as bytes and builds the response from them. `write` was absent entirely, making any chunked handler a runtime TypeError rather than a compile error. It is part of the surface this object claims to implement, so it is now implemented. Neither reaches a caller today: the document and runtime handlers only ever end with JSON strings, which took the string branch. Both are traps laid for the first handler that carries bytes. Both cases are pinned by tests that fail before this change -- the binary one on a byte comparison, the chunked one with a 500 from the TypeError. The existing adapter round-trip test, whose comment already called this the most bug-prone code in the route, covered only the string path. * fix(persistence): make the route adapter faithful where it carries bytes The embedded persistence route bridges a Node `RequestListener` to the Fetch API by hand, and its response object is cast `as unknown as ServerResponse`, so the compiler checks none of that surface. Five divergences are closed here; the first two are the ones that corrupt data, the last three were raised in review. **`end` decoded bytes as text.** It ran `Buffer.from(chunk).toString()` on a `Uint8Array` chunk, which is UTF-8, so every byte outside that encoding became U+FFFD -- silent corruption with no error anywhere. The adapter now buffers chunks as bytes. **`write` was absent**, making any chunked handler a runtime `TypeError` rather than a compile error. **`write` callbacks ran inline.** Node invokes them after the chunk is handed off. A handler that writes its next chunk from each callback therefore recursed synchronously and could overflow the stack, and a throwing callback threw in the wrong execution phase. They are now deferred. **The encoding argument was ignored.** `write(s, 'latin1')` emitted `0xc3 0xa9` where Node emits `0xe9`, corrupting the body and potentially invalidating a `Content-Length` the handler had set. Both `write` and `end` now parse their full overload set and pass the encoding through. Node's own behaviour for an invalid encoding was established by probing `ServerResponse` on Node 22 rather than assumed: it throws `ERR_UNKNOWN_ENCODING` synchronously, so the adapter validates with `Buffer.isEncoding` and lets `Buffer.from` raise the native error. **Bodyless statuses were not suppressed.** Node discards writes after `writeHead(204)` or `writeHead(304)`; this passed the buffered bytes to the Fetch `Response` constructor, which throws for those statuses, so the request became a 500 instead of the intended 204 or 304. Buffered data is now discarded for 204, 205 and 304, and for `HEAD`. None of the five reaches a caller today -- the document and runtime handlers only ever end with JSON strings. They are traps laid for the first handler that carries bytes, which is why they surfaced while wiring the asset server backend, whose byte routes hit the first two immediately. Each is pinned by a test that fails when its fix is reverted. --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
wyuc
|
3b4567076f |
refactor(generation): consume the package everywhere and delete lib/generation (#1090)
Part D of #1057 — the switch. All consumers import @openmaic/generation; lib/generation deleted (net -11,900 lines). PBL wiring preserved (single-call -> classified fallback -> injected loop), streaming outline route deduplicated onto buildOutlinePrompt, serverExternalPackages wired, and the change verified by a real-model four-kind end-to-end run against the production build with a rendered-classroom screenshot check. Closes #1057. |
||
|
|
fc66cc0603 |
feat(runtime): add learner whiteboard RuntimeStore foundation (#1075)
* feat(runtime): add whiteboard RuntimeStore foundation * Potential fix for pull request finding 'Comparison between inconvertible types' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * fix(runtime): reject invalid whiteboard appends before commit * fix(runtime): preserve special JSON keys during canonical clone * Potential fix for pull request finding 'Invalid prototype value' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * feat(runtime): add minimal learner whiteboard mutation --------- Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
5d5acd6215 |
feat: claude search (#393)
* Added search tool support for Claude * New locales for i18n * Updated translations * Prevent default web search base URLs from being prepopulated * fixed coding style issues * fixed coding style issues * fixing lint errors * fix: add SSRF protection to fetchPageContent in claude.ts Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/099e53fb-e1db-49f4-af7f-4f17cb3d364a Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fix: address all remaining review feedback from PR review thread Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/e1ab8fe2-fe0d-45a5-a0af-30945b99c268 Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fix: rename ambiguous variable names in settings store for clarity Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/e1ab8fe2-fe0d-45a5-a0af-30945b99c268 Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fixing lint errors * fix: address 5 items from second PR review thread Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/09469d80-8e91-449b-8bc2-9c11f83555df Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fixing model verification error * fix: address 3 items from third PR review thread Agent-Logs-Url: https://github.com/joseph-mpo-yeti/OpenMAIC/sessions/0aa49d19-df5b-4a08-a7e1-2295ae9cfc38 Co-authored-by: joseph-mpo-yeti <55380155+joseph-mpo-yeti@users.noreply.github.com> * fixing lint errors * fix: resolved Copilot review comments * fix: returning string promise for ssrf-guard * fix: returning string promise for ssrf-guard * fix: updated locales, added allowed_callers for claude search tools, handle duplicated claude model keys * fix: updated locales, added allowed_callers for claude search tools, handle duplicated claude model keys * updated tests * using Anthropic SDK for search and connection test in search settings. * uses @ai-sdk with a custom fetch including allowed_callers in payload instead of adding Anthropic SDK as a direct dependency * fixed format * removed @anthropic-ai/sdk as direct dependency * fix: resolve TypeScript errors in claude web-search tests Cast mock.calls through unknown to avoid tuple-index type errors on Vitest mock objects whose parameter types are inferred as empty tuples. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add Claude as a web-search provider on the multi-provider framework Forward-port of the original Claude web-search work onto main's searchWeb dispatcher, per review guidance on #393: - lib/web-search/claude.ts: searchWithClaude via the AI SDK Anthropic provider and Anthropic's native web_search server tool, called through callLLM for usage accounting. Returns the model's answer plus cited sources deduplicated by URL. The page-content enrichment fetch from the original PR is removed entirely, resolving the P1 SSRF finding (no server-side fetches of attacker-influenced result URLs remain). - allowed_callers: ["direct"] is still injected at the fetch layer; the AI SDK's provider-defined web_search tools do not emit it. - Tool version picked from the model: web_search_20260209 for 4.6+ Opus/Sonnet, basic web_search_20250305 for Haiku/older; maxResults maps to the tool's max_uses. - Registry/dispatcher/route wiring: claude provider entry, claudeModelId passthrough (web-search route, classroom generation, generation preview), WEB_SEARCH_CLAUDE_* env config with server-pinned model via WEB_SEARCH_CLAUDE_MODELS, and client base-URL allowlisting for api.anthropic.com. - Settings: per-provider modelId with a model selector in the Claude panel; locale keys added to all ten languages. - Tests: new adapter suite plus claude cases in the dispatcher, constants, route, provider-config, web-search-config, and settings store suites. Closes #392 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01E9KqAGmKCtBwMC4HtT4JKE * fix(web-search): trim whitespace around the Claude API key and base URL Keys are pasted by hand into Settings, and nothing in the client key path trimmed them: a trailing newline or space was sent verbatim as the x-api-key header value and came back from Anthropic as 'invalid x-api-key', with nothing in the message pointing at whitespace. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(web-search): correct Claude adapter tool selection, base URL, and timing Addresses the four P2 review findings on #393: - Route dated Claude 4.0 ids (claude-sonnet-4-20250514, claude-opus-4-20250514) to the basic web_search_20250305 tool. The blocklist regex missed them, so they were sent web_search_20260209, which pre-4.6 models reject. Replaced with an allowlist of the models that have dynamic filtering, so unrecognized ids also fall back to the basic tool -- the safe direction, since 4.6+ models accept both. - Pin allowed_callers: ["direct"] only on web_search_20250305. Injecting it into every request disabled the code-execution caller that provides dynamic filtering on web_search_20260209. - Normalize the bare Anthropic root to /v1 before handing the base URL to the SDK, which appends /messages verbatim and would otherwise 404. The adapter is the single choke point for the client allowlist, the server env var, YAML, and the default; other hosts pass through untouched. - Return responseTime in seconds, matching every sibling adapter's WebSearchResult contract instead of exposing milliseconds for Claude. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
wyuc
|
0ae21aa559 |
refactor(pbl): retire v1 write path and dead UI, converge on v2 (#1060)
* refactor(pbl): retire v1 write path and dead UI (#1058) Remove the legacy generator, renderer branches, chat API, and compatibility mirror writes. Keep stored v1 scenes readable through documented legacy adapters, and surface ordinary PBL planner double failures instead of degrading. * fix(pbl): contain planner failure to the scene, support legacy scenes in action regeneration, tighten legacy guards - Contain typed planner failures per scene while preserving retry and skip behavior. - Upgrade legacy PBL content before regenerating scene actions. - Apply video-export import boundaries to legacy readers. - Harden legacy rendering and empty-state handling. - Restore legacy upgrade compatibility coverage. - Remove retired translations and refresh fallback narration and legacy cover documentation. * fix(pbl): keep tolerant legacy configs renderable, guard invalid chat timestamps * chore(dsl): bump to 0.6.3 for README accuracy fix The README no longer references the deleted PBLProjectConfig symbol; the version gate requires a bump for any publishable package input change. * fix(pbl): harden legacy upgrade and persistence contract * fix(pbl): align action prompt with v2 workflow and validate project containers * fix(i18n): remove retired PBL v1 keys from the new fr-FR locale The fr-FR locale landed on main after this branch removed the v1 UI key families; prune the same 38 keys so all 10 locales align with en-US. * fix(generation): sync packaged pbl-actions template with the v2 wording The generation package that landed on main carries a byte-identical copy of the app's pbl-actions template; sync the v2 rewording and bump the package version. * fix(pbl): normalize legacy content and preserve planner status * fix(pbl): propagate provider status and ground action prompts * fix(pbl): classify planner failures for retry, default the project summary prompt input * fix(pbl): fall back to legacy data when v2 payloads are damaged, harden summary building * refactor(pbl): resolve hybrid content through a validity-aware resolver * fix(pbl): keep one v2 acceptance criterion across validator, resolver and persistence Persistence gates on the same container predicate as the validator and the renderer's resolver: payloads the renderer will not treat as v2 are inert bytes everywhere, avoiding runtime side effects for documents the write barrier would reject. An explicit null proficiencyAssessment stays processable. * fix(pbl): require runnable projects and harden legacy fallback * test(pbl): exercise the real save reload and hydration chain * fix(pbl): require an instructor and usable microtasks before v2 takes precedence * fix(pbl): fall back to the loop planner unless the provider failed or the user cancelled * fix(pbl): require role and task identities before v2 takes precedence Runtime normalization can repair status and create the instructor thread, but it cannot synthesize the role id that binds the thread, the microtask ids used by lookup and update paths, or the labels the workspace renders. The runnable predicate and its video-export mirror now require them, so identity-less payloads fall back to usable legacy data. |
||
|
wyuc
|
3f5c51bdba |
refactor(pbl-v2): split the planner core from the loop; inject the LLM call path (#1069)
Implements #1062. Pure planner-core extracted from the loop's file; both planners receive their LLM entry as a parameter (call sites pass the real callLLM unchanged); operations/ split into kernel vs runtime with a lint-enforced boundary; planner prompts pinned byte-identical via fixtures generated from the pre-refactor implementation. |
||
|
LING DUAN
|
61a7154f07 |
feat(pi): add evidence-aware Director context runtime (#971)
* feat(pi): add evidence-aware Director context runtime * fix(web-search): require completed native search * fix(orchestration): preserve whiteboard across agent handoffs * fix(pi): remove wrap-up turn bypass |
||
|
wyucandCodex
|
e39c64cd9e |
feat(persistence): one-command server-backed stack (compose profile + embedded API + docs) (#982)
* feat(persistence): one-command server-backed stack — embedded API + compose postgres profile docker compose --profile server-persistence up --build runs exactly two containers: the app (persistence HTTP server embedded at /api/persistence, enabled only when DATABASE_URL is set — unset keeps today's browser-only behavior byte-for-byte) and PostgreSQL 16. Client bootstrap configures both storage seams against the same-origin route when NEXT_PUBLIC_PERSISTENCE=1, riding the existing lazy migrations so prior browser data reaches the server per-course on first open. Dev auth (bearer token + client-asserted learner header) lives in one overridable module and is loudly documented as development-grade. Verified end-to-end against the real stack: document PUT/GET and runtime session create land in Postgres; bad tokens 401. Co-authored-by: Codex <codex@openai.com> * fix(persistence): review round — structural bootstrap ordering, retryable init, honest failure surfaces The bootstrap side-effect import moves into the two storage seam entry modules, so any module graph that can resolve a store necessarily evaluates it first — no client-entry ordering luck, and the two configure calls are all-or-nothing. A failed server init clears its cached promise (next request retries instead of permanent 500s). listStages rethrows backend failures and the dashboard surfaces a persistence-unavailable error instead of rendering an empty list that reads as data loss. Docs state the build-time nature of NEXT_PUBLIC_PERSISTENCE, the real token boundary (any visitor can extract it and impersonate any learner — localhost/trusted-network only), and the correct postgres password rotation. Dev-token compare is timing-safe; the handler singleton lives on globalThis so HMR cannot leak pools. Co-authored-by: Codex <codex@openai.com> * fix(persistence): review response — no-confidentiality wording + adapter round-trip coverage The token docs now state plainly that the NEXT_PUBLIC token is compiled into the public bundle and provides no confidentiality or user isolation (its only role is keeping unrelated scanners off a trusted network). The Fetch/Node adapter — the route's most bug-prone code — gains a round-trip integration test: PUT body streaming, 201 with multi-value headers, 204 empty body, and encoded path segments reaching the handler un-decoded. --------- Co-authored-by: Codex <codex@openai.com> |
||
|
杨慎andClaude Opus 4.8
|
84b1907255 |
feat(video-export): service-backed MP4 render + in-app one-click export (#866) (#937)
* feat(video-export): service-backed MP4 render + in-app one-click export (#866)
Adds the last mile of classroom video export: turning the self-contained
Hyperframes project ZIP (#865) into an MP4 via an isolated render service,
one-click in-app.
- render-service/: standalone Node 22 + Chromium + FFmpeg container wrapping
@hyperframes/producer's library API. Async job model (POST /render -> 202
jobId, GET poll, GET download, DELETE cancel). Swappable JobStore /
ArtifactStore seams (in-memory + local-disk now; Redis/S3 + presigned-302
download later) so it scales horizontally without changing the HTTP contract.
Concurrency + per-user guards are config knobs.
- App integration: thin Next proxy routes under app/api/export-video/* (forward
only, no rendering) + capability probe. use-render-video.ts uploads the ZIP,
polls via runPolledTask, downloads the MP4; shared buildExportZip prefix with
the existing ZIP path. Export menu gains resolution/fps/quality selectors and
a progress bar; degrades to ZIP download when RENDER_SERVICE_URL is unset.
- docker-compose: render-service under an opt-in "video-export" profile.
- Entry is main.ts (not server.ts): the producer auto-starts its own server on
:9847 when the process entry path ends with /src/server.ts.
Verified end-to-end in the container: rendered a real 640s (10.7 min) classroom
ZIP to a valid H.264 720p + AAC MP4 (duration matches source) in ~9.6 min
(~0.9x realtime, 4-worker frame capture). Degrade path, queued-cancel + cleanup,
and per-user 429 guard all exercised. pnpm check / lint / tsc / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(video-export): global render progress store, percent+ETA UI, ring on export button (#866)
Addresses two UX issues found while driving the in-app MP4 export:
1. Progress display was raw and unfriendly (showed producer's English stage
strings like "Capturing frame 5130/19220") and had no time estimate. Now the
menu shows only "<percent>% · about <remaining> left". ETA is computed from a
recent-speed estimate (percent-per-ms over the last sample), EMA-smoothed —
which tracks the render's non-uniform pace (prep -> frame capture with a
4->1 worker drop -> encode) far better than a whole-run average, and never
shows a stale/rising ETA.
2. Switching scenes mid-render unmounted the export menu and lost the progress
(and reset the local "already rendering" ref, allowing a duplicate submit).
The whole render lifecycle now lives in a global store
(lib/store/video-render.ts), so progress survives menu close / scene switch
and duplicate submits are guarded by status. A persistent CircularProgress
ring on the export button shows live progress whether or not the menu is open.
Also fixes the progress scale: the producer reports progress as 0..100, but our
HTTP contract (and success path) is 0..1 — the service now normalizes it, so the
client no longer showed "2000%".
- lib/store/video-render.ts: new global store owning submit->poll->download,
recent-speed ETA, duplicate-submit guard.
- lib/video-export-app/use-render-video.ts: thin facade over the store.
- components/ui/circular-progress.tsx: lightweight SVG progress ring.
- components/stage/{header-controls,video-export-menu}.tsx: ring on the export
button; menu shows percent + ETA, subscribes to the store.
- render-service/src/render-manager.ts: normalize producer progress 0..100 -> 0..1.
- i18n: percent/ETA strings across all 8 locales (drops the stage-based string).
- render-service/package-lock.json: complete integrity hashes (reproducible npm ci).
Verified: ETA logic checked against the real segmented render curve (worker drop
raises ETA, encode speedup drives it to ~0); progress scale fix confirmed live
against the container (0.2 -> 20%). tsc / lint / prettier / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): persist render options in the store, not the menu component
Selecting 720p/24fps/draft, switching scenes, and reopening the export menu
showed the defaults again (1080p/30/standard). The selections lived in the
VideoExportMenu component's local state, which reset when the menu unmounted on
a scene switch — the running render still used the chosen options, but the UI
misrepresented them.
Move resolution/fps/quality into the global video-render store (with a
setOptions action). The menu now reads/writes the store, so selections survive
menu close / scene switch, and while a render runs the selectors reflect the
options that render is actually using. startRender() reads options from the
store instead of taking them as an argument.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): deployment correctness + resource/isolation controls (PR #937 review)
Addresses the blocking findings from wyuc's review. Output fidelity was fine;
these harden deployment and production resource/isolation boundaries.
#1 Compose advertised MP4 but couldn't render in prod:
- Capability now probes the service's /health (checkRenderServiceHealth), so a
configured-but-absent service reports disabled and the UI degrades to ZIP
instead of 502-ing.
- RENDER_SERVICE_URL is operator-supplied trusted config, so the proxy no longer
runs it through the SSRF guard — the one-command `docker compose --profile
video-export up` now works without globally weakening SSRF via
ALLOW_LOCAL_NETWORKS. resolveRenderServiceUrl() is now synchronous.
- Client degrades to ZIP on any failed submit (not only 501).
#2 Unbounded upload/queue (ZIP-bomb / DoS):
- unzip.ts bounds the archive via fflate's filter BEFORE decompression: entry
count, per-entry and total expanded size, and compression ratio.
- Proxy rejects oversized uploads (413) by Content-Length before forwarding.
- RenderManager enforces a global queue-depth cap (RENDER_MAX_QUEUE).
- All limits are env-tunable knobs in config.ts.
#3 Per-user guard was ineffective + admission ran after extraction:
- Identity is derived server-side (client IP) and forwarded as x-openmaic-client;
the service ignores any client-supplied userId, and the proxy strips it.
- Admission is split into reserve()/submit()/release(): the slot is reserved
BEFORE extraction, so a rejected caller never triggers a decompression.
Additional risks:
- Per-job wall-clock watchdog (RENDER_JOB_DEADLINE_MS) aborts + fails a hung
render so it can't hold a slot/scratch forever.
- Download proxy bounds only the time-to-headers, not the body stream, so large
MP4s over slow links no longer truncate.
- Client cancels the server job (DELETE) when a started render fails/times out.
- Compose puts render-service on an internal:true network (no host/internet
route), sandboxing the Chromium that runs the uploaded HTML; the export ZIP is
self-contained so no outbound is needed. README documents the standalone caveat.
Not closing #866: the smoke/golden-render CI acceptance criterion remains a
follow-up (see PR description).
Verified in-container: legal render 202; ZIP-bomb (entry-count + compression-
ratio) rejected 400 before any decompression; per-identity guard 429 with a
spoofed multipart userId ignored; reserve-before-extract leaves no scratch dir
on rejection; watchdog aborts an overrunning job and frees the slot. tsc / lint /
prettier / i18n pass; render-service tsc passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): real audio durations + burned-in subtitles (PR #937 review)
Two export-fidelity issues found in wyuc's deeper E2E:
A. Narration was scheduled from estimated durations, cutting audio off mid-
sentence and advancing the timeline early. The scheduler trusted
AudioFileRecord.duration (recorded only since #861), so the many existing
classrooms without it fell back to text-length estimates — measured 4.35s
average / 10.23s max underestimate across 47 clips. timeline-deps now probes
the real duration from each narration blob via an off-document <audio>
(symmetric to the existing video probe), preferring it over the stored
duration, then the estimate only when no audio asset exists. Everything
downstream (narration starts, scene/total duration, subtitle cues) re-derives
from the corrected value in the pure compiler — no compiler change needed.
B. The final MP4 had no subtitles (only H.264+AAC), and the ZIP's SRT/VTT used
the same estimated boundaries. The emitter now renders a burned-in subtitle
overlay: one caption box + a hidden div per cue, revealed/hidden by the paused
GSAP timeline at each cue's start/end (corrected timings from A), so Chromium's
frame capture bakes them in. The producer has no subtitle track of its own, so
burn-in is the v1 approach.
Verified: emitter unit tests + snapshot updated (subtitle overlay + toggle
statements, escaped text, hidden-by-default); 82 video-export tests pass incl.
the determinism red-line proxy. Rendered a synthetic subtitle project through the
container and confirmed by pixel analysis that captions appear only within their
cue window (2429 near-white px in the caption band at t=1.5s vs 0 at t=0.05s).
tsc / lint / prettier / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): subtitle layout + upload/admission hardening (PR #937 review)
Address the three P1 blockers plus actionable P2s from the
|
||
|
|
bb8d4f30e7 |
feat(chat): add experimental Pi classroom runtime behind a flag (#914)
* feat(chat): add experimental Pi classroom runtime * fix(chat): harden Pi whiteboard action sequencing * style(chat): format session retirement flow * style(playback): format async scene transition * fix(chat): retire actions on session errors * fix(chat): close Pi runtime lifecycle gaps * fix(chat): avoid replaying historical whiteboard actions * fix(chat): prevent Pi structured-output residue from leaking and inflating turns Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(chat): expose whiteboard element/line ids and bound code-line context Surface each element's id (and code line ids) in the Pi whiteboard context so later child agents can target wb_delete / wb_edit_code, and cap the rendered code lines with a shared character budget so a code-heavy board cannot grow the prompt without bound. Return only the current turn's whiteboard ledger from the Pi director loop instead of carrying history forward across requests. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(chat): bound Pi whiteboard code context Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(chat): prioritize newer persisted whiteboard code in budget Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(chat): enforce whiteboard boundaries across Pi sessions * feat(chat): refine soft-closing discussion controls * fix(chat): make whiteboard clearing model-directed --------- Co-authored-by: apple <apple@GigideMacBook-Air.local> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: LING-6150 <duan.lin@northeastern.edu> |
||
|
Yizuki_Ameandwyuc
|
7c2aaafc91 |
fix(ssrf): harden provider redirects and ISATAP detection (#928)
* fix(ssrf): detect private IPv4 in ISATAP addresses * fix(ssrf): reject redirects in provider probes --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
cc5a6ab152 |
feat(agent): edit_elements — NL → EditIntent vertical (#895) (#896)
* feat(agent): add edit_elements NL → EditIntent vertical (#895) Give the editing surface its first real consumer: natural-language per-element style/geometry edits flow through the same EditIntent vocabulary as pointer gestures, with an all-or-nothing validation gate and one undo entry per instruction. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent): harden edit_elements against CR findings (#895) Re-validate intents at apply time (atomic refuse on missing/locked ids), type-check prop values, require an open edit session for undo, clamp line stroke width correctly, enforce group cohesion, nest shape text chrome, and surface refuseReason on the tool card. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agent): close remaining edit_elements CR residuals (#895) Feed apply outcomes into next-turn history and post-run chat corrections, phrase tool success as proposed-not-applied, and tighten gradient/filters/ enums/per-type value validation plus label-less shape text refusal. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(agent): derive edit_elements prop validation from the DSL schema (#895) Three review rounds each found a hand-written validator disagreeing with the DSL contract (a gradient pos range pinned at [0,1] against the percent contract, a textType enum matching the DSL in neither direction, per-type ownership gaps like vertical-on-shape and color-on-everything). Replace the class instead of the instances: - The gate now loads the dsl-generated JSON schema (@openmaic/dsl/schema/scene.schema.json) and validates every proposed prop value against the target element type's subschema via a compact schema-subset checker — per-type ownership, enums, and structured shapes (outline/shadow/gradient/filters) come from the single source of truth and stay correct as the DSL evolves. - Explicit policy layers stay on top: the content/identity blocklist (schema would allow what policy forbids), geometry clamps (coords, min sizes, line stroke width), and a numeric-range overlay for what plain number types cannot express — gradient stop pos is a percent in [0,100] (the previous [0,1] check refused canonical gradients and the unit test pinned the wrong contract), fontSize/lineHeight/radius bounds, filter value length, stop-count cap. - Consequences: color is refused on types that do not render it, with prompt guidance steering text color to defaultColor and shape body to fill; fill on charts is now allowed (the schema has it; the inventory already advertised it); textType accepts the full DSL enum. - Apply-time revalidation also refuses element-type drift via a gate-time targetElementTypes map carried in the tool details. - Refusal reasons render as visible card text, not only a hover title. - Test pins added for all of the above plus selection scoping, coord bounds, non-finite rotate, and empty batch/props. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(agent): harden the schema-derived gate — fail closed, assert schema source, unify numeric clamping (#895) - A schema construct the subset checker cannot judge (unresolved $ref, malformed anyOf, unsupported keywords) now refuses instead of accepting — the gate fails closed on the unknown. - A module-load assertion verifies the loaded dsl schema carries every definition the gate maps and a sane TextType enum, so a stale or missing build artifact fails loudly with a rebuild hint instead of validating an outdated contract silently. - Numeric-overshoot convention unified and documented: structural, enum, or ownership mismatches refuse (teaching the model the contract); numeric overshoot on a valid prop clamps (matching gesture behavior) — opacity now clamps like geometry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(agent): localize edit element refusals (#895) * fix(agent): preserve partial element edit semantics (#895) * fix(agent): prevent stale element edit application (#895) * fix(agent): preserve live canvas edit invariants (#895) * fix(agent): close element style contract gaps (#895) * fix(agent): reject non-persistent element styles (#895) * fix(agent): preserve rigid group geometry (#895) * fix(agent): tolerate decimal group translations (#895) * fix(agent): constrain line height edits (#895) * fix(agent): reject stale or shadowed element edits (#895) * fix(agent): close renderer contract gaps (#895) --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
Yanpeng Wangandwyuc
|
9670100363 |
feat(extraction): media (audio/video) extraction + AliDocMind provider (#887)
* feat(extraction): add AliDocMind provider + media extraction abstraction Adds AliDocMind (Aliyun Document Mind LLM version) as a new vendor alongside unpdf/MinerU, and introduces the media (audio/video) extraction layer that mirrors the document extraction one. Document side (file: pdf/docx/pptx/xlsx/images): - PDF_PROVIDERS gains an `alidocmind` entry; parseWithAliDocMind() maps layouts[] -> ParsedPdfContent. Flows through the existing document extractor registry, so AliDocMind is selectable anywhere MinerU is. - PDFParserConfig / DocumentExtractorConfig gain accessKeyId/accessKeySecret (AliDocMind uses AK/SK, not a single apiKey); env fallback via ALIDOCMIND_ACCESS_KEY_ID / ALIDOCMIND_ACCESS_KEY_SECRET. Media side (mp4/mp3/wav/... -> MediaArtifact): - New lib/media-parse/ domain mirroring lib/pdf/ (types/constants/providers). parseMedia() maps AliDocMind segments[]/audio_frames/video_frames -> MediaArtifact (transcript + keyframes). - MediaExtractorProvider interface + media registry + extractMedia() entry, symmetric to DocumentExtractorProvider / extractDocument(). - MediaArtifact and the ExtractionResult/Artifact/Error/Job envelope live in lib/document/types.ts (re-exported from @/lib/document). Shared AliDocMind SDK wrapper (lib/pdf/alidocmind-client.ts) handles the submit -> poll -> get flow for both sides via @alicloud/docmind-api20220711. Tests: env-gated smoke test (tests/document/alidocmind.smoke.test.ts) drives a real PDF and a real video through AliDocMind; media-artifact type test; extractor-registry test updated for the new provider. Part of #621 (MAIC ETL). Media extraction is the sibling to the document extraction landed in #741. * feat(extraction): wire AliDocMind AK/SK through settings UI + routes Surfaces AliDocMind in the (post-#837) Document Parsing settings panel as a peer of unpdf/MinerU — one panel, one credential entry, more supported formats. Threads Aliyun AccessKey ID/Secret from the store through the extraction routes. - Store: pdfProvidersConfig + setPDFProviderConfig gain accessKeyId/accessKeySecret; default alidocmind entry added. - pdf-settings.tsx: AliDocMind branch renders AccessKey ID + Secret inputs (secret masked with show/hide) and a Test Connection button; request-URL preview shows the DocMind endpoint. Provider auto-appears in the panel via PDF_PROVIDERS; supported-format badges come from #837's registry (ALIDOCMIND_MIMES added to lib/document/mime.ts). - verify-pdf-provider route: alidocmind branch verifies AK/SK via a lightweight authenticated probe (verifyAliDocMindCredentials) — auth-level errors fail, anything else passes. - extract-document route + generation flow (app/page.tsx, generation-preview): accessKeyId/accessKeySecret carried through session → FormData → config. - i18n: alidocmindAccessKeyId / alidocmindAccessKeySecret in 8 locales. - Icon: reuse /logos/bailian.svg (Aliyun family) instead of a missing asset. Verified end-to-end against the running app: AliDocMind panel renders, Test Connection returns "连接成功" through the real Aliyun API. Part of #886. * feat(extraction): route audio/video uploads through extractMedia() (reuse document path) Media uploads now flow through the same upload picker, /api/extract-document route, and generation pipeline as documents — no separate upload area. Only the extraction differs: media mimes dispatch to extractMedia() -> MediaArtifact, which is flattened to the text shape the generation pipeline already consumes. - mime.ts: register audio/video formats in DOCUMENT_FORMATS (accept string, extension map, badges resolve for them); add MEDIA_PROVIDER_SUPPORTED_MIME_TYPES + SUPPORTED_MEDIA_MIME_TYPES, kept separate from PROVIDER_SUPPORTED_MIME_TYPES so the document drift-guard stays document-only. mimesForProviders() folds in a provider's media mimes, so the existing upload helpers (getAcceptStringForProviders / isMimeSupportedByProviders / format badges) cover media automatically when the provider supports it. - extract-document route: media mimes dispatch to extractMedia(); MediaArtifact flattened to timestamped text (synopsis + transcript + keyframes). - Tests: 6 media cases in mime.test.ts (accept/validation/badges/normalization). Verified: uploading a real video through the route returns synopsis + timestamped transcript/keyframes as text (curl, real AliDocMind key). Part of #886. * feat(extraction): use Alibaba Cloud icon for AliDocMind provider Replace the placeholder bailian.svg with a dedicated Alibaba Cloud icon mark (the square symbol from the official wordmark, text removed) so the provider reads as Aliyun rather than Bailian. Square aspect matches the other provider icons. Source: Alibaba Cloud / Alibaba Group brand assets. * style: prettier format AliDocMind + media extraction files * fix(extraction): harden AliDocMind — SSRF guard, cred verify, env gate, table text Addresses review findings on the AliDocMind provider: - SSRF (high): the media branch of /api/extract-document now runs the same validateUrlForSSRF check on a client-supplied baseUrl as the document branch, so an audio/video upload can't point the server's Aliyun SDK at an internal host. - Credential verify (high): verifyAliDocMindCredentials now whitelists success signals instead of blacklisting auth errors. Probed against the real API: valid creds + bogus job returns a no-throw "BizIdNotExistOrResultExpired" body; invalid creds throw InvalidAccessKeyId.NotFound. Only a no-throw response or a "biz-not-found" business error counts as valid — an unreachable endpoint, a localized error, or throttling now correctly reports failure instead of a false "connection successful". - Env-fallback gate (med): resolveCredentials no longer reads ALIDOCMIND_ACCESS_KEY_ID/SECRET unconditionally. Env fallback is opt-in via allowEnvFallback, which the route sets only for a server-managed provider, so an unauthenticated client request can't silently run on the server account. - getDocParserResult error body (med): fetchResult throws on a non-200 result envelope instead of returning {} (empty text presented as success). - Media pagination (med): stop after the first page when segments[] are present — layoutNum/layoutStepSize address layout blocks, not media segments, so re-requesting would loop over the same segments to the safety cap. - Table content (med): tables/charts carry content in llmResult, not markdownContent; the layouts→text mapping now prefers llmResult for those types so table content isn't dropped (tables: true was advertised). - Dead code: remove getCurrentMediaParseConfig (referenced non-existent store fields; the single-panel UI reuses pdfProvidersConfig). - Dedup: media MIME list lives only in lib/document/mime.ts (ALIDOCMIND_MEDIA_MIMES); the media registry imports it. - Tests: smoke test asserts durationMs is in ms (>1000 for a ~52s clip) to guard a ms/s unit mismatch; uses allowEnvFallback for the env-cred path. Verified with the real key: PDF + video smoke tests pass; verify classifies valid vs invalid creds correctly. Part of #886. * feat(extraction): extract AliDocMind images to base64 (parity with unpdf/MinerU) AliDocMind embeds figure/picture image URLs inside each layout's markdownContent (markdown ``), not a dedicated field, and the URLs are short-lived OSS signed links. Previously we emitted `images: []` and left the expiring URLs inside the extracted text. Now, for figure/picture layouts we: - parse the OSS image URL out of markdownContent, - fetch it at extraction time (before the signature expires) and re-encode to PNG base64 via sharp — the same base64 `images[]` contract unpdf/MinerU produce, so downstream storeImages → IndexedDB → slide works unchanged, - populate metadata.pdfImages + imageMapping (the generation flow prefers pdfImages), - strip the remote-URL markdown from the emitted text so expiring links don't leak into the prompt. Downloads run concurrently; a failed/`sharp` image is dropped, never failing the whole parse. Also fixes the prior over-broad table handling: only `table` blocks read llmResult; `figure` is treated as an image (chart-figure llmResult still kept in text). Verified with the real key: a sample PDF yields 24 base64 images in both images[] and metadata.pdfImages, and no oss-cn-hangzhou URLs remain in text. Part of #886. * docs(test): note AliDocMind video smoke test is non-deterministic server-side * fix(extraction): correct AliDocMind pageCount (pageNum is 0-based) Verified against a real response: AliDocMind reports pageNum 0..13 and pageCountEstimate 13 for a 14-page document — both are 0-based. The metadata pageCount previously used pageCountEstimate directly, undercounting by one. Use the already-1-based maxPage, falling back to pageCountEstimate+1 only when no blocks were seen. Document the 0-based convention at the normalization site. * fix(extraction): address AliDocMind review — verify/SSRF/config/media (P1+P2) Resolves all P1/P2 findings from the cross-review on #887. P1 (blocking): 1. verifyAliDocMindCredentials now inspects the no-throw response body.code. An OSS-only key returns NoPermission without throwing; previously that was green-lit as "connection successful" then failed at extraction. Only a success/200 or the job-not-found probe code is accepted. Deterministic mocked test added (tests/document/alidocmind-verify.test.ts). 2. verify route trust boundary: managed → server-owned AK/SK + default endpoint only (ignore client values); unmanaged → client creds only, never env fallback, and the client endpoint is SSRF-validated before signing. 3. image fetch hardened: restricted to Aliyun OSS hosts, redirects disallowed, per-image byte cap, image-count cap, bounded concurrency (was unbounded Promise.all over provider-returned URLs). 4. AliDocMind is now selectable in the generation toolbar — availability recognizes the AK/SK pair, not just apiKey. P2 (correctness): 5. Explicit server-config for the AK/SK pair (applyAliDocMindFallback + resolveManagedAliDocMindCredentials); verify and extract now resolve managed/env identically instead of verify-uses-env / extract-rejects. 6. Poll loop checks body.code before status, so a body-level error (e.g. NoPermission) fails fast instead of retrying for the full 15 min. 7. Image page numbers preserved through fetch/filter — no longer hard-coded to page 1, so multi-page image→page association is correct. 8. Empty media extraction (no synopsis/transcript/keyframes) returns 422 PARSE_FAILED instead of HTTP 200 with empty text. 9. Format matrix trimmed to the official contract: images JPG/JPEG/PNG/BMP/GIF (dropped WebP/JP2), media MP4/MKV/AVI/MOV/WMV/MP3/WAV/AAC (dropped M4A). 10. A document-only provider (unpdf/mineru) uploaded with a media file now returns a clear 4xx instead of an opaque 500. P3: credential-verify failures return INVALID_CREDENTIALS 4xx (not INTERNAL_ERROR 500); formatTimestamp emits HH:MM:SS past one hour. Verified with the real key: verify classifies valid→ok and invalid→fail; PDF (24 images, correct page numbers) and MP4 extraction pass end-to-end. * fix(extraction): make AliDocMind selectable + correct analysis label for media Two UI/UX fixes found while manually testing the AliDocMind flow end-to-end: - Persisted-state backfill: add ensureBuiltInPDFProviders so a PDF/document provider added after a user's settings were persisted (AliDocMind) is backfilled into pdfProvidersConfig on rehydrate. Without it the provider never appeared in the store, so it couldn't be selected and never picked up its server-configured flag. Wired into both persist migrate() and merge(), mirroring the existing image/video/web-search backfills. - Analysis step label: getGenerationStepText showed "解析 document 文件" for audio/video (the type map fell through to the literal "document"). Documents keep their precise token (PDF/DOCX/PPTX/XLSX/images); audio/video now use a dedicated, locale-correct string (generation.analyzingMediaMaterial, added to all 8 locales) instead of forcing a format token into the "{{type}} 文件" template. Manually verified end-to-end (real key): PDF, MP4, and MP3 (audio extracted from the sample video) each extract and drive full course generation; a DocMind-restricted AK/SK (OSS-only) now correctly fails verification with NoPermission (400) instead of a false "connection successful". * fix(extraction): address 2nd-round AliDocMind review (managed creds, stream cap, empty verify) Resolves the three follow-up findings on #887: 1. [P1] YAML-managed AliDocMind creds now reach extraction. Both extract paths (document + media) previously cleared the managed AK/SK and relied on an env-only fallback, so a YAML-only deployment verified but failed to extract. They now resolve server-owned creds via the shared resolveManagedAliDocMindCredentials() (env OR YAML), matching the verifier. Regression test added for YAML-only creds with no ALIDOCMIND_* env. 2. [P1] Image download is now size-capped while streaming. Instead of buffering the whole response and checking length afterward, the body is read chunk by chunk with a cumulative byte count that aborts the moment it exceeds the cap — a missing/false Content-Length can no longer exhaust memory. Host allowlist tightened to oss-*.aliyuncs.com. Tests: non-OSS host refusal, no-Content-Length overflow abort, declared-oversize rejection. 3. [P2] Empty verification body no longer counts as success. Removed the codeStr === '' branch from the positive-signal whitelist (a working key always returns the job-not-found business code for the bogus probe id). Regression test added for empty and absent bodies. Rebased onto main (package.json: kept @openmaic/storage + AliCloud deps). Verified with the real key: PDF + MP4 extraction still pass end-to-end. * fix(extraction): merge AliDocMind AK/SK into a baseUrl-configured YAML entry Follow-up to the YAML-managed credential fix. When a YAML `pdf.alidocmind` entry also specifies `baseUrl`, the generic loadEnvSection() (pdf requires a baseUrl) creates the pdf.alidocmind entry copying only apiKey/baseUrl/models/ proxy — never AK/SK. applyAliDocMindFallback() then returned early because the entry already existed, so the provider was "managed" but had no usable credentials, and resolveManagedAliDocMindCredentials() returned undefined (verify + extract both silently lost the creds). Merge the AK/SK into the existing entry instead of returning early. Added a regression test with baseUrl + accessKeyId + accessKeySecret together (the previous test omitted baseUrl, so the generic loader skipped the entry and the bug was masked). * fix(extraction): don't mark AliDocMind managed without AK/SK; align poll code check Two edge cases from a follow-up adversarial pass: - [MED] A YAML `pdf.alidocmind` entry with `baseUrl` but no AK/SK (and no env AK/SK) made the generic loader create the entry → isServerConfigured=true (managed) → but resolveManagedAliDocMindCredentials() returned undefined, so the provider was locked out AND client-entered AK/SK were silently dropped. applyAliDocMindFallback now deletes a credential-less entry so the provider stays UNMANAGED (clients supply their own creds). Regression test added. - [LOW] verify accepted a `code: "success"` body but the extraction poll loop threw on any non-"200" code — a success-shaped status would pass verification then fail extraction. The poll loop now treats "200"/"success" as benign, matching verifyAliDocMindCredentials. --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
c56929510c |
feat(web-search): add SearXNG provider support (#842)
* feat(web-search): add SearXNG provider support Enable self-hosted SearXNG via SEARXNG_BASE_URL, with server-side provider fallback, settings sync, and tests. Co-authored-by: Cursor <cursoragent@cursor.com> * style: fix Prettier formatting in web-search files Restore pnpm check by applying Prettier to imports, headers, and test assertions. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(web-search): keep SearXNG base URLs server-managed only Close the SSRF primitive by ignoring client-supplied SearXNG URLs at the route boundary and rejecting them in resolveSafeClientWebSearchBaseUrl. Add negative route/config tests and an opt-in live smoke test for the JSON API. Co-authored-by: Cursor <cursoragent@cursor.com> * style: fix Prettier formatting for SearXNG web-search changes Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: 徐松 <song.xu@aieshanghai.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
jackefnandRowan_lxb
|
baff0998bb |
Feat/document bundles milestone 3 (#844)
* feat(document): support document bundles * fix(document): avoid server extractor imports in client bundle * fix(pdf): default mineru backend to pipeline * fix(document): address bundle review findings * fix(document): harden extraction limits and storage errors * fix(document): make analysis step format agnostic --------- Co-authored-by: Rowan_lxb <Lxb_savior@163.com> |
||
|
PhillLittlewoodandwyuc
|
a2886af92a |
Added ComfyUi to has Image Provider (#850)
* feat: integrate comfyui workflows into image provider selection * style: format code with prettier * fix: align i18n keys for russian locale * feat: implement requested changes and format code * feat: implement requested changes including renaming default workflow --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
Yanpeng Wang
|
b5e1963e48 |
feat(document): rename to Document Parsing, show supported formats, extend MinerU support
Squashed from PR #837 after review approval and green CI. |
||
|
Yanpeng Wang
|
d54e62f5c4 |
feat(token-plan): one-click token-plan setup + deployment usage dashboard (#784)
* feat(usage): add usage normalization, pricing, model-fetch, balance, storage Foundational layer for token-plan usage tracking (cc-switch-modeled): - lib/usage/normalize.ts: AI SDK v6 usage → four-class token shape - lib/usage/pricing.ts + defaults: per-class USD pricing table - lib/server/model-fetch.ts: /models candidate-URL multi-fallback (ported) - lib/usage/balance-providers.ts: built-in balance queries + detection - lib/server/usage-storage.ts: fire-and-forget jsonl logging to data/usage All pure/storage logic covered by vitest (32 tests). * feat(usage): capture token usage at the callLLM/streamLLM chokepoint - callLLM records result.usage before returning - streamLLM wraps onFinish to record totalUsage on stream completion, preserving any caller-supplied onFinish - provider/model derived from the model instance (no route changes) - fs-backed storage imported dynamically; fire-and-forget, never throws - include_usage is already sent by @ai-sdk/openai, so streaming is covered * feat(usage): add probe-models, balance, and usage API routes - POST /api/provider/probe-models: discover chat models via /models with candidate fallback; SSRF-guarded; filters non-chat ids; 401/404 typed - POST /api/provider/balance: built-in balance detection + billing fallback - GET /api/usage: aggregate jsonl by model/day/source with costIncomplete flag Verified e2e against the live MAIC gateway: 16 chat models (6 filtered), balance detected, and a real callLLM writes a costed usage row. * feat(settings): add token-plan preset picker to provider dialog - lib/config/token-plan-presets.ts: data-driven vendor presets (Huawei/MiniMax/ Xiaomi token plans, OpenRouter/SiliconFlow gateways, DeepSeek/GLM/Qwen/Hunyuan/ Doubao direct) with baseURL, protocol, optional modelsUrl, category - add-provider-dialog: '选择厂商/自定义' tabs; picking a preset auto-fills baseURL+protocol+modelsUrl, custom tab unchanged - ProviderSettings.modelsUrl carries the optional /models override - i18n keys added across all 8 locales Verified in-browser: preset picker renders grouped by category. * feat(settings): add Fetch Models button and balance bar to provider panel - 拉取模型: probes /models, merges discovered ids into the model list (dedupe, keeps manual additions), with success/no-endpoint/auth messages - 查询余额: queries /api/provider/balance, renders a balance bar or a 'check console' hint when unsupported - index.tsx: handleModelsFetched merges probe results into provider config - i18n keys across all 8 locales; removed a now-unused eslint-disable Verified in-browser against MAIC gateway: 16 models fetched, balance shown. * feat(settings): add usage dashboard to System Settings - usage-dashboard.tsx: echarts dual-axis daily trend (tokens + cost), totals cards, by-model table, refresh; reads GET /api/usage - mounted at the top of GeneralSettings (系统设置) - honest disclaimer + costIncomplete marker when a model lacks pricing - i18n keys across all 8 locales Verified in-browser: shows 1 request / 31 tokens / $0.0003 from a prior call. * feat(token-plan): multi-modal one-click setup in System Settings - token-plan-presets.ts: presets now declare per-modality targets (llm/image/video/tts/webSearch); MiniMax is the full-set template, others LLM-only — extend by adding entries (at-our-best adaptation) - apply-token-plan.ts: fills one key into every declared modality via injected store setters, isolating per-modality failures (TDD, 4 tests) - token-plan-settings.tsx: new sidebar page — pick plan, enter key, one-click apply lights up adapted modalities (+LLM model probe), shows 'not adapted yet' for the rest, balance bar reused - reverted add-provider dialog to plain custom form (preset picker moved here) - i18n across all 8 locales Verified in-browser: Token Plan page renders, MiniMax shows all 5 modalities, apply/balance UI wired. 859 tests pass, build clean. * feat(token-plan): add custom token plan entry to Token Plan page - Custom card at the bottom of the provider list: pick it to manually enter name + protocol + baseURL (LLM-only), then key + one-click apply via the same applyTokenPlan flow (mirrors cc-switch's custom provider) - effectivePreset unifies preset vs custom for apply/balance/probe - i18n keys (customGroup/customName/customHint) across all 8 locales Verified in-browser: custom card expands the manual form; preset flow intact. * feat(token-plan): reflect persisted config on the Token Plan page Other settings panels read the store directly, so they survive section switches; the Token Plan page only wrote the store, so it looked blank on return. Now it also reads providersConfig: - selecting a preset prefills its saved API key - configured presets show a '已配置/Configured' badge State persistence was never broken (apply writes zustand→localStorage); this fixes the missing read-back so the page reflects it. * fix(settings): isLLMProviderConfigured crashed on providers without models Root cause of 'token plan config disappears': applyTokenPlan writes a new LLM provider with no models yet (probe fills them later). isLLMProviderConfigured did config.models.length unguarded → threw inside setProviderConfig's resolver → the whole set() aborted → key/baseUrl/type never persisted. - guard models in isLLMProviderConfigured (shared validator; affects any provider written without a models array) - seed models:[] in applyTokenPlan's LLM write for a valid initial shape - regression tests: validator no longer throws; apply+probe write keeps apiKey Verified in-browser: DeepSeek persists key and shows 已配置 after section switch. * feat(token-plan): add remove/teardown for a configured token plan Apply had no inverse. Add removeTokenPlan: clears the API key and disables (enabled:false) every modality the plan declared; a custom LLM provider is deleted entirely (removeProvider), built-ins keep the cleared shape. - removeTokenPlan in apply-token-plan.ts (mirrors applyTokenPlan, isolated per-modality, injected setters; 3 tests) - trash button on configured preset cards in the Token Plan page; resets page state if the removed plan was selected - i18n 'remove' across all 8 locales Verified in-browser: removing DeepSeek empties its key and drops the 已配置 badge. * feat(usage): track multimodal usage, drop cost/pricing Reframe usage stats as pure usage (no cost), per user decision: - usage-storage: drop all cost fields; add kind (llm/image/video/tts/asr) + quantity + unit; LLM keeps token counts, others store quantity - delete lib/usage/pricing.ts + pricing-defaults.json + its test - instrument image/video/tts at the server-only API routes (not in the provider dispatch — those files are in the client graph and importing fs-backed usage-storage broke the client bundle) - /api/usage: aggregate by model/day/modality, no cost - dashboard: per-modality usage (tokens/images/seconds/chars), token-only daily trend, no '$'; i18n cost keys replaced with modality/unit keys - backward compatible: legacy rows (no kind, stray cost fields) read as llm 912 tests pass, build clean. * feat(usage): per-modality dashboard layout + softer dark-mode chart - group usage into per-modality sections (LLM/image/video/tts/asr), each table's usage column uses one consistent unit (token/image/sec/char) — no more mixed units in a single column - summary chips per modality with their own unit - trend chart now plots daily REQUESTS (unit-agnostic, works for any modality) instead of LLM-only tokens - theme-aware chart: faint thin line + soft gradient area, muted axis/grid colors via useTheme — fixes the harsh solid stroke in dark mode Verified in-browser (dark): TTS shows '字符', LLM shows 'Token', separate sections; chart no longer has a hard line. * refactor(usage): dedupe model/fetch/usage helpers, parallel balance probe Review cleanups on the token-plan/usage branch: - extract modelInfoFromId() (shared vision heuristic + ModelInfo shape) - extract fetchWithTimeout() shared by model-fetch and balance-providers - extract recordGenerationUsage() to dedupe the image/tts/video routes - queryBalance: fetch billing subscription + usage in parallel - parseOneApiBilling: report quota without remaining when usage endpoint is unavailable, instead of implying zero spend (full balance) - split the two jammed imports in settings/index.tsx * feat(token-plan): drop custom token-plan support Custom token plans (manual baseURL/protocol entry) added complexity for little gain — a one-off provider is better configured directly on the Providers page. Token Plan is now preset-only: - remove custom mode, manual fields, and the custom card from the UI - collapse effectivePreset back to the selected preset - drop the now-dead removeProvider action + custom-id branch in removeTokenPlan - remove orphaned customGroup/customName/customHint i18n keys (8 locales) * feat(token-plan): add Volcengine/Tencent/Bailian plans, drop balance feature Add three vendor token-plan presets (all map to existing built-in LLM providers, so it's data-only — no new adapters): - 火山方舟 Volcengine Ark → doubao, OpenAI /api/v3 - 腾讯 TokenHub Token Plan → tencent-hunyuan, OpenAI /plan/v3 (the plan-specific base; /v1 is the pay-as-you-go gateway) - 阿里百炼 Token Plan → qwen, cross-model plan (Qwen + DeepSeek/Kimi/ GLM/MiniMax) on one key; model list is probed/entered Remove the balance/quota feature entirely — we now track usage, not cost, and every vendor's balance query needs its own cloud AK/SK + signature (Volcengine SigV4 / Tencent TC3 / Aliyun BSS), which the Bearer-key billing-endpoint probe never supported anyway: - delete lib/usage/balance-providers.ts, /api/provider/balance, its test - strip the Check Balance button + balance bar from the token-plan page and the provider config panel - remove the 4 balance i18n keys across all 8 locales - restore an eslint-disable the branch had dropped in provider-config-panel * feat(token-plan): use cloud-brand logos for vendor token plans The three vendor plans are cloud offerings, not single-model products, so icon them with the cloud brand rather than a model logo: - 火山方舟 → volcengine.svg (was doubao.svg) - 腾讯 TokenHub → tencentcloud.svg (was hunyuan.svg) - 阿里百炼 → alibabacloud.svg (was bailian.svg) Logos are the colored brand variants from lobehub/lobe-icons, matching the existing colored-logo style (plain <img>, no dark:invert needed). * feat(token-plan): keep only MiniMax and Volcengine presets Trim the token-plan list to the two we want to ship: MiniMax (full-set template) and 火山方舟 Volcengine Ark. Drop the Tencent/Bailian plans and the OpenRouter/SiliconFlow/DeepSeek/GLM/Qwen entries. - remove the now-unused tencentcloud.svg / alibabacloud.svg logos (volcengine.svg stays; the other logos are still used by the provider registry) - retarget the LLM-only apply test from the deleted deepseek preset to volcengine-ark * feat(token-plan): restore aggregator/third-party presets Previous commit over-trimmed: the intent was to drop only the Tencent and Bailian token plans, not the OpenRouter/SiliconFlow/DeepSeek/GLM/Qwen entries. Bring those back; keep only Tencent/Bailian removed. - token_plan: MiniMax, 火山方舟 Volcengine Ark - aggregator: OpenRouter, SiliconFlow - third_party: DeepSeek, GLM, Qwen Revert the apply test back to the deepseek fixture (restored). tencentcloud.svg / alibabacloud.svg stay deleted (their plans are gone). * fix(token-plan): point Volcengine plan at the Coding Plan endpoint The plan's ark--prefixed API keys authenticate only against /api/coding/v3, not the general /api/v3 endpoint — the latter rejects them with "The API key format is incorrect", so model probing returned nothing. Switch the base URL to https://ark.cn-beijing.volces.com/api/coding/v3. * fix(token-plan): Volcengine is an Agent Plan (Anthropic /api/plan) Per the Ark Agent Plan docs, the ark--prefixed keys authenticate ONLY against the dedicated Anthropic-compatible base https://ark.cn-beijing. volces.com/api/plan ("其他 Base URL 无法在 Agent Plan 中使用"). The general /api/v3 and the Coding Plan /api/coding endpoints both reject the key as "API key format is incorrect", which is why model probing kept returning 0. - baseUrl → https://ark.cn-beijing.volces.com/api/plan/v1 (the /v1 lets the Anthropic SDK land on /api/plan/v1/messages) - apiFormat → anthropic - rename to 火山方舟 Agent Plan Probe still targets /api/plan/v1/models (the path exists); if the Anthropic gateway doesn't return an OpenAI-shaped list, users fall back to typing a model id like ark-code-latest. * fix(token-plan): Volcengine Agent Plan = OpenAI /api/plan/v3 + ark-code-latest Settled after probing the real key and reading cc-switch's approach: - The ark- plan key works on the OpenAI-compatible /api/plan/v3 endpoint (chat/completions returns 200); switch apiFormat back to openai. - The plan exposes NO /models list (every /api/plan/*/models is 404), which is why probing kept returning 0. cc-switch handles this by hardcoding a single ark-code-latest (an auto-routing alias valid on any tier) and does NOT use AK/SK for model listing — so we do the same. - Seed defaultModels: ['ark-code-latest'] only; users add specific ids by hand. Supporting machinery (kept, general-purpose): - applyTokenPlan seeds models from defaultModels instead of wiping to [] - handleApply uses defaultModels and skips the doomed probe when present - drop stray .playwright-mcp/ debug artifacts and gitignore them * style: fix prettier formatting in usage files CI runs prettier on the whole repo (prettier . --check); these four files predate this branch's formatting pass and tripped the check. * feat(token-plan): verify Volcengine Agent Plan's published model set The Agent Plan publishes a fixed model set but exposes no /models endpoint, so carry the documented models as CANDIDATES and verify each on apply: - add verifyModels flag to TokenPlanModalityTarget - new /api/provider/probe-chat-models route: sends a minimal chat request per candidate (OpenAI /chat/completions or Anthropic /messages) in parallel, returns the subset that succeeds; SSRF-guarded, auth-failure short-circuits - handleApply gains a verify branch (before the fixed-defaultModels fast path), falling back to the seeded list if verification fails - Volcengine preset now carries the 12 published Agent Plan text models (doubao-seed-2.0-*/deepseek-v4-*/minimax-m*/glm-5.2/kimi-k2.*) as candidates This auto-prunes retired (docs flag deepseek-v3.2/glm-5.1 as 即将下线) and tier-gated models without code changes. Verified all 12 resolve against a real plan key. * feat(token-plan): wire Volcengine Agent Plan image + video modalities Make the Ark seedream/seedance adapters path-configurable and light up the image/video modalities on the Volcengine plan: - seedream/seedance adapters: resolveArkRoot() uses baseUrl verbatim when it already carries an /api/... path (token plan's /api/plan/v3), else appends the standard /api/v3 — no regression for the pay-as-you-go default host. - applyTokenPlan: image/video branches inject a modality's defaultModels as customModels and set them as the active provider+model, so generation works out of the box. New optional setImageProvider/ModelId + setVideoProvider/ ModelId actions (UI passes the store setters; tests omit them). - Volcengine preset declares image (doubao-seedream-5.0-lite, verified 200 on /api/plan/v3/images/generations) and video (doubao-seedance-2.0/1.5-pro — Medium+ tiers only; lower tiers reject at call time, no code change needed to upgrade). Applying the plan overwrites the shared seedream/seedance slot with the plan config (same overwrite model as LLM); switching back to pay-as-you-go is a manual edit or plan removal. Verified image end-to-end with a real plan key. * feat(token-plan): verify image/video models on apply, disable unsupported tiers The Volcengine plan lit up video optimistically, but lower tiers (Small) don't include video — so using it 404'd with UnsupportedModel. Probe media models on apply and only keep what the tier actually supports: - generalize /api/provider/probe-chat-models with a `kind` (chat|image|video): image hits /images/generations, video hits /contents/generations/tasks with empty content. The model-support check (404 UnsupportedModel) runs before any billable work, so probing never starts a real image/video job; for media, "supported" = any non-404 response. - handleApply: after lighting up image/video, probe each verifyModels modality; prune to the verified model set + re-select a working model, or disable the modality entirely if none pass (no false "available"). - Volcengine preset: image/video targets gain verifyModels: true. - add settings.tokenPlan.tierUnsupported across 8 locales. Verified with a real Small-tier key: image (seedream-5.0-lite) passes and is kept; video (seedance-2.0/1.5-pro) 404s and is disabled. * feat(web-search): add Doubao (豆包搜索) provider Doubao Search (Custom 版) over its REST endpoint POST open.feedcoopapi.com/search_api/web_search with Bearer auth — the same endpoint the askecho-search-infinity MCP server wraps, so the Volcengine Agent Plan key authenticates directly. Mirrors the MiniMax adapter: maps Result.WebResults to WebSearchSource (prefers Summary, the query-relevant excerpt, over Snippet for LLM use) and surfaces errors from ResponseMetadata.Error. - register 'doubao' in WebSearchProviderId + WEB_SEARCH_PROVIDERS - searchWithDoubao adapter, searchWeb dispatch, store default config - SSRF allowlist entry for the search host * feat(audio): support Agent Plan single-key auth for Doubao TTS generateDoubaoTTS now picks auth + endpoint from the key shape, since Volcengine exposes Seed-TTS as two products with separate credentials (verified: a plan key 401s on the normal endpoint, and the plan endpoint rejects appId-style auth): - single key (no colon) -> X-Api-Key, for the Agent Plan /plan endpoint - appId:accessKey -> X-Api-App-Id + X-Api-Access-Key (unchanged) A malformed pair (empty half) fails clearly instead of sending an empty header. Reuses the existing NDJSON/base64-mp3 parsing and voice list. * fix(media): map MiniMax video 720p to its real 768P tier normalizeVideoOptions defaults minimax-video to '720p' (the first supported resolution), but Hailuo 2.3 only accepts 768P/1080P and rejects 720P with '2013 ... does not support resolution 720P'. MiniMax's mid tier is 768P, not 720P (the adapter already falls back to 768P, as does the connectivity test), so map the shared enum's '720p' to 768P. Regression tests lock the mapping. * feat(token-plan): add web search + TTS to Volcengine Agent Plan, widen image tiers Extend the volcengine-ark preset now that the adapters exist: - webSearch -> doubao (own host open.feedcoopapi.com, not the ark endpoint) - tts -> doubao-tts on the /api/plan/tts endpoint (single-key auth) - image defaultModels widened to a best-first Seedream 5.0/4.5/4.0 list so a higher tier keeps the strongest model while verifyModels prunes the rest; video keeps the 2.0 + 1.5-pro candidates Comments record the verified host/auth quirks of each modality. * feat(token-plan): show result panel only after probing, with two clear states Addresses review feedback that a green check implied generation works when it only meant 'configured'. The panel now renders after probing finishes (gated on results && !applying) so it reflects the final set, and uses two states: green when the modality is configured/usable, muted when a live probe proved it unavailable (e.g. video on a tier without it). * feat(token-plan): scope presets to true multi-modal token plans Drop the single-modality LLM presets (OpenRouter, SiliconFlow, DeepSeek, GLM, Qwen) from Token Plan. A token plan's defining trait is one key spanning many modalities; those entries are ordinary LLM API providers already covered by the add-provider flow, and listing them here muddied the 'one key, every modality' promise. Only MiniMax and the Volcengine Ark Agent Plan remain. The UI already hides categories with no entries. apply-token-plan's LLM-only test now uses a local fixture instead of the removed deepseek preset. * feat(token-plan): progressive reveal of probe results on apply The result panel previously rendered all at once after probing finished, reading as dead air during the model probe. Now rows appear immediately on Apply: modalities with a live probe in flight show a spinner ('pending') and resolve to lit/failed independently as each probe returns, while non-probe modalities show lit right away. Probes run in parallel (Promise.all) instead of sequentially. A row only turns green once its own probe confirms, so this reveals structure + live progress without a premature green — complementing the earlier 'render only after probing' intent rather than reverting it. Per review feedback from @wyuc on #784. * fix(token-plan): enrich seeded models with built-in thinking capability Token Plan built ModelInfo objects from probed ids via modelInfoFromId(), filling only streaming/tools/vision — so a model that supports configurable thinking lost capabilities.thinking and InlineThinkingControl was hidden. modelInfoFromId now takes an optional providerId and overlays the catalog thinking capability for that (provider, model) pair; applyTokenPlan does the same for its synchronously-seeded list. Added the Ark Agent Plan's dotted aliases to the metadata table: - native Doubao Seed 2.0 family (doubao-seed-2.0-pro/code/lite/mini) - cross-vendor models the plan serves through its OpenAI-compatible endpoint (deepseek-v4-pro/flash, glm-5.2, kimi-k2.7-code/k2.6, minimax-m3/m2.7, ark-code-latest) All verified against a live plan key: each accepts the gateway's unified reasoning_effort field (low/medium/high) and actually reasons. They share the doubao effort adapter, which disables via 'minimal' (not 'none') — matching what the plan endpoint accepts (it rejects reasoning_effort:'none'). Addresses review point #1 from @wyuc on #784. * Improve token plan capability setup UI * fix token plan setup flow * chore: prettier format tts-providers.ts |
||
|
jackefnandYanpeng Wang
|
3b7a0ca0c0 |
feat(document): add multi-format course material upload (#741)
* feat(document): add multi-format course material upload * fix(document): support office uploads with mineru cloud * fix(document): handle mineru cloud office edge cases * fix(document): align mineru cloud artifact metadata --------- Co-authored-by: Yanpeng Wang <yanpg.wang@gmail.com> |
||
|
|
cec3c6aaea |
Refactor widget actions into the scene action pipeline (#796)
* refactor: align widget actions with scene pipeline * feat(edit): migrate legacy teacherActions off interactive content on load The widget-actions pipeline no longer persists a teacherActions authoring layer; playback reads only scene.actions. Add migrateInteractiveContent to strip the dead field from existing documents as they load through migrateScene (wired into all store/classroom load paths). Satisfies the #794 forward-migration acceptance item. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(generation): default widget_setState.state to {} when the LLM omits it The removed TeacherAction->Action converter always defaulted state to {}. Restore that guard in the structured-output parser so an action missing state can't forward state: undefined into the widget iframe, where SET_WIDGET_STATE handlers dereference it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(edit): also strip nested widgetConfig.teacherActions in migration Legacy WidgetConfig variants each declared their own teacherActions, so existing documents can carry the dead field nested inside content.widgetConfig as well as at the top level. Extend migrateInteractiveContent to drop both, preserving all other widgetConfig fields. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(prompts): document stable widget-action selectors for diagram/game/viz3d The interactive-actions prompt documented stable target selectors only for procedural-skill and simulation widgets. Add the canonical selectors the diagram / game / visualization3d content prompts actually emit (node IDs via revealOrder; #zoom-in-btn / #canvas-container / #speed-slider / ...; #start-btn), plus a rule to use widget_setState or a speech-only beat instead of guessing a target when no stable selector is known (e.g. free-form code widgets). Hardens against widget actions silently no-oping on non-existent selectors. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
Yizuki_Ame
|
243e9f3a48 |
feat: auto-retry transient failures during scene generation (#788)
Adds automatic retry with exponential backoff for scene generation (content/actions/TTS): shared withGenerationRetry helper at the fetch boundary, smart error classification (retries 429/5xx/timeouts/network, fails fast on 401/403/404), AI SDK RetryError unwrapping, abort-aware backoff, per-route thinking config preserved, and upstream HTTP status surfaced for client retry classification. Fixes #755. |
||
|
杨慎andwyuc
|
0f44e9a46f |
Add PBL v2 runtime APIs and classroom UI (#799)
* feat: add PBL v2 runtime UI * fix: ship PBL v2 instructor avatar * fix: ship PBL v2 workspace logo * fix: prevent scenario PBL legacy fallback * chore: restore package lint comments * test: cover ordinary PBL legacy fallback * fix: front-load PBL single-call validation * fix: preserve PBL v2 content when assembling scenes --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
杨慎
|
09863af421 |
Add PBL v2 core schema and generation path (#795)
* feat: add PBL v2 core planner * chore: restore unrelated lint comments * fix: document PBL v2 scenario outline flags * fix: expose PBL scenario subtype in outlines * chore: clear PBL v2 completion stats lint warning * chore: address PBL instructor code quality comments * fix: address PBL v2 review findings * chore: restore unrelated lint comments |
||
|
wyuc
|
1d1ce80e04 |
feat(maic-agent): editor-agent line (v0) — Pro-mode "Edit with AI" (#777)
Promote the editor-agent line to main: read-then-act editor agent (read_scene_content, regenerate_scene, regenerate_scene_actions, edit_interactive_html), a routable maic-agent model stage, the agent sidebar UI, interactive-scene runtime-error capture, and the timeline/script editor. Cross-reviewed and smoke-verified; see #777. |
||
|
wyucandwyuc
|
267965d6ed |
feat(generation): infer concise courseTitle from outlines for readable course names (#756)
* feat(generation): infer concise courseTitle from outlines for readable course names
Generated courses were named after the raw user prompt (truncated to 500 chars
client-side, or 50 chars / first outline title server-side), which is long and
unreadable. The outline-generation step already understands the full course
semantics, so have it emit a concise `courseTitle` alongside `languageDirective`
and `outlines`, and use it as the stage name.
- Prompts (requirements-to-outlines, interactive-outlines): top-level JSON gains
a required `courseTitle` (≤30 chars, in the teaching language, noun phrase).
- Streaming route: extract `courseTitle` head-bound like `languageDirective`,
emit a `courseTitle` SSE event, and include it in the `done` event.
- Non-streaming outline-generator: parse and return optional `courseTitle`
(defensive trim + 120-char cap).
- Naming: stage.name = courseTitle || outlines[0]?.title || requirement fallback
(client preview path backfills stage.name after outlines resolve; server
classroom-generation uses it directly).
- Session: persist `courseTitle` on GenerationSessionState for reload safety.
- Tests: add courseTitle parsing coverage; e2e mock done event carries it.
Fully optional in the pipeline — every consumer falls back to the previous
behavior when the field is absent, so legacy/LLM-skipped cases are unaffected.
* fix(generation): make courseTitle propagation robust across all outline paths
Address review feedback on the courseTitle threading.
Prompts — align every outline-prompt schema statement to the 3-key shape.
The user-prompt "final reminder" / top-level shape blocks for
requirements-to-outlines, interactive-outlines, and task-engine-outlines still
demanded the old two-key {languageDirective, outlines} object; as the last
instruction the model reads, that caused it to omit courseTitle and the stage
name to fall back to the raw requirement. Task Engine prompts now emit
courseTitle as well.
Streaming route — normalize the streamed courseTitle (trim, ignore
whitespace-only, cap length) to match the non-streaming parser, and add a
full-buffer fallback so a title emitted after the outlines array or beyond the
head-scan window is still recovered before the done event.
Client — reset latched languageDirective/title on outline retry so a
succeeding attempt that omits them falls back instead of inheriting a failed
attempt's stale value; also carry courseTitle in the stream-end resolve path
that fires when the stream closes without an explicit done event.
courseTitle remains optional on every consumer path.
---------
Co-authored-by: wyuc <zdq1204@gmail.com>
|
||
|
wyucandClaude Opus 4.8
|
25cf58d19f |
feat(model): optional per-stage LLM model routing (#745) (#747)
* feat(model): optional per-stage LLM model routing (#745) Add a config-only stage -> model map consulted during model resolution, falling back to DEFAULT_MODEL when unset (zero behavior change unless opted in). - lib/server/model-routes.ts: parse/validate/cache MODEL_ROUTES env JSON; LLM_STAGES registry of routable stages; getStageModel(stage). - resolveModel: resolution order x-model > stage route > DEFAULT_MODEL > builtin; thread optional `stage` through resolveModelFromHeaders/FromRequest. - Wire each route's resolveModel call site to its stage; classroom-generation resolves generate-classroom and web-search-query-rewrite independently. - .env.example: document MODEL_ROUTES; unit tests for both modules. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): address cross-review findings for per-stage routing (#745) - classroom-generation: resolve the web-search-query-rewrite model lazily, only when explicitly routed and inside the web-search branch, with try/catch fallback to the classroom model. Fixes a regression where a misconfigured optional route (keyless/invalid provider) threw and aborted ALL classroom generation even with web search disabled; also avoids wasted resolution on the common no-search path and preserves classroom-model inheritance when unrouted. - resolve-model: type the stage param as LlmStage across resolveModel/ resolveModelFromHeaders/resolveModelFromRequest so a mistyped stage literal is a compile error instead of silently falling through to DEFAULT_MODEL. - model-routes: simplify the cache to a process singleton (matching provider-config), dropping the raw-string-keyed cache and redundant STAGE_SET. - .env.example: clarify x-model precedence and that a route to an unconfigured provider fails at request time (no startup validation), like a bad DEFAULT_MODEL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): stage route takes precedence over client x-model (#745) Cross-review (codex P2 + Claude tracer) found that the browser UI always sends its saved model as x-model, so the previous x-model > stage order made MODEL_ROUTES inert for normal UI traffic on exactly the heavy stages the RFC targets (scene-content, quiz, pbl-chat, chat). Flip resolution to stage route > x-model > DEFAULT_MODEL: a configured route is the operator's deliberate per-stage choice and wins, while unrouted stages still honor the client's x-model. Update docs and tests accordingly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(model): prettier-format changed files for #745 CI Run the repo formatter (pnpm check / prettier) on the changed routes, the new model-routes module, and the new tests so `prettier . --check` (CI) passes; also correct a stale precedence comment in the resolve-model test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): isolate routed model from client connection params (#745) When a stage route overrides the client's x-model and points at a different provider, the client-sent apiKey/baseUrl/providerType (for the client's model) must not bleed onto the routed provider — otherwise a routed Anthropic model would be constructed with the client's OpenAI providerType/key and fail. A routed model now resolves its connection params purely from server config, as if no x-model was sent. Unrouted stages still honor the client params. Adds tests covering both the drop (routed) and keep (unrouted) cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(model): correct graceful-degradation comment in classroom (#745) resolveModel does not throw on a missing key; clarify that a keyless route surfaces later in callLLM (degraded by the outer try/catch), while only a resolution-time failure is caught at the rewrite re-resolve. Comment-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(model): route scene-content per scene type (#745) Extend MODEL_ROUTES with composite keys scene-content:<type> for the four core scene types (slide/quiz/interactive/pbl). getStageModel now resolves composite `a:b` stages most-specific-first, trimming `:` segments, so scene-content:<type> falls back to the base scene-content route, then x-model, then DEFAULT_MODEL. The scene-content route derives its stage from outline.type. Backward compatible: with no composite key configured, behavior is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(model): per-stage thinking effort + drop hardcoded gpt fallback (#745) MODEL_ROUTES values may now be {model, effort} (not just a model string) to pin the thinking effort per stage. Arbitration mirrors model routing: a routed stage with effort set wins over the client's thinking (effort "none" disables); routed with no effort uses the model's default and drops client thinking; unrouted stages keep the client thinking. resolveModel is the single arbiter — body thinking is threaded in via resolveModelFromRequest. classroom-generation passes the resolved thinking into its callLLM calls so generate-classroom and web-search-query-rewrite honor route effort too. Also: removed the hardcoded `|| 'gpt-5.4-mini'` fallback in resolveModel — if no model resolves (no route / x-model / DEFAULT_MODEL) it now throws instead of silently picking a vendor default. model-routes: getStageRoute returns {model, effort?}; getStageModel delegates to it. Docs + tests updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(model): route value carries full ThinkingConfig (#745) Per-stage routing accepts the full unified ThinkingConfig in the route value ({model, thinking:{mode,effort,level,enabled,budgetTokens,excludeReasoningOutput}}) instead of just an effort string. The route's thinking is passed through resolveModel and normalized per the model's capability by callLLM, so budgetTokens (qwen), level (Gemini), enabled/mode, etc. all work. (qwen3.7-plus/max thinking capability now comes from upstream #753, so this no longer registers them.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model): honor routed thinking for chat-adapter; doc fixes (#745) Cross-review (codex P2 + Claude) found chat-adapter was the only routable stage that ignored the route's thinking: it took the routed model but built its own thinkingConfig from the request body, so operator-pinned thinking never applied and client thinking still leaked onto a routed model. Now pass the client thinking into resolveModel and use the resolved thinkingConfig (route-pinned for a routed stage, client otherwise; defaults to disabled for low-latency chat). Also: clarify DEFAULT_MODEL is now required for server-side stages (the hardcoded gpt-5.4-mini fallback was removed → resolveModel throws if nothing resolves), and fix a stale {model,effort}→{model,thinking} doc comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
jackefn
|
a7cb5b5813 | feat(document): add extractor provider foundation (#704) | ||
|
wyucandClaude Opus 4.8
|
3f80a92a28 |
fix(generate): make outline streaming + interactive post-processing linear (was O(n²)/O(K·L)) (#732)
Two streaming generation paths do super-linear work on the model output and can block the Node event loop (and balloon the heap) on large or pathological responses: 1. app/api/generate/scene-outlines-stream re-parses the ENTIRE accumulated buffer on every streamed chunk — extractLanguageDirective (full-buffer regex) and extractNewOutlines (char-walk from the array start). That is O(n²) over the stream; a ~300 KB response blocks the event loop for ~20 s. 2. lib/generation/interactive-post-processor convertLatexDelimiters restores <script> placeholders by rebuilding the whole HTML string once per block — O(blocks × length). A widget with many <script> tags blocks for tens of seconds and allocates GBs. Fixes: - extractNewOutlines: parse incrementally, resuming from a cursor so the buffer is scanned once across the stream (O(n)). Behaviour-equivalent — verified across chunk sizes, markdown fences, flat arrays, prose preambles, and braces/escapes inside string values. - extractLanguageDirective: bound the regex to the buffer head (the directive is the first wrapper key), O(1) per chunk. - Restore <script> blocks in a single pass with a function replacer ($-safe). - Stream-loop hardening: honor request abort and propagate the AbortSignal to the LLM call so upstream generation stops on disconnect; cap the accumulated buffer; guard controller.close() against double-close. Standalone repro (copied functions): 300 KB outline stream 20.8 s -> 1.3 s; 45 MB widget HTML >30 s + OOM -> 45 ms. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
RongHuiQiang
|
50c0aefd13 | fix(vocational): gate procedural content generation | ||
|
RongHuiQiang
|
170b7df98f | merge upstream/main into vocational task engine branch | ||
|
RongHuiQiang
|
e6debb093b | fix(vocational): address review feedback | ||
|
|
3ccd5da6da |
feat(generation): opt-in parallel scene-content generation (#660)
* feat(generation): opt-in parallel scene-content generation Scene generation was strictly serial — content -> actions -> TTS, one outline at a time — so an N-scene classroom paid N x (content + actions + TTS) of wall-clock latency, which dominates the post-outline wait. Add an opt-in hybrid two-phase path (#572): - Phase 1 fetches scene *content* concurrently (bounded). Content is the only per-scene step independent of cross-scene state, so it is safe to parallelise; a content failure marks just that outline and does not pause the batch. - Phase 2 keeps the existing in-order actions + TTS loop, so previousSpeeches threading, ordered addScene, the abort/epoch guards, and the pause-on-failure UX are all unchanged. Gated by a server-side PARALLEL_SCENE_CONCURRENCY (default 0 = off, clamped to 10), surfaced to the client through the existing /api/server-providers response. With it unset the content map is null and the loop is byte-for-byte the original serial path, so out-of-box behaviour is unchanged. The bounded worker pool is extracted to lib/utils/concurrency.ts (mapWithConcurrency, with an early-stop hook for abort/epoch) and unit-tested; getParallelSceneConcurrency env parsing/clamping is tested too. Closes #572 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(generation): pipeline content fetches instead of a barrier Address @wyuc's review on #660. The previous version awaited the whole bounded content pool before Phase 2, so for N > C scenes the user saw the first scene, then a stall on a single page for the entire content phase, then a burst — perceived as worse than serial despite the wall-clock win (the gap scales with ceil((N-1)/C)). Pipeline instead: lazyBoundedMap starts the content fetches (<= C in flight) and returns one promise per outline immediately; the serial loop awaits them in order, so each resolves as soon as its own content is ready (usually already done, hidden behind the previous scene's actions/TTS). The first scene now paints after content(1)+actions(1)+TTS(1) — same as serial — with the full concurrency benefit and no stall. Also from the review: - a content fetch can no longer take its siblings down: an unexpected throw is caught and returned as a failure result, routed through the same mark-failed path as the serial loop (symmetric); - drop the cosmetic Phase-1 setCurrentGeneratingOrder — the loop already sets it per outline; - note the intentional belt-and-suspenders clamp; - concurrency.test.ts asserts <= limit (not == limit), and gains lazyBoundedMap tests including the no-barrier property. mapWithConcurrency is kept as a thin await-all wrapper over lazyBoundedMap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
d7c0d80afe |
feat(tts): per-agent auto-voice quality + register-once timbre stability (#670) (#672)
* feat(tts): voxcpm voice-design types + deterministic voice id helpers (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): emit + persist per-agent voiceDesign descriptor (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): voxcpm voice registration backend client (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): voxcpm-voice ensure/register endpoint (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): client auto-voice registration + reference-clip cache (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): reference registered voice id in vLLM-Omni speech (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): thread voiceDesign + backend through tts call sites (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): match vLLM-Omni voice registration contract (#670) Live e2e against the real backend revealed the assumed multipart contract was wrong: POST /v1/audio/voices needs name + consent + audio_sample (not voice_id/file), and there is no per-name GET (405) — existence must list /v1/audio/voices and check membership. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(tts): make auto-voice register-once pattern provider-neutral (#670) The descriptor + register-once/reference-by-id pattern is not VoxCPM-specific. De-couple it into a provider-neutral seam so other registration-capable providers (ElevenLabs/MiniMax/Doubao voice cloning, …) can plug in: - lib/audio/voice-design.ts: VoiceDesign + buildVoiceDesignPrompt / normalizeVoiceDesign / getDeterministicVoiceId (neutral 'auto-<hash>' id, namespaced by providerId). - lib/audio/voice-registration.ts: VoiceRegistrationAdapter interface + providerId->adapter registry + supportsVoiceRegistration. - lib/audio/voice-registration-client.ts: neutral ensureRegisteredVoice() + IndexedDB clip cache. - app/api/generate/voice (replaces .../voxcpm-voice): dispatches by providerId. - voxcpm-registration.ts becomes the VoxCPM adapter (sole registered provider). - AgentConfig.voiceDesign + DB cache table renamed neutral. Behavior-preserving; VoxCPM-specific bits (inline (prompt)text, backend kinds, capability gate) stay in the voxcpm modules. Full suite green (674). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): build the voice clip Blob from an inline Uint8Array (#670) A factored-out helper returning Uint8Array widened to Uint8Array<ArrayBufferLike>, which next build (stricter than bare tsc) rejects as a BlobPart. Inline the buffer like the rest of the repo does. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): managed TTS providers resolve model server-side (#670) A server-managed TTS provider's model was still client-driven, but the managed-provider settings UI hides the model field — so a backend whose model id isn't the client default (e.g. VoxCPM/vLLM-Omni serving the full model path) always 500'd. resolveTTSModel() makes the model authoritative from server config (${PREFIX}_MODELS, first entry) for managed providers, like key/baseUrl; unmanaged/unconfigured providers keep the client model unchanged. Used by the tts and voice routes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): single agent-voice resolver + stable teacher narration voice (#670) Route all TTS voice resolution (narration, discussion, preview) through one resolveAgentVoiceOptions(agent, ...) that reads the agent profile and registers + references the voice by id. Eagerly warm up generated agents' voices on save. Critically fixes the teacher narration drifting (male/female jumps): the registry is always seeded with DEFAULT_AGENTS, so the old narration lookup find(role==='teacher') returned the default teacher (no voiceDesign) instead of the generated one — so narration never registered a voice and fell back to the inline prompt. pickNarratorAgent() now prefers the teacher carrying a voiceDesign. Also: drop language from the deterministic voice id (descriptor already encodes it) so narration (directive) and discussion (locale) resolve the same id; log the effective registeredVoiceId. Regression tests for pickNarratorAgent. Verified e2e: agent-profiles -> 1 voice registration -> 16 narration TTS all referencing the same registeredVoiceId; voice present on the backend. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(tts): fall back to persona as the voice seed for agents without a voiceDesign (#670) Preset/default agents carry no LLM voiceDesign. Rather than add a static field, derive the bootstrap descriptor from the agent's persona when voiceDesign is absent, so they still register one stable reference voice (stable-but-generic; persona is not a vocal spec). Generated agents keep their LLM voiceDesign. Also hardens replay: stage snapshots drop voiceDesign but keep persona. Unit-tested: resolveAgentVoiceOptions uses real voiceDesign when present, persona otherwise. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): match Auto Voice by its localized label in the agent voice picker (#670) The picker filtered on voice.name ('Auto Voice'), but Auto Voice is shown via its localized label (自动音色). Searching '自动' returned '没有匹配音色'. Match the localized label for the Auto Voice option so it's findable in any language. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): address code-review round 1 (#670) - strip parentheses in the voice-design prompt so a paren in the descriptor/ persona can't corrupt the (prompt)text bootstrap delimiter - gate POST /api/generate/voice on isServerTTSProviderDisabled (#665): a force-disabled provider was off for the tts route but not this sibling - check voiceExists before re-registering a client-cached clip, so a cached voice that's still live isn't needlessly re-uploaded every session - dedup concurrent ensureRegisteredVoice calls via an in-flight promise map (eager warm-up + first utterance no longer double bootstrap/register) - warm up only the narrator (teacher), not every generated agent, to avoid synthesizing voices at save time for agents that may never speak - collapse the duplicated vLLM-Omni speech payload into one object Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(tts): prettier formatting (#670) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): key auto-voice memo by (voiceId, backend) so a base-URL change re-registers (#670) Addresses review (cosarah): registeredThisSession/inFlight were keyed by voiceId alone, so switching the VoxCPM base URL mid-session made ensureRegisteredVoice short-circuit and return an id registered only on the old backend — TTS then sent that stale registeredVoiceId and skipped the inline fallback, failing with voice-not-found on the new backend. Memo key now includes the base URL; the IndexedDB clip cache stays keyed by voiceId (the reference clip is backend-independent and reused to re-register elsewhere). Regression test added. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tts): include API key in the auto-voice memo key (#670) Addresses review (cosarah, round 2): memoKeyFor was keyed by (voiceId, baseUrl) but not the API key. Registration/existence checks and speech calls are auth-scoped, so switching account/key on the same base URL could reuse a registeredVoiceId from the old credentials and skip re-validation. Memo key now includes the API key (in-memory only, never persisted/logged). Regression test covers the key-switch case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
jackefn
|
c58893e76f | feat(vocational): add task engine and procedural skill widgets | ||
|
|
3e1e9f083b |
feat(tts): unify provider-enablement model, default browser-native off (#665) (#666)
A single TTS provider-enablement model honored everywhere (picker,
auto-assignment, fallback, lecture/playback, server).
Model
- lib/audio/provider-enablement.ts: isTTSProviderConfigured /
isTTSProviderEnabled / listEnabledTTSProviderIds. Availability drops the
registry defaultBaseUrl fallback (fixes Lemonade's always-shown voices),
honors the per-provider `enabled` flag, server-disable overrides (server
precedence).
- voice-resolver: getEnabledProvidersWithVoices (enabled server/custom
providers) and getSelectableProvidersWithVoices (the ONE client source of
truth = enabled providers + opt-in browser-native when the browser exposes
voices). resolveAgentVoice picks deterministically among the given providers,
honors a voiceConfig only while its provider is present.
- Picker + discussion build their list from getSelectableProvidersWithVoices, so
the AgentBar and the live discussion agree on selectable voices (fixes
student agents going silent in auto mode). Students resolve via
resolveAgentVoice; the teacher uses the global lecture voice verbatim (same
voice the pre-generated lecture sends), so lecture and discussion never
diverge.
- Voice assignment vs fallback: an agent's voice is bound at generation (the
agent-profile LLM picks from availableVoices; preset uses the user's pick).
The fallback for a missing/invalid/unavailable voice is applied only at the
live TTS calls, not baked into the saved agent — so it reflects current
provider availability.
Defaults / opt-in
- Browser-native: first-class, default OFF, opt-in. Built-in providers default
enabled:true (only surface once configured).
- TTS master toggle defaults OFF; first server-sync auto-enables it when a
provider is configured (mirrors image/video).
- Persist v3->v4 migration normalizes the never-user-editable flags.
Server/admin disable (server precedence): server-providers.yml `enabled:false`
and env `TTS_<P>_ENABLED=false` (incl. browser-native). Surfaced as a per-provider
`disabled` flag on /api/server-providers; merged as serverDisabled; enforced at
the /api/generate/tts trust boundary, classroom media generation, and /api/health.
UI
- Settings TTS panel: per-provider enable toggle, disabled (with a hint) when the
provider is unavailable, locked off (with a note) when server-disabled.
- AgentBar voice pills render disabled with a "No voice" placeholder when no
provider is enabled. The toolbar media-popover TTS tab keeps its existing
on/off toggle (no picker/CTA).
Tests: predicates, resolver determinism + unified selectable list, server
force-disable, store merge + migration + TTS auto-enable, i18n parity. Verified
in a real browser (Playwright): settings toggle states + a full Gemini classroom
generation whose TTS requests used the picked voice.
Found on main @
|
||
|
|
2586cb5e05 |
fix(tts): surface provider rate-limit errors as HTTP 429 (#644)
TTS providers throw a plain `Error` on any non-OK response, and the `/api/generate/tts` route catches everything as `GENERATION_FAILED` (500), so an upstream 429 (quota/concurrency) was indistinguishable from a real failure — clients had no signal to retry/back off. `TTSRateLimitError` already existed for exactly this (with a TODO noting the route collapses everything uniformly), but only Doubao threw it and the route never mapped it. - Add a `RATE_LIMITED` API error code. - Map upstream HTTP 429 to `TTSRateLimitError` at every provider's `!response.ok` site via a shared `throwIfTtsRateLimited` helper. - Return `429 RATE_LIMITED` from the TTS route when a `TTSRateLimitError` is caught (all other errors unchanged). Purely additive: non-429 responses keep their existing error messages. Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyuc
|
2165e55bac | feat: add MiniMax web search provider (#634) | ||
|
|
35d36905d1 |
Make server-configured providers admin-managed, not client-overridable (#620) (#624)
* refactor(providers): make server-configured providers admin-managed, not client-overridable A server-configured provider is now authoritative: the operator owns its API key and base URL, and the client cannot override either. This collapses the tri-state (server key / client base URL / client key) that was the root of a recurring "missing API key" bug class — when the settings UI echoed a base URL with an empty key, routes that treated "any base URL ⇒ fully client-configured" dropped the server key. #533 patched that per-route via canUseServerApiKeyForBaseUrl; this removes the guard entirely by fixing the design. Server: - resolve*ApiKey/BaseUrl ignore client values when the provider is server- configured; add isServerConfiguredProvider(section, id); drop canUseServerApiKeyForBaseUrl + normalizeProviderBaseUrl. - getServer*Providers() no longer return base URLs — only allowed models (LLM/image) and the managed flag (presence) — so an internal gateway/proxy endpoint is never exposed to the client. - routes (chat/image/video/tts/asr/pdf + verify-*) resolve managed providers purely server-side; SSRF validation now applies only to unmanaged (client) base URLs. Client: - drop serverBaseUrl from ProviderSettings, the settings store, request-building (TTS base URL no longer echoes the server URL) and voice resolution. - settings UI renders managed providers read-only: the key/base-URL override inputs and region toggles are hidden, leaving the "managed by server" notice. Tradeoff: overriding a server provider's key with your own for the same provider is removed; add a separate custom provider instead. Closes #620. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers): refresh managed-provider copy, drop dead audio-settings, add #620 e2e - serverConfiguredNotice now says the provider is managed and not editable here, pointing users to add a separate provider for their own credentials (7 locales). - delete components/settings/audio-settings.tsx — unreferenced dead code. - add e2e/tests/managed-provider-620.spec.ts: managed provider hides the key/base-URL inputs; unmanaged provider keeps them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(providers): lock the model catalog for managed providers with a pinned list When the operator pins an allowed model list (MODELS env/yaml ⇒ serverModels), the model catalog is admin-managed too: the LLM provider panel shows the pinned models read-only with a "Server" badge and no add / reset / edit / delete affordances. Without a pinned list the server manages only credentials and the user keeps curating models. This removes the prior inconsistency where the settings panel let you add models to a pinned managed provider that the generation toolbar then filtered out. e2e: the managed-openai case now also asserts the catalog is locked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers): ignore (not reject) client overrides for managed web-search; clear legacy serverBaseUrl Addresses review feedback on #624: 1. Managed web-search providers now ignore a client-sent key/baseUrl instead of validating and rejecting it. A server-configured provider is admin-owned, so the server config is authoritative and a stale client base URL is dropped rather than 400-ing the request. The allowlist validation still runs for unmanaged providers, where the client base URL is actually used. 2. Drop the removed `serverBaseUrl` field from persisted localStorage on rehydrate (settings version 2 -> 3, stripped in both migrate and merge), so old clients don't keep a stale server URL in client state. Tests: web-search route test split into unmanaged-rejects-allowlist and managed-ignores-client-baseUrl; settings rehydrate test asserts a legacy serverBaseUrl is stripped. Full suite 381 pass; e2e 2 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
|
86c8e0c543 |
feat: inline external assets at export for offline classrooms (#596) (#613)
* feat(export): add external-asset reference scanner for HTML inlining * feat(export): add cached asset fetcher and data-uri encoder Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(export): recursively inline url() refs inside stylesheets Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(export): orchestrate inlining of links, scripts, images, and inline css Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(export): drop dead escapeAttr, harden importmap skip in scanner * feat(export): inline importmap modules incl three/addons graph closure Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(export): extract shared module to break inline-assets import cycle Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(export): inline interactive-scene assets into .maic.zip at export Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(export): inline interactive assets in Resource Pack export too * docs: document offline export asset inlining * style: prettier formatting * fix(test): resolve tsc type errors in export-classroom-inline test * fix(export): keep importmap prefix fallback on partial failure, chunk base64, preserve media attr, log pptx inline failures Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(export): parallelize asset fetching, share fetcher across scenes, promise-cache Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(export): retry transient asset fetches, bound concurrency, warn user on partial inline Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(export): inline only woff2 fonts, drop woff/ttf siblings to halve size Within @font-face blocks that have a woff2 url(), rewrite sibling woff/ttf/otf/eot refs to url(about:invalid) instead of inlining them. Falls back to full inline for ttf-only blocks. Drops katex output from ~1.79MB to ~0.71MB (~3x fewer font fetches). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * i18n(export): translate partial-inline warning and surface failed asset hosts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(proxy-media): follow redirects with per-hop SSRF revalidation (#398) Replace blanket redirect rejection with a bounded 5-hop follow loop that re-validates every Location target through validateUrlForSSRF before fetching, preventing redirect-to-internal SSRF while allowing CDN redirects (e.g. cdn.tailwindcss.com 302→/3.4.17). * feat(export): route asset inlining through proxy-media to bypass CORS Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(export): keep importmap prefix fallback, forward proxy 4xx, cap proxy size, re-export FetchAsset * style: prettier formatting for proxy-media + proxied-fetch * fix(export): report CSS-nested asset failures in inline report Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
LooThaoandwyuc
|
6d29bbe9af |
Fix server API key fallback when client echoes provider base URL (#533)
* Fix server API key fallback for provider base URLs * Format verify PDF provider route --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |