Reject classic inline interactive scripts that fail to parse at generation time (extracted with parse5, checked with node:vm Script without executing), and surface iframe runtime errors on the active interactive scene.
Addresses #1622 (partial: no recovery / regeneration UI).
Co-authored-by: Frank-zhu0404 <Frank-zhu0404@users.noreply.github.com>
* fix(tts): pace classroom TTS and classify MiniMax RPM limits
MiniMax reports RPM limits as HTTP 200 with base_resp.status_code 1002.
Classify that code as TTSRateLimitError, pace classroom TTS with
TTS_MIN_INTERVAL_MS (default 1000ms) and double the gap up to 15000ms on
each rate limit, and store ttsCoverage on the generation job so a partial
narration is not a clean success.
Fixes#1567
* fix(tts): fail closed on MiniMax status and skipped narration
Treat every non-zero MiniMax base_resp status as a failure, even when
audio bytes are present. Classify 1002, 1039, 1041, and 2045 as rate
limits. Floor rate-limit back-off independently of TTS_MIN_INTERVAL_MS,
cap the cumulative wait, and cancel those sleeps on abort. A skipped or
thrown TTS phase reports zero coverage and a warning.
* fix(tts): decay spacing, heartbeat TTS progress, default interval 0
Classroom TTS spacing steps back toward TTS_MIN_INTERVAL_MS after each
successful clip, and that interval defaults to 0 so pacing starts only
after a rate limit. Clip progress is reported through the existing job
onProgress path, so updatedAt keeps moving during a long narration phase.
A Retry-After that does not fit the remaining back-off budget skips that
clip instead of silencing every later clip.
generateTTSForClassroom always returns coverage. A skipped run that has
resolved a provider counts clips after the same speech split as synthesis.
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(web-search): stop leaking Brave's HTML challenge page into errors
Both Brave error paths echoed the upstream response body verbatim, so a
throttled or blocked request surfaced a full HTML page as Error.message
and a transient 429 was indistinguishable from a real outage.
Route both paths through formatBraveError, which gives 429 a dedicated
actionable message, drops markup and over-long bodies in favour of the
HTTP status text, and keeps short plain-text details.
Closes#1552
* fix(web-search): cancel Brave's HTML error body instead of reading it
Deciding whether an error body is worth surfacing still meant pulling the
whole challenge page into memory first. Check the content type up front and
cancel the stream when it is markup, so the large body is never read.
* fix(web-search): give Brave API-key 429 plan-specific guidance
The API-key path already sends a subscription token, so its 429 message
should not suggest configuring a Brave API key. Point it at the plan's
rate limit instead, keep the API-key suggestion for the keyless scrape
path, and add a test for the API-key 429.
---------
Co-authored-by: zhentang2 <zhentang2@iflytek.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(workbench): show effective upload limit in friendly errors
* fix(workbench): format the upload limit shown in composer toasts
A non-MiB cap was stringified as a long fraction, and nothing checked that the toast used the localized message.
Course documents, folders, materials and agent sessions are partitioned by an
owner id that comes from a 30-day anonymous cookie, because there is no host
auth layer. For one team behind one ACCESS_CODE that partition buys nothing —
those visitors already share the site password — and it costs: a second browser
shows an empty course list, and POST /api/stages/[id]/publish refuses every
owner, because publishing a cookie partition is not something the product
allows.
PERSISTENCE_SHARED_OWNER_ID resolves every request to that fixed id instead.
Unset — the default — changes nothing.
- **ACCESS_CODE is required.** Without one the middleware lets every request
through, so a single shared owner would expose one readable, editable and
publishable course library to anyone who can reach the deployment. That is
worse than the per-browser partitioning this setting removes, so the
combination refuses to start, from instrumentation.ts, with a message naming
both variables. The alternative the issue allowed — ignore it with a warning —
was rejected because a silently ignored setting is exactly the confusion this
feature exists to remove.
- The value is validated rather than trusted: 1-128 characters of
[A-Za-z0-9._-]. That excludes the reserved `anon:` prefix, which would still
be refused by publish and would alias onto a cookie owner, and any character
the material-key sanitiser rewrites, which could collide two ids onto one
object key. A malformed value fails startup on the same path; an empty value
is treated as unset, since KEY= in an env file cannot be told apart from an
operator who meant to leave the feature off.
- An explicit authenticatedOwnerId still outranks it, so adding a real auth
layer later does not require clearing the variable first.
- All four places that derive an owner honour it, including the Server Action
in lib/workbench/workspace-actions.ts, which re-reads the cookie itself. That
one is easy to miss and would make the workspace list and its row actions
disagree about who owns a session.
- The env docs state the trust model next to the variable, cross-reference
SECURITY.md, and note that this answers who owns a course while DATABASE_URL
answers where one lives — a second browser sees an empty list on Postgres too.
This is the stopgap asked for in #1550 — a PR in the spirit of the closed
#1551 — with threading a real authenticated owner left as the long-term fix.
AI-assisted commit
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
`buildSkillDirZip` names each zip entry with `path.relative(dir, file)`
directly. On Windows that separator is a backslash, so an exported
skill archive contains entries like `openmaic/references\extend.md`
instead of `openmaic/references/extend.md`.
Zip entry names are POSIX paths by spec, and consumers — the skill
import path, `unzip`, and the round-trip test — resolve entries by
forward-slash paths, so an archive exported on Windows cannot be read
back as a skill package.
Normalize the relative path to forward slashes before naming the
entry. On POSIX platforms `sep` is already `/`, so this is a no-op
there.
Co-authored-by: BeAIcoder <beaicode@example.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
#930 made the connectivity probes in the media adapters pass
`redirect: 'manual'`. The generation and poll calls in the same 14 files were
left following redirects. Those requests carry the provider credential and go
to a base URL that comes from provider settings a caller can supply, so a 3xx
would replay the credential at a host the caller chose — and the redirect
target can be an address the outbound guard already refused.
Every such call now passes `redirect: 'manual'` and rejects a 3xx through a
shared `assertNotRedirected` helper, which reports it as
"<provider>: Redirects are not allowed (HTTP <status>)" instead of letting the
generic failure path describe it as a provider error.
- 26 call sites across the image adapters (seedream, openai, qwen, grok,
lemonade, minimax, nano-banana), the video adapters (seedance, kling, grok,
happyhorse, minimax, veo) and ComfyUI's submit and image fetch.
- ComfyUI's `pollHistory` keeps its contract of handing the caller a retryable
failure rather than aborting the generation: it logs the refusal and returns
null.
- ComfyUI's same-origin workflow load is deliberately untouched — it reads the
app's own public/ asset, carries no credential and is not provider-influenced.
- The two adapters added since #930 (OpenRouter image and video) already did
this.
tests/media/adapter-redirects.test.ts covers one case per adapter family. Each
serves a 302 and asserts that the call rejects with the redirect message and
that every request carrying an init object asked fetch not to follow redirects;
each case fails if its adapter stops passing `redirect: 'manual'`.
The HappyHorse test asserted the exact request init, so it now includes the new
option.
AI-assisted commit
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Two independent Grok failures seen when the provider is reached through a relay
(a custom base URL) instead of api.x.ai directly:
- lib/ai/providers.ts: a long non-streaming chat completion was cut off by the
relay with a 504 at its idle timeout (~5 min), because nothing is sent
upstream until the model has the whole answer. Adding 'grok' to the existing
streaming-compat path (OPENAI_COMPAT_USE_STREAMING_CHAT=true) keeps bytes
flowing across the idle window; the SSE is buffered back into a normal JSON
response for the caller.
The path is for relays only. usesCustomOpenAIBaseUrl recognises OpenAI's
origin alone, so Grok's own api.x.ai also read as "custom" and was forced
onto the compat transport; the provider's native endpoint is now excluded.
- lib/media/adapters/grok-image-adapter.ts: response_format 'url' returns a
link on the relay's CDN host (imgen.x.ai), which may be unreachable from the
server's network. The generation then failed at the follow-up fetch through
/api/proxy-media even though the image had been produced successfully.
'b64_json' inlines the bytes and removes that second hop.
Inline bytes declare no media type, so the adapter reports one on
ImageGenerationResult and returns a typed data URL, which is the shape
openrouter-image-adapter already uses. Consumers take the type from there:
agent image persistence records it, the client's stored media row keeps the
type its data URL states, and classroom-media-generation names the file with
the matching extension. A JPEG is no longer stored, served or named as a PNG.
The process-wide undici timeout that previously accompanied these changes is
dropped: upstream #1404 now gives LLM calls their own undici headers/body
timeouts, which covers the same failure without raising the defaults globally.
Co-authored-by: ciclou1 <ciclou1@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Share corrected line bounds across renderer and React editing paths, preserve double-elbow routing, and translate PPTX points into the shape bounding box.
Refs #674 and the prior implementation/review in #675. AI-assisted.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(media): emit keyframe images as data URLs
The local media extractor stored keyframe bytes as raw base64 in
`DocumentAsset.data`. Every other extractor emits a data URL, and the
document bundle forwards this field as `pdfImages[].src` to `storeImages`,
which decodes it with `decodeBase64DataUrl`. With no `data:` prefix the
comma split yields no payload, so `atob(undefined)` throws and course
generation fails with "Failed to store image bundle at image img_1".
`pdf-compat`'s `dataUrlMimeType` also derives the image asset mime from
this same field, so the declared `image/webp` was silently dropped too.
Emitting `data:${mime};base64,...` matches `mineru-parser`, which already
normalizes prefix-less base64 the same way.
* fix(media): decode keyframe data URLs
* fix(media): accept legacy and data-url assets
* fix(media): reject malformed or empty media asset data URLs
A string that starts with data: but does not parse as a data URL used to
fall through to the raw-base64 path, where Node's decoder skips
non-alphabet characters and yields garbage bytes. Throw instead, and
reject an empty base64 payload rather than storing a 0-byte image.
Correct the keyframe test comment: local keyframe assets are consumed by
material extraction, and the test needs ffmpeg so it does not run in CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(export): surface video render rejection reasons
* fix(export): keep render diagnostics out of user toasts
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Discussion narration created a new Audio element per line, so on mobile every
line after the first had its programmatic play() refused with NotAllowedError:
the dialogue went silent while the lesson kept advancing. This is the
discussion side of #1474, which #1477 fixed for the narration player.
One module-scoped element now serves every line, kept separate from the
narration element in AudioPlayer so neither can cut the other off. Because
element identity can no longer tell lines apart, the stale-event guard became a
per-line token, handlers are assigned rather than added (a listener would
accumulate once per line), and finish()/cleanup() release the line by removing
the source attribute and reloading instead of leaving the element pointed at
the page's own URL.
Co-authored-by: talkman <jaxgen@163.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
ACCESS_CODE unset remains fail-open in middleware, document reads are
capability-by-id via the anonymous owner cookie (not x-learner-key),
and the README action/skill counts match the Action union and
skills/agent-runtime.
Closes#1587
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
The vocational gate keys off requirements.taskEngineMode in the
/api/generate/scene-content body, but only the first scene
(generation-preview) sent it. Scenes 2..N and retries run through
useSceneGenerator.generateRemaining / fetchSceneContent, which never
sent requirements, so resolveVocationalActive returned false and
applyOutlineFallbacks rewrote every later procedural-skill scene to
diagram — silently dropping the task-engine training mechanism for
most of a vocational course (the outline says procedural-skill while
the content path strips it).
Thread the persisted stage.taskEngineMode through GenerationParams into
both fetchSceneContent bodies (the retry path inherits it via
lastParamsRef), mirroring what the first-scene request already sends.
Server contract is unchanged; the flag is still ANDed with the
OPENMAIC_ENABLE_VOCATIONAL env gate server-side.
Closes#715
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* refactor(upload): derive the workbench material MIME policy from the shared format registry
#1498 fixed the Kylin generic-Office-MIME failure on both upload paths but
left the workbench with its own extension→MIME table, alias map, and
generic-MIME set next to the document registry, and the two had already
drifted. The workbench policy now derives every MIME/extension fact from
lib/document/mime.ts (the single source of truth); only the accepted-format
list stays workbench policy — fixed and extractor-independent, unlike the
classic path's provider-scoped whitelist.
- Register csv and webm in DOCUMENT_FORMATS (accepted by no document
provider, so classic-mode whitelists are unchanged) and add the
audio/x-m4a alias to m4a.
- material-upload-policy.ts keeps its export names (route, session-store,
and composer consumers unchanged) but resolves, whitelists, and builds
its accept string from registry helpers.
- The workbench gate now also accepts the registry's curated aliases it
previously missed: image/jpg, text/x-markdown, and audio/x-wav (stored
canonically as audio/wav).
Closes#1589.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* test(workbench): pin the audio/mp3 alias closure in the material policy
Deriving the workbench gate from the shared registry normalization
(#1589) also accepts the browser-reported audio/mp3 alias the hand-rolled
alias map rejected — the same gap class as image/jpg and text/x-markdown,
so pin it alongside them and name it in the comment.
Co-Authored-By: Claude Code <noreply@anthropic.com>
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat: sample declared interactive state before classroom questions
* test: exercise generated publication example through the iframe reader
* chore(generation): bump package version for observation prompt contract
* fix: keep interactive state optional on insecure HTTP origins
* fix(playback): keep component picking and separate state from reference
A declared state interface replaced the component picker with a forced
whole-area `#experiment` reference, removing the per-component selection
and outline that `main` already ships. Sampling was also gated on the
reference selector, so the only way to obtain state was to give up the
selection.
Reference identity and area state are now independent request-scoped
evidence items:
- `handleToggleElementPick` always arms the picker again, so a scene that
declares the interface keeps main's per-component selection, outline,
and send-time clearing.
- `sampleInteractiveState` follows the current Scene instead of the draft
reference, so an unreferenced follow-up still reports current facts and
never re-creates or extends a reference.
- The Host carries area state with or without a component reference. The
evidence header names both identities and refuses to present area facts
as properties of the referenced component.
- `metadata` is absent when only area state travels, so no element
identity and no Spotlight authorization can be derived from it, and the
accepted-reference receipt stays driven by explicit references only.
Review follow-ups in the same change:
- Client sampling follows `NEXT_PUBLIC_COURSEWARE_REFERENCE_ENABLED`.
An ungated packet turned an ordinary Pi question into a 400 while the
reference feature was disabled.
- A Scene that declares the interface always receives an availability
boundary, including when the browser produced no packet at all. It is
reported as `not-sampled` rather than the previous `no-interface`,
which was a false statement about an activity that does declare one.
Courseware without the interface keeps its unreferenced behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(generation): register the observation snippet as a packaged asset
The interactive-observation snippet is referenced by all six widget
content templates but was never added to the packaged-asset manifest, so
the asset test and the golden scene prompt both failed.
- `SNIPPET_IDS` now lists `interactive-observation`, restoring both the
"exactly the generation-owned templates and referenced snippets" check
and the "every referenced snippet is packaged" cross-check.
- The interactive system-prompt snapshot is re-pinned. The change is
purely additive: the snippet is appended to the simulation template.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pi): stop injecting state constraints while the feature is disabled
The route rejects a request that carries a reference or a state packet
while `NEXT_PUBLIC_COURSEWARE_REFERENCE_ENABLED` is off, but an ordinary
question carries neither. It still reached the Host, and a Scene that
declares the state interface then received the full page-state block —
several kilobytes of constraints about evidence the deployment can never
sample.
The Host now returns before building that note when the feature is off.
A route-level regression asserts that neither the Director prompt nor the
Child prompt gains `PAGE-REPORTED STATE` in that configuration; disabling
the guard makes it fail with exactly that symptom.
Also reopens the composer before the unreferenced follow-up in the
classroom browser spec. An accepted answer may close it, which made the
assertion flaky without changing the behaviour under test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pi): decouple reference Scene from state freshness and bound assembled evidence
Cross-review found two defects in the request-scoped state evidence.
The reference's Scene was folded into the sample's staleness test. The packet
is already bound to the current Scene by the identity check above it, so a
valid current-Scene sample was being discarded as `stale-sample` purely because
the student's component reference came from an earlier Scene. Reference and
area state are independent evidence items; freshness is a property of the
sample alone. With the coupling gone the two can now disagree on Scene, so the
note says so explicitly rather than letting the model attribute area facts to a
component that may not be on the current Scene.
The assembled evidence had no stated output budget. The static component packet
is bounded to 24,000 code points upstream, but that bound covers the static
packet alone; the note and the escaped observation JSON were appended without a
recheck. Escaping `<` for the prompt expands one code point into six, and `<` is
legal in a label or a fact value, so a packet the Host accepts could assemble to
149,385 code points. The budget is now declared as the static bound plus the room
the note frame needs, which is what makes the degradation terminate. Over budget,
the state body drops whole to an explicit `unavailable` statement: truncating the
JSON would emit a broken packet, and thinning a `complete` relation set would turn
an exhaustive set into a false one. The relationship summary degrades with it, so
the prose never asserts COMPLETE over a body that is gone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pi): keep slide references independent of activity state
* refactor(interactive): simplify declared state and unify iframe preparation
Accept any JSON report within byte and depth budgets, without generated field
requirements or relationship-completeness semantics. Keep publishState and an
optional rendered result in the generation guidance.
Prepare the observation responder through patchHtmlForIframe and let the pool
own document identity, preserving state across placeholder remounts. Settle
sampling failures locally and align browser/server nesting limits.
Cover permissive JSON delivery, resource limits, lifecycle, legacy behavior,
and real renderer remounts with focused regression tests.
* fix(generation): publish automatic activity changes with clear positions
* fix(interactive): report missing legacy scope as no interface
* fix(interactive): guard sampling capabilities and bind scopes lazily
* chore(generation): bump version after main integration
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
#1514 moved every lib/audio provider call to the npm undici package's
fetch so the pinned dispatcher is guaranteed to be honored. But the
adapters kept building multipart bodies with the platform-global
FormData — a class of Node's bundled undici — and undici's serializer
brand-checks a FormData body against its own class. A foreign FormData
fell through to the string branch and left the process as
`content-type: text/plain;charset=UTF-8` with the 17-byte literal
`[object FormData]` as the whole body: the audio bytes and every field
(including `model`) were dropped, downstream gateways fell back to
whisper-1 and answered 503, and every multipart audio request (ASR,
TTS FormData paths, voice registration/cloning) failed with a generic
internal error.
Bare Blob/File, string/JSON/Buffer and stream bodies were never
affected: undici 7.29.0 exports no File/Blob classes of its own and
its bare-body brand checks (and multipart part handling) bind the
platform classes.
Normalize the body in the transport, once, before either transport
path serializes it, so the adapters can keep the platform globals as
their public API boundary: a foreign FormData is re-created as
undici's own with every entry carried over verbatim — `append` (not
`set`) so repeated field names survive, and no filename argument so a
platform File part keeps its own name/type/lastModified. Everything
else passes through untouched.
Also drive real loopback regression tests with a platform-global
FormData (direct, with repeated field names, empty, and across a 307
redirect hop whose per-hop loop re-issues the normalized body) and a
bare Blob, asserting multipart on the wire instead of
`[object FormData]`, and update the voxcpm unit test that asserted
the buggy contract (a platform FormData at the undici boundary).
Fixes#1579
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
On mobile browsers classroom narration plays the first segment and then goes
silent for every following one, while the lesson keeps advancing: the player
created a new HTMLAudioElement per line, and only the first line is covered by
the user's gesture, so every programmatic play() after it is refused with
NotAllowedError and the engine falls back to its reading-time timer. #651/#652
fixed the blob leak from that rejection, not the missing voice.
Keep one element per player instead:
- getAudioElement() creates it on first use and every line reuses it, so the
element the first gesture activated stays playable for the rest of the lesson
- stopAudioElement() releases the line's state rather than the element: onended
cleared, src removed, load() called -- a stopped line must not keep reporting
speech it is no longer playing, nor retain narration bytes through a revoked
object URL until the next play()
- onended is assigned rather than added: the element now outlives a single line,
so a listener would accumulate once per segment and call the engine back
several times for one line
tests/audio/audio-player-element-reuse.test.ts stubs the mobile policy itself
(the first element plays, every element created after it is refused): the reuse
tests fail on main and pass with this change.
Verified on a live deployment at the fixed entry point (instrumented Chromium,
default autoplay policy, one real click on Play): a single element for 8+
consecutive lines, no refused play, currentTime advancing line by line.
Fixes#1474
Co-authored-by: Shaoxuhua <jaxgen@163.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat(media): add OpenRouter image and video providers
OpenMAIC ships six separate video providers (Veo, Kling, Seedance,
MiniMax, Grok, HappyHorse) and seven image providers, each needing its
own key. OpenRouter fronts those same model families behind one key and
one account, so this adds it as a provider on both sides.
Both use OpenRouter's dedicated media endpoints, not chat-completions:
- Image: POST /images -> { data: [{ b64_json }] }
- Video: POST /videos -> 202 { id, status }, poll GET /videos/{id},
then GET /videos/{id}/content for the mp4 bytes
The model list is fetched live from GET /images/models and
GET /videos/models through /api/openrouter-models rather than pinned in
the registry: OpenRouter hosts 48 image and 28 video models today and
adds more, so a hardcoded shortlist would decide for the operator which
models exist. The registry keeps a three-entry seed as an offline
fallback, and the existing custom-model UI still accepts any model id.
Both catalogs answer unauthenticated, so the picker fills before a key
is pasted; a key is forwarded when present for proxied base URLs.
Adapter contracts are covered by stubbed-fetch tests (request shape,
empty-response handling, and the video job state machine including
terminal failure). No test performs a billable call.
Closes#1355
* fix(media): validate the key and tolerate a pasted endpoint URL
Three fixes found while configuring the new provider:
1. Both connectivity probes hit the model catalogs, which answer 200
unauthenticated — so "Test Connection" reported success for any
string, including an invalid key. Probe GET /key instead: equally
cheap, and it actually rejects a bad key.
2. The settings field is labelled "Base URL" but the panel echoes it
back as "Request URL", so pasting the full endpoint
(https://openrouter.ai/api/v1/images) is the natural mistake. That
built /api/v1/images/images and 404'd. Trim a trailing slash and a
trailing /images or /videos so both forms work; a proxy path that
merely contains the word is left alone.
3. The image and video settings panels read `data.message` on a failed
test, but failures answer with `error` (apiError) and only successes
carry `message`. Every failing connectivity test — for any provider,
not just OpenRouter — rendered "connection failed: undefined" instead
of the reason. Pre-existing; surfaced by 1 and 2 above.
Closes#1355
* fix(media): make every OpenRouter model selectable, and always select a provider
Two gaps found while configuring the new provider.
The settings Models list is a read-only catalog for every provider; the
actual model picker is the media popover. That picker built its groups
from the static registry array, so OpenRouter offered only the
three-entry seed while settings listed the full live catalog — the
models were visible but not choosable. Feed the same live catalog into
the popover, fetched only once the provider is usable so an
unconfigured install makes no request.
Separately, `imageProviderId`/`videoProviderId` are empty until a
provider is chosen (first-run auto-config leaves them blank when the
server reports no media provider). Opening the settings panel on an
empty id selected nothing: the header rendered the missing name key as
"settings.undefined", and Test Connection posted a blank
x-image-provider/x-video-provider, so it failed with "No image/video
provider configured" whatever key was typed. Fall back to the first
catalog entry so the panel always has a selection. Pre-existing and not
specific to OpenRouter.
Closes#1355
* fix(tts): request a browser-playable format from custom providers
`generateOpenAITTS` serves every custom OpenAI-compatible TTS provider but
never sent `response_format`, so it inherited whatever each provider
defaults to. OpenAI defaults to mp3; OpenRouter's /audio/speech defaults
to raw `pcm`. The unknown content type then fell through to the `'mp3'`
default below, the client built `data:audio/mp3;base64,…` from headerless
PCM samples, and playback failed with "no supported source was found" —
while the server logged a clean 200, because the audio really was
generated. Name the format instead of inheriting it.
Also stop mislabelling an unrecognised body: `pcm`/`l16` now raises a
message naming the cause, and `aac`/`opus` are recognised.
Two supporting fixes:
- /api/openrouter-models normalises its base URL the way the adapters do
and falls back to the public catalog when a custom base URL fails, so a
typo in a free-text settings field cannot empty the model picker. Also
types the headers object so tsc accepts the conditional.
- provider-neutrality-guard pins exact per-vendor occurrence counts in
lib/server/provider-config.ts. Adding the image and video env entries
raises "openrouter" from 2 to 6 (each entry contributes both its key and
its value); CI failed without the bump.
Closes#1355
* fix(security): never send the operator key to a client-chosen host
Review found `/api/openrouter-models` was an SSRF and key-exfiltration
path, and the finding is correct. The route took `x-base-url` from the
caller at highest precedence while preferring the *server* env key, so any
caller could make the server send the operator's OpenRouter credential as
an `Authorization: Bearer` header to an arbitrary URL. The route's own
comment claimed it followed `/api/verify-image-provider`; that pattern
runs `validateUrlForSSRF` on client base URLs, and this route did not.
The boundary is now explicit: the server key travels only to the
operator's own base URL. A client-supplied URL is SSRF-validated and
carries only that caller's own `x-api-key` — the server key is dropped —
and the unauthenticated public-catalog fallback never forwards a
credential chosen for a different host. Redirects are no longer followed
(`redirect: 'manual'`), since a redirect would carry the Authorization
header off-host and reopen the same hole, and upstream reads are bounded
by a timeout.
The per-URL cache is now keyed by destination *and* a hash of the
credential, and bounded to 64 entries with oldest-first eviction, so
client-supplied URLs cannot grow it without limit and one caller's
key-authorised catalog is never served to another.
Also from the review:
- The image adapter discarded the reported `media_type`. The
orchestration layer wraps a bare `base64` as `data:image/png`
unconditionally, so jpeg/webp results were mislabelled; the adapter now
returns a data URL carrying the real type.
- Adapter generation and poll requests set `redirect: 'manual'`, matching
the `/key` probe that already did.
- `runPolledTask` accepts an `AbortSignal` so the sleep between polls is
cancellable; the video adapter passes the caller's signal. Without it a
cancelled generation still slept out a full 10s interval.
Tests cover the highest-risk paths the review named: which credential
reaches which URL, that an SSRF-rejected destination is never contacted,
that the fallback is unauthenticated, cache isolation between callers,
and MIME preservation.
Findings 2 (base-URL normalisation) and 4 (neutrality-guard debt) were
already fixed in d553a08, pushed after the review was submitted; CI is
green on that commit.
Closes#1355
* ci: retry flaky voice clone timeout
* fix(vercel): keep OpenRouter catalogs within Hobby function limit
* fix(vercel): avoid tracing self-hosted sharp binaries
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Browser TTS auto-detection only distinguished Chinese (CJK ratio) from
everything else (en-US), so Vietnamese narration was spoken by an English
voice. Add Vietnamese detection to the shared language helper and bind an
installed vi voice in the playback engine when the user has not picked one
explicitly:
- lib/audio/browser-tts-preview.ts: new detectSpeechLang() (zh-CN /
vi-VN / en-US), used by both the Test TTS preview and playback. A hit on
đ/ơ/ư or the U+1EA0-U+1EF9 precomposed block (ớ, ừ, ồ, ế, …) — absent
from French/Romanian Latin — marks Vietnamese; bare ă/â/ê/ô only count
toward a low ratio.
- lib/playback/engine.ts: use the shared helper; when it reports vi-VN,
prefer an installed vi voice so pronunciation is correct out of the box.
An explicitly configured voice still wins.
- tests/audio/detect-speech-lang.test.ts: zh/vi/en/fr cases.
Verified end-to-end (Playwright WebKit): utterances carry lang vi-VN with
an installed Vietnamese voice auto-selected, advancing through every line.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat(token-plan): add TokenDance one-key preset for every modality
TokenDance is a model gateway: chat and images are OpenAI-compatible at
/gateway/v1, and the same key authenticates vendor-protocol routes on the
same host (Ark, MiniMax, Bocha). The preset reuses the existing adapters
with those route prefixes as base URLs, so one key lights up LLM, image,
video, TTS and web search from Settings -> Token Plan.
- providers: add a built-in `tokendance` OpenAI-compatible provider
(TOKENDANCE_* env prefix, logo, provider name in all locales)
- token-plan: add the TokenDance preset (Seedream image, MiniMax H3 video,
MiniMax speech TTS, Bocha web search)
- seedream: use a base URL that already ends in a version segment verbatim,
so gateway routes like `/ark/v3` do not get `/api/v3` appended
- minimax-video: route H3-family models through the v2 task API (content
array submit, task-envelope poll); connectivity checks for H3 probe auth
on the v2 query route instead of submitting a billable task
- README: add a one-key quick example and replace the Gemini-specific
model recommendation with a provider-agnostic setup recommendation
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qrrq9CPwb718mpouz8Y2KL
* fix(token-plan): accept preset web-search base URLs and report H3 dimensions per ratio
- web-search: the client base URL allowlist also accepts the exact base URL
a built-in token plan preset writes for that provider, derived from
TOKEN_PLAN_PRESETS. Applying a plan whose web-search route is not an
official vendor host previously stored a URL that the route rejected with
400. Any other client URL is still rejected.
- minimax-video: report H3 v2 clip dimensions for 16:9, 9:16, 4:3 and 1:1
instead of assuming landscape for every non-portrait ratio.
- tests: pin the allowlist for every preset, the 1:1 H3 dimensions, and
clear TOKENDANCE_* in the provider-config env isolation list.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qrrq9CPwb718mpouz8Y2KL
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The workbench tools store generated bytes in the asset pool under the shared principal and write the allocated id into the document, the same discipline as the classic chain since #1392; the runner's putScene creates the reference rows and commits the allocations. The video completion patch rewrites every placeholder slot and retires anything that would shadow the new id; immediate render is preserved by leasing the id at the render boundary. A store-full refusal fails the tool with a model-readable error and writes nothing. Legacy /api/classroom-media documents keep rendering. Closes#1522.
Extracts commitToPool, the single client-side sequence for storing bytes in the asset pool, writing the allocated id back, and mirroring locally; routes the media pass, narration adoption and fresh TTS through it. A store-full refusal during TTS now retains the already-billed clip so the next load adopts it with zero provider calls. Closes#1467.
Audio provider requests (TTS, ASR, voice registration and voice cloning)
validated a client-supplied base URL once and then issued a plain fetch with
default redirect-follow and no pinned dispatcher. A base URL that resolved to a
public address but answered with a redirect to an internal one was followed, and
a DNS answer that changed between the guard's lookup and the connect reached an
internal host — both readable in-band.
- Route every lib/audio provider request through a new
lib/server/audio-provider-fetch.ts that combines per-hop redirect
re-validation with a pinned undici dispatcher, so the socket can only reach an
address the guard validated, on every hop.
- Select the public-vs-local policy server-side from isServerConfiguredProvider;
a client-supplied base URL is always strict public and can never reach a
private, loopback or cloud-metadata address, even with ALLOW_LOCAL_NETWORKS
set.
- Pin the result-audio download hop as well, keeping its host allowlist and
redirect:'error'.
- Add a coverage-matrix test that fails if any lib/audio module regains a raw
provider fetch.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
When ACCESS_CODE is set, verification tokens (timestamp.HMAC) never expired
because neither verifier checked the timestamp, and POST /api/access-code/verify
had no attempt throttling.
- Enforce a 7-day token lifetime in a shared, Edge-safe module used by both the
Node verifier and the middleware Web-Crypto verifier, and reject non-canonical
signatures in both.
- Rate-limit verification only when the client identity is trusted
(TRUST_PROXY_HEADERS=true): a per-client sliding window (10 failures / 60s)
whose attempt is reserved atomically at check time, returning 429 with
Retry-After. Without a trusted proxy the app cannot attribute requests to a
client, so no shared throttle is applied — a long random ACCESS_CODE is the
protection, and a warning is logged when it is short.
- Store bounded, copied identity keys so a large forwarding header cannot retain
memory.
- Document the behavior in .env.example, README, and configuration docs.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(classroom): keep transient load failures off the not-found card
/api/classroom 5xx and network errors used to become null, so ClassroomSurface treated them like a missing course and rendered the terminal not-found claim with no retry. Classify fetch answers as found/absent/unavailable and only show not-found on a positive 404/410 miss.
Fixes#1450
* fix(classroom): handle terminal load states
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
addCustomTTSProvider stores the dialog Base URL in customDefaultBaseUrl and leaves baseUrl empty. isTTSProviderConfigured ignored that field, so generation silently skipped narration even after Test TTS succeeded. Accept the dialog URL so already-saved custom providers start working without retyping the field.
Fixes#1471
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
App wiring for the @openmaic/storage 0.31.0 lifecycle: reference tracking and document references are paired unconditionally and declared at startup, course deletion withdraws references inside the tombstone transaction, ASSET_PENDING_TTL_MS configures the pending window, and the dead client-side reclamation code is removed.
* fix(agent-runtime): allow one owner material to be bound to multiple sessions
`agent_session_materials.id` is a global primary key, but
`bindOwnerMaterialsToSession` de-duplicated on `(sessionId, id)` and then
inserted the owner-side material id as the row id. Binding the same owner
material to a second session therefore raised a duplicate-key error
(23505) that the route surfaced as HTTP 500, so an owner material could
only ever be used by one course.
Keep row ids globally unique (every extraction method keys on `id` alone)
and store the shared owner id as metadata instead: the binder now mints a
fresh session row id, records `owner_material_id`, and looks that column
up for idempotent rebinding; a partial unique index on
`(session_id, owner_material_id)` adjudicates concurrent rebinds and the
loser adopts the winner's row. The schema change is additive and
idempotent (`ADD COLUMN IF NOT EXISTS` + `CREATE UNIQUE INDEX IF NOT
EXISTS`), so existing databases upgrade in place with `NULL` for legacy
rows. No FK from `owner_material_id` to the owner library, to keep
owner-library deletion decoupled from session rows.
Tests: backend-neutral contract test (same owner material bound to two
sessions, both readable), PGlite in-place upgrade test, pinned-schema
update, and a host-level test covering the route's 202 path.
* fix(agent-runtime): reuse legacy owner-material bindings and clean up losing uploads on concurrent bind
Rows written by the previous binder use the owner upload id as the session
row id and leave `owner_material_id` NULL, so the new owner-id lookup missed
them and a rebind created a duplicate row and copied the bytes again. The
binder now falls back to the session row id and adopts it only when it is
unambiguously that owner upload: source kind, matching title, no source URL
or derivative, no text, and the deterministic legacy object key with a byte
length that matches the owner record. Adoption stamps `owner_material_id`
through a conditional `backfillOwnerMaterialId` update that leaves extraction
columns untouched; a lost race re-reads the fast path. A different session is
still a fresh row.
The concurrent-bind loser also left its uploaded object behind after
adopting the winner. It now removes that object when the winner references a
different key, best-effort so a failed cleanup cannot fail a bind that
succeeded.
Tests: host-level PGlite regressions for the legacy rebind (id, row count,
byte copies, extraction state, and backfill) plus a different-session bind,
and a controllable byte store that parks the loser after its upload so the
winner commits first, asserting one row and only the winner's object remain.
A storage unit test pins the backfill contract. Bumps @openmaic/storage to
0.30.1 for the new public store method.
* fix(storage): keep jsonb writes valid when model output contains NUL or lone surrogates
PostgreSQL jsonb rejects the `\u0000` and lone UTF-16 surrogate escape
sequences that JSON.stringify emits verbatim, failing the enclosing
statement with SQLSTATE 22P05/22P02. In the agent-session store the failed
tree-entry/event write is treated as critical, so the whole run aborts and
all work in that run is lost.
Add a shared encodeJson at @openmaic/storage/src/pg-json.ts that replaces
U+0000 and unpaired surrogates with U+FFFD while preserving valid surrogate
pairs (emoji), sanitizing in-memory values and object keys before
JSON.stringify. Route every jsonb parameter through it: the agent-session,
document, runtime, asset, and material backends, plus owner-material
registration in lib/persistence.
The document/runtime/asset backends keep their existing assertJsonValue
guard, which rejects these code points with a readable error before the
shared serializer runs; the agent-session and material paths had no guard
and were the live 22P05 exposure. Literal backslash-u text is untouched.
* chore(storage): bump @openmaic/storage to 0.29.2
* fix(storage): keep colliding sanitized keys and own __proto__ members when encoding jsonb
encodeJson replaces NUL and lone surrogates in object keys, but two distinct
keys can sanitize to the same string: "a\u0000" and "a\uFFFD" both become
"a\uFFFD". The rebuild assigned each member by its sanitized key, so a later
member silently overwrote an earlier one and that earlier value was lost.
Keep the first member under the sanitized key and give each later colliding
member a deterministic suffix (#2, #3, ... appended until the key is unique
among the keys emitted so far, whether sanitized or original). Member order is
preserved. The suffix is ASCII and contains no NUL or surrogate, and the
sanitizer only ever emits U+FFFD, so a suffixed key can never be confused with
a bare replacement result.
The rebuild also used a plain {} object, so assigning an own "__proto__" member
invoked the prototype setter and dropped it from the output whenever a sibling
key needed sanitizing. Build the rebuilt object with a null prototype so
"__proto__" stays an own data property; JSON.stringify then emits it and
PostgreSQL round-trips it. Arrays and the no-op fast path (return the original
reference when nothing needs sanitizing) are unchanged.
* fix(settings): maintain asr language state invariant on provider selection (#1082)
* test(store): import ASRProviderId type in server sync tests
* fix(settings): validate custom ASR languages against CUSTOM_ASR_DEFAULT_LANGUAGES (#1082)
* test(store): refactor custom ASR sync and rehydration test suite for clarity
---------
Co-authored-by: cham <2577781125@qq.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(upload): resolve the generic Office MIME via filename extension (#1497)
On Linux, Chrome derives File.type from the XDG shared-mime-info database.
Older or trimmed databases (e.g. Kylin OS V10) map every OOXML extension to
the generic container `application/vnd.ms-office` instead of the concrete
format MIME, so .pptx/.docx/.xlsx uploads were rejected with "当前解析器不支持该格式"
on the generation toolbar and with 415 unsupported_mime on POST /api/materials
— while the same files upload fine on macOS/Windows.
Treat `application/vnd.ms-office` as a generic upload fallback alongside the
zip family, so the filename extension decides the concrete format:
- lib/document/mime.ts: add it to GENERIC_DOCUMENT_MIME_TYPES. This covers
the toolbar's provider-whitelist check and the extract-document API gate
(both call normalizeDocumentMimeType). It must NOT go into per-format
aliasMimes — canonicalFromAlias iterates DOCUMENT_FORMATS in declaration
order, so a shared alias would misroute x.pptx to application/msword.
- lib/workbench/material-upload-policy.ts: new resolveWorkbenchMaterialMime()
grants missing/octet-stream/zip/vnd.ms-office types the same extension
fallback the document path already had (the workbench gate previously
rejected empty and zip-family types outright on every OS).
- app/api/materials/route.ts + session-store.ts: gate, store, and send the
resolved MIME instead of the raw browser value.
The spoofing surface is unchanged: a specific but unsupported MIME still
falls through verbatim for the whitelist to reject, and an unknown extension
keeps the generic MIME. Full suite: 7992 passed.
Closes#1497
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(review): log the declared mime, harden the fallback chain, pin adversarial cases
Review follow-ups on #1498 (no correctness or security findings — hygiene only):
- materials route: the reject/context log now carries declaredMime alongside
the resolved mime whenever they differ, so a generic-header spoof stays
visible in upload failure logs instead of collapsing to the resolved type.
- uploadWorkbenchMaterial: the returned record's mimeType falls back to the
locally resolved mime before the raw browser value, which may still be the
generic Office container if the server echo is ever absent.
- material-upload-policy: export WORKBENCH_MATERIAL_EXTENSIONS and add a
drift guard asserting every accepted extension resolves (via a generic
MIME) to a whitelisted type — a hand-maintained map entry going missing
now fails tests instead of silently 415ing.
- tests: pin the legacy-.ppt provider split (mineru rejects, mineru-cloud
accepts), uppercase-extension resolution, and 415-before-400 precedence
for a generic mime with no filename.
Full suite: 7996 passed.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(upload): support WPS Office MIME aliases
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix: tolerate malformed authored CSS in classroom exports
Interactive-scene HTML authored by LLMs can carry browser-tolerated CSS
syntax errors (e.g. `-- crop-green: #22c55e`), which browsers silently
drop. The strict postcss pass used by resource-pack / classroom-zip
export threw on them, failing the whole export while PPTX (which never
parses interactive HTML) kept working.
Inlining is an enhancement, not a precondition: on strict-parse failure
the stylesheet/style attribute/SVG attribute is now kept byte-for-byte
with a warning, and export continues. Remote dependencies inside
unparseable CSS (url() / @import, absolute or relative, quoted with
parens) are still surfaced via a comment- and string-aware fallback
scanner so zip exporters warn and offline validation is not silently
bypassed.
* fix: stop the css fallback scanner at bad-string tokens
Per CSS syntax, an unescaped \n/\r/\f inside a quoted string produces a
bad-string token and the browser recovers right there. The fallback
scanner previously ran unterminated strings to EOF, swallowing every
following rule and hiding browser-active url()/@import dependencies from
dependency reporting and residual validation.
Both string-scanning loops now terminate at an unescaped newline and
resume scanning from it; an escaped newline (\ + \n, \ + \r\n, \ + \r,
\ + \f) continues the string, and escaped newlines are stripped from
captured URLs — in the fallback scanner and in the strict-path
collectors (cssUrlReferences / cssImportReference), which previously
kept the escape in the URL for valid CSS too.
* fix: keep EOF-truncated css strings as valid tokens
Per CSS Syntax 4.3.5, only an unescaped \n/\r/\f produces a bad-string
token; a quoted string truncated at EOF is a valid string token that
browsers keep (with the function auto-closed and a trailing backslash
consumed). The previous round wrongly dropped EOF-truncated url(".../
@import "... values, hiding real remote dependencies again — the exact
shape of an LLM-authored <style> cut off mid-generation.
Both string paths now share scanCssString with three-way termination
(quote / unescaped newline / EOF), and the collectors were aligned with
CSS token semantics on both the strict and fallback paths, with parity
tests: escaped newlines (\ + LF/CRLF/CR/FF) are stripped escape-aware
from quoted values; an unquoted url() containing a backslash-newline is
a bad-url and no dependency; non-whitespace (comments allowed) around a
quoted payload makes a bad-url, and a bad url consumes through its ')'
so nested url(...) text is not re-scanned; an escaped ')' belongs to an
unquoted url value.
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix: enable Pro workbench flag in Docker builds; allow non-TLS localhost cookie
Two deployment fixes discovered while self-hosting with Docker Compose:
1. Expose NEXT_PUBLIC_PRO_WORKBENCH_ENABLED as a Docker build arg.
Persistence and other NEXT_PUBLIC_* flags are already wired through
Dockerfile + docker-compose.yml, but the Pro workbench entry flag was
missing, so Docker deployments could not enable the workbench at all.
2. Add COOKIE_SECURE opt-out for the anonymous owner cookie.
Production builds always set `Secure` on the anonymous_id cookie. Safari
refuses to store Secure cookies served over plain http://localhost (it
does not special-case localhost like Chromium/Firefox), so every request
minted a fresh anonymous owner and owner-scoped document writes were
rejected with 403 — scenes never persisted and generation appeared to
hang after the first scene. COOKIE_SECURE=0 lets plain-HTTP deployments
opt out; the default (Secure in production) is unchanged.
Documents COOKIE_SECURE in .env.example alongside the persistence opt-ins.
* fix: share the COOKIE_SECURE opt-out with the Server Action cookie mint
Review follow-up on the COOKIE_SECURE opt-out.
- Extract anonymousCookieSecure() in owner.ts and use it in
lib/workbench/workspace-actions.ts, which re-implements the anonymous_id
mint: with COOKIE_SECURE=0 that path still sent a Secure cookie, so a
Server Action running before any /api/agent/* request (e.g. deleting a
workspace session) minted an ephemeral owner in Safari.
- State the opt-out accurately in owner.ts; the previous comment claimed the
flag never forces Secure off.
- Add the regression test beside the existing production case (verified it
fails when the opt-out is dropped).
- .env.example: document the security cost of dropping Secure and that only
the exact value 0 disables it.
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
