Reject classic inline interactive scripts that fail to parse at generation time (extracted with parse5, checked with node:vm Script without executing), and surface iframe runtime errors on the active interactive scene.
Addresses #1622 (partial: no recovery / regeneration UI).
Co-authored-by: Frank-zhu0404 <Frank-zhu0404@users.noreply.github.com>
* fix(tts): pace classroom TTS and classify MiniMax RPM limits
MiniMax reports RPM limits as HTTP 200 with base_resp.status_code 1002.
Classify that code as TTSRateLimitError, pace classroom TTS with
TTS_MIN_INTERVAL_MS (default 1000ms) and double the gap up to 15000ms on
each rate limit, and store ttsCoverage on the generation job so a partial
narration is not a clean success.
Fixes#1567
* fix(tts): fail closed on MiniMax status and skipped narration
Treat every non-zero MiniMax base_resp status as a failure, even when
audio bytes are present. Classify 1002, 1039, 1041, and 2045 as rate
limits. Floor rate-limit back-off independently of TTS_MIN_INTERVAL_MS,
cap the cumulative wait, and cancel those sleeps on abort. A skipped or
thrown TTS phase reports zero coverage and a warning.
* fix(tts): decay spacing, heartbeat TTS progress, default interval 0
Classroom TTS spacing steps back toward TTS_MIN_INTERVAL_MS after each
successful clip, and that interval defaults to 0 so pacing starts only
after a rate limit. Clip progress is reported through the existing job
onProgress path, so updatedAt keeps moving during a long narration phase.
A Retry-After that does not fit the remaining back-off budget skips that
clip instead of silencing every later clip.
generateTTSForClassroom always returns coverage. A skipped run that has
resolved a provider counts clips after the same speech split as synthesis.
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(generation): enforce quiz option value/label contract
When a model puts choice content in value and a bare A-Z letter in label,
swap those fields so persisted quiz options keep a letter as the selection
identity and the content as the label. Correct objects, plain strings, and
answer-key normalization stay as they were.
Closes#1375
* fix(generation): rewrite swapped quiz letter keys to option values
When a bare label is moved onto the option value, a raw answer token that
exactly equals that original label is rewritten to the uppercased letter
before exact alignment. The stored key is then the letter QuizView submits,
so that submission grades correct. A stored "a" on an already-correct
value "A" is left unresolved.
Closes#1375
* fix(generation): reject invalid quiz option shapes instead of swapping
Stop rewriting quiz options when a model puts content in value and a
letter in label. The quiz prompt now requires value to be one ASCII
letter A-Z and label to be the option content. A choice question that
still breaks that contract, or whose answer does not name an option
value, is invalid model output and regenerates.
Closes#1375
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(web-search): stop leaking Brave's HTML challenge page into errors
Both Brave error paths echoed the upstream response body verbatim, so a
throttled or blocked request surfaced a full HTML page as Error.message
and a transient 429 was indistinguishable from a real outage.
Route both paths through formatBraveError, which gives 429 a dedicated
actionable message, drops markup and over-long bodies in favour of the
HTTP status text, and keeps short plain-text details.
Closes#1552
* fix(web-search): cancel Brave's HTML error body instead of reading it
Deciding whether an error body is worth surfacing still meant pulling the
whole challenge page into memory first. Check the content type up front and
cancel the stream when it is markup, so the large body is never read.
* fix(web-search): give Brave API-key 429 plan-specific guidance
The API-key path already sends a subscription token, so its 429 message
should not suggest configuring a Brave API key. Point it at the plan's
rate limit instead, keep the API-key suggestion for the keyless scrape
path, and add a test for the API-key 429.
---------
Co-authored-by: zhentang2 <zhentang2@iflytek.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(workbench): show effective upload limit in friendly errors
* fix(workbench): format the upload limit shown in composer toasts
A non-MiB cap was stringified as a long fraction, and nothing checked that the toast used the localized message.
Course documents, folders, materials and agent sessions are partitioned by an
owner id that comes from a 30-day anonymous cookie, because there is no host
auth layer. For one team behind one ACCESS_CODE that partition buys nothing —
those visitors already share the site password — and it costs: a second browser
shows an empty course list, and POST /api/stages/[id]/publish refuses every
owner, because publishing a cookie partition is not something the product
allows.
PERSISTENCE_SHARED_OWNER_ID resolves every request to that fixed id instead.
Unset — the default — changes nothing.
- **ACCESS_CODE is required.** Without one the middleware lets every request
through, so a single shared owner would expose one readable, editable and
publishable course library to anyone who can reach the deployment. That is
worse than the per-browser partitioning this setting removes, so the
combination refuses to start, from instrumentation.ts, with a message naming
both variables. The alternative the issue allowed — ignore it with a warning —
was rejected because a silently ignored setting is exactly the confusion this
feature exists to remove.
- The value is validated rather than trusted: 1-128 characters of
[A-Za-z0-9._-]. That excludes the reserved `anon:` prefix, which would still
be refused by publish and would alias onto a cookie owner, and any character
the material-key sanitiser rewrites, which could collide two ids onto one
object key. A malformed value fails startup on the same path; an empty value
is treated as unset, since KEY= in an env file cannot be told apart from an
operator who meant to leave the feature off.
- An explicit authenticatedOwnerId still outranks it, so adding a real auth
layer later does not require clearing the variable first.
- All four places that derive an owner honour it, including the Server Action
in lib/workbench/workspace-actions.ts, which re-reads the cookie itself. That
one is easy to miss and would make the workspace list and its row actions
disagree about who owns a session.
- The env docs state the trust model next to the variable, cross-reference
SECURITY.md, and note that this answers who owns a course while DATABASE_URL
answers where one lives — a second browser sees an empty list on Postgres too.
This is the stopgap asked for in #1550 — a PR in the spirit of the closed
#1551 — with threading a real authenticated owner left as the long-term fix.
AI-assisted commit
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
#930 made the connectivity probes in the media adapters pass
`redirect: 'manual'`. The generation and poll calls in the same 14 files were
left following redirects. Those requests carry the provider credential and go
to a base URL that comes from provider settings a caller can supply, so a 3xx
would replay the credential at a host the caller chose — and the redirect
target can be an address the outbound guard already refused.
Every such call now passes `redirect: 'manual'` and rejects a 3xx through a
shared `assertNotRedirected` helper, which reports it as
"<provider>: Redirects are not allowed (HTTP <status>)" instead of letting the
generic failure path describe it as a provider error.
- 26 call sites across the image adapters (seedream, openai, qwen, grok,
lemonade, minimax, nano-banana), the video adapters (seedance, kling, grok,
happyhorse, minimax, veo) and ComfyUI's submit and image fetch.
- ComfyUI's `pollHistory` keeps its contract of handing the caller a retryable
failure rather than aborting the generation: it logs the refusal and returns
null.
- ComfyUI's same-origin workflow load is deliberately untouched — it reads the
app's own public/ asset, carries no credential and is not provider-influenced.
- The two adapters added since #930 (OpenRouter image and video) already did
this.
tests/media/adapter-redirects.test.ts covers one case per adapter family. Each
serves a 302 and asserts that the call rejects with the redirect message and
that every request carrying an init object asked fetch not to follow redirects;
each case fails if its adapter stops passing `redirect: 'manual'`.
The HappyHorse test asserted the exact request init, so it now includes the new
option.
AI-assisted commit
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Two independent Grok failures seen when the provider is reached through a relay
(a custom base URL) instead of api.x.ai directly:
- lib/ai/providers.ts: a long non-streaming chat completion was cut off by the
relay with a 504 at its idle timeout (~5 min), because nothing is sent
upstream until the model has the whole answer. Adding 'grok' to the existing
streaming-compat path (OPENAI_COMPAT_USE_STREAMING_CHAT=true) keeps bytes
flowing across the idle window; the SSE is buffered back into a normal JSON
response for the caller.
The path is for relays only. usesCustomOpenAIBaseUrl recognises OpenAI's
origin alone, so Grok's own api.x.ai also read as "custom" and was forced
onto the compat transport; the provider's native endpoint is now excluded.
- lib/media/adapters/grok-image-adapter.ts: response_format 'url' returns a
link on the relay's CDN host (imgen.x.ai), which may be unreachable from the
server's network. The generation then failed at the follow-up fetch through
/api/proxy-media even though the image had been produced successfully.
'b64_json' inlines the bytes and removes that second hop.
Inline bytes declare no media type, so the adapter reports one on
ImageGenerationResult and returns a typed data URL, which is the shape
openrouter-image-adapter already uses. Consumers take the type from there:
agent image persistence records it, the client's stored media row keeps the
type its data URL states, and classroom-media-generation names the file with
the matching extension. A JPEG is no longer stored, served or named as a PNG.
The process-wide undici timeout that previously accompanied these changes is
dropped: upstream #1404 now gives LLM calls their own undici headers/body
timeouts, which covers the same failure without raising the defaults globally.
Co-authored-by: ciclou1 <ciclou1@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Share corrected line bounds across renderer and React editing paths, preserve double-elbow routing, and translate PPTX points into the shape bounding box.
Refs #674 and the prior implementation/review in #675. AI-assisted.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(media): emit keyframe images as data URLs
The local media extractor stored keyframe bytes as raw base64 in
`DocumentAsset.data`. Every other extractor emits a data URL, and the
document bundle forwards this field as `pdfImages[].src` to `storeImages`,
which decodes it with `decodeBase64DataUrl`. With no `data:` prefix the
comma split yields no payload, so `atob(undefined)` throws and course
generation fails with "Failed to store image bundle at image img_1".
`pdf-compat`'s `dataUrlMimeType` also derives the image asset mime from
this same field, so the declared `image/webp` was silently dropped too.
Emitting `data:${mime};base64,...` matches `mineru-parser`, which already
normalizes prefix-less base64 the same way.
* fix(media): decode keyframe data URLs
* fix(media): accept legacy and data-url assets
* fix(media): reject malformed or empty media asset data URLs
A string that starts with data: but does not parse as a data URL used to
fall through to the raw-base64 path, where Node's decoder skips
non-alphabet characters and yields garbage bytes. Throw instead, and
reject an empty base64 payload rather than storing a 0-byte image.
Correct the keyframe test comment: local keyframe assets are consumed by
material extraction, and the test needs ffmpeg so it does not run in CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(export): surface video render rejection reasons
* fix(export): keep render diagnostics out of user toasts
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Discussion narration created a new Audio element per line, so on mobile every
line after the first had its programmatic play() refused with NotAllowedError:
the dialogue went silent while the lesson kept advancing. This is the
discussion side of #1474, which #1477 fixed for the narration player.
One module-scoped element now serves every line, kept separate from the
narration element in AudioPlayer so neither can cut the other off. Because
element identity can no longer tell lines apart, the stale-event guard became a
per-line token, handlers are assigned rather than added (a listener would
accumulate once per line), and finish()/cleanup() release the line by removing
the source attribute and reloading instead of leaving the element pointed at
the page's own URL.
Co-authored-by: talkman <jaxgen@163.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* refactor(upload): derive the workbench material MIME policy from the shared format registry
#1498 fixed the Kylin generic-Office-MIME failure on both upload paths but
left the workbench with its own extension→MIME table, alias map, and
generic-MIME set next to the document registry, and the two had already
drifted. The workbench policy now derives every MIME/extension fact from
lib/document/mime.ts (the single source of truth); only the accepted-format
list stays workbench policy — fixed and extractor-independent, unlike the
classic path's provider-scoped whitelist.
- Register csv and webm in DOCUMENT_FORMATS (accepted by no document
provider, so classic-mode whitelists are unchanged) and add the
audio/x-m4a alias to m4a.
- material-upload-policy.ts keeps its export names (route, session-store,
and composer consumers unchanged) but resolves, whitelists, and builds
its accept string from registry helpers.
- The workbench gate now also accepts the registry's curated aliases it
previously missed: image/jpg, text/x-markdown, and audio/x-wav (stored
canonically as audio/wav).
Closes#1589.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* test(workbench): pin the audio/mp3 alias closure in the material policy
Deriving the workbench gate from the shared registry normalization
(#1589) also accepts the browser-reported audio/mp3 alias the hand-rolled
alias map rejected — the same gap class as image/jpg and text/x-markdown,
so pin it alongside them and name it in the comment.
Co-Authored-By: Claude Code <noreply@anthropic.com>
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat: sample declared interactive state before classroom questions
* test: exercise generated publication example through the iframe reader
* chore(generation): bump package version for observation prompt contract
* fix: keep interactive state optional on insecure HTTP origins
* fix(playback): keep component picking and separate state from reference
A declared state interface replaced the component picker with a forced
whole-area `#experiment` reference, removing the per-component selection
and outline that `main` already ships. Sampling was also gated on the
reference selector, so the only way to obtain state was to give up the
selection.
Reference identity and area state are now independent request-scoped
evidence items:
- `handleToggleElementPick` always arms the picker again, so a scene that
declares the interface keeps main's per-component selection, outline,
and send-time clearing.
- `sampleInteractiveState` follows the current Scene instead of the draft
reference, so an unreferenced follow-up still reports current facts and
never re-creates or extends a reference.
- The Host carries area state with or without a component reference. The
evidence header names both identities and refuses to present area facts
as properties of the referenced component.
- `metadata` is absent when only area state travels, so no element
identity and no Spotlight authorization can be derived from it, and the
accepted-reference receipt stays driven by explicit references only.
Review follow-ups in the same change:
- Client sampling follows `NEXT_PUBLIC_COURSEWARE_REFERENCE_ENABLED`.
An ungated packet turned an ordinary Pi question into a 400 while the
reference feature was disabled.
- A Scene that declares the interface always receives an availability
boundary, including when the browser produced no packet at all. It is
reported as `not-sampled` rather than the previous `no-interface`,
which was a false statement about an activity that does declare one.
Courseware without the interface keeps its unreferenced behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(generation): register the observation snippet as a packaged asset
The interactive-observation snippet is referenced by all six widget
content templates but was never added to the packaged-asset manifest, so
the asset test and the golden scene prompt both failed.
- `SNIPPET_IDS` now lists `interactive-observation`, restoring both the
"exactly the generation-owned templates and referenced snippets" check
and the "every referenced snippet is packaged" cross-check.
- The interactive system-prompt snapshot is re-pinned. The change is
purely additive: the snippet is appended to the simulation template.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pi): stop injecting state constraints while the feature is disabled
The route rejects a request that carries a reference or a state packet
while `NEXT_PUBLIC_COURSEWARE_REFERENCE_ENABLED` is off, but an ordinary
question carries neither. It still reached the Host, and a Scene that
declares the state interface then received the full page-state block —
several kilobytes of constraints about evidence the deployment can never
sample.
The Host now returns before building that note when the feature is off.
A route-level regression asserts that neither the Director prompt nor the
Child prompt gains `PAGE-REPORTED STATE` in that configuration; disabling
the guard makes it fail with exactly that symptom.
Also reopens the composer before the unreferenced follow-up in the
classroom browser spec. An accepted answer may close it, which made the
assertion flaky without changing the behaviour under test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pi): decouple reference Scene from state freshness and bound assembled evidence
Cross-review found two defects in the request-scoped state evidence.
The reference's Scene was folded into the sample's staleness test. The packet
is already bound to the current Scene by the identity check above it, so a
valid current-Scene sample was being discarded as `stale-sample` purely because
the student's component reference came from an earlier Scene. Reference and
area state are independent evidence items; freshness is a property of the
sample alone. With the coupling gone the two can now disagree on Scene, so the
note says so explicitly rather than letting the model attribute area facts to a
component that may not be on the current Scene.
The assembled evidence had no stated output budget. The static component packet
is bounded to 24,000 code points upstream, but that bound covers the static
packet alone; the note and the escaped observation JSON were appended without a
recheck. Escaping `<` for the prompt expands one code point into six, and `<` is
legal in a label or a fact value, so a packet the Host accepts could assemble to
149,385 code points. The budget is now declared as the static bound plus the room
the note frame needs, which is what makes the degradation terminate. Over budget,
the state body drops whole to an explicit `unavailable` statement: truncating the
JSON would emit a broken packet, and thinning a `complete` relation set would turn
an exhaustive set into a false one. The relationship summary degrades with it, so
the prose never asserts COMPLETE over a body that is gone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pi): keep slide references independent of activity state
* refactor(interactive): simplify declared state and unify iframe preparation
Accept any JSON report within byte and depth budgets, without generated field
requirements or relationship-completeness semantics. Keep publishState and an
optional rendered result in the generation guidance.
Prepare the observation responder through patchHtmlForIframe and let the pool
own document identity, preserving state across placeholder remounts. Settle
sampling failures locally and align browser/server nesting limits.
Cover permissive JSON delivery, resource limits, lifecycle, legacy behavior,
and real renderer remounts with focused regression tests.
* fix(generation): publish automatic activity changes with clear positions
* fix(interactive): report missing legacy scope as no interface
* fix(interactive): guard sampling capabilities and bind scopes lazily
* chore(generation): bump version after main integration
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
#1514 moved every lib/audio provider call to the npm undici package's
fetch so the pinned dispatcher is guaranteed to be honored. But the
adapters kept building multipart bodies with the platform-global
FormData — a class of Node's bundled undici — and undici's serializer
brand-checks a FormData body against its own class. A foreign FormData
fell through to the string branch and left the process as
`content-type: text/plain;charset=UTF-8` with the 17-byte literal
`[object FormData]` as the whole body: the audio bytes and every field
(including `model`) were dropped, downstream gateways fell back to
whisper-1 and answered 503, and every multipart audio request (ASR,
TTS FormData paths, voice registration/cloning) failed with a generic
internal error.
Bare Blob/File, string/JSON/Buffer and stream bodies were never
affected: undici 7.29.0 exports no File/Blob classes of its own and
its bare-body brand checks (and multipart part handling) bind the
platform classes.
Normalize the body in the transport, once, before either transport
path serializes it, so the adapters can keep the platform globals as
their public API boundary: a foreign FormData is re-created as
undici's own with every entry carried over verbatim — `append` (not
`set`) so repeated field names survive, and no filename argument so a
platform File part keeps its own name/type/lastModified. Everything
else passes through untouched.
Also drive real loopback regression tests with a platform-global
FormData (direct, with repeated field names, empty, and across a 307
redirect hop whose per-hop loop re-issues the normalized body) and a
bare Blob, asserting multipart on the wire instead of
`[object FormData]`, and update the voxcpm unit test that asserted
the buggy contract (a platform FormData at the undici boundary).
Fixes#1579
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
On mobile browsers classroom narration plays the first segment and then goes
silent for every following one, while the lesson keeps advancing: the player
created a new HTMLAudioElement per line, and only the first line is covered by
the user's gesture, so every programmatic play() after it is refused with
NotAllowedError and the engine falls back to its reading-time timer. #651/#652
fixed the blob leak from that rejection, not the missing voice.
Keep one element per player instead:
- getAudioElement() creates it on first use and every line reuses it, so the
element the first gesture activated stays playable for the rest of the lesson
- stopAudioElement() releases the line's state rather than the element: onended
cleared, src removed, load() called -- a stopped line must not keep reporting
speech it is no longer playing, nor retain narration bytes through a revoked
object URL until the next play()
- onended is assigned rather than added: the element now outlives a single line,
so a listener would accumulate once per segment and call the engine back
several times for one line
tests/audio/audio-player-element-reuse.test.ts stubs the mobile policy itself
(the first element plays, every element created after it is refused): the reuse
tests fail on main and pass with this change.
Verified on a live deployment at the fixed entry point (instrumented Chromium,
default autoplay policy, one real click on Play): a single element for 8+
consecutive lines, no refused play, currentTime advancing line by line.
Fixes#1474
Co-authored-by: Shaoxuhua <jaxgen@163.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat(media): add OpenRouter image and video providers
OpenMAIC ships six separate video providers (Veo, Kling, Seedance,
MiniMax, Grok, HappyHorse) and seven image providers, each needing its
own key. OpenRouter fronts those same model families behind one key and
one account, so this adds it as a provider on both sides.
Both use OpenRouter's dedicated media endpoints, not chat-completions:
- Image: POST /images -> { data: [{ b64_json }] }
- Video: POST /videos -> 202 { id, status }, poll GET /videos/{id},
then GET /videos/{id}/content for the mp4 bytes
The model list is fetched live from GET /images/models and
GET /videos/models through /api/openrouter-models rather than pinned in
the registry: OpenRouter hosts 48 image and 28 video models today and
adds more, so a hardcoded shortlist would decide for the operator which
models exist. The registry keeps a three-entry seed as an offline
fallback, and the existing custom-model UI still accepts any model id.
Both catalogs answer unauthenticated, so the picker fills before a key
is pasted; a key is forwarded when present for proxied base URLs.
Adapter contracts are covered by stubbed-fetch tests (request shape,
empty-response handling, and the video job state machine including
terminal failure). No test performs a billable call.
Closes#1355
* fix(media): validate the key and tolerate a pasted endpoint URL
Three fixes found while configuring the new provider:
1. Both connectivity probes hit the model catalogs, which answer 200
unauthenticated — so "Test Connection" reported success for any
string, including an invalid key. Probe GET /key instead: equally
cheap, and it actually rejects a bad key.
2. The settings field is labelled "Base URL" but the panel echoes it
back as "Request URL", so pasting the full endpoint
(https://openrouter.ai/api/v1/images) is the natural mistake. That
built /api/v1/images/images and 404'd. Trim a trailing slash and a
trailing /images or /videos so both forms work; a proxy path that
merely contains the word is left alone.
3. The image and video settings panels read `data.message` on a failed
test, but failures answer with `error` (apiError) and only successes
carry `message`. Every failing connectivity test — for any provider,
not just OpenRouter — rendered "connection failed: undefined" instead
of the reason. Pre-existing; surfaced by 1 and 2 above.
Closes#1355
* fix(media): make every OpenRouter model selectable, and always select a provider
Two gaps found while configuring the new provider.
The settings Models list is a read-only catalog for every provider; the
actual model picker is the media popover. That picker built its groups
from the static registry array, so OpenRouter offered only the
three-entry seed while settings listed the full live catalog — the
models were visible but not choosable. Feed the same live catalog into
the popover, fetched only once the provider is usable so an
unconfigured install makes no request.
Separately, `imageProviderId`/`videoProviderId` are empty until a
provider is chosen (first-run auto-config leaves them blank when the
server reports no media provider). Opening the settings panel on an
empty id selected nothing: the header rendered the missing name key as
"settings.undefined", and Test Connection posted a blank
x-image-provider/x-video-provider, so it failed with "No image/video
provider configured" whatever key was typed. Fall back to the first
catalog entry so the panel always has a selection. Pre-existing and not
specific to OpenRouter.
Closes#1355
* fix(tts): request a browser-playable format from custom providers
`generateOpenAITTS` serves every custom OpenAI-compatible TTS provider but
never sent `response_format`, so it inherited whatever each provider
defaults to. OpenAI defaults to mp3; OpenRouter's /audio/speech defaults
to raw `pcm`. The unknown content type then fell through to the `'mp3'`
default below, the client built `data:audio/mp3;base64,…` from headerless
PCM samples, and playback failed with "no supported source was found" —
while the server logged a clean 200, because the audio really was
generated. Name the format instead of inheriting it.
Also stop mislabelling an unrecognised body: `pcm`/`l16` now raises a
message naming the cause, and `aac`/`opus` are recognised.
Two supporting fixes:
- /api/openrouter-models normalises its base URL the way the adapters do
and falls back to the public catalog when a custom base URL fails, so a
typo in a free-text settings field cannot empty the model picker. Also
types the headers object so tsc accepts the conditional.
- provider-neutrality-guard pins exact per-vendor occurrence counts in
lib/server/provider-config.ts. Adding the image and video env entries
raises "openrouter" from 2 to 6 (each entry contributes both its key and
its value); CI failed without the bump.
Closes#1355
* fix(security): never send the operator key to a client-chosen host
Review found `/api/openrouter-models` was an SSRF and key-exfiltration
path, and the finding is correct. The route took `x-base-url` from the
caller at highest precedence while preferring the *server* env key, so any
caller could make the server send the operator's OpenRouter credential as
an `Authorization: Bearer` header to an arbitrary URL. The route's own
comment claimed it followed `/api/verify-image-provider`; that pattern
runs `validateUrlForSSRF` on client base URLs, and this route did not.
The boundary is now explicit: the server key travels only to the
operator's own base URL. A client-supplied URL is SSRF-validated and
carries only that caller's own `x-api-key` — the server key is dropped —
and the unauthenticated public-catalog fallback never forwards a
credential chosen for a different host. Redirects are no longer followed
(`redirect: 'manual'`), since a redirect would carry the Authorization
header off-host and reopen the same hole, and upstream reads are bounded
by a timeout.
The per-URL cache is now keyed by destination *and* a hash of the
credential, and bounded to 64 entries with oldest-first eviction, so
client-supplied URLs cannot grow it without limit and one caller's
key-authorised catalog is never served to another.
Also from the review:
- The image adapter discarded the reported `media_type`. The
orchestration layer wraps a bare `base64` as `data:image/png`
unconditionally, so jpeg/webp results were mislabelled; the adapter now
returns a data URL carrying the real type.
- Adapter generation and poll requests set `redirect: 'manual'`, matching
the `/key` probe that already did.
- `runPolledTask` accepts an `AbortSignal` so the sleep between polls is
cancellable; the video adapter passes the caller's signal. Without it a
cancelled generation still slept out a full 10s interval.
Tests cover the highest-risk paths the review named: which credential
reaches which URL, that an SSRF-rejected destination is never contacted,
that the fallback is unauthenticated, cache isolation between callers,
and MIME preservation.
Findings 2 (base-URL normalisation) and 4 (neutrality-guard debt) were
already fixed in d553a08, pushed after the review was submitted; CI is
green on that commit.
Closes#1355
* ci: retry flaky voice clone timeout
* fix(vercel): keep OpenRouter catalogs within Hobby function limit
* fix(vercel): avoid tracing self-hosted sharp binaries
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Browser TTS auto-detection only distinguished Chinese (CJK ratio) from
everything else (en-US), so Vietnamese narration was spoken by an English
voice. Add Vietnamese detection to the shared language helper and bind an
installed vi voice in the playback engine when the user has not picked one
explicitly:
- lib/audio/browser-tts-preview.ts: new detectSpeechLang() (zh-CN /
vi-VN / en-US), used by both the Test TTS preview and playback. A hit on
đ/ơ/ư or the U+1EA0-U+1EF9 precomposed block (ớ, ừ, ồ, ế, …) — absent
from French/Romanian Latin — marks Vietnamese; bare ă/â/ê/ô only count
toward a low ratio.
- lib/playback/engine.ts: use the shared helper; when it reports vi-VN,
prefer an installed vi voice so pronunciation is correct out of the box.
An explicitly configured voice still wins.
- tests/audio/detect-speech-lang.test.ts: zh/vi/en/fr cases.
Verified end-to-end (Playwright WebKit): utterances carry lang vi-VN with
an installed Vietnamese voice auto-selected, advancing through every line.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(generation): tolerate non-array mediaGenerations in generated outlines
LLM-generated outline JSON can carry mediaGenerations as a string or
object instead of an array. Every downstream consumer (media
orchestrator, video manifest, scene generator) requires an array, and
uniquifyMediaElementIds called .map() after only a falsy check, so one
malformed outline aborted course generation with
"outline.mediaGenerations.map is not a function".
- sanitize at parse time: drop non-array mediaGenerations from each
enriched outline (outline-generator)
- harden uniquifyMediaElementIds: treat non-array values as absent,
strip them, and open the early-return guard on field presence so
all-malformed outlines still get sanitized
- add unit tests covering array, string, object, number, and missing
shapes
* chore(generation): bump package version to 0.3.8
Required by the package version bump check: the PR changes the
publishable @openmaic/generation package sources.
* test(generation): use correct SceneOutline fixture fields in outline media tests
The first commit of this branch accidentally staged an earlier draft of
the test file (sceneType instead of type, stale id regex). Re-stage the
final version that passes tsc and the full suite locally.
---------
Co-authored-by: PassCode023 <269712126+PassCode023@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat(token-plan): add TokenDance one-key preset for every modality
TokenDance is a model gateway: chat and images are OpenAI-compatible at
/gateway/v1, and the same key authenticates vendor-protocol routes on the
same host (Ark, MiniMax, Bocha). The preset reuses the existing adapters
with those route prefixes as base URLs, so one key lights up LLM, image,
video, TTS and web search from Settings -> Token Plan.
- providers: add a built-in `tokendance` OpenAI-compatible provider
(TOKENDANCE_* env prefix, logo, provider name in all locales)
- token-plan: add the TokenDance preset (Seedream image, MiniMax H3 video,
MiniMax speech TTS, Bocha web search)
- seedream: use a base URL that already ends in a version segment verbatim,
so gateway routes like `/ark/v3` do not get `/api/v3` appended
- minimax-video: route H3-family models through the v2 task API (content
array submit, task-envelope poll); connectivity checks for H3 probe auth
on the v2 query route instead of submitting a billable task
- README: add a one-key quick example and replace the Gemini-specific
model recommendation with a provider-agnostic setup recommendation
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qrrq9CPwb718mpouz8Y2KL
* fix(token-plan): accept preset web-search base URLs and report H3 dimensions per ratio
- web-search: the client base URL allowlist also accepts the exact base URL
a built-in token plan preset writes for that provider, derived from
TOKEN_PLAN_PRESETS. Applying a plan whose web-search route is not an
official vendor host previously stored a URL that the route rejected with
400. Any other client URL is still rejected.
- minimax-video: report H3 v2 clip dimensions for 16:9, 9:16, 4:3 and 1:1
instead of assuming landscape for every non-portrait ratio.
- tests: pin the allowlist for every preset, the 1:1 H3 dimensions, and
clear TOKENDANCE_* in the provider-config env isolation list.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qrrq9CPwb718mpouz8Y2KL
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The workbench tools store generated bytes in the asset pool under the shared principal and write the allocated id into the document, the same discipline as the classic chain since #1392; the runner's putScene creates the reference rows and commits the allocations. The video completion patch rewrites every placeholder slot and retires anything that would shadow the new id; immediate render is preserved by leasing the id at the render boundary. A store-full refusal fails the tool with a model-readable error and writes nothing. Legacy /api/classroom-media documents keep rendering. Closes#1522.
Detach and stop the active playback engine before asynchronous scene teardown so switching to a non-playable scene cannot leave narration, timers, or effects running.\n\nRecheck engine ownership after lecture-session creation in manual and auto-play paths, close stale sessions, and reject progress callbacks from detached engines. Add regression coverage for all three races.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Extracts commitToPool, the single client-side sequence for storing bytes in the asset pool, writing the allocated id back, and mirroring locally; routes the media pass, narration adoption and fresh TTS through it. A store-full refusal during TTS now retains the already-billed clip so the next load adopts it with zero provider calls. Closes#1467.
Audio provider requests (TTS, ASR, voice registration and voice cloning)
validated a client-supplied base URL once and then issued a plain fetch with
default redirect-follow and no pinned dispatcher. A base URL that resolved to a
public address but answered with a redirect to an internal one was followed, and
a DNS answer that changed between the guard's lookup and the connect reached an
internal host — both readable in-band.
- Route every lib/audio provider request through a new
lib/server/audio-provider-fetch.ts that combines per-hop redirect
re-validation with a pinned undici dispatcher, so the socket can only reach an
address the guard validated, on every hop.
- Select the public-vs-local policy server-side from isServerConfiguredProvider;
a client-supplied base URL is always strict public and can never reach a
private, loopback or cloud-metadata address, even with ALLOW_LOCAL_NETWORKS
set.
- Pin the result-audio download hop as well, keeping its host allowlist and
redirect:'error'.
- Add a coverage-matrix test that fails if any lib/audio module regains a raw
provider fetch.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
When ACCESS_CODE is set, verification tokens (timestamp.HMAC) never expired
because neither verifier checked the timestamp, and POST /api/access-code/verify
had no attempt throttling.
- Enforce a 7-day token lifetime in a shared, Edge-safe module used by both the
Node verifier and the middleware Web-Crypto verifier, and reject non-canonical
signatures in both.
- Rate-limit verification only when the client identity is trusted
(TRUST_PROXY_HEADERS=true): a per-client sliding window (10 failures / 60s)
whose attempt is reserved atomically at check time, returning 429 with
Retry-After. Without a trusted proxy the app cannot attribute requests to a
client, so no shared throttle is applied — a long random ACCESS_CODE is the
protection, and a warning is logged when it is short.
- Store bounded, copied identity keys so a large forwarding header cannot retain
memory.
- Document the behavior in .env.example, README, and configuration docs.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(classroom): keep transient load failures off the not-found card
/api/classroom 5xx and network errors used to become null, so ClassroomSurface treated them like a missing course and rendered the terminal not-found claim with no retry. Classify fetch answers as found/absent/unavailable and only show not-found on a positive 404/410 miss.
Fixes#1450
* fix(classroom): handle terminal load states
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
addCustomTTSProvider stores the dialog Base URL in customDefaultBaseUrl and leaves baseUrl empty. isTTSProviderConfigured ignored that field, so generation silently skipped narration even after Test TTS succeeded. Accept the dialog URL so already-saved custom providers start working without retyping the field.
Fixes#1471
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Why:
- Keep the image URL-guard regression deterministic when contributors export generic OpenAI credentials.
- Prevent this test file from deleting inherited environment values for later tests in the same worker.
What:
- Stub image-provider and generic OpenAI fallback variables as absent with restorable Vitest environment stubs.
- Restore environment stubs after each case.
- Assert the allowed-local-network case reaches generation with the client API key.
Risk:
- Test-only change; production provider selection and SSRF behavior are unchanged.
Tests:
- 120 focused server security and provider tests with OPENAI_API_KEY=openmaic-test-sentinel
- pnpm test with OPENAI_API_KEY=openmaic-test-sentinel (7894 passed, 81 skipped)
- pnpm check; pnpm lint; tsc --noEmit; pnpm check:i18n-keys
- pnpm build
- package version, internal dependency range, and Node engine checks
Live Docs:
- Not applicable; test isolation defect tracked in #1475.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
App wiring for the @openmaic/storage 0.31.0 lifecycle: reference tracking and document references are paired unconditionally and declared at startup, course deletion withdraws references inside the tombstone transaction, ASSET_PENDING_TTL_MS configures the pending window, and the dead client-side reclamation code is removed.
* fix(agent-runtime): allow one owner material to be bound to multiple sessions
`agent_session_materials.id` is a global primary key, but
`bindOwnerMaterialsToSession` de-duplicated on `(sessionId, id)` and then
inserted the owner-side material id as the row id. Binding the same owner
material to a second session therefore raised a duplicate-key error
(23505) that the route surfaced as HTTP 500, so an owner material could
only ever be used by one course.
Keep row ids globally unique (every extraction method keys on `id` alone)
and store the shared owner id as metadata instead: the binder now mints a
fresh session row id, records `owner_material_id`, and looks that column
up for idempotent rebinding; a partial unique index on
`(session_id, owner_material_id)` adjudicates concurrent rebinds and the
loser adopts the winner's row. The schema change is additive and
idempotent (`ADD COLUMN IF NOT EXISTS` + `CREATE UNIQUE INDEX IF NOT
EXISTS`), so existing databases upgrade in place with `NULL` for legacy
rows. No FK from `owner_material_id` to the owner library, to keep
owner-library deletion decoupled from session rows.
Tests: backend-neutral contract test (same owner material bound to two
sessions, both readable), PGlite in-place upgrade test, pinned-schema
update, and a host-level test covering the route's 202 path.
* fix(agent-runtime): reuse legacy owner-material bindings and clean up losing uploads on concurrent bind
Rows written by the previous binder use the owner upload id as the session
row id and leave `owner_material_id` NULL, so the new owner-id lookup missed
them and a rebind created a duplicate row and copied the bytes again. The
binder now falls back to the session row id and adopts it only when it is
unambiguously that owner upload: source kind, matching title, no source URL
or derivative, no text, and the deterministic legacy object key with a byte
length that matches the owner record. Adoption stamps `owner_material_id`
through a conditional `backfillOwnerMaterialId` update that leaves extraction
columns untouched; a lost race re-reads the fast path. A different session is
still a fresh row.
The concurrent-bind loser also left its uploaded object behind after
adopting the winner. It now removes that object when the winner references a
different key, best-effort so a failed cleanup cannot fail a bind that
succeeded.
Tests: host-level PGlite regressions for the legacy rebind (id, row count,
byte copies, extraction state, and backfill) plus a different-session bind,
and a controllable byte store that parks the loser after its upload so the
winner commits first, asserting one row and only the winner's object remain.
A storage unit test pins the backfill contract. Bumps @openmaic/storage to
0.30.1 for the new public store method.
