ZipParseLimits was optional field by field and both import paths called
parseZip with no argument, so every bound the parser already implemented was
inert: a .pptx could inflate without limit.
- apply the defaults per field, so a caller overriding one bound no longer
silently drops the others, and Number.POSITIVE_INFINITY switches a single
bound off
- add maxCompressionRatio for text parts, defaulting to 200:1. It applies to
slides, layouts, themes and chart XML only. Uncompressed bitmaps and silent
PCM are ordinary deck content sitting at the top of DEFLATE's range — a
solid-colour 24-bit BMP measures ~1027:1, 30s of silent stereo PCM ~1016:1,
and a 1920x1080 white screenshot with a grid ~297:1 — so a fixed ratio there
rejects real decks. Media and embedded objects are bounded by
maxEntryUncompressedBytes and maxMediaBytes instead, which for an honest
archive are checked before anything is inflated. Text parts have no such
problem: real decks peak around 19:1, so 200 leaves ample headroom.
- reject a supplied NaN and a non-integer maxEntries rather than letting them
compare false against every bound and disable it silently
- export the defaults and the limit type so callers can extend them
- add the first tests for this parser: the regression itself (a call with no
limits must still apply the bounds), the per-field defaulting invariant, each
bound pinned, the binary-part exemption pinned with a solid-colour BMP, a
silent WAV and an embedded object, and the repo's own regression deck
The bounds read the sizes the archive declares about itself, which an archive
is free to understate. One that declares small and inflates large passes all of
them and is stopped only by JSZip's own size check, after that entry has been
inflated — peak RSS in the reviewed case was ~1.19 GB, and with maxConcurrency 8
several such entries inflate in parallel. Bounding the allocation itself needs a
byte-budgeted inflate inside the read path, which is a larger change than
activating the bounds that already existed here.
Version to 0.3.0: ^0.2.6 admits 0.2.7, and this changes default behaviour for
existing callers. Consumers pinned to ^0.2.x need to move deliberately.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
#930 made the connectivity probes in the media adapters pass
`redirect: 'manual'`. The generation and poll calls in the same 14 files were
left following redirects. Those requests carry the provider credential and go
to a base URL that comes from provider settings a caller can supply, so a 3xx
would replay the credential at a host the caller chose — and the redirect
target can be an address the outbound guard already refused.
Every such call now passes `redirect: 'manual'` and rejects a 3xx through a
shared `assertNotRedirected` helper, which reports it as
"<provider>: Redirects are not allowed (HTTP <status>)" instead of letting the
generic failure path describe it as a provider error.
- 26 call sites across the image adapters (seedream, openai, qwen, grok,
lemonade, minimax, nano-banana), the video adapters (seedance, kling, grok,
happyhorse, minimax, veo) and ComfyUI's submit and image fetch.
- ComfyUI's `pollHistory` keeps its contract of handing the caller a retryable
failure rather than aborting the generation: it logs the refusal and returns
null.
- ComfyUI's same-origin workflow load is deliberately untouched — it reads the
app's own public/ asset, carries no credential and is not provider-influenced.
- The two adapters added since #930 (OpenRouter image and video) already did
this.
tests/media/adapter-redirects.test.ts covers one case per adapter family. Each
serves a 302 and asserts that the call rejects with the redirect message and
that every request carrying an init object asked fetch not to follow redirects;
each case fails if its adapter stops passing `redirect: 'manual'`.
The HappyHorse test asserted the exact request init, so it now includes the new
option.
AI-assisted commit
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Two independent Grok failures seen when the provider is reached through a relay
(a custom base URL) instead of api.x.ai directly:
- lib/ai/providers.ts: a long non-streaming chat completion was cut off by the
relay with a 504 at its idle timeout (~5 min), because nothing is sent
upstream until the model has the whole answer. Adding 'grok' to the existing
streaming-compat path (OPENAI_COMPAT_USE_STREAMING_CHAT=true) keeps bytes
flowing across the idle window; the SSE is buffered back into a normal JSON
response for the caller.
The path is for relays only. usesCustomOpenAIBaseUrl recognises OpenAI's
origin alone, so Grok's own api.x.ai also read as "custom" and was forced
onto the compat transport; the provider's native endpoint is now excluded.
- lib/media/adapters/grok-image-adapter.ts: response_format 'url' returns a
link on the relay's CDN host (imgen.x.ai), which may be unreachable from the
server's network. The generation then failed at the follow-up fetch through
/api/proxy-media even though the image had been produced successfully.
'b64_json' inlines the bytes and removes that second hop.
Inline bytes declare no media type, so the adapter reports one on
ImageGenerationResult and returns a typed data URL, which is the shape
openrouter-image-adapter already uses. Consumers take the type from there:
agent image persistence records it, the client's stored media row keeps the
type its data URL states, and classroom-media-generation names the file with
the matching extension. A JPEG is no longer stored, served or named as a PNG.
The process-wide undici timeout that previously accompanied these changes is
dropped: upstream #1404 now gives LLM calls their own undici headers/body
timeouts, which covers the same failure without raising the defaults globally.
Co-authored-by: ciclou1 <ciclou1@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Share corrected line bounds across renderer and React editing paths, preserve double-elbow routing, and translate PPTX points into the shape bounding box.
Refs #674 and the prior implementation/review in #675. AI-assisted.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(media): emit keyframe images as data URLs
The local media extractor stored keyframe bytes as raw base64 in
`DocumentAsset.data`. Every other extractor emits a data URL, and the
document bundle forwards this field as `pdfImages[].src` to `storeImages`,
which decodes it with `decodeBase64DataUrl`. With no `data:` prefix the
comma split yields no payload, so `atob(undefined)` throws and course
generation fails with "Failed to store image bundle at image img_1".
`pdf-compat`'s `dataUrlMimeType` also derives the image asset mime from
this same field, so the declared `image/webp` was silently dropped too.
Emitting `data:${mime};base64,...` matches `mineru-parser`, which already
normalizes prefix-less base64 the same way.
* fix(media): decode keyframe data URLs
* fix(media): accept legacy and data-url assets
* fix(media): reject malformed or empty media asset data URLs
A string that starts with data: but does not parse as a data URL used to
fall through to the raw-base64 path, where Node's decoder skips
non-alphabet characters and yields garbage bytes. Throw instead, and
reject an empty base64 payload rather than storing a 0-byte image.
Correct the keyframe test comment: local keyframe assets are consumed by
material extraction, and the test needs ffmpeg so it does not run in CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(export): surface video render rejection reasons
* fix(export): keep render diagnostics out of user toasts
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Discussion narration created a new Audio element per line, so on mobile every
line after the first had its programmatic play() refused with NotAllowedError:
the dialogue went silent while the lesson kept advancing. This is the
discussion side of #1474, which #1477 fixed for the narration player.
One module-scoped element now serves every line, kept separate from the
narration element in AudioPlayer so neither can cut the other off. Because
element identity can no longer tell lines apart, the stale-event guard became a
per-line token, handlers are assigned rather than added (a listener would
accumulate once per line), and finish()/cleanup() release the line by removing
the source attribute and reloading instead of leaving the element pointed at
the page's own URL.
Co-authored-by: talkman <jaxgen@163.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
ACCESS_CODE unset remains fail-open in middleware, document reads are
capability-by-id via the anonymous owner cookie (not x-learner-key),
and the README action/skill counts match the Action union and
skills/agent-runtime.
Closes#1587
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
The comment implied the old-space limit prevents a VM-level OOM. It only
makes the V8 heap limit explicit; measurements showed the package build
completing under a 1 GiB container limit, not a guarantee against host OOM.
Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(docker): build workspace packages in the builder stage, not during install
The root postinstall builds nine workspace packages inside `pnpm install`.
In the Docker deps stage that step peaks above 1 GiB, almost entirely from
the importer's rollup + terser pass, which exhausts small Docker VMs and
hangs the build (a 1 GiB container fails with a JS heap OOM).
- deps: `pnpm install --frozen-lockfile --ignore-scripts`, so the stage only
resolves and links dependencies (peak ~1051 MiB -> ~315 MiB).
- builder: run the same chain explicitly via the new `build:packages` script,
with `--max-old-space-size=1024` so a runaway build fails with a clear heap
error instead of taking down the VM; public/vendor is produced here now.
- `postinstall` delegates to `build:packages`, so local installs are unchanged.
- CI: build the deps + builder stages when an image input changes; the main
Dockerfile was not built in CI before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf
* ci(docker): build the main image for any non-docs change
The builder stage copies the whole build context, so application sources
and config such as next.config.ts are image inputs too. Only skip the
image build when every changed file is documentation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The vocational gate keys off requirements.taskEngineMode in the
/api/generate/scene-content body, but only the first scene
(generation-preview) sent it. Scenes 2..N and retries run through
useSceneGenerator.generateRemaining / fetchSceneContent, which never
sent requirements, so resolveVocationalActive returned false and
applyOutlineFallbacks rewrote every later procedural-skill scene to
diagram — silently dropping the task-engine training mechanism for
most of a vocational course (the outline says procedural-skill while
the content path strips it).
Thread the persisted stage.taskEngineMode through GenerationParams into
both fetchSceneContent bodies (the retry path inherits it via
lastParamsRef), mirroring what the first-scene request already sends.
Server contract is unchanged; the flag is still ANDed with the
OPENMAIC_ENABLE_VOCATIONAL env gate server-side.
Closes#715
Co-authored-by: ly-wang19 <ly-wang19@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* refactor(upload): derive the workbench material MIME policy from the shared format registry
#1498 fixed the Kylin generic-Office-MIME failure on both upload paths but
left the workbench with its own extension→MIME table, alias map, and
generic-MIME set next to the document registry, and the two had already
drifted. The workbench policy now derives every MIME/extension fact from
lib/document/mime.ts (the single source of truth); only the accepted-format
list stays workbench policy — fixed and extractor-independent, unlike the
classic path's provider-scoped whitelist.
- Register csv and webm in DOCUMENT_FORMATS (accepted by no document
provider, so classic-mode whitelists are unchanged) and add the
audio/x-m4a alias to m4a.
- material-upload-policy.ts keeps its export names (route, session-store,
and composer consumers unchanged) but resolves, whitelists, and builds
its accept string from registry helpers.
- The workbench gate now also accepts the registry's curated aliases it
previously missed: image/jpg, text/x-markdown, and audio/x-wav (stored
canonically as audio/wav).
Closes#1589.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* test(workbench): pin the audio/mp3 alias closure in the material policy
Deriving the workbench gate from the shared registry normalization
(#1589) also accepts the browser-reported audio/mp3 alias the hand-rolled
alias map rejected — the same gap class as image/jpg and text/x-markdown,
so pin it alongside them and name it in the comment.
Co-Authored-By: Claude Code <noreply@anthropic.com>
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(generation): keep narration speech TTS-readable — no formulas or LaTeX
Narration speech authored by the four *-actions prompts could carry raw
formula notation (a^2 x / y) or LaTeX (\frac{a}{b}), which TTS engines
read out as gibberish. The prompts had no TTS readability rules, and the
element list fed to them includes raw LaTeX via `Formula:` entries,
inviting verbatim copying into speech.
Add a shared `speech-tts-readability` snippet and compose it into the
speech sections of slide-actions, quiz-actions, interactive-actions, and
pbl-actions via the existing {{snippet:...}} mechanism. The name
references speech/TTS so it cannot be confused with slide-content's
visual LaTeX rules or reused there.
Refs #1585
Co-Authored-By: Claude Code <noreply@anthropic.com>
* chore(generation): bump to 0.3.10, main already released 0.3.9
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat: sample declared interactive state before classroom questions
* test: exercise generated publication example through the iframe reader
* chore(generation): bump package version for observation prompt contract
* fix: keep interactive state optional on insecure HTTP origins
* fix(playback): keep component picking and separate state from reference
A declared state interface replaced the component picker with a forced
whole-area `#experiment` reference, removing the per-component selection
and outline that `main` already ships. Sampling was also gated on the
reference selector, so the only way to obtain state was to give up the
selection.
Reference identity and area state are now independent request-scoped
evidence items:
- `handleToggleElementPick` always arms the picker again, so a scene that
declares the interface keeps main's per-component selection, outline,
and send-time clearing.
- `sampleInteractiveState` follows the current Scene instead of the draft
reference, so an unreferenced follow-up still reports current facts and
never re-creates or extends a reference.
- The Host carries area state with or without a component reference. The
evidence header names both identities and refuses to present area facts
as properties of the referenced component.
- `metadata` is absent when only area state travels, so no element
identity and no Spotlight authorization can be derived from it, and the
accepted-reference receipt stays driven by explicit references only.
Review follow-ups in the same change:
- Client sampling follows `NEXT_PUBLIC_COURSEWARE_REFERENCE_ENABLED`.
An ungated packet turned an ordinary Pi question into a 400 while the
reference feature was disabled.
- A Scene that declares the interface always receives an availability
boundary, including when the browser produced no packet at all. It is
reported as `not-sampled` rather than the previous `no-interface`,
which was a false statement about an activity that does declare one.
Courseware without the interface keeps its unreferenced behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(generation): register the observation snippet as a packaged asset
The interactive-observation snippet is referenced by all six widget
content templates but was never added to the packaged-asset manifest, so
the asset test and the golden scene prompt both failed.
- `SNIPPET_IDS` now lists `interactive-observation`, restoring both the
"exactly the generation-owned templates and referenced snippets" check
and the "every referenced snippet is packaged" cross-check.
- The interactive system-prompt snapshot is re-pinned. The change is
purely additive: the snippet is appended to the simulation template.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pi): stop injecting state constraints while the feature is disabled
The route rejects a request that carries a reference or a state packet
while `NEXT_PUBLIC_COURSEWARE_REFERENCE_ENABLED` is off, but an ordinary
question carries neither. It still reached the Host, and a Scene that
declares the state interface then received the full page-state block —
several kilobytes of constraints about evidence the deployment can never
sample.
The Host now returns before building that note when the feature is off.
A route-level regression asserts that neither the Director prompt nor the
Child prompt gains `PAGE-REPORTED STATE` in that configuration; disabling
the guard makes it fail with exactly that symptom.
Also reopens the composer before the unreferenced follow-up in the
classroom browser spec. An accepted answer may close it, which made the
assertion flaky without changing the behaviour under test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pi): decouple reference Scene from state freshness and bound assembled evidence
Cross-review found two defects in the request-scoped state evidence.
The reference's Scene was folded into the sample's staleness test. The packet
is already bound to the current Scene by the identity check above it, so a
valid current-Scene sample was being discarded as `stale-sample` purely because
the student's component reference came from an earlier Scene. Reference and
area state are independent evidence items; freshness is a property of the
sample alone. With the coupling gone the two can now disagree on Scene, so the
note says so explicitly rather than letting the model attribute area facts to a
component that may not be on the current Scene.
The assembled evidence had no stated output budget. The static component packet
is bounded to 24,000 code points upstream, but that bound covers the static
packet alone; the note and the escaped observation JSON were appended without a
recheck. Escaping `<` for the prompt expands one code point into six, and `<` is
legal in a label or a fact value, so a packet the Host accepts could assemble to
149,385 code points. The budget is now declared as the static bound plus the room
the note frame needs, which is what makes the degradation terminate. Over budget,
the state body drops whole to an explicit `unavailable` statement: truncating the
JSON would emit a broken packet, and thinning a `complete` relation set would turn
an exhaustive set into a false one. The relationship summary degrades with it, so
the prose never asserts COMPLETE over a body that is gone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(pi): keep slide references independent of activity state
* refactor(interactive): simplify declared state and unify iframe preparation
Accept any JSON report within byte and depth budgets, without generated field
requirements or relationship-completeness semantics. Keep publishState and an
optional rendered result in the generation guidance.
Prepare the observation responder through patchHtmlForIframe and let the pool
own document identity, preserving state across placeholder remounts. Settle
sampling failures locally and align browser/server nesting limits.
Cover permissive JSON delivery, resource limits, lifecycle, legacy behavior,
and real renderer remounts with focused regression tests.
* fix(generation): publish automatic activity changes with clear positions
* fix(interactive): report missing legacy scope as no interface
* fix(interactive): guard sampling capabilities and bind scopes lazily
* chore(generation): bump version after main integration
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
#1514 moved every lib/audio provider call to the npm undici package's
fetch so the pinned dispatcher is guaranteed to be honored. But the
adapters kept building multipart bodies with the platform-global
FormData — a class of Node's bundled undici — and undici's serializer
brand-checks a FormData body against its own class. A foreign FormData
fell through to the string branch and left the process as
`content-type: text/plain;charset=UTF-8` with the 17-byte literal
`[object FormData]` as the whole body: the audio bytes and every field
(including `model`) were dropped, downstream gateways fell back to
whisper-1 and answered 503, and every multipart audio request (ASR,
TTS FormData paths, voice registration/cloning) failed with a generic
internal error.
Bare Blob/File, string/JSON/Buffer and stream bodies were never
affected: undici 7.29.0 exports no File/Blob classes of its own and
its bare-body brand checks (and multipart part handling) bind the
platform classes.
Normalize the body in the transport, once, before either transport
path serializes it, so the adapters can keep the platform globals as
their public API boundary: a foreign FormData is re-created as
undici's own with every entry carried over verbatim — `append` (not
`set`) so repeated field names survive, and no filename argument so a
platform File part keeps its own name/type/lastModified. Everything
else passes through untouched.
Also drive real loopback regression tests with a platform-global
FormData (direct, with repeated field names, empty, and across a 307
redirect hop whose per-hop loop re-issues the normalized body) and a
bare Blob, asserting multipart on the wire instead of
`[object FormData]`, and update the voxcpm unit test that asserted
the buggy contract (a platform FormData at the undici boundary).
Fixes#1579
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
setup-node v5 turns on package-manager caching automatically when
package.json declares `packageManager`, which requires pnpm on PATH. The
`publish` and `mark` jobs only use npm on prebuilt tarballs and never
install pnpm, so the first package publish after the Node 24 action
migration failed in setup-node with "Unable to locate executable file:
pnpm".
Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
On mobile browsers classroom narration plays the first segment and then goes
silent for every following one, while the lesson keeps advancing: the player
created a new HTMLAudioElement per line, and only the first line is covered by
the user's gesture, so every programmatic play() after it is refused with
NotAllowedError and the engine falls back to its reading-time timer. #651/#652
fixed the blob leak from that rejection, not the missing voice.
Keep one element per player instead:
- getAudioElement() creates it on first use and every line reuses it, so the
element the first gesture activated stays playable for the rest of the lesson
- stopAudioElement() releases the line's state rather than the element: onended
cleared, src removed, load() called -- a stopped line must not keep reporting
speech it is no longer playing, nor retain narration bytes through a revoked
object URL until the next play()
- onended is assigned rather than added: the element now outlives a single line,
so a listener would accumulate once per segment and call the engine back
several times for one line
tests/audio/audio-player-element-reuse.test.ts stubs the mobile policy itself
(the first element plays, every element created after it is refused): the reuse
tests fail on main and pass with this change.
Verified on a live deployment at the fixed entry point (instrumented Chromium,
default autoplay policy, one real click on Play): a single element for 8+
consecutive lines, no refused play, currentTime advancing line by line.
Fixes#1474
Co-authored-by: Shaoxuhua <jaxgen@163.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat(media): add OpenRouter image and video providers
OpenMAIC ships six separate video providers (Veo, Kling, Seedance,
MiniMax, Grok, HappyHorse) and seven image providers, each needing its
own key. OpenRouter fronts those same model families behind one key and
one account, so this adds it as a provider on both sides.
Both use OpenRouter's dedicated media endpoints, not chat-completions:
- Image: POST /images -> { data: [{ b64_json }] }
- Video: POST /videos -> 202 { id, status }, poll GET /videos/{id},
then GET /videos/{id}/content for the mp4 bytes
The model list is fetched live from GET /images/models and
GET /videos/models through /api/openrouter-models rather than pinned in
the registry: OpenRouter hosts 48 image and 28 video models today and
adds more, so a hardcoded shortlist would decide for the operator which
models exist. The registry keeps a three-entry seed as an offline
fallback, and the existing custom-model UI still accepts any model id.
Both catalogs answer unauthenticated, so the picker fills before a key
is pasted; a key is forwarded when present for proxied base URLs.
Adapter contracts are covered by stubbed-fetch tests (request shape,
empty-response handling, and the video job state machine including
terminal failure). No test performs a billable call.
Closes#1355
* fix(media): validate the key and tolerate a pasted endpoint URL
Three fixes found while configuring the new provider:
1. Both connectivity probes hit the model catalogs, which answer 200
unauthenticated — so "Test Connection" reported success for any
string, including an invalid key. Probe GET /key instead: equally
cheap, and it actually rejects a bad key.
2. The settings field is labelled "Base URL" but the panel echoes it
back as "Request URL", so pasting the full endpoint
(https://openrouter.ai/api/v1/images) is the natural mistake. That
built /api/v1/images/images and 404'd. Trim a trailing slash and a
trailing /images or /videos so both forms work; a proxy path that
merely contains the word is left alone.
3. The image and video settings panels read `data.message` on a failed
test, but failures answer with `error` (apiError) and only successes
carry `message`. Every failing connectivity test — for any provider,
not just OpenRouter — rendered "connection failed: undefined" instead
of the reason. Pre-existing; surfaced by 1 and 2 above.
Closes#1355
* fix(media): make every OpenRouter model selectable, and always select a provider
Two gaps found while configuring the new provider.
The settings Models list is a read-only catalog for every provider; the
actual model picker is the media popover. That picker built its groups
from the static registry array, so OpenRouter offered only the
three-entry seed while settings listed the full live catalog — the
models were visible but not choosable. Feed the same live catalog into
the popover, fetched only once the provider is usable so an
unconfigured install makes no request.
Separately, `imageProviderId`/`videoProviderId` are empty until a
provider is chosen (first-run auto-config leaves them blank when the
server reports no media provider). Opening the settings panel on an
empty id selected nothing: the header rendered the missing name key as
"settings.undefined", and Test Connection posted a blank
x-image-provider/x-video-provider, so it failed with "No image/video
provider configured" whatever key was typed. Fall back to the first
catalog entry so the panel always has a selection. Pre-existing and not
specific to OpenRouter.
Closes#1355
* fix(tts): request a browser-playable format from custom providers
`generateOpenAITTS` serves every custom OpenAI-compatible TTS provider but
never sent `response_format`, so it inherited whatever each provider
defaults to. OpenAI defaults to mp3; OpenRouter's /audio/speech defaults
to raw `pcm`. The unknown content type then fell through to the `'mp3'`
default below, the client built `data:audio/mp3;base64,…` from headerless
PCM samples, and playback failed with "no supported source was found" —
while the server logged a clean 200, because the audio really was
generated. Name the format instead of inheriting it.
Also stop mislabelling an unrecognised body: `pcm`/`l16` now raises a
message naming the cause, and `aac`/`opus` are recognised.
Two supporting fixes:
- /api/openrouter-models normalises its base URL the way the adapters do
and falls back to the public catalog when a custom base URL fails, so a
typo in a free-text settings field cannot empty the model picker. Also
types the headers object so tsc accepts the conditional.
- provider-neutrality-guard pins exact per-vendor occurrence counts in
lib/server/provider-config.ts. Adding the image and video env entries
raises "openrouter" from 2 to 6 (each entry contributes both its key and
its value); CI failed without the bump.
Closes#1355
* fix(security): never send the operator key to a client-chosen host
Review found `/api/openrouter-models` was an SSRF and key-exfiltration
path, and the finding is correct. The route took `x-base-url` from the
caller at highest precedence while preferring the *server* env key, so any
caller could make the server send the operator's OpenRouter credential as
an `Authorization: Bearer` header to an arbitrary URL. The route's own
comment claimed it followed `/api/verify-image-provider`; that pattern
runs `validateUrlForSSRF` on client base URLs, and this route did not.
The boundary is now explicit: the server key travels only to the
operator's own base URL. A client-supplied URL is SSRF-validated and
carries only that caller's own `x-api-key` — the server key is dropped —
and the unauthenticated public-catalog fallback never forwards a
credential chosen for a different host. Redirects are no longer followed
(`redirect: 'manual'`), since a redirect would carry the Authorization
header off-host and reopen the same hole, and upstream reads are bounded
by a timeout.
The per-URL cache is now keyed by destination *and* a hash of the
credential, and bounded to 64 entries with oldest-first eviction, so
client-supplied URLs cannot grow it without limit and one caller's
key-authorised catalog is never served to another.
Also from the review:
- The image adapter discarded the reported `media_type`. The
orchestration layer wraps a bare `base64` as `data:image/png`
unconditionally, so jpeg/webp results were mislabelled; the adapter now
returns a data URL carrying the real type.
- Adapter generation and poll requests set `redirect: 'manual'`, matching
the `/key` probe that already did.
- `runPolledTask` accepts an `AbortSignal` so the sleep between polls is
cancellable; the video adapter passes the caller's signal. Without it a
cancelled generation still slept out a full 10s interval.
Tests cover the highest-risk paths the review named: which credential
reaches which URL, that an SSRF-rejected destination is never contacted,
that the fallback is unauthenticated, cache isolation between callers,
and MIME preservation.
Findings 2 (base-URL normalisation) and 4 (neutrality-guard debt) were
already fixed in d553a08, pushed after the review was submitted; CI is
green on that commit.
Closes#1355
* ci: retry flaky voice clone timeout
* fix(vercel): keep OpenRouter catalogs within Hobby function limit
* fix(vercel): avoid tracing self-hosted sharp binaries
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Browser TTS auto-detection only distinguished Chinese (CJK ratio) from
everything else (en-US), so Vietnamese narration was spoken by an English
voice. Add Vietnamese detection to the shared language helper and bind an
installed vi voice in the playback engine when the user has not picked one
explicitly:
- lib/audio/browser-tts-preview.ts: new detectSpeechLang() (zh-CN /
vi-VN / en-US), used by both the Test TTS preview and playback. A hit on
đ/ơ/ư or the U+1EA0-U+1EF9 precomposed block (ớ, ừ, ồ, ế, …) — absent
from French/Romanian Latin — marks Vietnamese; bare ă/â/ê/ô only count
toward a low ratio.
- lib/playback/engine.ts: use the shared helper; when it reports vi-VN,
prefer an installed vi voice so pronunciation is correct out of the box.
An explicitly configured voice still wins.
- tests/audio/detect-speech-lang.test.ts: zh/vi/en/fr cases.
Verified end-to-end (Playwright WebKit): utterances carry lang vi-VN with
an installed Vietnamese voice auto-selected, advancing through every line.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(docker): create /app/data owned by the runtime user before dropping privileges
* docs(deployment): note one-time ownership repair for pre-existing data volumes
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(generation): tolerate non-array mediaGenerations in generated outlines
LLM-generated outline JSON can carry mediaGenerations as a string or
object instead of an array. Every downstream consumer (media
orchestrator, video manifest, scene generator) requires an array, and
uniquifyMediaElementIds called .map() after only a falsy check, so one
malformed outline aborted course generation with
"outline.mediaGenerations.map is not a function".
- sanitize at parse time: drop non-array mediaGenerations from each
enriched outline (outline-generator)
- harden uniquifyMediaElementIds: treat non-array values as absent,
strip them, and open the early-return guard on field presence so
all-malformed outlines still get sanitized
- add unit tests covering array, string, object, number, and missing
shapes
* chore(generation): bump package version to 0.3.8
Required by the package version bump check: the PR changes the
publishable @openmaic/generation package sources.
* test(generation): use correct SceneOutline fixture fields in outline media tests
The first commit of this branch accidentally staged an earlier draft of
the test file (sceneType instead of type, stale id regex). Re-stage the
final version that passes tsc and the full suite locally.
---------
Co-authored-by: PassCode023 <269712126+PassCode023@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat(token-plan): add TokenDance one-key preset for every modality
TokenDance is a model gateway: chat and images are OpenAI-compatible at
/gateway/v1, and the same key authenticates vendor-protocol routes on the
same host (Ark, MiniMax, Bocha). The preset reuses the existing adapters
with those route prefixes as base URLs, so one key lights up LLM, image,
video, TTS and web search from Settings -> Token Plan.
- providers: add a built-in `tokendance` OpenAI-compatible provider
(TOKENDANCE_* env prefix, logo, provider name in all locales)
- token-plan: add the TokenDance preset (Seedream image, MiniMax H3 video,
MiniMax speech TTS, Bocha web search)
- seedream: use a base URL that already ends in a version segment verbatim,
so gateway routes like `/ark/v3` do not get `/api/v3` appended
- minimax-video: route H3-family models through the v2 task API (content
array submit, task-envelope poll); connectivity checks for H3 probe auth
on the v2 query route instead of submitting a billable task
- README: add a one-key quick example and replace the Gemini-specific
model recommendation with a provider-agnostic setup recommendation
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qrrq9CPwb718mpouz8Y2KL
* fix(token-plan): accept preset web-search base URLs and report H3 dimensions per ratio
- web-search: the client base URL allowlist also accepts the exact base URL
a built-in token plan preset writes for that provider, derived from
TOKEN_PLAN_PRESETS. Applying a plan whose web-search route is not an
official vendor host previously stored a URL that the route rejected with
400. Any other client URL is still rejected.
- minimax-video: report H3 v2 clip dimensions for 16:9, 9:16, 4:3 and 1:1
instead of assuming landscape for every non-portrait ratio.
- tests: pin the allowlist for every preset, the 1:1 H3 dimensions, and
clear TOKENDANCE_* in the provider-config env isolation list.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qrrq9CPwb718mpouz8Y2KL
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The workbench tools store generated bytes in the asset pool under the shared principal and write the allocated id into the document, the same discipline as the classic chain since #1392; the runner's putScene creates the reference rows and commits the allocations. The video completion patch rewrites every placeholder slot and retires anything that would shadow the new id; immediate render is preserved by leasing the id at the render boundary. A store-full refusal fails the tool with a model-readable error and writes nothing. Legacy /api/classroom-media documents keep rendering. Closes#1522.
Detach and stop the active playback engine before asynchronous scene teardown so switching to a non-playable scene cannot leave narration, timers, or effects running.\n\nRecheck engine ownership after lecture-session creation in manual and auto-play paths, close stale sessions, and reject progress callbacks from detached engines. Add regression coverage for all three races.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Extracts commitToPool, the single client-side sequence for storing bytes in the asset pool, writing the allocated id back, and mirroring locally; routes the media pass, narration adoption and fresh TTS through it. A store-full refusal during TTS now retains the already-billed clip so the next load adopts it with zero provider calls. Closes#1467.
Bump the application version to 1.0.3 and record the changelog. This is a
security release closing three advisories — access-code token expiry and
verification throttling, a render-service network policy on untrusted HTML, and
audio provider redirect and DNS-rebinding validation — and upgrading Next.js to
patch a critical RCE, plus the fixes and features merged since 1.0.2.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Audio provider requests (TTS, ASR, voice registration and voice cloning)
validated a client-supplied base URL once and then issued a plain fetch with
default redirect-follow and no pinned dispatcher. A base URL that resolved to a
public address but answered with a redirect to an internal one was followed, and
a DNS answer that changed between the guard's lookup and the connect reached an
internal host — both readable in-band.
- Route every lib/audio provider request through a new
lib/server/audio-provider-fetch.ts that combines per-hop redirect
re-validation with a pinned undici dispatcher, so the socket can only reach an
address the guard validated, on every hop.
- Select the public-vs-local policy server-side from isServerConfiguredProvider;
a client-supplied base URL is always strict public and can never reach a
private, loopback or cloud-metadata address, even with ALLOW_LOCAL_NETWORKS
set.
- Pin the result-audio download hop as well, keeping its host allowlist and
redirect:'error'.
- Add a coverage-matrix test that fails if any lib/audio module regains a raw
provider fetch.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Headless Chromium renders caller-supplied HTML on two paths, and neither applied
the Content-Security-Policy the app packager injects for exports. Inline script
in a preview scene or an uploaded render project could reach loopback and
internal addresses, and on the render path the response is painted into the
returned MP4.
- Add a single untrusted-HTML CSP and an injector that places the policy as the
first node the parser processes (ASCII whitespace only; a leading BOM is
stripped; a byte-level injector preserves non-UTF-8 bodies).
- Inject the policy into the interactive preview srcDoc and add a Puppeteer
request guard that blocks non-data/blob/about requests from the untrusted
frame and non-about main-frame navigations.
- Harden every extracted project HTML, and sanitize framed same-origin .svg and
.xhtml documents (scripts, on* handlers, javascript: URLs, foreignObject,
nested frames removed via parse5), which cannot carry a meta CSP.
- Document the residual top-level-navigation risk on the render path, which the
egress lockdown must contain.
- Add real-Chromium boundary tests (zero listener hits incl. WebSocket) and a
packager-equivalence test for the policy.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
