* fix(quiz): resolve AI answer keys to option values before grading
AI-generated quiz answer keys are unreliable: the model may write the
correct answer as option CONTENT ("(6, 2)"), as a LETTER ("A"), or as a
formatting variant (full-width chars, extra spaces) of either. Grading
compares exact option values, so a learner picking the correct option
was still graded incorrect whenever the key was written as content or
in a different format.
- generation: normalizeQuizAnswer now resolves every answer variant to
the matching option value (NFKC + whitespace-stripped + letter-prefix
canonical matching against option values and labels); unknown answers
pass through untouched
- grading: resolveAnswerKeyToValue retro-fixes already-generated courses
whose stored keys hold content variants, mapping them to option values
when exactly one option matches canonically; ambiguous keys are left
alone instead of being silently re-pointed
- tests: letter/content/full-width/trailing-space/multi-answer/unknown
cases + end-to-end grading with a content-variant key
* chore(generation): bump version to 0.3.2 for answer-key fix
The repo requires every publishable @openmaic package change to ship with
a version bump (CI check-package-version-bumps).
* style(generation): prettier-format package.json after version bump
* fix(quiz): use resolved answer representation in review renderers; prettier
- QuestionCard + quiz-view review modes highlight the option canonically
matching the stored answer key (answerIncludesOption), so persisted
content-variant keys highlight correctly after the grading fix
- prettier formatting on touched files
* test(quiz): guard possibly-undefined answer in multi-choice test
* fix(quiz): narrow answer-key matching to exact unique alignment; project review UI from the same resolver
Per review: matching policy narrows to the DSL option-identity contract.
- Generation: exact value or unique exact label alignment; unknown and
ambiguous keys stay unresolved (fail closed). Prompt example unified to
the DSL object-options + value-answer shape.
- Grading: persisted keys resolve via the same exact alignment; the
resolver returns the option's actual value when exactly one option
matches canonically-or-exactly; ambiguous keys stay untouched.
- Review UI: QuestionCard and both quiz-view review modes highlight via
the same resolver projection (answerIncludesOption(q, optionValue)).
- Removed the broader equivalence rules (NFKC, whitespace stripping, case
folding, wrapped/prefixed-letter probe) — semantic synonyms such as
正交/垂直 remain uncovered, hence Refs #1326.
* fix(quiz): resolve answer keys for persisted keys only; align quiz template with the DSL
- grading: compatibility resolution (label -> value) now applies to the stored
answer key only. A learner submission is compared as submitted, so an option
label can no longer be accepted as a different option's value.
- quiz-content/user.md: replace the string-options/`correctAnswer` example with
the DSL shape the system prompt defines ({ label, value } options and
`answer` as an array of option VALUES), so generated keys are values.
- tests: regression that a label submission grades incorrect while the same
question's value submission grades correct.
* test(generation): update scene-prompt golden snapshot for the DSL-shaped quiz template
The golden test pins the rendered user prompt for every scene kind; the
quiz-content template now emits the DSL shape, so the snapshot follows.
* fix(quiz): read label-stored correct answers in the editor's option rows
`quiz-edit-ops.toRows` decided correctness with raw value equality, so an
existing label-keyed quiz showed A/B as correct (the review UI and the grader
both resolve labels) while every row read as incorrect internally — and the
next option edit or reorder rebuilt `answer` from those rows and silently
dropped the key.
- `toRows` now uses the same resolved projection (`answerIncludesOption`).
- `normalizeQuizAnswer`'s docstring now describes the exact-only behavior it
has had since the review: formatting variants are not normalized and
ambiguous entries pass through untouched.
- Regressions: an unrelated option-label edit and an option reorder both keep
a label-stored answer, and a multiple-choice toggle preserves it. All three
fail against the previous raw comparison.
---------
Co-authored-by: CrazyPandaP <CrazyPandaP@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Zhipu released GLM-5.3 (2026-08-19, text flagship) and GLM-5.3-Flash
(2026-08-26, native multimodal, 320B MoE / 18B active). Both serve a
1M-token context with 128K max output and always-on thinking controlled
by reasoning_effort low/high/max — the API rejects thinking
{"type":"disabled"} with code 1210 ("该模型始终思考，不支持关闭思考").
- catalog: glm-5.3 (text) and glm-5.3-flash (vision) entries, 1M/128K
- metadata: glm53Effort capability — effort-only, not toggleable,
default effort max (same forced-thinking pattern as Fable 5)
- adapter: for non-toggleable GLM models, degrade a "disabled" thinking
request to the lightest effort instead of sending the API-rejected
{type:'disabled'}, mirroring the guard in the Anthropic adapter;
this also un-breaks POST /api/verify-model, which hardcodes
thinking off
Co-authored-by: XingJia He <hexingjia@mobirit.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Two sharp versions resolve in the dependency tree (0.34.5 transitive ->
@img/sharp-libvips-* 1.2.4, 0.35.4 direct -> 1.3.3). sharp's native
libvips libraries are loaded via dlopen and are not statically analyzable,
so Next.js standalone output tracing omitted them; it traced the 1.2.4
libs while the runtime loaded sharp 0.35.4, which dlopens libvips 8.18.6.
The result was a boot failure on self-hosted Docker (Alpine/musl):
ERR_DLOPEN_FAILED: Error loading shared library libvips-cpp.so.8.18.6:
No such file or directory (needed by .../@img/sharp-linuxmusl-arm64/
lib/sharp-linuxmusl-arm64-0.35.4.node)
This surfaced as "Agent runtime startup failed" and broke PDF/image
material extraction. The bug is platform-independent: it stems from the
standalone tracing dropping the wrong libvips version, so it affects any
self-hosted Docker deployment (arm64 or x64), not a specific architecture.
Force-include every @img/sharp-libvips-* native lib directory in
outputFileTracingIncludes so the libvips matching the loaded sharp binary
is always present in the standalone output, across platforms and versions.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(ssrf): keep cloud metadata endpoints blocked under ALLOW_LOCAL_NETWORKS
ALLOW_LOCAL_NETWORKS=true exists so self-hosted deployments can point
providers at loopback, RFC1918 and split-horizon targets. It also
returned early from validateUrlForSSRF before any classification, so a
client-supplied base URL of 169.254.169.254 (or metadata.google.internal,
100.100.100.200, fd00:ec2::254) was accepted on such deployments.
Cloud instance-metadata endpoints are now rejected regardless of the
flag: literal hosts and IPv4-mapped forms are classified directly, and
non-IP hostnames are resolved so an answer that lands on a metadata
address is rejected too. DNS failure under the flag still fails open,
as before, because split-horizon DNS is an explicit use case of the flag.
Without the flag the DNS path now also rejects answers on metadata
addresses that are not RFC1918 (100.100.100.200).
The route tests that used the metadata address as their example of a
target the flag allows now use a private-network address instead.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ
* fix(ssrf): widen the metadata set and bound the flagged DNS lookup
Review follow-up. Add the AWS ECS task credential and EKS Pod Identity
endpoints (169.254.170.2, 169.254.170.23, fd00:ec2::23), the Azure
WireServer address (168.63.129.16) and the legacy OCI IMDS address
(192.0.0.192) to the blocked set; the last two are neither RFC1918 nor
link-local, so they were reachable even without the flag. Recognise the
metadata addresses when carried inside 6to4, Teredo, ISATAP and NAT64
literals. Bound the DNS lookup done under the flag to three seconds and
fail open on expiry, matching the existing fail-open on error. Say in
.env.example that the set is a fixed list and that DNS failure is
allowed through.
Use the same private-network fixture for the reject and allow cases of
the four route guard tests so a partial NODE_ENV re-gate of the guard
goes red again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ
* fix(ssrf): share the tunnel decoder between the private and metadata checks
Review follow-up. assertSafeIp now classifies metadata addresses through
isCloudMetadataAddress, so an ISATAP identifier under a globally
routable prefix that carries 168.63.129.16, 192.0.0.192 or
100.100.100.200 is rejected on the strict-fetch path too. isPrivateIP
uses the same tunnelEmbeddedIPv4 helper instead of its own copies of the
6to4, Teredo and ISATAP decoders, which also gives it NAT64. Tests cover
the globally-routable ISATAP forms, the NAT64 private case, and the
false-positive direction (tunnel literals carrying public or RFC1918
addresses stay allowed under the flag).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ
* fix(ssrf): reject metadata hostnames before the flag branch
Review follow-up. The metadata hostname and address check now runs
before the ALLOW_LOCAL_NETWORKS branch, so metadata.google.internal is
rejected by name in both flag states (previously only under the flag),
and metadata literals get the metadata message rather than the one that
suggests setting the flag. Pin the tunnel decoder boundaries in tests
(2001:db8 is not Teredo, 64:ff9b:1 is not NAT64, 2003 is not 6to4).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Add a triage section that says how severity is assigned and that
severity objections are answered in the advisory thread before
publication, and a disclosure note that maintainers request the CVE
through GitHub at publication time and keep the published advisory
consistent with the CVE record.
Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* feat(render-service): emit render and preview lifecycle events
The service is silent apart from one startup banner, which leaves a deployment
able to see *that* rendering happens — CPU, memory — but unable to answer how
many renders succeeded, how long they took, or how long they waited for a slot.
Three things compound to make the outcome of a render unobservable from outside
the process:
- Job state lives in `JobStore`, in memory by default, so every record is gone
on restart.
- A failed render is reported to the client as HTTP 200 whose *body* carries
`status: 'failed'`, so any proxy, gateway or dashboard counting status codes
sees a perfectly healthy service.
- Nothing between admission and completion writes a line anywhere.
This adds one JSON line per lifecycle transition on stdout, which every
container log pipeline already collects:
- `render_job_submitted` — with the queue and running depth at admission
- `render_job_started` — with `queueWaitMs`, the time the job spent queued
behind other renders. Nothing else exposes this today, and it is the signal
that distinguishes "renders are slow" from "renders are waiting".
- `render_job_finished` — with the outcome (succeeded / failed / cancelled),
the total duration, and, for a failure, its machine-readable code
- `render_admission_rejected` — with the reason a 429 was returned, on both
`/render` and `/preview`
- `preview_request` — status and duration for every response the synchronous
preview route produces, emitted from middleware so 200, 413, 429, 504 and 500
are all covered without touching the handler's return paths
Events carry only bounded, low-cardinality dimensions: an opaque job id, a
state, a duration, a reason code. No client identity, no file or directory
names, no scene content, and no raw error strings — a render works on
untrusted, model-authored input, and none of it belongs in an operational log.
A test asserts that a failure's free-text message, which can contain a scratch
path, never reaches the log while its fixed-vocabulary code does.
`RenderCoordinator` takes the sink through its existing options bag
(`onEvent`), defaulting to the stdout emitter, so tests assert on events
without going through the console.
* fix(render-service): close the lifecycle for a job cancelled while queued
Cancelling a job that has not started yet takes a different path from
cancelling a running one: it updates the job store directly rather than going
through `finishNonSuccess`. So it emitted no `render_job_finished` event — a
submitted job was simply never heard from again, and any success rate computed
from the event stream would have been quietly wrong. It also left the job's
submission timestamp in the map, which is the one piece of per-job state these
events keep.
Both are fixed by closing the lifecycle in that branch too. The added test
asserts a queued-then-cancelled job reports a terminal event, and fails without
the fix.
* fix(render-service): make the lifecycle stream summable and its fields honest
An independent review of this branch found the event stream could not actually
be summed into the rates it exists to produce, plus two tests that could not
detect the regressions they were named for. All of these were demonstrated.
- A job could emit `render_job_finished` twice. The success event was emitted
before the terminal `jobs.update`, so a rejected write landed in the catch and
closed the same job again as `failed` — one render counted as both a success
and a failure. `JobStore` is a seam a Redis-backed store can fill, so a
rejected terminal write is a real case, not a hypothetical. The success event
now follows the last fallible await, and `finishEvent` is idempotent: it keys
off the timing entry it consumes, so any second close is a no-op.
- `durationMs` on a finished job was submission-to-finish while its doc claimed
"time spent in the state the event closes", so p95 "render duration" tracked
queue depth. It keeps that meaning, now documented, and the event also carries
`queueWaitMs` and `renderMs` so a consumer can tell a slow render from a long
queue without joining back to the started event.
- `running` counts jobs from the moment they leave the queue, but the execution
permit is shared with `/preview`, so a counted job may still be blocked. The
field is documented as "admitted to run" rather than claiming to be a count of
renders in flight.
- `render_job_submitted` counted the submitting job in `queued`, so an idle
service reported `queued: 1` and the obvious "alert when queued > 0" rule
could never be written. It now excludes self, matching `render_job_started`.
- `render_admission_rejected` and `preview_request` called the module-level
emitter directly, so an embedder injecting a sink silently received three of
five event types. `AppDeps` now takes the same `onEvent` seam.
- The serializer walked `Object.entries(event)`. Excess-property checking made
that safe for object literals but not for spreads, which are exempt — so the
header's promise that no identity, path or content is ever logged rested on
every future caller avoiding one. It now serializes an explicit field list.
Tests: the route tests passed an `executionGate` that `AppDeps` does not have,
silently discarded, and unnoticed because `tsconfig` includes only `src` so
typecheck never sees `test/`. Removed, and they now assert through the injected
sink instead of spying on the console. The leak test asserted only on event
shape and still passed with the cleanup deleted; it now observes the tracked
state through a `trackedJobs` accessor. Added regressions for the double close,
the timing split and the queue depth — each verified to fail without its fix.
* harden input validation and outbound URL handling
- Validate the classroom id on the write path with the same allowlist the
read path already applies, and assert in the storage layer that a resolved
classroom file stays inside CLASSROOMS_DIR.
- Apply validateUrlForSSRF at every call site regardless of NODE_ENV; the
guard already owns the ALLOW_LOCAL_NETWORKS escape hatch, so the extra
environment condition only disabled the check outside production builds.
Add a repository-scanning test so a gated call site cannot reappear.
- Re-validate every redirect hop of an outbound provider request through a
shared transport, mirroring the per-hop loop proxy-media already uses.
- Restrict stored slide HTML to the formatting vocabulary the renderer
produces, sanitizing at the classroom persistence boundary on both write
and read, with a separate policy for KaTeX snapshots.
- Refuse the development persistence authenticator under NODE_ENV=production
unless an explicit opt-in is set, and document it in .env.example.
* drop credential headers on a cross-origin redirect hop
The manual redirect loop reused the caller's init on every hop, so provider
credentials were re-sent to the redirect target even on a different origin.
The platform fetch drops Authorization itself when it follows redirects, so
mirror that: strip authorization, api-key, x-api-key and x-goog-api-key
before a cross-origin hop, matching case-insensitively and preserving the
shape of the caller's headers. Same-origin hops are untouched.
A streaming request body cannot be replayed onto the next hop, so fail with
a clear message rather than sending an empty body.
---------
Co-authored-by: wyuc <dhq1024@proton.me>
* feat(deepseek): register deepseek-v4-flash-vision-exp with vision capability
DeepSeek's docs list deepseek-v4-flash-vision-exp as an official model
on the same OpenAI-compatible base URL (https://api.deepseek.com):
1M context, 384K max output, JSON output / tool calls / Responses API /
Anthropic API all supported, priced same as deepseek-v4-flash.
Register it in the deepseek catalog with vision: true so scene
generation can pass document images as vision content parts instead of
silently dropping them for this provider.
* fix(deepseek): register vision model in thinking metadata and update pinned lists
- add deepseek-v4-flash-vision-exp to THINKING_CAPABILITIES (deepseekEffort,
same as v4-flash per official docs) so the model-metadata drift test passes
- update the pinned deepseekModels list in thinking-config.test.ts to include
the new model
* docs(env): include deepseek-v4-flash-vision-exp in DEEPSEEK_MODELS example
* Revert "docs(env): include deepseek-v4-flash-vision-exp in DEEPSEEK_MODELS example"
This reverts commit 8040bac441.
* style(deepseek): apply prettier formatting
* test(deepseek): pin vision model capabilities; document vision-exp in env example
Review asks from #1329:
- assert deepseek-v4-flash-vision-exp capabilities (vision: true,
streaming, tools) so a regression to vision: false fails the suite
- add the model to the DEEPSEEK_MODELS example in .env.example
---------
Co-authored-by: CrazyPandaP <CrazyPandaP@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
parseUserSkillZip guarded the SKILL.md size against the zip central
directory's self-declared uncompressedSize and then handed the entry to
JSZip's async('string'), which inflates the real deflate stream no
matter what the header claims. A malicious zip declaring a tiny size
slipped past the guard, and with the upload route's 1 MiB archive cap
and deflate's ~1032:1 ratio, one request could expand ~1 GiB in the
server process — repeatable per request for memory-exhaustion DoS.
Inflate the entry's compressed bytes with node:zlib
inflateRawSync({ maxOutputLength: 70_000 }) so zlib aborts the moment
output crosses the cap, whatever the headers say. The declared-size
check stays as a fast path that refuses honest oversized zips without
inflating; STORE entries are byte-length checked directly since their
"compressed" content is the content. Over-limit output raises the same
UserSkillUploadError('The archive SKILL.md is too large.'), so callers,
route responses, and status codes are unchanged.
Tests: a forged-header bomb (a real zip whose central directory is
patched to declare a small uncompressedSize, with the forgery asserted
to have taken on reload) is refused; an honest oversized zip is refused
by the fast path; a forged STORE entry is refused by the byte-length
check; a STORE zip round-trips; a corrupt deflate stream still maps to
invalid-zip rather than a size refusal.
Co-authored-by: Claude Code <noreply@anthropic.com>
* feat(render-service): add PreviewGate admission control for previews
Independent admission gate (global in-flight cap + per-identity cap) so
synchronous previews never compete with the MP4 export queue. Release is
idempotent so every route exit path can call it safely.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* feat(render-service): add single-page preview renderer
Renders a scene to a PNG via SlideCanvas + static markup using the existing
Chromium executable configuration, egress policy, and semaphore
infrastructure, with a deadline/AbortSignal.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* feat(render-service): add preview config and resource pixel limits
Adds previewDeadlineMs / previewMaxInFlight / previewMaxPerUser config and
resource-profile viewport pixel limits for previews; declares the renderer
runtime dependencies installed with the render-service container.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* feat(render-service): add POST /preview endpoint
Wires the preview endpoint: declared-size rejection, gate admission before
buffering, byte-capped payload parsing, scene/stage/viewport validation,
deadline/cancellation, PNG response, and 400/413/429/504/500 mapping. Fixes
the render_scene_preview tool's broken link (previously 404).
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix preview rendering and deadlines
* fix(render-service): enforce preview deadline, memory bound, and shared Chromium limit
- Stalled preview uploads now observe the deadline signal, error the consumer,
and cancel the reader; deadline aborts map to 504 and release the permit.
- Previews retain the extraction permit through rendering so parsed payloads
cannot accumulate beyond the memory admission bound.
- RenderCoordinator owns a shared execution semaphore so previews and MP4
renders respect the same global Chromium concurrency limit.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(render-service): harden preview admission and rendering
* docs(render-service): define the preview deployment contract
* fix(render-service): enforce self-contained previews
* fix(render-service): allow fragment CSS URLs in previews
* fix(render-service): allow background-only previews
* fix(render-service): control malformed preview rejections
* style(render-service): apply prettier formatting after rebase
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
* feat(canvas): add double-click text insertion
* fix(ci): format canvas double-click text insertion
* fix(canvas): prevent text insertion on double-click of locked elements
- Add guard to check if double-click event target is inside an editable element
- Prevents text box insertion when double-clicking locked elements (backgrounds/watermarks)
- Fixes event-propagation bug where locked element selection handler returns before stopPropagation()
- Add regression test suite for canvas double-click handler covering locked elements and blank canvas cases
Addresses review feedback from CxuPercy on PR #1310
* fix: remove unused import in canvas-double-click-handler test
* fix: improve test implementation to avoid DOM dependency
- Rewrite tests to use mock objects instead of actual DOM elements
- Properly type MockTarget interface to satisfy TypeScript checks
- All tests now pass locally without DOM environment requirement
* test(e2e): add end-to-end tests for canvas double-click text insertion
- Add test for double-clicking blank canvas area to create new text element
- Add test to verify double-clicking existing elements does NOT insert new text
- Tests actual editor interactions, not just mock targets
- Addresses CxuPercy's review feedback for proper regression test coverage
* chore: format code with prettier
* test(e2e): simplify double-click text insertion test
- Remove flaky second test that checks element visibility state
- Focus on core functionality: double-click on blank canvas creates new element
- Use more lenient assertion (toBeGreaterThan) to account for dynamic elements
- Increase wait time for slide editor to fully render (1000ms)
* fix(storage): increase test timeout for replace/release operations
The 'replace retires rather than revokes an issued snapshot; release revokes both'
test performs multiple async operations (put, resolve, replace, release) that can
exceed the default timeout. Increased timeout to 10 seconds to accommodate I/O operations.
* feat(render-service): expose machine-readable admission state
429 rejections carried only prose, so clients couldn't distinguish
deployment-wide backpressure from a personal limit, and /health gave
load balancers no way to shed traffic before probing with a render.
- RenderRejectedError now carries a reason code: queue_full (global
RENDER_MAX_QUEUE cap) or per_identity_limit (RENDER_MAX_JOBS_PER_USER
guard). The internal reservation invariant stays reason-less.
- POST /render serializes the reason on 429 bodies via spread-omission,
so reason-less rejections don't emit a reason field at all.
- /health gains `accepting: boolean` (queue below the global cap).
Deliberately aggregate-only: no occupancy counts and never
per-identity data — identity keys are client IPs when
TRUST_PROXY_HEADERS=true, so the map must not leave the process.
- README documents the 429 reason codes, the accepting flag and why it
is boolean-only, and the shared-`direct`-identity model that makes the
default Compose disable the per-user guard.
Refs #1348
Co-Authored-By: Claude Code <noreply@anthropic.com>
* refactor(render-service): extract 429 body serialization and shared test helpers
Review findings on the admission-observability change: the 429 body was
built identically in both catch sites, and the parked-executor /
bounded-poll-loop shapes were duplicated across the route and
coordinator admission tests. The spec review also flagged that
/health's negative space was only verified by inspection.
- rejectionBody(error) serializes an admission rejection once; both
RenderRejectedError catch sites collapse back to one-line mappings.
- The /health inline comment now cites RenderCoordinator#accepting
instead of restating its aggregate-only rationale.
- parkingExecutor() moves to test/support/fakes.ts beside
succeedingExecutor; waitUntil() lands in test/support/async.ts and
backs both waitForPoll (route) and the coordinator settle check, with
descriptive timeout messages.
- A new route test pins /health to its exact key set (accepting, ok,
resourceProfile, versions) and a boolean accepting, so any new field
must be a deliberate, reviewed change.
Refs #1348
Co-Authored-By: Claude Code <noreply@anthropic.com>
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* docs(readme): sync with current code, multi-workbench skill support, v1.0 videos
- Add missing PostgreSQL persistence and MP4 video export sections to README-zh (EN already had them)
- Update project structure: @openmaic/* SDK packages (dsl/generation/renderer/importer/editor/storage), render-service, 26 API endpoint groups
- Add storage layer to key architecture; fix outdated endpoint count
- Document that the OpenMAIC Skill works beyond OpenClaw: Codex, DeepSeek, WorkBuddy and other agent workbenches (rename section + anchors)
- Replace intro videos with v1.0 versions (EN + ZH)
* docs(readme): replace unverified cp command with import description for other agent workbenches
---------
Co-authored-by: Percy <percy@PercydeMacBook-Pro.local>
* fix(skills): normalize paths to POSIX before passing to pi-agent-core loader
On Windows, NodeExecutionEnv returns backslash-separated paths from
fileInfo() and listDir(). The pi-agent-core skill loader passes these to
the `ignore` package, which requires POSIX-style relative paths and
rejects absolute or backslash paths with:
RangeError: path should be a `path.relative()`d string
Wrap NodeExecutionEnv with PosixNormalizingEnv that converts all
FileInfo.path values to forward slashes, and normalize skillsDir itself
before it reaches the loader.
Fixes#1295
* fix(skills): normalize FileInfo.name alongside path for Windows skill discovery
Address CR from HaningZS: on Windows, pi-agent-core's fileInfoFromStats()
returns the full backslash path as name, so entry.name === 'SKILL.md' never
matches. normalizeSkillFileInfo now derives name from the normalized path.
Add deterministic tests for normalizeSkillFileInfo and a production-boundary
test asserting listSkills() returns built-in skills.
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Why:
- The root Node 20.9 contract predates direct runtime dependencies whose declared minimums now reach Node 22.19.
- README, contributor, localized docs, and the OpenMAIC extension skill repeated the stale supported-version claim.
What:
- Raise the root Node minimum to 22.19 and align every operator, contributor, locale, and skill prerequisite.
- Add a check that compares the root minimum with installed direct production dependency engine minimums.
- Run the new contract check in CI after the frozen dependency install.
Risk:
- This changes the declared minimum only; no upper bound is added and Node 24 compatibility remains a separate concern.
- The localized docs build was validated with the independent #1306 boundary fix from PR #1307, which is not included here.
Tests:
- RED on the base: engine check reported pi-agent-core, pi-ai, svg-pathdata, and undici floors
- GREEN: root minimum 22.19 satisfies 35 engine-constrained direct dependencies
- Node 20 lockfile-only install reports the root unsupported-engine warning
- Node 22 frozen install and postinstall
- Docs build with PR #1307 boundary: 34 pages and all locale postexport checks; docs types:check
- Root Prettier, ESLint, TypeScript, i18n, package-version, and internal-dependency gates
- Root pnpm test: 7137 passed, 81 skipped
Live Docs:
- GitHub issue #1304 tracks the Node contract; #1306 / PR #1307 tracks the separate docs-build prerequisite.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(generation): keep diagram node position off the hover transform
A CSS `transform` overrides the SVG `transform` presentation attribute, so a
generated `.node:hover { transform: ... }` rule wipes out the `translate()`
that positions the node and snaps it toward the origin while hovered. The
diagram prompt only asked to "avoid hover transform conflicts", which was too
vague to prevent this.
Spell out the contract instead: the outer `<g>` carries positioning only, and
hover/active animations apply to an inner group.
Refs #1300
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(generation): bump package version to 0.3.4
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix: keep videos playing inline on mobile
Add playsInline to every playback video path, including the legacy app renderer, renderer-backed playback, and the shared renderer package. This keeps mobile browsers such as WeChat on Android from interrupting inline playback and restarting videos after an interruption.
Add regression coverage for both app renderer modes and the package renderer.
* chore(renderer): bump package version
* chore(renderer): bump package version to 0.1.6
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(workbench): render LaTeX in assistant messages
* fix(workbench): harden streamed math rendering
* fix(workbench): preserve streamed math boundaries
* refactor(workbench): use standard math syntax
* fix(workbench): preserve math in course link labels
* fix(workbench): disambiguate single-dollar math
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(settings): turn TTS and media generation on from the Settings page
The global enable flags only auto-set on first provider sync, and Settings
had no switch for them, so a later API key never started narration or
image/video generation. Add the Settings toggles and turn each flag on
when that provider first becomes usable.
* fix(settings): route media enablement through isUsableMediaProvider
A force-disabled image or video provider could still flip the global flag
from a key paste, and keyless providers never did. Use the same usable
transition as TTS, including a baseUrl for lemonade and comfyui.
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(asr): bypass AI SDK for custom providers to handle extra response fields
Custom ASR providers (e.g. SiliconFlow) return OpenAI-compatible JSON but
include extra fields like segments, duration, and usage that the
Vercel AI SDK's strict Zod response schema does not expect, causing it to
throw an 'Invalid JSON response' error.
Previously, custom-asr-* providers were routed through
transcribeOpenAIWhisper() which delegates to the AI SDK's
experimental_transcribe. The SDK validates the raw HTTP response against a
strict schema, so any provider-specific extra field fails parsing even when
the ext value itself is perfectly valid.
Fix: introduce transcribeCustomOpenAICompatibleASR() which calls
/audio/transcriptions directly via raw fetch (the same approach already
used by funasr-asr and lemonade-asr). It extracts only data.text from
the response, making it tolerant of any extra fields a provider may return.
The official openai-whisper case is unchanged and still uses the SDK.
Fixes#1277
* style: fix prettier formatting in asr-providers.ts
* fix(asr): require model ID for custom providers, add regression test
Address review feedback from CXuPercy (#1283):
- The previous implementation silently omitted the model FormData field
when config.modelId was falsy. Most OpenAI-compatible transcription
endpoints treat model as required, so a custom provider configured
without a model ID would go from working (via transcribeOpenAIWhisper's
fallback) to silently sending a modelless request and getting a 400.
- Fix (Option A — fail fast): validate config.modelId before building
the request and throw a descriptive error if it is missing, so the user
gets a clear message rather than an opaque 400 from the upstream API.
- UI contract: the settings panel already shows an amber warning
'Please add at least one model before using this provider' and the
Add button is disabled when the model ID field is blank, so the
UI is already aligned with the fail-fast behaviour.
- Update all 12 test cases to supply a modelId, and add a dedicated
regression test 'throws when no model ID is configured' to prevent
this from regressing silently again.
* fix: replace rm -rf with cross-platform fs.rmSync in package scripts
The build/clean scripts used the Unix \
m -rf dist\ command, which fails on Windows where cmd.exe has no rm. Replace with Node's built-in fs.rmSync, available on all platforms (Node >=20.9).
* chore: bump @openmaic package versions for cross-platform script fix
Generated Pyodide widgets can import micropip before it is loaded, aborting initialization. Add ordered prompt guidance, cover the packaged asset, and bump the generation package patch version.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(providers): apply server-pinned models for image and video providers
The server-providers API already exposes image models, and the server
resolves them correctly, but getServerVideoProviders() omitted the
models field entirely, and fetchServerProviders() on the client side
discarded the models payload for both image and video — it only set
isServerConfigured and serverDisabled.
This caused the client to fall back to built-in model IDs, which fail
on endpoints that require different IDs (e.g. Volcengine Ark Agent Plan
uses dotted aliases like doubao-seedream-5.0-lite).
Server: getServerVideoProviders() now exposes models, mirroring the
image listing. Client: fetchServerProviders() stores the server model
list as customModels with replaceBuiltInModels: true for both image and
video, matching the existing LLM provider model-filtering pattern.
Fixes#1251
* style(tests): wrap long assertion to satisfy prettier printWidth
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* fix(docs): swap English/Chinese guide links in README badges
* fix(docs): swap English/Chinese guide links in README-zh badges
* fix(docs): swap English/Chinese user guide links only
Revert unrelated changes (apiKey, database URL) and keep only
the four guide-link swaps in README.md and README-zh.md.
Addresses review feedback from YizukiAme.
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Why:
- The docs Turbopack project walked up to the repository root and compiled the product middleware.
- The v1.0 middleware's product-only feature-flag alias made every current docs build fail before static export.
What:
- Resolve the absolute packages/docs directory from the Next config URL.
- Set that directory as the standalone docs app's Turbopack root.
Risk:
- The setting affects only the docs Turbopack project; root product middleware and aliases are unchanged.
- Webpack-specific behavior is unchanged because the docs workflow builds with Turbopack.
Tests:
- packages/docs pnpm build: 34 static pages and all locale postexport checks passed
- packages/docs pnpm types:check
- Root Prettier, ESLint, TypeScript, and i18n gates
- Root pnpm test: 7137 passed, 81 skipped
Live Docs:
- GitHub issue #1306 tracks the regression and build evidence.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Why:
- A shell-exported OPENAI_API_KEY enables the generic image fallback inside a test that expects every configured media provider to be disabled.
- The resulting fetch-mock failure includes the Authorization header in local test output.
What:
- Mask the generic OpenAI key only for the force-disabled media test before provider configuration is imported.
- Reuse the existing Vitest environment cleanup to restore the caller's shell environment after the test.
Risk:
- Test-only change; production provider fallback behavior and the global shell-environment policy remain unchanged.
- Dedicated provider-config coverage continues to exercise the generic OpenAI image fallback.
Tests:
- Focused force-disabled test with OPENAI_API_KEY=openmaic-test-sentinel
- Full classroom-media-generation test with and without the sentinel: 8 passed
- tests/server/provider-config.test.ts: 107 passed
- Full root suite with the sentinel: 7137 passed, 81 skipped
- Prettier, ESLint, TypeScript, and i18n quality gates
Live Docs:
- GitHub issue #1302; test-environment policy remains documented by PR #1162.
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
* feat(agent): make generate_video asynchronous with a placeholder ref
The generate_video tool awaited the whole provider submit/poll/download/
persist cycle inside the tool call, blocking the agent turn for minutes
(and effectively capping it at the 10-minute tool budget, below its own
15-minute internal budget).
The tool now validates synchronously, mints a gen_vid_<id> placeholder
(the scheme the outline flow already uses), and returns immediately so
the agent can patch_stage the ref onto a video element and keep
working; the element renders the existing skeleton while pending. A
detached background job (own 15-minute timeout, deliberately decoupled
from the caller's abort so a cancelled chat cannot silently orphan a
billable provider job) runs the provider cycle, persists the bytes, and
then:
- patches the persisted document: every slide video element still
referencing the placeholder gets the concrete server-hosted src
(same runStageMutation discipline as the generation tools; skipped
silently when the element was changed meanwhile), and
- appends a media_ready lifecycle event to the session's durable log
via the session-level control channel (valid post-run, unlike the
runner's lease-guarded emit); the workbench folds it into the media
generation store so the skeleton resolves instantly, on live stream
and on replay.
Pending tasks live in a process-local registry; a server restart
orphans in-flight jobs (the placeholder keeps its skeleton), matching
the classic flow's client-local durability caveats. Tracked as an
accepted v1 limitation.
Closes#1266
* fix(agent): unfence the background video patch from the run lease
Review findings on the async generate_video change:
- P1: the completion patch wrote through the run's owner-bound store,
whose mutation fence asserts the run lease on every write. A video
job settles minutes after its run ended and the lease is released,
so every post-run patch threw AgentSessionLeaseLostError and the job
wrongly settled failed. The runner now builds a dedicated owner-bound
store for media jobs fenced only by the stage-mutation discipline,
and the tool takes it as a separate backgroundStore dep.
- P2: patchStageVideoPlaceholder rewrote whole scenes from one
minutes-old loadDocument snapshot. It now re-reads each candidate
scene immediately before its write and applies the placeholder swap
to the freshest state, so concurrent user/agent edits survive; the
residual read-write window matches the stage edit API's own
read-modify-write discipline.
- P3: drop the dead 'emit' progress marker (setPendingMediaStage is a
no-op after settle), emit a media_ready failed frame from the
last-resort crash guard so a bug path cannot leave the client on a
permanent skeleton, and log when appendControlEvent silently drops a
frame for a deleted session.
* test(agent): cover the concurrent placeholder-element removal case
Round-2 review leftovers: pin that patchStageVideoPlaceholder skips
rather than resurrects a placeholder element deleted between the
candidate scan and the write, and keep the detached job's crash guard
synchronous (never-rejecting helpers only) so a future throw inside it
cannot become the unhandled rejection the guard exists to contain.
* fix(agent): isolate the completion patch from the provider budget
Round-3 review leftovers:
- The patch shared the job's 15-minute signal, so a provider cycle that
nearly exhausted the budget could fail mid-patch and rebrand a
persisted, downloadable asset as failed. The patch now runs on its
own 60-second budget and a patch failure is logged while the job
still settles and emits done (the done frame's src renders fine).
- A user edit that replaced the placeholder with a concrete src while
the job ran no longer gets clobbered by the stale mediaRef: the swap
only writes while src is absent or still the placeholder.
Accepted, documented: the sub-second read-write window between two
concurrent jobs on the same scene (the client-side media fold renders
either way), and the failed-state fold requiring an attached chat
stream in v1.
* fix(agent): close the regeneration gap in the placeholder src guard
Delta review found the new patch-failure test never attempted the write
(no element carried the runtime ref, so putScene was never called), and
that the src guard also skipped legitimate patches: an element
re-pointed at a new job via mediaRef while still carrying the previous
generated src (which keeps rendering the old video), and an empty src.
The guard now also writes when src is empty or a previously generated
/api/classroom-media/ URL; the test seeds the ref so the failing write
is really attempted and asserts the logged patch failure.
* fix(agent): harden the placeholder src guard
- Total predicate: a malformed non-string src no longer throws inside
the element map and aborts the whole stage patch.
- Recognize the absolute-form generated src the classic pipeline
persists, and scope the generated-src arm to the stage's own media
root so a user's pick copied from another stage is preserved.
* feat(agent): let generate_scene pass widgetType and widgetOutline for interactive pages
The generate_scene tool schema had no widget fields, so every interactive
page it generated fell back to a simulation widget in the generation
pipeline. Add optional widgetType (simulation/diagram/code/game/
visualization3d; procedural-skill stays gated behind task-engine mode)
and widgetOutline parameters, reject them for non-interactive page
types, and carry them into the constructed SceneOutline. When only one
of the two is provided, mirror the generator's own defaults
(widgetType: 'simulation', widgetOutline: { concept: title }) so
half-specified calls still generate.
Closes#1258
* fix(agent): harden generate_scene widget params from review findings
- Accept procedural-skill in the widgetType union; the handler already
enables it via allowProceduralSkill and skill constraints can require
it, so excluding it only produced an opaque schema error.
- Validate widgetOutline is a plain object and reject malformed values
(null, strings, arrays) with an invalid-widget-outline error instead
of silently degrading the prompt or surfacing a misleading generic
generation failure.
- Reject widget fields on non-interactive pages by definedness rather
than truthiness, matching the tool description's contract.
- Pin the widgetOutline passthrough test to a non-default diagramType
(mindmap), cover the handler's single-field defaults, cover
procedural-skill passthrough, and exercise the tool schema through
TypeBox Value.Check.
* fix(agent): keep procedural-skill gated out of generate_scene
Review feedback: main gates procedural-skill behind taskEngineMode and
OPENMAIC_ENABLE_VOCATIONAL in every generation path, and no vocational
signal reaches the agent runtime, so accepting the literal here would
ungate the feature in every deployment. Drop it from the widgetType
union (back to the first commit's semantics), say so in the schema
description, and pin the gate in the schema validation test. Ungating
for vocational agent sessions can come back as its own change once a
task-engine signal is plumbed into the tool layer.
* perf(classroom): defer non-priority media blob hydration off the load path
Entering a classroom awaited the full mediaFiles restore — one object URL
per restored image/video blob — before the loading gate opened. For
video-heavy courses that is hundreds of MB of IndexedDB materialization
blocking first paint.
Split the restore into two phases:
- The awaited phase now builds metadata-complete task entries but only
creates object URLs for failed rows (none needed) and for media
referenced by the scene the classroom opens on (persisted cursor, else
the first scene). buildRestoredMediaTasks gains an optional
shouldHydrateBlob predicate, defaulting to eager hydration so existing
callers are unchanged.
- Remaining blob-backed records hydrate in the background, chunked over
requestIdleCallback (setTimeout fallback). Each deferred task stays
'done' — so generation resume never re-runs it — but carries no
objectUrl yet, which the media resolution state machine already renders
as a pending skeleton until the URL lands.
Background hydration is guarded per record: a task that was replaced
(classroom switch, regeneration, retry) or that belongs to another stage
is skipped and its freshly minted URLs are revoked immediately.
* perf(classroom): lazy-render slide thumbnails in the playback sidebar
The playback scene sidebar mounted a full SlideCanvas for every slide
scene the moment the classroom opened (and every off-screen video element
opened a preload="metadata" fetch), because SlideThumbnail's existing
visible prop was never passed.
Extract the editor nav rail's near-viewport IntersectionObserver hook to
lib/hooks/use-near-viewport.ts and gate the sidebar's slide thumbnails
through it: only scenes within 200px of the viewport render the live
canvas; the rest show SlideThumbnail's existing placeholder until scrolled
near. The placeholder keeps the same box size, so gating never shifts
layout.
The shared hook now starts hidden instead of eager: with the previous
eager-initial state, opening a long deck still mounted every canvas for a
frame before the observer could flip off-screen items off. The observer's
guaranteed initial delivery flips near-viewport items within a frame; the
no-IntersectionObserver fallback (e.g. jsdom) defers to a microtask so
the effect never synchronously re-renders.
* perf(media): lazy image decoding and HTTP Range support for classroom media
Renderer: BaseImageElement's <img> now carries loading="lazy" and
decoding="async", so thumbnail-heavy surfaces (playback sidebar, editor
nav rail, course cards) no longer fetch and decode every slide image up
front. In-viewport images are unaffected — the browser fetches them
immediately. slideToPng forces eager loading inside its permanently
off-screen snapshot tree, where lazy images would otherwise never fetch
and exports would capture blank slides.
Server: GET /api/classroom-media/[classroomId]/[...path] now answers
single byte-range requests with 206 Partial Content (Content-Range,
Accept-Ranges, correct Content-Length), enabling progressive playback and
seeking for hosted video/audio instead of downloading whole files. Suffix
ranges are supported; unsatisfiable ranges get 416 with the full size;
unsupported units or multi-range sets fall back to the plain 200 full-body
response, which is always a legal answer. Existing caching headers are
kept on every response shape.
* chore(renderer): bump version to 0.1.4 for image lazy-loading change
* fix(classroom): guard deferred hydration against restarted tasks and bound idle waits
A deferred record is only ever 'done' without an objectUrl; a task that
regeneration or retry restarted passes through pending/generating, so skip
those instead of attaching stale persisted bytes. Also give the idle
scheduling a timeout so a busy main thread cannot starve hydration.
* fix(classroom): stop superseded deferred hydration before minting URLs
A restore epoch captured at apply time now gates each idle chunk: loading
another classroom or reloading the same one invalidates any older hydration
loop, so it neither keeps scheduling work for an abandoned classroom nor
attaches bytes read by an older load to a newer load's tasks.
* fix(classroom): keep 416s uncached and classify element-keyed media as priority
A 416 with public immutable caching can poison the media URL for later
valid requests, so range errors now send Cache-Control: no-store. Priority
classification also collects the opening scene's media element ids, since
task lookup binds records keyed stage:<elementId> even when the slide slot
carries a different opaque ref.
* fix(classroom): tie deferred hydration liveness to the classroom load token
The restore epoch only advanced when the next load reached apply, so an
abandoned classroom kept hydrating during the next load's storage/network
phase. Compose the epoch with the load's isCurrent (load token plus effect
cleanup) so navigation stops the loop at the next idle boundary.
* fix(classroom): classify legacy-recovered media before deferring it
Legacy singleton video recovery assigns a placeholderRef only at the end of
the task build, so a record keyed by an allocated id without placeholderRef
was deferred even when it backs the opening scene's gen_vid_* element. Run a
metadata-only build first to learn each record's effective ref and classify
against it, keeping the first visible page's legacy video eager.
* fix(action): wait for deferred video bytes before starting play_video
executePlayVideo treated status done as immediately playable, but a deferred
restore is done without an objectUrl: the renderer shows a skeleton, no
<video> exists, and the later hydration never retriggers play, leaving the
action stuck until the safety timeout. Readiness now follows the renderer's
contract (only done-with-bytes is playable) at the initial check, the
subscription exit, and the post-subscription recheck; the failed skip applies
whether or not a wait happened.
* fix(classroom): include the stage whiteboard in priority media refs
The stage-level whiteboard stays open across standalone classroom switches,
so its media can be visible before any scene is. Classifying it as deferred
left visible whiteboard media pending behind idle hydration chunks.
Remember explicit workspace home visits and validate remembered session entries against the existing workspace session list before resuming. Stale entries now fall back to the workspace home and are forgotten.
Closes#1270
* feat(chat): add PPT element references to Pi
* fix(chat): include chart values in reference routing
* fix(chat): harden slide element reference evidence
* fix(playback): clear element reference on scene navigation
* fix(pi): preserve element grounding across child retries
---------
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
