mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-16 05:19:50 +08:00
@openmaic/dsl@0.11.2
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 wyucandClaude Opus 4.8
|
7420d26ebb |
fix(render-service): contain untrusted HTML with a network policy on preview and render (#1512)
Headless Chromium renders caller-supplied HTML on two paths, and neither applied the Content-Security-Policy the app packager injects for exports. Inline script in a preview scene or an uploaded render project could reach loopback and internal addresses, and on the render path the response is painted into the returned MP4. - Add a single untrusted-HTML CSP and an injector that places the policy as the first node the parser processes (ASCII whitespace only; a leading BOM is stripped; a byte-level injector preserves non-UTF-8 bodies). - Inject the policy into the interactive preview srcDoc and add a Puppeteer request guard that blocks non-data/blob/about requests from the untrusted frame and non-about main-frame navigations. - Harden every extracted project HTML, and sanitize framed same-origin .svg and .xhtml documents (scripts, on* handlers, javascript: URLs, foreignObject, nested frames removed via parse5), which cannot carry a meta CSP. - Document the residual top-level-navigation risk on the render path, which the egress lockdown must contain. - Add real-Chromium boundary tests (zero listener hits incl. WebSocket) and a packager-equivalence test for the policy. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Mohammedthowfiqandwyuc
|
98765db8b3 |
fix(render): preserve plan audio during chunk assembly (#1358)
* fix(render): preserve plan audio during chunk assembly Signed-off-by: Mohammedthowfiq <49336054+MohammedThowfiq@users.noreply.github.com> * fix(render): trust plan metadata for audio assembly Signed-off-by: Mohammedthowfiq <49336054+MohammedThowfiq@users.noreply.github.com> --------- Signed-off-by: Mohammedthowfiq <49336054+MohammedThowfiq@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
5873a1f657 |
fix(render): keep preview browser evaluation self-contained (#1421)
Co-authored-by: Bryan Nathan <bryan@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
98e75479ba |
fix(render): enforce chunk worker cap (#1359)
Co-authored-by: Bryan Nathan <bryan@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyuc
|
64621adfa0 |
feat(render-service): emit render and preview lifecycle events (#1397)
* feat(render-service): emit render and preview lifecycle events The service is silent apart from one startup banner, which leaves a deployment able to see *that* rendering happens — CPU, memory — but unable to answer how many renders succeeded, how long they took, or how long they waited for a slot. Three things compound to make the outcome of a render unobservable from outside the process: - Job state lives in `JobStore`, in memory by default, so every record is gone on restart. - A failed render is reported to the client as HTTP 200 whose *body* carries `status: 'failed'`, so any proxy, gateway or dashboard counting status codes sees a perfectly healthy service. - Nothing between admission and completion writes a line anywhere. This adds one JSON line per lifecycle transition on stdout, which every container log pipeline already collects: - `render_job_submitted` — with the queue and running depth at admission - `render_job_started` — with `queueWaitMs`, the time the job spent queued behind other renders. Nothing else exposes this today, and it is the signal that distinguishes "renders are slow" from "renders are waiting". - `render_job_finished` — with the outcome (succeeded / failed / cancelled), the total duration, and, for a failure, its machine-readable code - `render_admission_rejected` — with the reason a 429 was returned, on both `/render` and `/preview` - `preview_request` — status and duration for every response the synchronous preview route produces, emitted from middleware so 200, 413, 429, 504 and 500 are all covered without touching the handler's return paths Events carry only bounded, low-cardinality dimensions: an opaque job id, a state, a duration, a reason code. No client identity, no file or directory names, no scene content, and no raw error strings — a render works on untrusted, model-authored input, and none of it belongs in an operational log. A test asserts that a failure's free-text message, which can contain a scratch path, never reaches the log while its fixed-vocabulary code does. `RenderCoordinator` takes the sink through its existing options bag (`onEvent`), defaulting to the stdout emitter, so tests assert on events without going through the console. * fix(render-service): close the lifecycle for a job cancelled while queued Cancelling a job that has not started yet takes a different path from cancelling a running one: it updates the job store directly rather than going through `finishNonSuccess`. So it emitted no `render_job_finished` event — a submitted job was simply never heard from again, and any success rate computed from the event stream would have been quietly wrong. It also left the job's submission timestamp in the map, which is the one piece of per-job state these events keep. Both are fixed by closing the lifecycle in that branch too. The added test asserts a queued-then-cancelled job reports a terminal event, and fails without the fix. * fix(render-service): make the lifecycle stream summable and its fields honest An independent review of this branch found the event stream could not actually be summed into the rates it exists to produce, plus two tests that could not detect the regressions they were named for. All of these were demonstrated. - A job could emit `render_job_finished` twice. The success event was emitted before the terminal `jobs.update`, so a rejected write landed in the catch and closed the same job again as `failed` — one render counted as both a success and a failure. `JobStore` is a seam a Redis-backed store can fill, so a rejected terminal write is a real case, not a hypothetical. The success event now follows the last fallible await, and `finishEvent` is idempotent: it keys off the timing entry it consumes, so any second close is a no-op. - `durationMs` on a finished job was submission-to-finish while its doc claimed "time spent in the state the event closes", so p95 "render duration" tracked queue depth. It keeps that meaning, now documented, and the event also carries `queueWaitMs` and `renderMs` so a consumer can tell a slow render from a long queue without joining back to the started event. - `running` counts jobs from the moment they leave the queue, but the execution permit is shared with `/preview`, so a counted job may still be blocked. The field is documented as "admitted to run" rather than claiming to be a count of renders in flight. - `render_job_submitted` counted the submitting job in `queued`, so an idle service reported `queued: 1` and the obvious "alert when queued > 0" rule could never be written. It now excludes self, matching `render_job_started`. - `render_admission_rejected` and `preview_request` called the module-level emitter directly, so an embedder injecting a sink silently received three of five event types. `AppDeps` now takes the same `onEvent` seam. - The serializer walked `Object.entries(event)`. Excess-property checking made that safe for object literals but not for spreads, which are exempt — so the header's promise that no identity, path or content is ever logged rested on every future caller avoiding one. It now serializes an explicit field list. Tests: the route tests passed an `executionGate` that `AppDeps` does not have, silently discarded, and unnoticed because `tsconfig` includes only `src` so typecheck never sees `test/`. Removed, and they now assert through the injected sink instead of spying on the console. The leak test asserted only on event shape and still passed with the cleanup deleted; it now observes the tracked state through a `trackedJobs` accessor. Added regressions for the double close, the timing split and the queue depth — each verified to fail without its fix. |
||
|
wyucandClaude Code
|
0128212da4 |
feat(render-service): add POST /preview endpoint (fixes render_scene_preview 404) (#1285)
* feat(render-service): add PreviewGate admission control for previews Independent admission gate (global in-flight cap + per-identity cap) so synchronous previews never compete with the MP4 export queue. Release is idempotent so every route exit path can call it safely. Co-Authored-By: Claude Code <noreply@anthropic.com> * feat(render-service): add single-page preview renderer Renders a scene to a PNG via SlideCanvas + static markup using the existing Chromium executable configuration, egress policy, and semaphore infrastructure, with a deadline/AbortSignal. Co-Authored-By: Claude Code <noreply@anthropic.com> * feat(render-service): add preview config and resource pixel limits Adds previewDeadlineMs / previewMaxInFlight / previewMaxPerUser config and resource-profile viewport pixel limits for previews; declares the renderer runtime dependencies installed with the render-service container. Co-Authored-By: Claude Code <noreply@anthropic.com> * feat(render-service): add POST /preview endpoint Wires the preview endpoint: declared-size rejection, gate admission before buffering, byte-capped payload parsing, scene/stage/viewport validation, deadline/cancellation, PNG response, and 400/413/429/504/500 mapping. Fixes the render_scene_preview tool's broken link (previously 404). Co-Authored-By: Claude Code <noreply@anthropic.com> * fix preview rendering and deadlines * fix(render-service): enforce preview deadline, memory bound, and shared Chromium limit - Stalled preview uploads now observe the deadline signal, error the consumer, and cancel the reader; deadline aborts map to 504 and release the permit. - Previews retain the extraction permit through rendering so parsed payloads cannot accumulate beyond the memory admission bound. - RenderCoordinator owns a shared execution semaphore so previews and MP4 renders respect the same global Chromium concurrency limit. Co-Authored-By: Claude Code <noreply@anthropic.com> * fix(render-service): harden preview admission and rendering * docs(render-service): define the preview deployment contract * fix(render-service): enforce self-contained previews * fix(render-service): allow fragment CSS URLs in previews * fix(render-service): allow background-only previews * fix(render-service): control malformed preview rejections * style(render-service): apply prettier formatting after rebase --------- Co-authored-by: Claude Code <noreply@anthropic.com> |
||
|
|
f760f58a70 |
feat(render-service): machine-readable admission state (429 reasons + health accepting) (#1351)
* feat(render-service): expose machine-readable admission state 429 rejections carried only prose, so clients couldn't distinguish deployment-wide backpressure from a personal limit, and /health gave load balancers no way to shed traffic before probing with a render. - RenderRejectedError now carries a reason code: queue_full (global RENDER_MAX_QUEUE cap) or per_identity_limit (RENDER_MAX_JOBS_PER_USER guard). The internal reservation invariant stays reason-less. - POST /render serializes the reason on 429 bodies via spread-omission, so reason-less rejections don't emit a reason field at all. - /health gains `accepting: boolean` (queue below the global cap). Deliberately aggregate-only: no occupancy counts and never per-identity data — identity keys are client IPs when TRUST_PROXY_HEADERS=true, so the map must not leave the process. - README documents the 429 reason codes, the accepting flag and why it is boolean-only, and the shared-`direct`-identity model that makes the default Compose disable the per-user guard. Refs #1348 Co-Authored-By: Claude Code <noreply@anthropic.com> * refactor(render-service): extract 429 body serialization and shared test helpers Review findings on the admission-observability change: the 429 body was built identically in both catch sites, and the parked-executor / bounded-poll-loop shapes were duplicated across the route and coordinator admission tests. The spec review also flagged that /health's negative space was only verified by inspection. - rejectionBody(error) serializes an admission rejection once; both RenderRejectedError catch sites collapse back to one-line mappings. - The /health inline comment now cites RenderCoordinator#accepting instead of restating its aggregate-only rationale. - parkingExecutor() moves to test/support/fakes.ts beside succeedingExecutor; waitUntil() lands in test/support/async.ts and backs both waitForPoll (route) and the coordinator settle check, with descriptive timeout messages. - A new route test pins /health to its exact key set (accepting, ok, resourceProfile, versions) and a boolean accepting, so any new field must be a deliberate, reviewed change. Refs #1348 Co-Authored-By: Claude Code <noreply@anthropic.com> --------- Co-authored-by: Claude Code <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
杨慎
|
7df8e50054 |
feat(video-export): add bounded local chunk executor (#1115)
* feat(video-export): add bounded local chunk executor * fix(video-export): preserve producer plan hash during chunk execution * fix(video-export): reuse producer chunk sidecars * style(video-export): format render service entrypoint * fix(video-export): tighten chunk cancellation and cache validation * fix(video-export): address chunk lifecycle review findings * test(video-export): cover chunk cancellation and drain * fix(video-export): validate observed chunk capture mode * fix(video-export): enforce chunk plan and resource bounds * fix(video-export): terminate chunks at job deadline * fix(video-export): guard chunk worker IPC teardown |
||
|
杨慎
|
234ea2f540 |
fix(video-export): constrain GenUI iframe visibility (#1125)
* fix(video-export): constrain interactive iframe visibility * fix(render-service): allow screenshot fallback in standard profile * fix(render-service): lower standard memory floor to 8 GiB * refactor(video-export): address review findings |
||
|
杨慎andwyuc
|
019fe6e6ac |
feat(video-export): add explicit CPU resource profiles (#1105)
* feat(video-export): add explicit CPU resource profiles * fix(video-export): preserve failure capture metrics * fix(video-export): report observed failure capture mode * docs(video-export): keep benchmark evidence out of service docs * fix(render-service): build from dated Debian snapshot --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
杨慎andwyuc
|
27b6d082c4 |
refactor(video-export): extract stable RenderExecutor seam (#1104)
* refactor(video-export): extract render executor seam * fix(video-export): preserve executor failure semantics * test(video-export): lock the render HTTP contract --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
杨慎
|
e9d8a33b56 |
fix(render-service): make parallel capture effective (#1042)
* fix(render-service): make parallel capture effective * fix(render-service): preserve adaptive workers and beginframe * fix(render-service): restore adaptive compose sizing |
||
|
杨慎andClaude Opus 4.8
|
84b1907255 |
feat(video-export): service-backed MP4 render + in-app one-click export (#866) (#937)
* feat(video-export): service-backed MP4 render + in-app one-click export (#866)
Adds the last mile of classroom video export: turning the self-contained
Hyperframes project ZIP (#865) into an MP4 via an isolated render service,
one-click in-app.
- render-service/: standalone Node 22 + Chromium + FFmpeg container wrapping
@hyperframes/producer's library API. Async job model (POST /render -> 202
jobId, GET poll, GET download, DELETE cancel). Swappable JobStore /
ArtifactStore seams (in-memory + local-disk now; Redis/S3 + presigned-302
download later) so it scales horizontally without changing the HTTP contract.
Concurrency + per-user guards are config knobs.
- App integration: thin Next proxy routes under app/api/export-video/* (forward
only, no rendering) + capability probe. use-render-video.ts uploads the ZIP,
polls via runPolledTask, downloads the MP4; shared buildExportZip prefix with
the existing ZIP path. Export menu gains resolution/fps/quality selectors and
a progress bar; degrades to ZIP download when RENDER_SERVICE_URL is unset.
- docker-compose: render-service under an opt-in "video-export" profile.
- Entry is main.ts (not server.ts): the producer auto-starts its own server on
:9847 when the process entry path ends with /src/server.ts.
Verified end-to-end in the container: rendered a real 640s (10.7 min) classroom
ZIP to a valid H.264 720p + AAC MP4 (duration matches source) in ~9.6 min
(~0.9x realtime, 4-worker frame capture). Degrade path, queued-cancel + cleanup,
and per-user 429 guard all exercised. pnpm check / lint / tsc / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(video-export): global render progress store, percent+ETA UI, ring on export button (#866)
Addresses two UX issues found while driving the in-app MP4 export:
1. Progress display was raw and unfriendly (showed producer's English stage
strings like "Capturing frame 5130/19220") and had no time estimate. Now the
menu shows only "<percent>% · about <remaining> left". ETA is computed from a
recent-speed estimate (percent-per-ms over the last sample), EMA-smoothed —
which tracks the render's non-uniform pace (prep -> frame capture with a
4->1 worker drop -> encode) far better than a whole-run average, and never
shows a stale/rising ETA.
2. Switching scenes mid-render unmounted the export menu and lost the progress
(and reset the local "already rendering" ref, allowing a duplicate submit).
The whole render lifecycle now lives in a global store
(lib/store/video-render.ts), so progress survives menu close / scene switch
and duplicate submits are guarded by status. A persistent CircularProgress
ring on the export button shows live progress whether or not the menu is open.
Also fixes the progress scale: the producer reports progress as 0..100, but our
HTTP contract (and success path) is 0..1 — the service now normalizes it, so the
client no longer showed "2000%".
- lib/store/video-render.ts: new global store owning submit->poll->download,
recent-speed ETA, duplicate-submit guard.
- lib/video-export-app/use-render-video.ts: thin facade over the store.
- components/ui/circular-progress.tsx: lightweight SVG progress ring.
- components/stage/{header-controls,video-export-menu}.tsx: ring on the export
button; menu shows percent + ETA, subscribes to the store.
- render-service/src/render-manager.ts: normalize producer progress 0..100 -> 0..1.
- i18n: percent/ETA strings across all 8 locales (drops the stage-based string).
- render-service/package-lock.json: complete integrity hashes (reproducible npm ci).
Verified: ETA logic checked against the real segmented render curve (worker drop
raises ETA, encode speedup drives it to ~0); progress scale fix confirmed live
against the container (0.2 -> 20%). tsc / lint / prettier / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): persist render options in the store, not the menu component
Selecting 720p/24fps/draft, switching scenes, and reopening the export menu
showed the defaults again (1080p/30/standard). The selections lived in the
VideoExportMenu component's local state, which reset when the menu unmounted on
a scene switch — the running render still used the chosen options, but the UI
misrepresented them.
Move resolution/fps/quality into the global video-render store (with a
setOptions action). The menu now reads/writes the store, so selections survive
menu close / scene switch, and while a render runs the selectors reflect the
options that render is actually using. startRender() reads options from the
store instead of taking them as an argument.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): deployment correctness + resource/isolation controls (PR #937 review)
Addresses the blocking findings from wyuc's review. Output fidelity was fine;
these harden deployment and production resource/isolation boundaries.
#1 Compose advertised MP4 but couldn't render in prod:
- Capability now probes the service's /health (checkRenderServiceHealth), so a
configured-but-absent service reports disabled and the UI degrades to ZIP
instead of 502-ing.
- RENDER_SERVICE_URL is operator-supplied trusted config, so the proxy no longer
runs it through the SSRF guard — the one-command `docker compose --profile
video-export up` now works without globally weakening SSRF via
ALLOW_LOCAL_NETWORKS. resolveRenderServiceUrl() is now synchronous.
- Client degrades to ZIP on any failed submit (not only 501).
#2 Unbounded upload/queue (ZIP-bomb / DoS):
- unzip.ts bounds the archive via fflate's filter BEFORE decompression: entry
count, per-entry and total expanded size, and compression ratio.
- Proxy rejects oversized uploads (413) by Content-Length before forwarding.
- RenderManager enforces a global queue-depth cap (RENDER_MAX_QUEUE).
- All limits are env-tunable knobs in config.ts.
#3 Per-user guard was ineffective + admission ran after extraction:
- Identity is derived server-side (client IP) and forwarded as x-openmaic-client;
the service ignores any client-supplied userId, and the proxy strips it.
- Admission is split into reserve()/submit()/release(): the slot is reserved
BEFORE extraction, so a rejected caller never triggers a decompression.
Additional risks:
- Per-job wall-clock watchdog (RENDER_JOB_DEADLINE_MS) aborts + fails a hung
render so it can't hold a slot/scratch forever.
- Download proxy bounds only the time-to-headers, not the body stream, so large
MP4s over slow links no longer truncate.
- Client cancels the server job (DELETE) when a started render fails/times out.
- Compose puts render-service on an internal:true network (no host/internet
route), sandboxing the Chromium that runs the uploaded HTML; the export ZIP is
self-contained so no outbound is needed. README documents the standalone caveat.
Not closing #866: the smoke/golden-render CI acceptance criterion remains a
follow-up (see PR description).
Verified in-container: legal render 202; ZIP-bomb (entry-count + compression-
ratio) rejected 400 before any decompression; per-identity guard 429 with a
spoofed multipart userId ignored; reserve-before-extract leaves no scratch dir
on rejection; watchdog aborts an overrunning job and frees the slot. tsc / lint /
prettier / i18n pass; render-service tsc passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): real audio durations + burned-in subtitles (PR #937 review)
Two export-fidelity issues found in wyuc's deeper E2E:
A. Narration was scheduled from estimated durations, cutting audio off mid-
sentence and advancing the timeline early. The scheduler trusted
AudioFileRecord.duration (recorded only since #861), so the many existing
classrooms without it fell back to text-length estimates — measured 4.35s
average / 10.23s max underestimate across 47 clips. timeline-deps now probes
the real duration from each narration blob via an off-document <audio>
(symmetric to the existing video probe), preferring it over the stored
duration, then the estimate only when no audio asset exists. Everything
downstream (narration starts, scene/total duration, subtitle cues) re-derives
from the corrected value in the pure compiler — no compiler change needed.
B. The final MP4 had no subtitles (only H.264+AAC), and the ZIP's SRT/VTT used
the same estimated boundaries. The emitter now renders a burned-in subtitle
overlay: one caption box + a hidden div per cue, revealed/hidden by the paused
GSAP timeline at each cue's start/end (corrected timings from A), so Chromium's
frame capture bakes them in. The producer has no subtitle track of its own, so
burn-in is the v1 approach.
Verified: emitter unit tests + snapshot updated (subtitle overlay + toggle
statements, escaped text, hidden-by-default); 82 video-export tests pass incl.
the determinism red-line proxy. Rendered a synthetic subtitle project through the
container and confirmed by pixel analysis that captions appear only within their
cue window (2429 near-white px in the caption band at t=1.5s vs 0 at t=0.05s).
tsc / lint / prettier / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): subtitle layout + upload/admission hardening (PR #937 review)
Address the three P1 blockers plus actionable P2s from the
|