mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-02 09:24:43 +08:00
@openmaic/generation@0.3.14
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 LING DUANandwyuc
|
80367c0d76 |
feat(render): add admission and per-task resource budgets (#1492)
* feat(render): add admission and per-task resource budgets * fix(ci): isolate patched producer test configuration * fix(render): bind portable resource builds and installed validation * fix(render): preserve deadline errors and service exit ownership * style(render): format verified lifecycle changes * fix(render): retain cleanup evidence and enforce task pid limits * refactor(render): narrow resource patch and acceptance documentation Revert cleanup adapters in two unreferenced Producer helpers. Remove historical runtime claims from the patched README, retain the current Linux NOT_RUN qualification, and map remaining B4 evidence to concrete owner and platform boundaries. Validation: fixed-source patch application and all 44 source hashes pass; 43 retained patch sections and both dependency locks are unchanged. Source/package tests: 17 passed. Cleanup/context/supervisor tests: 30 passed. No native build, VM, installed Linux execution, or export was run. * test(render): cover remaining B4 lifecycle acceptance paths Extend the existing installed runner with guardian unlink failure, overlapping task reference settlement, and successful new-supervisor service after explicit platform takeover. Keep original per-task and outer limits; only the overlap case reserves two task budgets through the existing Producer API. Local validation: syntax, formatting, lint, two evidence regressions, and fixed-source patch/hash verification pass. Installed Linux cases remain NOT_RUN: the first SSH preflight timed out before authentication or any remote command ran. No runtime implementation or dependency version changed. * docs(render): keep runtime qualification tied to candidate evidence Keep the validation document as a reproducible contract rather than a stale NOT_RUN snapshot. Current native and Producer/B4 evidence passed for 6be78252; HTTP remains unexecuted after an adapter input-preflight failure. Runtime status and remaining work are tracked in the PR. * fix(render): harden resource startup and simplify owner transport * docs(render): clarify post-commit cleanup settlement * style(render): format outer resource fence files * fix(render-service): harden resource-mode closeout * fix(render): bind resource cleanup to trusted directory identity Use nonrecursive stage cleanup and preserve quarantine when project identity cannot be verified. Bind mount, publication and cleanup to verified directory descriptors, contain recovery bookkeeping errors, and cover helper cancellation and closeout gates. --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyucandClaude Opus 4.8
|
7420d26ebb |
fix(render-service): contain untrusted HTML with a network policy on preview and render (#1512)
Headless Chromium renders caller-supplied HTML on two paths, and neither applied the Content-Security-Policy the app packager injects for exports. Inline script in a preview scene or an uploaded render project could reach loopback and internal addresses, and on the render path the response is painted into the returned MP4. - Add a single untrusted-HTML CSP and an injector that places the policy as the first node the parser processes (ASCII whitespace only; a leading BOM is stripped; a byte-level injector preserves non-UTF-8 bodies). - Inject the policy into the interactive preview srcDoc and add a Puppeteer request guard that blocks non-data/blob/about requests from the untrusted frame and non-about main-frame navigations. - Harden every extracted project HTML, and sanitize framed same-origin .svg and .xhtml documents (scripts, on* handlers, javascript: URLs, foreignObject, nested frames removed via parse5), which cannot carry a meta CSP. - Document the residual top-level-navigation risk on the render path, which the egress lockdown must contain. - Add real-Chromium boundary tests (zero listener hits incl. WebSocket) and a packager-equivalence test for the policy. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
wyucandClaude Code
|
0128212da4 |
feat(render-service): add POST /preview endpoint (fixes render_scene_preview 404) (#1285)
* feat(render-service): add PreviewGate admission control for previews Independent admission gate (global in-flight cap + per-identity cap) so synchronous previews never compete with the MP4 export queue. Release is idempotent so every route exit path can call it safely. Co-Authored-By: Claude Code <noreply@anthropic.com> * feat(render-service): add single-page preview renderer Renders a scene to a PNG via SlideCanvas + static markup using the existing Chromium executable configuration, egress policy, and semaphore infrastructure, with a deadline/AbortSignal. Co-Authored-By: Claude Code <noreply@anthropic.com> * feat(render-service): add preview config and resource pixel limits Adds previewDeadlineMs / previewMaxInFlight / previewMaxPerUser config and resource-profile viewport pixel limits for previews; declares the renderer runtime dependencies installed with the render-service container. Co-Authored-By: Claude Code <noreply@anthropic.com> * feat(render-service): add POST /preview endpoint Wires the preview endpoint: declared-size rejection, gate admission before buffering, byte-capped payload parsing, scene/stage/viewport validation, deadline/cancellation, PNG response, and 400/413/429/504/500 mapping. Fixes the render_scene_preview tool's broken link (previously 404). Co-Authored-By: Claude Code <noreply@anthropic.com> * fix preview rendering and deadlines * fix(render-service): enforce preview deadline, memory bound, and shared Chromium limit - Stalled preview uploads now observe the deadline signal, error the consumer, and cancel the reader; deadline aborts map to 504 and release the permit. - Previews retain the extraction permit through rendering so parsed payloads cannot accumulate beyond the memory admission bound. - RenderCoordinator owns a shared execution semaphore so previews and MP4 renders respect the same global Chromium concurrency limit. Co-Authored-By: Claude Code <noreply@anthropic.com> * fix(render-service): harden preview admission and rendering * docs(render-service): define the preview deployment contract * fix(render-service): enforce self-contained previews * fix(render-service): allow fragment CSS URLs in previews * fix(render-service): allow background-only previews * fix(render-service): control malformed preview rejections * style(render-service): apply prettier formatting after rebase --------- Co-authored-by: Claude Code <noreply@anthropic.com> |
||
|
|
f760f58a70 |
feat(render-service): machine-readable admission state (429 reasons + health accepting) (#1351)
* feat(render-service): expose machine-readable admission state 429 rejections carried only prose, so clients couldn't distinguish deployment-wide backpressure from a personal limit, and /health gave load balancers no way to shed traffic before probing with a render. - RenderRejectedError now carries a reason code: queue_full (global RENDER_MAX_QUEUE cap) or per_identity_limit (RENDER_MAX_JOBS_PER_USER guard). The internal reservation invariant stays reason-less. - POST /render serializes the reason on 429 bodies via spread-omission, so reason-less rejections don't emit a reason field at all. - /health gains `accepting: boolean` (queue below the global cap). Deliberately aggregate-only: no occupancy counts and never per-identity data — identity keys are client IPs when TRUST_PROXY_HEADERS=true, so the map must not leave the process. - README documents the 429 reason codes, the accepting flag and why it is boolean-only, and the shared-`direct`-identity model that makes the default Compose disable the per-user guard. Refs #1348 Co-Authored-By: Claude Code <noreply@anthropic.com> * refactor(render-service): extract 429 body serialization and shared test helpers Review findings on the admission-observability change: the 429 body was built identically in both catch sites, and the parked-executor / bounded-poll-loop shapes were duplicated across the route and coordinator admission tests. The spec review also flagged that /health's negative space was only verified by inspection. - rejectionBody(error) serializes an admission rejection once; both RenderRejectedError catch sites collapse back to one-line mappings. - The /health inline comment now cites RenderCoordinator#accepting instead of restating its aggregate-only rationale. - parkingExecutor() moves to test/support/fakes.ts beside succeedingExecutor; waitUntil() lands in test/support/async.ts and backs both waitForPoll (route) and the coordinator settle check, with descriptive timeout messages. - A new route test pins /health to its exact key set (accepting, ok, resourceProfile, versions) and a boolean accepting, so any new field must be a deliberate, reviewed change. Refs #1348 Co-Authored-By: Claude Code <noreply@anthropic.com> --------- Co-authored-by: Claude Code <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
杨慎
|
7df8e50054 |
feat(video-export): add bounded local chunk executor (#1115)
* feat(video-export): add bounded local chunk executor * fix(video-export): preserve producer plan hash during chunk execution * fix(video-export): reuse producer chunk sidecars * style(video-export): format render service entrypoint * fix(video-export): tighten chunk cancellation and cache validation * fix(video-export): address chunk lifecycle review findings * test(video-export): cover chunk cancellation and drain * fix(video-export): validate observed chunk capture mode * fix(video-export): enforce chunk plan and resource bounds * fix(video-export): terminate chunks at job deadline * fix(video-export): guard chunk worker IPC teardown |
||
|
杨慎
|
234ea2f540 |
fix(video-export): constrain GenUI iframe visibility (#1125)
* fix(video-export): constrain interactive iframe visibility * fix(render-service): allow screenshot fallback in standard profile * fix(render-service): lower standard memory floor to 8 GiB * refactor(video-export): address review findings |
||
|
杨慎andwyuc
|
019fe6e6ac |
feat(video-export): add explicit CPU resource profiles (#1105)
* feat(video-export): add explicit CPU resource profiles * fix(video-export): preserve failure capture metrics * fix(video-export): report observed failure capture mode * docs(video-export): keep benchmark evidence out of service docs * fix(render-service): build from dated Debian snapshot --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
杨慎andwyuc
|
27b6d082c4 |
refactor(video-export): extract stable RenderExecutor seam (#1104)
* refactor(video-export): extract render executor seam * fix(video-export): preserve executor failure semantics * test(video-export): lock the render HTTP contract --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
杨慎
|
e9d8a33b56 |
fix(render-service): make parallel capture effective (#1042)
* fix(render-service): make parallel capture effective * fix(render-service): preserve adaptive workers and beginframe * fix(render-service): restore adaptive compose sizing |
||
|
杨慎andClaude Opus 4.8
|
6d29241bb0 |
feat(video-export): fidelity polish — spotlight geometry, video clips, formulas, subtitles (#952)
* fix(video-export): bridge play_video element id → media ref Generated-media records are keyed by the element's media ref (gen_vid_…), but a play_video action targets the slide element by its .id. The app-side compiler deps looked media/durations up by the raw element id, so every generated video missed → present:false → no <video> emitted, no bytes collected → the clip vanished from the export (only its static first frame, baked into the base PNG, remained). Build an elementId → mediaRef bridge across all slide scenes in createVideoTimelineDeps (mirroring the live engine's resolveMediaPlaceholderId) and resolve both media() and videoDurationMs() through it. Refs #867. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): re-measure KaTeX fit-scale after fonts settle KaTeX sizes large delimiters (\left\{, \begin{cases}) from its KaTeX_Size faces, which load asynchronously. KatexContent measured its shrink-to-fit scale once on mount, against the fallback font, and never recomputed — so a cold slide snapshot baked a stale scale and the big brace desynced from the piecewise body ("大括号后面的分段函数与大括号错位"). - Re-run the fit-scale measurement on document.fonts.ready and on each `loadingdone`, keeping the shrink-only (cap-at-1) behavior. - slideToPng waits two extra frames before capture so the font-triggered React re-measure commits into the DOM html2canvas reads. Refs #867. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): place spotlight/laser/video on the rendered content box The pure geometry helper resolves an element's authored outer box against a fixed 1000×562.5 base, but the live overlays — and the exported frame PNG — measure the element's `.element-content` box, which for horizontal text is auto-height plus 10px content padding. So a spotlight/laser sat offset from where the text actually renders ("spotlight 与对应元素的位置有偏差", #867 item 5). Add an optional GeometryProbe to the compiler's DI boundary: when supplied, the geometry pass prefers each element's measured content-box geometry (video keeps its rotation from the authored element), degrading to the pure authored-box calc on a miss so the compiler stays deterministic in unit contexts. App side: measureSlideElementGeometry (new, in @openmaic/renderer/snapshot) mounts the slide off-screen and reads the same `.element-content`-against- container box the live SpotlightOverlay uses; createVideoTimelineDeps pre-measures every spotlight/laser/play_video target per scene and serves the probe as a synchronous table lookup. Tests: geometry.test.ts quantifies the outer-box vs content-box delta and proves probe-wins / fallback / rotation-preservation; timeline-deps.test.ts covers the pre-measure wiring. Refs #867. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): decode a first-frame poster for posterless videos A play_video clip only shows during its play window; the rest of the time the static base-frame PNG backs that spot. But html2canvas can't draw a <video>, so the snapshot swaps it for its poster <img> — and generated videos usually carry no poster, leaving the video area blank whenever the clip isn't playing. When a video record has no stored poster bytes, decode the video blob's first frame (seek slightly off 0, draw to a canvas) and set element.poster before the base snapshot, so that spot shows the first frame — the "paused on frame one" look — instead of blank. Falls back to no poster on decode failure / timeout / CORS taint, never failing the export. Refs #867. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(video-export): split subtitles, make burn-in optional, centered modal Burned-in captions used one cue per whole narration paragraph, so a long cue became a 4–6 line block covering the lower half of the slide, and there was no way to turn it off. - Split cues (pure, compiler layer): lib/video-export/split-cue.ts splits each cue on sentence → clause → hard-wrap to ≤~40 CJK-equiv units, distributing the parent window by character weight (last piece pinned to the parent end, sub-1.2s slivers merged). Wired into the timeline pass, so the burned-in overlay and the SRT/VTT sidecar share one split track and can't drift. - Burn-in optional (default off, #867 item 2): emitHyperframes gains burnInSubtitles; off → clean video + sidecar SRT/VTT only. Caption layout hard- clamped to 2 lines so an outlier can't grow tall again. - Download subtitles without rendering: compileSubtitles() runs only the compiler (no assets/zip/service); new useDownloadSubtitles saves an .srt. - Centered modal: video-export-dialog.tsx replaces the cramped dropdown section, grouping Output / Subtitles / Actions / Progress with a stable lifecycle for multi-minute renders. header-controls opens it from an "Export Video" entry; old video-export-menu.tsx removed. i18n keys added across all 8 locales. Refs #867. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style: prettier format video-export fidelity changes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: drop unused playVideo helper in timeline-deps test The media-bridge tests assert via deps.assets.media(...) directly, so the playVideo helper (and its PlayVideoAction import) were dead. Flagged by the code-quality bot on #952. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): resolve cross-review findings + lower subtitle band - emit-hyperframes: drop duplicate `display:-webkit-box` on subtitle cues so they start truly `display:none` (a second declaration overrode the `none`, showing every cue stacked at t=0 when burn-in was enabled). - geometry: don't feed the measured content-box AABB to a rotated video clip — it already encloses the rotation, so re-applying `rotate` doubled it. Rotated elements fall back to the authored box (one, un-doubled source of truth). - build-export-zip: extract shared `compileStageIr` so buildExportZip and compileSubtitles can't drift on the timing/assets/geometry wiring. - split-cue: join merged CJK cue pieces without a spurious ASCII space. - subtitles: lower the caption band (bottomRatio 0.055 → 0.01) and hoist the band's magic numbers into a named SUBTITLE constants block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): stop the snapshot onclone override from reshaping KaTeX The export snapshot injects a neutral-kerning / geometricPrecision style over `.slide-renderer-prose *` to keep CJK prose+table glyphs from mis-advancing under html2canvas-pro. KaTeX formulas (output:'html') also live inside `.slide-renderer-prose`, so the override was reshaping their math glyph advances — a burned-in formula rendered subtly differently from the live canvas, which applies no such override. Empirical check (Playwright + system Chrome, cases+fraction formula, native paint vs html2canvas): the override raised the formula's mean per-pixel delta vs the live paint from 120.1 to 139.2 (~16%) and moved ~75% of inked pixels. Restoring native font-kerning/text-rendering for `.katex` subtrees brings the formula's delta back to 120.1 (bit-identical to the no-override capture) while prose and table text keep the override. The residual 120.1 is the deeper html2canvas rasterization gap (vlist/frac-line/delimiter), a separate track. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): render slides via native paint, bundle KaTeX font embed Root-fix the exported-vs-live formula distortion (and the filter/mask/video gaps) by rasterizing slides with native Chrome paint instead of html2canvas's reimplementation. - slideToPng now captures via html-to-image (foreignObject → the same Chrome engine that paints the live classroom rasterizes the same DOM). KaTeX formulas, CSS filter, soft-edge masks, and mixed CJK/Latin text come out as the classroom shows them. html2canvas-pro stays as a fallback (with the filter/mask bakes + KaTeX text-rendering reset) for when native paint can't run — e.g. a cross-origin image taints the canvas. - A foreignObject SVG can't reach the document font registry, so fonts must be inlined. KaTeX math faces are prepended from the bundled woff2 (KATEX_FONT_EMBED_CSS, generated by scripts/generate-katex-fonts.mjs), NOT left to runtime getFontEmbedCSS — which reads cssRules and silently drops a cross-origin KaTeX stylesheet, collapsing a large brace to a fallback glyph (the observed "formula render failure" on e.g. a \begin{cases} with \text{}). A per-face guard verifies every KaTeX face the formula references is embedded, falling back to html2canvas otherwise. Verified in a Playwright spike (native vs html2canvas vs html-to-image): formula inked-delta 143→53; filter/mask/video match native; and with getFontEmbedCSS forced empty, the bundled embed alone still renders the failing formula correctly. Renderer package tests: 226 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): prettier-format the generated KaTeX font-embed module The generator emitted an unformatted TS file (double quotes, long unwrapped CSS literal) that failed CI `prettier --check`. Run the generated output through the repo's Prettier config before writing, so every rebuild produces CI-clean, stable output. Regenerated the committed file to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lint): rename reserved `module` var in katex-fonts generator; ignore .scratch - generate-katex-fonts.mjs assigned to a variable named `module`, tripping @next/next/no-assign-module-variable (the one hard eslint error). Renamed to `fileContents`; generated output is byte-identical (deterministic). - Add `.scratch/**` to the eslint globalIgnores (alongside .claude/.worktrees) so local throwaway verification scaffolding doesn't pollute `pnpm lint`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): address PR #952 review — scene-scoped media bridge, safer cue splitting Resolves the two blocking correctness issues and three follow-ups from the cross-review on #952. - Scope the elementId→mediaRef bridge by scene (was deck-wide/last-writer-wins): slide-local ids like `video_001` now resolve within their own scene. `assets.media` uses its scene arg; `timing.videoDurationMs` resolves via action object identity. Adds a cross-scene collision test. - Stop treating every ASCII period as a sentence boundary: a lone `.` splits only when not between word chars (3.14, v1.2) and not an abbreviation dot (e.g.), collapsing repeated-punctuation runs. Adds decimal/version/abbrev/ellipsis tests. - Share the module-level in-flight guard across ZIP export and subtitle download (extracted to export-in-flight.ts); dialog `busy` folds in `downloading`. - Skip the off-screen geometry render for the subtitles-only path (skipGeometry); duration probes are kept so sidecar cue timing stays in sync with the video. - Align measure.ts font settling with slideToPng: force-load each used face before fonts.ready so measured content-boxes don't bake fallback advance widths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(video-export): prettier-format split-cue test Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(video-export): enable beginFrame capture via PRODUCER_BROWSER_GPU_MODE=hardware The producer defaults to `software` GPU mode, which silently force-enables the CPU-bound `Page.captureScreenshot` fallback: SwiftShader pins every core on rasterization and caps frame capture at ~10 fps. Selecting `hardware` keeps Chromium's `HeadlessExperimental.beginFrame` capture active instead. Measured on a 900-frame (30s) clip, no real GPU on the host: - capture ~10 -> ~19 fps, total 100s -> 62s (~38% faster) - CPU peak ~810% -> ~98%; peak mem 2.2 GiB -> 0.9 GiB - static frames pixel-identical; animation frames differ only in sub-pixel edge antialiasing (same-path re-render is deterministic/inf PSNR) No real GPU is required — the win is the faster capture API, not hardware rasterization; beginFrame falls back gracefully if the host lacks a GPU. Also eases the RENDER_JOB_DEADLINE_MS timeout risk on long videos. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(video-export): sentence splitting via Intl.Segmenter + abbreviation merge Addresses the PR #952 follow-up: separate sentence-boundary detection from cue shaping and stop growing a custom punctuation heuristic. - splitSentences now delegates boundary detection to Intl.Segmenter (locale- agnostic ICU), the multilingual/CJK baseline. It already preserves 3.14, v1.2, e.g., U.S., ellipses, and CJK 。！？… without any hand-rolled period scan. - ICU still ends a sentence after a titlecase abbreviation before a capitalized word (Dr. Smith, Fig. 3) — the edge case the reviewer flagged. A small abbreviation-aware post-merge re-joins those, driven by an ABBREVIATIONS set. - Semicolons are no longer sentence ends (they join clauses); moved ；; into the clause-level SECONDARY splitter so over-budget sentences still break there. - Cue shaping (budget / clause / hard-wrap / weighted timing) is unchanged, so detection and shaping are now distinct stages. Not using `sbd` as suggested: lib/video-export/*.ts sits behind the machine- enforced eslint purity boundary (imports limited to @openmaic/dsl, zod, ../choreography, siblings), so a new npm dep is rejected. Intl.Segmenter is a pure Node >=20.9 builtin, matching the reviewer's "small abbreviation-aware post-merge layer" alternative. Tests: add Dr. Smith / Fig. 3 / U.S. coverage; the rhetorical ellipsis now stays one under-budget cue (was split); add a semicolon over-budget case. 22/22 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(video-export): prettier-format split-cue sentence refactor Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * i18n(es-MX): add 7 subtitle/video export keys to align with en-US main added the Spanish (Mexico) locale (#942) after this branch's subtitle keys landed, so es-MX was the only locale missing export.subtitles* and export.videoBurnSubtitles* — which failed the i18n key-alignment CI check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): restrict title post-merge so it never deletes real boundaries Addresses the blocking re-review on the Intl.Segmenter change: the abbreviation post-merge unconditionally rejoined the next ICU segment whenever the previous one ended in a listed abbreviation, which erased genuine sentence boundaries — `It was made by Acme Inc. Next point.` collapsed into one cue, and even an explicit newline (`Acme Inc.\nNext topic`) was rejoined. The merge now fires only for the case ICU actually mis-splits — a title (Mr/Mrs/Ms/Dr/Prof/Sr/Jr) before a capitalized proper noun — and only when: - the previous segment does not end at a newline (hard boundary), and - the next segment starts with a capital letter. Sentence-trailing abbreviations (Inc./Ltd./etc./vs./No.) are dropped from the set entirely: they legitimately end sentences, and ICU already keeps a title before a number (No. 5, Fig. 3) or a lowercase continuation (etc. and…) whole without any merge, so nothing there needs rejoining. Regression tests: Inc./etc. boundary preserved, newline-after-title preserved, lowercase continuation kept whole, plus the retained Dr. Smith / Fig. 3 / U.S. cases. 26 split-cue tests pass; full video-export suite 123 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
杨慎andClaude Opus 4.8
|
84b1907255 |
feat(video-export): service-backed MP4 render + in-app one-click export (#866) (#937)
* feat(video-export): service-backed MP4 render + in-app one-click export (#866)
Adds the last mile of classroom video export: turning the self-contained
Hyperframes project ZIP (#865) into an MP4 via an isolated render service,
one-click in-app.
- render-service/: standalone Node 22 + Chromium + FFmpeg container wrapping
@hyperframes/producer's library API. Async job model (POST /render -> 202
jobId, GET poll, GET download, DELETE cancel). Swappable JobStore /
ArtifactStore seams (in-memory + local-disk now; Redis/S3 + presigned-302
download later) so it scales horizontally without changing the HTTP contract.
Concurrency + per-user guards are config knobs.
- App integration: thin Next proxy routes under app/api/export-video/* (forward
only, no rendering) + capability probe. use-render-video.ts uploads the ZIP,
polls via runPolledTask, downloads the MP4; shared buildExportZip prefix with
the existing ZIP path. Export menu gains resolution/fps/quality selectors and
a progress bar; degrades to ZIP download when RENDER_SERVICE_URL is unset.
- docker-compose: render-service under an opt-in "video-export" profile.
- Entry is main.ts (not server.ts): the producer auto-starts its own server on
:9847 when the process entry path ends with /src/server.ts.
Verified end-to-end in the container: rendered a real 640s (10.7 min) classroom
ZIP to a valid H.264 720p + AAC MP4 (duration matches source) in ~9.6 min
(~0.9x realtime, 4-worker frame capture). Degrade path, queued-cancel + cleanup,
and per-user 429 guard all exercised. pnpm check / lint / tsc / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(video-export): global render progress store, percent+ETA UI, ring on export button (#866)
Addresses two UX issues found while driving the in-app MP4 export:
1. Progress display was raw and unfriendly (showed producer's English stage
strings like "Capturing frame 5130/19220") and had no time estimate. Now the
menu shows only "<percent>% · about <remaining> left". ETA is computed from a
recent-speed estimate (percent-per-ms over the last sample), EMA-smoothed —
which tracks the render's non-uniform pace (prep -> frame capture with a
4->1 worker drop -> encode) far better than a whole-run average, and never
shows a stale/rising ETA.
2. Switching scenes mid-render unmounted the export menu and lost the progress
(and reset the local "already rendering" ref, allowing a duplicate submit).
The whole render lifecycle now lives in a global store
(lib/store/video-render.ts), so progress survives menu close / scene switch
and duplicate submits are guarded by status. A persistent CircularProgress
ring on the export button shows live progress whether or not the menu is open.
Also fixes the progress scale: the producer reports progress as 0..100, but our
HTTP contract (and success path) is 0..1 — the service now normalizes it, so the
client no longer showed "2000%".
- lib/store/video-render.ts: new global store owning submit->poll->download,
recent-speed ETA, duplicate-submit guard.
- lib/video-export-app/use-render-video.ts: thin facade over the store.
- components/ui/circular-progress.tsx: lightweight SVG progress ring.
- components/stage/{header-controls,video-export-menu}.tsx: ring on the export
button; menu shows percent + ETA, subscribes to the store.
- render-service/src/render-manager.ts: normalize producer progress 0..100 -> 0..1.
- i18n: percent/ETA strings across all 8 locales (drops the stage-based string).
- render-service/package-lock.json: complete integrity hashes (reproducible npm ci).
Verified: ETA logic checked against the real segmented render curve (worker drop
raises ETA, encode speedup drives it to ~0); progress scale fix confirmed live
against the container (0.2 -> 20%). tsc / lint / prettier / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): persist render options in the store, not the menu component
Selecting 720p/24fps/draft, switching scenes, and reopening the export menu
showed the defaults again (1080p/30/standard). The selections lived in the
VideoExportMenu component's local state, which reset when the menu unmounted on
a scene switch — the running render still used the chosen options, but the UI
misrepresented them.
Move resolution/fps/quality into the global video-render store (with a
setOptions action). The menu now reads/writes the store, so selections survive
menu close / scene switch, and while a render runs the selectors reflect the
options that render is actually using. startRender() reads options from the
store instead of taking them as an argument.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): deployment correctness + resource/isolation controls (PR #937 review)
Addresses the blocking findings from wyuc's review. Output fidelity was fine;
these harden deployment and production resource/isolation boundaries.
#1 Compose advertised MP4 but couldn't render in prod:
- Capability now probes the service's /health (checkRenderServiceHealth), so a
configured-but-absent service reports disabled and the UI degrades to ZIP
instead of 502-ing.
- RENDER_SERVICE_URL is operator-supplied trusted config, so the proxy no longer
runs it through the SSRF guard — the one-command `docker compose --profile
video-export up` now works without globally weakening SSRF via
ALLOW_LOCAL_NETWORKS. resolveRenderServiceUrl() is now synchronous.
- Client degrades to ZIP on any failed submit (not only 501).
#2 Unbounded upload/queue (ZIP-bomb / DoS):
- unzip.ts bounds the archive via fflate's filter BEFORE decompression: entry
count, per-entry and total expanded size, and compression ratio.
- Proxy rejects oversized uploads (413) by Content-Length before forwarding.
- RenderManager enforces a global queue-depth cap (RENDER_MAX_QUEUE).
- All limits are env-tunable knobs in config.ts.
#3 Per-user guard was ineffective + admission ran after extraction:
- Identity is derived server-side (client IP) and forwarded as x-openmaic-client;
the service ignores any client-supplied userId, and the proxy strips it.
- Admission is split into reserve()/submit()/release(): the slot is reserved
BEFORE extraction, so a rejected caller never triggers a decompression.
Additional risks:
- Per-job wall-clock watchdog (RENDER_JOB_DEADLINE_MS) aborts + fails a hung
render so it can't hold a slot/scratch forever.
- Download proxy bounds only the time-to-headers, not the body stream, so large
MP4s over slow links no longer truncate.
- Client cancels the server job (DELETE) when a started render fails/times out.
- Compose puts render-service on an internal:true network (no host/internet
route), sandboxing the Chromium that runs the uploaded HTML; the export ZIP is
self-contained so no outbound is needed. README documents the standalone caveat.
Not closing #866: the smoke/golden-render CI acceptance criterion remains a
follow-up (see PR description).
Verified in-container: legal render 202; ZIP-bomb (entry-count + compression-
ratio) rejected 400 before any decompression; per-identity guard 429 with a
spoofed multipart userId ignored; reserve-before-extract leaves no scratch dir
on rejection; watchdog aborts an overrunning job and frees the slot. tsc / lint /
prettier / i18n pass; render-service tsc passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): real audio durations + burned-in subtitles (PR #937 review)
Two export-fidelity issues found in wyuc's deeper E2E:
A. Narration was scheduled from estimated durations, cutting audio off mid-
sentence and advancing the timeline early. The scheduler trusted
AudioFileRecord.duration (recorded only since #861), so the many existing
classrooms without it fell back to text-length estimates — measured 4.35s
average / 10.23s max underestimate across 47 clips. timeline-deps now probes
the real duration from each narration blob via an off-document <audio>
(symmetric to the existing video probe), preferring it over the stored
duration, then the estimate only when no audio asset exists. Everything
downstream (narration starts, scene/total duration, subtitle cues) re-derives
from the corrected value in the pure compiler — no compiler change needed.
B. The final MP4 had no subtitles (only H.264+AAC), and the ZIP's SRT/VTT used
the same estimated boundaries. The emitter now renders a burned-in subtitle
overlay: one caption box + a hidden div per cue, revealed/hidden by the paused
GSAP timeline at each cue's start/end (corrected timings from A), so Chromium's
frame capture bakes them in. The producer has no subtitle track of its own, so
burn-in is the v1 approach.
Verified: emitter unit tests + snapshot updated (subtitle overlay + toggle
statements, escaped text, hidden-by-default); 82 video-export tests pass incl.
the determinism red-line proxy. Rendered a synthetic subtitle project through the
container and confirmed by pixel analysis that captions appear only within their
cue window (2429 near-white px in the caption band at t=1.5s vs 0 at t=0.05s).
tsc / lint / prettier / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): subtitle layout + upload/admission hardening (PR #937 review)
Address the three P1 blockers plus actionable P2s from the
|