126 Commits

Include renames

Author	SHA1	Message	Date
HEOJUNFOandClaude Opus 5.5	1c70e86a13	fix(media): poll and download Veo videos the Gemini API way (#1695) ... * fix(media): poll and download Veo videos the Gemini API way The Veo adapter targets generativelanguage.googleapis.com but polled with Vertex AI's models/{model}:fetchPredictOperation and read inline response.videos[].bytesBase64Encoded. The Gemini API reads operations with GET v1beta/{name} and returns response.generateVideoResponse.generatedSamples[].video.uri, fetched with the same API key. - Poll with GET v1beta/{operationName}. - Download the sample URI with x-goog-api-key into a data URL, always through the configured base URL's origin and without following redirects, like the other adapter calls. Inline bytes are still accepted when present. - Report RAI-filtered operations as a failure with their reasons. - Catalog: Gemini API model IDs for Veo 3.1 (-preview, plus Lite) and only the 16:9 / 9:16 aspect ratios the API accepts. Fixes #1694 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(media): follow the Veo download redirect through a validated transport The Gemini API file URI can answer with a redirect to storage (Google's own example downloads it with `curl -L`), which the download refused, so a generation could fail after a successful poll. - VideoGenerationConfig gets an optional downloadFetchImpl. fetchImpl is the pinned transport that refuses every 3xx, so the download needs its own one. - lib/server/media-provider-fetch.ts adds mediaDownloadFetch and managedMediaDownloadFetch: providerFetch with redirects followed, each hop re-validated under the operator address policy, pinned to the vetted DNS answers, and stripped of x-goog-api-key once it leaves the origin (fetchWithRedirectValidation). withVideoProviderFetch installs both transports. - The server video call sites (the generate route, classroom media generation, the agent runtime) inject it; the adapter still does not import server code. - downloadVideo follows redirects only through the injected transport and keeps redirect: 'manual' + assertNotRedirected without it. - Test: a 302 to another origin yields the data URL and the second hop carries no x-goog-api-key; without the transport the 302 is refused. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-28 23:00:04 +08:00
wyucandClaude Opus 5.5	cd7d6a1a59	fix(providers): pin caller-chosen provider requests and keep upstream detail out of errors (#1704) ... * fix(pdf): pin verify-pdf-provider probes to the strict provider transport The MinerU Cloud and self-hosted connectivity probes validated a caller-supplied base URL once and then issued a plain fetch that resolved DNS again, so a rebinding hostname could pass validation and connect to a loopback or private address. The response also echoed the target's status, its 401/403 body, and per-errno connection errors. Both probes now go through providerFetch with reject-redirects and the operator address policy (the same ALLOW_LOCAL_NETWORKS policy the route already validates against), so the connect address is pinned to the vetted DNS answers. A 3xx still maps to REDIRECT_NOT_ALLOWED. Authentication failures return a fixed message, every other connection failure returns a single generic message, and the success payload no longer includes the target status. Details are logged server-side only. Tests drive the real route, guard and pinned transport against loopback servers, covering rebinding, auth body suppression, identical refused/not-found/timeout answers, redirect refusal and the managed path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(pdf): pin self-hosted MinerU parsing to the strict provider transport Self-hosted MinerU parsing validated a caller-supplied base URL once and then posted the document with a plain fetch that re-resolved DNS and followed redirects, so a public host could hand the upload (and API key) to a loopback or private address. The /file_parse request now goes through providerFetch under the operator address policy with redirects refused. Transport failures, policy blocks and redirects collapse into one fixed message; an unknown error status is reported without its body (the missing-dependency classification stays), a non-JSON body no longer surfaces the parser's input snippet, and the empty-result error no longer lists response keys. MinerU Cloud control plane, upload and ZIP errors likewise stop quoting raw response bodies. Tests drive the real parse route and pinned transport against loopback servers: rebinding, redirect refusal, body suppression, the multipart upload shape and the managed path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(tts): pin the Azure voice list request to the strict provider transport The voice list request validated the caller-supplied base URL once and then fetched it with a plain fetch that resolved DNS again. It also mirrored the target's HTTP status as the route's own status and returned the full error body, plus any JSON the target answered on success. The request now goes through providerFetch under the operator address policy with redirects refused and a 20s deadline. Non-2xx answers return a fixed 502 (authentication failures get their own fixed message), only a JSON array is returned as the voice list, and transport failures share one fixed 500 message. Details are logged server-side only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(providers): pin model-list probing to the strict provider transport Model discovery validated the caller-supplied base URL and models URL once and then issued a plain fetch per candidate, which resolved DNS again. Non-2xx answers carried up to 512 bytes of the provider's body into the route response, and a non-JSON success body surfaced the parser's input snippet. fetchModels now defaults to providerFetch under the operator address policy with redirects refused (a refused hop keeps the REDIRECT_NOT_ALLOWED contract), and accepts an injected transport for tests. Error bodies are never read; the route keeps its 401/404/403 contracts, reports other HTTP failures by status class only, answers a non-JSON list and every transport failure with fixed messages, and returns 400 for a malformed request body. The rejected-redirect detector is shared from the transport module instead of being copied per caller. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(media): run image/video provider requests on the strict provider transport The image and video routes validated a caller-supplied provider base URL once, then every adapter issued plain fetch calls that resolved DNS again. Adapter errors carried full non-2xx bodies into the route responses, the auth-only probes returned the 401/403 body, and connectivity failures echoed the transport error text. Adapters now take a `fetchImpl` from their config and use it for every provider request (submit, poll, download and connectivity probe). The adapters are also imported by the settings UI, so they cannot import the server transport; every server caller (the four routes, classroom media generation and the agent runtime tools) injects mediaProviderFetch, which is providerFetch under the operator address policy with redirects refused. Connectivity results are fixed text: authentication failures, redirects, other HTTP statuses and transport failures each have one message, and no provider body is read. The generation routes log the adapter error and answer a fixed message, keeping the content-safety classification. A malformed Kling key is still reported before any request. The rejected-redirect detector moves to a dependency-free module so the browser-bundled probe helper can share it with the server transport. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(llm): pin LLM calls to a client-supplied base URL resolveModel validated a client-supplied base URL once and then handed the AI SDK a redirect-validating fetch whose only dispatcher was the timeout-only agent, so connect-time DNS was resolved again without the guard. verify-model also echoed the provider's error message (which can carry its response body) and distinguished errno classes. A client-supplied base URL now gets providerFetch under the operator address policy with redirects refused. The pinned dispatcher accepts headers/body timeouts, and this path uses the same 15-minute budget as the default LLM dispatcher; the transport's body normalization and streaming are unchanged. Operator-configured endpoints keep the existing redirect-validating transport. verify-model now classifies failures by the provider's HTTP status only (401/403, 404, 429, other status class) and answers every transport or parse failure with one fixed message. Tests stream a real SSE chat completion through the pinned path and check that deltas arrive before the server finishes, plus rebinding, redirect refusal and the managed path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(pdf): accept only official AliDocMind endpoints from clients AliDocMind requests go through the vendor SDK, which builds its own HTTPS agent and resolves the endpoint itself, so a client-supplied endpoint that passed the URL guard could not be pinned to the validated address. The credential check also returned the SDK's error code and message, which told a refused port from a TLS failure or a timeout. When the provider is not server-managed, verify-pdf-provider, parse-pdf and extract-document (document and media paths) now accept a client endpoint only when it is an official docmind-api.<region>.aliyuncs.com host over https with no port, path or credentials, and answer INVALID_URL before any SDK call otherwise. The accepted endpoint is passed on as the normalized host. Server-managed endpoints are unchanged. Credential verification failures now return fixed messages and log the SDK detail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(agent-runtime): download provider result URLs under the strict public policy The agent-runtime image and video tools downloaded the URL a provider returned with a plain fetch, validating each redirect hop with the operator policy but connecting without pinning, and accepting http. A data: URL from an adapter that inlines its result (the OpenRouter video adapter) was rejected instead of decoded. The classroom download helper's policy is extracted into fetchProviderResultUrl: data: URLs are decoded locally, anything else must be https and pass the strict public policy (allowLocalNetworks false, regardless of ALLOW_LOCAL_NETWORKS), and the request goes through providerFetch, which re-validates redirect hops under the same policy and pins connect-time DNS. The agent-runtime tools and the classroom helper share it; the existing bounded reads and content-type checks are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): note pinned provider transports and body-free provider errors Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(providers): keep vendor endpoint rules out of provider-neutral modules The provider-neutrality guard keeps vendor knowledge out of the capability routes and model resolution. The AliDocMind endpoint rule now sits behind provider-neutral helpers (checkClientDocumentExtractorBaseUrl and checkClientMediaExtractorBaseUrl), which apply the official-endpoint rule to extractors whose SDK cannot be pinned and the URL guard to the rest. The pinned LLM transport policy moves out of resolve-model into its own module. Behavior is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(providers): hold IP-literal request hosts to the transport address policy The pinned dispatcher judges hostnames in its connect-time lookup, but Node never runs that lookup for an IP-literal host, so a request to a loopback or private IP reached it without the local-network opt-in. The provider transport now checks an IP-literal origin against the same policy before connecting, so it enforces the policy on its own rather than relying on every caller to validate the URL first. Tests that reached loopback IP literals without the opt-in now set it, as a self-hosted deployment would; cloud metadata stays refused under every policy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(providers): let operator-configured local providers connect without the opt-in Image/video providers and self-hosted MinerU moved to the pinned transport under the operator address policy for every base URL, so a server-managed endpoint on a local network (for example a local Lemonade image server or a MinerU container) stopped working unless ALLOW_LOCAL_NETWORKS was set. A server-managed base URL is operator configuration: it now runs with local networks allowed, still pinned and still refusing redirects, while cloud metadata and reserved ranges stay refused. Caller-supplied base URLs keep the operator policy. Media routes pick the transport by the provider's managed flag; server-internal media generation uses the managed transport; document extraction carries a `managed` flag to the MinerU parsers and the PDF verification probe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(llm): pin every caller-chosen endpoint and keep transport detail out of errors An unmanaged provider picked by the caller without a base URL fell back to its catalog default (for example a localhost Ollama or Lemonade endpoint) on the operator transport, which neither validated nor pinned the origin. The pinned client transport is now chosen whenever the caller picked the model or sent a base URL, and the effective endpoint (client URL or catalog default) is validated under the operator policy first. A model the operator selected through MODEL_ROUTES or DEFAULT_MODEL keeps the operator transport. Routes relay LLM error messages, and the AI SDK builds them from the fetch failure cause and from the provider's error body. On the caller-chosen transport a failed request now surfaces a fixed reason ("connection failed", "request timed out" or "redirects are not allowed") with the system error logged server-side, and an HTTP error response reaches the SDK with an empty body and the standard reason phrase, keeping its status and headers for retry and status classification. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(pdf): keep MinerU Cloud envelope text out of errors and refuse query/fragment base URLs MinerU Cloud errors quoted the endpoint's envelope `msg`, a failed row's `err_msg` and the batch id, and the parse routes relay error messages. They now report the context with the HTTP status or, for a rejected request, the numeric code only; the endpoint text is logged server-side. Provider paths are appended to a base URL as text, so a client base URL ending in `?` or `#` (or carrying a query) absorbs the fixed path and leaves the request target to the caller. Client-supplied provider base URLs are now refused when they contain a query string or fragment, across the LLM, media, document extraction, PDF verification, model probe and Azure voice routes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(media): bound provider data: URL results before decoding them `fetchProviderResultUrl` decoded a provider-returned `data:` URL in full and left the size check to the caller's body reader, so an oversized payload was materialized first. Callers now pass their byte limit, and the payload size is estimated from the encoded length (base64: 3/4 less padding; percent-encoded: at least one byte per three characters) and refused over the limit before any buffer is built, then checked exactly after decoding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): describe provider transport behavior changes accurately Note which base URLs refuse redirects (MinerU Cloud API roots still follow validated, pinned hops), the query/fragment base URL refusal, IP-literal hosts under the address policy, validation of an unmanaged LLM provider's built-in default, server-configured local providers working without the opt-in, HTTPS for agent-runtime video and poster downloads, pinned requests bypassing the environment proxy, and the fixed LLM and MinerU Cloud error text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(llm): bound the logged error body and map out-of-range statuses to 502 An error response from a caller-chosen LLM endpoint was read in full only to log 500 characters, and a status of 600-999 (passed through by the transport) made the replacement Response constructor throw. Read at most 1 KB (or 1 s) of the body before cancelling it, and report statuses above 599 as 502. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(pdf): hold MinerU Cloud redirect hops to the operator policy for a managed root A server-managed MinerU Cloud API root runs with local networks allowed, and the same policy applied to every redirect it answered with, so a hop to a private address was followed without ALLOW_LOCAL_NETWORKS. The provider transport now takes a separate address policy for redirect hops (`redirectAllowLocalNetworks`): each hop is validated and pinned on a dispatcher for that policy. A managed MinerU Cloud root keeps local access for itself while its hops use the operator policy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(providers): match server-configured providers by own keys only Provider ids come from requests and were looked up on plain config objects, so an id such as `constructor` or `toString` read an inherited property and counted as server-configured (and its key, base URL and models resolved from that property). All per-provider config lookups now go through one helper that accepts only the section's own keys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(audio): let server-configured TTS/ASR endpoints reach local networks Server-configured image/video, MinerU and LLM providers on a local network work without ALLOW_LOCAL_NETWORKS, but a server-configured TTS/ASR or voice-registration endpoint given as a loopback or private IP literal (for example a VoxCPM server at 127.0.0.1:8000) was refused unless the opt-in was set. The routes and the server-side narration, classroom TTS and voice-clone paths now mark server-configured providers as `managed`; their endpoint requests run with local networks allowed, still pinned, with cloud metadata and reserved ranges refused, and redirect hops held to the operator policy. Client-supplied endpoints keep the strict public policy, and an unmanaged provider's catalog default keeps the operator policy. Provider-returned result URLs are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): server-configured TTS/ASR local endpoints and config id lookups Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-28 18:01:22 +08:00
小魔andwyuc	a41bbac239	feat(llm): retry once on a fallback model for retryable failures (#1614) ... * feat: configurable model escalation scheduler ## Motivation Course generation can stall when the model configured for a heavy stage (e.g. `scene-content:interactive` / `scene-content:pbl`) is repeatedly rate-limited, times out, or returns empty output. Today the operator must manually switch models in `MODEL_ROUTES` and re-trigger the page; long-running generations die on a single bad call. This PR makes the fallback automatic and observable: an optional per-stage **escalation policy** that re-runs the failed scene once with an explicitly chosen upgrade model, and logs every decision to a durable ledger viewable from a new Settings section. ## Design - **Disabled by default, zero behavior change.** No `data/model-schedule.json` → the engine behaves exactly as before. The config is hot-read per call (250 ms cache), so saving from the panel takes effect without a server restart. - **Explicit escalation only.** The upgrade model is resolved through the existing `resolveModel` pipeline **outside** `MODEL_ROUTES` (unrouted model string wins), reusing the full provider/model parsing (`openai:...`, `qwen:...`). - **Trigger semantics.** `onTimeout` requires a timeout error; `onRetryableError` accepts any retryable generation error (`isRetryableGenerationError`). Content-safety style rejections are **not** retryable and **never** escalate — the safety boundary is preserved. - **Budget guard.** Optional `budget.dailyEscalationCap` stops escalations for the day once exhausted; per-stage `max` limits retries. - **Ledger.** Every escalation appends one line to `data/schedule-events.jsonl` (stage, scene, base → used, error class, reason); the Settings panel shows the last 20. ## Screenshots Settings > Model Scheduling (config present):  Ledger after an escalation (example entry):  ## Verification - `pnpm lint` — 0 errors - `pnpm check` (prettier) / `pnpm check:i18n-keys` — pass (12 locales) - `pnpm build` — Next.js production build passes - Engine logic script — 8/8 assertions (disabled default, policy parsing, budget guard, ledger I/O) - API A/B flow — GET null without config; PUT template → served immediately (hot reload); DELETE → back to null ## Notes - The config schema is minimal free-form JSON (optional `models` / `budget` / `escalation` / `strictMode`); a stricter TS schema can follow in a later PR. - `strictMode` is reserved (not enforced yet). - The 3-tier preset template lives in the panel's "Load 3-tier preset template" button; no example file is shipped. * feat: configurable model escalation scheduler ## Motivation Course generation can stall when the model configured for a heavy stage (e.g. `scene-content:interactive` / `scene-content:pbl`) is repeatedly rate-limited, times out, or returns empty output. Today the operator must manually switch models in `MODEL_ROUTES` and re-trigger the page; long-running generations die on a single bad call. This PR makes the fallback automatic and observable: an optional per-stage **escalation policy** that re-runs the failed scene once with an explicitly chosen upgrade model, and logs every decision to a durable ledger viewable from a new Settings section. ## Design - **Disabled by default, zero behavior change.** No `data/model-schedule.json` → the engine behaves exactly as before. The config is hot-read per call (250 ms cache), so saving from the panel takes effect without a server restart. - **Explicit escalation only.** The upgrade model is resolved through the existing `resolveModel` pipeline **outside** `MODEL_ROUTES` (unrouted model string wins), reusing the full provider/model parsing (`openai:...`, `qwen:...`). - **Trigger semantics.** `onTimeout` requires a timeout error; `onRetryableError` accepts any retryable generation error (`isRetryableGenerationError`). Content-safety style rejections are **not** retryable and **never** escalate — the safety boundary is preserved. - **Budget guard.** Optional `budget.dailyEscalationCap` stops escalations for the day once exhausted; per-stage `max` limits retries. - **Ledger.** Every escalation appends one line to `data/schedule-events.jsonl` (stage, scene, base → used, error class, reason); the Settings panel shows the last 20. ## Screenshots Settings > Model Scheduling (config present):  Ledger after an escalation (example entry):  ## Verification - `pnpm lint` — 0 errors - `pnpm check` (prettier) / `pnpm check:i18n-keys` — pass (12 locales) - `pnpm build` — Next.js production build passes - Engine logic script — 8/8 assertions (disabled default, policy parsing, budget guard, ledger I/O) - API A/B flow — GET null without config; PUT template → served immediately (hot reload); DELETE → back to null ## Notes - The config schema is minimal free-form JSON (optional `models` / `budget` / `escalation` / `strictMode`); a stricter TS schema can follow in a later PR. - `strictMode` is reserved (not enforced yet). - The 3-tier preset template lives in the panel's "Load 3-tier preset template" button; no example file is shipped. * Delete omai-upload-tree directory * fix: ensure trailing newline in locale files fix: ensure trailing newline in locale files * chore: english comments for upstream review chore: english comments for upstream review * Delete components/settings/model-schedule-settings.tsx * Delete lib/server/model-schedule.ts * Delete app/api/model-schedule/route.ts * Delete assets/model-schedule/ledger-with-entry.png * Delete assets/model-schedule/settings-panel.png * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Update .env.example * Enhance access control comments in .env.example Added additional context and warnings for access control configuration. * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * Add files via upload * run ci * Add files via upload * Add files via upload * Add files via upload * Supplement. env.example Supplement. env.example * Add files via upload * Update .env.example * Update fallback notes in .env.example Clarified fallback behavior in callLLM layer documentation. * Update llm.ts * Update llm-fallback.test.ts * fix(llm): round-4 fallback hardening fix(llm): round-4 fallback hardening — server-managed gate, content-filter refusal, APICallError unwrap stop, outlines fullStream errors * fix(llm): rebase round-4 onto main v1.1.1 fix(llm): rebase round-4 onto main v1.1.1 * fix(llm): round4c-prettier formatting+retry test fix(llm): round4c-prettier formatting+retry test * test(llm): round-4d-align outline-stream test(llm): align outline-stream and title-generator tests with round-4 call signatures --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-28 17:24:55 +08:00
wyucandClaude Opus 5.5	44ad72beba	fix(persistence): serve the course library without the agent runtime; exit on invalid boot config ... * fix(persistence): serve the course library without the agent runtime The course library and folder routes (/api/stages/** and /api/folders/**) gated on isAgentRuntimeConfigured(), so a deployment with server persistence (DATABASE_URL set, client built with NEXT_PUBLIC_PERSISTENCE=1) but the agent runtime off answered 404 there: the home library showed an empty, "persistence unavailable" list and folder creation failed, although imports persisted. These routes only need the persistence provider and the owner-bound document store, so they now gate on isServerPersistenceConfigured(). Agent features (/api/agent/**, /api/skills/**, /api/materials/**) keep the runtime gate, and without a DATABASE_URL every route still answers 404. GET /api/agent/runtime also reports `persistence`; `enabled` and `runtimeEnabled` are unchanged and the workbench entry still keys on `enabled`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(boot): exit the server when boot validation refuses the configuration Next.js logs a throw from the instrumentation register() hook as "Failed to prepare server" but keeps listening and answers every request with 500, so a refused configuration (malformed asset quota, pending TTL or owner lock waits, PERSISTENCE_SHARED_OWNER_ID without ACCESS_CODE, the removed OWNER_AUTHENTICATOR / TRUSTED_PROXY_* variables, invalid owner auth or host hook registrations) left a process that looked alive and served nothing. The fatal validations now run in one validateBootConfiguration() step. When it throws, register() calls exitOnInvalidBootConfiguration(), which prints one "[boot] Invalid server configuration" line carrying the original message, waits for stderr to flush, and exits with code 1. The wrapper lives in its own module so tests stub it (or process.exit) and still assert the thrown error. register() already returns before any validation on the Edge runtime, and warnings (unset ACCESS_CODE, model-routing checks) never exit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: document library gating and exit on invalid boot configuration Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(boot): tell refused configuration apart from other startup failures Every throw from the boot validation step was printed as "Invalid server configuration" with its message only, so a startup failure that has nothing to do with settings (a module missing from a standalone build, a bug in startup code, a host registration call that throws) sent operators looking for a bad environment variable. Each fatal configuration check now runs through runConfigurationCheck(), which marks what it throws as an InvalidBootConfigurationError with the same message. exitOnBootFailure() (renamed from exitOnInvalidBootConfiguration) keeps the one-line message for those, and prints anything else as "[boot] Server startup failed" with its stack and cause. Both exit with code 1. The README (EN and zh) and CHANGELOG now list exactly the settings the boot validation refuses, adding OWNER_CLAIM_TRIGGER, the ASSET_S3_BUCKET conflict with a registered byte store and the sharedTeamAuthMethod() placement rule. Also fixes a stray comment marker in feature-flags.ts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-28 17:03:36 +08:00
3 people	ba7d707b0f	fix(media): route classroom media downloads through strict provider transport (#1692) ... * fix(media): route classroom media downloads through strict provider transport * style: apply prettier to classroom media transport changes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-28 14:45:42 +08:00
wyucandClaude Opus 5.5	a686481e73	feat(auth): owner identity seam — composable auth methods, owner-keyed persistence, host hooks, claims ... * ci: run CI for the owner identity seam integration branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(auth): owner identity seam, phase 1 — pluggable authenticator with today's behavior * feat(auth): add the owner identity seam with built-in authenticators Introduce lib/server/identity/: an OwnerAuthenticator contract (SubjectKind, OwnerPrincipal, AuthOutcome), a single-shot, boot-validated configureOwnerAuthenticator() registry, and the anonymousCookie and sharedTeam built-ins that reproduce today's owner resolution exactly (same anonymous_id cookie, owner id strings, statuses and env semantics). Every owner-scoped route handler and the workspace Server Action now resolve their owner through one memoized per-request resolution instead of the three previous copies (agent-runtime/owner.ts, with-owner.ts and the Server Action re-implementation). Publish and unpublish decide from the principal's course:publish role instead of an anon: id prefix. An invalid credential from an authenticator is answered with 401 on every surface and never falls back to an anonymous owner. A source scan keeps the anonymous owner cookie and anon: prefix checks out of code outside the identity module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(auth): describe the owner identity seam and host registration Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(auth): tighten the identity boundary guard and Server Action cookies - Scan workspace package sources too, flag imports of the concrete built-in authenticators outside lib/server/identity, and police the slice/substring/indexOf/includes spellings of an anon: id-shape check. Stop re-exporting the built-ins from the host-facing index. - Refuse setCookies returned by authenticateFromContext, as the authenticate() fallback already did, and document that it must write cookies itself through next/headers. - Let the route-test owner stub carry explicit kind/roles, and cover the 403 forbidden branch of publish and unpublish for a signed-in owner without course:publish. - README: registration conflicts fail the boot; an invalid principal is rejected per request with a 500. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(auth): owner identity seam, phase 2 — runtime and assets keyed by the owner * feat(auth): owner identity seam, phase 2 — runtime and assets keyed by the owner Runtime sessions and assets on /api/persistence now take their identity from the same memoized owner resolution as documents, instead of a client-chosen x-learner-key behind a development bearer token that shipped in the public bundle. - Runtime: the storage handler's learner key is principal.ownerId. A path or body naming another learner answers 403 FORBIDDEN_LEARNER and another learner's session answers 404, whatever x-learner-key says; the header is no longer read. The browser learns its key from the new GET /api/persistence/learner-key and sends no credential of its own. The Pi whiteboard routes derive the same key from the owner. PERSISTENCE_DEV_TOKEN, NEXT_PUBLIC_PERSISTENCE_TOKEN and PERSISTENCE_ALLOW_INSECURE_DEV_AUTH are gone from the server path and the client, and server-auth.ts is deleted. Learner merge stays refused. - Assets: allocations land in a per-owner partition (owner:<ownerId>), so quota is per owner and replace/delete reach only the owner's entries. Reads stay capability-by-id where a course viewer needs them: another owner's committed entry is readable while a live course references it. Legacy entries in the old shared partition stay readable by id, and may be replaced or deleted only by an owner who owns every course referencing them. Server-generated media is stored under the run owner's partition; asset-id extraction and vision image resolution use the same rule. - Tombstones: runtime of a deleted course reads as absent (404, empty lists) and takes no new sessions or records, through a guard in the app's runtime composition; no package schema change. The identity boundary scan now also fails on any source that reads x-learner-key or the retired development token variables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(persistence): describe owner-keyed runtime and assets, drop the dev token Remove the development persistence token from the server-persistence recipe, .env.example, the Docker build arguments and the deployment docs; describe runtime learner keys, per-owner asset partitions and quota, the legacy shared-partition rule and the tombstone behavior; and record the breaking change for existing server-persistence runtime data under Unreleased. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(storage): scope asset references to principals, typed stage refusal, uniform create collision @openmaic/storage 0.32.0. - PgDocumentStore takes assetReferencePrincipals: with reference tracking on, a write references and commits only entries held by the listed principals. An id naming another principal's entry records no row and changes no lifecycle column, like an unknown id, and the write is never refused. Without the option any held entry is referenced, as before. AssetCollector takes a matching per-document-owner function so the one-time backfill scopes each document the same way. This keeps a document naming a leaked id from committing, exposing or pinning another owner's allocation. - RuntimeStageNotFoundError: a store may refuse createSession for a stage its host considers absent; the runtime handler answers 404 STAGE_NOT_FOUND, recognizing the error by class or by code. - A session create over a taken id answers 409 SESSION_ALREADY_EXISTS whoever holds it. It used to answer 403 for another learner's session and 409 for one's own, which told a caller whose session an id was. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): close owner-seam boundary gaps in references, tombstones and legacy assets - Document writes reference and commit only the writer's own asset entries and legacy shared ones (owner-bound document store and the collector backfill), so another owner's course naming a pending id can no longer commit, expose or pin it. - The foreign-read rule additionally requires the referencing live course to belong to the entry's owner, as defense in depth for reference rows that did not come from a scoped write. - The tombstone refusal for new runtime sessions moved from a raw path comparison in the route into the guarded store's createSession (RuntimeStageNotFoundError, 404 STAGE_NOT_FOUND), so no encoded spelling of the path routes around it. - Replacing or deleting a legacy shared entry checks reference ownership and mutates in one transaction under the entry row lock, which a concurrent reference insert must wait for. - Tests build the uncommitted-but-referenced, tombstoned-with-rows and foreign-reference states directly so each half of the read rule is observable, and a PostgreSQL suite holds a competing reference open across a legacy delete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(persistence): upgrade note for dev-token-gated deployments, owner-scoped references Tell operators whose only access gate was the development token to put the deployment behind an access code or gateway, register an owner authenticator, or turn server persistence off before upgrading, and describe that a course references only its owner's media. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(storage): per-owner reference principals and a typed taken-id conflict - PgDocumentStore's assetReferencePrincipals is now a function of the store's bound owner, evaluated per write, instead of a fixed list. A store re-bound with forOwner therefore scopes to the new owner rather than carrying the previous owner's principals onto its writes. It has the same shape as AssetCollector's backfill option, so one function serves both. - PgRuntimeStore and BrowserRuntimeStore raise RuntimeSessionExistsError for a taken session id, and the runtime handler classifies it as 409 SESSION_ALREADY_EXISTS even when its re-read cannot see the holder (for example a session a host-side wrapper hides). Recognized by class or by code. Still the unreleased 0.32.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): guard the Pi whiteboard runtime and answer hidden collisions with 409 - Every server-side runtime writer now uses the tombstone-guarded store: the Pi native whiteboard service gets it through the same helper as /api/persistence/runtime/*, so a deleted course takes no new whiteboard runtime either. - Creating a session whose id is held by a session a tombstone hides answers the uniform 409 SESSION_ALREADY_EXISTS instead of a 500, covered on PGlite and on real PostgreSQL. - The owner-bound document store passes its reference-principal function, so the scoping follows the bound owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(auth): owner identity seam, phase 3 — accounts through an identity gateway (trustedProxyHeader) * feat(auth): owner identity seam, phase 3 — trusted-proxy-header authenticator Add the trustedProxyHeader built-in: real accounts through an identity gateway (oauth2-proxy, Authelia, a Keycloak-based proxy, an institutional reverse proxy) that signs the user in and forwards the verified user in a request header. OpenMAIC never handles passwords or tokens. - Selected with OWNER_AUTHENTICATOR=trusted-proxy and validated in instrumentation register(). TRUSTED_PROXY_SECRET (32-1024 printable ASCII characters) is required; header names default to x-openmaic-proxy-secret and x-forwarded-user, and an optional groups header with TRUSTED_PROXY_ADMIN_GROUPS grants admin. An unknown selector, a TRUSTED_PROXY_* variable without the selector, a shared owner id or a host-registered authenticator alongside it, and malformed, reserved or repeated header names fail the boot. - Trust boundary: Next.js exposes no TCP peer address to route handlers, middleware or Server Actions, and fills x-forwarded-for from the socket only when the client sent none, so no address allowlist is offered. The gateway secret is compared in constant time (hashed, timingSafeEqual). - Per request: a missing or wrong secret, a missing or blank user, a user header with a comma (duplicate lines arrive comma-joined), or a user whose proxy:<user> id fails the owner id guard is 401 INVALID_CREDENTIAL, never an anonymous owner. The guard failure is a 401 rather than a 500 because the value comes with the request. The user is trimmed and keeps its case. Every gateway user holds course:publish; kind user, assurance verified, channel proxy; no cookie. Route handlers and Server Actions share one code path. - The unset-ACCESS_CODE boot warning is skipped in this mode; the gateway is the access gate and ACCESS_CODE stays independent. - The identity boundary scan now also fails on gateway identity headers (x-forwarded-user/-groups/-email, x-auth-request-*, remote-user, the secret header) and TRUSTED_PROXY_* read outside lib/server/identity. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(auth): document accounts through an identity gateway Describe the trustedProxyHeader built-in in the README owner identity section (and its Chinese counterpart): configuration, request semantics, the trust boundary and why it is a shared secret, the interaction with ACCESS_CODE, boot-time validation, and an oauth2-proxy example that injects the user, groups and secret headers. Add the .env.example block, a SECURITY.md note and an Unreleased changelog entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(auth): harden the trusted-proxy authenticator after review - Reserve the header names HTTP, Next.js and forwarding proxies set themselves: configuring the user, groups or secret header to rsc, next-action, forwarded, x-forwarded-for/-host/-proto/-port, x-real-ip, or anything starting with x-middleware-, x-invoke-, x-nextjs- or next-router- now fails the boot. - Log a one-time boot warning when TRUSTED_PROXY_ADMIN_GROUPS is set: the admin role is granted from the groups header, which is trusted on the strength of the shared secret alone, so the gateway must overwrite or strip it. - POST /api/chat/pi resolves the owner up front and answers 401 when the authenticator rejects the credential, like every other owner-resolving route, instead of continuing without an owner. The anonymous default always resolves, so it is unaffected. - The identity boundary scan also covers the Authentik, Azure App Service authentication, WebAuth, Cloudflare Access, Google IAP and AWS ALB OIDC header families. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(auth): admin-groups warning and a corrected oauth2-proxy example - Warn next to TRUSTED_PROXY_ADMIN_GROUPS (README, README-zh, .env.example) that the groups header is trusted on the strength of the shared secret alone, and list the reserved header names. - The oauth2-proxy example now states it targets v7.14+ (nested claimSource/secretSource), uses the canonical bindAddress, sets preserveRequestValue: false on all three headers and insecureSkipNonce: false, forwards the subject with `claim: user`, and notes that the secret variable holds the literal secret. It adds the legacy-option migration caveat, a warning not to exempt app routes via skip-auth or trusted-ip options, and notes that the IdP must release the groups claim and group names must not contain commas. - Changelog: the new boot checks and the /api/chat/pi 401. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(persistence): owner identity seam, phase 4 — host extension hooks * feat(persistence): owner identity seam, phase 4 — host extension hooks Let a host add product behavior at four points without forking a route, registered once from instrumentation.ts register() like the owner authenticator and sealed on first use. Nothing changes when no hook is registered. - configurePersistenceHooks({ name, authorizeCreate, onCreate, library, beforeAssetAllocate }): - authorizeCreate / onCreate run once per created course, inside the create transaction, in the transaction that inserts the course's ownership row. Saves of a course the owner already holds, and a create that loses a race to a concurrent create of the same id, are updates and run no create hook. A refusal answers 403 CREATE_REFUSED; a throw rolls the course, its ownership row and the host's writes back together. The hooks receive the request principal, or only the owner id for a background agent run. - library chooses which stage ids GET /api/stages lists; the route builds the usual summaries in the provider's order and drops every id the read path would refuse (deleted or unclaimed). folderId is shown only on the principal's own courses. - beforeAssetAllocate runs inside the storage handler's asset authorization step for POST /assets, after the owner is resolved and before the body is read; a returned Response is sent instead, with nothing stored and no quota counted. - configureAssetByteStore({ name, create, signsReadUrls }) replaces the ASSET_S3_BUCKET switch for both the persistence provider and the asset collector. A host store must keep bytes outside the registry database. Under ASSET_BYTE_EGRESS=redirect a store that does not declare signsReadUrls stops the server at boot; the built-in layers keep their behavior. - @openmaic/storage 0.33.0: DocumentWriteRefusedError lets a store refuse a write as policy; the document HTTP handler answers 403 with its code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): harden the host extension hooks after review Correctness: - Registration reads every hook once and stores it bound to the object the host passed, so hooks defined on a class prototype or as non-enumerable properties are kept. A spread copy dropped them silently, including the authorizeCreate and beforeAssetAllocate gates. Same for configureAssetByteStore. The unknown-key check now applies to plain objects only, since class instances carry their own state fields. - The owner-bound document store carries each call's operation in its own async context (AsyncLocalStorage) instead of a field on the instance. One store is shared by every tool of an agent run; a concurrent call could overwrite the field before the transaction read it, so a create was gated as a read (no ownership row, no create hooks) or claimed another stage. create_stage also runs sequentially within a tool batch. Hardening: - isDocumentWriteRefusedError recognizes a refusal from another copy of the storage package by name and shape (not by code alone). The document handler and POST /api/stages use it. - beforeAssetAllocate also gates PUT /assets/{id}/content. The request carries operation ('create' | 'replace') and the decoded assetId from the handler's own routing. - Agent runs: a refusal on a background write carries a fixed message, and create_stage answers it with a stable tool result. The host's message never reaches the model. - A byte store declaring signsReadUrls without a signer degrades to direct bytes with a warning instead of failing reads, and the collector no longer requires signing. - DocumentActor is discriminated by source ('request' with the principal, 'background' without one). - Library ids the read path cannot address are dropped, and a provider answer is capped at 5000 ids. - The boot-time ownership backfill logs how many courses it adopted without hooks. - The server persistence provider no longer exposes an unscoped document store. - The writesOutsideRegistryDatabase trust boundary and the scope of upload admission (HTTP uploads, not server-generated media) are documented. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): refuse misspelled hooks on host classes too A class instance skipped the unknown-key check, so a misspelled hook such as authorizeCreat registered cleanly and never ran, leaving creation ungated. Both configurePersistenceHooks and configureAssetByteStore now reject any function-valued property of a class instance (own or on its prototype chain up to Object.prototype, constructor excluded) that is not a known key, and any unknown key of a plain object. The error names the key and suggests the known key within edit distance 2. Host classes keep their helpers private (#helper) or register a plain object; the README says so. Also: - isDocumentWriteRefusedError requires a string message from a cross-copy candidate, and documents that the name is the effective discriminator. - CHANGELOG: ServerPersistenceProvider no longer exposes an unscoped documentStore; use the owner-bound store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(persistence): owner identity seam, phase 5 — tenancy lives in stage_meta only * feat(persistence): owner identity seam, phase 5 — tenancy lives in stage_meta only Course ownership was recorded twice: in stage_meta, which already answers every access decision, and in document_stages.owner_id, which the storage package scoped by. Two tenancy records double what a later claim has to re-key, and a host that keeps ownership in its own table had to rewrite the package's SQL to strip the column. @openmaic/storage 0.34.0: - PgDocumentStore never reads or writes document_stages.owner_id. An owner-bound store scopes listings, writes, deletes, the freshness manifest and folder membership through the host's ownership relation, named with the new documentOwnership option ({ table, stageIdColumn, ownerIdColumn, tombstoneColumn, claimOnCreate }, or false for no document scoping). Identifiers are validated, never quoted into SQL from arbitrary input. - forOwner / ownerId without documentOwnership throws at construction, so a direct consumer cannot upgrade into a store that silently scopes nothing. An unbound store is tenant-agnostic. - claimOnCreate inserts the ownership row in the create transaction; a concurrent create of the same id by another owner rolls back. - folders: false lets a host whose document_stages has no folder_id use the store; folder ids are unique per owner, so membership goes through the ownership relation too. - AssetCollector reads each document's owner from documentOwnership for assetReferencePrincipals, and refuses the function without it. - Schema: fresh installs get no owner_id column and none of its indexes; an existing column is kept for one release, made nullable with its default dropped (each ALTER guarded by a catalog check, so a replay takes no lock), and its two indexes are dropped. document_stages_folder_idx replaces the owner/folder index. Application: - The owner-bound store binds the package to stage_meta (STAGE_META_OWNERSHIP) and lists through it in one query; the asset collector schedule passes the same relation. - The boot backfill copies column-only owners into stage_meta before any store is built, only when the legacy column exists, idempotently, and logs how many it adopted and how many disagree (stage_meta stands). Tests cover a fresh install and an upgrade from the previous schema on PGlite and PostgreSQL 16, a host table with neither ownership nor folder columns, folder isolation across owners sharing a folder id, and the concurrent-create race. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): serialize schema bootstrap; guard unscoped owner-bound stores Review follow-ups for the tenancy change. - Concurrent starts: two instances starting against one database raced on the schema DDL (a duplicate pg_class / pg_type entry, or "tuple concurrently updated"), failing one instance's first request. Every schema bootstrap the application runs -- the persistence provider's sequence (runtime, documents, stage_meta and its ownership backfill, owner materials, assets), the agent-session, session-material and user-skill stores, and the asset collector's -- now runs under one PostgreSQL advisory lock held on a dedicated connection and released in finally. It lives in the application because what needs serializing is the whole sequence, including tables the package does not own. A PostgreSQL test boots eight instances at once, four rounds each, on a fresh and on an upgraded database; without the lock it fails every run. - documentOwnership: false on an owner-bound PgDocumentStore now also requires allowCrossOwnerDocumentAccess: true, so binding an owner for folders or asset principals cannot silently expose every owner's documents. - Documented that an ownership relation must cascade with the document rows (a leftover row keeps the id reserved for its owner, which is also how a host keeps retired ids from being reused), with tests for both. Leftover rows are deliberately not made claimable: a host cannot be told apart from one that keeps them as retirement markers. - Upgrade notes: the one-time table locks on the first start, and the out-of-band CREATE INDEX CONCURRENTLY / DROP INDEX CONCURRENTLY path that leaves the start with nothing to build or drop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(persistence): leave the schema bootstrap lock out of the held-lock check The create-hooks suite asserts that its host lock is released with the create transaction by counting granted advisory locks database-wide. A suite booting a provider against the same database at that moment now holds the schema bootstrap lock, which is not the lock under test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(auth): owner identity seam, phase 6 — claiming anonymous work on sign-in * feat(storage): owner merge primitives for claiming anonymous work (0.35.0) Add the per-store pieces a host needs to move one owner's data to another inside a single transaction of its own: - reassignDocumentFolders(tx, { fromOwnerId, toOwnerId, documentOwnership }) moves folders and filing: a same-named folder merges into the target's, a colliding id is renumbered, and filing is rewritten in one statement from the old ids. - PgAssetStore.reassignPrincipal(fromKey, toKey) moves a principal's entries under both principals' write locks, in key order; quota is not re-checked. - PgUserSkillStore.mergeOwner(from, to) moves skills, renaming a live handle the target already uses with the first free numeric suffix. - PgUserSkillStore takes a resolveFinalOwner hook, run as the create transaction's first statement, like the agent-session store's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(auth): owner identity seam, phase 6 — claiming anonymous work on sign-in A visitor who works anonymously and then signs in can move that work to their account in one transaction, and the anonymous id is retired afterwards. - Principal: pendingClaim { fromOwnerId, assurance }. The trusted-proxy built-in sets it when a gateway request also carries a valid anonymous cookie; the anonymous and shared-team built-ins never do. Authenticators may implement describeStoredOwner, canonicalize and clearPendingClaim; principalFromStoredOwner describes a stored id for work without a request. - Claims: claimOwner / claimPendingOwner and registerClaimParticipant (sealed on first claim). One transaction, both owners' identity locks taken exclusively in key order, participants in a fixed order (folders, courses, materials, agent sessions, skills, runtime, assets), recorded in the new owner_merges table. Idempotent per pair; refuses a non-anonymous source, an anonymous target, a source already claimed elsewhere, and chains. Same-named folders merge, colliding folder ids are renumbered, colliding skill handles get a suffix, and quotas are not applied to what moves. - Identity lock: every owner write (documents, folders, asset allocations, material registration, agent sessions, skills) takes the owner's advisory lock in shared mode first, so a write racing a claim is moved or refused, never orphaned or deadlocked. - Retired ids: request writes are refused with 403 OWNER_RETIRED; agent runs and media generation that started before the claim follow the id to the account (forwarding document store, canonicalized stage probes, forwarded generated assets and skills). - Trigger: POST /api/identity/claim (same-origin JSON only; clears the anonymous cookie), or OWNER_CLAIM_TRIGGER=auto on the first request carrying a pending claim. The runtime contract's learner merge now performs the same claim, allowed only for the request's own pending claim. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(persistence): count only the suite's own advisory locks after concurrent creates The check that a host's transaction-scoped lock is released counted every granted advisory lock in the database. Suites that run against the same database at the same time hold advisory locks of their own (a schema bootstrap, the owner identity locks a claim takes), which made it fail depending on timing. Count only locks held by this suite's connections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(auth): use a neutral example table for host claim participants Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(storage): owner merge fixes and store policy errors over HTTP (review round 1) - PgUserSkillStore.mergeOwner reserves every handle either owner holds before renaming: a claimed skill whose handle the target uses could be renamed into a handle another claimed skill still held, which violated the per-owner unique index and made that merge fail every time. - PgUserSkillStore.list orders ties by id. - PgRuntimeStore takes resolveFinalLearner(tx, learnerKey), run as the create transaction's first statement, so a host can fence session creates against its owner merges; reassignLearner(from, to) re-keys sessions without re-validating them, so one session a newer version wrote cannot block a merge. - Every HTTP handler (runtime, documents, assets) answers a DocumentWriteRefusedError as 403 with its code, and the new StorageBusyError (code, message, retryAfterSeconds) as 503 with Retry-After. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(auth): claims review round 1 — fenced runtime creates, bounded locks, retired-cookie recovery - Runtime session creates take the owner's identity lock as the create transaction's first statement and refuse a retired owner, closing the window in which a create racing a claim was left under the retired id. - Lock waits are bounded: a claim waits OWNER_CLAIM_LOCK_WAIT_MS (default 5 s) for its identity locks, a write OWNER_WRITE_LOCK_WAIT_MS (default 30 s); running out, or losing a deadlock or serialization race, answers 503 OWNER_BUSY with Retry-After on the claim routes and on every fenced write. - Every 403 OWNER_RETIRED carries the Set-Cookie values that drop the retired anonymous credential (the anonymous built-in now implements clearPendingClaim), so a browser whose claim response was lost recovers. Writes by id to rows that moved (skill delete, session messages) answer OWNER_RETIRED instead of not-found or a bare 403. - OWNER_CLAIM_TRIGGER=auto no longer claims ahead of the explicit claim routes, which now report their own claim instead of a false refusal. - The owner-events stream tells a retired owner only that it moved, never the claiming account's id. - Creates of one course id take turns (a per-id advisory lock before the ownership probe), so a concurrent second create saves as an update instead of failing as reserved-document. - A claim's source must be described as anonymous by the authenticator; the write fences skip the retirement lookup for every other owner. The host canonicalize hook is removed: forwarding is core's owner_merges alone. - The folders participant locks the source's stage_meta rows first; runtime sessions are re-keyed without re-validation; the owner asset store fences every allocation and refuses to allocate without transactions; background skill reads and patches follow a mid-run claim; the forwarding document store forwards methods only. - Docs: when a pending claim can arise with the shipped authenticators, the anonymous cookie as a bearer credential, bounded waits, the exact scope of OWNER_RETIRED, and claims narrowed to what the tests show. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(auth): claims review round 2 — merge-record guard, boot-checked lock waits, stream credential drop - owner_merges records claims of anonymous owners only, since the write fences enforce retirement only for ids the authenticator describes as anonymous. Reading a row that retires any other owner now fails loudly instead of being followed but not enforced. The comment that pointed hosts at claimOwner for their own account merges is corrected: a host merging two signed-in accounts moves the rows itself and refuses the merged-away account in its authenticator. describeStoredOwner must classify ids stably. - OWNER_WRITE_LOCK_WAIT_MS and OWNER_CLAIM_LOCK_WAIT_MS are validated at boot with the same parser the lock paths use, so a malformed value stops the server instead of failing every write. - The owner-events stream answers a connect by an already retired identity with one owner_moved event and the Set-Cookie values that drop the retired credential, so a stale tab's reconnect loop ends after one round trip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * chore(auth): final polish before landing the owner identity seam * chore(auth): final polish before landing the owner identity seam - Pi routes refuse a rejected credential with the seam's INVALID_CREDENTIAL body. - Drop the unused LegacyAssetMutations alias and resolveLockWaitMs re-export. - Scope the document_stages.owner_id deprecation wording: the store no longer reads or writes it; the boot backfill reads it and a claim mirrors ownership into it for rollback. - Document OWNER_CLAIM_TRIGGER and the two lock-wait variables in .env.example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(storage): 0.35.1 for the README wording fix Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(identity): one identity entry point with composable auth methods * feat(identity): compose owner auth methods behind one entry point Owner resolution now asks an ordered list of owner auth methods instead of one configured authenticator. Each method answers `authenticated`, `not-applicable` (no credential of its kind) or `invalid` (its credential is present but bad). Core keeps the first `authenticated` answer, refuses any `invalid` answer with 401 INVALID_CREDENTIAL without asking later methods, and when no method applies falls back to the anonymous cookie owner, or answers 401 when the host turned the fallback off. Route handlers and Server Actions share the same order and per-request memoization. - configureOwnerAuthentication({ methods, anonymousFallback? }) replaces configureOwnerAuthenticator: single-shot, sealed on first use, and validated at registration and at boot. - anonymousCookie becomes the built-in fallback method; sharedTeam becomes a method, selected by PERSISTENCE_SHARED_OWNER_ID as before when nothing is registered, and kept beside host methods only when the host lists sharedTeamAuthMethod() last. The variable set beside a registration that leaves it out fails the boot. - Core attaches pendingClaim itself: a host method's non-anonymous principal on a request that also carries a valid anonymous cookie. A method may not set one. Claim cookie clearing and OWNER_RETIRED recovery go through the anonymous method's clearCredential. - principalFromStoredOwner consults the anonymous method first, then the configured methods' describeStoredOwner in order. - Remove the built-in trusted-proxy header authenticator with OWNER_AUTHENTICATOR and every TRUSTED_PROXY_* variable. The boundary test now also keeps the resolution internals inside lib/server/identity/ and asserts no source reads the removed variables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(identity): document owner auth methods and a gateway JWT recipe Rewrite the owner identity section around the ordered auth methods: the three answers and what resolution does with each, the anonymous fallback, registration and its boot checks, and the sharedTeam rules. Replace the removed built-in gateway authenticator's documentation with a short host-owned recipe that verifies a gateway-forwarded signed JWT (oauth2-proxy, Cloudflare Access, IAP) against the identity provider's keys with the jose library. Update the claim section for the core-owned claim candidate, and the Chinese README, SECURITY.md, CHANGELOG and the deployment docs in every locale to match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(identity): tighten retired-owner cookies, retired config and the boundary - On 403 OWNER_RETIRED, core now calls clearCredential only on the anonymous fallback and on methods that declare `issuesAnonymousOwners: true`, so an account method's session cookie is never cleared there. A clearCredential that throws is logged and skipped instead of turning the refusal into a 500. - Setting any retired OWNER_AUTHENTICATOR / TRUSTED_PROXY_* variable fails the boot with a pointer to the auth-methods docs, instead of silently serving every request anonymously. Only the names are matched, in one module the boundary test checks never reads a value. - Add lib/server/identity/host/ for host auth methods. The boundary test now scans core identity files for gateway identity headers too, polices reads of an incoming Authorization header everywhere but the host directory, and keeps resolution internals out of host code. - Document on OwnerAuthMethodResult that a present but unusable credential must answer `invalid`, never `not-applicable`, and that a method that cannot decide throws. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(identity): fix the gateway JWT recipe's error split and add notes - The recipe now answers `invalid` only for token failures (expired, claim validation, malformed JWS/JWT, bad signature, no or several matching keys, disallowed or unsupported algorithm) and rethrows everything else: jose reports a key endpoint that is down, answers non-200 or returns unparsable data as a generic JOSEError, which the previous version turned into a 401 for every user. - The recipe file goes in lib/server/identity/host/, with an accurate description of what the boundary test enforces. - Document invalid versus not-applicable for malformed credentials, issuesAnonymousOwners, the boot failure on retired variables, and that the claim candidate is an unsigned bearer cookie a sibling subdomain can shadow (serve on a registrable domain of its own; cookie hardening is deferred). Mirror the notes in the Chinese README, SECURITY.md and the changelog. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-28 11:54:26 +08:00
wyucandClaude Opus 5.5	44762d9db2	fix(pdf): route MinerU Cloud requests through the strict provider transport (#1688) ... The MinerU Cloud parser issued its API-root, presigned-upload and result-ZIP requests with the default fetch, so a URL carried in a provider response could direct a further request at an arbitrary destination and redirects were followed without re-validation. Route every MinerU Cloud request through the strict provider transport — the redirect-validating, DNS-pinning helper already used by the audio providers, now re-exported under the neutral `providerFetch` name: - The configured API root runs under the operator's address policy (the ALLOW_LOCAL_NETWORKS opt-in applies), preserving self-hosted and BYOK reachability while keeping per-hop redirect re-validation and DNS pinning. - The response-supplied presigned upload URL always uses the strict public policy with redirects rejected outright: a 3xx answer is a hard failure and the body is never forwarded to a redirect target. - The response-supplied result ZIP URL uses the strict public policy and requires HTTPS on every followed redirect hop, via a new opt-in `requireHttps` transport policy that defaults off so audio behavior is unchanged. - Address-policy refusals are terminal and are not retried; the retry wrapper rethrows the original UnsafeNetworkTargetError when one is present. Bound the untrusted response bodies: - JSON control-plane responses are capped at 8 MiB and the result ZIP at 256 MiB, enforced from content-length and while streaming. - Before extraction the result archive is limited to 10,000 entries and to a 512 MiB declared uncompressed total: a cheap preflight over header fields that can understate the real payload. While decompressing, every text entry is capped at 64 MiB, every image at 32 MiB, and a 512 MiB running total is enforced from bytes counted as they stream out of the decompressor, so extraction aborts as soon as a limit is crossed instead of buffering the whole entry and checking afterwards. This also surfaces the decompressor's own size-mismatch error for entries whose declared size does not match their payload. Also classify the deprecated IPv4-compatible IPv6 range (::/96, excluding the unspecified `::` and loopback `::1`) by its embedded IPv4 in the shared SSRF guard, so `[::7f00:1]` and `[::a9fe:a9fe]` are refused at both the URL layer and the connect-time pinned lookup. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-27 00:07:35 +08:00
PercyandPercy	16d6d09f58	feat(attribution): send X-APP-URL on TokenDance gateway requests (#1675) ... Co-authored-by: Percy <percy@PercydeMacBook-Pro.local>	2026-09-24 13:46:38 +08:00
4 people	a87c94c89d	feat(settings): restructure settings IA with course model config and token plan controls [AI-assisted] (#1644) ... * feat(generation): propagate per-stage model routes via x-model-routes header Course model config lets each pipeline stage pin its own model; send the user-level stage routes alongside the main model credentials so servers resolve overrides consistently. Server-side model-routes/resolve-model learn to merge them; scene renderers, the scene generator and the generation preview pass the header through. * feat(settings): token plan enrollment markers, plan toggle and store rework - Plans connect through an explicit tokenPlanEnrollments marker: a personal key on a shared provider (minimax/tokendance/doubao) is never mistaken for a connected plan, so startup reconciliation no longer hijacks its catalogue - New per-plan authorization toggle (tokenPlanDisabled): toggling cascades to the plan's per-modality provider enabled flags while skipping providers another usable plan still owns; re-enabling clears the seed fingerprint and re-seeds through reconciliation - Multi-plan priority: exclusive slots (main model, stage routes, per-modality selections) yield to plans earlier in TOKEN_PLAN_PRESETS order regardless of write order; picker ordering follows the same order - Reconciliation skips plans disabled by the toggle; disconnect restores the built-in provider's enabled flag and clears stale authorization markers - Stage routes are pruned when a provider becomes unusable; the homepage web-search toggle migrates into the settings store; server-provider init waits for persist hydration * feat(settings): five-section IA with course model config and provider panels - Settings dialog reorganized into Token Plan / Course Model Config / Model Services / Skills / System - New Course Model Config section: station cards along the generation pipeline with a per-station inspector for model overrides, mainline model plus fine-grained sub-stage routes, invalid selections kept as disabled rows instead of blank triggers - Model Services becomes a provider list + detail panel with a per-provider authorization toggle (disabled providers leave course model config); Qwen TTS voice-clone area redesigned (45:55 split, add-clone CTA) - Token Plan panel gains the Enable-this-plan switch mirroring the provider toggle, connected/off status copy, and priority-ordered model groups * refactor(generation): drop the media popover now that media providers live in settings Media provider/model selection moved into Model Services and Course Model Config; the generation-bar media popover (520 lines) goes away and the toolbar slims to the remaining toggles. * fix(ui): keep provider popovers inside the viewport Voice/config popovers now cap their height against --radix-popover-content-available-height and their width against the viewport, so search boxes no longer clip near screen edges. * feat(i18n): complete course-model stations and settings copy across 12 locales - Translate stations.* labels and descriptions for the 9 locales that fell back to English; convert zh-TW descriptions to Traditional Chinese - Add subStages.*, mainlineUnset, pickModel/pickProvider, optionInvalid and the token-plan toggle copy across all locales * test(audio): mock getStageRoutesHeaderValue in refused-narration test Companion mock to the x-model-routes plumbing commit. * style: apply prettier to files touched by the settings IA refactor * fix(settings): restore the homepage model pill and set-up CTA lost in the IA refactor The refactor dropped the toolbar's model picker and "Set up model" CTA, leaving the homepage without any model affordance (#580 dead-end) and breaking the picker stress tests (#1447), the managed-provider specs (#620) and the KV-persistence checks that anchor on the pill. - Re-add the pill on the shared ModelPicker component (aria-label "Provider / Model"), backed by the extracted useLLMPickerGroups hook so the toolbar and course model config share one filtered, priority-ordered provider list - Re-add the amber "Set up model" CTA when no usable provider exists; it opens settings at Model Services - Specs: navigate to Model Services in #620 and scope its catalog assertion to the dialog; drop the provider-tab step in #1447 (providers are group headings now) * fix(settings): write the full runtime stage set for classroom interaction The interaction station wrote only chat-adapter, while quiz grading and the PBL v2 runtime resolve quiz-grade / pbl-v2-runtime:*, so the override never reached them and calls fell back to the main model (review P0). - Add lib/config/station-stage-keys.ts as the single station -> runtime stage-keys contract; interaction now writes chat-adapter, quiz-grade, pbl-chat and pbl-v2-runtime together (PBL composite sub-stages inherit the base key via the existing colon fallback) - Override and restore-follow now write/clear the whole key group, thinking-config changes included - Regression tests: contract keys are LLM_STAGES members, no orphan runtime stages, the interaction override resolves for every runtime endpoint, and explicit sub-stage overrides keep precedence over inherited parents * fix(settings): enforce provider authorization across sync fallback and gates A disabled plan's provider could be resurrected on refresh: validateProvider correctly rejected it, but the auto-recover branch adopted fallback[0], whose order still included authorization-disabled entries; the homepage generate gate (isLLMProviderConfigured) also ignored the toggle (review P0-02). Excluded disabled entries from the fallback order (extracted as buildUsableFallbackOrder; image/video keep adopting server-configured entries whose enabled:false means not adopted yet) and taught isLLMProviderConfigured to reject disabled providers, so an empty selection is retained and submission stays blocked. * fix(token-plan): assign shared provider credentials to the top enabled plan Applying a plan overwrote shared provider slots (seedream) last-writer- wins, and disabling the owning plan left the slot enabled with the disabled plan's endpoint and key (review P0-03). Shared slots now always belong to the highest-priority enabled plan targeting them: apply yields credentials and seed catalogues to a higher-priority owner regardless of connect order, and disable/enable/remove transitions hand the slot back (restoreSharedProviderCredentials rewrites endpoint, key — taken from the owner's LLM slot — and catalogue of the new owner). * fix(token-plan): include the re-enabled plan in shared ownership and clean up on removal Two transitions still mis-owned shared slots after the previous fix (re-review of P0-03): re-enabling a higher-priority plan excluded itself from ownership resolution, so the slot stayed with the lower-priority plan; and removal skipped cleanup when another plan was merely enrolled but disabled, leaving the removed plan's endpoint, key and model catalogue active on the shared provider. - restoreSharedProviderCredentials takes excludeConcerned: disabling and removal exclude the concerned plan, re-enabling includes it, so the newly enabled plan reclaims its slot and a lower-priority enable does not steal one it does not own - removeTokenPlan skips shared-slot cleanup only for another USABLE plan; otherwise the slot is cleared and disabled instead of surviving behind an enrolled-but-disabled plan * fix(chat): honor per-stage user routes for classroom chat The classroom-interaction model override is sent by the client as the `x-model-routes` header, but the two multi-agent classroom chat routes resolved their `chat-adapter` stage from the request body without parsing it, so a per-stage override was silently ignored and the main model was used instead. Forward the parsed user routes into `resolveModel` for both /api/chat and /api/chat/pi (credentials still come from the body, not x-* headers), and attach the header from the client fetches. Operator MODEL_ROUTES still wins server-side. Regression tests cover the user-route precedence, the operator override, the x-model-routes parser, and both client fetches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(settings): drop the pro-mode override the server never honors The Course Model Config UI offered a "Pro Mode" station that wrote a user-level route for `maic-agent-driver`, but that stage is resolved only from the operator's MODEL_ROUTES (it also needs an explicit api dialect and contextWindow), so the override could never take effect. Remove the station from the map and its pipeline card, stop seeding the route from the TokenDance preset, and drop the now-unused label/i18n keys in every locale. The settings store's rehydrate sanitizer also drops any previously persisted `maic-agent-driver` user route, so upgraded installs do not keep a dead entry. The "no orphan runtime stage" contract test now expresses `maic-agent-driver` as an explicit operator-only exception. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(settings): neutralize token-plan comments Rewrite comments that referenced an external deployment or implementation into neutral descriptions of this file's own behavior and design intent. No behavior change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Percy <percy@PercydeMacBook-Pro.local> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-23 18:11:21 +08:00
Frank Zhuandwyuc	4b537dca96	fix(tts): pace classroom TTS and classify MiniMax RPM limits (#1650) ... * fix(tts): pace classroom TTS and classify MiniMax RPM limits MiniMax reports RPM limits as HTTP 200 with base_resp.status_code 1002. Classify that code as TTSRateLimitError, pace classroom TTS with TTS_MIN_INTERVAL_MS (default 1000ms) and double the gap up to 15000ms on each rate limit, and store ttsCoverage on the generation job so a partial narration is not a clean success. Fixes #1567 * fix(tts): fail closed on MiniMax status and skipped narration Treat every non-zero MiniMax base_resp status as a failure, even when audio bytes are present. Classify 1002, 1039, 1041, and 2045 as rate limits. Floor rate-limit back-off independently of TTS_MIN_INTERVAL_MS, cap the cumulative wait, and cancel those sleeps on abort. A skipped or thrown TTS phase reports zero coverage and a warning. * fix(tts): decay spacing, heartbeat TTS progress, default interval 0 Classroom TTS spacing steps back toward TTS_MIN_INTERVAL_MS after each successful clip, and that interval defaults to 0 so pacing starts only after a rate limit. Clip progress is reported through the existing job onProgress path, so updatedAt keeps moving during a long narration phase. A Retry-After that does not fit the remaining back-off budget skips that clip instead of silencing every later clip. generateTTSForClassroom always returns coverage. A skipped run that has resolved a provider counts clips after the same speech split as synthesis. --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-23 13:26:04 +08:00
AbelWangYaBoandwyuc	05d782090f	feat(persistence): resolve single-tenant requests to a shared owner id (#1639) ... Course documents, folders, materials and agent sessions are partitioned by an owner id that comes from a 30-day anonymous cookie, because there is no host auth layer. For one team behind one ACCESS_CODE that partition buys nothing — those visitors already share the site password — and it costs: a second browser shows an empty course list, and POST /api/stages/[id]/publish refuses every owner, because publishing a cookie partition is not something the product allows. PERSISTENCE_SHARED_OWNER_ID resolves every request to that fixed id instead. Unset — the default — changes nothing. - **ACCESS_CODE is required.** Without one the middleware lets every request through, so a single shared owner would expose one readable, editable and publishable course library to anyone who can reach the deployment. That is worse than the per-browser partitioning this setting removes, so the combination refuses to start, from instrumentation.ts, with a message naming both variables. The alternative the issue allowed — ignore it with a warning — was rejected because a silently ignored setting is exactly the confusion this feature exists to remove. - The value is validated rather than trusted: 1-128 characters of [A-Za-z0-9._-]. That excludes the reserved `anon:` prefix, which would still be refused by publish and would alias onto a cookie owner, and any character the material-key sanitiser rewrites, which could collide two ids onto one object key. A malformed value fails startup on the same path; an empty value is treated as unset, since KEY= in an env file cannot be told apart from an operator who meant to leave the feature off. - An explicit authenticatedOwnerId still outranks it, so adding a real auth layer later does not require clearing the variable first. - All four places that derive an owner honour it, including the Server Action in lib/workbench/workspace-actions.ts, which re-reads the cookie itself. That one is easy to miss and would make the workspace list and its row actions disagree about who owns a session. - The env docs state the trust model next to the variable, cross-reference SECURITY.md, and note that this answers who owns a course while DATABASE_URL answers where one lives — a second browser sees an empty list on Postgres too. This is the stopgap asked for in #1550 — a PR in the spirit of the closed #1551 — with threading a real authenticated owner left as the long-term fix. AI-assisted commit Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-22 17:33:02 +08:00
3 people	1e867e8448	fix(skills): keep exported zip entry names POSIX-separated on Windows (#1641) ... `buildSkillDirZip` names each zip entry with `path.relative(dir, file)` directly. On Windows that separator is a backslash, so an exported skill archive contains entries like `openmaic/references\extend.md` instead of `openmaic/references/extend.md`. Zip entry names are POSIX paths by spec, and consumers — the skill import path, `unzip`, and the round-trip test — resolve entries by forward-slash paths, so an archive exported on Windows cannot be read back as a skill package. Normalize the relative path to forward slashes before naming the entry. On POSIX platforms `sep` is already `/`, so this is a no-op there. Co-authored-by: BeAIcoder <beaicode@example.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-22 17:21:00 +08:00
4 people	b0e481eaa5	fix: keep long Grok relay generations alive and inline image bytes (#1364) ... Two independent Grok failures seen when the provider is reached through a relay (a custom base URL) instead of api.x.ai directly: - lib/ai/providers.ts: a long non-streaming chat completion was cut off by the relay with a 504 at its idle timeout (~5 min), because nothing is sent upstream until the model has the whole answer. Adding 'grok' to the existing streaming-compat path (OPENAI_COMPAT_USE_STREAMING_CHAT=true) keeps bytes flowing across the idle window; the SSE is buffered back into a normal JSON response for the caller. The path is for relays only. usesCustomOpenAIBaseUrl recognises OpenAI's origin alone, so Grok's own api.x.ai also read as "custom" and was forced onto the compat transport; the provider's native endpoint is now excluded. - lib/media/adapters/grok-image-adapter.ts: response_format 'url' returns a link on the relay's CDN host (imgen.x.ai), which may be unreachable from the server's network. The generation then failed at the follow-up fetch through /api/proxy-media even though the image had been produced successfully. 'b64_json' inlines the bytes and removes that second hop. Inline bytes declare no media type, so the adapter reports one on ImageGenerationResult and returns a typed data URL, which is the shape openrouter-image-adapter already uses. Consumers take the type from there: agent image persistence records it, the client's stored media row keeps the type its data URL states, and classroom-media-generation names the file with the matching extension. A JPEG is no longer stored, served or named as a PNG. The process-wide undici timeout that previously accompanied these changes is dropped: upstream #1404 now gives LLM calls their own undici headers/body timeouts, which covers the same failure without raising the defaults globally. Co-authored-by: ciclou1 <ciclou1@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-22 12:04:20 +08:00
3 people	6dfeb62dd8	fix(media): emit keyframe images as data URLs [AI-assisted] (#1444) ... * fix(media): emit keyframe images as data URLs The local media extractor stored keyframe bytes as raw base64 in `DocumentAsset.data`. Every other extractor emits a data URL, and the document bundle forwards this field as `pdfImages[].src` to `storeImages`, which decodes it with `decodeBase64DataUrl`. With no `data:` prefix the comma split yields no payload, so `atob(undefined)` throws and course generation fails with "Failed to store image bundle at image img_1". `pdf-compat`'s `dataUrlMimeType` also derives the image asset mime from this same field, so the declared `image/webp` was silently dropped too. Emitting `data:${mime};base64,...` matches `mineru-parser`, which already normalizes prefix-less base64 the same way. * fix(media): decode keyframe data URLs * fix(media): accept legacy and data-url assets * fix(media): reject malformed or empty media asset data URLs A string that starts with data: but does not parse as a data URL used to fall through to the raw-base64 path, where Node's decoder skips non-alphabet characters and yields garbage bytes. Throw instead, and reject an empty base64 payload rather than storing a 0-byte image. Correct the keyframe test comment: local keyframe assets are consumed by material extraction, and the test needs ffmpeg so it does not run in CI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-09-21 21:48:02 +08:00
Yizuki_Ameandwyuc	54d626bc61	fix(export): surface video render rejection reasons (#1414) ... * fix(export): surface video render rejection reasons * fix(export): keep render diagnostics out of user toasts --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-21 18:49:25 +08:00
Huang Geyangandwyuc	67f568848a	fix(server): warn when access-code protection is disabled (#1599) ... Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-20 19:14:38 +08:00
3 people	d51f4b8350	fix(audio): rebuild platform FormData bodies for the undici transport (#1580) ... #1514 moved every lib/audio provider call to the npm undici package's fetch so the pinned dispatcher is guaranteed to be honored. But the adapters kept building multipart bodies with the platform-global FormData — a class of Node's bundled undici — and undici's serializer brand-checks a FormData body against its own class. A foreign FormData fell through to the string branch and left the process as `content-type: text/plain;charset=UTF-8` with the 17-byte literal `[object FormData]` as the whole body: the audio bytes and every field (including `model`) were dropped, downstream gateways fell back to whisper-1 and answered 503, and every multipart audio request (ASR, TTS FormData paths, voice registration/cloning) failed with a generic internal error. Bare Blob/File, string/JSON/Buffer and stream bodies were never affected: undici 7.29.0 exports no File/Blob classes of its own and its bare-body brand checks (and multipart part handling) bind the platform classes. Normalize the body in the transport, once, before either transport path serializes it, so the adapters can keep the platform globals as their public API boundary: a foreign FormData is re-created as undici's own with every entry carried over verbatim — `append` (not `set`) so repeated field names survive, and no filename argument so a platform File part keeps its own name/type/lastModified. Everything else passes through untouched. Also drive real loopback regression tests with a platform-global FormData (direct, with repeated field names, empty, and across a 307 redirect hop whose per-hop loop re-issues the normalized body) and a bare Blob, asserting multipart on the wire instead of `[object FormData]`, and update the voxcpm unit test that asserted the buggy contract (a platform FormData at the undici boundary). Fixes #1579 Co-authored-by: Claude Code <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-18 17:39:02 +08:00
3 people	a8726ec4f3	feat(media): add OpenRouter image and video providers (#1356) ... * feat(media): add OpenRouter image and video providers OpenMAIC ships six separate video providers (Veo, Kling, Seedance, MiniMax, Grok, HappyHorse) and seven image providers, each needing its own key. OpenRouter fronts those same model families behind one key and one account, so this adds it as a provider on both sides. Both use OpenRouter's dedicated media endpoints, not chat-completions: - Image: POST /images -> { data: [{ b64_json }] } - Video: POST /videos -> 202 { id, status }, poll GET /videos/{id}, then GET /videos/{id}/content for the mp4 bytes The model list is fetched live from GET /images/models and GET /videos/models through /api/openrouter-models rather than pinned in the registry: OpenRouter hosts 48 image and 28 video models today and adds more, so a hardcoded shortlist would decide for the operator which models exist. The registry keeps a three-entry seed as an offline fallback, and the existing custom-model UI still accepts any model id. Both catalogs answer unauthenticated, so the picker fills before a key is pasted; a key is forwarded when present for proxied base URLs. Adapter contracts are covered by stubbed-fetch tests (request shape, empty-response handling, and the video job state machine including terminal failure). No test performs a billable call. Closes #1355 * fix(media): validate the key and tolerate a pasted endpoint URL Three fixes found while configuring the new provider: 1. Both connectivity probes hit the model catalogs, which answer 200 unauthenticated — so "Test Connection" reported success for any string, including an invalid key. Probe GET /key instead: equally cheap, and it actually rejects a bad key. 2. The settings field is labelled "Base URL" but the panel echoes it back as "Request URL", so pasting the full endpoint (https://openrouter.ai/api/v1/images) is the natural mistake. That built /api/v1/images/images and 404'd. Trim a trailing slash and a trailing /images or /videos so both forms work; a proxy path that merely contains the word is left alone. 3. The image and video settings panels read `data.message` on a failed test, but failures answer with `error` (apiError) and only successes carry `message`. Every failing connectivity test — for any provider, not just OpenRouter — rendered "connection failed: undefined" instead of the reason. Pre-existing; surfaced by 1 and 2 above. Closes #1355 * fix(media): make every OpenRouter model selectable, and always select a provider Two gaps found while configuring the new provider. The settings Models list is a read-only catalog for every provider; the actual model picker is the media popover. That picker built its groups from the static registry array, so OpenRouter offered only the three-entry seed while settings listed the full live catalog — the models were visible but not choosable. Feed the same live catalog into the popover, fetched only once the provider is usable so an unconfigured install makes no request. Separately, `imageProviderId`/`videoProviderId` are empty until a provider is chosen (first-run auto-config leaves them blank when the server reports no media provider). Opening the settings panel on an empty id selected nothing: the header rendered the missing name key as "settings.undefined", and Test Connection posted a blank x-image-provider/x-video-provider, so it failed with "No image/video provider configured" whatever key was typed. Fall back to the first catalog entry so the panel always has a selection. Pre-existing and not specific to OpenRouter. Closes #1355 * fix(tts): request a browser-playable format from custom providers `generateOpenAITTS` serves every custom OpenAI-compatible TTS provider but never sent `response_format`, so it inherited whatever each provider defaults to. OpenAI defaults to mp3; OpenRouter's /audio/speech defaults to raw `pcm`. The unknown content type then fell through to the `'mp3'` default below, the client built `data:audio/mp3;base64,…` from headerless PCM samples, and playback failed with "no supported source was found" — while the server logged a clean 200, because the audio really was generated. Name the format instead of inheriting it. Also stop mislabelling an unrecognised body: `pcm`/`l16` now raises a message naming the cause, and `aac`/`opus` are recognised. Two supporting fixes: - /api/openrouter-models normalises its base URL the way the adapters do and falls back to the public catalog when a custom base URL fails, so a typo in a free-text settings field cannot empty the model picker. Also types the headers object so tsc accepts the conditional. - provider-neutrality-guard pins exact per-vendor occurrence counts in lib/server/provider-config.ts. Adding the image and video env entries raises "openrouter" from 2 to 6 (each entry contributes both its key and its value); CI failed without the bump. Closes #1355 * fix(security): never send the operator key to a client-chosen host Review found `/api/openrouter-models` was an SSRF and key-exfiltration path, and the finding is correct. The route took `x-base-url` from the caller at highest precedence while preferring the *server* env key, so any caller could make the server send the operator's OpenRouter credential as an `Authorization: Bearer` header to an arbitrary URL. The route's own comment claimed it followed `/api/verify-image-provider`; that pattern runs `validateUrlForSSRF` on client base URLs, and this route did not. The boundary is now explicit: the server key travels only to the operator's own base URL. A client-supplied URL is SSRF-validated and carries only that caller's own `x-api-key` — the server key is dropped — and the unauthenticated public-catalog fallback never forwards a credential chosen for a different host. Redirects are no longer followed (`redirect: 'manual'`), since a redirect would carry the Authorization header off-host and reopen the same hole, and upstream reads are bounded by a timeout. The per-URL cache is now keyed by destination *and* a hash of the credential, and bounded to 64 entries with oldest-first eviction, so client-supplied URLs cannot grow it without limit and one caller's key-authorised catalog is never served to another. Also from the review: - The image adapter discarded the reported `media_type`. The orchestration layer wraps a bare `base64` as `data:image/png` unconditionally, so jpeg/webp results were mislabelled; the adapter now returns a data URL carrying the real type. - Adapter generation and poll requests set `redirect: 'manual'`, matching the `/key` probe that already did. - `runPolledTask` accepts an `AbortSignal` so the sleep between polls is cancellable; the video adapter passes the caller's signal. Without it a cancelled generation still slept out a full 10s interval. Tests cover the highest-risk paths the review named: which credential reaches which URL, that an SSRF-rejected destination is never contacted, that the fallback is unauthenticated, cache isolation between callers, and MIME preservation. Findings 2 (base-URL normalisation) and 4 (neutrality-guard debt) were already fixed in d553a08, pushed after the review was submitted; CI is green on that commit. Closes #1355 * ci: retry flaky voice clone timeout * fix(vercel): keep OpenRouter catalogs within Hobby function limit * fix(vercel): avoid tracing self-hosted sharp binaries --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-18 00:01:48 +08:00
LeoParkerOuandwyuc	784f2a9f95	fix: bound model discovery response body timeout (#1478) ... Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-17 23:07:23 +08:00
xuyuanwei678	f7b8769e7f	fix(importer, renderer): preserve PPTX text, chart, and image styling (#1534) ... * fix: preserve PPTX text, chart, and image styling * fix(importer): preserve hyperlink semantics and remove inferred alignment * test(charts): replace untyped assertions to satisfy CI * fix(importer): honor source-linked chart axis number formats * fix(renderer): preserve picture bar clustering and clipping * fix(pptx): address chart scales and soft-edge review findings * fix(pptx): preserve percent-stack labels and inherited text sizes * fix(pptx): synchronize agent chart schema and axis defaults	2026-09-17 21:41:12 +08:00
wyucandClaude Opus 5	2cbd011c1f	feat(token-plan): add TokenDance one-key preset for every modality (#1525) ... * feat(token-plan): add TokenDance one-key preset for every modality TokenDance is a model gateway: chat and images are OpenAI-compatible at /gateway/v1, and the same key authenticates vendor-protocol routes on the same host (Ark, MiniMax, Bocha). The preset reuses the existing adapters with those route prefixes as base URLs, so one key lights up LLM, image, video, TTS and web search from Settings -> Token Plan. - providers: add a built-in `tokendance` OpenAI-compatible provider (TOKENDANCE_* env prefix, logo, provider name in all locales) - token-plan: add the TokenDance preset (Seedream image, MiniMax H3 video, MiniMax speech TTS, Bocha web search) - seedream: use a base URL that already ends in a version segment verbatim, so gateway routes like `/ark/v3` do not get `/api/v3` appended - minimax-video: route H3-family models through the v2 task API (content array submit, task-envelope poll); connectivity checks for H3 probe auth on the v2 query route instead of submitting a billable task - README: add a one-key quick example and replace the Gemini-specific model recommendation with a provider-agnostic setup recommendation Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qrrq9CPwb718mpouz8Y2KL * fix(token-plan): accept preset web-search base URLs and report H3 dimensions per ratio - web-search: the client base URL allowlist also accepts the exact base URL a built-in token plan preset writes for that provider, derived from TOKEN_PLAN_PRESETS. Applying a plan whose web-search route is not an official vendor host previously stored a URL that the route rejected with 400. Any other client URL is still rejected. - minimax-video: report H3 v2 clip dimensions for 16:9, 9:16, 4:3 and 1:1 instead of assuming landscape for every non-portrait ratio. - tests: pin the allowlist for every preset, the 1:1 H3 dimensions, and clear TOKENDANCE_* in the provider-config env isolation list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qrrq9CPwb718mpouz8Y2KL --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-09-16 13:28:09 +08:00
wyuc	b5605ef979	feat(agent-runtime): workbench generate_image / generate_video write through the asset pool (#1007 part 6) (#1524) ... The workbench tools store generated bytes in the asset pool under the shared principal and write the allocated id into the document, the same discipline as the classic chain since #1392; the runner's putScene creates the reference rows and commits the allocations. The video completion patch rewrites every placeholder slot and retires anything that would shadow the new id; immediate render is preserved by leasing the id at the render boundary. A store-full refusal fails the tool with a model-readable error and writes nothing. Legacy /api/classroom-media documents keep rendering. Closes #1522.	2026-09-16 11:57:54 +08:00
wyucandClaude Opus 4.8	87c4524b4d	fix(audio): validate redirects and pin connections on provider requests (#1514) ... Audio provider requests (TTS, ASR, voice registration and voice cloning) validated a client-supplied base URL once and then issued a plain fetch with default redirect-follow and no pinned dispatcher. A base URL that resolved to a public address but answered with a redirect to an internal one was followed, and a DNS answer that changed between the guard's lookup and the connect reached an internal host — both readable in-band. - Route every lib/audio provider request through a new lib/server/audio-provider-fetch.ts that combines per-hop redirect re-validation with a pinned undici dispatcher, so the socket can only reach an address the guard validated, on every hop. - Select the public-vs-local policy server-side from isServerConfiguredProvider; a client-supplied base URL is always strict public and can never reach a private, loopback or cloud-metadata address, even with ALLOW_LOCAL_NETWORKS set. - Pin the result-audio download hop as well, keeping its host allowlist and redirect:'error'. - Add a coverage-matrix test that fails if any lib/audio module regains a raw provider fetch. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-09-15 17:47:30 +08:00
wyucandClaude Opus 4.8	ee79762ef9	fix(access-code): expire tokens server-side and throttle verification behind a trusted proxy (#1513) ... When ACCESS_CODE is set, verification tokens (timestamp.HMAC) never expired because neither verifier checked the timestamp, and POST /api/access-code/verify had no attempt throttling. - Enforce a 7-day token lifetime in a shared, Edge-safe module used by both the Node verifier and the middleware Web-Crypto verifier, and reject non-canonical signatures in both. - Rate-limit verification only when the client identity is trusted (TRUST_PROXY_HEADERS=true): a per-client sliding window (10 failures / 60s) whose attempt is reserved atomically at check time, returning 429 with Retry-After. Without a trusted proxy the app cannot attribute requests to a client, so no shared throttle is applied — a long random ACCESS_CODE is the protection, and a warning is logged when it is short. - Store bounded, copied identity keys so a large forwarding header cannot retain memory. - Document the behavior in .env.example, README, and configuration docs. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-09-15 17:47:14 +08:00
wyuc	ee7a7b64df	fix(agent-runtime): allow one owner material to be bound to multiple sessions (#1500) ... * fix(agent-runtime): allow one owner material to be bound to multiple sessions `agent_session_materials.id` is a global primary key, but `bindOwnerMaterialsToSession` de-duplicated on `(sessionId, id)` and then inserted the owner-side material id as the row id. Binding the same owner material to a second session therefore raised a duplicate-key error (23505) that the route surfaced as HTTP 500, so an owner material could only ever be used by one course. Keep row ids globally unique (every extraction method keys on `id` alone) and store the shared owner id as metadata instead: the binder now mints a fresh session row id, records `owner_material_id`, and looks that column up for idempotent rebinding; a partial unique index on `(session_id, owner_material_id)` adjudicates concurrent rebinds and the loser adopts the winner's row. The schema change is additive and idempotent (`ADD COLUMN IF NOT EXISTS` + `CREATE UNIQUE INDEX IF NOT EXISTS`), so existing databases upgrade in place with `NULL` for legacy rows. No FK from `owner_material_id` to the owner library, to keep owner-library deletion decoupled from session rows. Tests: backend-neutral contract test (same owner material bound to two sessions, both readable), PGlite in-place upgrade test, pinned-schema update, and a host-level test covering the route's 202 path. * fix(agent-runtime): reuse legacy owner-material bindings and clean up losing uploads on concurrent bind Rows written by the previous binder use the owner upload id as the session row id and leave `owner_material_id` NULL, so the new owner-id lookup missed them and a rebind created a duplicate row and copied the bytes again. The binder now falls back to the session row id and adopts it only when it is unambiguously that owner upload: source kind, matching title, no source URL or derivative, no text, and the deterministic legacy object key with a byte length that matches the owner record. Adoption stamps `owner_material_id` through a conditional `backfillOwnerMaterialId` update that leaves extraction columns untouched; a lost race re-reads the fast path. A different session is still a fresh row. The concurrent-bind loser also left its uploaded object behind after adopting the winner. It now removes that object when the winner references a different key, best-effort so a failed cleanup cannot fail a bind that succeeded. Tests: host-level PGlite regressions for the legacy rebind (id, row count, byte copies, extraction state, and backfill) plus a different-session bind, and a controllable byte store that parks the loser after its upload so the winner commits first, asserting one row and only the winner's object remain. A storage unit test pins the backfill contract. Bumps @openmaic/storage to 0.30.1 for the new public store method.	2026-09-14 18:43:31 +02:00
Nguyen Quang Thiepandwyuc	3e3cac5f76	fix: enable Pro workbench flag in Docker builds; allow non-TLS localhost cookie (#1484) ... * fix: enable Pro workbench flag in Docker builds; allow non-TLS localhost cookie Two deployment fixes discovered while self-hosting with Docker Compose: 1. Expose NEXT_PUBLIC_PRO_WORKBENCH_ENABLED as a Docker build arg. Persistence and other NEXT_PUBLIC_* flags are already wired through Dockerfile + docker-compose.yml, but the Pro workbench entry flag was missing, so Docker deployments could not enable the workbench at all. 2. Add COOKIE_SECURE opt-out for the anonymous owner cookie. Production builds always set `Secure` on the anonymous_id cookie. Safari refuses to store Secure cookies served over plain http://localhost (it does not special-case localhost like Chromium/Firefox), so every request minted a fresh anonymous owner and owner-scoped document writes were rejected with 403 — scenes never persisted and generation appeared to hang after the first scene. COOKIE_SECURE=0 lets plain-HTTP deployments opt out; the default (Secure in production) is unchanged. Documents COOKIE_SECURE in .env.example alongside the persistence opt-ins. * fix: share the COOKIE_SECURE opt-out with the Server Action cookie mint Review follow-up on the COOKIE_SECURE opt-out. - Extract anonymousCookieSecure() in owner.ts and use it in lib/workbench/workspace-actions.ts, which re-implements the anonymous_id mint: with COOKIE_SECURE=0 that path still sent a Secure cookie, so a Server Action running before any /api/agent/* request (e.g. deleting a workspace session) minted an ephemeral owner in Safari. - State the opt-out accurately in owner.ts; the previous comment claimed the flag never forces Secure off. - Add the regression test beside the existing production case (verified it fails when the opt-out is dropped). - .env.example: document the security cost of dropping Secure and that only the exact value 0 disables it. --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-14 17:01:59 +02:00
wyuc	68aed2932a	fix(proxy-media): connect only to addresses the SSRF guard validated (#1488) ... * fix(proxy-media): connect only to addresses the SSRF guard validated The route validated a URL with validateUrlForSSRF and then handed the hostname to fetch(), which resolved it again at connect time. An attacker-controlled name could answer a public address to the guard and an internal address (loopback, RFC1918, link-local, cloud metadata) to the socket, and every redirect hop repeated the same validate-then-refetch gap. Add a shared policy-aware pinned dispatcher (lib/server/pinned-dispatcher.ts) whose connect.lookup resolves all addresses, runs the same local-network policy as validateUrlForSSRF over every answer (cloud metadata is always blocked, even with ALLOW_LOCAL_NETWORKS=true), and passes only the vetted list to the connection. proxy-media creates one dispatcher per request and uses it on every hop, mapping a connect-time refusal to 403 INVALID_URL instead of a generic 500. The agent-runtime pinned agent now reuses the same builder with its strict assertSafeIp policy unchanged. The other routes that validate with validateUrlForSSRF and then hand the URL to provider SDKs (parse-pdf, transcription, generate/*, verify-*, probe-models, azure-voices) are not covered; that is a follow-up. Tests: new tests/server/proxy-media-rebinding.test.ts covers the two-answer rebind, the ALLOW_LOCAL_NETWORKS opt-in through the pinned path, metadata refusal under the opt-in, and a redirect-hop rebind. * fix(proxy-media): destroy the per-request dispatcher and align non-unicast policy across layers Follow-up hardening for the proxy-media SSRF pin. 1. Handler hang: the route awaited `dispatcher.close()` in `finally`. undici's `close()` drains in-flight requests, and the response body is intentionally left unread on the 30x (`redirect: 'manual'`), non-2xx, oversize and connect-refusal paths, so an upstream that keeps its body open never completes and the POST never settles. Tear the pool down with a non-awaited `destroy()` instead; the response blob is already in memory by the time `finally` runs. Regression tests drive a loopback server that answers 404, a 30x, and an oversize 200 while trickling its body forever and assert the route settles in well under a second. 2. Headline rebinding test sensitivity: install an attacker-controlled global dispatcher (`setGlobalDispatcher` plus a lookup that always answers loopback) in `beforeEach` so the unpinned fetch path really reaches the loopback server. Removing the route's dispatcher now fails on `internal.requests() === 0` rather than on an unrelated DNS resolution error. The positive control also asserts the pinned connect lookup was invoked, so a runtime that silently ignores `init.dispatcher` goes red. 3. Non-unicast policy parity: `validateUrlForSSRF` only classified IP-literal URLs through `isPrivateIP`, so a literal in CGNAT 100.64/10 or an IANA reserved/special-use range (240/4, 198.18/15, the TEST-NET blocks, 192.0.0.0/24, multicast, broadcast) passed the URL layer while the same address reached through a hostname was refused at connect time. Both layers now share `isNeverAllowedRange`, applied to IP-literal URLs and to every resolved answer, with or without `ALLOW_LOCAL_NETWORKS`. `connectionAddressBlockReason` drops its blunt `range() !== 'unicast'` test so the two layers agree exactly, while private, loopback and link-local answers keep following the opt-in. Behaviour change: a hostname resolving into CGNAT 100.64/10 or an IANA reserved/special-use range is now refused (403 INVALID_URL) where it was previously proxied, including under `ALLOW_LOCAL_NETWORKS=true`, since these ranges are never legitimate proxy targets. 4. Do not echo an unparseable connect-time answer in the refusal. Return the generic block message instead of `Unable to classify network address: <value>`; the route relays that message to the client in a 403 body. Tests: new trickle-body regression tests and CGNAT/reserved cases for both literals and resolved answers, plus a `connectionAddressBlockReason` parity suite. One existing fixture is updated from the IANA documentation prefix 2001:db8::/32 (reserved) to a real public prefix 2001:4860::/32. * fix(ssrf-guard): let the local-network opt-in cover CGNAT and decode local-use NAT64 prefixes	2026-09-13 18:00:56 +02:00
wyuc	1fd4348f21	fix(classroom): generate classroom ids server-side and create files exclusively (#1489) ... * fix(classroom): generate classroom ids server-side and create files exclusively The legacy file-backed classroom store accepted a caller-chosen stage.id and renamed a temp file over the target, so anyone who knew a public share URL could POST the same id and replace that classroom's content. POST /api/classroom now mints the id itself (nanoid, 10 characters, matching the generation pipeline), ignores any client-supplied stage.id, and rebinds the stage and every scene to the generated id. Both the route and classroom-generation persist through a new exclusive create that hard-links a temp file into place, so an existing classroom is never replaced; EEXIST is retried with a fresh id a bounded number of times and then reported as 409. Unchanged: the server-backed persistence path (app/api/persistence, lib/persistence), the read/GET path, middleware, and the access-code gate; writeJsonFileAtomic keeps its overwrite semantics for the other callers. Adds regression tests pinning non-overwrite, the typed EEXIST error, collision retry/exhaustion, and generation-path exclusivity. * fix(classroom): fall back to exclusive open without hard links and reserve ids before media generation writeJsonFileExclusive kept `fs.link` as its only route to the destination, so classroom creation failed hard on mounts without hard links (gcsfuse, s3fs and some FUSE gateways return ENOSYS/ENOTSUP/EOPNOTSUPP/EPERM/EXDEV). Keep `link` as the fast path and, for exactly those codes, fall back to an exclusive `wx` open: still never replaces an existing file and still maps EEXIST to ClassroomAlreadyExistsError. Any other code keeps propagating, and the temp file is always removed. The generation pipeline wrote media and TTS into <CLASSROOMS_DIR>/<id>/{media,audio} before the exclusive create, so a collision would have landed the new classroom's files in an existing classroom's directory and the retried document's URLs would still point at that other id. Reserve the id first by exclusively creating the classroom file with a placeholder document (`reserved: true`, empty scenes). The file is the only token that atomically covers the whole collision namespace: a classroom created through POST /api/classroom has a JSON file but no directory, so reserving a directory would miss it. readClassroom treats a reserved document as absent, so an in-flight or crashed reservation is never served as an empty classroom. The retry on EEXIST now happens at reservation time (bounded, before any media), and the final persist is an ordinary overwrite of the id the process owns, so the post-media id retry is gone. Document OPENMAIC_CLASSROOMS_DIR in .env.example, including that CLASSROOM_JOBS_DIR does not move with it. * fix(classroom): release an unused reservation when generation fails	2026-09-13 17:49:57 +02:00
Yizuki_Ameandwyuc	c460bceb60	feat(export): preflight render queue availability (#1455) ... Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-11 14:33:51 +02:00
wyucandClaude Fable 5.1	d5be393317	test(agent-runtime): make the material search budget test deterministic (#1453) ... search_material stopped on whichever of the character budget or the 100 ms wall-clock budget fired first, but the budget test asserted the exact character count, so it failed on slow CI runners when the time budget won (observed scannedChars 786432 instead of 1000000). Route the search path's clock through the injectable `now` dependency the wait tool already uses, freeze the clock in the character-budget test so only that budget can stop the scan, and add a time-budget test that trips the deadline after exactly one chunk. Closes #1452 Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-11 05:33:42 +02:00
Mahesh Singh	57782f928f	fix(tts): reject non-audio 200 responses before storing or billing (#1395) (#1405) ... * fix(tts): reject non-audio 200 responses and resolve audioUrl envelopes (#1395) * fix(tts): address review feedback on #1405 — drop follower, protect raw pcm, test billing invariant	2026-09-10 20:52:35 +02:00
wyucandClaude Fable 5.1	29735f10d0	fix(ssrf): keep cloud metadata endpoints blocked under ALLOW_LOCAL_NETWORKS (#1419) ... * fix(ssrf): keep cloud metadata endpoints blocked under ALLOW_LOCAL_NETWORKS ALLOW_LOCAL_NETWORKS=true exists so self-hosted deployments can point providers at loopback, RFC1918 and split-horizon targets. It also returned early from validateUrlForSSRF before any classification, so a client-supplied base URL of 169.254.169.254 (or metadata.google.internal, 100.100.100.200, fd00:ec2::254) was accepted on such deployments. Cloud instance-metadata endpoints are now rejected regardless of the flag: literal hosts and IPv4-mapped forms are classified directly, and non-IP hostnames are resolved so an answer that lands on a metadata address is rejected too. DNS failure under the flag still fails open, as before, because split-horizon DNS is an explicit use case of the flag. Without the flag the DNS path now also rejects answers on metadata addresses that are not RFC1918 (100.100.100.200). The route tests that used the metadata address as their example of a target the flag allows now use a private-network address instead. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ * fix(ssrf): widen the metadata set and bound the flagged DNS lookup Review follow-up. Add the AWS ECS task credential and EKS Pod Identity endpoints (169.254.170.2, 169.254.170.23, fd00:ec2::23), the Azure WireServer address (168.63.129.16) and the legacy OCI IMDS address (192.0.0.192) to the blocked set; the last two are neither RFC1918 nor link-local, so they were reachable even without the flag. Recognise the metadata addresses when carried inside 6to4, Teredo, ISATAP and NAT64 literals. Bound the DNS lookup done under the flag to three seconds and fail open on expiry, matching the existing fail-open on error. Say in .env.example that the set is a fixed list and that DNS failure is allowed through. Use the same private-network fixture for the reject and allow cases of the four route guard tests so a partial NODE_ENV re-gate of the guard goes red again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ * fix(ssrf): share the tunnel decoder between the private and metadata checks Review follow-up. assertSafeIp now classifies metadata addresses through isCloudMetadataAddress, so an ISATAP identifier under a globally routable prefix that carries 168.63.129.16, 192.0.0.192 or 100.100.100.200 is rejected on the strict-fetch path too. isPrivateIP uses the same tunnelEmbeddedIPv4 helper instead of its own copies of the 6to4, Teredo and ISATAP decoders, which also gives it NAT64. Tests cover the globally-routable ISATAP forms, the NAT64 private case, and the false-positive direction (tunnel literals carrying public or RFC1918 addresses stay allowed under the flag). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ * fix(ssrf): reject metadata hostnames before the flag branch Review follow-up. The metadata hostname and address check now runs before the ALLOW_LOCAL_NETWORKS branch, so metadata.google.internal is rejected by name in both flag states (previously only under the flag), and metadata literals get the metadata message rather than the one that suggests setting the flag. Pin the tunnel decoder boundaries in tests (2001:db8 is not Teredo, 64:ff9b:1 is not NAT64, 2003 is not 6to4). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GRaNc5E88r41GUWt2Y3uiQ --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-08 19:44:20 +02:00
wyucandwyuc	0bab621e09	harden input validation and outbound URL handling (#1387) ... * harden input validation and outbound URL handling - Validate the classroom id on the write path with the same allowlist the read path already applies, and assert in the storage layer that a resolved classroom file stays inside CLASSROOMS_DIR. - Apply validateUrlForSSRF at every call site regardless of NODE_ENV; the guard already owns the ALLOW_LOCAL_NETWORKS escape hatch, so the extra environment condition only disabled the check outside production builds. Add a repository-scanning test so a gated call site cannot reappear. - Re-validate every redirect hop of an outbound provider request through a shared transport, mirroring the per-hop loop proxy-media already uses. - Restrict stored slide HTML to the formatting vocabulary the renderer produces, sanitizing at the classroom persistence boundary on both write and read, with a separate policy for KaTeX snapshots. - Refuse the development persistence authenticator under NODE_ENV=production unless an explicit opt-in is set, and document it in .env.example. * drop credential headers on a cross-origin redirect hop The manual redirect loop reused the caller's init on every hop, so provider credentials were re-sent to the redirect target even on a different origin. The platform fetch drops Authorization itself when it follows redirects, so mirror that: strip authorization, api-key, x-api-key and x-goog-api-key before a cross-origin hop, matching case-insensitively and preserving the shape of the caller's headers. Same-origin hops are untouched. A streaming request body cannot be replayed onto the next hop, so fail with a clear message rather than sending an empty body. --------- Co-authored-by: wyuc <dhq1024@proton.me>	2026-09-05 23:56:37 -07:00
Yizuki_Ame	eb2cbc701a	feat(pro): generate concise conversation titles (#1275) ... * feat(storage): add automatic conversation title lifecycle * feat(agent): generate concise conversation titles * feat(agent): schedule automatic conversation titles * fix(agent): harden automatic title lifecycle * fix(agent): refine conversation title prompt * fix(agent): harden automatic title boundaries	2026-09-04 10:52:25 -04:00
杨慎andClaude Code	3a0612e962	fix(server): bound skill zip inflation instead of trusting declared sizes (#1374) ... parseUserSkillZip guarded the SKILL.md size against the zip central directory's self-declared uncompressedSize and then handed the entry to JSZip's async('string'), which inflates the real deflate stream no matter what the header claims. A malicious zip declaring a tiny size slipped past the guard, and with the upload route's 1 MiB archive cap and deflate's ~1032:1 ratio, one request could expand ~1 GiB in the server process — repeatable per request for memory-exhaustion DoS. Inflate the entry's compressed bytes with node:zlib inflateRawSync({ maxOutputLength: 70_000 }) so zlib aborts the moment output crosses the cap, whatever the headers say. The declared-size check stays as a fast path that refuses honest oversized zips without inflating; STORE entries are byte-length checked directly since their "compressed" content is the content. Over-limit output raises the same UserSkillUploadError('The archive SKILL.md is too large.'), so callers, route responses, and status codes are unchanged. Tests: a forged-header bomb (a real zip whose central directory is patched to declare a small uncompressedSize, with the forgery asserted to have taken on reload) is refused; an honest oversized zip is refused by the fast path; a forged STORE entry is refused by the byte-length check; a STORE zip round-trips; a corrupt deflate stream still maps to invalid-zip rather than a size refusal. Co-authored-by: Claude Code <noreply@anthropic.com>	2026-09-04 02:23:09 -04:00
3 people	dd74cfe23b	feat(web-search): add Exa provider (#1342) ... Co-authored-by: white638 <212083322+white638@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-03 06:14:05 -04:00
宙见·星野andwyuc	9b7b42c4c4	fix: normalize GPT Image 2 generation sizes (#1346) ... Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-03 04:48:42 -04:00
Yizuki_Ameandwyuc	e71b00c5c9	fix(agent): preserve generate_scene failure causes (#1316) ... * fix(agent): preserve generate_scene failure causes * test(agent): harden scene failure coverage * test(agent): isolate warning log format * test(agent): isolate warning log level --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-03 02:15:53 -04:00
mehmet turacandwyuc	0352ffde4d	fix(skills): normalize paths to POSIX before passing to pi-agent-core loader (#1297) ... * fix(skills): normalize paths to POSIX before passing to pi-agent-core loader On Windows, NodeExecutionEnv returns backslash-separated paths from fileInfo() and listDir(). The pi-agent-core skill loader passes these to the `ignore` package, which requires POSIX-style relative paths and rejects absolute or backslash paths with: RangeError: path should be a `path.relative()`d string Wrap NodeExecutionEnv with PosixNormalizingEnv that converts all FileInfo.path values to forward slashes, and normalize skillsDir itself before it reaches the loader. Fixes #1295 * fix(skills): normalize FileInfo.name alongside path for Windows skill discovery Address CR from HaningZS: on Windows, pi-agent-core's fileInfoFromStats() returns the full backslash path as name, so entry.name === 'SKILL.md' never matches. normalizeSkillFileInfo now derives name from the normalized path. Add deterministic tests for normalizeSkillFileInfo and a production-boundary test asserting listSkills() returns built-in skills. --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-02 06:01:18 -04:00
mehmet turacandwyuc	a7e9dc8337	fix(providers): apply server-pinned models for image and video providers (#1298) ... * fix(providers): apply server-pinned models for image and video providers The server-providers API already exposes image models, and the server resolves them correctly, but getServerVideoProviders() omitted the models field entirely, and fetchServerProviders() on the client side discarded the models payload for both image and video — it only set isServerConfigured and serverDisabled. This caused the client to fall back to built-in model IDs, which fail on endpoints that require different IDs (e.g. Volcengine Ark Agent Plan uses dotted aliases like doubao-seedream-5.0-lite). Server: getServerVideoProviders() now exposes models, mirroring the image listing. Client: fetchServerProviders() stores the server model list as customModels with replaceBuiltInModels: true for both image and video, matching the existing LLM provider model-filtering pattern. Fixes #1251 * style(tests): wrap long assertion to satisfy prettier printWidth --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>	2026-09-01 04:22:49 -04:00
wyuc	f6cf8fd4b7	feat(agent): make generate_video asynchronous with a placeholder ref (#1267) ... * feat(agent): make generate_video asynchronous with a placeholder ref The generate_video tool awaited the whole provider submit/poll/download/ persist cycle inside the tool call, blocking the agent turn for minutes (and effectively capping it at the 10-minute tool budget, below its own 15-minute internal budget). The tool now validates synchronously, mints a gen_vid_<id> placeholder (the scheme the outline flow already uses), and returns immediately so the agent can patch_stage the ref onto a video element and keep working; the element renders the existing skeleton while pending. A detached background job (own 15-minute timeout, deliberately decoupled from the caller's abort so a cancelled chat cannot silently orphan a billable provider job) runs the provider cycle, persists the bytes, and then: - patches the persisted document: every slide video element still referencing the placeholder gets the concrete server-hosted src (same runStageMutation discipline as the generation tools; skipped silently when the element was changed meanwhile), and - appends a media_ready lifecycle event to the session's durable log via the session-level control channel (valid post-run, unlike the runner's lease-guarded emit); the workbench folds it into the media generation store so the skeleton resolves instantly, on live stream and on replay. Pending tasks live in a process-local registry; a server restart orphans in-flight jobs (the placeholder keeps its skeleton), matching the classic flow's client-local durability caveats. Tracked as an accepted v1 limitation. Closes #1266 * fix(agent): unfence the background video patch from the run lease Review findings on the async generate_video change: - P1: the completion patch wrote through the run's owner-bound store, whose mutation fence asserts the run lease on every write. A video job settles minutes after its run ended and the lease is released, so every post-run patch threw AgentSessionLeaseLostError and the job wrongly settled failed. The runner now builds a dedicated owner-bound store for media jobs fenced only by the stage-mutation discipline, and the tool takes it as a separate backgroundStore dep. - P2: patchStageVideoPlaceholder rewrote whole scenes from one minutes-old loadDocument snapshot. It now re-reads each candidate scene immediately before its write and applies the placeholder swap to the freshest state, so concurrent user/agent edits survive; the residual read-write window matches the stage edit API's own read-modify-write discipline. - P3: drop the dead 'emit' progress marker (setPendingMediaStage is a no-op after settle), emit a media_ready failed frame from the last-resort crash guard so a bug path cannot leave the client on a permanent skeleton, and log when appendControlEvent silently drops a frame for a deleted session. * test(agent): cover the concurrent placeholder-element removal case Round-2 review leftovers: pin that patchStageVideoPlaceholder skips rather than resurrects a placeholder element deleted between the candidate scan and the write, and keep the detached job's crash guard synchronous (never-rejecting helpers only) so a future throw inside it cannot become the unhandled rejection the guard exists to contain. * fix(agent): isolate the completion patch from the provider budget Round-3 review leftovers: - The patch shared the job's 15-minute signal, so a provider cycle that nearly exhausted the budget could fail mid-patch and rebrand a persisted, downloadable asset as failed. The patch now runs on its own 60-second budget and a patch failure is logged while the job still settles and emits done (the done frame's src renders fine). - A user edit that replaced the placeholder with a concrete src while the job ran no longer gets clobbered by the stale mediaRef: the swap only writes while src is absent or still the placeholder. Accepted, documented: the sub-second read-write window between two concurrent jobs on the same scene (the client-side media fold renders either way), and the failed-state fold requiring an attached chat stream in v1. * fix(agent): close the regeneration gap in the placeholder src guard Delta review found the new patch-failure test never attempted the write (no element carried the runtime ref, so putScene was never called), and that the src guard also skipped legitimate patches: an element re-pointed at a new job via mediaRef while still carrying the previous generated src (which keeps rendering the old video), and an empty src. The guard now also writes when src is empty or a previously generated /api/classroom-media/ URL; the test seeds the ref so the failing write is really attempted and asserts the logged patch failure. * fix(agent): harden the placeholder src guard - Total predicate: a malformed non-string src no longer throws inside the element map and aborts the whole stage patch. - Recognize the absolute-form generated src the classic pipeline persists, and scope the generated-src arm to the stage's own media root so a user's pick copied from another stage is preserved.	2026-08-31 07:22:27 -04:00
wyuc	aa7d5fd585	feat(agent): let generate_scene pass widgetType and widgetOutline for interactive pages (#1259) ... * feat(agent): let generate_scene pass widgetType and widgetOutline for interactive pages The generate_scene tool schema had no widget fields, so every interactive page it generated fell back to a simulation widget in the generation pipeline. Add optional widgetType (simulation/diagram/code/game/ visualization3d; procedural-skill stays gated behind task-engine mode) and widgetOutline parameters, reject them for non-interactive page types, and carry them into the constructed SceneOutline. When only one of the two is provided, mirror the generator's own defaults (widgetType: 'simulation', widgetOutline: { concept: title }) so half-specified calls still generate. Closes #1258 * fix(agent): harden generate_scene widget params from review findings - Accept procedural-skill in the widgetType union; the handler already enables it via allowProceduralSkill and skill constraints can require it, so excluding it only produced an opaque schema error. - Validate widgetOutline is a plain object and reject malformed values (null, strings, arrays) with an invalid-widget-outline error instead of silently degrading the prompt or surfacing a misleading generic generation failure. - Reject widget fields on non-interactive pages by definedness rather than truthiness, matching the tool description's contract. - Pin the widgetOutline passthrough test to a non-default diagramType (mindmap), cover the handler's single-field defaults, cover procedural-skill passthrough, and exercise the tool schema through TypeBox Value.Check. * fix(agent): keep procedural-skill gated out of generate_scene Review feedback: main gates procedural-skill behind taskEngineMode and OPENMAIC_ENABLE_VOCATIONAL in every generation path, and no vocational signal reaches the agent runtime, so accepting the literal here would ungate the feature in every deployment. Drop it from the widgetType union (back to the first commit's semantics), say so in the schema description, and pin the gate in the schema validation test. Ungating for vocational agent sessions can come back as its own change once a task-engine signal is plumbed into the tool layer.	2026-08-31 07:11:24 -04:00
wyuc	dfebbcf33f	perf(classroom): speed up classroom loading (media hydration, sidebar thumbnails, media range requests) (#1276) ... * perf(classroom): defer non-priority media blob hydration off the load path Entering a classroom awaited the full mediaFiles restore — one object URL per restored image/video blob — before the loading gate opened. For video-heavy courses that is hundreds of MB of IndexedDB materialization blocking first paint. Split the restore into two phases: - The awaited phase now builds metadata-complete task entries but only creates object URLs for failed rows (none needed) and for media referenced by the scene the classroom opens on (persisted cursor, else the first scene). buildRestoredMediaTasks gains an optional shouldHydrateBlob predicate, defaulting to eager hydration so existing callers are unchanged. - Remaining blob-backed records hydrate in the background, chunked over requestIdleCallback (setTimeout fallback). Each deferred task stays 'done' — so generation resume never re-runs it — but carries no objectUrl yet, which the media resolution state machine already renders as a pending skeleton until the URL lands. Background hydration is guarded per record: a task that was replaced (classroom switch, regeneration, retry) or that belongs to another stage is skipped and its freshly minted URLs are revoked immediately. * perf(classroom): lazy-render slide thumbnails in the playback sidebar The playback scene sidebar mounted a full SlideCanvas for every slide scene the moment the classroom opened (and every off-screen video element opened a preload="metadata" fetch), because SlideThumbnail's existing visible prop was never passed. Extract the editor nav rail's near-viewport IntersectionObserver hook to lib/hooks/use-near-viewport.ts and gate the sidebar's slide thumbnails through it: only scenes within 200px of the viewport render the live canvas; the rest show SlideThumbnail's existing placeholder until scrolled near. The placeholder keeps the same box size, so gating never shifts layout. The shared hook now starts hidden instead of eager: with the previous eager-initial state, opening a long deck still mounted every canvas for a frame before the observer could flip off-screen items off. The observer's guaranteed initial delivery flips near-viewport items within a frame; the no-IntersectionObserver fallback (e.g. jsdom) defers to a microtask so the effect never synchronously re-renders. * perf(media): lazy image decoding and HTTP Range support for classroom media Renderer: BaseImageElement's <img> now carries loading="lazy" and decoding="async", so thumbnail-heavy surfaces (playback sidebar, editor nav rail, course cards) no longer fetch and decode every slide image up front. In-viewport images are unaffected — the browser fetches them immediately. slideToPng forces eager loading inside its permanently off-screen snapshot tree, where lazy images would otherwise never fetch and exports would capture blank slides. Server: GET /api/classroom-media/[classroomId]/[...path] now answers single byte-range requests with 206 Partial Content (Content-Range, Accept-Ranges, correct Content-Length), enabling progressive playback and seeking for hosted video/audio instead of downloading whole files. Suffix ranges are supported; unsatisfiable ranges get 416 with the full size; unsupported units or multi-range sets fall back to the plain 200 full-body response, which is always a legal answer. Existing caching headers are kept on every response shape. * chore(renderer): bump version to 0.1.4 for image lazy-loading change * fix(classroom): guard deferred hydration against restarted tasks and bound idle waits A deferred record is only ever 'done' without an objectUrl; a task that regeneration or retry restarted passes through pending/generating, so skip those instead of attaching stale persisted bytes. Also give the idle scheduling a timeout so a busy main thread cannot starve hydration. * fix(classroom): stop superseded deferred hydration before minting URLs A restore epoch captured at apply time now gates each idle chunk: loading another classroom or reloading the same one invalidates any older hydration loop, so it neither keeps scheduling work for an abandoned classroom nor attaches bytes read by an older load to a newer load's tasks. * fix(classroom): keep 416s uncached and classify element-keyed media as priority A 416 with public immutable caching can poison the media URL for later valid requests, so range errors now send Cache-Control: no-store. Priority classification also collects the opening scene's media element ids, since task lookup binds records keyed stage:<elementId> even when the slide slot carries a different opaque ref. * fix(classroom): tie deferred hydration liveness to the classroom load token The restore epoch only advanced when the next load reached apply, so an abandoned classroom kept hydrating during the next load's storage/network phase. Compose the epoch with the load's isCurrent (load token plus effect cleanup) so navigation stops the loop at the next idle boundary. * fix(classroom): classify legacy-recovered media before deferring it Legacy singleton video recovery assigns a placeholderRef only at the end of the task build, so a record keyed by an allocated id without placeholderRef was deferred even when it backs the opening scene's gen_vid_* element. Run a metadata-only build first to learn each record's effective ref and classify against it, keeping the first visible page's legacy video eager. * fix(action): wait for deferred video bytes before starting play_video executePlayVideo treated status done as immediately playable, but a deferred restore is done without an objectUrl: the renderer shows a skeleton, no <video> exists, and the later hydration never retriggers play, leaving the action stuck until the safety timeout. Readiness now follows the renderer's contract (only done-with-bytes is playable) at the initial check, the subscription exit, and the post-subscription recheck; the failed skip applies whether or not a wait happened. * fix(classroom): include the stage whiteboard in priority media refs The stage-level whiteboard stays open across standalone classroom switches, so its media can be visible before any scene is. Classifying it as deferred left visible whiteboard media pending behind idle hydration chunks.	2026-08-30 03:48:04 +08:00
wyucandClaude Code	f8b30ef0f0	fix(storage): prune redundant intermediate message_update frames (#1279) ... * fix(storage): prune redundant message update frames * fix(storage): prune all message update runs * chore(storage): release 0.27.0 for pruneMessageUpdates Co-Authored-By: Claude Code <noreply@anthropic.com> --------- Co-authored-by: Claude Code <noreply@anthropic.com>	2026-08-30 01:31:58 +08:00
wyuc	96be971d1c	fix(agent): bound import_pptx parse with size cap and timeout (#1268) (#1269) ... import_pptx could hang forever on large PowerPoint files: the isolated worker had no deadline, ignored the tool AbortSignal, and the slide cap only applied after a successful parse. Reject oversized decks before parse and terminate the worker on abort or parse timeout so the existing parse-failed path can return.	2026-08-28 16:20:22 +08:00
wyuc	fcb464bdf0	fix(dsl): migrate away legacy rotate/height fields on line elements (#1261) ... * fix(dsl): strip legacy rotate/height from line elements on migration and import (#1260) * fix(dsl): cover whiteboard slides in the legacy line strip, drop redundant import wiring Round-1 review findings (#1261): the strip only walked scenes[*].content.canvas.elements, leaving the same bug class latent on interactive whiteboard slides (scenes[*].whiteboards[*].elements); and the import-site sanitize was redundant because the document store already runs the ladder on save, while its comment misdescribed that mechanism. Extend the strip to whiteboards and let the ladder alone own the cleanup. * fix(dsl): cover stage-level whiteboard boards and row envelopes in the legacy line strip Round-2 terminal audit findings (#1261): the strip walked only the document envelope's scene surfaces, so dirty line elements under stage.whiteboard (the stage-level explainer boards) and under bare Scene/Stage row envelopes passed through while the runner stamped the document current. Walk every line-element surface of every migratable envelope: scene canvas, scene whiteboards, and stage whiteboard, at document, Scene-row, and Stage-row roots. * fix(dsl): gate the canvas strip on the slide discriminant; run the ladder on legacy-only exports Round-3 terminal audit findings (#1261): the canvas walk fired on any canvas-shaped content regardless of the scene kind, so a non-slide scene carrying a canvas-shaped app extension would have fields deleted from it — gate on content.type ('slide' or absent, the dirty-line epoch predates schema enforcement). And the lock-free exportDatabase fallback hand-stamped DSL_VERSION onto a payload the ladder never walked, so a backup could read as current on restore and permanently skip the migration — build the export unstamped and let migrate stamp it. * fix(agent-runtime): bring stale-stamped documents current before incremental scene writes Round-4 audit finding (#1261): putScene rejects documents whose stored DSL stamp is older than the current one, and the aggregate read migrates in memory only, so the first server-side tool write into a course stored at an older version (every pre-bump course, including previously imported ones) failed with a version error. Route the server tools' incremental scene writes through a wrapper whose not-current fallback reloads the migrated aggregate and full-saves it with the scene spliced in — the server-side counterpart of the app autosave's catch-and-full-save. * style: prettier * docs(agent-runtime): record the reviewed concurrency window on the stale-stamp fallback	2026-08-28 14:06:03 +08:00
wyucandClaude Fable 5	04621578de	release: OpenMAIC 1.0.0 — the agent workbench (#1228) ... * feat(storage): add an agent-session store with PG backend and layered contracts (#1163) * feat(storage): add agent-session store with PG backend and layered contracts * test(storage): avoid BigInt literals for pre-ES2020 root typecheck * fix(storage): close agent-session store review findings * docs(storage): align hook ordering and contention-probe claims with the code * ci: run on the agent-workbench integration branch * chore(storage): bump to 0.5.0 for the agent-session store * fix(storage): carry replay compaction across page boundaries * feat(agent): add the driver model contract and stage route dialect (#1165) * feat(agent): add the driver model contract and stage route dialect * fix(agent): validate route context windows and clarify dialect precedence * feat(agent): adapt the agent-session store and runtime foundations (#1167) * feat(agent): adapt the agent-session store and runtime foundations * feat(agent): resolve request owner identity via an anonymous cookie * docs(agent): document the opt-in compaction default and harden edge cases * feat(agent): add the background session runner (#1169) * feat(agent): add the background session runner * feat(agent): wire the runner into startup behind feature flags * fix(agent): stop clean interruptions from consuming the attempt budget * fix(storage): charge the attempt budget for abandoned leases but not clean parks * docs(storage): document the attempt-charging contract and decouple its tests * feat(agent): add agent session and owner event streams (#1170) * feat(agent): add agent session and owner event streams * fix(agent): close the session-existence oracle and document the owner seam * feat(agent): add agent session lifecycle routes (#1171) * feat(agent): add agent session lifecycle routes * fix(agent): validate session-create input and preserve the owner cookie on errors * refactor(storage): drop the unused active-stage API from the agent-session contract (#1174) * refactor(storage): drop the unused active-stage API from the agent-session contract Tools address stages explicitly on every call, so the store keeps no mutable session-level stage pointer. Removes resolveActiveStage and setActiveStage from the store interface, their PG implementations, the active_stage_changed lifecycle event, the session_active_stage owner event variant, and the contract tests pinning them. The active_stage_id column and the DDL check constraint stay untouched for schema compatibility. * chore(storage): bump @openmaic/storage to 0.7.0 for the contract removal * docs: document the agent runtime configuration surface (#1176) * fix(agent): repair orphaned and late tool results across interruption boundaries (#1180) * fix(agent): repair orphaned and late tool results across interruption boundaries A crash, shutdown, or provider failure can leave the durable transcript with tool calls that have no result, or with results ordered illegally for the provider. Three failure modes were fixed: - Orphaned tool calls: a run that died between an assistant tool-call frame and its result left a dangling call in the entry tree. Resume no longer synthesizes and persists receipts for it: interrupted results are a read-time provider view owned by a shared read-boundary repair, which returns the original array for a healthy transcript and never mutates the tree. - Late parallel results: a parallel tool can finish while pi unwinds an aborted assistant frame, leaving result(A), assistant(aborted), result(B) in durable order. Strict providers reject non-contiguous results, so the read-boundary repair moves existing results next to their owning assistant frame (in call order), omits incomplete unwind frames, and synthesizes receipts only for genuinely missing calls. - Interrupted calls at the write boundary: a call still in flight when the run winds down (shutdown, lease loss, cancellation, provider failure) had no receipt at all. The runner now tracks in-flight calls from their assistant frames and, before the terminal flush, appends an interrupted-result receipt for each still-orphaned call through the same attempt-fenced write chain, so a lease-stealing zombie never writes and the next claim sees a provider-safe transcript. * test(agent): pin the runner wiring for interruption-boundary tool repair * feat(agent): add neutral tool foundation libraries (#1184) * feat(agent): register a web_search tool on the session runner (#1185) * feat(storage): add a per-session URL trust gate (#1186) * feat(agent): add the skills system (#1189) * feat(agent): add the skills system (builtin directories and durable user skills) * fix(storage): serialize the user-skill quota check-and-insert per owner Two concurrent creates at the 50-skill boundary both counted 49 rows and both inserted (READ COMMITTED, no lock), overshooting the quota contract. The create transaction now takes a per-owner pg_advisory_xact_lock first, and the same-name idempotency check runs before the count check so an at-least-once retry of the create that committed as the owner's 50th row still returns its durable receipt instead of a quota error. The 23505 backstop is retained for writes that do not take the lock. * fix(agent): share unstorable-character validation and align skill lookup * feat(agent): add session materials and a fetch_url tool behind the URL trust gate (#1190) * feat(agent): add session materials and a fetch_url tool behind the URL trust gate * fix(agent): harden session material fetching * feat(storage): add an ownership scope to stage documents (#1191) * feat(agent): add material read and search tools (#1192) * feat(agent): add stage read and patch tools (#1194) * feat(agent): add page generation and deck editing tools (#1198) * test(storage): keep the PG contract suite order-independent (#1200) * fix(agent): revoke deleted-session URL authority and reject private ISATAP endpoints (#1199) * fix(storage): revoke deleted session URL authority * fix(ssrf): reject private ISATAP endpoints in strict fetches * chore(storage): bump to 0.11.1 for the session-URL authority fix * feat(agent): add roster and voice registration tools (#1201) * feat(agent): add folder organisation tools (#1202) * feat(api): add stage and material HTTP routes (#1203) * feat(workbench): add the client data layer (#1204) * feat(workbench): add the client data layer * docs(workbench): write the ported comments in English * chore(edit): remove the in-editor agent panel (#1210) * chore(edit): remove the in-editor agent panel * style: apply prettier formatting * fix(agent): report the runtime as unusable without a database (#1207) * fix(agent): report the runtime as unusable without a database * style: apply prettier formatting * feat(agent): add image, video and pptx import tools (#1211) * feat(workbench): add the agent chat surface (#1205) * feat(workbench): add the agent chat surface * docs(workbench): write the ported comments in English * fix(workbench): label the folder and rename tools on the timeline * fix(workbench): label the roster and voice tools on the timeline The reconciliation test iterates every tool the runner registers and requires a display label of its own. The roster and voice-clone tools (list_voices, set_roster, clip_audio, register_voice) reached the integration base with the roster/voice-registration tools but never gained presentation rows, so they fell through to the default branch and rendered their wire names. Port their rows from the reference implementation (labels and i18n keys verbatim) and extend the reconciliation allowlist with ROSTER_TOOL_NAMES and VOICE_CLONE_TOOL_NAMES, so a future tool cannot enter the product without a label. * feat(agent): add the material extraction lifecycle (#1212) * feat(storage): add material extraction lifecycle * feat(agent): execute queued material extraction * style: apply prettier formatting * style: satisfy prefer-const in the extraction runner * test: give material fixtures the extraction lifecycle fields The media-tools slice and the extraction lifecycle slice were each green in isolation but never compiled together: the lifecycle made derivedFrom and extraction required on AgentSessionMaterial while the media-tool fixtures predate them. * chore: remove stray task notes * fix(workbench): label the extraction lifecycle tools on the timeline * feat(workbench): add the workspace shell (#1206) * feat(workbench): add the workspace shell * docs(workbench): write the ported comments in English * i18n(workbench): align workspace keys across locales * fix(workbench): adopt the landed data layer and label the extraction tools - replace the sibling-slice seam stubs with the real data-layer modules - drop ambient declarations now shadowed by landed files - port timeline labels for the extraction lifecycle tools from the reference - align the new i18n keys across all locales * ci: retrigger * feat(api): folder routes, stage-meta viewer surfaces, and the material upload contract (#1215) * fix(storage): restore capability-based stage access * fix(api): bind document access to request owner * fix(agent): restore three-state stage access on the tool layer Port probeStageAccess and the three-state StageAccess (owned / foreign / missing / tombstoned) and gate every stageId-bearing stage tool on an owned probe, mirroring the reference per tool: - move_to_folder, rename_stage, read_stage_outline refuse a non-owned stage with the single not-yours message before touching the store. - The course/DSL toolset and the roster toolset are wrapped by withOwnerStageAuthorization: read_stage, patch_stage, grep_stage and every writer refuse a foreign stage with the same message and refusal shape. - Scene preview keeps its own probe and its own refusal text, and is registered beside the course toolset (never double-gated). - The runner injects one probe factory at the three call sites. Tests: the dsl cross-owner test premise (a foreign stage is readable by id) encoded an invented capability-read policy that the reference does not have at the tool layer; it now asserts foreign read/patch/grep are all refused while the owner still reads. Curriculum cross-owner assertions were already the reference's and now pass with the probes in place. * docs: correct per-file test counts in the fidelity report * test: fix type errors in stage-access fidelity test * test: adapt media-tool and gate suites to the owner-scoped store seam * feat(api): add owner-scoped course-folder HTTP routes Port the reference implementation's /api/folders family (list, create, rename, delete with ungroup/remove modes, and folder membership) onto the owner-bound document store, replacing its provider-based auth with the existing withRequestOwnerId / owner-scoped store seams. The storage package's folder store grows the pieces the routes need: DocumentFolder.order (schema column + max+1 assignment + ordering), renameFolder, deleteFolder(mode) with captured member ids, and setStageFolder(stageId, folderId | null) with idempotent un-filing. FolderNameError moves into folder-name-validation.ts (stage-storage re-exports it, keeping import sites intact). Every route gates on the configured agent runtime (plain 404 when off or unconfigured), keeps the reference's machine codes and envelopes, and is covered by gate tests plus a behavior suite. * feat(api): add stage-meta viewer surfaces for the classroom Port the reference implementation's viewer-facing stage state — can-edit / collected / published / generation-complete — on top of the stage-access base (stage_meta + tombstones). stage_meta gains published_at and generation_complete columns plus a stage_bookmarks table; the reference's deployment-specific origin/claimed_at columns are stripped. New gated routes: GET /api/stage-meta/[stageId] (per-viewer facts, 404 for absent/tombstoned, never returns the owner id), GET /api/stages/[id]/status, POST generation-complete / publish / unpublish (owner-only), POST /api/bookmarks. The resolver lives in lib/server/stage-access.ts. Wiring: a fetchStageMeta client with the reference's three-outcome contract, stage-store isOwner/isBookmarked/readOnly fields (upstream single-user defaults, no-op until the sidecar answers) plus setViewerAccess, the classroom apply path computing readOnly = !(isOwner || isBookmarked), the Stage editability gate, and a sidecar probe after each classroom load. A sidecar 'absent' answer keeps the editable default here because the classroom also serves local-only courses; server writes stay owner-enforced. * feat(api): port the reference material upload contract Rewrite POST /api/materials to the reference implementation's upload shape so the workbench uploader (uploadWorkbenchMaterial, which posts no session id and expects a flat 201 view) works unchanged: owner-scoped upload with mime normalization/validation (415), per-class size caps checked on the declared content-length and the streamed body (413), empty body (400), quota (429), sha256 reserve->store->finalize lifecycle with abandon on failure, flat { materialId, originalName, bytes, mime, extraction } 201, and an x-request-id echo. Adds the owner-scoped material library (owner_material table + quota + 24h lazy sweep, bytes in the host's asset registry as the neutral replacement for the reference's object-storage byte path) and the material cap configuration. The session-scoped GET list is left as-is; the reference's owner-material extraction worker is not ported (the branch's session-material extraction lifecycle already covers extraction). Gate tests now cover all 23 persistence routes across the three runtime env states; the materials behavior suite pins the new contract. * feat(media): add an optional local ffmpeg media extractor (#1213) Adds a local ffmpeg/ffprobe pipeline as a second media extraction provider behind the extractor registry, ported faithfully from the reference implementation: duration probing, keyframe-safe chunking, per-chunk ASR with timeout and deadline budgets, and timestamped transcript assembly. - Availability probing feeds the registry's candidate selection: the provider simply is not a candidate when ffmpeg/ffprobe are absent. - With neither ffmpeg nor a cloud provider configured, extraction fails with an actionable message naming both enablement paths. - Media materials route through the same extraction lifecycle and lease fence as documents; no parallel queue. - Tests inject the executable resolver so the missing-ffmpeg path is the default-tested one; the real pipeline test is skip-if-unavailable. - @openmaic/storage 0.13.0 -> 0.14.0 (media routing in the material lifecycle surface). * feat(storage): per-scene monotonic revisions via database triggers (#1214) * feat(storage): per-scene monotonic revisions via database triggers Restore the reference implementation's freshness granularity: a per-scene monotonic revision maintained by database triggers, so every writer (HTTP routes, agent tools, jobs, manual SQL) bumps it without application cooperation. - Companion revision tables + trigger functions in the storage package's idempotent schema bootstrap, with the lock-order invariant, pg_notify wakeup and the suppression switch for batch writers. - ensureDocumentSchema gained a dollar-quote-aware statement splitter. - The freshness and manifest routes serve per-scene revisions. - Mutation-verified: dropping the triggers turns the revision tests red. - @openmaic/storage 0.13.0 -> 0.14.0. * fix: forward the freshness manifest through the owner-bound store * feat(workbench): add the Pro entry points and preserve the mode-transition semantics (#1208) * feat(workbench): add the Pro entry points * feat(workbench): preserve Pro mode transition semantics * fix(workbench): drop ambient declarations shadowed by landed slices * fix(workbench): drop ambient declarations shadowed by the landed shell * feat: port workspace shell sibling modules Port the 16 leaf modules the Pro workspace shell imports but that were only ambient-declared, replacing the compile-time bridge with real implementations adapted from the sibling-slice reference: pure workbench helpers (session title, rail tab, course-chat bootstrap, created-course tabs, course-tabs memory, workspace navigation, pane navigation, pro-edit sizing, existing-course minting, first-message session), the neutral brand context and course-rename server API, the server-action session delete, the home discovery hook, the classroom pane host with its load-policy leaf, the theme toggle and floating-layer owner, plus the floating-layer-owner wiring the dialog/dropdown/tooltip portals stamp. Also add the workbench-shell locale copy for all 12 locales, port the reference tests for the ported modules, and drop types/workbench-sibling-slices.d.ts now that every declaration has a real implementation. * docs: keep ported comments in English and deployment-neutral * docs: announce 1.0.0 and refresh the feature overview (#1216) * docs: announce 1.0.0 and refresh the feature overview * docs: finalize 1.0.0 README after feature merge * fix(agent): control-plane routes answer 404, not 500, without a database The agent control-plane routes gated only on the runtime flag, so an enabled-but-unconfigured deployment (flag on, DATABASE_URL empty) answered 500 from a store that cannot connect. Gate them on the configured check instead, matching the stage/material routes: the whole surface is cleanly absent until both the flag and the database are present. The status probe keeps reporting both bits. * test: mock both runtime gate exports in the control-plane route suites * fix(agent): abort in-flight TTS on cancel and bound each provider request with a timeout (#1217) The generate_tts / scene-tts path checked the runner's AbortSignal between actions but never created the provider HTTP requests with it, so a session cancel left a hung synthesis fetch in flight until a restart repaired the tool result. Thread the signal end-to-end: TTSModelConfig carries an optional signal, generateTTS combines it with a per-request timeout (TTS_REQUEST_TIMEOUT_MS, default 30s, ported from the reference runtime's TTS bounds) via AbortSignal.any, and every provider fetch (openai, azure, glm, qwen incl. voice-clone + audio download, voxcpm, minimax, doubao, elevenlabs, lemonade) is created with that signal. A timeout now fails the tool call with TTSRequestTimeoutError (a clear retryable error) instead of wedging the session; a caller cancel propagates as the interruption so the runner settles the session as cancelled without a restart. Tests: hung-provider simulation rejects at the timeout with the retryable error; abort mid-flight aborts the captured request signal and surfaces the interrupted shape; removing the signal wiring makes the abort tests fail (red), restoring them turns green. * fix(workbench): PG-mode home listing via owner stages; keep the interrupted terminal course card (#1218) Finding 1: with server persistence on, listStages resolved to the generic GET /api/persistence/documents listing, which the capability model deliberately answers 403 FORBIDDEN_DOCUMENTS for (reads by id, listings owner-only). The home/workspace library now lists through the owner-scoped GET /api/stages surface (same anonymous-owner cookie the workbench uses) when server persistence is enabled; the server-side 403 is untouched. Finding 2: a run interrupted (session_interrupted) and repaired (session_resumed) that ends cancelled before agent_end stranded its pending classroom sightings, so the timeline's terminal card lost the course the answer produced. session_end (cancelled) now flushes the pending sightings into the same course card set agent_end paints, before the stopped caption. * chore(workbench): remove the bookmark concept and the saved-courses drawer (#1219) * chore(classroom): remove the bookmark ('collected') concept entirely The stage-meta viewer port introduced a bookmark surface (stage_bookmarks table, POST /api/bookmarks, the isBookmarked sidecar field, and a readOnly rule that let a saved course stay editable). The product has no such concept, so remove it as a closure: - delete the /api/bookmarks route and the stage_bookmarks table plus its query helpers from the persistence bootstrap - drop isBookmarked from GET /api/stage-meta/[stageId] - simplify the classroom read-only rule to readOnly = !isOwner across the sidecar client, ownership signal, classroom load, stage store and the classroom page - keep publish/unpublish, generation-complete, isOwner and isPublic exactly as they were - update the gate and stage-meta route suites and the README mentions The workspace rail's Bookmark glyphs and comments describe the upstream saved-courses (favorites) section, which is driven by isOwner and renders no collect affordance; they are kept as unrelated homonyms. * chore(workbench): remove the saved-courses drawer UI The first pass removed the bookmark data model but kept the rail's "Saved courses" drawer, judging it a separate surface driven by `isOwner === false`. The home/workspace listing is owner-scoped, so that flag can never occur: `allSaved` is permanently empty and the drawer (plus the collapsed-rail Bookmark mini-button) is a dead affordance. Remove it: the SavedDrawer component and its mount, the savedOpen / savedSection state, the allSaved / matchedSaved derivations, the 'saved' variant of the course-list renderers, the mini Bookmark glyph, the drawer-only CSS, and the drawer's i18n keys from all 12 locales. The courses tab is now exactly one folders tree. The authored/favorites split in workspace-tree.ts goes with it; the tree module no longer reads `isOwner`. The discovery course type keeps the field — the shell still reads it for read-only gating. Upstream has no collect concept; the drawer could only ever render empty here. The reference implementation HAS this drawer (its favorites come from its account system), so this removal is a deliberate upstream product decision, not a fidelity bug. * fix(workbench): restore the attach entry, add the rail settings entry, pin all three entry points (#1221) * fix(workbench): restore the composer attach entry by gating it on the live runtime The AttachButton's rollout probe read a `materialsEnabled` field that this branch's /api/agent/runtime never answers (the materials routes gate on the runtime itself, like the stages), so the gate could never pass and the attach button never rendered — the Pro launch and chat composers showed only the @-mention and enhance glyphs. Substitute the field with the runtime's `enabled` value, which IS the upload action's precondition: POST /api/materials answers 404 whenever it is false, so the render condition now equals the action precondition (no dead button). The button's label (`proMode.attach`) is a user-visible string that becomes visible again; port the reference implementation's own translations verbatim into the 11 locales that still carried the Chinese copy. * feat(workbench): add the settings entry to the rail's bottom-left cluster The reference's rail foot carries a cluster of utilities (its saved-courses drawer, the language switcher, the display toggle). This branch removed the drawer — it could only ever render empty here — and the product decision is to fill that freed spot with the settings entry. Add a settings trigger to the foot cluster (expanded rail, beside the language and display toggles, and on the collapsed strip) and mount the model/provider SettingsDialog in the rail, wired to the trigger. It is the same dialog the classic home opens from its header pill; the workspace had no settings entry of its own, so nothing is duplicated within a surface. * test(workbench): pin the restored upload, attach, and settings entry points Covers the three restored entry points: - the courses-tab upload control: rendered beside the course name filter, wired to the discovery hook's ZIP import trigger, disabled while an import runs, and gated by the same condition as its action (the courses tab); - the composer attach control: an actual render of AttachButton under both probe answers (visible when the runtime says the upload path is live, hidden otherwise), its mounts in the launch and chat composers, the branch's runtime-field substitution in the probe, and the reference's own `proMode.attach` copy in all 12 locales; - the settings entry: the trigger in the rail's foot cluster (expanded and collapsed), beside the language and display toggles, opening the SettingsDialog the rail mounts. * chore(config): the Pro workbench flag implies the MAIC Editor gate (#1223) A workbench build without the editor toggle has no way to edit a course: enabling NEXT_PUBLIC_PRO_WORKBENCH_ENABLED while forgetting NEXT_PUBLIC_MAIC_EDITOR_ENABLED produced exactly that split-brain bundle. The workbench IS Pro mode, so its flag now implies the editor gate; the standalone flag remains for deployments that want the classroom editor without the workbench. Documents both flags in .env.example. * fix(agent): wake SSE tails and the runner on durable deltas (streaming fidelity) (#1222) The Pro workbench chat did not stream: the session/owner SSE routes polled the durable event log on a 5s/30s clock with no wakeup, so message_update deltas (written at 150ms cadence) reached the browser in poll-sized blocks and the thinking strip only mounted after the whole reasoning text had accumulated. Port the reference's LISTEN/NOTIFY delta path: - storage: add in-transaction wake hooks (onSessionEventAppended, onOwnerEventAppended, onCancelRequested) so a host queues pg_notify in the same transaction as the durable append; align readEventsAfterForReplay to rank the bounded page so the first delta after the cursor is always kept (the live tail can never starve). Bump @openmaic/storage to 0.18.0. - app: port the process-wide event-notify bus (dedicated LISTEN client, self-check probe, reconnect backoff; notify through the storage transaction surface), wire the store hooks, subscribe both SSE routes before the initial read with the reference's initializing gate, and give the runner one {kind:'session'} subscription whose wake runs the cancel check and the message drain. Polls stay as the lossy-NOTIFY backstop. - lifecycle: start/stop the bus from instrumentation. Tests: storage hook + compaction contract; route wakeup latency; runner wakeup wiring with a fake agent; bus unit tests; PG contracts proving a real append wakes the routes and a live SSE route forwards a message_update on the wakeup, and that a rolled-back append never wakes. Also fix the pre-existing park-attempt-budget PG test TRUNCATE (missing CASCADE against newer FK tables). * fix(storage): asset writes self-deadlocked against pooled PostgreSQL (#1225) * fix(storage): refuse the non-transactional byte-write deadlock configuration A byte store whose plain write() runs on its own pooled connection cannot be invoked from inside a registry write transaction: after the transaction has claimed the blob-row lock, that write blocks on the lock the transaction just took while the transaction waits on the write - a self-deadlock PostgreSQL cannot detect (one side is idle in transaction). There is no lock-safe ordering for such a writer: bytes must be written after the row claim (writing before it lets the collector delete the bytes while the upsert waits), and any second-connection write after the claim is the deadlock. The configuration is therefore detected and refused: - AssetByteStore gains writesOutsideRegistryDatabase?: true, declaring that the layer's plain byte operations cannot contend for the registry's row locks. - PgAssetStore refuses put()/replace() up front (and defends coordinatedWrite) when the byte store has no writeWith and does not declare the flag, throwing a clear configuration error before any row is claimed. - The collector mirrors the guard on its delete path (deleteWith or a declared out-of-registry layer, else a configuration error). - The object store declares the flag (its out-of-transaction write remains legitimate); the in-registry PostgreSQL byte column provides writeWith / deleteWith instead. - Write transactions (put/replace/remove) set SET LOCAL lock_timeout = 30s so any future lock-contention variant fails loudly instead of hanging. Bumps @openmaic/storage to 0.18.0. * fix(persistence): forward the transactional byte methods through the lazy asset byte-store wrapper The no-bucket case of lazyAssetByteStore returned a bare { write, read, delete } and dropped writeWith/readWith even though the underlying PgAssetByteStore has them. The registry's hasTransactionalWriter duck check then failed and put() fell back to the byte store's own pooled connection, which blocks forever on the blob-row lock the registry transaction just took when the bytes live in the same PostgreSQL - the production self-deadlock. The no-bucket layer is statically PgAssetByteStore, so its transaction-pinned methods are forwarded eagerly (typed against the real signatures via PgForwardedByteStore). The bucket case keeps its lazy-probing semantics: no transactional writer exists there, the signed-URL method stays absent or lazy exactly as documented, and the wrapper now declares writesOutsideRegistryDatabase so the registry may run the plain write inside its transaction. New tests pin the wrapper's transactional capability red-to-green and assert put()/resolve() route byte traffic through the transaction-pinned queryable. * fix(home): cap the generate-prep ingest drain at 3s so Generate never waits the full server budget The classic home flow's Generate click drained in-flight ingests for the full 15s server budget. Cap the wait at GENERATE_DRAIN_CAP_MS (3000ms, documented as a UX bound) and reuse the existing timeout fallback: sources that miss the cap proceed on the legacy byte path and each late-resolving id is released. * chore(storage): bump to 0.19.0 over the concurrently landed 0.18.0 * fix(agent): bound every tool call with a timeout; never resurrect a cancelled session (#1226) * fix(agent): bound every tool call with a global timeout and settle it on cancel A tool await that neither resolves nor rejects wedges the session forever: the lease keeps heartbeating and the driver never reaches its next cancel checkpoint. Race every tool execution (in buildAgent) against a hard budget (OPENMAIC_AGENT_TOOL_TIMEOUT_MS, default 10 min, per-tool overrides for known long runners) and against the caller's AbortSignal, so even a signal-ignoring await cannot keep a cancelled session running. On timeout the call rejects with AgentToolTimeoutError; the agent loop turns the rejection into a structured error tool-result the agent can retry or proceed from, and the abort signal is delivered to the tool's in-flight work through a derived controller. Zombie-tool updates after settlement are dropped. * fix(storage): never re-lease a cancel-requested session; settle it as cancelled on claim The claim scan treated a session with cancel_requested_at set as a normal claim candidate: after a restart it re-leased the same session for attempt N+1 and resumed generating despite the pending cancel. claimNextSession now settles such candidates as cancelled under the claim lock (status cancelled, attempt reset, lease and cancel request cleared, terminal session_end event and owner projection) instead of leasing them, then keeps scanning. Bump @openmaic/storage to 0.18.0. * docs: takeaway-style 1.0.0 announcement with bilingual guide links The 1.0.0 head is now a short takeaway block — badge links to the official user guides (English and Chinese), five one-line highlights, and pointers into Features and the workbench setup section — instead of six dense paragraphs. The detailed provider-neutrality and freshness notes move into the Features workbench section, phrased database- neutrally (the announcement no longer names a specific database). Release date corrected to August 27. * fix(workbench): restore editor chrome, mode transition, streaming, materials, mentions, folders (#1229) * fix(workbench): wire workspace folder routes * fix(editor): restore reference workbench chrome * fix(workbench): persist composer materials and course refs * fix(workbench): preserve live reasoning frames * fix(persistence): back off failed streaming saves * chore(workbench): retire stale slice seams * test(editor): cover element pin layer * chore(storage): bump to 0.21.0 for the user-message ref/material fields * chore(editor): translate ported code comments to English * fix(agent): fence durable tool writes and consume cancel requests atomically (#1230) * fix(agent): enforce provider force-off in agent tools and scrub vendor identity from tool results (#1231) * fix(materials): serialize per-owner quota reservations and make crashed uploads reclaimable (#1232) * fix(editor): resolve dock-bar i18n keys, remove dock height drag, wire element referencing (#1233) * fix(workbench): send the opening session message exactly once with refs intact (#1234) * feat(editor): port timeline TTS preview single-flight and voice-all state latching (#1235) * fix(media): restore the reference classic media chain (#1236) * fix(import): adapt imported PPTX canvas size so decks render without overflow (#1237) * fix(editor): complete element referencing — renderer DOM contract and GenUI picking aligned with the reference (#1238) * test(providers): reconcile the provider-config vendor-token debt count after the main merge The integration line's AK/SK fallback for the managed document provider adds occurrences that main's allowlist snapshot predates. Same mixed-composition debt category the group already documents; no new vendor behavior. * test(providers): reconcile vendor-token debt counts with the integration line The main-merge brought main's neutrality-guard snapshot next to integration features it predates (media-extractor fallback chain, local voice-profile deletion semantics, the enabled-TTS helper). Same debt categories the guard already documents; counts updated to the guard's own tally and two grouped entries added. No new vendor behavior. * fix(agent): carry reasoning through the completions dialect so the thinking strip renders (#1239) * feat(skills): add Feynman and spiral curriculum methods (#1240) * feat(agent): port missing reference tools and skills (parity audit) (#1241) * feat(media): retire asset-registry wiring; media and materials follow the reference byte model (#1242) * fix(classroom): center adapted canvases in the stage and send back navigation home during generation (#1243) * feat(settings): skill management with real list, download, delete, and upload (#1244) * feat(settings): skill management section with real list, detail, and zip download * feat(skills): owner skill delete and upload across storage, API, and settings * fixup! feat(settings): skill management section with real list, detail, and zip download chore: neutralize a reference note in the settings header comment * fix(media): persist origin-independent classroom-media references from the agent runtime (#1245) * feat(editor): float the insert toolbar in the outer frame with collapse (#1246) The insert strip was bounded to the slide card, so it could only ever sit on top of slide content: the card's overflow clipped it and it could not be parked in the padding beside the slide. Move it into the studio frame the element picker's panel already roams (CanvasOverlayPortal + the frame selector), so both canvas overlays share one bounding container and their handles behave the same. While picking, the strip rises over the picker and goes inert, which is the z-order CANVAS_OVERLAY_Z already documents. Add a fold beside the grip: the chevron collapses the strip to that grip row and back, with the buttons unmounted rather than hidden. The fold is session-local state owned by EditShell, next to the drag offset, so a surface swap keeps it; nothing is persisted. Expanding a strip parked at the bottom edge re-clamps through the same bounds rule the keyboard move uses. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(workbench): align the chat timeline's left edge with the composer (#1247) * fix(agent): fence session claims while an ask_user question is outstanding (#1248) * fix(agent): settle-time rescue tracks real delivery instead of a count offset (#1249) * fix(persistence): migrate owner_material to oss_key and drop legacy asset_id (#1250) * docs(readme): surface the 1.0.0 user guide badges at the top (#1253) * fix(workbench): show newly created folders in the sidebar without reload (#1254) * docs(readme): add the release version prefix and drop the opt-in framing * fix(workbench): single-source the chat gutter so timeline and composer share a left edge (#1255) The transcript and the composer each established their own column: their own `px-*` gutter and their own `mx-auto w-full max-w-*` centering wrapper. Equal padding values were never enough, because the two columns are centered inside different containing blocks — the transcript's is a scroll container, whose content box is narrower than the composer footer's by the scrollbar's width: transcript text left = pad + (pane - 2*pad - scrollbar - measure) / 2 composer box left = pad + (pane - 2*pad - measure) / 2 The padding cancels out of the difference and what remains is `-scrollbar/2` at every padding value, so the transcript sat half a scrollbar to the left of the composer and tuning the two paddings against each other could not move it. The column is now established once, by the nearest common ancestor of both (`chatColumn`), and the scroll viewport and the composer footer are siblings inside it that add no horizontal inset of their own. The cap carries the gutter on top of the 760px reading measure, so the text column keeps its width. The handed-over question row drops the padding that indented it past the agent's prose; framed rows keep their own inner padding, which is what a card's border sitting on the column edge means. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(workbench): lock pane-embedded classroom to edit mode (#1256) The workspace right pane painted the full learning chrome — speed control, play button, learner avatars, mic bar — for a course the agent had just created, then flipped to edit once the first scene landed. resolveStageChromeMode treated playback as the DEFAULT branch for a hosted classroom, so every shortfall fell into it: a course whose tab opens at stage_link time has no scenes yet, so currentSceneId is null and isHostedSceneEditable is false. A folded pane parked the playback root behind the fold and cross-faded it out over the pane on unfold, and a failed editor chunk dropped into playback permanently. Lock it at the pane instead of defaulting per entry path: - WorkbenchPanelProvider — the single element that mounts a classroom into the workspace — publishes editPinned (visible && !playback). Every entry path passes through it, so none of them decides. - The hosted resolution can no longer degrade to playback. Start Learning (workbenchLearning, new input, split out from pane visibility) is the one door; everything else resolves between the neutral loading shell and edit. - Stage's chrome dispatch is exhaustive on chromeMode, so the playback root is no longer the else-branch of a condition about the current scene. No flicker: chromeMode is resolved during render, and preloadEditor now answers synchronously (isEditorPreloaded) so a remount with the chunk already registered paints edit on the first frame. A failed import is no longer cached forever, so the lock cannot strand the pane. Standalone classrooms keep their stored mode unchanged. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-08-27 19:26:25 +08:00
wyuc	98904c1c5d	fix(providers): three silent-behavior fixes from the provider audit (#1196) ... * fix(extract): fail loudly instead of silently falling back to MinerU Cloud A request that selects the self-hosted MinerU extractor without a configured base URL used to fall back to MinerU Cloud whenever a cloud API key was available, sending documents to a third party without the operator's knowledge. The cloud fallback now requires an explicit operator opt-in (ALLOW_MINERU_CLOUD_FALLBACK, default off, documented in .env.example); otherwise the route answers a 422 that names what was configured (self-hosted MinerU) and what was unavailable (its base URL), and points at both remedies. * fix(media): remove the selectable video provider that has no adapter The Sora entry in the video provider catalog could be selected in the settings UI and server config, but had no connectivity or generation dispatch case, so choosing it failed only at execution time. A catalog entry that cannot execute is worse than an absent one: the entry, its type-union member, UI names/icons, store defaults, and the VIDEO_SORA env mapping are removed. A new catalog test pins that every selectable video provider dispatches to its own adapter in both switches. * fix(storage): delete the obsolete no-op storage provider abstraction getStorageProvider() unconditionally returned a NoopStorageProvider and the module swallowed every operation into silence, so any caller believing it had storage got neither bytes nor an error. No real caller exists anywhere in the repo (only lib/storage/client.ts, a separate module with a documented null contract and a live caller, remains). The dead entry point, its type file, and the no-op provider are deleted; a test pins the removal and that the real client upload helper is kept.	2026-08-27 11:49:23 +08:00
wyuc	06f105ed4d	feat(server): validate model routing config at startup (#1182) ... * feat(server): validate model routing config at startup and deprecate bare model ids * fix(server): keep the bare-id deprecation warning out of request paths parseModelString is reachable with request-controlled strings, so warning there lets clients drive log volume and grow the dedupe set without bound; the warning now fires only from boot-time validation of config-derived sites, whose surface is finite. DEFAULT_MODEL missing-key wording no longer claims requests will fail (unrouted resolution honors client keys), and a keyless provider with a default base URL is not flagged as unconfigured.	2026-08-26 10:59:47 +08:00
wyuc	b8fbac1a54	feat(providers): uniform capability force-off and consistent missing-key contract (#1181) ... * feat(providers): uniform capability force-off and consistent missing-key contract * fix(providers): close force-off bypasses found in review	2026-08-26 10:59:43 +08:00