trending/OpenMAIC
1
0
Fork 0
mirror of https://github.com/THU-MAIC/OpenMAIC.git synced 2026-10-21 06:10:53 +08:00
Code Issues Packages Projects Releases Wiki Activity
648 Commits 303 Branches 136 Tags
@openmaic/generation@0.3.14
Commit Graph
125 Commits
Author SHA1 Message Date
HEOJUNFOandClaude Opus 5.5 1c70e86a13 fix(media): poll and download Veo videos the Gemini API way (#1695) 
* fix(media): poll and download Veo videos the Gemini API way

The Veo adapter targets generativelanguage.googleapis.com but polled
with Vertex AI's models/{model}:fetchPredictOperation and read inline
response.videos[].bytesBase64Encoded. The Gemini API reads operations
with GET v1beta/{name} and returns
response.generateVideoResponse.generatedSamples[].video.uri, fetched
with the same API key.

- Poll with GET v1beta/{operationName}.
- Download the sample URI with x-goog-api-key into a data URL, always
  through the configured base URL's origin and without following
  redirects, like the other adapter calls. Inline bytes are still
  accepted when present.
- Report RAI-filtered operations as a failure with their reasons.
- Catalog: Gemini API model IDs for Veo 3.1 (-preview, plus Lite) and
  only the 16:9 / 9:16 aspect ratios the API accepts.

Fixes #1694

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(media): follow the Veo download redirect through a validated transport

The Gemini API file URI can answer with a redirect to storage (Google's
own example downloads it with `curl -L`), which the download refused, so
a generation could fail after a successful poll.

- VideoGenerationConfig gets an optional downloadFetchImpl. fetchImpl is
  the pinned transport that refuses every 3xx, so the download needs its
  own one.
- lib/server/media-provider-fetch.ts adds mediaDownloadFetch and
  managedMediaDownloadFetch: providerFetch with redirects followed, each
  hop re-validated under the operator address policy, pinned to the
  vetted DNS answers, and stripped of x-goog-api-key once it leaves the
  origin (fetchWithRedirectValidation). withVideoProviderFetch installs
  both transports.
- The server video call sites (the generate route, classroom media
  generation, the agent runtime) inject it; the adapter still does not
  import server code.
- downloadVideo follows redirects only through the injected transport and
  keeps redirect: 'manual' + assertNotRedirected without it.
- Test: a 302 to another origin yields the data URL and the second hop
  carries no x-goog-api-key; without the transport the 302 is refused.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-28 23:00:04 +08:00
wyucandClaude Opus 5.5 cd7d6a1a59 fix(providers): pin caller-chosen provider requests and keep upstream detail out of errors (#1704) 
* fix(pdf): pin verify-pdf-provider probes to the strict provider transport

The MinerU Cloud and self-hosted connectivity probes validated a
caller-supplied base URL once and then issued a plain fetch that resolved
DNS again, so a rebinding hostname could pass validation and connect to a
loopback or private address. The response also echoed the target's status,
its 401/403 body, and per-errno connection errors.

Both probes now go through providerFetch with reject-redirects and the
operator address policy (the same ALLOW_LOCAL_NETWORKS policy the route
already validates against), so the connect address is pinned to the vetted
DNS answers. A 3xx still maps to REDIRECT_NOT_ALLOWED. Authentication
failures return a fixed message, every other connection failure returns a
single generic message, and the success payload no longer includes the
target status. Details are logged server-side only.

Tests drive the real route, guard and pinned transport against loopback
servers, covering rebinding, auth body suppression, identical
refused/not-found/timeout answers, redirect refusal and the managed path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(pdf): pin self-hosted MinerU parsing to the strict provider transport

Self-hosted MinerU parsing validated a caller-supplied base URL once and
then posted the document with a plain fetch that re-resolved DNS and
followed redirects, so a public host could hand the upload (and API key)
to a loopback or private address.

The /file_parse request now goes through providerFetch under the operator
address policy with redirects refused. Transport failures, policy blocks
and redirects collapse into one fixed message; an unknown error status is
reported without its body (the missing-dependency classification stays),
a non-JSON body no longer surfaces the parser's input snippet, and the
empty-result error no longer lists response keys. MinerU Cloud control
plane, upload and ZIP errors likewise stop quoting raw response bodies.

Tests drive the real parse route and pinned transport against loopback
servers: rebinding, redirect refusal, body suppression, the multipart
upload shape and the managed path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(tts): pin the Azure voice list request to the strict provider transport

The voice list request validated the caller-supplied base URL once and then
fetched it with a plain fetch that resolved DNS again. It also mirrored the
target's HTTP status as the route's own status and returned the full error
body, plus any JSON the target answered on success.

The request now goes through providerFetch under the operator address
policy with redirects refused and a 20s deadline. Non-2xx answers return a
fixed 502 (authentication failures get their own fixed message), only a
JSON array is returned as the voice list, and transport failures share one
fixed 500 message. Details are logged server-side only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(providers): pin model-list probing to the strict provider transport

Model discovery validated the caller-supplied base URL and models URL once
and then issued a plain fetch per candidate, which resolved DNS again. Non-2xx
answers carried up to 512 bytes of the provider's body into the route
response, and a non-JSON success body surfaced the parser's input snippet.

fetchModels now defaults to providerFetch under the operator address policy
with redirects refused (a refused hop keeps the REDIRECT_NOT_ALLOWED
contract), and accepts an injected transport for tests. Error bodies are
never read; the route keeps its 401/404/403 contracts, reports other HTTP
failures by status class only, answers a non-JSON list and every transport
failure with fixed messages, and returns 400 for a malformed request body.

The rejected-redirect detector is shared from the transport module instead
of being copied per caller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(media): run image/video provider requests on the strict provider transport

The image and video routes validated a caller-supplied provider base URL
once, then every adapter issued plain fetch calls that resolved DNS again.
Adapter errors carried full non-2xx bodies into the route responses, the
auth-only probes returned the 401/403 body, and connectivity failures
echoed the transport error text.

Adapters now take a `fetchImpl` from their config and use it for every
provider request (submit, poll, download and connectivity probe). The
adapters are also imported by the settings UI, so they cannot import the
server transport; every server caller (the four routes, classroom media
generation and the agent runtime tools) injects mediaProviderFetch, which
is providerFetch under the operator address policy with redirects refused.

Connectivity results are fixed text: authentication failures, redirects,
other HTTP statuses and transport failures each have one message, and no
provider body is read. The generation routes log the adapter error and
answer a fixed message, keeping the content-safety classification. A
malformed Kling key is still reported before any request.

The rejected-redirect detector moves to a dependency-free module so the
browser-bundled probe helper can share it with the server transport.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(llm): pin LLM calls to a client-supplied base URL

resolveModel validated a client-supplied base URL once and then handed the
AI SDK a redirect-validating fetch whose only dispatcher was the
timeout-only agent, so connect-time DNS was resolved again without the
guard. verify-model also echoed the provider's error message (which can
carry its response body) and distinguished errno classes.

A client-supplied base URL now gets providerFetch under the operator
address policy with redirects refused. The pinned dispatcher accepts
headers/body timeouts, and this path uses the same 15-minute budget as the
default LLM dispatcher; the transport's body normalization and streaming
are unchanged. Operator-configured endpoints keep the existing
redirect-validating transport.

verify-model now classifies failures by the provider's HTTP status only
(401/403, 404, 429, other status class) and answers every transport or
parse failure with one fixed message.

Tests stream a real SSE chat completion through the pinned path and check
that deltas arrive before the server finishes, plus rebinding, redirect
refusal and the managed path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(pdf): accept only official AliDocMind endpoints from clients

AliDocMind requests go through the vendor SDK, which builds its own HTTPS
agent and resolves the endpoint itself, so a client-supplied endpoint that
passed the URL guard could not be pinned to the validated address. The
credential check also returned the SDK's error code and message, which
told a refused port from a TLS failure or a timeout.

When the provider is not server-managed, verify-pdf-provider, parse-pdf and
extract-document (document and media paths) now accept a client endpoint
only when it is an official docmind-api.<region>.aliyuncs.com host over
https with no port, path or credentials, and answer INVALID_URL before any
SDK call otherwise. The accepted endpoint is passed on as the normalized
host. Server-managed endpoints are unchanged. Credential verification
failures now return fixed messages and log the SDK detail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(agent-runtime): download provider result URLs under the strict public policy

The agent-runtime image and video tools downloaded the URL a provider
returned with a plain fetch, validating each redirect hop with the operator
policy but connecting without pinning, and accepting http. A data: URL
from an adapter that inlines its result (the OpenRouter video adapter) was
rejected instead of decoded.

The classroom download helper's policy is extracted into
fetchProviderResultUrl: data: URLs are decoded locally, anything else must
be https and pass the strict public policy (allowLocalNetworks false,
regardless of ALLOW_LOCAL_NETWORKS), and the request goes through
providerFetch, which re-validates redirect hops under the same policy and
pins connect-time DNS. The agent-runtime tools and the classroom helper
share it; the existing bounded reads and content-type checks are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): note pinned provider transports and body-free provider errors

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(providers): keep vendor endpoint rules out of provider-neutral modules

The provider-neutrality guard keeps vendor knowledge out of the capability
routes and model resolution. The AliDocMind endpoint rule now sits behind
provider-neutral helpers (checkClientDocumentExtractorBaseUrl and
checkClientMediaExtractorBaseUrl), which apply the official-endpoint rule
to extractors whose SDK cannot be pinned and the URL guard to the rest.
The pinned LLM transport policy moves out of resolve-model into its own
module. Behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(providers): hold IP-literal request hosts to the transport address policy

The pinned dispatcher judges hostnames in its connect-time lookup, but Node
never runs that lookup for an IP-literal host, so a request to a loopback or
private IP reached it without the local-network opt-in. The provider transport
now checks an IP-literal origin against the same policy before connecting,
so it enforces the policy on its own rather than relying on every caller to
validate the URL first.

Tests that reached loopback IP literals without the opt-in now set it, as a
self-hosted deployment would; cloud metadata stays refused under every policy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(providers): let operator-configured local providers connect without the opt-in

Image/video providers and self-hosted MinerU moved to the pinned transport
under the operator address policy for every base URL, so a server-managed
endpoint on a local network (for example a local Lemonade image server or a
MinerU container) stopped working unless ALLOW_LOCAL_NETWORKS was set.

A server-managed base URL is operator configuration: it now runs with local
networks allowed, still pinned and still refusing redirects, while cloud
metadata and reserved ranges stay refused. Caller-supplied base URLs keep the
operator policy. Media routes pick the transport by the provider's managed
flag; server-internal media generation uses the managed transport; document
extraction carries a `managed` flag to the MinerU parsers and the PDF
verification probe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(llm): pin every caller-chosen endpoint and keep transport detail out of errors

An unmanaged provider picked by the caller without a base URL fell back to its
catalog default (for example a localhost Ollama or Lemonade endpoint) on the
operator transport, which neither validated nor pinned the origin. The pinned
client transport is now chosen whenever the caller picked the model or sent a
base URL, and the effective endpoint (client URL or catalog default) is
validated under the operator policy first. A model the operator selected
through MODEL_ROUTES or DEFAULT_MODEL keeps the operator transport.

Routes relay LLM error messages, and the AI SDK builds them from the fetch
failure cause and from the provider's error body. On the caller-chosen
transport a failed request now surfaces a fixed reason ("connection failed",
"request timed out" or "redirects are not allowed") with the system error
logged server-side, and an HTTP error response reaches the SDK with an empty
body and the standard reason phrase, keeping its status and headers for retry
and status classification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(pdf): keep MinerU Cloud envelope text out of errors and refuse query/fragment base URLs

MinerU Cloud errors quoted the endpoint's envelope `msg`, a failed row's
`err_msg` and the batch id, and the parse routes relay error messages. They now
report the context with the HTTP status or, for a rejected request, the numeric
code only; the endpoint text is logged server-side.

Provider paths are appended to a base URL as text, so a client base URL
ending in `?` or `#` (or carrying a query) absorbs the fixed path and leaves
the request target to the caller. Client-supplied provider base URLs are now
refused when they contain a query string or fragment, across the LLM, media,
document extraction, PDF verification, model probe and Azure voice routes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(media): bound provider data: URL results before decoding them

`fetchProviderResultUrl` decoded a provider-returned `data:` URL in full and
left the size check to the caller's body reader, so an oversized payload was
materialized first. Callers now pass their byte limit, and the payload size is
estimated from the encoded length (base64: 3/4 less padding; percent-encoded:
at least one byte per three characters) and refused over the limit before any
buffer is built, then checked exactly after decoding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): describe provider transport behavior changes accurately

Note which base URLs refuse redirects (MinerU Cloud API roots still follow
validated, pinned hops), the query/fragment base URL refusal, IP-literal hosts
under the address policy, validation of an unmanaged LLM provider's built-in
default, server-configured local providers working without the opt-in, HTTPS
for agent-runtime video and poster downloads, pinned requests bypassing the
environment proxy, and the fixed LLM and MinerU Cloud error text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(llm): bound the logged error body and map out-of-range statuses to 502

An error response from a caller-chosen LLM endpoint was read in full only
to log 500 characters, and a status of 600-999 (passed through by the
transport) made the replacement Response constructor throw. Read at most
1 KB (or 1 s) of the body before cancelling it, and report statuses above
599 as 502.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(pdf): hold MinerU Cloud redirect hops to the operator policy for a managed root

A server-managed MinerU Cloud API root runs with local networks allowed,
and the same policy applied to every redirect it answered with, so a hop
to a private address was followed without ALLOW_LOCAL_NETWORKS.

The provider transport now takes a separate address policy for redirect
hops (`redirectAllowLocalNetworks`): each hop is validated and pinned on
a dispatcher for that policy. A managed MinerU Cloud root keeps local
access for itself while its hops use the operator policy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(providers): match server-configured providers by own keys only

Provider ids come from requests and were looked up on plain config
objects, so an id such as `constructor` or `toString` read an inherited
property and counted as server-configured (and its key, base URL and
models resolved from that property). All per-provider config lookups now
go through one helper that accepts only the section's own keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(audio): let server-configured TTS/ASR endpoints reach local networks

Server-configured image/video, MinerU and LLM providers on a local
network work without ALLOW_LOCAL_NETWORKS, but a server-configured
TTS/ASR or voice-registration endpoint given as a loopback or private IP
literal (for example a VoxCPM server at 127.0.0.1:8000) was refused
unless the opt-in was set.

The routes and the server-side narration, classroom TTS and voice-clone
paths now mark server-configured providers as `managed`; their endpoint
requests run with local networks allowed, still pinned, with cloud
metadata and reserved ranges refused, and redirect hops held to the
operator policy. Client-supplied endpoints keep the strict public
policy, and an unmanaged provider's catalog default keeps the operator
policy. Provider-returned result URLs are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): server-configured TTS/ASR local endpoints and config id lookups

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 18:01:22 +08:00
小魔andwyuc a41bbac239 feat(llm): retry once on a fallback model for retryable failures (#1614) 
* feat: configurable model escalation scheduler

## Motivation

Course generation can stall when the model configured for a heavy stage (e.g. `scene-content:interactive` / `scene-content:pbl`) is repeatedly rate-limited, times out, or returns empty output. Today the operator must manually switch models in `MODEL_ROUTES` and re-trigger the page; long-running generations die on a single bad call.

This PR makes the fallback automatic and observable: an optional per-stage **escalation policy** that re-runs the failed scene once with an explicitly chosen upgrade model, and logs every decision to a durable ledger viewable from a new Settings section.

## Design

- **Disabled by default, zero behavior change.** No `data/model-schedule.json` → the engine behaves exactly as before. The config is hot-read per call (250 ms cache), so saving from the panel takes effect without a server restart.
- **Explicit escalation only.** The upgrade model is resolved through the existing `resolveModel` pipeline **outside** `MODEL_ROUTES` (unrouted model string wins), reusing the full provider/model parsing (`openai:...`, `qwen:...`).
- **Trigger semantics.** `onTimeout` requires a timeout error; `onRetryableError` accepts any retryable generation error (`isRetryableGenerationError`). Content-safety style rejections are **not** retryable and **never** escalate — the safety boundary is preserved.
- **Budget guard.** Optional `budget.dailyEscalationCap` stops escalations for the day once exhausted; per-stage `max` limits retries.
- **Ledger.** Every escalation appends one line to `data/schedule-events.jsonl` (stage, scene, base → used, error class, reason); the Settings panel shows the last 20.

## Screenshots

Settings > Model Scheduling (config present):

![Model Scheduling panel](assets/model-schedule/settings-panel.png)

Ledger after an escalation (example entry):

![Ledger with entry](assets/model-schedule/ledger-with-entry.png)

## Verification

- `pnpm lint` — 0 errors
- `pnpm check` (prettier) / `pnpm check:i18n-keys` — pass (12 locales)
- `pnpm build` — Next.js production build passes
- Engine logic script — 8/8 assertions (disabled default, policy parsing, budget guard, ledger I/O)
- API A/B flow — GET null without config; PUT template → served immediately (hot reload); DELETE → back to null

## Notes

- The config schema is minimal free-form JSON (optional `models` / `budget` / `escalation` / `strictMode`); a stricter TS schema can follow in a later PR.
- `strictMode` is reserved (not enforced yet).
- The 3-tier preset template lives in the panel's "Load 3-tier preset template" button; no example file is shipped.

* feat: configurable model escalation scheduler

## Motivation

Course generation can stall when the model configured for a heavy stage (e.g. `scene-content:interactive` / `scene-content:pbl`) is repeatedly rate-limited, times out, or returns empty output. Today the operator must manually switch models in `MODEL_ROUTES` and re-trigger the page; long-running generations die on a single bad call.

This PR makes the fallback automatic and observable: an optional per-stage **escalation policy** that re-runs the failed scene once with an explicitly chosen upgrade model, and logs every decision to a durable ledger viewable from a new Settings section.

## Design

- **Disabled by default, zero behavior change.** No `data/model-schedule.json` → the engine behaves exactly as before. The config is hot-read per call (250 ms cache), so saving from the panel takes effect without a server restart.
- **Explicit escalation only.** The upgrade model is resolved through the existing `resolveModel` pipeline **outside** `MODEL_ROUTES` (unrouted model string wins), reusing the full provider/model parsing (`openai:...`, `qwen:...`).
- **Trigger semantics.** `onTimeout` requires a timeout error; `onRetryableError` accepts any retryable generation error (`isRetryableGenerationError`). Content-safety style rejections are **not** retryable and **never** escalate — the safety boundary is preserved.
- **Budget guard.** Optional `budget.dailyEscalationCap` stops escalations for the day once exhausted; per-stage `max` limits retries.
- **Ledger.** Every escalation appends one line to `data/schedule-events.jsonl` (stage, scene, base → used, error class, reason); the Settings panel shows the last 20.

## Screenshots

Settings > Model Scheduling (config present):

![Model Scheduling panel](assets/model-schedule/settings-panel.png)

Ledger after an escalation (example entry):

![Ledger with entry](assets/model-schedule/ledger-with-entry.png)

## Verification

- `pnpm lint` — 0 errors
- `pnpm check` (prettier) / `pnpm check:i18n-keys` — pass (12 locales)
- `pnpm build` — Next.js production build passes
- Engine logic script — 8/8 assertions (disabled default, policy parsing, budget guard, ledger I/O)
- API A/B flow — GET null without config; PUT template → served immediately (hot reload); DELETE → back to null

## Notes

- The config schema is minimal free-form JSON (optional `models` / `budget` / `escalation` / `strictMode`); a stricter TS schema can follow in a later PR.
- `strictMode` is reserved (not enforced yet).
- The 3-tier preset template lives in the panel's "Load 3-tier preset template" button; no example file is shipped.

* Delete omai-upload-tree directory

* fix: ensure trailing newline in locale files

fix: ensure trailing newline in locale files

* chore: english comments for upstream review

chore: english comments for upstream review

* Delete components/settings/model-schedule-settings.tsx

* Delete lib/server/model-schedule.ts

* Delete app/api/model-schedule/route.ts

* Delete assets/model-schedule/ledger-with-entry.png

* Delete assets/model-schedule/settings-panel.png

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Update .env.example

* Enhance access control comments in .env.example

Added additional context and warnings for access control configuration.

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* run ci

* Add files via upload

* Add files via upload

* Add files via upload

* Supplement. env.example

Supplement. env.example

* Add files via upload

* Update .env.example

* Update fallback notes in .env.example

Clarified fallback behavior in callLLM layer documentation.

* Update llm.ts

* Update llm-fallback.test.ts

* fix(llm): round-4 fallback hardening

fix(llm): round-4 fallback hardening — server-managed gate, content-filter refusal, APICallError unwrap stop, outlines fullStream errors

* fix(llm): rebase round-4 onto main v1.1.1

fix(llm): rebase round-4 onto main v1.1.1

* fix(llm): round4c-prettier formatting+retry test 

fix(llm): round4c-prettier formatting+retry test

* test(llm): round-4d-align outline-stream

test(llm): align outline-stream and title-generator tests with round-4 call signatures

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-28 17:24:55 +08:00
wyucandClaude Opus 5.5 44ad72beba fix(persistence): serve the course library without the agent runtime; exit on invalid boot config 
* fix(persistence): serve the course library without the agent runtime

The course library and folder routes (/api/stages/** and /api/folders/**)
gated on isAgentRuntimeConfigured(), so a deployment with server persistence
(DATABASE_URL set, client built with NEXT_PUBLIC_PERSISTENCE=1) but the agent
runtime off answered 404 there: the home library showed an empty,
"persistence unavailable" list and folder creation failed, although imports
persisted.

These routes only need the persistence provider and the owner-bound document
store, so they now gate on isServerPersistenceConfigured(). Agent features
(/api/agent/**, /api/skills/**, /api/materials/**) keep the runtime gate, and
without a DATABASE_URL every route still answers 404.

GET /api/agent/runtime also reports `persistence`; `enabled` and
`runtimeEnabled` are unchanged and the workbench entry still keys on
`enabled`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(boot): exit the server when boot validation refuses the configuration

Next.js logs a throw from the instrumentation register() hook as "Failed to
prepare server" but keeps listening and answers every request with 500, so a
refused configuration (malformed asset quota, pending TTL or owner lock
waits, PERSISTENCE_SHARED_OWNER_ID without ACCESS_CODE, the removed
OWNER_AUTHENTICATOR / TRUSTED_PROXY_* variables, invalid owner auth or host
hook registrations) left a process that looked alive and served nothing.

The fatal validations now run in one validateBootConfiguration() step. When
it throws, register() calls exitOnInvalidBootConfiguration(), which prints
one "[boot] Invalid server configuration" line carrying the original message,
waits for stderr to flush, and exits with code 1. The wrapper lives in its own
module so tests stub it (or process.exit) and still assert the thrown error.
register() already returns before any validation on the Edge runtime, and
warnings (unset ACCESS_CODE, model-routing checks) never exit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: document library gating and exit on invalid boot configuration

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(boot): tell refused configuration apart from other startup failures

Every throw from the boot validation step was printed as "Invalid server
configuration" with its message only, so a startup failure that has nothing
to do with settings (a module missing from a standalone build, a bug in
startup code, a host registration call that throws) sent operators looking
for a bad environment variable.

Each fatal configuration check now runs through runConfigurationCheck(),
which marks what it throws as an InvalidBootConfigurationError with the same
message. exitOnBootFailure() (renamed from exitOnInvalidBootConfiguration)
keeps the one-line message for those, and prints anything else as
"[boot] Server startup failed" with its stack and cause. Both exit with
code 1.

The README (EN and zh) and CHANGELOG now list exactly the settings the boot
validation refuses, adding OWNER_CLAIM_TRIGGER, the ASSET_S3_BUCKET conflict
with a registered byte store and the sharedTeamAuthMethod() placement rule.
Also fixes a stray comment marker in feature-flags.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 17:03:36 +08:00
wyucandClaude Opus 5.5 a686481e73 feat(auth): owner identity seam — composable auth methods, owner-keyed persistence, host hooks, claims 
* ci: run CI for the owner identity seam integration branch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(auth): owner identity seam, phase 1 — pluggable authenticator with today's behavior

* feat(auth): add the owner identity seam with built-in authenticators

Introduce lib/server/identity/: an OwnerAuthenticator contract
(SubjectKind, OwnerPrincipal, AuthOutcome), a single-shot, boot-validated
configureOwnerAuthenticator() registry, and the anonymousCookie and
sharedTeam built-ins that reproduce today's owner resolution exactly
(same anonymous_id cookie, owner id strings, statuses and env semantics).

Every owner-scoped route handler and the workspace Server Action now
resolve their owner through one memoized per-request resolution instead
of the three previous copies (agent-runtime/owner.ts, with-owner.ts and
the Server Action re-implementation). Publish and unpublish decide from
the principal's course:publish role instead of an anon: id prefix.

An invalid credential from an authenticator is answered with 401 on
every surface and never falls back to an anonymous owner. A source scan
keeps the anonymous owner cookie and anon: prefix checks out of code
outside the identity module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(auth): describe the owner identity seam and host registration

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(auth): tighten the identity boundary guard and Server Action cookies

- Scan workspace package sources too, flag imports of the concrete
  built-in authenticators outside lib/server/identity, and police the
  slice/substring/indexOf/includes spellings of an anon: id-shape check.
  Stop re-exporting the built-ins from the host-facing index.
- Refuse setCookies returned by authenticateFromContext, as the
  authenticate() fallback already did, and document that it must write
  cookies itself through next/headers.
- Let the route-test owner stub carry explicit kind/roles, and cover the
  403 forbidden branch of publish and unpublish for a signed-in owner
  without course:publish.
- README: registration conflicts fail the boot; an invalid principal is
  rejected per request with a 500.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* feat(auth): owner identity seam, phase 2 — runtime and assets keyed by the owner

* feat(auth): owner identity seam, phase 2 — runtime and assets keyed by the owner

Runtime sessions and assets on /api/persistence now take their identity
from the same memoized owner resolution as documents, instead of a
client-chosen x-learner-key behind a development bearer token that
shipped in the public bundle.

- Runtime: the storage handler's learner key is principal.ownerId. A
  path or body naming another learner answers 403 FORBIDDEN_LEARNER and
  another learner's session answers 404, whatever x-learner-key says; the
  header is no longer read. The browser learns its key from the new
  GET /api/persistence/learner-key and sends no credential of its own.
  The Pi whiteboard routes derive the same key from the owner.
  PERSISTENCE_DEV_TOKEN, NEXT_PUBLIC_PERSISTENCE_TOKEN and
  PERSISTENCE_ALLOW_INSECURE_DEV_AUTH are gone from the server path and
  the client, and server-auth.ts is deleted. Learner merge stays refused.
- Assets: allocations land in a per-owner partition (owner:<ownerId>),
  so quota is per owner and replace/delete reach only the owner's
  entries. Reads stay capability-by-id where a course viewer needs them:
  another owner's committed entry is readable while a live course
  references it. Legacy entries in the old shared partition stay
  readable by id, and may be replaced or deleted only by an owner who
  owns every course referencing them. Server-generated media is stored
  under the run owner's partition; asset-id extraction and vision image
  resolution use the same rule.
- Tombstones: runtime of a deleted course reads as absent (404, empty
  lists) and takes no new sessions or records, through a guard in the
  app's runtime composition; no package schema change.

The identity boundary scan now also fails on any source that reads
x-learner-key or the retired development token variables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(persistence): describe owner-keyed runtime and assets, drop the dev token

Remove the development persistence token from the server-persistence
recipe, .env.example, the Docker build arguments and the deployment
docs; describe runtime learner keys, per-owner asset partitions and
quota, the legacy shared-partition rule and the tombstone behavior; and
record the breaking change for existing server-persistence runtime data
under Unreleased.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(storage): scope asset references to principals, typed stage refusal, uniform create collision

@openmaic/storage 0.32.0.

- PgDocumentStore takes assetReferencePrincipals: with reference
  tracking on, a write references and commits only entries held by the
  listed principals. An id naming another principal's entry records no
  row and changes no lifecycle column, like an unknown id, and the write
  is never refused. Without the option any held entry is referenced, as
  before. AssetCollector takes a matching per-document-owner function so
  the one-time backfill scopes each document the same way. This keeps a
  document naming a leaked id from committing, exposing or pinning
  another owner's allocation.
- RuntimeStageNotFoundError: a store may refuse createSession for a
  stage its host considers absent; the runtime handler answers
  404 STAGE_NOT_FOUND, recognizing the error by class or by code.
- A session create over a taken id answers 409 SESSION_ALREADY_EXISTS
  whoever holds it. It used to answer 403 for another learner's session
  and 409 for one's own, which told a caller whose session an id was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(persistence): close owner-seam boundary gaps in references, tombstones and legacy assets

- Document writes reference and commit only the writer's own asset
  entries and legacy shared ones (owner-bound document store and the
  collector backfill), so another owner's course naming a pending id
  can no longer commit, expose or pin it.
- The foreign-read rule additionally requires the referencing live
  course to belong to the entry's owner, as defense in depth for
  reference rows that did not come from a scoped write.
- The tombstone refusal for new runtime sessions moved from a raw path
  comparison in the route into the guarded store's createSession
  (RuntimeStageNotFoundError, 404 STAGE_NOT_FOUND), so no encoded
  spelling of the path routes around it.
- Replacing or deleting a legacy shared entry checks reference
  ownership and mutates in one transaction under the entry row lock,
  which a concurrent reference insert must wait for.
- Tests build the uncommitted-but-referenced, tombstoned-with-rows and
  foreign-reference states directly so each half of the read rule is
  observable, and a PostgreSQL suite holds a competing reference open
  across a legacy delete.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(persistence): upgrade note for dev-token-gated deployments, owner-scoped references

Tell operators whose only access gate was the development token to put
the deployment behind an access code or gateway, register an owner
authenticator, or turn server persistence off before upgrading, and
describe that a course references only its owner's media.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(storage): per-owner reference principals and a typed taken-id conflict

- PgDocumentStore's assetReferencePrincipals is now a function of the
  store's bound owner, evaluated per write, instead of a fixed list. A
  store re-bound with forOwner therefore scopes to the new owner rather
  than carrying the previous owner's principals onto its writes. It has
  the same shape as AssetCollector's backfill option, so one function
  serves both.
- PgRuntimeStore and BrowserRuntimeStore raise RuntimeSessionExistsError
  for a taken session id, and the runtime handler classifies it as
  409 SESSION_ALREADY_EXISTS even when its re-read cannot see the holder
  (for example a session a host-side wrapper hides). Recognized by class
  or by code.

Still the unreleased 0.32.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(persistence): guard the Pi whiteboard runtime and answer hidden collisions with 409

- Every server-side runtime writer now uses the tombstone-guarded store:
  the Pi native whiteboard service gets it through the same helper as
  /api/persistence/runtime/*, so a deleted course takes no new
  whiteboard runtime either.
- Creating a session whose id is held by a session a tombstone hides
  answers the uniform 409 SESSION_ALREADY_EXISTS instead of a 500,
  covered on PGlite and on real PostgreSQL.
- The owner-bound document store passes its reference-principal
  function, so the scoping follows the bound owner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* feat(auth): owner identity seam, phase 3 — accounts through an identity gateway (trustedProxyHeader)

* feat(auth): owner identity seam, phase 3 — trusted-proxy-header authenticator

Add the trustedProxyHeader built-in: real accounts through an identity
gateway (oauth2-proxy, Authelia, a Keycloak-based proxy, an institutional
reverse proxy) that signs the user in and forwards the verified user in a
request header. OpenMAIC never handles passwords or tokens.

- Selected with OWNER_AUTHENTICATOR=trusted-proxy and validated in
  instrumentation register(). TRUSTED_PROXY_SECRET (32-1024 printable
  ASCII characters) is required; header names default to
  x-openmaic-proxy-secret and x-forwarded-user, and an optional groups
  header with TRUSTED_PROXY_ADMIN_GROUPS grants admin. An unknown
  selector, a TRUSTED_PROXY_* variable without the selector, a shared
  owner id or a host-registered authenticator alongside it, and malformed,
  reserved or repeated header names fail the boot.
- Trust boundary: Next.js exposes no TCP peer address to route handlers,
  middleware or Server Actions, and fills x-forwarded-for from the socket
  only when the client sent none, so no address allowlist is offered. The
  gateway secret is compared in constant time (hashed, timingSafeEqual).
- Per request: a missing or wrong secret, a missing or blank user, a user
  header with a comma (duplicate lines arrive comma-joined), or a user
  whose proxy:<user> id fails the owner id guard is 401
  INVALID_CREDENTIAL, never an anonymous owner. The guard failure is a
  401 rather than a 500 because the value comes with the request. The
  user is trimmed and keeps its case. Every gateway user holds
  course:publish; kind user, assurance verified, channel proxy; no
  cookie. Route handlers and Server Actions share one code path.
- The unset-ACCESS_CODE boot warning is skipped in this mode; the gateway
  is the access gate and ACCESS_CODE stays independent.
- The identity boundary scan now also fails on gateway identity headers
  (x-forwarded-user/-groups/-email, x-auth-request-*, remote-user, the
  secret header) and TRUSTED_PROXY_* read outside lib/server/identity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(auth): document accounts through an identity gateway

Describe the trustedProxyHeader built-in in the README owner identity
section (and its Chinese counterpart): configuration, request semantics,
the trust boundary and why it is a shared secret, the interaction with
ACCESS_CODE, boot-time validation, and an oauth2-proxy example that
injects the user, groups and secret headers. Add the .env.example block,
a SECURITY.md note and an Unreleased changelog entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(auth): harden the trusted-proxy authenticator after review

- Reserve the header names HTTP, Next.js and forwarding proxies set
  themselves: configuring the user, groups or secret header to rsc,
  next-action, forwarded, x-forwarded-for/-host/-proto/-port, x-real-ip,
  or anything starting with x-middleware-, x-invoke-, x-nextjs- or
  next-router- now fails the boot.
- Log a one-time boot warning when TRUSTED_PROXY_ADMIN_GROUPS is set: the
  admin role is granted from the groups header, which is trusted on the
  strength of the shared secret alone, so the gateway must overwrite or
  strip it.
- POST /api/chat/pi resolves the owner up front and answers 401 when the
  authenticator rejects the credential, like every other owner-resolving
  route, instead of continuing without an owner. The anonymous default
  always resolves, so it is unaffected.
- The identity boundary scan also covers the Authentik, Azure App Service
  authentication, WebAuth, Cloudflare Access, Google IAP and AWS ALB OIDC
  header families.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(auth): admin-groups warning and a corrected oauth2-proxy example

- Warn next to TRUSTED_PROXY_ADMIN_GROUPS (README, README-zh,
  .env.example) that the groups header is trusted on the strength of the
  shared secret alone, and list the reserved header names.
- The oauth2-proxy example now states it targets v7.14+ (nested
  claimSource/secretSource), uses the canonical bindAddress, sets
  preserveRequestValue: false on all three headers and
  insecureSkipNonce: false, forwards the subject with `claim: user`, and
  notes that the secret variable holds the literal secret. It adds the
  legacy-option migration caveat, a warning not to exempt app routes via
  skip-auth or trusted-ip options, and notes that the IdP must release the
  groups claim and group names must not contain commas.
- Changelog: the new boot checks and the /api/chat/pi 401.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* feat(persistence): owner identity seam, phase 4 — host extension hooks

* feat(persistence): owner identity seam, phase 4 — host extension hooks

Let a host add product behavior at four points without forking a route,
registered once from instrumentation.ts register() like the owner
authenticator and sealed on first use. Nothing changes when no hook is
registered.

- configurePersistenceHooks({ name, authorizeCreate, onCreate, library,
  beforeAssetAllocate }):
  - authorizeCreate / onCreate run once per created course, inside the
    create transaction, in the transaction that inserts the course's
    ownership row. Saves of a course the owner already holds, and a create
    that loses a race to a concurrent create of the same id, are updates and
    run no create hook. A refusal answers 403 CREATE_REFUSED; a throw rolls
    the course, its ownership row and the host's writes back together. The
    hooks receive the request principal, or only the owner id for a
    background agent run.
  - library chooses which stage ids GET /api/stages lists; the route builds
    the usual summaries in the provider's order and drops every id the read
    path would refuse (deleted or unclaimed). folderId is shown only on the
    principal's own courses.
  - beforeAssetAllocate runs inside the storage handler's asset
    authorization step for POST /assets, after the owner is resolved and
    before the body is read; a returned Response is sent instead, with
    nothing stored and no quota counted.
- configureAssetByteStore({ name, create, signsReadUrls }) replaces the
  ASSET_S3_BUCKET switch for both the persistence provider and the asset
  collector. A host store must keep bytes outside the registry database.
  Under ASSET_BYTE_EGRESS=redirect a store that does not declare
  signsReadUrls stops the server at boot; the built-in layers keep their
  behavior.
- @openmaic/storage 0.33.0: DocumentWriteRefusedError lets a store refuse a
  write as policy; the document HTTP handler answers 403 with its code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(persistence): harden the host extension hooks after review

Correctness:
- Registration reads every hook once and stores it bound to the object the
  host passed, so hooks defined on a class prototype or as non-enumerable
  properties are kept. A spread copy dropped them silently, including the
  authorizeCreate and beforeAssetAllocate gates. Same for
  configureAssetByteStore. The unknown-key check now applies to plain
  objects only, since class instances carry their own state fields.
- The owner-bound document store carries each call's operation in its own
  async context (AsyncLocalStorage) instead of a field on the instance. One
  store is shared by every tool of an agent run; a concurrent call could
  overwrite the field before the transaction read it, so a create was gated
  as a read (no ownership row, no create hooks) or claimed another stage.
  create_stage also runs sequentially within a tool batch.

Hardening:
- isDocumentWriteRefusedError recognizes a refusal from another copy of the
  storage package by name and shape (not by code alone). The document
  handler and POST /api/stages use it.
- beforeAssetAllocate also gates PUT /assets/{id}/content. The request
  carries operation ('create' | 'replace') and the decoded assetId from the
  handler's own routing.
- Agent runs: a refusal on a background write carries a fixed message, and
  create_stage answers it with a stable tool result. The host's message never
  reaches the model.
- A byte store declaring signsReadUrls without a signer degrades to direct
  bytes with a warning instead of failing reads, and the collector no longer
  requires signing.
- DocumentActor is discriminated by source ('request' with the principal,
  'background' without one).
- Library ids the read path cannot address are dropped, and a provider
  answer is capped at 5000 ids.
- The boot-time ownership backfill logs how many courses it adopted without
  hooks.
- The server persistence provider no longer exposes an unscoped document
  store.
- The writesOutsideRegistryDatabase trust boundary and the scope of upload
  admission (HTTP uploads, not server-generated media) are documented.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(persistence): refuse misspelled hooks on host classes too

A class instance skipped the unknown-key check, so a misspelled hook such
as authorizeCreat registered cleanly and never ran, leaving creation
ungated. Both configurePersistenceHooks and configureAssetByteStore now
reject any function-valued property of a class instance (own or on its
prototype chain up to Object.prototype, constructor excluded) that is not a
known key, and any unknown key of a plain object. The error names the key
and suggests the known key within edit distance 2. Host classes keep their
helpers private (#helper) or register a plain object; the README says so.

Also:
- isDocumentWriteRefusedError requires a string message from a cross-copy
  candidate, and documents that the name is the effective discriminator.
- CHANGELOG: ServerPersistenceProvider no longer exposes an unscoped
  documentStore; use the owner-bound store.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* feat(persistence): owner identity seam, phase 5 — tenancy lives in stage_meta only

* feat(persistence): owner identity seam, phase 5 — tenancy lives in stage_meta only

Course ownership was recorded twice: in stage_meta, which already answers
every access decision, and in document_stages.owner_id, which the storage
package scoped by. Two tenancy records double what a later claim has to
re-key, and a host that keeps ownership in its own table had to rewrite the
package's SQL to strip the column.

@openmaic/storage 0.34.0:
- PgDocumentStore never reads or writes document_stages.owner_id. An
  owner-bound store scopes listings, writes, deletes, the freshness manifest
  and folder membership through the host's ownership relation, named with
  the new documentOwnership option ({ table, stageIdColumn, ownerIdColumn,
  tombstoneColumn, claimOnCreate }, or false for no document scoping).
  Identifiers are validated, never quoted into SQL from arbitrary input.
- forOwner / ownerId without documentOwnership throws at construction,
  so a direct consumer cannot upgrade into a store that silently scopes
  nothing. An unbound store is tenant-agnostic.
- claimOnCreate inserts the ownership row in the create transaction; a
  concurrent create of the same id by another owner rolls back.
- folders: false lets a host whose document_stages has no folder_id use
  the store; folder ids are unique per owner, so membership goes through
  the ownership relation too.
- AssetCollector reads each document's owner from documentOwnership for
  assetReferencePrincipals, and refuses the function without it.
- Schema: fresh installs get no owner_id column and none of its indexes;
  an existing column is kept for one release, made nullable with its
  default dropped (each ALTER guarded by a catalog check, so a replay takes
  no lock), and its two indexes are dropped. document_stages_folder_idx
  replaces the owner/folder index.

Application:
- The owner-bound store binds the package to stage_meta
  (STAGE_META_OWNERSHIP) and lists through it in one query; the asset
  collector schedule passes the same relation.
- The boot backfill copies column-only owners into stage_meta before any
  store is built, only when the legacy column exists, idempotently, and
  logs how many it adopted and how many disagree (stage_meta stands).

Tests cover a fresh install and an upgrade from the previous schema on
PGlite and PostgreSQL 16, a host table with neither ownership nor folder
columns, folder isolation across owners sharing a folder id, and the
concurrent-create race.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(persistence): serialize schema bootstrap; guard unscoped owner-bound stores

Review follow-ups for the tenancy change.

- Concurrent starts: two instances starting against one database raced on
  the schema DDL (a duplicate pg_class / pg_type entry, or "tuple
  concurrently updated"), failing one instance's first request. Every
  schema bootstrap the application runs -- the persistence provider's
  sequence (runtime, documents, stage_meta and its ownership backfill,
  owner materials, assets), the agent-session, session-material and
  user-skill stores, and the asset collector's -- now runs under one
  PostgreSQL advisory lock held on a dedicated connection and released in
  finally. It lives in the application because what needs serializing is
  the whole sequence, including tables the package does not own. A
  PostgreSQL test boots eight instances at once, four rounds each, on a
  fresh and on an upgraded database; without the lock it fails every run.
- documentOwnership: false on an owner-bound PgDocumentStore now also
  requires allowCrossOwnerDocumentAccess: true, so binding an owner for
  folders or asset principals cannot silently expose every owner's
  documents.
- Documented that an ownership relation must cascade with the document
  rows (a leftover row keeps the id reserved for its owner, which is also
  how a host keeps retired ids from being reused), with tests for both.
  Leftover rows are deliberately not made claimable: a host cannot be
  told apart from one that keeps them as retirement markers.
- Upgrade notes: the one-time table locks on the first start, and the
  out-of-band CREATE INDEX CONCURRENTLY / DROP INDEX CONCURRENTLY path
  that leaves the start with nothing to build or drop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(persistence): leave the schema bootstrap lock out of the held-lock check

The create-hooks suite asserts that its host lock is released with the
create transaction by counting granted advisory locks database-wide. A suite
booting a provider against the same database at that moment now holds the
schema bootstrap lock, which is not the lock under test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* feat(auth): owner identity seam, phase 6 — claiming anonymous work on sign-in

* feat(storage): owner merge primitives for claiming anonymous work (0.35.0)

Add the per-store pieces a host needs to move one owner's data to another
inside a single transaction of its own:

- reassignDocumentFolders(tx, { fromOwnerId, toOwnerId, documentOwnership })
  moves folders and filing: a same-named folder merges into the target's, a
  colliding id is renumbered, and filing is rewritten in one statement from
  the old ids.
- PgAssetStore.reassignPrincipal(fromKey, toKey) moves a principal's entries
  under both principals' write locks, in key order; quota is not re-checked.
- PgUserSkillStore.mergeOwner(from, to) moves skills, renaming a live handle
  the target already uses with the first free numeric suffix.
- PgUserSkillStore takes a resolveFinalOwner hook, run as the create
  transaction's first statement, like the agent-session store's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(auth): owner identity seam, phase 6 — claiming anonymous work on sign-in

A visitor who works anonymously and then signs in can move that work to their
account in one transaction, and the anonymous id is retired afterwards.

- Principal: pendingClaim { fromOwnerId, assurance }. The trusted-proxy
  built-in sets it when a gateway request also carries a valid anonymous
  cookie; the anonymous and shared-team built-ins never do. Authenticators may
  implement describeStoredOwner, canonicalize and clearPendingClaim;
  principalFromStoredOwner describes a stored id for work without a request.
- Claims: claimOwner / claimPendingOwner and registerClaimParticipant (sealed
  on first claim). One transaction, both owners' identity locks taken
  exclusively in key order, participants in a fixed order (folders, courses,
  materials, agent sessions, skills, runtime, assets), recorded in the new
  owner_merges table. Idempotent per pair; refuses a non-anonymous source, an
  anonymous target, a source already claimed elsewhere, and chains. Same-named
  folders merge, colliding folder ids are renumbered, colliding skill handles
  get a suffix, and quotas are not applied to what moves.
- Identity lock: every owner write (documents, folders, asset allocations,
  material registration, agent sessions, skills) takes the owner's advisory
  lock in shared mode first, so a write racing a claim is moved or refused,
  never orphaned or deadlocked.
- Retired ids: request writes are refused with 403 OWNER_RETIRED; agent runs
  and media generation that started before the claim follow the id to the
  account (forwarding document store, canonicalized stage probes, forwarded
  generated assets and skills).
- Trigger: POST /api/identity/claim (same-origin JSON only; clears the
  anonymous cookie), or OWNER_CLAIM_TRIGGER=auto on the first request carrying
  a pending claim. The runtime contract's learner merge now performs the same
  claim, allowed only for the request's own pending claim.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(persistence): count only the suite's own advisory locks after concurrent creates

The check that a host's transaction-scoped lock is released counted every
granted advisory lock in the database. Suites that run against the same
database at the same time hold advisory locks of their own (a schema
bootstrap, the owner identity locks a claim takes), which made it fail
depending on timing. Count only locks held by this suite's connections.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(auth): use a neutral example table for host claim participants

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(storage): owner merge fixes and store policy errors over HTTP (review round 1)

- PgUserSkillStore.mergeOwner reserves every handle either owner holds before
  renaming: a claimed skill whose handle the target uses could be renamed into
  a handle another claimed skill still held, which violated the per-owner
  unique index and made that merge fail every time.
- PgUserSkillStore.list orders ties by id.
- PgRuntimeStore takes resolveFinalLearner(tx, learnerKey), run as the
  create transaction's first statement, so a host can fence session creates
  against its owner merges; reassignLearner(from, to) re-keys sessions without
  re-validating them, so one session a newer version wrote cannot block a
  merge.
- Every HTTP handler (runtime, documents, assets) answers a
  DocumentWriteRefusedError as 403 with its code, and the new StorageBusyError
  (code, message, retryAfterSeconds) as 503 with Retry-After.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(auth): claims review round 1 — fenced runtime creates, bounded locks, retired-cookie recovery

- Runtime session creates take the owner's identity lock as the create
  transaction's first statement and refuse a retired owner, closing the window
  in which a create racing a claim was left under the retired id.
- Lock waits are bounded: a claim waits OWNER_CLAIM_LOCK_WAIT_MS (default 5 s)
  for its identity locks, a write OWNER_WRITE_LOCK_WAIT_MS (default 30 s);
  running out, or losing a deadlock or serialization race, answers 503
  OWNER_BUSY with Retry-After on the claim routes and on every fenced write.
- Every 403 OWNER_RETIRED carries the Set-Cookie values that drop the retired
  anonymous credential (the anonymous built-in now implements
  clearPendingClaim), so a browser whose claim response was lost recovers.
  Writes by id to rows that moved (skill delete, session messages) answer
  OWNER_RETIRED instead of not-found or a bare 403.
- OWNER_CLAIM_TRIGGER=auto no longer claims ahead of the explicit claim
  routes, which now report their own claim instead of a false refusal.
- The owner-events stream tells a retired owner only that it moved, never the
  claiming account's id.
- Creates of one course id take turns (a per-id advisory lock before the
  ownership probe), so a concurrent second create saves as an update instead
  of failing as reserved-document.
- A claim's source must be described as anonymous by the authenticator; the
  write fences skip the retirement lookup for every other owner. The host
  canonicalize hook is removed: forwarding is core's owner_merges alone.
- The folders participant locks the source's stage_meta rows first; runtime
  sessions are re-keyed without re-validation; the owner asset store fences
  every allocation and refuses to allocate without transactions; background
  skill reads and patches follow a mid-run claim; the forwarding document
  store forwards methods only.
- Docs: when a pending claim can arise with the shipped authenticators, the
  anonymous cookie as a bearer credential, bounded waits, the exact scope of
  OWNER_RETIRED, and claims narrowed to what the tests show.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(auth): claims review round 2 — merge-record guard, boot-checked lock waits, stream credential drop

- owner_merges records claims of anonymous owners only, since the write
  fences enforce retirement only for ids the authenticator describes as
  anonymous. Reading a row that retires any other owner now fails loudly
  instead of being followed but not enforced. The comment that pointed hosts
  at claimOwner for their own account merges is corrected: a host merging two
  signed-in accounts moves the rows itself and refuses the merged-away account
  in its authenticator. describeStoredOwner must classify ids stably.
- OWNER_WRITE_LOCK_WAIT_MS and OWNER_CLAIM_LOCK_WAIT_MS are validated at boot
  with the same parser the lock paths use, so a malformed value stops the
  server instead of failing every write.
- The owner-events stream answers a connect by an already retired identity
  with one owner_moved event and the Set-Cookie values that drop the retired
  credential, so a stale tab's reconnect loop ends after one round trip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* chore(auth): final polish before landing the owner identity seam

* chore(auth): final polish before landing the owner identity seam

- Pi routes refuse a rejected credential with the seam's INVALID_CREDENTIAL body.
- Drop the unused LegacyAssetMutations alias and resolveLockWaitMs re-export.
- Scope the document_stages.owner_id deprecation wording: the store no longer
  reads or writes it; the boot backfill reads it and a claim mirrors ownership
  into it for rollback.
- Document OWNER_CLAIM_TRIGGER and the two lock-wait variables in .env.example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(storage): 0.35.1 for the README wording fix

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* feat(identity): one identity entry point with composable auth methods

* feat(identity): compose owner auth methods behind one entry point

Owner resolution now asks an ordered list of owner auth methods instead
of one configured authenticator. Each method answers `authenticated`,
`not-applicable` (no credential of its kind) or `invalid` (its credential
is present but bad). Core keeps the first `authenticated` answer, refuses
any `invalid` answer with 401 INVALID_CREDENTIAL without asking later
methods, and when no method applies falls back to the anonymous cookie
owner, or answers 401 when the host turned the fallback off. Route
handlers and Server Actions share the same order and per-request
memoization.

- configureOwnerAuthentication({ methods, anonymousFallback? }) replaces
  configureOwnerAuthenticator: single-shot, sealed on first use, and
  validated at registration and at boot.
- anonymousCookie becomes the built-in fallback method; sharedTeam
  becomes a method, selected by PERSISTENCE_SHARED_OWNER_ID as before
  when nothing is registered, and kept beside host methods only when the
  host lists sharedTeamAuthMethod() last. The variable set beside a
  registration that leaves it out fails the boot.
- Core attaches pendingClaim itself: a host method's non-anonymous
  principal on a request that also carries a valid anonymous cookie. A
  method may not set one. Claim cookie clearing and OWNER_RETIRED
  recovery go through the anonymous method's clearCredential.
- principalFromStoredOwner consults the anonymous method first, then the
  configured methods' describeStoredOwner in order.
- Remove the built-in trusted-proxy header authenticator with
  OWNER_AUTHENTICATOR and every TRUSTED_PROXY_* variable. The boundary
  test now also keeps the resolution internals inside
  lib/server/identity/ and asserts no source reads the removed variables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(identity): document owner auth methods and a gateway JWT recipe

Rewrite the owner identity section around the ordered auth methods: the
three answers and what resolution does with each, the anonymous
fallback, registration and its boot checks, and the sharedTeam rules.
Replace the removed built-in gateway authenticator's documentation with
a short host-owned recipe that verifies a gateway-forwarded signed JWT
(oauth2-proxy, Cloudflare Access, IAP) against the identity provider's
keys with the jose library. Update the claim section for the core-owned
claim candidate, and the Chinese README, SECURITY.md, CHANGELOG and the
deployment docs in every locale to match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(identity): tighten retired-owner cookies, retired config and the boundary

- On 403 OWNER_RETIRED, core now calls clearCredential only on the
  anonymous fallback and on methods that declare
  `issuesAnonymousOwners: true`, so an account method's session cookie is
  never cleared there. A clearCredential that throws is logged and
  skipped instead of turning the refusal into a 500.
- Setting any retired OWNER_AUTHENTICATOR / TRUSTED_PROXY_* variable fails
  the boot with a pointer to the auth-methods docs, instead of silently
  serving every request anonymously. Only the names are matched, in one
  module the boundary test checks never reads a value.
- Add lib/server/identity/host/ for host auth methods. The boundary test
  now scans core identity files for gateway identity headers too, polices
  reads of an incoming Authorization header everywhere but the host
  directory, and keeps resolution internals out of host code.
- Document on OwnerAuthMethodResult that a present but unusable
  credential must answer `invalid`, never `not-applicable`, and that a
  method that cannot decide throws.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(identity): fix the gateway JWT recipe's error split and add notes

- The recipe now answers `invalid` only for token failures (expired,
  claim validation, malformed JWS/JWT, bad signature, no or several
  matching keys, disallowed or unsupported algorithm) and rethrows
  everything else: jose reports a key endpoint that is down, answers
  non-200 or returns unparsable data as a generic JOSEError, which the
  previous version turned into a 401 for every user.
- The recipe file goes in lib/server/identity/host/, with an accurate
  description of what the boundary test enforces.
- Document invalid versus not-applicable for malformed credentials,
  issuesAnonymousOwners, the boot failure on retired variables, and that
  the claim candidate is an unsigned bearer cookie a sibling subdomain can
  shadow (serve on a registrable domain of its own; cookie hardening is
  deferred). Mirror the notes in the Chinese README, SECURITY.md and the
  changelog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 11:54:26 +08:00
LING DUANandwyuc 04baad7c45 fix(pi): distinguish state errors from stale whiteboard references (#1681) 
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-27 01:01:20 +08:00
LING DUANandClaude Opus 5.5 d23bc104dc feat(pi): reference a whiteboard element in classroom chat (#1656) 
Let a learner pick one element on the classroom whiteboard and ask about
it, alongside the existing PPT element and interactive component
references behind the courseware-reference flag.

- Add a `whiteboard_element` reference: { kind, whiteboardId, elementId }.
  The Host resolves it only from this request's Stage snapshot
  (`stage.whiteboard[0]`), projects the element with the existing slide
  element projection, and passes it through the existing Director ->
  call_agent evidence path. It grants no Spotlight or other tool access.
- Reuse the playback pick overlay inside the whiteboard, scoped to the
  whiteboard DOM so pan/zoom and overlapping elements resolve correctly,
  and keep a highlight on the selected element.
- Share one displayed-whiteboard selector for display, picking, and the
  pre-send check. A runtime (RuntimeStore) whiteboard disables picking;
  runtime-board references are out of scope.
- Before sending, re-check the reference against the snapshot actually
  POSTed; if it is stale, keep the learner's text and ask them to pick
  again. Server rejections carry a `whiteboard_reference_changed` reason.
- Describe whiteboard references correctly next to page-reported state,
  and rename the entry to "Reference content" in all locales.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 11:13:03 +08:00
PercyClaude Opus 5.5Percywyuc
 		a87c94c89d feat(settings): restructure settings IA with course model config and token plan controls [AI-assisted] (#1644) 
* feat(generation): propagate per-stage model routes via x-model-routes header

Course model config lets each pipeline stage pin its own model; send the
user-level stage routes alongside the main model credentials so servers
resolve overrides consistently. Server-side model-routes/resolve-model
learn to merge them; scene renderers, the scene generator and the
generation preview pass the header through.

* feat(settings): token plan enrollment markers, plan toggle and store rework

- Plans connect through an explicit tokenPlanEnrollments marker: a personal
  key on a shared provider (minimax/tokendance/doubao) is never mistaken for
  a connected plan, so startup reconciliation no longer hijacks its catalogue
- New per-plan authorization toggle (tokenPlanDisabled): toggling cascades
  to the plan's per-modality provider enabled flags while skipping providers
  another usable plan still owns; re-enabling clears the seed fingerprint and
  re-seeds through reconciliation
- Multi-plan priority: exclusive slots (main model, stage routes, per-modality
  selections) yield to plans earlier in TOKEN_PLAN_PRESETS order regardless
  of write order; picker ordering follows the same order
- Reconciliation skips plans disabled by the toggle; disconnect restores the
  built-in provider's enabled flag and clears stale authorization markers
- Stage routes are pruned when a provider becomes unusable; the homepage
  web-search toggle migrates into the settings store; server-provider init
  waits for persist hydration

* feat(settings): five-section IA with course model config and provider panels

- Settings dialog reorganized into Token Plan / Course Model Config /
  Model Services / Skills / System
- New Course Model Config section: station cards along the generation
  pipeline with a per-station inspector for model overrides, mainline
  model plus fine-grained sub-stage routes, invalid selections kept as
  disabled rows instead of blank triggers
- Model Services becomes a provider list + detail panel with a per-provider
  authorization toggle (disabled providers leave course model config);
  Qwen TTS voice-clone area redesigned (45:55 split, add-clone CTA)
- Token Plan panel gains the Enable-this-plan switch mirroring the provider
  toggle, connected/off status copy, and priority-ordered model groups

* refactor(generation): drop the media popover now that media providers live in settings

Media provider/model selection moved into Model Services and Course Model
Config; the generation-bar media popover (520 lines) goes away and the
toolbar slims to the remaining toggles.

* fix(ui): keep provider popovers inside the viewport

Voice/config popovers now cap their height against
--radix-popover-content-available-height and their width against the
viewport, so search boxes no longer clip near screen edges.

* feat(i18n): complete course-model stations and settings copy across 12 locales

- Translate stations.* labels and descriptions for the 9 locales that fell
  back to English; convert zh-TW descriptions to Traditional Chinese
- Add subStages.*, mainlineUnset, pickModel/pickProvider, optionInvalid and
  the token-plan toggle copy across all locales

* test(audio): mock getStageRoutesHeaderValue in refused-narration test

Companion mock to the x-model-routes plumbing commit.

* style: apply prettier to files touched by the settings IA refactor

* fix(settings): restore the homepage model pill and set-up CTA lost in the IA refactor

The refactor dropped the toolbar's model picker and "Set up model" CTA,
leaving the homepage without any model affordance (#580 dead-end) and
breaking the picker stress tests (#1447), the managed-provider specs
(#620) and the KV-persistence checks that anchor on the pill.

- Re-add the pill on the shared ModelPicker component (aria-label
  "Provider / Model"), backed by the extracted useLLMPickerGroups hook so
  the toolbar and course model config share one filtered, priority-ordered
  provider list
- Re-add the amber "Set up model" CTA when no usable provider exists; it
  opens settings at Model Services
- Specs: navigate to Model Services in #620 and scope its catalog
  assertion to the dialog; drop the provider-tab step in #1447 (providers
  are group headings now)

* fix(settings): write the full runtime stage set for classroom interaction

The interaction station wrote only chat-adapter, while quiz grading and
the PBL v2 runtime resolve quiz-grade / pbl-v2-runtime:*, so the override
never reached them and calls fell back to the main model (review P0).

- Add lib/config/station-stage-keys.ts as the single station -> runtime
  stage-keys contract; interaction now writes chat-adapter, quiz-grade,
  pbl-chat and pbl-v2-runtime together (PBL composite sub-stages inherit
  the base key via the existing colon fallback)
- Override and restore-follow now write/clear the whole key group,
  thinking-config changes included
- Regression tests: contract keys are LLM_STAGES members, no orphan
  runtime stages, the interaction override resolves for every runtime
  endpoint, and explicit sub-stage overrides keep precedence over
  inherited parents

* fix(settings): enforce provider authorization across sync fallback and gates

A disabled plan's provider could be resurrected on refresh: validateProvider
correctly rejected it, but the auto-recover branch adopted fallback[0],
whose order still included authorization-disabled entries; the homepage
generate gate (isLLMProviderConfigured) also ignored the toggle (review
P0-02). Excluded disabled entries from the fallback order (extracted as
buildUsableFallbackOrder; image/video keep adopting server-configured
entries whose enabled:false means not adopted yet) and taught
isLLMProviderConfigured to reject disabled providers, so an empty
selection is retained and submission stays blocked.

* fix(token-plan): assign shared provider credentials to the top enabled plan

Applying a plan overwrote shared provider slots (seedream) last-writer-
wins, and disabling the owning plan left the slot enabled with the
disabled plan's endpoint and key (review P0-03). Shared slots now always
belong to the highest-priority enabled plan targeting them: apply yields
credentials and seed catalogues to a higher-priority owner regardless of
connect order, and disable/enable/remove transitions hand the slot back
(restoreSharedProviderCredentials rewrites endpoint, key — taken from the
owner's LLM slot — and catalogue of the new owner).

* fix(token-plan): include the re-enabled plan in shared ownership and clean up on removal

Two transitions still mis-owned shared slots after the previous fix
(re-review of P0-03): re-enabling a higher-priority plan excluded itself
from ownership resolution, so the slot stayed with the lower-priority
plan; and removal skipped cleanup when another plan was merely enrolled
but disabled, leaving the removed plan's endpoint, key and model
catalogue active on the shared provider.

- restoreSharedProviderCredentials takes excludeConcerned: disabling and
  removal exclude the concerned plan, re-enabling includes it, so the
  newly enabled plan reclaims its slot and a lower-priority enable does
  not steal one it does not own
- removeTokenPlan skips shared-slot cleanup only for another USABLE plan;
  otherwise the slot is cleared and disabled instead of surviving behind
  an enrolled-but-disabled plan

* fix(chat): honor per-stage user routes for classroom chat

The classroom-interaction model override is sent by the client as the
`x-model-routes` header, but the two multi-agent classroom chat routes
resolved their `chat-adapter` stage from the request body without parsing
it, so a per-stage override was silently ignored and the main model was
used instead.

Forward the parsed user routes into `resolveModel` for both /api/chat and
/api/chat/pi (credentials still come from the body, not x-* headers),
and attach the header from the client fetches. Operator MODEL_ROUTES still
wins server-side.

Regression tests cover the user-route precedence, the operator override,
the x-model-routes parser, and both client fetches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(settings): drop the pro-mode override the server never honors

The Course Model Config UI offered a "Pro Mode" station that wrote a
user-level route for `maic-agent-driver`, but that stage is resolved only
from the operator's MODEL_ROUTES (it also needs an explicit api dialect
and contextWindow), so the override could never take effect.

Remove the station from the map and its pipeline card, stop seeding the
route from the TokenDance preset, and drop the now-unused label/i18n keys
in every locale. The settings store's rehydrate sanitizer also drops any
previously persisted `maic-agent-driver` user route, so upgraded installs
do not keep a dead entry. The "no orphan runtime stage" contract test now
expresses `maic-agent-driver` as an explicit operator-only exception.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(settings): neutralize token-plan comments

Rewrite comments that referenced an external deployment or implementation
into neutral descriptions of this file's own behavior and design intent.
No behavior change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Percy <percy@PercydeMacBook-Pro.local>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 18:11:21 +08:00
Yizuki_Ame 7de474f4b2 fix(workbench): show effective upload limits in friendly errors (#1418) 
* fix(workbench): show effective upload limit in friendly errors

* fix(workbench): format the upload limit shown in composer toasts

A non-MiB cap was stringified as a long fraction, and nothing checked that the toast used the localized message.
 2026-09-23 11:43:08 +08:00
LING DUAN 2a77a8a476 feat(chat): make Pi classroom runtime the default (#1628) 
* feat(chat): make Pi classroom runtime the default

* chore(chat): align docs and E2E with Pi default
 2026-09-22 11:41:13 +08:00
Yizuki_Ameandwyuc 54d626bc61 fix(export): surface video render rejection reasons (#1414) 
* fix(export): surface video render rejection reasons

* fix(export): keep render diagnostics out of user toasts

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-21 18:49:25 +08:00
Huang Geyangandwyuc 67f568848a fix(server): warn when access-code protection is disabled (#1599) 
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-20 19:14:38 +08:00
Easternandwyuc 33ff189a11 fix(persistence): correlate 5xx responses with request ids (#1601) 
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-20 16:30:53 +08:00
LING DUANandClaude Opus 5 f29bbc4daa feat: sample declared interactive state before classroom questions (#1508) 
* feat: sample declared interactive state before classroom questions

* test: exercise generated publication example through the iframe reader

* chore(generation): bump package version for observation prompt contract

* fix: keep interactive state optional on insecure HTTP origins

* fix(playback): keep component picking and separate state from reference

A declared state interface replaced the component picker with a forced
whole-area `#experiment` reference, removing the per-component selection
and outline that `main` already ships. Sampling was also gated on the
reference selector, so the only way to obtain state was to give up the
selection.

Reference identity and area state are now independent request-scoped
evidence items:

- `handleToggleElementPick` always arms the picker again, so a scene that
  declares the interface keeps main's per-component selection, outline,
  and send-time clearing.
- `sampleInteractiveState` follows the current Scene instead of the draft
  reference, so an unreferenced follow-up still reports current facts and
  never re-creates or extends a reference.
- The Host carries area state with or without a component reference. The
  evidence header names both identities and refuses to present area facts
  as properties of the referenced component.
- `metadata` is absent when only area state travels, so no element
  identity and no Spotlight authorization can be derived from it, and the
  accepted-reference receipt stays driven by explicit references only.

Review follow-ups in the same change:

- Client sampling follows `NEXT_PUBLIC_COURSEWARE_REFERENCE_ENABLED`.
  An ungated packet turned an ordinary Pi question into a 400 while the
  reference feature was disabled.
- A Scene that declares the interface always receives an availability
  boundary, including when the browser produced no packet at all. It is
  reported as `not-sampled` rather than the previous `no-interface`,
  which was a false statement about an activity that does declare one.
  Courseware without the interface keeps its unreferenced behaviour.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(generation): register the observation snippet as a packaged asset

The interactive-observation snippet is referenced by all six widget
content templates but was never added to the packaged-asset manifest, so
the asset test and the golden scene prompt both failed.

- `SNIPPET_IDS` now lists `interactive-observation`, restoring both the
  "exactly the generation-owned templates and referenced snippets" check
  and the "every referenced snippet is packaged" cross-check.
- The interactive system-prompt snapshot is re-pinned. The change is
  purely additive: the snippet is appended to the simulation template.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(pi): stop injecting state constraints while the feature is disabled

The route rejects a request that carries a reference or a state packet
while `NEXT_PUBLIC_COURSEWARE_REFERENCE_ENABLED` is off, but an ordinary
question carries neither. It still reached the Host, and a Scene that
declares the state interface then received the full page-state block —
several kilobytes of constraints about evidence the deployment can never
sample.

The Host now returns before building that note when the feature is off.
A route-level regression asserts that neither the Director prompt nor the
Child prompt gains `PAGE-REPORTED STATE` in that configuration; disabling
the guard makes it fail with exactly that symptom.

Also reopens the composer before the unreferenced follow-up in the
classroom browser spec. An accepted answer may close it, which made the
assertion flaky without changing the behaviour under test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(pi): decouple reference Scene from state freshness and bound assembled evidence

Cross-review found two defects in the request-scoped state evidence.

The reference's Scene was folded into the sample's staleness test. The packet
is already bound to the current Scene by the identity check above it, so a
valid current-Scene sample was being discarded as `stale-sample` purely because
the student's component reference came from an earlier Scene. Reference and
area state are independent evidence items; freshness is a property of the
sample alone. With the coupling gone the two can now disagree on Scene, so the
note says so explicitly rather than letting the model attribute area facts to a
component that may not be on the current Scene.

The assembled evidence had no stated output budget. The static component packet
is bounded to 24,000 code points upstream, but that bound covers the static
packet alone; the note and the escaped observation JSON were appended without a
recheck. Escaping `<` for the prompt expands one code point into six, and `<` is
legal in a label or a fact value, so a packet the Host accepts could assemble to
149,385 code points. The budget is now declared as the static bound plus the room
the note frame needs, which is what makes the degradation terminate. Over budget,
the state body drops whole to an explicit `unavailable` statement: truncating the
JSON would emit a broken packet, and thinning a `complete` relation set would turn
an exhaustive set into a false one. The relationship summary degrades with it, so
the prose never asserts COMPLETE over a body that is gone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(pi): keep slide references independent of activity state

* refactor(interactive): simplify declared state and unify iframe preparation

Accept any JSON report within byte and depth budgets, without generated field
requirements or relationship-completeness semantics. Keep publishState and an
optional rendered result in the generation guidance.

Prepare the observation responder through patchHtmlForIframe and let the pool
own document identity, preserving state across placeholder remounts. Settle
sampling failures locally and align browser/server nesting limits.

Cover permissive JSON delivery, resource limits, lifecycle, legacy behavior,
and real renderer remounts with focused regression tests.

* fix(generation): publish automatic activity changes with clear positions

* fix(interactive): report missing legacy scope as no interface

* fix(interactive): guard sampling capabilities and bind scopes lazily

* chore(generation): bump version after main integration

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-19 18:17:30 +08:00
XHaoandwyuc e14c3a1d12 fix(api): validate pdfContent input (#1578) 
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-18 12:36:29 +08:00
wyucandClaude Opus 4.8 87c4524b4d fix(audio): validate redirects and pin connections on provider requests (#1514) 
Audio provider requests (TTS, ASR, voice registration and voice cloning)
validated a client-supplied base URL once and then issued a plain fetch with
default redirect-follow and no pinned dispatcher. A base URL that resolved to a
public address but answered with a redirect to an internal one was followed, and
a DNS answer that changed between the guard's lookup and the connect reached an
internal host — both readable in-band.

- Route every lib/audio provider request through a new
  lib/server/audio-provider-fetch.ts that combines per-hop redirect
  re-validation with a pinned undici dispatcher, so the socket can only reach an
  address the guard validated, on every hop.
- Select the public-vs-local policy server-side from isServerConfiguredProvider;
  a client-supplied base URL is always strict public and can never reach a
  private, loopback or cloud-metadata address, even with ALLOW_LOCAL_NETWORKS
  set.
- Pin the result-audio download hop as well, keeping its host allowlist and
  redirect:'error'.
- Add a coverage-matrix test that fails if any lib/audio module regains a raw
  provider fetch.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-09-15 17:47:30 +08:00
wyucandClaude Opus 4.8 ee79762ef9 fix(access-code): expire tokens server-side and throttle verification behind a trusted proxy (#1513) 
When ACCESS_CODE is set, verification tokens (timestamp.HMAC) never expired
because neither verifier checked the timestamp, and POST /api/access-code/verify
had no attempt throttling.

- Enforce a 7-day token lifetime in a shared, Edge-safe module used by both the
  Node verifier and the middleware Web-Crypto verifier, and reject non-canonical
  signatures in both.
- Rate-limit verification only when the client identity is trusted
  (TRUST_PROXY_HEADERS=true): a per-client sliding window (10 failures / 60s)
  whose attempt is reserved atomically at check time, returning 429 with
  Retry-After. Without a trusted proxy the app cannot attribute requests to a
  client, so no shared throttle is applied — a long random ACCESS_CODE is the
  protection, and a warning is logged when it is short.
- Store bounded, copied identity keys so a large forwarding header cannot retain
  memory.
- Document the behavior in .env.example, README, and configuration docs.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-09-15 17:47:14 +08:00
wyuc c605e9e6ef feat(persistence): turn on the server-owned asset lifecycle and release assets on course deletion (#1007 amendment, part 2) (#1473) 
App wiring for the @openmaic/storage 0.31.0 lifecycle: reference tracking and document references are paired unconditionally and declared at startup, course deletion withdraws references inside the tombstone transaction, ASSET_PENDING_TTL_MS configures the pending window, and the dead client-side reclamation code is removed.
 2026-09-15 09:55:26 +02:00
杨慎Claude Codewyuc
 		90cf53f142 fix(upload): resolve the generic Office MIME (application/vnd.ms-office) via filename extension (#1498) 
* fix(upload): resolve the generic Office MIME via filename extension (#1497)

On Linux, Chrome derives File.type from the XDG shared-mime-info database.
Older or trimmed databases (e.g. Kylin OS V10) map every OOXML extension to
the generic container `application/vnd.ms-office` instead of the concrete
format MIME, so .pptx/.docx/.xlsx uploads were rejected with "当前解析器不支持该格式"
on the generation toolbar and with 415 unsupported_mime on POST /api/materials
— while the same files upload fine on macOS/Windows.

Treat `application/vnd.ms-office` as a generic upload fallback alongside the
zip family, so the filename extension decides the concrete format:

- lib/document/mime.ts: add it to GENERIC_DOCUMENT_MIME_TYPES. This covers
  the toolbar's provider-whitelist check and the extract-document API gate
  (both call normalizeDocumentMimeType). It must NOT go into per-format
  aliasMimes — canonicalFromAlias iterates DOCUMENT_FORMATS in declaration
  order, so a shared alias would misroute x.pptx to application/msword.
- lib/workbench/material-upload-policy.ts: new resolveWorkbenchMaterialMime()
  grants missing/octet-stream/zip/vnd.ms-office types the same extension
  fallback the document path already had (the workbench gate previously
  rejected empty and zip-family types outright on every OS).
- app/api/materials/route.ts + session-store.ts: gate, store, and send the
  resolved MIME instead of the raw browser value.

The spoofing surface is unchanged: a specific but unsupported MIME still
falls through verbatim for the whitelist to reject, and an unknown extension
keeps the generic MIME. Full suite: 7992 passed.

Closes #1497

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(review): log the declared mime, harden the fallback chain, pin adversarial cases

Review follow-ups on #1498 (no correctness or security findings — hygiene only):

- materials route: the reject/context log now carries declaredMime alongside
  the resolved mime whenever they differ, so a generic-header spoof stays
  visible in upload failure logs instead of collapsing to the resolved type.
- uploadWorkbenchMaterial: the returned record's mimeType falls back to the
  locally resolved mime before the raw browser value, which may still be the
  generic Office container if the server echo is ever absent.
- material-upload-policy: export WORKBENCH_MATERIAL_EXTENSIONS and add a
  drift guard asserting every accepted extension resolves (via a generic
  MIME) to a whitelisted type — a hand-maintained map entry going missing
  now fails tests instead of silently 415ing.
- tests: pin the legacy-.ppt provider split (mineru rejects, mineru-cloud
  accepts), uppercase-extension resolution, and 415-before-400 precedence
  for a generic mime with no filename.

Full suite: 7996 passed.

Co-Authored-By: Claude Code <noreply@anthropic.com>

* fix(upload): support WPS Office MIME aliases

---------

Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-14 17:53:08 +02:00
wyuc 68aed2932a fix(proxy-media): connect only to addresses the SSRF guard validated (#1488) 
* fix(proxy-media): connect only to addresses the SSRF guard validated

The route validated a URL with validateUrlForSSRF and then handed the
hostname to fetch(), which resolved it again at connect time. An
attacker-controlled name could answer a public address to the guard and an
internal address (loopback, RFC1918, link-local, cloud metadata) to the
socket, and every redirect hop repeated the same validate-then-refetch gap.

Add a shared policy-aware pinned dispatcher
(lib/server/pinned-dispatcher.ts) whose connect.lookup resolves all
addresses, runs the same local-network policy as validateUrlForSSRF over
every answer (cloud metadata is always blocked, even with
ALLOW_LOCAL_NETWORKS=true), and passes only the vetted list to the
connection. proxy-media creates one dispatcher per request and uses it on
every hop, mapping a connect-time refusal to 403 INVALID_URL instead of a
generic 500. The agent-runtime pinned agent now reuses the same builder
with its strict assertSafeIp policy unchanged.

The other routes that validate with validateUrlForSSRF and then hand the
URL to provider SDKs (parse-pdf, transcription, generate/*, verify-*,
probe-models, azure-voices) are not covered; that is a follow-up.

Tests: new tests/server/proxy-media-rebinding.test.ts covers the two-answer
rebind, the ALLOW_LOCAL_NETWORKS opt-in through the pinned path, metadata
refusal under the opt-in, and a redirect-hop rebind.

* fix(proxy-media): destroy the per-request dispatcher and align non-unicast policy across layers

Follow-up hardening for the proxy-media SSRF pin.

1. Handler hang: the route awaited `dispatcher.close()` in `finally`.
   undici's `close()` drains in-flight requests, and the response body is
   intentionally left unread on the 30x (`redirect: 'manual'`), non-2xx,
   oversize and connect-refusal paths, so an upstream that keeps its body open
   never completes and the POST never settles. Tear the pool down with a
   non-awaited `destroy()` instead; the response blob is already in memory by
   the time `finally` runs. Regression tests drive a loopback server that
   answers 404, a 30x, and an oversize 200 while trickling its body forever and
   assert the route settles in well under a second.

2. Headline rebinding test sensitivity: install an attacker-controlled global
   dispatcher (`setGlobalDispatcher` plus a lookup that always answers
   loopback) in `beforeEach` so the unpinned fetch path really reaches the
   loopback server. Removing the route's dispatcher now fails on
   `internal.requests() === 0` rather than on an unrelated DNS resolution
   error. The positive control also asserts the pinned connect lookup was
   invoked, so a runtime that silently ignores `init.dispatcher` goes red.

3. Non-unicast policy parity: `validateUrlForSSRF` only classified IP-literal
   URLs through `isPrivateIP`, so a literal in CGNAT 100.64/10 or an IANA
   reserved/special-use range (240/4, 198.18/15, the TEST-NET blocks,
   192.0.0.0/24, multicast, broadcast) passed the URL layer while the same
   address reached through a hostname was refused at connect time. Both layers
   now share `isNeverAllowedRange`, applied to IP-literal URLs and to every
   resolved answer, with or without `ALLOW_LOCAL_NETWORKS`.
   `connectionAddressBlockReason` drops its blunt `range() !== 'unicast'` test
   so the two layers agree exactly, while private, loopback and link-local
   answers keep following the opt-in.

   Behaviour change: a hostname resolving into CGNAT 100.64/10 or an IANA
   reserved/special-use range is now refused (403 INVALID_URL) where it was
   previously proxied, including under `ALLOW_LOCAL_NETWORKS=true`, since these
   ranges are never legitimate proxy targets.

4. Do not echo an unparseable connect-time answer in the refusal. Return the
   generic block message instead of `Unable to classify network address:
   <value>`; the route relays that message to the client in a 403 body.

Tests: new trickle-body regression tests and CGNAT/reserved cases for both
literals and resolved answers, plus a `connectionAddressBlockReason` parity
suite. One existing fixture is updated from the IANA documentation prefix
2001:db8::/32 (reserved) to a real public prefix 2001:4860::/32.

* fix(ssrf-guard): let the local-network opt-in cover CGNAT and decode local-use NAT64 prefixes
 2026-09-13 18:00:56 +02:00
wyuc 1fd4348f21 fix(classroom): generate classroom ids server-side and create files exclusively (#1489) 
* fix(classroom): generate classroom ids server-side and create files exclusively

The legacy file-backed classroom store accepted a caller-chosen stage.id and
renamed a temp file over the target, so anyone who knew a public share URL
could POST the same id and replace that classroom's content. POST
/api/classroom now mints the id itself (nanoid, 10 characters, matching the
generation pipeline), ignores any client-supplied stage.id, and rebinds the
stage and every scene to the generated id. Both the route and
classroom-generation persist through a new exclusive create that hard-links a
temp file into place, so an existing classroom is never replaced; EEXIST is
retried with a fresh id a bounded number of times and then reported as 409.

Unchanged: the server-backed persistence path (app/api/persistence,
lib/persistence), the read/GET path, middleware, and the access-code gate;
writeJsonFileAtomic keeps its overwrite semantics for the other callers.
Adds regression tests pinning non-overwrite, the typed EEXIST error, collision
retry/exhaustion, and generation-path exclusivity.

* fix(classroom): fall back to exclusive open without hard links and reserve ids before media generation

writeJsonFileExclusive kept `fs.link` as its only route to the destination, so
classroom creation failed hard on mounts without hard links (gcsfuse, s3fs and
some FUSE gateways return ENOSYS/ENOTSUP/EOPNOTSUPP/EPERM/EXDEV). Keep `link`
as the fast path and, for exactly those codes, fall back to an exclusive `wx`
open: still never replaces an existing file and still maps EEXIST to
ClassroomAlreadyExistsError. Any other code keeps propagating, and the temp
file is always removed.

The generation pipeline wrote media and TTS into
<CLASSROOMS_DIR>/<id>/{media,audio} before the exclusive create, so a collision
would have landed the new classroom's files in an existing classroom's
directory and the retried document's URLs would still point at that other id.
Reserve the id first by exclusively creating the classroom file with a
placeholder document (`reserved: true`, empty scenes). The file is the only
token that atomically covers the whole collision namespace: a classroom created
through POST /api/classroom has a JSON file but no directory, so reserving a
directory would miss it. readClassroom treats a reserved document as absent, so
an in-flight or crashed reservation is never served as an empty classroom. The
retry on EEXIST now happens at reservation time (bounded, before any media),
and the final persist is an ordinary overwrite of the id the process owns, so
the post-media id retry is gone.

Document OPENMAIC_CLASSROOMS_DIR in .env.example, including that
CLASSROOM_JOBS_DIR does not move with it.

* fix(classroom): release an unused reservation when generation fails
 2026-09-13 17:49:57 +02:00
LING DUANandwyuc ebf665f316 feat(playback): reference static GenUI components (#1281) 
* feat(playback): reference static GenUI components

* feat(playback): gate courseware references

* fix(pi): bound Interactive source HTML before parsing

* fix(pi): bound interactive reference HTML processing

* fix(playback): close interactive reference review gaps

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-11 19:16:10 +02:00
Yizuki_Ameandwyuc c460bceb60 feat(export): preflight render queue availability (#1455) 
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-11 14:33:51 +02:00
wyucClaude Opus 5杨慎
 		85ef52d599 feat(media): write generated media through the asset pool under server-backed persistence (#1392) 
* feat(media): store generated media in the asset pool when persistence is server-backed

With server-backed persistence the document is durable and shared, but
generated media stayed in the producing browser: the document kept its
gen_img_* / gen_vid_* placeholder and narration kept a browser-derived audio
id. Every new browser that opened such a course re-ran generation for every
slide, and it never converged, because the address of the generated bytes was
never written back into the document.

Under server-backed persistence only, the classic generation chain now stores
bytes in the asset pool first and writes the id the pool allocated into the
document.

- The client bootstrap configures the asset seam alongside the document and
  runtime seams: an HttpAssetStore over the persistence endpoint carrying the
  same credentials the document store carries, marked server-backed. The seam
  preflight now covers all three, so a failure still cannot half-configure
  persistence.
- Image, video and TTS generation commit in one fixed order: provider, pool,
  document, local cache, task. A reference reaches the document only after put
  returned an id, so a document can never name bytes that were not stored. A
  failure before the write-back leaves the placeholder with the provider called
  exactly once; the retry happens on the next owner load.
- The write-back is a per-slot rewrite through mutateDocument, which re-reads
  the current document under the per-stage lock, so it cannot clobber a newer
  scene. The open course is refreshed with the same rewrite without being
  marked dirty.
- "Has this already been generated?" is answered by the document (the slide
  exists and no longer holds the placeholder) instead of by this browser's task
  table.
- The classroom's resume effect fails closed on ownership: only a resolved
  owner starts generation, so a viewer opening a shared course spends nothing.
- The local media and audio tables become a per-tab cache. A failed cache write
  costs a re-download, never the media.

Browser-only mode is unchanged: every new call site sits behind the
server-backed gate, the local tables stay authoritative there, and placeholders
stay in the document.

Rendering and export needed no changes. HttpAssetStore.resolve mints an object
URL exactly as the browser store does, and the export byte resolver was already
pool-first.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(classroom): make the generation owner gate a three-outcome rule and apply it everywhere

The gate refused everything but a resolved owner, which read a sidecar that
answered "no ownership fact exists for this course" as a reason to block. That
is the answer a deployment without the sidecar's server-side prerequisites
gives for every course, and the answer a course with no ownership record gives:
in both, there is nobody the operator's budget needs protecting from, and
refusing strands the course's own author behind a question that can never be
answered.

Ownership is now four states over the sidecar's three outcomes. A definite
answer splits into owner and not-owner. An absent record is its own answer,
ownerless, and generation proceeds — the behaviour such a deployment had before
the gate existed. Only the absence of an answer, a transport failure or a load
that has not asked yet, stays unresolved and fails closed: "we could not ask"
must never be read as "nobody owns this". One mapper turns a sidecar result
into that state, and one predicate decides on it.

The workbench classroom pane runs the same resume effect and had no ownership
input at all, so a viewer opening a shared course there could still spend the
budget. It now asks the sidecar once per course, in parallel with its load and
feeding only the generation gate, so its read-only and edit behaviour is
unchanged. The shared progressive-load policy carries the gate for it, with
both new inputs required rather than defaulted so a future caller cannot omit
them into an open budget. Its stale comment claiming ownership could not be
expressed here is corrected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): make the write-back survive autosave, arrive before the scene does, and never leak

Independent reviews of the write-back found three ways a durable document could
still end up naming a placeholder, and two ways the gate that protects the
operator's budget could be walked around.

An autosave round captures the store synchronously and writes that capture, so
a round already in flight when a rewrite landed wrote the placeholder straight
back over the allocated id, and nothing marked the store dirty again to correct
it. The rewrite now marks the units it changed, which leaves a corrective flush
queued behind the stale one; re-saving a scene that already holds the id is
idempotent, losing the id is not.

Media is generated from outlines in parallel with scene content and usually
finishes first, so the slide that will carry the placeholder does not exist yet
and the write-back has nothing to rewrite. That was the ordinary path, not a
tail case, and its result was discarded: the task was marked done, the scene was
added afterwards with its placeholder intact, and a second pass in the same run
could call the provider again. The allocation is now held under the placeholder
— which also answers the skip test, so nothing pays twice — and applied when
that scene is committed, before its first save. One complete pass now leaves no
placeholder behind.

A failed commit used to abandon what it had already allocated. A poster upload
that failed threw away a stored video and sent the retry to submit the most
expensive job in the system again; a rejected write-back left registry rows that
name bytes nothing references, which the byte collector cannot reclaim because
it only collects blobs no row names. A poster failure now costs the poster, and
a write-back that reached nothing reclaims what it allocated. A partial write is
left alone, because the document already names it.

The ownership gate is fail-closed again. Treating the sidecar's 404 as
permission was wrong: the client cannot tell "this course has no owner" from
"this deployment told me nothing", so a visitor who opened a shared course could
bill the operator. The root cause was the sidecar itself, which gated on the
agent runtime although every persisted course has an owner regardless — the
persistence route resolves one for every request. It now gates on server
persistence, so the configuration that made 404 the universal answer has real
ownership facts to report, and the gate can refuse everything but a named owner.

Retry affordances answered to no gate at all. A viewer of a shared course with
one failed image was shown a Retry button that called the provider. Both retry
entry points and every surface that draws them now read one shared permission,
so what is offered and what is allowed are the same value.

Also: narration regeneration no longer pretends it can replace bytes behind a
live id — the exclusivity proof that would allow it is refused by construction
once references leave the browser, so it forks to a fresh id and says so; the
"already generated" test lets a finished deck answer from the document alone,
since scene order stops identifying an outline once slides are inserted or
deleted; stored assets record a specific media type rather than a generic
transfer type; the pane no longer asks the sidecar in browser-only mode; and the
funnel's docstring now states what the per-stage lock actually guarantees, which
is same-browser serialization and not a cross-browser compare-and-swap.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): park allocations in the deciding turn, never reclaim on an ambiguous write

A delta review of the write-back found the first-pass fix still had a window,
and the reclamation it added could delete media the document already names.

The allocation was parked after an awaited local cache write. A scene committed
in that window reconciled against a registry that did not hold it yet, so the
document kept the placeholder — and the entry recorded a moment later then
answered the skip test as "already handled", so nothing could correct it. Parking
now happens inside the write-back, in the same synchronous turn as the decision
that nothing could take the reference; no await separates the live check from the
park. Allocations parked by an earlier pass are handed to their slides at the
start of the next one, before anything decides what still needs generating, so a
held allocation whose scene has since arrived becomes a rewrite rather than an
answer.

Reclaiming on a rejected write was unsound: a rejection does not prove the server
did not apply the write, so deleting the asset could break the scene that now
names it. The funnel decides instead, and says so: it reclaims only when no store
write was ever issued and nothing took the reference. Anything else is placed if
its slide exists and parked if it does not, so the next pass reuses the bytes
instead of paying for them again. When a write fails after part of it landed, the
live store is brought up to the document before the error is rethrown — otherwise
the next ordinary flush would overwrite the half that did land, with the ids
deliberately not reclaimed.

Parked allocations are now cleared with the course. Classic placeholders are
reused across runs, so one surviving an interrupted run would be handed to a
different slide of the next deck: the previous picture, on a slide whose provider
was never asked. Both classroom surfaces clear the arriving course, the deletion
cascade clears the deleted one, and clearing the database clears them all.

Two more ways generation could start without asking the gate are closed. An
overlapping pass — an outline retry re-enters generation with every outline while
the first is still working — re-requested elements whose provider call was
already in flight; a task that is not done is an answered request, not an
unanswered one. And narration regeneration in the timeline editor called the TTS
provider and allocated a pool asset with no ownership check at all; it now reads
the same permission, which withholds both the per-line and whole-timeline
controls and refuses the call.

Finally, a pane opened during the stage-link availability gap recorded the
sidecar's 404 for a course that was moments from existing and never asked again,
leaving the real owner locked out of generation until it remounted. Ownership is
re-fetched once the document becomes available; the gate stays closed until an
answer arrives, so asking again can only open it for someone entitled to it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): make the asset routes reachable, and stale snapshots harmless

A full-branch audit found that the deployment this project documents could not
store a single generated asset, and that several routes into durable storage
could still write a placeholder over a reference that had already landed.

The persistence route sent asset requests through the development
authenticator, which refuses outright in a production build that has not
explicitly opted into it — and the documented server-persistence recipe
produces exactly that build. Every store and every read answered 401, so images
and video failed on every slide while re-billing the provider on each retry, and
a narration failure stopped the deck at its first slide. Assets live in one
shared partition by design, so there was never anything per-caller for that
authenticator to decide: the route now resolves the asset principal itself,
alongside the owner it already resolves for documents. Runtime sessions are
genuinely per-learner and keep the development authenticator until real session
verification replaces it. And narration that cannot be stored no longer fails
its scene: the line stays unvoiced and retryable, which is what an image that
cannot be stored does to its slide.

Placeholders could also come back from behind. A queued autosave's snapshot, an
editor-history entry replayed by an undo, the departing save a course switch
flushes — each captures content at its own moment, and any of those moments can
predate a write-back. Point fixes at each producer would leave the next producer
to rediscover the bug, so the check lives at the write boundary every producer
passes through, and the allocation record it consults now outlives the parked
queue: a placeholder whose rewrite landed long ago is exactly the case it
catches.

Two ways generation could be lost or repeated are closed. A pass now claims the
elements it will reach and releases them however it ends, so an overlapping pass
stands down while an aborted one strands nothing — previously its tasks stayed
`pending` and every later pass skipped them with no retry control to recover
them. And the media abort controller is aborted before being replaced, so a
superseded pass stops calling providers instead of running on for a course the
user has left.

The remaining two are narrower. The workbench pane asks for ownership only after
a document load succeeds, and after every later one, mirroring the page route:
the load is what creates the ownership row the first time a course is opened, so
asking beforehand asked about a course that did not exist yet and locked its
author out for the mount. And the ownership gate on the timeline editor now
withholds narration regeneration alone; listening back to existing narration and
seeing whether a line has any spend nothing and stay available.

Known limitation, unchanged and now stated plainly in the comments that used to
point at it as a solution: nothing reclaims an unreferenced pool asset. The
registry sweep is written but not wired up, and the byte collector only reclaims
blobs no registry row names, so every narration regeneration and every abandoned
allocation leaves storage behind.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): gate asset mutations, and make claims and allocation records survive a handoff

Opening the asset routes opened all of them. Reads and allocations are meant to
be as open as document reads and creates already are, but no authorization hook
was supplied, so the handler's default admitted PUT and DELETE too — and those
scope by principal key alone, which is one shared constant. Any caller who
learned an id, and a document read hands out every id its slides name, could
overwrite or destroy another author's media. Mutations now require the
deployment's credential, which in a production build without the development-auth
opt-in means they are refused outright; reads and allocations stay open. The
route comment says what the posture is and what it is not: the deployment-level
fence is the access code, and no per-principal quota is configured. The client's
own reclaim is best effort to match — losing an argument about deleting an asset
must not cost a task its retry, and the bytes are left for server-side
reclamation.

The pass claim could not survive the handoff it was written for. A retry aborts
the live media pass and starts its replacement in the same synchronous block,
long before the aborted pass's cleanup runs, so the replacement saw every element
still claimed, collected nothing, and returned — leaving each unreached element
at pending with nobody coming back for it and no retry control to recover it,
which is the exact failure the claim was introduced to prevent. A claim now
carries its pass's signal and is retired the moment that signal aborts, and a
pass releases only claims it still owns, so a late unwind cannot take its
replacement's work. Claims are also acquired at the single point every request
passes through, so a single-task retry participates too — previously a retry
awaiting its provider was invisible to a pass starting alongside it and both
called it.

The allocation record could outlive the bytes it named. It was written before the
write-back attempted anything and survived the reclaim that followed a failure,
so when the slide finally arrived the write boundary stamped a deleted id into
the document — and the placeholder it replaced was gone, which reads as already
generated and stops anything from retrying. The record is now written only where
the allocation is retained, and forgotten wherever a reclaim removes the bytes,
including the narration rollback path.

The tests follow. The route test drives the real storage handler against an
in-memory registry instead of a stub, so it can see what the resolved principal
is then allowed to do; the handoff test performs a real abort mid-pass rather
than starting from an already-aborted signal; and the guards that could only
assert file layout now assert the property they care about, or have been replaced
by behaviour.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): make media passes serial per course instead of tracking element ownership

Three rounds of per-element claims each produced a new way to lose an element.
Whole-pass reservations swallowed a Retry for an element the same pass had
already failed, leaving it pending with the affordance gone. Retiring a claim by
its signal freed an element whose commit was still uploading, so the replacement
pass paid for it twice. A claim held for a failed element stranded its retry. The
bookkeeping is the defect: every refinement of "who owns this element right now"
answered the question at a moment when the answer was already stale.

Passes for one course are now serial. A replacement aborts its predecessor, as
before, and then waits for it to settle before collecting. That removes the
question entirely: a commit already under way finishes — its bytes stored and its
reference written, so the new pass sees a resolved slide and skips it — and an
element the aborted pass never reached is still a placeholder and gets collected
like any other. The claim set, the reservations, the signal retirement and the
identity-checked release are all gone.

The task table is consulted for one thing only: an element that is generating
right now is a single-element retry running alongside the pass, and taking it too
would pay twice. Pending is deliberately not a skip reason — it means a pass once
intended to reach an element, which an abandoned pass leaves behind with nobody
acting on it, and reading that as answered is what stranded elements before. A
retry runs concurrently with a pass, because a pass never revisits an element it
has processed, and it re-reads the task after its own await and refuses before
touching it: marking first and refusing afterwards destroyed the failed state
that draws the affordance.

Browser-only mode is back to exactly what it was. The abort is now conditional,
the waiting does not apply, and the original status-based skip is restored
verbatim. Two baseline lines remain changed in each of the two files, and both
are behind a server-backed fork whose else-branch is the original.

Two smaller things. The allocation record becomes visible when a write goes on
the wire rather than when the round trip ends, and the write boundary reconciles
under the document lock rather than before it — a save queued during a write-back
was otherwise captured with the placeholder and, for a course the user had left,
had no corrective flush to follow. And the comments that said a refused reclaim
leaves its bytes for server-side reclamation were wrong: nothing collects them,
because the registry entry still names its blob and the sweep that would remove
it is not wired up. They now say the bytes leak.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): make a deferred pass re-earn its right to run, and bound the commit it waits on

Serializing passes moved their body out of the block that launched them, and
three things followed from that.

A pass now wakes when its predecessor settles, which can be after the user has
left the course. It enqueued before it looked at its signal, into a task table
keyed by element id alone — and placeholder ids are not unique across courses,
which is why the classroom clears that table on arrival. So a departing course's
pass seeded the arriving course's table with tasks carrying the wrong stage id,
and a Retry routes by that id: the reference went into the wrong document. A pass
now re-validates after the wait, before touching anything shared.

The same lateness broke the skip test. `documentSkipIndex` answers only while the
live store is on the pass's stage, and returning nothing put the collection loop
on the browser-only rule — a silent demotion from "the document is the authority"
to "this browser's task table is", on exactly the path where that table has just
been cleared. Every element the predecessor had committed was collected again,
paid for again, and its second write-back found no placeholder to rewrite, so its
bytes were parked where nothing will ever reference them. In server-backed mode
an unreadable document now means the pass stands down.

And waiting was unbounded. A commit is uncancellable: the asset client takes no
signal, and a document write cannot be half-undone. One stalled upload therefore
froze the course's media generation for the session — the replacement never
collected, the element sat on a skeleton that draws no Retry, and only a reload
recovered. The pass's signal is now threaded into the media proxy fetch, and the
commit is bounded by a deadline. The deadline is on the wait, not the work: the
commit carries on, and if it lands late the document simply ends up correct,
while the element becomes retryable and the queue moves on.

The tests that were meant to pin the previous round were not sensitive to it.
Two asserted end states where the mechanism only changes ordering, and one of
them rigged the document read so the assertion held whether or not the pass had
waited; a third covered half of what it claimed. They now observe the ordering
directly — nothing is issued while another pass for the course is working; in
browser-only mode a second pass reaches its provider immediately — and the
reconciliation under the document lock has a test that fails when it moves back
outside it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* revert(media): drop the commit deadline and the abortable download

The deadline bought less than it cost. Abandoning a commit after two minutes
makes the element retryable while the real commit is still running, so a Retry
starts a second commit for the same placeholder against the first: two provider
calls, two allocations, and whichever lands second stamps its result over the
other's task by element id. The allocation record is keyed by placeholder, so the
loser's cleanup erases the winner's record, and the write boundary then puts the
raw placeholder back into the document. That is the overlap serial passes were
built to remove, reopened through the one door serialization never covered.

So a stalled commit holds the course's media queue until it settles or the page
is reloaded, and that is written down rather than papered over. The wait is
unbounded on purpose: every ceiling on it turns out to be a way of running two
commits for one element.

Threading the pass signal into the download was also a mistake, in the other
direction. The provider call that produced the URL has already been billed, so
cancelling the download throws away work that is paid for — and the shared proxy
cache records a cancelled request as a transient failure against that URL, which
after three of them blocks it for every consumer in the session. Browser-only
mode never asked for this: it had no way to observe an abort there, which is
exactly why the bytes were kept. The signal is gone from the download again, and
`fetchAsBlob` is byte-for-byte what it was before this branch.

The regression guard for the stranded-element rule is restored alongside the
timing test that was meant to supersede it. It catches a different rule — a task
left pending being read as answered — and nothing else does: making the pass skip
pending leaves every other suite green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): stop asking the pool for refs it never issued, bound it, and adopt cached bytes

Four things a deployment found once this was running for real.

A reference this application mints itself — a generation placeholder, a derived
narration key — was never in the pool, because the pool allocates every id it
holds. Asking anyway used to be an IndexedDB miss; once the pool is server-backed
it is a request that answers 404, one per element per load, forever on a course
that still holds placeholders. Every lease and probe now checks first. The check
is a negative test on shapes this application owns, not an id validator: the
pool's id domain stays unconstrained, and anything that is not one of ours is
still asked about.

The asset store can bound how much one principal holds, and enforces it inside
the write transaction, but nothing ever passed the number. It does now, with a
default rather than an opt-in: allocation is reachable by any caller a
deployment admits, and with one shared principal an unbounded store is
unbounded database growth with no operator-visible brake.

Refusing asset mutations to unauthenticated callers was not enough, because
every authenticated caller resolves to that same shared principal — so
authentication decided nothing, and any signed-in visitor could delete any id
they learned. Since this branch began storing media the registry is the only
copy a course has. Replacing and deleting are now refused to everyone, and the
browser no longer tries: an entry nothing references waits for server-side
reclamation instead. What a browser must still do is forget its own record of an
allocation that reached nothing, or a later save would stamp an id the document
has no reason to trust.

And a course generated before any of this holds placeholders in its document
with its bytes only in the author's browser. Those bytes are paid for, so the
author's next load converts them — stored to the pool and written back through
the ordinary commit path, with no provider call — instead of buying them again.
A row that records only a hosted URL is treated as absent: that URL is the
provider's address, not something a document may hold.

One renderer expectation moved with this. An untracked placeholder used to paint
as pending on first render because asking the pool left a lease in flight; it
settled to disabled a moment later either way, and now says so from the start.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): surface a full store as a refusal and convert legacy narration

A quota refusal reached the browser as HTTP 500 with a generic message, which
reads as a transient failure: the element kept a Retry that would pay a
provider again and be refused again. The store raises the contract's own error
and the handler maps it to 507, but the store answering a request is not always
built by the same bundle as the handler -- the persistence provider is reached
from the route bundle and from instrumentation, which is why its state lives on
a Symbol.for global -- and `instanceof` is false across that boundary while the
declared code is still right. Classify on the code as well as the class, and
make the code a permanent, persisted refusal in the browser: recorded locally so
it survives a reload, shown as "storage is full", and refused by the retry entry
point so a stale button cannot buy a second generation. Every other storage
failure stays retryable.

Convert what a pre-server-backed course still holds. Generated media is adopted
under either key this application has used for it -- the placeholder, and the
allocated id of a course converted once and later rolled back -- instead of only
the first. Narration is converted by a load-time pass over the open course's
speech actions, since nothing re-enters generation for an action that already
has an id: bytes to the pool, id written back through a funnel that mirrors the
media one, owner-only and server-backed-only. A line whose bytes are in no
browser is left alone rather than re-synthesized.

Also: the pool guard is now a positive `ast_` test rather than an enumeration of
the shapes we mint (imports never reach the pool, so this is safe in both
modes); the slide ref collection is an exported pure function so its four lease
sites are covered behaviourally; ASSET_QUOTA_BYTES treats every spelling of zero
as opting out and refuses a malformed value at startup instead of falling back;
the abort signal is re-checked after the cache read, before an uncancellable
commit; and the unused `removeAsset` and pool `replace` surfaces are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* chore(storage): release 0.29.1

The asset HTTP handler now recognises a store refusal by the contract code it
declares as well as by its class, so a quota refusal raised in another module
realm answers 507 instead of 500. Same contract, stricter recognition, no API
change: a patch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): make a full store recoverable and adoption course-safe

Narration adoption read the local audio row by its derived key alone. That key
carries no stage id and the table is keyed by id alone, so two courses can mint
the same one -- a PPTX import numbers its scenes and actions deterministically,
which gives every imported deck's first slide `tts_s1_speech-scene-p1`. Locally
a collision only means one course plays another's clip in one browser; adopting
it wrote that clip into the shared document permanently, for every device and
every visitor. A row that names a course is now adopted only into that course,
and a row from before that column existed only when the text it recorded is the
text of the action being converted.

A full asset store was made permanent last round, which was wrong three times
over: it overwrote the refused bytes with an empty blob -- on the conversion
path that row is a course's only copy of its own media -- it kept sending the
rest of the deck to a provider against a ceiling it already knew was reached,
and it left no way back once an operator raised that ceiling. A full store is
neither the content's fault nor the configuration's, so it is now its own case:
the bytes are kept, the pass stops at the first refusal, and the element shows
the reason together with a Retry that re-attempts the upload from those bytes.
Nothing retries automatically, so no one is re-billed.

The narration write-back now reaches the write boundary every producer of a
durable write passes through, not only the dirty mark: adoption never deletes
the derived row, so a snapshot that reverts the rewrite is adopted again on the
next load and allocates a fresh asset every time. Adoption is also mounted by
both classroom surfaces rather than one, takes the course's abort signal, and
re-validates that this browser still has the course open before each write.

ASSET_QUOTA_BYTES is validated from instrumentation, where the README and the
docstring already claimed it was: its only other consumer is lazy and memoised,
so a malformed ceiling let the process boot and then failed every persistence
request, documents and runtime included.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): remember a full store per course, and never lose retained bytes

A stopped pass left the elements it never reached as placeholders with no
persisted record -- deliberately, since nothing was attempted for them. But
that left the next load with no reason not to try: it called a provider for the
next placeholder and was refused at exactly the same point, once per reload,
indefinitely. A full store is not a property of any slide. It belongs to the
deployment and changes for reasons the document knows nothing about, so it is
now remembered once per course in the browser's device KV. A pass that finds
the marker stands down before spending anything and leaves every placeholder
its "storage is full" state and its Retry; the first upload that succeeds
clears it and the next pass runs normally.

Narration adoption latched per course so it runs once per load, and the latch
outlived the abort that leaving a course performs. On a surface that stays
mounted across switches -- the workbench pane is one component for every course
it shows -- owner course A, visitor course B, then back to A skipped exactly
the clips the abort had cut off, and nothing else converts them. The latch is
released with the abort now, and a course adopts one run at a time so a
re-entry cannot hand a clip a second allocation while the previous run's
uncancellable tail is still settling.

A quota-blocked element retried into a network error or a 500 lost the bytes
that were kept for it: the retry deleted the row before attempting the upload
and wrote no replacement for an error carrying no structured code, so the next
retry went back to a provider for media this browser had a moment earlier. The
row now survives until an upload succeeds, the failure handler keeps whatever
bytes the attempt was given, and the retry asks the question a pass asks --
does this browser already hold bytes for this element -- rather than reading an
error code that a second failure has already overwritten.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XypYpLBtk8DB5nT5jyqZ5D

* fix(media): adopt real legacy narration, queue re-entries, report attempt outcomes

Narration adoption admitted a stage-less row only when the text it recorded
matched the action being converted. Both of those columns were added to the
local audio table by the very change that moved narration onto allocated ids,
so a row still carrying a derived key has neither: the rule refused every real
pre-allocation course and passed only on fixtures built from post-allocation
rows. What the row cannot say, the key can. A derived key names two clips only
when two courses share a scene order and an action id, and an action id repeats
only when something other than the generator minted it -- an import numbers
them by slide position. So a key built from a generated action id is adopted on
that basis, a key an import could have reproduced still needs matching text,
and a row that names another course is refused however unique its key looks.

Handing a re-entering caller the adoption run already in flight undid the latch
release it was paired with: that run is bound to the signal the departure just
aborted, so it stops at its next clip while the caller -- which has the course
open and a live signal -- is told the work is done, and an effect replayed as
mount, cleanup, mount adopts nothing at all. A later caller now waits for the
uncancellable tail and scans again, which costs a lookup on a course that has
nothing left and finishes the clips the abort cut off on one that does.

One attempt at an element now reports both facts its callers need instead of a
bare boolean: whether the store refused it for room, and whether bytes actually
reached the store. Leaving a course clears the task table, so a retry that
landed afterwards read "no failed task" as success and deleted the row holding
the only copy of the media. Nothing is inferred from that table any more.

Reading the localStorage property can throw where storage is denied by policy,
typeof included, so the availability check moved inside the guard: this metadata
is best-effort, and a rejection here strands a generation pass that has already
enqueued its tasks.

A retry is never blocked by the per-course "store is full" marker, but a retry
that is refused again re-sets it, and adoption now reads and writes the same
marker rather than issuing one refused upload per clip on every load. The two
canvas element renderers and both thumbnail renderers show the reason beside
the Retry, so a full store does not look like an ordinary failure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): probe a full store instead of standing down, and pair notices with a Retry

Narration adoption was given both halves of the per-course "the store is full"
marker last round: it stood down when the marker was set, and it set the marker
when its own upload was refused for room. Those halves are only safe together
if something can lift the marker, and for adoption nothing could. It has no
affordance of its own, it stood down before reaching its own clear, the media
pass returns before its marker gate when there is nothing to generate -- so a
narration-only deck, or one whose slides are already satisfied, painted no
storage-full element and offered no Retry -- and narration generated rather
than adopted allocates directly rather than through the media commit. The
course's cached narration was then lost for good, where before it converted on
the first load after the ceiling was raised.

The gate is a probe now. A marked course attempts exactly one clip per load:
refused, it stops and the marker stands, which costs what standing down cost;
stored, it lifts the marker and finishes the course. Adoption spends no
provider money, so the whole cost of probing a store that is still full is one
refused upload. Generated narration lifts the marker too.

The three surfaces that gained a failure notice last round drew it for any
failure with a reason, including the one refusal that is reachable without
server-backed persistence, so a browser-only deck painted something it had not
painted before. The notice is drawn beside a Retry and nowhere else, which is
what it was added for and what leaves browser-only output unchanged. Both are
now asserted through the render harness the surface matrix already had.

A caller arriving while a rescan is queued shares it rather than appending
another. One rescan converts whatever the run in flight left and every later
one would find an allocated id on every action, so a chain bought nothing and
turned a single stalled upload into a course that never adopts again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): treat a refusal for room as a fact about one clip, not the deck

The asset store checks each write against the headroom it has left, so a store
that refuses a long opening clip can still hold every short clip behind it.
Narration adoption assumed the opposite: it broke the deck at the first refusal
and then re-attempted that same first clip on every later load, because the
document names it first. A deck whose longest clip exceeds current headroom
therefore never converted the clips that would have fit, with no affordance to
recover it -- the state the probe was introduced to remove, reached through a
narrower door.

An unmarked load now attempts every clip, skipping the ones that do not fit,
and remembers the condition only if the load ends with clips it still could not
store. A marked load spends its single upload on the smallest clip left rather
than the first one named: that is the clip that answers the question the marker
asks, because if the smallest does not fit nothing does. The media pass keeps
stopping at its first refusal, and for a reason adoption does not share --
every element it attempts costs a provider call.

A rescan several callers share took the newest caller's signal, and the newest
caller is not necessarily the one still there: a surface that opened a course
and closed it again would stop work a surface still showing that course was
waiting for, and that surface is latched, so it would never ask again. The
shared run now takes a signal that is aborted only once every caller has left.

The comment claiming the shared rescan contains a stalled upload was wrong --
the rescan is chained off the run in flight, so a stalled upload leaves every
caller pending exactly as a chain would. It claims the bounded queue it
actually provides, and the stall is recorded as a limitation.

The failed-state containers took their stacking classes unconditionally, so
markup differed in browser-only mode even though nothing moved on screen. Those
classes are applied only when there is a notice to stack, and the tests assert
the exact class attribute rather than a substring.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): stop narration adoption writing the media pass's store-full marker

The marker means "do not call a provider for this course". A path is entitled
to write it only if its own refusal cost a provider call, and narration
adoption's refusals cost nothing: it uploads bytes this browser already holds.
The store also checks each write against the headroom it has left, so a clip
that does not fit says nothing about whether a slide's image would. Adoption
was writing it anyway, and one over-long narration clip was therefore enough to
stand a course's entire image pass down on every later load -- on a store that
had just accepted adoption's other clips. The author could still recover each
element by hand, every load, for ever.

Three rounds of narrowing this seam produced a finding each time, so it is
removed rather than narrowed again. Gone: the marker read, the single-clip
probe, the smallest-clip selection, and the up-front read of every row into an
array -- which also retires a sampled-then-stale flag and the retention of a
whole deck's blobs for the length of a run, and returns the loop to streaming
one row at a time.

Adoption's rule is now that every load attempts every clip it holds, once; any
failure skips that clip and the load continues. The noise the coupling was
meant to avoid does not arise, because after the first load the clips still
outstanding are exactly the ones that did not fit -- normally none, or one.

A successful write still clears the marker, and that is a different kind of
statement: a write that went through is a fact this run established, where a
refusal is an inference about what some other write would cost. For a course
whose media needs nothing, adoption and generated narration are also the only
paths that can establish it.

The failure module still documented the deck-wide premise this contradicts. It
now says what is true: the check is per write, and the media pass stops the
deck as a judgement about cost rather than about certainty.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): bound a full store's cost from the store's own arithmetic

Removing the store-full marker from narration adoption removed its bound too,
and the code then asserted the bound was unnecessary. It is, on a store with
room for most of a deck. On the store the whole mechanism exists for -- the
ceiling reached, nothing fitting -- the outstanding set after every load is the
entire deck, so a thirty-clip course posted thirty full blobs on every load,
indefinitely. Each of those is not a cheap refusal: the bytes are uploaded, the
server hashes the whole payload, and only then takes a per-principal lock and
sums every entry that principal owns before saying no.

The bound needs no flag, no key and nothing carried between loads. The store
asks whether `used + addedBytes` exceeds the ceiling, and `used` only grows
while a run is uploading, so a clip refused for want of room implies every clip
at least that large is refused for the rest of that run. The run keeps the
smallest size it has been refused and skips anything no smaller without
uploading it; a smaller clip is still attempted, because it may fit. A deck the
store refuses entirely now costs one upload per successive size minimum instead
of one per clip, and a deck it has room for costs nothing extra, because
nothing is refused. Only a refusal for room lowers the bar: a dropped
connection says nothing about how much room there is.

The deck-wide certainty premise the failure module retracted last round still
stood verbatim at the site that implements the stand-down. Both copies now say
the same thing: the check is per write, and the pass stops the deck as a
judgement about cost rather than about certainty.

The comment on adoption's marker clear now names its price. Narration of a few
hundred bytes fits in headroom an image does not, so a proven write can let the
next pass buy one more image that is refused again -- bounded at one, and the
price of the alternative being a course whose media never generates again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(media): state the adoption bound exactly, and stop three comments describing the old rule

The comment introducing the in-load bound gave its cost as "at most a handful,
and the first load pays the most". Neither clause is a property of the rule. A
clip is skipped only when something no larger was already refused, so a
fully-refused deck costs one upload per successive size minimum in document
order: one when the clips grow, about ln N for an arbitrary order, and one per
clip when they only shrink -- a long opener followed by terser lines is exactly
that shape. And no load is cheaper than the first, because the bound resets per
run and a refused clip stays outstanding. The comment now says that, and points
at what would make it exactly one for any ordering: the store returning its
remaining headroom in the refusal's existing details channel, which the server
leaves empty today.

Two other comments still described the previous rule -- "attempts every clip it
holds, every load" -- one of them twenty lines above the paragraph that
introduces the bound, in the same block. Both now say what the code does.

The bound's soundness is worth stating where a maintainer will look for it:
quota is charged at full length with no discount for a duplicate, the sum it is
checked against joins entries to blobs so the collector cannot lower it, the
check takes a per-principal lock before summing, and replace and delete are
refused to every browser. Nothing a run can do makes room appear inside it.

One test installed a row implementation and replaced it wholesale a few lines
later, so the first was dead and the survivor dropped the text the first clip's
import-shaped key needs for the ownership rule -- it passed on the coincidence
that the fixture's default text is the action's. Merged into one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(media): keep a Retry from re-buying parked media, and state the store seam once

Five findings from an inline review.

The standalone classroom route asked the ownership sidecar once per load and
recorded only stage ownership when that ask failed. Every non-answer fails
closed, so one transient 5xx left the genuine author with no resume, no Retry
affordance and no legacy narration converted for the rest of the load, with
nothing to change it short of a reload. The failure now records the fail-closed
answer explicitly -- an answer an earlier load established must not outlive the
failure that replaced it -- and an unresolved answer is asked for again, a few
times over a few seconds. A real answer, however unwelcome, is final.

A Retry could pay a provider for media the pool already held. When the bytes
are stored and only the write-back fails in a way that keeps the allocation, it
is parked and no local row exists, because that row is written only after a
successful write-back. Retry now reads the parked queue exactly as the pass
does and re-attempts the write-back: it re-keys the task done when the document
takes it, leaves the entry parked when the slide still does not exist, and
stays failed and retryable when the document refuses again.

Object URLs a parked allocation owns are revoked when the entry is dropped. The
commit path leaves them alone while the entry is parked, because it is then the
only thing holding bytes this tab can render, so a course switch or a stage
deletion was pinning the whole blob for the life of the tab. An entry a slide
has already taken is left alone: the task table is displaying those URLs.

The fallback lookup for cached bytes is a stage-scoped scan, and the keyed
lookup misses for every row the commit path writes, so a pass was materializing
and sorting the course's whole media table once per element. One scan per pass
now, built on the first miss. It is sound and not merely cheaper: an element
asks only for its own placeholder, and every row a pass writes carries the
placeholder of the element that wrote it.

"The store accepted a write, so it is not out of room" was enforced at three
call sites under slightly different conditions, which made it a convention the
next pool write path could silently break. It is stated once, in putAsset, for
the course whose bytes it just stored.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-09-11 19:38:08 +08:00
puxiao 2baf616843 fix(materials): sanitize owner id in byte-store key so uploads work on Windows (#1426) 2026-09-11 12:42:22 +02:00
Mahesh Singh 57782f928f fix(tts): reject non-audio 200 responses before storing or billing (#1395) (#1405) 
* fix(tts): reject non-audio 200 responses and resolve audioUrl envelopes (#1395)

* fix(tts): address review feedback on #1405 — drop follower, protect raw pcm, test billing invariant
 2026-09-10 20:52:35 +02:00
wyucandwyuc 0bab621e09 harden input validation and outbound URL handling (#1387) 
* harden input validation and outbound URL handling

- Validate the classroom id on the write path with the same allowlist the
  read path already applies, and assert in the storage layer that a resolved
  classroom file stays inside CLASSROOMS_DIR.
- Apply validateUrlForSSRF at every call site regardless of NODE_ENV; the
  guard already owns the ALLOW_LOCAL_NETWORKS escape hatch, so the extra
  environment condition only disabled the check outside production builds.
  Add a repository-scanning test so a gated call site cannot reappear.
- Re-validate every redirect hop of an outbound provider request through a
  shared transport, mirroring the per-hop loop proxy-media already uses.
- Restrict stored slide HTML to the formatting vocabulary the renderer
  produces, sanitizing at the classroom persistence boundary on both write
  and read, with a separate policy for KaTeX snapshots.
- Refuse the development persistence authenticator under NODE_ENV=production
  unless an explicit opt-in is set, and document it in .env.example.

* drop credential headers on a cross-origin redirect hop

The manual redirect loop reused the caller's init on every hop, so provider
credentials were re-sent to the redirect target even on a different origin.
The platform fetch drops Authorization itself when it follows redirects, so
mirror that: strip authorization, api-key, x-api-key and x-goog-api-key
before a cross-origin hop, matching case-insensitively and preserving the
shape of the caller's headers. Same-origin hops are untouched.

A streaming request body cannot be replayed onto the next hop, so fail with
a clear message rather than sending an empty body.

---------

Co-authored-by: wyuc <dhq1024@proton.me>
 2026-09-05 23:56:37 -07:00
Yizuki_Ame eb2cbc701a feat(pro): generate concise conversation titles (#1275) 
* feat(storage): add automatic conversation title lifecycle

* feat(agent): generate concise conversation titles

* feat(agent): schedule automatic conversation titles

* fix(agent): harden automatic title lifecycle

* fix(agent): refine conversation title prompt

* fix(agent): harden automatic title boundaries
 2026-09-04 10:52:25 -04:00
white638white638wyuc
 		dd74cfe23b feat(web-search): add Exa provider (#1342) 
Co-authored-by: white638 <212083322+white638@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-03 06:14:05 -04:00
宙见·星野andwyuc 9b7b42c4c4 fix: normalize GPT Image 2 generation sizes (#1346) 
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-09-03 04:48:42 -04:00
Yizuki_Ame 9d16e68296 fix(workbench): persist Pro conversation titles (#1273) 
* feat(storage): persist manual session titles

* feat(agent): add session title patch route

* fix(workbench): preserve renamed session snapshots

* fix: use typed session title store

* style(storage): format session title tests

* test(storage): align agent session schema contract

* fix(workbench): fence stale session title bootstrap

* fix(workbench): close session title races

* fix(workbench): harden session title consistency

* fix(workbench): preserve unconfirmed session titles

* fix: harden session title reconciliation

* fix: finalize session title consistency

* test(workbench): document session title recency
 2026-08-31 04:21:01 -04:00
wyuc dfebbcf33f perf(classroom): speed up classroom loading (media hydration, sidebar thumbnails, media range requests) (#1276) 
* perf(classroom): defer non-priority media blob hydration off the load path

Entering a classroom awaited the full mediaFiles restore — one object URL
per restored image/video blob — before the loading gate opened. For
video-heavy courses that is hundreds of MB of IndexedDB materialization
blocking first paint.

Split the restore into two phases:

- The awaited phase now builds metadata-complete task entries but only
  creates object URLs for failed rows (none needed) and for media
  referenced by the scene the classroom opens on (persisted cursor, else
  the first scene). buildRestoredMediaTasks gains an optional
  shouldHydrateBlob predicate, defaulting to eager hydration so existing
  callers are unchanged.
- Remaining blob-backed records hydrate in the background, chunked over
  requestIdleCallback (setTimeout fallback). Each deferred task stays
  'done' — so generation resume never re-runs it — but carries no
  objectUrl yet, which the media resolution state machine already renders
  as a pending skeleton until the URL lands.

Background hydration is guarded per record: a task that was replaced
(classroom switch, regeneration, retry) or that belongs to another stage
is skipped and its freshly minted URLs are revoked immediately.

* perf(classroom): lazy-render slide thumbnails in the playback sidebar

The playback scene sidebar mounted a full SlideCanvas for every slide
scene the moment the classroom opened (and every off-screen video element
opened a preload="metadata" fetch), because SlideThumbnail's existing
visible prop was never passed.

Extract the editor nav rail's near-viewport IntersectionObserver hook to
lib/hooks/use-near-viewport.ts and gate the sidebar's slide thumbnails
through it: only scenes within 200px of the viewport render the live
canvas; the rest show SlideThumbnail's existing placeholder until scrolled
near. The placeholder keeps the same box size, so gating never shifts
layout.

The shared hook now starts hidden instead of eager: with the previous
eager-initial state, opening a long deck still mounted every canvas for a
frame before the observer could flip off-screen items off. The observer's
guaranteed initial delivery flips near-viewport items within a frame; the
no-IntersectionObserver fallback (e.g. jsdom) defers to a microtask so
the effect never synchronously re-renders.

* perf(media): lazy image decoding and HTTP Range support for classroom media

Renderer: BaseImageElement's <img> now carries loading="lazy" and
decoding="async", so thumbnail-heavy surfaces (playback sidebar, editor
nav rail, course cards) no longer fetch and decode every slide image up
front. In-viewport images are unaffected — the browser fetches them
immediately. slideToPng forces eager loading inside its permanently
off-screen snapshot tree, where lazy images would otherwise never fetch
and exports would capture blank slides.

Server: GET /api/classroom-media/[classroomId]/[...path] now answers
single byte-range requests with 206 Partial Content (Content-Range,
Accept-Ranges, correct Content-Length), enabling progressive playback and
seeking for hosted video/audio instead of downloading whole files. Suffix
ranges are supported; unsatisfiable ranges get 416 with the full size;
unsupported units or multi-range sets fall back to the plain 200 full-body
response, which is always a legal answer. Existing caching headers are
kept on every response shape.

* chore(renderer): bump version to 0.1.4 for image lazy-loading change

* fix(classroom): guard deferred hydration against restarted tasks and bound idle waits

A deferred record is only ever 'done' without an objectUrl; a task that
regeneration or retry restarted passes through pending/generating, so skip
those instead of attaching stale persisted bytes. Also give the idle
scheduling a timeout so a busy main thread cannot starve hydration.

* fix(classroom): stop superseded deferred hydration before minting URLs

A restore epoch captured at apply time now gates each idle chunk: loading
another classroom or reloading the same one invalidates any older hydration
loop, so it neither keeps scheduling work for an abandoned classroom nor
attaches bytes read by an older load to a newer load's tasks.

* fix(classroom): keep 416s uncached and classify element-keyed media as priority

A 416 with public immutable caching can poison the media URL for later
valid requests, so range errors now send Cache-Control: no-store. Priority
classification also collects the opening scene's media element ids, since
task lookup binds records keyed stage:<elementId> even when the slide slot
carries a different opaque ref.

* fix(classroom): tie deferred hydration liveness to the classroom load token

The restore epoch only advanced when the next load reached apply, so an
abandoned classroom kept hydrating during the next load's storage/network
phase. Compose the epoch with the load's isCurrent (load token plus effect
cleanup) so navigation stops the loop at the next idle boundary.

* fix(classroom): classify legacy-recovered media before deferring it

Legacy singleton video recovery assigns a placeholderRef only at the end of
the task build, so a record keyed by an allocated id without placeholderRef
was deferred even when it backs the opening scene's gen_vid_* element. Run a
metadata-only build first to learn each record's effective ref and classify
against it, keeping the first visible page's legacy video eager.

* fix(action): wait for deferred video bytes before starting play_video

executePlayVideo treated status done as immediately playable, but a deferred
restore is done without an objectUrl: the renderer shows a skeleton, no
<video> exists, and the later hydration never retriggers play, leaving the
action stuck until the safety timeout. Readiness now follows the renderer's
contract (only done-with-bytes is playable) at the initial check, the
subscription exit, and the post-subscription recheck; the failed skip applies
whether or not a wait happened.

* fix(classroom): include the stage whiteboard in priority media refs

The stage-level whiteboard stays open across standalone classroom switches,
so its media can be visible before any scene is. Classifying it as deferred
left visible whiteboard media pending behind idle hydration chunks.
 2026-08-30 03:48:04 +08:00
LING DUANandwyuc 9a28ca39d5 feat(chat): add single PPT element references to Pi (#1224) 
* feat(chat): add PPT element references to Pi

* fix(chat): include chart values in reference routing

* fix(chat): harden slide element reference evidence

* fix(playback): clear element reference on scene navigation

* fix(pi): preserve element grounding across child retries

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-29 14:38:15 +08:00
wyucandClaude Fable 5 04621578de release: OpenMAIC 1.0.0 — the agent workbench (#1228) 
* feat(storage): add an agent-session store with PG backend and layered contracts (#1163)

* feat(storage): add agent-session store with PG backend and layered contracts

* test(storage): avoid BigInt literals for pre-ES2020 root typecheck

* fix(storage): close agent-session store review findings

* docs(storage): align hook ordering and contention-probe claims with the code

* ci: run on the agent-workbench integration branch

* chore(storage): bump to 0.5.0 for the agent-session store

* fix(storage): carry replay compaction across page boundaries

* feat(agent): add the driver model contract and stage route dialect (#1165)

* feat(agent): add the driver model contract and stage route dialect

* fix(agent): validate route context windows and clarify dialect precedence

* feat(agent): adapt the agent-session store and runtime foundations (#1167)

* feat(agent): adapt the agent-session store and runtime foundations

* feat(agent): resolve request owner identity via an anonymous cookie

* docs(agent): document the opt-in compaction default and harden edge cases

* feat(agent): add the background session runner (#1169)

* feat(agent): add the background session runner

* feat(agent): wire the runner into startup behind feature flags

* fix(agent): stop clean interruptions from consuming the attempt budget

* fix(storage): charge the attempt budget for abandoned leases but not clean parks

* docs(storage): document the attempt-charging contract and decouple its tests

* feat(agent): add agent session and owner event streams (#1170)

* feat(agent): add agent session and owner event streams

* fix(agent): close the session-existence oracle and document the owner seam

* feat(agent): add agent session lifecycle routes (#1171)

* feat(agent): add agent session lifecycle routes

* fix(agent): validate session-create input and preserve the owner cookie on errors

* refactor(storage): drop the unused active-stage API from the agent-session contract (#1174)

* refactor(storage): drop the unused active-stage API from the agent-session contract

Tools address stages explicitly on every call, so the store keeps no
mutable session-level stage pointer. Removes resolveActiveStage and
setActiveStage from the store interface, their PG implementations, the
active_stage_changed lifecycle event, the session_active_stage owner
event variant, and the contract tests pinning them. The active_stage_id
column and the DDL check constraint stay untouched for schema
compatibility.

* chore(storage): bump @openmaic/storage to 0.7.0 for the contract removal

* docs: document the agent runtime configuration surface (#1176)

* fix(agent): repair orphaned and late tool results across interruption boundaries (#1180)

* fix(agent): repair orphaned and late tool results across interruption boundaries

A crash, shutdown, or provider failure can leave the durable transcript with
tool calls that have no result, or with results ordered illegally for the
provider. Three failure modes were fixed:

- Orphaned tool calls: a run that died between an assistant tool-call frame
  and its result left a dangling call in the entry tree. Resume no longer
  synthesizes and persists receipts for it: interrupted results are a
  read-time provider view owned by a shared read-boundary repair, which
  returns the original array for a healthy transcript and never mutates the
  tree.

- Late parallel results: a parallel tool can finish while pi unwinds an
  aborted assistant frame, leaving result(A), assistant(aborted), result(B)
  in durable order. Strict providers reject non-contiguous results, so the
  read-boundary repair moves existing results next to their owning assistant
  frame (in call order), omits incomplete unwind frames, and synthesizes
  receipts only for genuinely missing calls.

- Interrupted calls at the write boundary: a call still in flight when the
  run winds down (shutdown, lease loss, cancellation, provider failure) had
  no receipt at all. The runner now tracks in-flight calls from their
  assistant frames and, before the terminal flush, appends an
  interrupted-result receipt for each still-orphaned call through the same
  attempt-fenced write chain, so a lease-stealing zombie never writes and
  the next claim sees a provider-safe transcript.

* test(agent): pin the runner wiring for interruption-boundary tool repair

* feat(agent): add neutral tool foundation libraries (#1184)

* feat(agent): register a web_search tool on the session runner (#1185)

* feat(storage): add a per-session URL trust gate (#1186)

* feat(agent): add the skills system (#1189)

* feat(agent): add the skills system (builtin directories and durable user skills)

* fix(storage): serialize the user-skill quota check-and-insert per owner

Two concurrent creates at the 50-skill boundary both counted 49 rows and
both inserted (READ COMMITTED, no lock), overshooting the quota contract.
The create transaction now takes a per-owner pg_advisory_xact_lock first,
and the same-name idempotency check runs before the count check so an
at-least-once retry of the create that committed as the owner's 50th row
still returns its durable receipt instead of a quota error. The 23505
backstop is retained for writes that do not take the lock.

* fix(agent): share unstorable-character validation and align skill lookup

* feat(agent): add session materials and a fetch_url tool behind the URL trust gate (#1190)

* feat(agent): add session materials and a fetch_url tool behind the URL trust gate

* fix(agent): harden session material fetching

* feat(storage): add an ownership scope to stage documents (#1191)

* feat(agent): add material read and search tools (#1192)

* feat(agent): add stage read and patch tools (#1194)

* feat(agent): add page generation and deck editing tools (#1198)

* test(storage): keep the PG contract suite order-independent (#1200)

* fix(agent): revoke deleted-session URL authority and reject private ISATAP endpoints (#1199)

* fix(storage): revoke deleted session URL authority

* fix(ssrf): reject private ISATAP endpoints in strict fetches

* chore(storage): bump to 0.11.1 for the session-URL authority fix

* feat(agent): add roster and voice registration tools (#1201)

* feat(agent): add folder organisation tools (#1202)

* feat(api): add stage and material HTTP routes (#1203)

* feat(workbench): add the client data layer (#1204)

* feat(workbench): add the client data layer

* docs(workbench): write the ported comments in English

* chore(edit): remove the in-editor agent panel (#1210)

* chore(edit): remove the in-editor agent panel

* style: apply prettier formatting

* fix(agent): report the runtime as unusable without a database (#1207)

* fix(agent): report the runtime as unusable without a database

* style: apply prettier formatting

* feat(agent): add image, video and pptx import tools (#1211)

* feat(workbench): add the agent chat surface (#1205)

* feat(workbench): add the agent chat surface

* docs(workbench): write the ported comments in English

* fix(workbench): label the folder and rename tools on the timeline

* fix(workbench): label the roster and voice tools on the timeline

The reconciliation test iterates every tool the runner registers and
requires a display label of its own. The roster and voice-clone tools
(list_voices, set_roster, clip_audio, register_voice) reached the
integration base with the roster/voice-registration tools but never
gained presentation rows, so they fell through to the default branch
and rendered their wire names. Port their rows from the reference
implementation (labels and i18n keys verbatim) and extend the
reconciliation allowlist with ROSTER_TOOL_NAMES and
VOICE_CLONE_TOOL_NAMES, so a future tool cannot enter the product
without a label.

* feat(agent): add the material extraction lifecycle (#1212)

* feat(storage): add material extraction lifecycle

* feat(agent): execute queued material extraction

* style: apply prettier formatting

* style: satisfy prefer-const in the extraction runner

* test: give material fixtures the extraction lifecycle fields

The media-tools slice and the extraction lifecycle slice were each green
in isolation but never compiled together: the lifecycle made derivedFrom
and extraction required on AgentSessionMaterial while the media-tool
fixtures predate them.

* chore: remove stray task notes

* fix(workbench): label the extraction lifecycle tools on the timeline

* feat(workbench): add the workspace shell (#1206)

* feat(workbench): add the workspace shell

* docs(workbench): write the ported comments in English

* i18n(workbench): align workspace keys across locales

* fix(workbench): adopt the landed data layer and label the extraction tools

- replace the sibling-slice seam stubs with the real data-layer modules
- drop ambient declarations now shadowed by landed files
- port timeline labels for the extraction lifecycle tools from the reference
- align the new i18n keys across all locales

* ci: retrigger

* feat(api): folder routes, stage-meta viewer surfaces, and the material upload contract (#1215)

* fix(storage): restore capability-based stage access

* fix(api): bind document access to request owner

* fix(agent): restore three-state stage access on the tool layer

Port probeStageAccess and the three-state StageAccess (owned / foreign /
missing / tombstoned) and gate every stageId-bearing stage tool on an owned
probe, mirroring the reference per tool:

- move_to_folder, rename_stage, read_stage_outline refuse a non-owned stage
  with the single not-yours message before touching the store.
- The course/DSL toolset and the roster toolset are wrapped by
  withOwnerStageAuthorization: read_stage, patch_stage, grep_stage and every
  writer refuse a foreign stage with the same message and refusal shape.
- Scene preview keeps its own probe and its own refusal text, and is
  registered beside the course toolset (never double-gated).
- The runner injects one probe factory at the three call sites.

Tests: the dsl cross-owner test premise (a foreign stage is readable by id)
encoded an invented capability-read policy that the reference does not have
at the tool layer; it now asserts foreign read/patch/grep are all refused
while the owner still reads. Curriculum cross-owner assertions were already
the reference's and now pass with the probes in place.

* docs: correct per-file test counts in the fidelity report

* test: fix type errors in stage-access fidelity test

* test: adapt media-tool and gate suites to the owner-scoped store seam

* feat(api): add owner-scoped course-folder HTTP routes

Port the reference implementation's /api/folders family (list, create,
rename, delete with ungroup/remove modes, and folder membership) onto the
owner-bound document store, replacing its provider-based auth with the
existing withRequestOwnerId / owner-scoped store seams.

The storage package's folder store grows the pieces the routes need:
DocumentFolder.order (schema column + max+1 assignment + ordering),
renameFolder, deleteFolder(mode) with captured member ids, and
setStageFolder(stageId, folderId | null) with idempotent un-filing.
FolderNameError moves into folder-name-validation.ts (stage-storage re-exports
it, keeping import sites intact).

Every route gates on the configured agent runtime (plain 404 when off or
unconfigured), keeps the reference's machine codes and envelopes, and is
covered by gate tests plus a behavior suite.

* feat(api): add stage-meta viewer surfaces for the classroom

Port the reference implementation's viewer-facing stage state — can-edit /
collected / published / generation-complete — on top of the stage-access
base (stage_meta + tombstones). stage_meta gains published_at and
generation_complete columns plus a stage_bookmarks table; the reference's
deployment-specific origin/claimed_at columns are stripped.

New gated routes: GET /api/stage-meta/[stageId] (per-viewer facts, 404 for
absent/tombstoned, never returns the owner id), GET /api/stages/[id]/status,
POST generation-complete / publish / unpublish (owner-only), POST
/api/bookmarks. The resolver lives in lib/server/stage-access.ts.

Wiring: a fetchStageMeta client with the reference's three-outcome contract,
stage-store isOwner/isBookmarked/readOnly fields (upstream single-user
defaults, no-op until the sidecar answers) plus setViewerAccess, the
classroom apply path computing readOnly = !(isOwner || isBookmarked), the
Stage editability gate, and a sidecar probe after each classroom load. A
sidecar 'absent' answer keeps the editable default here because the
classroom also serves local-only courses; server writes stay owner-enforced.

* feat(api): port the reference material upload contract

Rewrite POST /api/materials to the reference implementation's upload shape
so the workbench uploader (uploadWorkbenchMaterial, which posts no session
id and expects a flat 201 view) works unchanged: owner-scoped upload with
mime normalization/validation (415), per-class size caps checked on the
declared content-length and the streamed body (413), empty body (400),
quota (429), sha256 reserve->store->finalize lifecycle with abandon on
failure, flat { materialId, originalName, bytes, mime, extraction } 201,
and an x-request-id echo.

Adds the owner-scoped material library (owner_material table + quota +
24h lazy sweep, bytes in the host's asset registry as the neutral
replacement for the reference's object-storage byte path) and the material
cap configuration. The session-scoped GET list is left as-is; the
reference's owner-material extraction worker is not ported (the branch's
session-material extraction lifecycle already covers extraction).

Gate tests now cover all 23 persistence routes across the three runtime
env states; the materials behavior suite pins the new contract.

* feat(media): add an optional local ffmpeg media extractor (#1213)

Adds a local ffmpeg/ffprobe pipeline as a second media extraction
provider behind the extractor registry, ported faithfully from the
reference implementation: duration probing, keyframe-safe chunking,
per-chunk ASR with timeout and deadline budgets, and timestamped
transcript assembly.

- Availability probing feeds the registry's candidate selection: the
  provider simply is not a candidate when ffmpeg/ffprobe are absent.
- With neither ffmpeg nor a cloud provider configured, extraction fails
  with an actionable message naming both enablement paths.
- Media materials route through the same extraction lifecycle and lease
  fence as documents; no parallel queue.
- Tests inject the executable resolver so the missing-ffmpeg path is the
  default-tested one; the real pipeline test is skip-if-unavailable.
- @openmaic/storage 0.13.0 -> 0.14.0 (media routing in the material
  lifecycle surface).

* feat(storage): per-scene monotonic revisions via database triggers (#1214)

* feat(storage): per-scene monotonic revisions via database triggers

Restore the reference implementation's freshness granularity: a
per-scene monotonic revision maintained by database triggers, so every
writer (HTTP routes, agent tools, jobs, manual SQL) bumps it without
application cooperation.

- Companion revision tables + trigger functions in the storage package's
  idempotent schema bootstrap, with the lock-order invariant, pg_notify
  wakeup and the suppression switch for batch writers.
- ensureDocumentSchema gained a dollar-quote-aware statement splitter.
- The freshness and manifest routes serve per-scene revisions.
- Mutation-verified: dropping the triggers turns the revision tests red.
- @openmaic/storage 0.13.0 -> 0.14.0.

* fix: forward the freshness manifest through the owner-bound store

* feat(workbench): add the Pro entry points and preserve the mode-transition semantics (#1208)

* feat(workbench): add the Pro entry points

* feat(workbench): preserve Pro mode transition semantics

* fix(workbench): drop ambient declarations shadowed by landed slices

* fix(workbench): drop ambient declarations shadowed by the landed shell

* feat: port workspace shell sibling modules

Port the 16 leaf modules the Pro workspace shell imports but that were only
ambient-declared, replacing the compile-time bridge with real implementations
adapted from the sibling-slice reference: pure workbench helpers (session
title, rail tab, course-chat bootstrap, created-course tabs, course-tabs
memory, workspace navigation, pane navigation, pro-edit sizing,
existing-course minting, first-message session), the neutral brand context and
course-rename server API, the server-action session delete, the home
discovery hook, the classroom pane host with its load-policy leaf, the theme
toggle and floating-layer owner, plus the floating-layer-owner wiring the
dialog/dropdown/tooltip portals stamp.

Also add the workbench-shell locale copy for all 12 locales, port the
reference tests for the ported modules, and drop
types/workbench-sibling-slices.d.ts now that every declaration has a real
implementation.

* docs: keep ported comments in English and deployment-neutral

* docs: announce 1.0.0 and refresh the feature overview (#1216)

* docs: announce 1.0.0 and refresh the feature overview

* docs: finalize 1.0.0 README after feature merge

* fix(agent): control-plane routes answer 404, not 500, without a database

The agent control-plane routes gated only on the runtime flag, so an
enabled-but-unconfigured deployment (flag on, DATABASE_URL empty)
answered 500 from a store that cannot connect. Gate them on the
configured check instead, matching the stage/material routes: the
whole surface is cleanly absent until both the flag and the database
are present. The status probe keeps reporting both bits.

* test: mock both runtime gate exports in the control-plane route suites

* fix(agent): abort in-flight TTS on cancel and bound each provider request with a timeout (#1217)

The generate_tts / scene-tts path checked the runner's AbortSignal between
actions but never created the provider HTTP requests with it, so a session
cancel left a hung synthesis fetch in flight until a restart repaired the
tool result. Thread the signal end-to-end: TTSModelConfig carries an
optional signal, generateTTS combines it with a per-request timeout
(TTS_REQUEST_TIMEOUT_MS, default 30s, ported from the reference runtime's
TTS bounds) via AbortSignal.any, and every provider fetch (openai, azure,
glm, qwen incl. voice-clone + audio download, voxcpm, minimax, doubao,
elevenlabs, lemonade) is created with that signal.

A timeout now fails the tool call with TTSRequestTimeoutError (a clear
retryable error) instead of wedging the session; a caller cancel propagates
as the interruption so the runner settles the session as cancelled without
a restart.

Tests: hung-provider simulation rejects at the timeout with the retryable
error; abort mid-flight aborts the captured request signal and surfaces the
interrupted shape; removing the signal wiring makes the abort tests fail
(red), restoring them turns green.

* fix(workbench): PG-mode home listing via owner stages; keep the interrupted terminal course card (#1218)

Finding 1: with server persistence on, listStages resolved to the generic
GET /api/persistence/documents listing, which the capability model
deliberately answers 403 FORBIDDEN_DOCUMENTS for (reads by id, listings
owner-only). The home/workspace library now lists through the owner-scoped
GET /api/stages surface (same anonymous-owner cookie the workbench uses)
when server persistence is enabled; the server-side 403 is untouched.

Finding 2: a run interrupted (session_interrupted) and repaired
(session_resumed) that ends cancelled before agent_end stranded its pending
classroom sightings, so the timeline's terminal card lost the course the
answer produced. session_end (cancelled) now flushes the pending sightings
into the same course card set agent_end paints, before the stopped caption.

* chore(workbench): remove the bookmark concept and the saved-courses drawer (#1219)

* chore(classroom): remove the bookmark ('collected') concept entirely

The stage-meta viewer port introduced a bookmark surface (stage_bookmarks
table, POST /api/bookmarks, the isBookmarked sidecar field, and a
readOnly rule that let a saved course stay editable). The product has no
such concept, so remove it as a closure:

- delete the /api/bookmarks route and the stage_bookmarks table plus its
  query helpers from the persistence bootstrap
- drop isBookmarked from GET /api/stage-meta/[stageId]
- simplify the classroom read-only rule to readOnly = !isOwner across the
  sidecar client, ownership signal, classroom load, stage store and the
  classroom page
- keep publish/unpublish, generation-complete, isOwner and isPublic
  exactly as they were
- update the gate and stage-meta route suites and the README mentions

The workspace rail's Bookmark glyphs and comments describe the upstream
saved-courses (favorites) section, which is driven by isOwner and renders
no collect affordance; they are kept as unrelated homonyms.

* chore(workbench): remove the saved-courses drawer UI

The first pass removed the bookmark data model but kept the rail's
"Saved courses" drawer, judging it a separate surface driven by
`isOwner === false`. The home/workspace listing is owner-scoped, so that
flag can never occur: `allSaved` is permanently empty and the drawer
(plus the collapsed-rail Bookmark mini-button) is a dead affordance.
Remove it: the SavedDrawer component and its mount, the savedOpen /
savedSection state, the allSaved / matchedSaved derivations, the 'saved'
variant of the course-list renderers, the mini Bookmark glyph, the
drawer-only CSS, and the drawer's i18n keys from all 12 locales. The
courses tab is now exactly one folders tree.

The authored/favorites split in workspace-tree.ts goes with it; the tree
module no longer reads `isOwner`. The discovery course type keeps the
field — the shell still reads it for read-only gating.

Upstream has no collect concept; the drawer could only ever render empty
here. The reference implementation HAS this drawer (its favorites come
from its account system), so this removal is a deliberate upstream
product decision, not a fidelity bug.

* fix(workbench): restore the attach entry, add the rail settings entry, pin all three entry points (#1221)

* fix(workbench): restore the composer attach entry by gating it on the live runtime

The AttachButton's rollout probe read a `materialsEnabled` field that this
branch's /api/agent/runtime never answers (the materials routes gate on the
runtime itself, like the stages), so the gate could never pass and the attach
button never rendered — the Pro launch and chat composers showed only the
@-mention and enhance glyphs.

Substitute the field with the runtime's `enabled` value, which IS the upload
action's precondition: POST /api/materials answers 404 whenever it is false,
so the render condition now equals the action precondition (no dead button).

The button's label (`proMode.attach`) is a user-visible string that becomes
visible again; port the reference implementation's own translations verbatim
into the 11 locales that still carried the Chinese copy.

* feat(workbench): add the settings entry to the rail's bottom-left cluster

The reference's rail foot carries a cluster of utilities (its saved-courses
drawer, the language switcher, the display toggle). This branch removed the
drawer — it could only ever render empty here — and the product decision is
to fill that freed spot with the settings entry.

Add a settings trigger to the foot cluster (expanded rail, beside the
language and display toggles, and on the collapsed strip) and mount the
model/provider SettingsDialog in the rail, wired to the trigger. It is the
same dialog the classic home opens from its header pill; the workspace had no
settings entry of its own, so nothing is duplicated within a surface.

* test(workbench): pin the restored upload, attach, and settings entry points

Covers the three restored entry points:

- the courses-tab upload control: rendered beside the course name filter,
  wired to the discovery hook's ZIP import trigger, disabled while an import
  runs, and gated by the same condition as its action (the courses tab);
- the composer attach control: an actual render of AttachButton under both
  probe answers (visible when the runtime says the upload path is live,
  hidden otherwise), its mounts in the launch and chat composers, the
  branch's runtime-field substitution in the probe, and the reference's own
  `proMode.attach` copy in all 12 locales;
- the settings entry: the trigger in the rail's foot cluster (expanded and
  collapsed), beside the language and display toggles, opening the
  SettingsDialog the rail mounts.

* chore(config): the Pro workbench flag implies the MAIC Editor gate (#1223)

A workbench build without the editor toggle has no way to edit a course:
enabling NEXT_PUBLIC_PRO_WORKBENCH_ENABLED while forgetting
NEXT_PUBLIC_MAIC_EDITOR_ENABLED produced exactly that split-brain bundle.
The workbench IS Pro mode, so its flag now implies the editor gate; the
standalone flag remains for deployments that want the classroom editor
without the workbench. Documents both flags in .env.example.

* fix(agent): wake SSE tails and the runner on durable deltas (streaming fidelity) (#1222)

The Pro workbench chat did not stream: the session/owner SSE routes polled
the durable event log on a 5s/30s clock with no wakeup, so message_update
deltas (written at 150ms cadence) reached the browser in poll-sized blocks
and the thinking strip only mounted after the whole reasoning text had
accumulated.

Port the reference's LISTEN/NOTIFY delta path:

- storage: add in-transaction wake hooks (onSessionEventAppended,
  onOwnerEventAppended, onCancelRequested) so a host queues pg_notify in
  the same transaction as the durable append; align readEventsAfterForReplay
  to rank the bounded page so the first delta after the cursor is always
  kept (the live tail can never starve). Bump @openmaic/storage to 0.18.0.
- app: port the process-wide event-notify bus (dedicated LISTEN client,
  self-check probe, reconnect backoff; notify through the storage
  transaction surface), wire the store hooks, subscribe both SSE routes
  before the initial read with the reference's initializing gate, and give
  the runner one {kind:'session'} subscription whose wake runs the cancel
  check and the message drain. Polls stay as the lossy-NOTIFY backstop.
- lifecycle: start/stop the bus from instrumentation.

Tests: storage hook + compaction contract; route wakeup latency; runner
wakeup wiring with a fake agent; bus unit tests; PG contracts proving a
real append wakes the routes and a live SSE route forwards a message_update
on the wakeup, and that a rolled-back append never wakes. Also fix the
pre-existing park-attempt-budget PG test TRUNCATE (missing CASCADE against
newer FK tables).

* fix(storage): asset writes self-deadlocked against pooled PostgreSQL (#1225)

* fix(storage): refuse the non-transactional byte-write deadlock configuration

A byte store whose plain write() runs on its own pooled connection cannot be
invoked from inside a registry write transaction: after the transaction has
claimed the blob-row lock, that write blocks on the lock the transaction just
took while the transaction waits on the write - a self-deadlock PostgreSQL
cannot detect (one side is idle in transaction).

There is no lock-safe ordering for such a writer: bytes must be written after
the row claim (writing before it lets the collector delete the bytes while the
upsert waits), and any second-connection write after the claim is the
deadlock. The configuration is therefore detected and refused:

- AssetByteStore gains writesOutsideRegistryDatabase?: true, declaring that the
  layer's plain byte operations cannot contend for the registry's row locks.
- PgAssetStore refuses put()/replace() up front (and defends coordinatedWrite)
  when the byte store has no writeWith and does not declare the flag, throwing
  a clear configuration error before any row is claimed.
- The collector mirrors the guard on its delete path (deleteWith or a declared
  out-of-registry layer, else a configuration error).
- The object store declares the flag (its out-of-transaction write remains
  legitimate); the in-registry PostgreSQL byte column provides writeWith /
  deleteWith instead.
- Write transactions (put/replace/remove) set SET LOCAL lock_timeout = 30s so
  any future lock-contention variant fails loudly instead of hanging.

Bumps @openmaic/storage to 0.18.0.

* fix(persistence): forward the transactional byte methods through the lazy asset byte-store wrapper

The no-bucket case of lazyAssetByteStore returned a bare { write, read,
delete } and dropped writeWith/readWith even though the underlying
PgAssetByteStore has them. The registry's hasTransactionalWriter duck check
then failed and put() fell back to the byte store's own pooled connection,
which blocks forever on the blob-row lock the registry transaction just took
when the bytes live in the same PostgreSQL - the production self-deadlock.

The no-bucket layer is statically PgAssetByteStore, so its transaction-pinned
methods are forwarded eagerly (typed against the real signatures via
PgForwardedByteStore). The bucket case keeps its lazy-probing semantics: no
transactional writer exists there, the signed-URL method stays absent or lazy
exactly as documented, and the wrapper now declares writesOutsideRegistryDatabase
so the registry may run the plain write inside its transaction.

New tests pin the wrapper's transactional capability red-to-green and assert
put()/resolve() route byte traffic through the transaction-pinned queryable.

* fix(home): cap the generate-prep ingest drain at 3s so Generate never waits the full server budget

The classic home flow's Generate click drained in-flight ingests for the full
15s server budget. Cap the wait at GENERATE_DRAIN_CAP_MS (3000ms, documented as
a UX bound) and reuse the existing timeout fallback: sources that miss the cap
proceed on the legacy byte path and each late-resolving id is released.

* chore(storage): bump to 0.19.0 over the concurrently landed 0.18.0

* fix(agent): bound every tool call with a timeout; never resurrect a cancelled session (#1226)

* fix(agent): bound every tool call with a global timeout and settle it on cancel

A tool await that neither resolves nor rejects wedges the session forever:
the lease keeps heartbeating and the driver never reaches its next cancel
checkpoint. Race every tool execution (in buildAgent) against a hard budget
(OPENMAIC_AGENT_TOOL_TIMEOUT_MS, default 10 min, per-tool overrides for
known long runners) and against the caller's AbortSignal, so even a
signal-ignoring await cannot keep a cancelled session running.

On timeout the call rejects with AgentToolTimeoutError; the agent loop turns
the rejection into a structured error tool-result the agent can retry or
proceed from, and the abort signal is delivered to the tool's in-flight work
through a derived controller. Zombie-tool updates after settlement are
dropped.

* fix(storage): never re-lease a cancel-requested session; settle it as cancelled on claim

The claim scan treated a session with cancel_requested_at set as a normal
claim candidate: after a restart it re-leased the same session for attempt
N+1 and resumed generating despite the pending cancel. claimNextSession now
settles such candidates as cancelled under the claim lock (status cancelled,
attempt reset, lease and cancel request cleared, terminal session_end event
and owner projection) instead of leasing them, then keeps scanning.

Bump @openmaic/storage to 0.18.0.

* docs: takeaway-style 1.0.0 announcement with bilingual guide links

The 1.0.0 head is now a short takeaway block — badge links to the
official user guides (English and Chinese), five one-line highlights,
and pointers into Features and the workbench setup section — instead of
six dense paragraphs. The detailed provider-neutrality and freshness
notes move into the Features workbench section, phrased database-
neutrally (the announcement no longer names a specific database).
Release date corrected to August 27.

* fix(workbench): restore editor chrome, mode transition, streaming, materials, mentions, folders (#1229)

* fix(workbench): wire workspace folder routes

* fix(editor): restore reference workbench chrome

* fix(workbench): persist composer materials and course refs

* fix(workbench): preserve live reasoning frames

* fix(persistence): back off failed streaming saves

* chore(workbench): retire stale slice seams

* test(editor): cover element pin layer

* chore(storage): bump to 0.21.0 for the user-message ref/material fields

* chore(editor): translate ported code comments to English

* fix(agent): fence durable tool writes and consume cancel requests atomically (#1230)

* fix(agent): enforce provider force-off in agent tools and scrub vendor identity from tool results (#1231)

* fix(materials): serialize per-owner quota reservations and make crashed uploads reclaimable (#1232)

* fix(editor): resolve dock-bar i18n keys, remove dock height drag, wire element referencing (#1233)

* fix(workbench): send the opening session message exactly once with refs intact (#1234)

* feat(editor): port timeline TTS preview single-flight and voice-all state latching (#1235)

* fix(media): restore the reference classic media chain (#1236)

* fix(import): adapt imported PPTX canvas size so decks render without overflow (#1237)

* fix(editor): complete element referencing — renderer DOM contract and GenUI picking aligned with the reference (#1238)

* test(providers): reconcile the provider-config vendor-token debt count after the main merge

The integration line's AK/SK fallback for the managed document provider adds
occurrences that main's allowlist snapshot predates. Same mixed-composition
debt category the group already documents; no new vendor behavior.

* test(providers): reconcile vendor-token debt counts with the integration line

The main-merge brought main's neutrality-guard snapshot next to integration
features it predates (media-extractor fallback chain, local voice-profile
deletion semantics, the enabled-TTS helper). Same debt categories the guard
already documents; counts updated to the guard's own tally and two grouped
entries added. No new vendor behavior.

* fix(agent): carry reasoning through the completions dialect so the thinking strip renders (#1239)

* feat(skills): add Feynman and spiral curriculum methods (#1240)

* feat(agent): port missing reference tools and skills (parity audit) (#1241)

* feat(media): retire asset-registry wiring; media and materials follow the reference byte model (#1242)

* fix(classroom): center adapted canvases in the stage and send back navigation home during generation (#1243)

* feat(settings): skill management with real list, download, delete, and upload (#1244)

* feat(settings): skill management section with real list, detail, and zip download

* feat(skills): owner skill delete and upload across storage, API, and settings

* fixup! feat(settings): skill management section with real list, detail, and zip download

chore: neutralize a reference note in the settings header comment

* fix(media): persist origin-independent classroom-media references from the agent runtime (#1245)

* feat(editor): float the insert toolbar in the outer frame with collapse (#1246)

The insert strip was bounded to the slide card, so it could only ever sit on
top of slide content: the card's overflow clipped it and it could not be parked
in the padding beside the slide. Move it into the studio frame the element
picker's panel already roams (CanvasOverlayPortal + the frame selector), so both
canvas overlays share one bounding container and their handles behave the same.
While picking, the strip rises over the picker and goes inert, which is the
z-order CANVAS_OVERLAY_Z already documents.

Add a fold beside the grip: the chevron collapses the strip to that grip row and
back, with the buttons unmounted rather than hidden. The fold is session-local
state owned by EditShell, next to the drag offset, so a surface swap keeps it;
nothing is persisted. Expanding a strip parked at the bottom edge re-clamps
through the same bounds rule the keyboard move uses.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* fix(workbench): align the chat timeline's left edge with the composer (#1247)

* fix(agent): fence session claims while an ask_user question is outstanding (#1248)

* fix(agent): settle-time rescue tracks real delivery instead of a count offset (#1249)

* fix(persistence): migrate owner_material to oss_key and drop legacy asset_id (#1250)

* docs(readme): surface the 1.0.0 user guide badges at the top (#1253)

* fix(workbench): show newly created folders in the sidebar without reload (#1254)

* docs(readme): add the release version prefix and drop the opt-in framing

* fix(workbench): single-source the chat gutter so timeline and composer share a left edge (#1255)

The transcript and the composer each established their own column: their own
`px-*` gutter and their own `mx-auto w-full max-w-*` centering wrapper. Equal
padding values were never enough, because the two columns are centered inside
different containing blocks — the transcript's is a scroll container, whose
content box is narrower than the composer footer's by the scrollbar's width:

  transcript text left = pad + (pane - 2*pad - scrollbar - measure) / 2
  composer box  left   = pad + (pane - 2*pad             - measure) / 2

The padding cancels out of the difference and what remains is `-scrollbar/2` at
every padding value, so the transcript sat half a scrollbar to the left of the
composer and tuning the two paddings against each other could not move it.

The column is now established once, by the nearest common ancestor of both
(`chatColumn`), and the scroll viewport and the composer footer are siblings
inside it that add no horizontal inset of their own. The cap carries the gutter
on top of the 760px reading measure, so the text column keeps its width. The
handed-over question row drops the padding that indented it past the agent's
prose; framed rows keep their own inner padding, which is what a card's border
sitting on the column edge means.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* fix(workbench): lock pane-embedded classroom to edit mode (#1256)

The workspace right pane painted the full learning chrome — speed control,
play button, learner avatars, mic bar — for a course the agent had just
created, then flipped to edit once the first scene landed.

resolveStageChromeMode treated playback as the DEFAULT branch for a hosted
classroom, so every shortfall fell into it: a course whose tab opens at
stage_link time has no scenes yet, so currentSceneId is null and
isHostedSceneEditable is false. A folded pane parked the playback root
behind the fold and cross-faded it out over the pane on unfold, and a
failed editor chunk dropped into playback permanently.

Lock it at the pane instead of defaulting per entry path:

- WorkbenchPanelProvider — the single element that mounts a classroom into
  the workspace — publishes editPinned (visible && !playback). Every entry
  path passes through it, so none of them decides.
- The hosted resolution can no longer degrade to playback. Start Learning
  (workbenchLearning, new input, split out from pane visibility) is the one
  door; everything else resolves between the neutral loading shell and edit.
- Stage's chrome dispatch is exhaustive on chromeMode, so the playback root
  is no longer the else-branch of a condition about the current scene.

No flicker: chromeMode is resolved during render, and preloadEditor now
answers synchronously (isEditorPreloaded) so a remount with the chunk
already registered paints edit on the first frame. A failed import is no
longer cached forever, so the lock cannot strand the pane.

Standalone classrooms keep their stored mode unchanged.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-27 19:26:25 +08:00
wyuc 98904c1c5d fix(providers): three silent-behavior fixes from the provider audit (#1196) 
* fix(extract): fail loudly instead of silently falling back to MinerU Cloud

A request that selects the self-hosted MinerU extractor without a
configured base URL used to fall back to MinerU Cloud whenever a cloud
API key was available, sending documents to a third party without the
operator's knowledge. The cloud fallback now requires an explicit
operator opt-in (ALLOW_MINERU_CLOUD_FALLBACK, default off, documented
in .env.example); otherwise the route answers a 422 that names what was
configured (self-hosted MinerU) and what was unavailable (its base
URL), and points at both remedies.

* fix(media): remove the selectable video provider that has no adapter

The Sora entry in the video provider catalog could be selected in the
settings UI and server config, but had no connectivity or generation
dispatch case, so choosing it failed only at execution time. A catalog
entry that cannot execute is worse than an absent one: the entry, its
type-union member, UI names/icons, store defaults, and the VIDEO_SORA
env mapping are removed. A new catalog test pins that every selectable
video provider dispatches to its own adapter in both switches.

* fix(storage): delete the obsolete no-op storage provider abstraction

getStorageProvider() unconditionally returned a NoopStorageProvider and
the module swallowed every operation into silence, so any caller
believing it had storage got neither bytes nor an error. No real caller
exists anywhere in the repo (only lib/storage/client.ts, a separate
module with a documented null contract and a live caller, remains). The
dead entry point, its type file, and the no-op provider are deleted; a
test pins the removal and that the real client upload helper is kept.
 2026-08-27 11:49:23 +08:00
wyuc b8fbac1a54 feat(providers): uniform capability force-off and consistent missing-key contract (#1181) 
* feat(providers): uniform capability force-off and consistent missing-key contract

* fix(providers): close force-off bypasses found in review
 2026-08-26 10:59:43 +08:00
wyuc 4a9f906a13 feat(media): resolve image, video, and ASR models from server config (#1175) 
* feat(media): resolve image, video, and ASR models from server config

The image generation chain never consulted the server-side
IMAGE_<PREFIX>_MODELS config: the route only forwarded the client
x-image-model header and each adapter silently fell back to a hardcoded
vendor default, so a managed provider could generate with a model the
operator never chose. The provider itself also defaulted to a hardcoded
vendor id when the client sent no preference.

- Add resolveImageModel / resolveVideoModel / resolveASRModel and
  resolveServerImageProviderId / resolveServerVideoProviderId to the
  server provider config, mirroring how TTS already resolves
  server-managed settings.
- Routes resolve provider and model server-side when the client sends
  no preference and fail loud with a clear 400 when nothing resolves;
  no more hardcoded vendor defaults.
- Adapters require an explicit model via requireModel; connectivity
  probes keep a fixed probe payload since they are auth handshakes,
  not user-facing generation.
- Tests cover the resolver precedence and the fail-loud paths.

* fix(media): align model resolution semantics across capabilities

resolveImageModel and resolveASRModel let the server pin win
unconditionally, while resolveVideoModel implements allowlist semantics
(client choice wins when it is in the pinned list, otherwise the first
pinned entry). Align image and ASR to the video allowlist semantics so a
client picking any server-pinned model is honored, and update the
resolver docstrings and tests to pin the aligned behavior for all three
capabilities.

The classroom media path has no client model and no HTTP response to
fail loud with: it now falls back to the first catalog model when the
operator pins no _MODELS list, so key-only deployments keep generating
media instead of silently skipping every element via the adapter's
requireModel backstop.

* fix(media): normalize configured model lists and client model ids
 2026-08-25 22:42:22 +08:00
wyucandClaude Fable 5 4f00cf7ad6 feat(materials): material library manifest and id-addressed generation images (#1153 part 2) (#1168) 
* feat(materials): material library manifest and id-addressed generation images (#1153 part 2)

* fix(materials): library-owned references and failure-path parity (#1153 part 2)

* fix(generation): resolve-then-slice vision images and close allocation leaks (#1153 part 2)

* fix(generation): bound vision resolution and make library writes leak-safe (#1153 part 2)

Bound the resolve-with-refill phase with an aggregate 15s budget raced
against every probe plus a 3-consecutive-failure fuse — either stop strips
unresolved and unprobed ids from the mapping and proceeds (text-only at
worst), never failing the request. Restore the original interleaved text
ordering for non-vision runs with a mapping present. Guard the library's
failed-entry-write release with an ambiguous-commit re-read: release only
when the entry provably does not point at the new id, keep it when the
write actually landed, and prefer a recoverable leak over a dangling entry
when the re-read fails. Drop the unused package export.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(generation): bump @openmaic/generation to 0.3.1 (#1153 part 2)

The package's inputs changed (partitionImagesForVision, resolvedVisionImages
option, non-vision ordering fix); the merge-time gate requires a version bump.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-24 18:08:40 +08:00
wyuc e3fd494658 feat(tts): add Qwen TTS voice cloning (#1160) 
* feat(tts): add Qwen TTS voice cloning via the registration adapter

* feat(settings): add Qwen TTS clone-voice manager

* fix(tts): harden Qwen voice cloning integration

* fix(tts): enforce voice-bound Qwen synthesis

* fix(tts): close Qwen voice clone tail issues

* fix(tts): close final voice clone review issues

* fix(agents): harden voice token parsing

* fix(settings): hide the voice-clone model from manual selection

* fix(agents): pin the narrator voice to the user's selected voice

* fix(agents): keep narrator fallback alive when the pinned voice is unusable

* fix(i18n): correct the narration fallback notice copy

* fix(tts): bound the narrator voice retry to a single fallback hop

* fix(tts): tolerate transient vendor errors in the voice existence precheck
 2026-08-24 14:33:43 +08:00
LING DUANandwyuc a0dccbe78d feat(pi): add Native Child additive whiteboard tools (#1156) 
* feat(pi): add Native Child additive whiteboard tools

* chore(renderer): bump package version to 0.1.3

* fix(pi): reject invisible whiteboard mutations

* fix(pi): close native whiteboard review blockers

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-24 12:12:07 +08:00
wyucandClaude Fable 5 5a3e81c4d9 feat(document): ingest uploaded sources into the asset pool and extract by id (#1153 part 0) (#1154) 
* feat(document): declare a version on every extractor provider (#1153 part 0)

* feat(api): accept asset-id input in the extract-document route (#1153 part 0)

* feat(upload): ingest selected course materials into the asset pool (#1153 part 0)

* fix(document): harden asset-id extraction inputs and close upload ingest leaks (#1153 part 0)

* fix(document): scope extraction fallback and harden the asset-id path (#1153 part 0)

- G1: byte fallback only on pre-extraction failures; PARSE_FAILED
  (422/500) is surfaced without re-running the paid extractor
- G2: freeze the course-material set for the duration of generate-prep
  (guarded add/remove + disabled toolbar affordances, session built
  from the click-time snapshot, belt-and-braces drain)
- G3: bound the pre-generation ingest await (~15s); timed-out ingests
  go byte-path and their late-resolving ids are released
- G4: reject non-object JSON bodies (null/array/string/number) as 400
- G5: JSON-path provider pre-validation with generic messages, generic
  JSON-path PARSE_FAILED/422 responses, control-char-free log values
- G6: size cap enforced via identify() before resolve() materializes
  bytes; too_large maps to the existing 413

* fix(document): mirror provider-hint semantics on the asset-id form and freeze prep-time controls (#1153 part 0)

* fix(document): keep the asset-id form's responses free of caller input (#1153 part 0)

Address review findings: answer the extractor-selection failure with a
generic static message on the asset-id form (the registry's interpolated
message carries the caller's MIME type; multipart keeps it verbatim, both
sides now pinned by tests), dedupe same-batch material additions by content
fingerprint inside the pure updater, document the network-error retry
exception in shouldRetryWithByteUpload's docstring, and fix a test name
asserting 500 but named 400.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-24 11:45:31 +08:00
LING DUAN 2cbe4ae293 feat(pi): add Native Child whiteboard tools (#1152) 
* feat(pi): add native whiteboard tools

* style(pi): format native whiteboard tests

* fix(pi): clarify whiteboard CAS handoff

* fix(whiteboard): reconcile runtime projection on open
 2026-08-21 18:07:49 +08:00
LING DUANandwyuc 59adbef03b Simplify Pi agent tool ownership and completion (#1148) 
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-20 00:26:07 +08:00
LING DUANandwyuc 0d308ed78d feat(pi): add Native Child Web Search (#1133) 
* feat(pi): add native child web search

* fix(pi): align native web search with provider system

* fix(pi): harden native web search configuration and cancellation

* fix(pi): guide Native Child web search selection

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-19 10:35:44 +08:00
LING DUANandwyuc cdf949a017 feat(pi): add native child runtime with scoped Spotlight (#1111) 
* feat(pi): add native child runtime with scoped Spotlight

* fix(pi): address native child review feedback

* fix(pi): suppress prefaced structured child output

* fix(pi): restore native child text streaming

---------

Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-16 00:05:23 +08:00
wyucand杨慎 0a10af11cf feat(storage): opt-in indirect asset byte egress (#1007) (#1100) 
* feat(storage): add opt-in indirect asset byte egress

A deployment can now opt an asset byte GET into a 302 to a short-lived
signed URL when the byte layer can sign. It is off by default, and
byte-for-byte unchanged when off. The byte layer gains an optional
signReadUrl capability; S3AssetByteStore implements it through the new
optional @aws-sdk/s3-request-presigner peer, resolved lazily exactly
like the client SDK, with a signer seam mirroring the commands seam so
tests can bind doubles. The signed URL pins the contract's response
headers -- the media type after the renderable allowlist, the fixed
disposition, and the no-store cache posture -- via S3 response-header
overrides. PgAssetStore mints the URL from the same ownership-checked
transactional read a direct resolve performs, so authorization still
runs per read before any URL exists, and a store whose byte layer
cannot sign falls back to direct bytes. The contract documents the
shape and the disclosure tradeoff: a Location into a hash-keyed byte
layer names the content hash, so deployments that need the
no-disclosure property keep direct egress.

Refs #1007

* feat(app): wire ASSET_BYTE_EGRESS into the persistence route

ASSET_BYTE_EGRESS=redirect opts asset byte GETs into the storage
package's indirect egress; unset, direct, or an unrecognized value
keeps the default byte-for-byte behavior, with a warning for the
unrecognized case. The lazy byte-store wrapper now forwards
signReadUrl and answers undefined when the resolved layer has no
signer, so the PostgreSQL byte column degrades to direct bytes and
the S3 layer signs without the wrapper knowing which it holds. The
collector path is untouched: it builds its byte store through
createAssetByteStore as before and only ever calls delete.

Refs #1007

* fix(storage): let the packaged client survive redirect egress

Cross-review round 1 on the indirect byte egress change found two
problems.

A byte response that followed a redirect carries the pinned content
type but no X-Asset-Revision, and HttpAssetStore required both from
the same response, so every cold resolve under redirect egress failed
MALFORMED_RESPONSE. The client now detects the shape once -- a
redirected byte response with no revision -- latches it, and probes
HEAD before each cold GET from then on, taking the label from the
probe. The probe predates the download on purpose: a replacement in
between labels newer bytes with an older revision, which the next
revalidation detects and corrects, while the reverse order could pin
stale bytes under a fresh revision with no signal to repair it. The
probe doubles as the miss check, so a redirect-mode miss costs no
download. The direct path is byte-for-byte unchanged; the one double
download per client lifetime on first contact is documented in the
contract, which gains the client-side paragraph this behavior
implements.

signedUrlTtlSeconds also accepted lifetimes beyond the seven-day
SigV4 presigning maximum, minting redirects the object store would
reject. The option is now capped at construction.

Refs #1007

* fix(storage): cap signed URL lifetime below the reclamation grace

A signed URL whose lifetime exceeds the collector grace period can
outlive its object: the last reference goes, the grace elapses, the
collector deletes the object, and the still-valid URL errors at the
object store. The previous ceiling -- the seven-day SigV4 maximum --
made that window days wide against the one-hour default grace. The
cap is now fifteen minutes, and the contract text states both the
ceiling and the rule for deployments that shorten their grace.

Refs #1007

* fix(storage): decline signing when the optional presigner is absent

An unresolvable @aws-sdk/s3-request-presigner made signReadUrl throw,
which the registry surfaced as a failed asset read -- a 500 for a
deployment whose only fault is a missing optional peer. The contract's
answer for a byte layer that cannot sign is the capability fallback:
return undefined and let the caller serve the bytes directly. Signing
errors from a resolved signer still fail loud; only the missing
capability declines.

Refs #1007

* fix(storage): answer redirect egress as a descriptor to asking clients

Following a 302 is not header-neutral: the platform fetch forwards the
original request's headers to the object store's origin, stripping only
Authorization, so a deployment whose credential travels in a custom
header would hand it to the object store -- and the preflight the
forwarded custom headers provoke commonly fails there besides. This
replaces the latch-and-probe client shape with an explicit one: the
client sends X-Asset-Egress: descriptor on every byte GET, a
redirect-egress server answers 200 with a JSON { url, revision } body
instead of a Location, and the client fetches the signed URL with no
deployment headers at all. The revision comes from the descriptor, so
the probe-first ordering and its one wasted download are gone too. The
302 remains the answer for consumers that did not ask; the descriptor
response is marked by its own header so a JSON-media asset can never
parse as one.

Refs #1007

* fix(storage): negotiate the descriptor through Accept, not a custom header

A custom X-Asset-Egress request header is not CORS-safelisted, so every
byte GET to a cross-origin server became a preflighted request -- a
regression for direct-egress deployments that never opted into
anything, against servers with no reason to allow the header. The
negotiation now rides Accept with a vendor media type, which is
safelisted and costs no preflight, and the descriptor answer is
identified by that media type as its Content-Type rather than by a
marker header.

Refs #1007

* fix(storage): sign outside the registry transaction and reserve the descriptor type

Two review findings on the indirect egress path.

resolveIndirect awaited the signer inside the coordinated read, but a
signer on refreshable credentials can wait on the network, and a
database connection plus the blob row's FOR SHARE lock would be held
across that delay. The read now closes first -- the lock still puts
the hash, label and revision in one snapshot -- and signing runs
afterwards, which cannot observe anything the read did not.

The descriptor media type is also reserved from renderableTypes: the
client identifies a descriptor answer by that exact Content-Type, and
an allowlisted asset served with it would be misread as a descriptor
and fail MALFORMED_RESPONSE. The handler now rejects the configuration
at construction, and the contract states the reservation.

Refs #1007

* fix(storage): ship the signing deps and negotiate without ambiguity

Three review findings on the indirect egress path.

The app now declares @aws-sdk/client-s3 and
@aws-sdk/s3-request-presigner as runtime dependencies, and the
standalone build force-includes them for the persistence route: both
are reached through deliberately untraced dynamic imports, so the
shipped image could not resolve them, and a deployment opting into S3
or redirect egress would have found the capability silently absent.

The descriptor negotiation matches exactly on both sides: the client
compares the Content-Type essence, so a longer media type that merely
begins with the reserved value still serves as bytes, and the server
parses Accept media ranges, so a descriptor range sent with q=0
selects the redirect instead of the descriptor it explicitly rejects.

Refs #1007

* fix(app): refuse redirect egress when the collection grace is shorter

A signed URL must never outlive its object: with the handler's
60-second default lifetime, a deployment that sets
ASSET_COLLECTION_GRACE_MS below ten minutes lets the collector delete
an object while a URL minted against it is still valid. The route now
refuses the combination at handler initialization, naming both
variables -- a deterministic misconfiguration, loud at startup rather
than silent at read time.

Refs #1007

* fix(app): ship the signing SDKs without loading them

Two findings on the standalone deployment of redirect egress.

The outputFileTracingIncludes globs did not follow pnpm's scoped
symlinks, so the standalone image carried neither AWS package and both
S3 mode and redirect egress could not resolve their SDK at runtime.
The packages are now server-external and referenced from literal --
but never called -- dynamic import thunks in the persistence byte-store
wiring, which gets them traced into the image while module resolution
still happens only on first S3 use. Verified against a local
standalone build: node_modules/@aws-sdk now contains both packages,
and the route tests that pin the never-resolve-unless-configured
discipline still pass.

Media type comparisons are also case-insensitive now, as HTTP
requires, on both the Accept parse and the descriptor recognition.

Refs #1007

* fix(app): treat an empty collection grace as unset

Number('') is 0, so a present-but-empty ASSET_COLLECTION_GRACE_MS
failed the redirect-egress coordination check and took persistence
down with it, while the collector's own parsing reads the same value
as the one-hour default. The check now trims and treats empty as
unset, matching durationEnv.

Refs #1007

* fix(storage): accept bytes too, omit signing on the known-PG wrapper, state the full tradeoff

Three review findings.

The client's descriptor request now advertises both representations --
the vendor descriptor type preferred, */* accepted -- so a strict
content-negotiating layer can never answer 406 to a client that does
in fact consume plain bytes.

The lazy byte-store wrapper no longer advertises signReadUrl when no
bucket is configured: the layer is known to be PostgreSQL at
construction, and carrying the method made resolveIndirect take its
ownership query and blob-row lock just to decline, then repeat them
in resolve, on every cold GET. With a bucket configured the lazy
validation is preserved and the wrapper still declines when the
resolved layer cannot sign.

And the contract's disclosure tradeoff now states the second edge:
object-store GETs carry ETag and Last-Modified, response overrides
cannot strip them, and because deduplicated PUTs rewrite the
hash-keyed object, Last-Modified moves when another principal
re-uploads the same bytes -- shared-object write timing, beyond byte
equality. Deployments for whom that signal is sensitive must keep
direct egress.

Refs #1007

* fix(app): degrade on bad grace, share the ttl invariant, cover the real boundary

The latest review round, four findings.

A misconfigured ASSET_COLLECTION_GRACE_MS now degrades redirect
egress to direct with a warning instead of failing the shared
handler's initialization -- the asset backend is optional, and its
misconfiguration must never take document and runtime traffic down.

The package exports assertSignedUrlTtlWithinGrace so a deployment
wiring the handler and the collector separately validates the
invariant once at its own boundary: the handler caps the lifetime,
but only the deployment knows both numbers.

The contract states that nosniff does not survive the redirect --
response-header overrides cannot set it -- and why the client-minted
blob makes that safe for conforming clients.

And the route gains a real-boundary test: the Fetch<->Node adapter,
the real createStorageHttpHandler, and the egress wiring, exercised
with an actual byte GET through the composed stack instead of mocks.

Refs #1007

* fix(storage): make the unsafe egress configuration unrepresentable

Three findings on the indirect egress path, all resolved by moving a
rule into a place where it cannot be bypassed rather than by adding a
check a consumer has to remember.

The grace/TTL invariant now lives in the option shape. byteEgress takes
'direct' or { mode: 'redirect', collectionGraceMs, signedUrlTtlSeconds? },
so enabling indirect egress without declaring the reclamation window it
lives inside no longer type-checks, and the handler validates both
numbers at construction. The flat signedUrlTtlSeconds option, its
cross-field "requires byteEgress redirect" check, and the standalone
900-second cap as the only guard all go away; the ceiling stays, but
alongside the ratio rather than in place of it. This is what the earlier
exported assertSignedUrlTtlWithinGrace could not do -- it protected only
consumers who called it -- and it costs nothing in compatibility because
the option is new in this branch. The app resolves the grace through one
shared parser with the collector, so the number the handler checks is the
number the collector runs with.

A signed URL whose object is gone is now a miss. The client maps a 404
from the byte fetch to ASSET_NOT_FOUND, matching the direct path: the
entry was owned and readable when the URL was minted, so a reclaimed
object is the same physical state the direct read reports as a miss, and
what a read means must not depend on the deployment's egress setting.
Signing still does not probe for the object -- that would restore the
round trip this feature removes and make the mint's price vary with prior
presence -- so the contract instead requires the byte layer to answer 404
for an absent object, and states that S3 needs s3:ListBucket to do so.

Route coverage now spans the whole egress matrix through the real
composed handler rather than only the direct case: 302 for a consumer
that did not ask, descriptor for one that did, direct bytes when the
byte layer declines to sign, and direct bytes when a short grace degraded
the configured mode.

Refs #1007

* feat(storage): forward byte egress options through reference server

* fix(storage): fail closed on redirects and unconfirmed 404s in indirect byte egress

A signed-fetch 404 now maps to a miss only when the object store's XML
body declares NoSuchKey; every other 404 from the signed fetch fails loud,
so a wrong bucket, access point, or endpoint can no longer make a live
asset read as absent. The descriptor byte GET is sent with redirect:
'manual' and any 3xx answer is treated as an error, so a server that
ignores the descriptor negotiation can never forward the deployment's
custom credential headers to a redirect target.

The ASSET_BYTE_EGRESS switch is now documented in .env.example and the
server-persistence deployment section, together with the object-store CORS
and s3:ListBucket prerequisites it requires, and the asset descriptor
media type is exported from the package root and the asset/http subpath.

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-08-14 23:57:21 +08:00
wyucClaude Opus 5杨慎
 		969574bee4 feat(storage): wire the asset backend into the app (#1007) (#1089) 
* fix(persistence): stop corrupting binary response bodies in the route adapter

The embedded persistence route bridges a Node `RequestListener` to the
Fetch API by hand. Its response object is cast with `as unknown as
ServerResponse`, so the compiler checks none of that surface, and two
parts of it were wrong in ways only a byte-carrying handler would hit.

`end` decoded any `Uint8Array` chunk with `Buffer.from(chunk).toString()`,
which is UTF-8. `ServerResponse.end` accepts a `Uint8Array`, and those
bytes are not necessarily valid UTF-8, so every unpaired byte became
U+FFFD -- silent corruption with no error anywhere. The adapter now
buffers chunks as bytes and builds the response from them.

`write` was absent entirely, making any chunked handler a runtime
TypeError rather than a compile error. It is part of the surface this
object claims to implement, so it is now implemented.

Neither reaches a caller today: the document and runtime handlers only
ever end with JSON strings, which took the string branch. Both are traps
laid for the first handler that carries bytes.

Both cases are pinned by tests that fail before this change -- the
binary one on a byte comparison, the chunked one with a 500 from the
TypeError. The existing adapter round-trip test, whose comment already
called this the most bug-prone code in the route, covered only the
string path.

* feat(storage): wire the asset backend into the app (#1007)

The server asset registry shipped with #1007 but was reachable from
nothing: the persistence route never passed an `assetStore`, and no file
under `app/`, `lib/` or `components/` imported the asset server or client
modules. This connects it, additively -- no DSL change, no version bump,
and no behavioural change when server persistence is off.

The route now builds a `PgAssetStore` on the same pool and transaction as
the document store, with `PgAssetByteStore` by default and
`S3AssetByteStore` when `ASSET_S3_BUCKET` opts a deployment into it. The
AWS SDK stays an optional peer, reached only through a dynamic import on
that branch. The offline collector is deliberately not scheduled here,
and the route says so where a reader would look for it, because leaving
it unscheduled grows storage without bound and scheduling it is the
deployment's decision.

On the browser side the asset pool gains a single-shot configuration seam
modelled on the document store's, so that with server persistence enabled
the pool is an `HttpAssetStore` carrying the same auth headers as the
other stores.

Three browser-specific mechanisms needed decisions rather than
translation:

`clearAssetPool` deletes an IndexedDB database today. In server mode
there is nothing local to delete, and a "clear cache" action must not
remove server-side assets -- that would destroy user data from a button
promising to free space. It now revokes local object URLs and closes the
client, and deletes nothing remote.

The cross-tab replacement broadcast is unchanged. Its job is to make
another tab drop a warm object URL when the bytes behind an id change,
and that need is identical in server mode.

The exclusivity proof that decides whether an in-place `replace` is safe
reasons over this browser only: local documents, unflushed state, and a
cross-tab presence probe. In server mode those cover a strict subset of
the holders, since another device can reference the same id and no probe
can see it -- and asking the server who else references an id would be an
existence oracle over other principals. The proof therefore returns not
exclusive unconditionally in server mode, and regeneration takes the
existing fork path. That is the fail-closed direction and a graceful
degradation rather than a break.

The development authenticator had to grow the asset principal's required
`key`; without it every asset request would have been correctly denied.
It derives from the same learner partition, and a principal without one
still gets no asset access. The route now states plainly that this key
comes from a client-supplied header, so the cross-principal isolation the
asset contract describes is not in force under this authenticator -- that
warning previously lived only in the auth module's own header, where
nobody mounting the route would see it.

* fix(storage): close three defects in the asset app wiring (#1007)

**The S3 branch could not load, and isolated nothing.** The route
dynamically imported the storage package's S3 byte store, but that module
statically imported the AWS SDK, so merely resolving it pulled the SDK
into resolution and bundling. The route then separately performed an
ignored native import of the SDK from the app's own scope, which fails
with ERR_MODULE_NOT_FOUND because the SDK lives under the storage
workspace package -- so a deployment that set the bucket got a broken
store rather than an S3 one. There is now one lazy loader in one
resolution scope, the storage module no longer imports the SDK at module
load, and the bucket is validated at initialization rather than treated
as valid because it is non-empty. A test mocks the SDK to throw on
resolution and asserts that importing the byte-store module never
resolves it.

**A cross-tab replacement could leave a mounted lease stale.** The
broadcast handler re-resolves, and `HttpAssetStore.resolve` coalesces
concurrent calls for one id onto a single in-flight request -- correct on
its own, and required by the contract. Together they meant a resolve
triggered by the broadcast could merge into a GET that started before the
peer's replacement, return the old revision, and leave the consumer on
the old URL with nothing to dislodge it. The client gains an invalidation
that retires its cached snapshot and advances the per-id generation
without revoking any URL already issued; `release` would have been wrong
here, since the contract keeps an issued URL valid until its holder
releases it. Removing the invalidation call fails the new race test.

**Clearing a server-backed pool left the closed client installed.** In
server mode `clearAssetPool` closed the client and returned before
clearing the module singleton, so every later `getAssetPool()` handed
back a closed store until a page reload happened to intervene. Only the
IndexedDB deletion is skipped in server mode now; the singleton is always
cleared so the next call rebuilds through the configured factory.

Also adds the tests whose absence let these through: a route round-trip
carrying invalid UTF-8, the SDK-resolution assertion above, the broadcast
race, and reopening the pool after a clear. The earlier tests were not
wrong so much as aimed past the defect -- the route test used JSON text,
and the S3 test injected a loader rather than exercising the imports.

* chore(storage): bump package version to 0.3.0

* fix(persistence): make the route adapter faithful where it carries bytes

The embedded persistence route bridges a Node `RequestListener` to the
Fetch API by hand, and its response object is cast
`as unknown as ServerResponse`, so the compiler checks none of that
surface. Five divergences are closed here; the first two are the ones
that corrupt data, the last three were raised in review.

**`end` decoded bytes as text.** It ran `Buffer.from(chunk).toString()`
on a `Uint8Array` chunk, which is UTF-8, so every byte outside that
encoding became U+FFFD -- silent corruption with no error anywhere. The
adapter now buffers chunks as bytes.

**`write` was absent**, making any chunked handler a runtime `TypeError`
rather than a compile error.

**`write` callbacks ran inline.** Node invokes them after the chunk is
handed off. A handler that writes its next chunk from each callback
therefore recursed synchronously and could overflow the stack, and a
throwing callback threw in the wrong execution phase. They are now
deferred.

**The encoding argument was ignored.** `write(s, 'latin1')` emitted
`0xc3 0xa9` where Node emits `0xe9`, corrupting the body and potentially
invalidating a `Content-Length` the handler had set. Both `write` and
`end` now parse their full overload set and pass the encoding through.
Node's own behaviour for an invalid encoding was established by probing
`ServerResponse` on Node 22 rather than assumed: it throws
`ERR_UNKNOWN_ENCODING` synchronously, so the adapter validates with
`Buffer.isEncoding` and lets `Buffer.from` raise the native error.

**Bodyless statuses were not suppressed.** Node discards writes after
`writeHead(204)` or `writeHead(304)`; this passed the buffered bytes to
the Fetch `Response` constructor, which throws for those statuses, so the
request became a 500 instead of the intended 204 or 304. Buffered data is
now discarded for 204, 205 and 304, and for `HEAD`.

None of the five reaches a caller today -- the document and runtime
handlers only ever end with JSON strings. They are traps laid for the
first handler that carries bytes, which is why they surfaced while
wiring the asset server backend, whose byte routes hit the first two
immediately.

Each is pinned by a test that fails when its fix is reverted.

* fix(storage): keep { client, bucket } working for the S3 byte store

`S3AssetByteStoreOptions.commands` was required and the store's methods
called it directly, so the published `./asset/s3-bytes` export stopped
accepting the `{ client, bucket }` construction it had always taken.
Anyone not going through `loadS3AssetByteStore` either failed to compile
or hit an undefined command factory at runtime.

Make `commands` optional. Omitted, the store resolves the AWS SDK's own
command constructors through a dynamic import on its first `write` /
`read` / `delete` and caches them, so `{ client, bucket }` works again.
Importing the module and constructing a store still never reach the
optional peer dependency: a deployment that does not select S3 never
resolves it. When the SDK cannot be resolved the call rejects naming
`@aws-sdk/client-s3` rather than crashing on an undefined property, and
the resolution is left uncached so installing the dependency fixes an
already-constructed store.

The lazily bound store is run against the byte-store contract, and the
new isolation tests pin both that construction does not resolve the SDK
and that an unresolvable SDK is reported by name.

With the break gone this branch's change to the package is additive, so
the version becomes a patch above main (0.2.4 -> 0.2.5) rather than the
minor it was carrying.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(persistence): run the asset collector in the shipped deployment

`PgAssetStore.remove`, and a `replace` that changes content, only stamp
`asset_blobs.unreferenced_at`. `AssetCollector.collect` is the only path
that deletes anything, and the persistence route deliberately did not
schedule it, on the grounds that scheduling is the deployment's decision.

The deployment this repository ships is docker-compose.yml: the app and
PostgreSQL, and nothing else that could make that decision. "The
deployment decides" therefore meant "it never runs", and ordinary asset
churn retained PostgreSQL bytes or S3 objects forever.

Schedule it from instrumentation.ts, which Next runs once per server
process — a route module can be instantiated more than once and has no
shutdown hook, so a schedule started from one is really started per
instantiation. Next 15 and later pick the file up with no config, so
next.config.ts is unchanged.

Collection is on by default with a 15-minute interval and the package's
one-hour grace period, so the Compose stack is correct with no operator
action. ASSET_COLLECTION_INTERVAL_MS, ASSET_COLLECTION_GRACE_MS, and
ASSET_COLLECTION_ENABLED change or disable it, each documented where it
is read. Without DATABASE_URL nothing is scheduled at all, and a failed
pass is logged and retried on the next tick rather than ending the
schedule or the process.

Several instances may collect concurrently: each candidate blob row is
re-checked and locked FOR UPDATE inside its own transaction, so
collectors serialize on the row rather than racing. The comment says so,
so nobody adds a distributed lock that is not needed.

The byte-store selection moves to lib/persistence/asset-byte-store so the
collector reclaims through the same layer the request path wrote through.
A collector on the PostgreSQL byte store while the route wrote to S3
would drop the blob row and orphan the object permanently.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(storage): bound each collector pass and report whether it filled

An unbounded collect() is sized by however long a deployment ran before
collection was scheduled: one statement selecting every eligible blob,
then a transaction and a byte-layer delete each, in a loop nothing
interrupts. The first pass over the Compose deployment that grew a
backlog before collection was scheduled is exactly that pass.

A pass now takes at most batchSize blobs (default one thousand) and
returns. Ordinary churn between two scheduled passes is far below the
cap, so a healthy deployment behaves as it did when a pass was
unbounded; a capped pass costs the remainder one scheduling interval,
which is what the interval is for.

collect() still answers with the count, and the count alone cannot tell
an empty backlog from a full batch -- a re-referenced or concurrently
taken candidate is skipped, so even a full batch can collect less than
batchSize. collectPass() returns the count together with capped, which
is exactly "this batch was full, run again"; a caller draining a
backlog loops while it is true.

Candidates are taken oldest-unreferenced first, with the content hash
as tiebreaker within one timestamp. Ordering by the hash alone would
starve: digests are uniformly distributed, so a high-sorting blob
waits behind every lower digest stamped after it, and under steady
arrivals those keep coming. The tests pin the ordering against a
planner that happens to answer the right rows without an ORDER BY.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(persistence): keep asset backend faults off the shared handler

Two review findings on the app wiring.

A concrete asset pool instance is single-lifecycle, but
resolveConfiguredAssetPoolStore handed the same object out again after
clearAssetPool() had closed it, reinstalling a dead store as the live
pool. The second handout now refuses loudly and points at the factory
form, which rebuilds on every resolution.

createPersistenceHandler eagerly awaited the asset byte store, so an
invalid ASSET_S3_BUCKET or an unresolvable AWS SDK rejected the shared
persistence handler and took document and runtime traffic down with an
optional backend. Byte-store construction is now deferred to the first
asset byte operation, and a failed construction is not cached, so the
next asset request retries -- the route's own no-poisoned-singleton
rule.

Refs #1007

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-11 13:01:16 +08:00
wyucand杨慎 1498ca52b2 fix(persistence): stop corrupting binary response bodies in the route adapter (#1088) 
* fix(persistence): stop corrupting binary response bodies in the route adapter

The embedded persistence route bridges a Node `RequestListener` to the
Fetch API by hand. Its response object is cast with `as unknown as
ServerResponse`, so the compiler checks none of that surface, and two
parts of it were wrong in ways only a byte-carrying handler would hit.

`end` decoded any `Uint8Array` chunk with `Buffer.from(chunk).toString()`,
which is UTF-8. `ServerResponse.end` accepts a `Uint8Array`, and those
bytes are not necessarily valid UTF-8, so every unpaired byte became
U+FFFD -- silent corruption with no error anywhere. The adapter now
buffers chunks as bytes and builds the response from them.

`write` was absent entirely, making any chunked handler a runtime
TypeError rather than a compile error. It is part of the surface this
object claims to implement, so it is now implemented.

Neither reaches a caller today: the document and runtime handlers only
ever end with JSON strings, which took the string branch. Both are traps
laid for the first handler that carries bytes.

Both cases are pinned by tests that fail before this change -- the
binary one on a byte comparison, the chunked one with a 500 from the
TypeError. The existing adapter round-trip test, whose comment already
called this the most bug-prone code in the route, covered only the
string path.

* fix(persistence): make the route adapter faithful where it carries bytes

The embedded persistence route bridges a Node `RequestListener` to the
Fetch API by hand, and its response object is cast
`as unknown as ServerResponse`, so the compiler checks none of that
surface. Five divergences are closed here; the first two are the ones
that corrupt data, the last three were raised in review.

**`end` decoded bytes as text.** It ran `Buffer.from(chunk).toString()`
on a `Uint8Array` chunk, which is UTF-8, so every byte outside that
encoding became U+FFFD -- silent corruption with no error anywhere. The
adapter now buffers chunks as bytes.

**`write` was absent**, making any chunked handler a runtime `TypeError`
rather than a compile error.

**`write` callbacks ran inline.** Node invokes them after the chunk is
handed off. A handler that writes its next chunk from each callback
therefore recursed synchronously and could overflow the stack, and a
throwing callback threw in the wrong execution phase. They are now
deferred.

**The encoding argument was ignored.** `write(s, 'latin1')` emitted
`0xc3 0xa9` where Node emits `0xe9`, corrupting the body and potentially
invalidating a `Content-Length` the handler had set. Both `write` and
`end` now parse their full overload set and pass the encoding through.
Node's own behaviour for an invalid encoding was established by probing
`ServerResponse` on Node 22 rather than assumed: it throws
`ERR_UNKNOWN_ENCODING` synchronously, so the adapter validates with
`Buffer.isEncoding` and lets `Buffer.from` raise the native error.

**Bodyless statuses were not suppressed.** Node discards writes after
`writeHead(204)` or `writeHead(304)`; this passed the buffered bytes to
the Fetch `Response` constructor, which throws for those statuses, so the
request became a 500 instead of the intended 204 or 304. Buffered data is
now discarded for 204, 205 and 304, and for `HEAD`.

None of the five reaches a caller today -- the document and runtime
handlers only ever end with JSON strings. They are traps laid for the
first handler that carries bytes, which is why they surfaced while
wiring the asset server backend, whose byte routes hit the first two
immediately.

Each is pinned by a test that fails when its fix is reverted.

---------

Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>
 2026-08-10 16:14:42 +08:00
wyuc 3b4567076f refactor(generation): consume the package everywhere and delete lib/generation (#1090) 
Part D of #1057 — the switch. All consumers import @openmaic/generation; lib/generation deleted (net -11,900 lines). PBL wiring preserved (single-call -> classified fallback -> injected loop), streaming outline route deduplicated onto buildOutlinePrompt, serverExternalPackages wired, and the change verified by a real-model four-kind end-to-end run against the production build with a rendered-classroom screenshot check. Closes #1057.
 2026-08-10 03:58:33 -04:00
LING DUANCopilot Autofix powered by AIwyuc
 		fc66cc0603 feat(runtime): add learner whiteboard RuntimeStore foundation (#1075) 
* feat(runtime): add whiteboard RuntimeStore foundation

* Potential fix for pull request finding 'Comparison between inconvertible types'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* fix(runtime): reject invalid whiteboard appends before commit

* fix(runtime): preserve special JSON keys during canonical clone

* Potential fix for pull request finding 'Invalid prototype value'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* feat(runtime): add minimal learner whiteboard mutation

---------

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
 2026-08-09 12:32:28 -04:00