mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-21 06:10:53 +08:00
@openmaic/generation@0.3.13
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
9fe650d994 |
fix(quiz): resolve AI answer keys to option values before grading (#1328)
* fix(quiz): resolve AI answer keys to option values before grading
AI-generated quiz answer keys are unreliable: the model may write the
correct answer as option CONTENT ("(6, 2)"), as a LETTER ("A"), or as a
formatting variant (full-width chars, extra spaces) of either. Grading
compares exact option values, so a learner picking the correct option
was still graded incorrect whenever the key was written as content or
in a different format.
- generation: normalizeQuizAnswer now resolves every answer variant to
the matching option value (NFKC + whitespace-stripped + letter-prefix
canonical matching against option values and labels); unknown answers
pass through untouched
- grading: resolveAnswerKeyToValue retro-fixes already-generated courses
whose stored keys hold content variants, mapping them to option values
when exactly one option matches canonically; ambiguous keys are left
alone instead of being silently re-pointed
- tests: letter/content/full-width/trailing-space/multi-answer/unknown
cases + end-to-end grading with a content-variant key
* chore(generation): bump version to 0.3.2 for answer-key fix
The repo requires every publishable @openmaic package change to ship with
a version bump (CI check-package-version-bumps).
* style(generation): prettier-format package.json after version bump
* fix(quiz): use resolved answer representation in review renderers; prettier
- QuestionCard + quiz-view review modes highlight the option canonically
matching the stored answer key (answerIncludesOption), so persisted
content-variant keys highlight correctly after the grading fix
- prettier formatting on touched files
* test(quiz): guard possibly-undefined answer in multi-choice test
* fix(quiz): narrow answer-key matching to exact unique alignment; project review UI from the same resolver
Per review: matching policy narrows to the DSL option-identity contract.
- Generation: exact value or unique exact label alignment; unknown and
ambiguous keys stay unresolved (fail closed). Prompt example unified to
the DSL object-options + value-answer shape.
- Grading: persisted keys resolve via the same exact alignment; the
resolver returns the option's actual value when exactly one option
matches canonically-or-exactly; ambiguous keys stay untouched.
- Review UI: QuestionCard and both quiz-view review modes highlight via
the same resolver projection (answerIncludesOption(q, optionValue)).
- Removed the broader equivalence rules (NFKC, whitespace stripping, case
folding, wrapped/prefixed-letter probe) — semantic synonyms such as
正交/垂直 remain uncovered, hence Refs #1326.
* fix(quiz): resolve answer keys for persisted keys only; align quiz template with the DSL
- grading: compatibility resolution (label -> value) now applies to the stored
answer key only. A learner submission is compared as submitted, so an option
label can no longer be accepted as a different option's value.
- quiz-content/user.md: replace the string-options/`correctAnswer` example with
the DSL shape the system prompt defines ({ label, value } options and
`answer` as an array of option VALUES), so generated keys are values.
- tests: regression that a label submission grades incorrect while the same
question's value submission grades correct.
* test(generation): update scene-prompt golden snapshot for the DSL-shaped quiz template
The golden test pins the rendered user prompt for every scene kind; the
quiz-content template now emits the DSL shape, so the snapshot follows.
* fix(quiz): read label-stored correct answers in the editor's option rows
`quiz-edit-ops.toRows` decided correctness with raw value equality, so an
existing label-keyed quiz showed A/B as correct (the review UI and the grader
both resolve labels) while every row read as incorrect internally — and the
next option edit or reorder rebuilt `answer` from those rows and silently
dropped the key.
- `toRows` now uses the same resolved projection (`answerIncludesOption`).
- `normalizeQuizAnswer`'s docstring now describes the exact-only behavior it
has had since the review: formatting variants are not normalized and
ambiguous entries pass through untouched.
- Regressions: an unrelated option-label edit and an option reorder both keep
a label-stored answer, and a multiple-choice toggle preserves it. All three
fail against the previous raw comparison.
---------
Co-authored-by: CrazyPandaP <CrazyPandaP@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>
|
||
|
|
3eea9dc542 |
feat(runtime): complete the #869 learner-data cutover — quiz + playback onto RuntimeStore (#955)
* feat(runtime): persist quiz attempts in RuntimeStore * fix(runtime): coalesce quiz draft snapshots * fix(runtime): recover concurrent quiz attempts * fix(runtime): handle quiz completion races * fix(runtime): commit quiz review atomically * fix(runtime): dedupe concurrent quiz writes * fix(runtime): drop stale quiz drafts * fix: harden quiz runtime review recovery * fix: serialize quiz attempt identity * feat: read quiz state from runtime store * fix: persist quiz retries before resetting * fix: preserve authoritative quiz outcomes * fix: preserve legacy quiz retries during cutover * fix: reconcile legacy quiz snapshots safely * fix: drain rollover quiz write queues * fix: drain completed quiz retry queues * fix: reuse concurrent quiz retries * fix(quiz): preserve drafts across abrupt reloads * fix(quiz): recover empty retry sessions * fix(chat): abort stalled runtime state reads * fix(quiz): expose queued phases to readers * fix(quiz): retain concurrent writer tails * fix(quiz): canonicalize retry branches * fix(quiz): keep retry rollovers monotonic * fix(quiz): validate skipped retry siblings * fix(quiz): close read cutover races * test(classroom): cover legacy quiz summaries * fix(quiz): reset async consumers on scene changes * fix(quiz): close scene transition windows * test(pbl): cover launch freshness guards * fix(quiz): close cutover concurrency gaps * fix(quiz): harden retry and context freshness * fix(quiz): reject malformed legacy answers * fix(quiz): validate legacy answer values * test(quiz): cover legacy multi-answer migration * fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover Main's three call sites built static storeState blocks that runAgentLoopFn never consumed (it always rebuilds fresh state via getStoreState); the quiz read cutover replaced that callback with the async two-phase RuntimeStore read, leaving the template field with zero consumers. Drop it. * feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError semantics existed only in the browser backend; server-backed deployments would silently accept conflicting appends and leave completed sessions active. Forward the options over the wire, detect conflicts atomically under the PG transaction, map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error client-side so quiz retry logic works across every backend. Co-authored-by: Codex <codex@openai.com> * fix(chat): Pi single requests build storeState via the async runtime quiz read Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the request template straight to /api/chat/pi, which rejects bodies without storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read with the scene-transition guard) and call it on the Pi path too. * ci: whitelist the runtime-data-cutover integration trunk for PR checks * feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956) * feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869) The fourth and last runtime family. Consumed-discussion facts become append-only 'playback' records folded into a set at read (at-least-once appends, no conflict machinery); the resume cursor is device-scoped last-write-wins KV per the amended #779/#869 split. sessionStorage keeps same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie playbackState machinery is retired, with a one-time lazy migration of any legacy row (cursor half + facts half) before deletion, and stage deletion now clears both the KV cursor and any unmigrated legacy row. Co-authored-by: Codex <codex@openai.com> * test(runtime): include playbackState in the stage-delete db mock --------- Co-authored-by: Codex <codex@openai.com> * fix(playback): persist discussion facts on every consumption path (#957) * fix(playback): persist discussion facts on every consumption path (final-review P0+P1s) - The engine now publishes a progress snapshot the moment a discussion is consumed (join / skip / unselected-agent auto-skip). onProgress otherwise fires before the discussion action executes and a discussion is the scene's last action, so the fact never reached persistence. - Reads fold records across ALL playback sessions in the learner partition (mergeLearner deliberately preserves same-kind sessions from both keys). - Legacy migration appends only not-yet-durable facts, so an interrupted migration resumes instead of dropping the tail. - recordConsumedDiscussion reports durability; the component drops failed ids from its observed set so a later progress tick retries (at-least-once). * test(e2e): live verification of the playback persistence chain Seeds a deterministic stage straight into the Dexie DB, starts the lecture via the canvas overlay, and asserts the full chain: discussion auto-skip appends a discussionConsumed record to maic-runtime, the device cursor lands in KV, and both survive a fresh browsing context (empty sessionStorage). * refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959) Product ruling on #869's fourth family: playback learner state is front-end ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined discussions' content already lives in chat runtime records, and no replay export / analytics consumer exists — so durable facts bought nothing over in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half with real UX value), the engine's consumption-time progress snapshot (cursor freshness), and the legacy Dexie retirement (cursor half migrates, row deletes, consumed ids are dropped). * fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966) * fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair) Review findings on #955: didActiveSceneRemainUnchanged compared the active scene by object identity, so a store update reallocating the scene during the async quiz read dropped the learner's graded answers from that turn's request — the scene id is the real boundary. The records route now classifies a malformed sessionTransition as a validation failure instead of letting the store's throw surface as a 500. * fix(playback): superseded-engine cursor guard + migration write-window recheck Second-vendor review of the #959 shrink (requested after the cross-review noted it had single-vendor coverage) found: an engine orphaned by a scene switch during async lecture resume could pass the idle-only recheck, be resurrected, and publish its old scene's progress over the new scene's debounced cursor — the resume continuation now requires identity with the installed engine, and onProgress drops snapshots from superseded engines. The legacy cursor migration also rechecks KV immediately before its write so a concurrent tab's newer cursor cannot be overwritten and orphaned by the legacy-row delete. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> * fix(review): approval follow-up P3 nits (#967) * release: v0.3.1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard - The records route's sessionTransition guard now requires an ISO updatedAt (isIsoTimestamp), matching the sibling PATCH /status route - Dead vi.mock factories for the deleted playback-storage module dropped - A corrupt legacy playback timestamp falls back to 'now' instead of wedging migration into a permanent re-throw that disabled resume --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Dustin Persek
|
9ffdd72f77 |
fix(quiz): render formulas in quiz text (#833)
* fix(quiz): render formulas in quiz text * fix(quiz): tighten formula rendering heuristics |
||
|
wyuc
|
1d1ce80e04 |
feat(maic-agent): editor-agent line (v0) — Pro-mode "Edit with AI" (#777)
Promote the editor-agent line to main: read-then-act editor agent (read_scene_content, regenerate_scene, regenerate_scene_actions, edit_interactive_html), a routable maic-agent model stage, the agent sidebar UI, interactive-scene runtime-error capture, and the timeline/script editor. Cross-reviewed and smoke-verified; see #777. |
||
|
|
f86c0482dc |
feat: end-of-course completion page + persistent quiz state (#484)
* refactor(quiz): extract grading helpers into lib/quiz/grading * feat(classroom): add summarizeScenes aggregation helper * feat(i18n): add classroomComplete translations * feat(classroom): add ClassroomCompletePage component * feat(stage): render ClassroomCompletePage when course finishes * feat(stage): show 'Course complete' slot in sidebar * chore: apply prettier formatting Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(classroom): richer completion visuals — confetti, gold trophy, stats cards - Replace lucide Trophy with custom gradient SVG (3x larger, base + stem + star) - Add one-shot confetti burst (55 particles, 7-color palette, no new dep) - Add looping sparkles + pulsing halo around trophy - Redesign stats from thin icon row into metric cards with count-up numbers - Upgrade quiz block to gradient card with animated gold progress ring - Add gradient ribbon badge above title, bg-clip gradient title - Sidebar thumbnail: amber gradient + sparkles + larger filled trophy - Respect prefers-reduced-motion (skips confetti/sparkles/count-up) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(stage): show course-complete title in header; tone down sidebar thumbnail - Header top-left was showing loading fallback on pending slot; now renders t('stage.courseComplete') when isCourseComplete && isPendingScene - Sidebar thumbnail was too saturated next to neutral slide thumbnails; back to soft amber-50 base with a subtle radial glow, smaller muted sparkles, and colored amber-500 trophy (no more filled white on gradient) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(classroom): persist submitted quiz answers for completion page Previously handleSubmit cleared the quizDraft cache, leaving nothing for the completion page to read — score always showed 0/N. Now submit also writes to a separate quizAnswers:<sceneId> key that is not cleared by the normal in-progress draft lifecycle. - quiz-view: on submit, mirror answers into quizAnswers:<sceneId> - quiz-view: on retry, remove that persistent key too - classroom-complete: read quizAnswers first; fall back to quizDraft so an unsubmitted attempt still shows partial progress rather than 0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(quiz): persist submitted quiz state across unmounts Before: remounting a quiz scene after submit restarted it from 'not_started' because the draft cache was cleared at submit time. Reopening meant redoing everything. Now on mount, read quizAnswers:<sceneId> + quizResults:<sceneId> and restore: - Both present → jump to 'reviewing' with graded answers and feedback - Only answers (mid-grading unmount) → 'answering' with answers pre-filled - Neither → fall through to existing draft-restore path Results are persisted when the grading useEffect transitions to 'reviewing', and both keys are cleared together by handleRetry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(home): prevent horizontal shift when recent classrooms expand Expanding recent classrooms could grow the page past the viewport height, making a vertical scrollbar appear. Because the scrollbar takes layout width, the centered header and trigger row jumped horizontally. Reserve the scrollbar gutter on <html> so the gutter is always present and layout stays stable across expand/collapse. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(home): force always-visible scrollbar to eliminate expand/collapse shift scrollbar-gutter: stable alone wasn't enough (browser/config dependent). Adding overflow-y: scroll on html forces the vertical scrollbar to always occupy layout width, so expanding recent classrooms past the viewport no longer shifts the centered trigger row. Trade-off: home page now always shows a scrollbar track, even when content fits viewport. Acceptable cost for stable layout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(quiz): extract persistence module + sweep localStorage on delete Addresses code review on #484: 1. New lib/quiz/persistence.ts centralizes the three per-scene localStorage keys (draft / answers / results) behind readSubmittedState, readAnswersForSummary, writeSubmittedAnswers, writeSubmittedResults, clearSubmitted, clearAllForScene. quiz-view and classroom-complete now both consume this module instead of hand-rolling localStorage calls, removing the schema-coupling between the two components. 2. deleteStageData now collects scene ids before deletion and sweeps the matching quiz persistence keys, fixing the storage leak when a classroom is deleted. 3. ClassroomCompletePage gets role=status / aria-live=polite / aria-label for screen readers to announce the completion moment. 4. Tests: 11 new unit tests covering the persistence module (read/write/clear roundtrips, draft fallback, corrupt-JSON tolerance, scene-isolation). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(classroom): address review on PR #484 (a11y, motion, coupling) Addresses cosarah's review: 1. prefers-reduced-motion now fully honored — wrap ClassroomCompletePage in MotionConfig with reducedMotion='always' when the OS preference is on. This neutralizes the previously-uncovered animations (trophy float repeat:Infinity, halo pulse repeat:Infinity, page/cards/trophy entrance fade+scale+spring). 2. Move role='status' off the noisy outer wrapper; replace with a single sr-only span carrying the title text so screen readers get one clean announcement instead of every count-up tick. Outer is now a plain <section aria-label=...>. 3. Wrap ClassroomCompletePageConnected in a motion.div inside AnimatePresence so leaving the pending slot fades instead of snapping (matches the loading-overlay branch). 4. quiz-view's draft cache key now uses the draftKey() helper exported from lib/quiz/persistence so the prefix has one source of truth (no silent desync if the schema evolves). 5. isCourseComplete now requires scenes.length === outlines.length, so a partial generation with failed outlines doesn't falsely trigger. 6. aria-hidden on decorative motion wrappers (Confetti, Sparkles, trophy entrance) for cleaner SR experience. 7. Comment on summarizeScenes useMemo explaining the mount-fresh assumption. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: prettier-format review-fix changes CI's format check (pnpm check) failed because the previous commit's changes weren't run through prettier locally before push. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |