mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-04 02:08:36 +08:00
9fe650d994
* fix(quiz): resolve AI answer keys to option values before grading
AI-generated quiz answer keys are unreliable: the model may write the
correct answer as option CONTENT ("(6, 2)"), as a LETTER ("A"), or as a
formatting variant (full-width chars, extra spaces) of either. Grading
compares exact option values, so a learner picking the correct option
was still graded incorrect whenever the key was written as content or
in a different format.
- generation: normalizeQuizAnswer now resolves every answer variant to
the matching option value (NFKC + whitespace-stripped + letter-prefix
canonical matching against option values and labels); unknown answers
pass through untouched
- grading: resolveAnswerKeyToValue retro-fixes already-generated courses
whose stored keys hold content variants, mapping them to option values
when exactly one option matches canonically; ambiguous keys are left
alone instead of being silently re-pointed
- tests: letter/content/full-width/trailing-space/multi-answer/unknown
cases + end-to-end grading with a content-variant key
* chore(generation): bump version to 0.3.2 for answer-key fix
The repo requires every publishable @openmaic package change to ship with
a version bump (CI check-package-version-bumps).
* style(generation): prettier-format package.json after version bump
* fix(quiz): use resolved answer representation in review renderers; prettier
- QuestionCard + quiz-view review modes highlight the option canonically
matching the stored answer key (answerIncludesOption), so persisted
content-variant keys highlight correctly after the grading fix
- prettier formatting on touched files
* test(quiz): guard possibly-undefined answer in multi-choice test
* fix(quiz): narrow answer-key matching to exact unique alignment; project review UI from the same resolver
Per review: matching policy narrows to the DSL option-identity contract.
- Generation: exact value or unique exact label alignment; unknown and
ambiguous keys stay unresolved (fail closed). Prompt example unified to
the DSL object-options + value-answer shape.
- Grading: persisted keys resolve via the same exact alignment; the
resolver returns the option's actual value when exactly one option
matches canonically-or-exactly; ambiguous keys stay untouched.
- Review UI: QuestionCard and both quiz-view review modes highlight via
the same resolver projection (answerIncludesOption(q, optionValue)).
- Removed the broader equivalence rules (NFKC, whitespace stripping, case
folding, wrapped/prefixed-letter probe) — semantic synonyms such as
正交/垂直 remain uncovered, hence Refs #1326.
* fix(quiz): resolve answer keys for persisted keys only; align quiz template with the DSL
- grading: compatibility resolution (label -> value) now applies to the stored
answer key only. A learner submission is compared as submitted, so an option
label can no longer be accepted as a different option's value.
- quiz-content/user.md: replace the string-options/`correctAnswer` example with
the DSL shape the system prompt defines ({ label, value } options and
`answer` as an array of option VALUES), so generated keys are values.
- tests: regression that a label submission grades incorrect while the same
question's value submission grades correct.
* test(generation): update scene-prompt golden snapshot for the DSL-shaped quiz template
The golden test pins the rendered user prompt for every scene kind; the
quiz-content template now emits the DSL shape, so the snapshot follows.
* fix(quiz): read label-stored correct answers in the editor's option rows
`quiz-edit-ops.toRows` decided correctness with raw value equality, so an
existing label-keyed quiz showed A/B as correct (the review UI and the grader
both resolve labels) while every row read as incorrect internally — and the
next option edit or reorder rebuilt `answer` from those rows and silently
dropped the key.
- `toRows` now uses the same resolved projection (`answerIncludesOption`).
- `normalizeQuizAnswer`'s docstring now describes the exact-only behavior it
has had since the review: formatting variants are not normalized and
ambiguous entries pass through untouched.
- Regressions: an unrelated option-label edit and an option reorder both keep
a label-stored answer, and a multiple-choice toggle preserves it. All three
fail against the previous raw comparison.
---------
Co-authored-by: CrazyPandaP <CrazyPandaP@users.noreply.github.com>
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn>