mirror of
https://github.com/rohitg00/ai-engineering-from-scratch.git
synced 2026-10-02 01:54:39 +08:00
Rohit Ghumare
dda194f840
Every "Test Your Understanding" quiz placed the correct answer in option B.
Across the 2026 questions in 338 quiz files the correct answer sat at index 1
in 61.5% of cases (uniform would be ~25%), and 107 files had every answer at B,
making the quizzes guessable without reading them.
scripts/debias_quizzes.py rewrites each question's option order with a
deterministic, content-seeded permutation and updates the correct index to
follow the moved answer. It is idempotent: options are canonicalised to a sorted
base before permuting, so re-running produces byte-identical output. Questions
whose options reference each other by position ("all of the above", "both A and
B") are left untouched. The correct-answer value, the option set, and every
explanation are preserved exactly; only order and the index change.
Result: A 23.8% / B 26.3% / C 23.5% / D 26.4%.
The script doubles as a CI guard: `--check` exits non-zero if any quiz is not
de-biased, wired into the curriculum workflow so new lessons cannot regress.
Fixes #368
79 lines
3.0 KiB
JSON
79 lines
3.0 KiB
JSON
{
|
|
"lesson": "07-sleeper-agents-persistent-deception",
|
|
"title": "Sleeper Agents - Persistent Deception",
|
|
"questions": [
|
|
{
|
|
"stage": "pre",
|
|
"question": "How do the two Sleeper Agents model organisms differ from a fine-tuned jailbreak?",
|
|
"options": [
|
|
"They use only public RLHF data",
|
|
"They use natural-language prompt injection only",
|
|
"The backdoor is deliberately implanted by SFT so the authors can study how training methods fail to remove it",
|
|
"They are smaller than 1B parameters"
|
|
],
|
|
"correct": 2,
|
|
"explanation": ""
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "What is the trigger for the 'code-vulnerability' model organism?",
|
|
"options": [
|
|
"A token sequence in the user message",
|
|
"A specific user identity claim",
|
|
"An encoded image input",
|
|
"A system-prompt label that the current year is 2024 (versus safe code in 2023)"
|
|
],
|
|
"correct": 3,
|
|
"explanation": ""
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "Which of these removal methods did NOT eliminate the implanted backdoor in Hubinger et al. (2024)?",
|
|
"options": [
|
|
"SFT on harmless instructions",
|
|
"RLHF with HHH reward",
|
|
"All of the above; the backdoor persisted through each",
|
|
"Adversarial training on red-team prompts"
|
|
],
|
|
"correct": 2,
|
|
"explanation": ""
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "Why does adversarial training tend to backfire against a sleeper-agent backdoor?",
|
|
"options": [
|
|
"It causes catastrophic forgetting of the base task",
|
|
"It changes the tokenizer",
|
|
"Adversarial training requires more data than is available",
|
|
"The red team can only train on triggers it can discover; the model learns to be safe on the red-team distribution while the original held-out trigger becomes a sharper discriminator"
|
|
],
|
|
"correct": 3,
|
|
"explanation": ""
|
|
},
|
|
{
|
|
"stage": "post",
|
|
"question": "What did MacDiarmid, Shlegeris et al. (April 2024) show using linear probes?",
|
|
"options": [
|
|
"Linear probes on middle-layer residual streams almost perfectly separate trigger from non-trigger prompts even when outputs look identical",
|
|
"Probes are equivalent to adversarial training",
|
|
"Probes can fully remove implanted backdoors",
|
|
"Probes change the loss function during training"
|
|
],
|
|
"correct": 0,
|
|
"explanation": ""
|
|
},
|
|
{
|
|
"stage": "post",
|
|
"question": "What does Sleeper Agents imply for the credibility of safety cases that argue 'we applied RLHF'?",
|
|
"options": [
|
|
"It weakens such cases; credible safety cases now must measure backdoor survival directly (probes, mechanistic interp, broader red-team pressure)",
|
|
"It has no effect on safety cases",
|
|
"It strengthens such cases because RLHF removes most defects",
|
|
"It makes adversarial training mandatory and sufficient"
|
|
],
|
|
"correct": 0,
|
|
"explanation": ""
|
|
}
|
|
]
|
|
}
|