mirror of
https://github.com/rohitg00/ai-engineering-from-scratch.git
synced 2026-10-02 01:54:39 +08:00
Rohit Ghumare
dda194f840
Every "Test Your Understanding" quiz placed the correct answer in option B.
Across the 2026 questions in 338 quiz files the correct answer sat at index 1
in 61.5% of cases (uniform would be ~25%), and 107 files had every answer at B,
making the quizzes guessable without reading them.
scripts/debias_quizzes.py rewrites each question's option order with a
deterministic, content-seeded permutation and updates the correct index to
follow the moved answer. It is idempotent: options are canonicalised to a sorted
base before permuting, so re-running produces byte-identical output. Questions
whose options reference each other by position ("all of the above", "both A and
B") are left untouched. The correct-answer value, the option set, and every
explanation are preserved exactly; only order and the index change.
Result: A 23.8% / B 26.3% / C 23.5% / D 26.4%.
The script doubles as a CI guard: `--check` exits non-zero if any quiz is not
de-biased, wired into the curriculum workflow so new lessons cannot regress.
Fixes #368
103 lines
3.6 KiB
JSON
103 lines
3.6 KiB
JSON
{
|
|
"lesson": "01-text-processing",
|
|
"title": "Text Processing — Tokenization, Stemming, Lemmatization",
|
|
"questions": [
|
|
{
|
|
"stage": "pre",
|
|
"question": "Why does a language model need preprocessing before training?",
|
|
"options": [
|
|
"Models read strings directly",
|
|
"Preprocessing improves GPU utilization",
|
|
"Models consume discrete integer tokens, not raw text",
|
|
"Preprocessing is required only for languages without whitespace"
|
|
],
|
|
"correct": 2,
|
|
"explanation": "Models operate on integer token IDs; preprocessing bridges continuous language to discrete inputs."
|
|
},
|
|
{
|
|
"stage": "pre",
|
|
"question": "Which preprocessing operation is rule-based suffix stripping?",
|
|
"options": [
|
|
"POS tagging",
|
|
"Stemming",
|
|
"Lemmatization",
|
|
"Tokenization"
|
|
],
|
|
"correct": 1,
|
|
"explanation": "Stemming chops suffixes with rules; it is fast but can be wrong."
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "What does the Porter stemmer step 1a return for the word 'ponies'?",
|
|
"options": [
|
|
"poni",
|
|
"ponies",
|
|
"pony",
|
|
"ponie"
|
|
],
|
|
"correct": 0,
|
|
"explanation": "The 'ies' rule replaces the suffix with 'i', producing 'poni'; step 1b in real Porter cleans it up."
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "Why does lemmatization usually need a POS tag?",
|
|
"options": [
|
|
"Unicode handling",
|
|
"Grammatical context disambiguates forms like 'better' (ADJ) versus 'better' (VERB)",
|
|
"To reduce memory usage",
|
|
"Speed"
|
|
],
|
|
"correct": 1,
|
|
"explanation": "Lemmas depend on grammatical role; without the tag the lookup is ambiguous."
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "When translating NLTK Penn Treebank POS tags to WordNet tags, what does a tag starting with 'V' map to?",
|
|
"options": [
|
|
"'r' (adverb)",
|
|
"'a' (adjective)",
|
|
"'v' (verb)",
|
|
"'n' (noun)"
|
|
],
|
|
"correct": 2,
|
|
"explanation": "Verb-prefixed Penn Treebank tags map to WordNet 'v'."
|
|
},
|
|
{
|
|
"stage": "post",
|
|
"question": "What is 'training/inference mismatch' in NLP preprocessing?",
|
|
"options": [
|
|
"A GPU memory issue",
|
|
"Training and serving apply different preprocessing, so the model sees an unfamiliar distribution at inference",
|
|
"Mixing tokenizer libraries within a notebook",
|
|
"Different batch sizes between train and test"
|
|
],
|
|
"correct": 1,
|
|
"explanation": "Different preprocessing at training vs inference is the most common production NLP failure."
|
|
},
|
|
{
|
|
"stage": "post",
|
|
"question": "Which tool would you reach for in a transformer pipeline instead of classical preprocessing?",
|
|
"options": [
|
|
"NLTK word_tokenize",
|
|
"The model's own tokenizer (e.g. via tokenizers / transformers)",
|
|
"spaCy en_core_web_sm",
|
|
"A regex tokenizer"
|
|
],
|
|
"correct": 1,
|
|
"explanation": "Transformer models ship a paired tokenizer; classical preprocessing is bypassed."
|
|
},
|
|
{
|
|
"stage": "post",
|
|
"question": "Why pin NLTK and spaCy versions in requirements?",
|
|
"options": [
|
|
"Newer versions are slower",
|
|
"Older versions support more languages",
|
|
"Tokenizer and lemmatizer behavior shifts between minor releases, silently changing training distribution",
|
|
"License compatibility"
|
|
],
|
|
"correct": 2,
|
|
"explanation": "Library upgrades can change tokenization output, drifting from the training distribution."
|
|
}
|
|
]
|
|
}
|