mirror of
https://github.com/rohitg00/ai-engineering-from-scratch.git
synced 2026-10-02 01:54:39 +08:00
Rohit Ghumare
dda194f840
Every "Test Your Understanding" quiz placed the correct answer in option B.
Across the 2026 questions in 338 quiz files the correct answer sat at index 1
in 61.5% of cases (uniform would be ~25%), and 107 files had every answer at B,
making the quizzes guessable without reading them.
scripts/debias_quizzes.py rewrites each question's option order with a
deterministic, content-seeded permutation and updates the correct index to
follow the moved answer. It is idempotent: options are canonicalised to a sorted
base before permuting, so re-running produces byte-identical output. Questions
whose options reference each other by position ("all of the above", "both A and
B") are left untouched. The correct-answer value, the option set, and every
explanation are preserved exactly; only order and the index change.
Result: A 23.8% / B 26.3% / C 23.5% / D 26.4%.
The script doubles as a CI guard: `--check` exits non-zero if any quiz is not
de-biased, wired into the curriculum workflow so new lessons cannot regress.
Fixes #368
79 lines
2.7 KiB
JSON
79 lines
2.7 KiB
JSON
{
|
|
"lesson": "11-multi-region-kv-locality",
|
|
"title": "Multi-Region LLM Serving and KV Cache Locality",
|
|
"questions": [
|
|
{
|
|
"stage": "pre",
|
|
"question": "Why is round-robin load balancing actively harmful for cached LLM inference?",
|
|
"options": [
|
|
"Round-robin breaks TLS",
|
|
"Round-robin is only valid for stateful databases",
|
|
"Round-robin requires sticky sessions",
|
|
"A request that does not land on the node holding its prefix pays full prefill cost instead of a cache hit"
|
|
],
|
|
"correct": 3,
|
|
"explanation": ""
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "What two inputs does a cache-aware router consume?",
|
|
"options": [
|
|
"Only the user_id and tenant_id",
|
|
"Round-robin counters and TLS keys",
|
|
"KV-cache events from replicas and a prefix hash on the incoming request",
|
|
"Random shuffles and request size"
|
|
],
|
|
"correct": 2,
|
|
"explanation": ""
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "Roughly what is the TTFT gap between a cache hit and a cold prefill on a 2K-token prompt for Llama 3.3 70B FP8?",
|
|
"options": [
|
|
"About 1000x",
|
|
"About 10x (~80 ms vs ~800 ms)",
|
|
"About 1.1x",
|
|
"Identical"
|
|
],
|
|
"correct": 1,
|
|
"explanation": ""
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "Why does cross-region routing not always beat regional routing for cache hits?",
|
|
"options": [
|
|
"Cache-aware routing is impossible across regions",
|
|
"Inter-region routing is forbidden by all hyperscalers",
|
|
"GORGO research found cache hits do not help latency",
|
|
"Saved prefill can be dwarfed by network RTT, e.g. 440 ms round-trip can dwarf an 800-to-80 ms prefill saving"
|
|
],
|
|
"correct": 3,
|
|
"explanation": ""
|
|
},
|
|
{
|
|
"stage": "post",
|
|
"question": "What does the lesson cite as the 32% LLM DR failure driver?",
|
|
"options": [
|
|
"Backups that include weights but miss tokenizer files or quantization configs",
|
|
"Region quota exhaustion",
|
|
"Misconfigured load balancers",
|
|
"Unencrypted backups"
|
|
],
|
|
"correct": 0,
|
|
"explanation": ""
|
|
},
|
|
{
|
|
"stage": "post",
|
|
"question": "What does the lesson say about commercial cross-region inference offerings such as Bedrock CRI?",
|
|
"options": [
|
|
"They optimize availability, not TTFT, and treat inference as opaque — you still need an app-layer cache-aware router",
|
|
"They are forbidden under GDPR",
|
|
"They are KV-cache-aware and replace app-layer routing",
|
|
"They only work in us-east-1"
|
|
],
|
|
"correct": 0,
|
|
"explanation": ""
|
|
}
|
|
]
|
|
}
|