mirror of
https://github.com/rohitg00/ai-engineering-from-scratch.git
synced 2026-10-02 01:54:39 +08:00
Rohit Ghumare
dda194f840
Every "Test Your Understanding" quiz placed the correct answer in option B.
Across the 2026 questions in 338 quiz files the correct answer sat at index 1
in 61.5% of cases (uniform would be ~25%), and 107 files had every answer at B,
making the quizzes guessable without reading them.
scripts/debias_quizzes.py rewrites each question's option order with a
deterministic, content-seeded permutation and updates the correct index to
follow the moved answer. It is idempotent: options are canonicalised to a sorted
base before permuting, so re-running produces byte-identical output. Questions
whose options reference each other by position ("all of the above", "both A and
B") are left untouched. The correct-answer value, the option set, and every
explanation are preserved exactly; only order and the index change.
Result: A 23.8% / B 26.3% / C 23.5% / D 26.4%.
The script doubles as a CI guard: `--check` exits non-zero if any quiz is not
de-biased, wired into the curriculum workflow so new lessons cannot regress.
Fixes #368
91 lines
3.2 KiB
JSON
91 lines
3.2 KiB
JSON
{
|
|
"lesson": "18-agno-and-mastra-runtimes",
|
|
"title": "Production Agent Runtimes",
|
|
"questions": [
|
|
{
|
|
"stage": "pre",
|
|
"question": "Which language pairing does the lesson recommend for each runtime?",
|
|
"options": [
|
|
"Agno for TypeScript, Mastra for Python",
|
|
"Both are Go-first",
|
|
"Agno for Python, Mastra for TypeScript",
|
|
"Both are Rust-first"
|
|
],
|
|
"correct": 2,
|
|
"explanation": "Agno is Python (FastAPI-shaped); Mastra is TypeScript (Vercel AI SDK-shaped)."
|
|
},
|
|
{
|
|
"stage": "pre",
|
|
"question": "What is Agno's recommended production deployment shape?",
|
|
"options": [
|
|
"A long-lived stateful daemon",
|
|
"A serverless cron worker only",
|
|
"A stateless session-scoped FastAPI backend; each request starts a fresh agent and session state lives in a DB",
|
|
"A WebSocket-only server"
|
|
],
|
|
"correct": 2,
|
|
"explanation": "Stateless FastAPI per request; session state externalized to a DB."
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "What are Mastra's three primitives?",
|
|
"options": [
|
|
"Node, Edge, State",
|
|
"Plan, Worker, Solver",
|
|
"Agents, Tools, Workflows",
|
|
"Actor, Message, Inbox"
|
|
],
|
|
"correct": 2,
|
|
"explanation": "Agents (LLM + role), Tools (Zod-typed), and Workflows are Mastra's three primitives."
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "Roughly what agent-instantiation cost does Agno target per its docs?",
|
|
"options": [
|
|
"About 1 second and 1 GiB per agent",
|
|
"About 2 microseconds with about 3.75 KiB per agent",
|
|
"About 200 milliseconds and 100 MiB per agent",
|
|
"About 10 minutes and 4 GiB per agent"
|
|
],
|
|
"correct": 1,
|
|
"explanation": "Agno's docs cite about 2 microseconds and about 3.75 KiB per agent."
|
|
},
|
|
{
|
|
"stage": "check",
|
|
"question": "What does Mastra's Unified Model Router give?",
|
|
"options": [
|
|
"A vector DB layer",
|
|
"A single client surface for thousands of models across many providers",
|
|
"A graph checkpointer",
|
|
"A queue for tool calls"
|
|
],
|
|
"correct": 1,
|
|
"explanation": "Mastra's Unified Model Router cites 3,300+ models across 94 providers."
|
|
},
|
|
{
|
|
"stage": "post",
|
|
"question": "When is perf-for-perf's-sake the wrong reason to pick Agno?",
|
|
"options": [
|
|
"When using Langfuse",
|
|
"When the workload is one slow agent call per request and overhead is not the bottleneck",
|
|
"When using Python 3.13",
|
|
"When deploying to AWS"
|
|
],
|
|
"correct": 1,
|
|
"explanation": "2 microseconds matters at chat fan-in scale, not for a single slow call per request."
|
|
},
|
|
{
|
|
"stage": "post",
|
|
"question": "What licensing surface should you read carefully before forking Mastra?",
|
|
"options": [
|
|
"There is no license file",
|
|
"ee/ directories are source-available rather than Apache 2.0 and restrict commercial use",
|
|
"All of Mastra is GPL",
|
|
"Mastra requires CLA but no license review"
|
|
],
|
|
"correct": 1,
|
|
"explanation": "Mastra is Apache 2.0 except for ee/ which is source-available; check the restrictions before forking."
|
|
}
|
|
]
|
|
}
|