trending/ai-engineering-from-scratch
1
0
Fork 0
mirror of https://github.com/rohitg00/ai-engineering-from-scratch.git synced 2026-10-02 01:54:39 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,810 Commits 20 Branches 4 Tags
main
Commit Graph
7 Commits
Author SHA1 Message Date
Rohit Ghumare a05d392590 fix: repair broken lesson references and guard homepage storage (#495) 
* fix(site): guard theme storage access on the homepage and about page

Reading or writing localStorage throws a SecurityError when storage is
blocked (strict privacy settings, some embedded webviews). site/app.js
read the saved theme outside any try/catch, before it registered its
DOMContentLoaded handler, so the throw stopped the whole homepage from
initializing. The about page's inline theme script had the same
unguarded read and write.

Wrap all four accesses the way the other pages already do and fall
back to the system theme. Bump the app.js cache key so browsers pick
up the fix, and give app.js its own release constant in the cache-key
test.

Fixes #490

* fix(lessons): repair dataset, model, and tool references that no longer resolve

Lesson snippets pointed at resources that are gone or never existed, so
they fail when a student runs them:

- wikimedia/wikipedia only ships 20231101.* configs; 20220301.en is gone
- MMAU-Pro lives at gamma-lab-umd/MMAU-Pro, with audio_path and answer
  fields; open-ended rows are filtered out of the exact-match score
- meta-llama/Llama-3-70B-Instruct is meta-llama/Meta-Llama-3-70B-Instruct
- Depth Anything V2 for transformers is
  depth-anything/Depth-Anything-V2-Large-hf
- microsoft/deberta-v3-large-mnli and microsoft/BEATs-base are not on
  the Hub; BEATs checkpoints ship with microsoft/unilm
- sayakpaul/sd-lora-ghibli does not exist; use a published SD 1.5
  Ghibli LoRA with its trigger phrase
- Qwen/Qwen3-0.6B-spec and meta-llama/Llama-3.2-1B-Instruct-spec are
  not real draft models
- gemini-3-pro is not a Gemini model code, gemini-1.5-pro is shut down,
  and the google.generativeai SDK reached end of life on 2025-11-30
- vLLM replaced --speculative-model and --num-speculative-tokens with
  --speculative-config, and --dtype has no float8_e4m3fn choice
- convert_hf_to_gguf.py cannot write q4_k_m; llama-quantize does
- AutoGPTQ and AutoAWQ are archived; GPTQModel and LLM Compressor are
  the maintained successors, and vLLM reads their quantization method
  from the checkpoint config

Fixes #493

* fix(lessons): replace dead reference links

A link check over every lesson found 46 references that return 404 or
410 or no longer resolve. Each replacement was fetched and matched
against the cited title or content:

- pages that moved on the same site: vLLM, librosa, Letta, Apollo
  Research, Stability AI, NVIDIA, NeurIPS, the MCP spec, the Claude
  docs, the A2A spec, the Julia docs, W&B, Baseten, and Arena (formerly
  LMSYS Chatbot Arena, whose old domain no longer resolves)
- papers and books pointed at DOIs or publisher pages: Friedman on
  gradient boosting, Golub and Van Loan, Kuttruff, Littman's thesis,
  Milne and Witten, and Zave and Jackson (whose DOI was wrong)
- Wayback snapshots where the source only survives in the archive:
  Poynton's color space tour and a model-routing article
- citations of pages that never existed replaced with the real source:
  the DINOv2 paper, the MMAU-Pro project page, Anthropic's Contextual
  Retrieval post, the February 2026 risk report, and the correct
  Anthropic alignment post
- removed where no source exists: a Spinning Up DQN page, the
  andrewgarst/agentic_harness repo, and an Akira blog post

URLs in code-file reference headers get the same treatment.

* chore(site): rebuild data.js

* Revert "chore(site): rebuild data.js"

The main-only CI job rebuilds site/data.js after merge, so the PR
should not carry it; a stale "Last built" line would conflict.

This reverts commit e78e5def.

* fix(lessons): correct claims flagged in review

Each finding was checked against its primary source before changing
anything:

- vLLM: the v0.18.0 feature matrix marks speculative decoding as
  compatible with chunked prefill (and incompatible with LoRA), so the
  "draft model plus chunked prefill does not compile" gotcha was false.
  It is replaced in both lessons, their skills, the scheduler and
  EAGLE-3 diagrams, and the quiz question that asserted it
- alignment faking: the cited Anthropic post tests interrogation
  training, scratchpad length penalties, and process supervision, not
  a compliance-gap loss or faithful-CoT training. The section, learning
  objective, exercise, reference, skill, diagram, and quiz now match it
- Gemini: Google limits the 2.5 models to existing users and points new
  projects to 3.5 Flash-Lite or 3.8 Flash, so both examples use
  gemini-3.8-flash (GA, caching supported, 1M context), and the
  context comparison names GPT-4o's 128K window instead of Claude
- vLLM FP8: --quantization fp8_per_tensor, which the 0.30.0 release
  accepts and main recommends over the deprecated fp8
- GGUF: the converter has no K-quant output; it does write f32 and
  ternary files, which the old wording ruled out
- MMAU-Pro: the exact-match loop is labeled a sanity check, with the
  official evaluator (embedding match, LLM judge, regex rules) named
- Depth Anything: the pipeline example is labeled V2, with the separate
  depth_anything_3 package described for V3, and the missing numpy
  import added
- audio classification: Step 5 is titled for the AST example it runs,
  with BEATs loading described beneath it

The repo's quiz de-biaser moved one rewritten question's correct answer
to its assigned position.

* fix(lessons): drop the speculative decoding and LoRA incompatibility

The v0.18.0 docs matrix marks speculative decoding with LoRA as
unsupported, but vLLM has supported LoRA with speculative decoding on
the V1 GPU engine since vllm-project/vllm#21068 (merged 2025-11-08), so
calling the pair incompatible steers learners away from valid setups.
Remove the claim from both serving lessons, their skills, and both
diagrams. The rollout skill's hard reject now names an incompatibility
the speculative-decoding docs do list: pipeline parallelism on vLLM
0.15.0 or earlier.
 2026-09-27 15:38:59 +05:30
Rohit Ghumare cb55ea0cc6 feat(site): curriculum-wide interactive figure system (134 widgets, 13 modules) (#279) 
* feat(site): interactive training-foundations figures in 5 lessons

Add five theme-aware interactive widgets to lesson-figures.js, embedded
via the existing ```figure fence:

- gradient-descent (P1.08 optimization): drag learning rate, watch the
  descent path converge or diverge past lr > 1
- softmax-temperature (P3.04 activations): divide logits by T, reshape
  the distribution from argmax to uniform
- bias-variance (P2.10): slide model complexity across the U-shaped
  test-error curve, see the sweet spot move
- l2-regularization (P3.07): raise lambda, watch every weight shrink
- lr-schedule (P3.09): compare warmup, cosine, step, exponential decay

Validated headless: all five mount with no console errors, sliders and
selects drive re-render, both light and dark themes render correctly.

* feat(site): interactive LLM-internals figures in 5 lessons

Batch 2, building on the same widget system:

- sampling-decoder (P10.04 mini-gpt): temperature then top-k then top-p
  filtering over the logits, survivors renormalized
- scaling-laws (P7.13): Chinchilla loss from params and tokens, with the
  20-tokens-per-parameter compute-optimal rule
- quantization (P10.11): bits per weight against model size and the
  precision lost at fp16/int8/int4/int2
- rope-explorer (P7.04): rotary frequencies across position and dimension,
  base controls wavelength and usable context
- lora-params (P11.08): rank against the 2r/d trainable fraction

Validated headless: all five mount with no console errors, sliders and
selects drive re-render, both light and dark render correctly.

* feat(site): interactive evaluation and representation figures in 5 lessons

Batch 3, same widget system:

- precision-recall-threshold (P2.09 model-evaluation): slide the cutoff
  across two class distributions, watch precision/recall/F1 trade
- cross-entropy-loss (P3.05 loss-functions): -log(p_true), the price of
  being confident and wrong
- cosine-similarity (P11.04 embeddings): the angle between two vectors is
  the similarity, magnitude drops out
- tokenizer-tradeoff (P10.01 tokenizers): vocab size against tokens-per-word
  and the embedding table cost
- rag-chunking (P11.06 rag): chunk size, overlap, and top-k against chunk
  count and context tokens per query

Validated headless: all five mount with no console errors, math checks out
(thr 0.8 -> P 1.00/R 0.11, -ln(0.05)=2.996, cos 90 deg = 0, 224 chunks),
sliders drive re-render, both light and dark render correctly.

* feat(site): interactive figure system — 74 new widgets across 11 phases

Expand the lesson-figure system from a handful of widgets into a curriculum-wide
library. Refactor lesson-figures.js to expose a shared LF toolkit (el, svgEl,
slider, select, fmtInt, clamp, lerp, raf, register) and split widgets into eight
per-phase module files that plug in via LF.register.

New module files (3,682 LOC) and the concepts they make draggable:
- figures-math.js (P1, 11): vector projection, matrix transform + determinant,
  eigenvectors, derivative tangent, chain rule, gaussian, bayes update,
  entropy/KL, PCA axes, fourier synthesis, convex vs nonconvex
- figures-ml.js (P2, 10): regression fit/MSE, logistic boundary, SVM margin,
  kNN smoothness, k-means steps, tree depth, feature scaling, naive bayes,
  class imbalance, k-fold CV
- figures-dl.js (P3, 9): perceptron boundary, MLP forward pass, vanishing
  gradients, optimizer trajectories, weight-init variance, dropout, batchnorm,
  learning curves, gradient clipping
- figures-vision-speech.js (P4/P6, 8): convolution kernel, pooling, receptive
  field, conv output size, CNN params, spectrogram window, mel scale, aliasing
- figures-transformers.js (P5/P7, 9): attention heatmap, multihead split, causal
  mask, sqrt(d_k) scaling, word2vec arithmetic, BPE merges, GQA sharing,
  residual stream, flash-attention memory
- figures-genai-rl.js (P8/P9, 9): diffusion denoise, noise schedule, VAE latent,
  GAN minimax, Q-learning gridworld, value iteration, epsilon-greedy, discount
  horizon, policy-gradient ascent
- figures-llms-systems.js (P10/P12/P13, 9): beam search, speculative decoding,
  MoE routing, context window, perplexity, continuous batching, ViT patches,
  multimodal fusion, MCP round trip
- figures-agents-alignment.js (P11/P14/P16/P18, 9): agent loop, ReAct trace,
  tool routing, swarm message scaling, supervisor tree, RLHF reward-KL,
  DPO margin, context budget, guardrail gates

Each widget embedded in its lesson via the figure fence (74 lessons). All
theme-aware through CSS vars, vanilla ES5, no dependencies.

Validated headless: all 90 registered figures (16 prior + 74) mount with zero
console errors in a master harness; rich SVG visualizations (attention heatmap,
gridworld policy, convolution feature map, swarm graphs) render correctly in
both light and dark.

* feat(site): 44 more interactive figures — NLP, LLM internals, infra, autonomy

Wave 2 extends the figure system into the phases that were still bare,
plus deeper coverage of the large NLP and LLM phases. Five new module
files (2,219 LOC), each plugging into the shared LF toolkit:

- figures-math2.js (P1, 9): SVD low-rank reconstruction, tensor broadcasting,
  log-sum-exp stability, Lp unit balls, monte-carlo pi, system conditioning,
  random-walk diffusion, roots of unity, graph degree
- figures-nlp2.js (P5, 8): BoW/TF-IDF, RNN unroll, LSTM gates, seq2seq
  alignment, edit distance, n-gram backoff, BIO tagging, sentiment logits
- figures-llms2.js (P10, 9): RMSNorm vs LayerNorm, SwiGLU, RLHF pipeline,
  DPO loss, paged KV cache, expert capacity, sliding-window attention,
  differential attention, weight tying
- figures-infra.js (P17, 9): data/tensor/pipeline parallelism, ZeRO sharding,
  GPU memory breakdown, throughput-latency, autoscaling, cost-per-token,
  roofline
- figures-frontier.js (P15/P19, 9): task decomposition, reflection loop,
  memory consolidation, world-model rollout, autonomy oversight, pass@k,
  eval-harness matrix, canary rollout, trace spans

Embedded in 44 lessons via the figure fence. Validated headless: all 134
registered figures (16 core + 118 module) mount with zero console errors in
a full harness; pipeline-bubble, SVD energy, and trace-span visualizations
render correctly in light and dark.

* fix(site): address review findings on figure widgets

- sampling-decoder: formula now reads 'cumulative >= p' (nucleus keeps the
  smallest set covering p, matching the implementation)
- supervisor-hierarchy: drop the dead capped-total accumulator; show the exact
  geometric total and note when the diagram caps a level at 64 so the number
  and the drawn nodes stay consistent; handle b=1 (total = depth + 1) instead
  of the closed form that is undefined at b=1
- image-patch-tokens: use ceil(size/patch) so non-divisible sizes count the
  partial patch row; formula shows the ceil and meta notes the padded size
- debugging-neural-networks: normalize the one-off Type 'Practice' to 'Build'

Verified in browser: all three widgets render with the corrected text/math,
no console errors.

Skipped: the 'figure fence is not an approved language tag' findings. lesson.html
keys on codeLang === 'figure' to emit the widget mount point; the fence body is
the figure id. Renaming the fence to the figure id would stop it rendering.
There is no fence-language allowlist for these lesson docs.
 2026-06-10 19:35:56 +01:00
Rohit Ghumare 7d49b5725e fix(phase-09/02): escape pipes inside inline code so markdown tables render 2026-05-27 21:28:06 +01:00
Rohit Ghumare 38d71570e9 Revert "chore(phase-09): scrub in-prose banned reference-repo mentions" 
This reverts commit 236198f6da.
 2026-04-23 10:06:45 +01:00
Rohit Ghumare 236198f6da chore(phase-09): scrub in-prose banned reference-repo mentions 2026-04-23 10:05:36 +01:00
Rohit Ghumare a8d005e57f fix(phase-09/02): add Bertsekas and Tsitsiklis neuro-DP as bridge to deep RL lessons 2026-04-23 00:45:43 +01:00
Rohit Ghumare 4389b47469 feat(phase-09/02): dynamic programming (policy/value iteration) 2026-04-22 23:52:09 +01:00