Rohit Ghumare 5f1ed24014 Add lesson: Activation Functions (ReLU, Sigmoid, GELU and Why) ...

Phase 3, Lesson 04. Covers linearity collapse proof, sigmoid/tanh
vanishing gradients (0.25^10 math), ReLU breakthrough and dead neuron
problem, GELU/Swish for transformers, softmax for classification.
Build It implements all functions with derivatives, vanishing gradient
experiment, dead neuron detector, and training comparison showing
GELU > ReLU > sigmoid convergence. Includes activation selector prompt.

2026-03-28 00:53:53 +00:00

..

prompt-activation-selector.md

Add lesson: Activation Functions (ReLU, Sigmoid, GELU and Why)

2026-03-28 00:53:53 +00:00