mirror of
https://github.com/rohitg00/ai-engineering-from-scratch.git
synced 2026-10-02 01:54:39 +08:00
BIO tagging scheme with round-trip helpers (spans_to_bio and bio_to_spans with tested equivalence). Rule-based gazetteer baseline, hand-crafted feature extraction (word shape, suffix, neighbors) used by classical CRFs, and sketched BiLSTM-CRF architecture. Walks the evolution: rule-based (zero OOV coverage) -> CRF (hand features, enforces valid BIO) -> BiLSTM-CRF (learned features) -> transformer fine-tune (best accuracy, most compute). Each step names the limitation its successor solved. Ship artifact: ner-picker skill that matches approach to domain, label set, language, latency, and data volume. Rejects transformer fine-tune under 500 labeled examples without a pretrained domain model. ~75 minutes. Prerequisites lesson 05/02.