Library
Cuantum trackFull access

Under the Hood of Large Language Models

8 chapters and 47 canonical sections synced from the Cuantum content database.

Author
Cuantum Tech.
Chapters
8
Reading time
~ 22h
Level
Professional
Language
English
Edition
2025
Your progress0%
Start reading

Chapters & sections

8 chapters - 47 sections

Chapter 1: What Are LLMs? From Transformers to Titans

0/5
1.1 From GPT to LLaMA, Claude, Gemini, Mistral, DeepSeek76m1.2 Decoder-Only vs Encoder-Decoder vs Mixture-of-Experts (MoE)91m1.3 Scaling Laws: Kaplan, Chinchilla, and Data–Model Trade-Offs47mPractical Exercises – Chapter 15mChapter 1 Summary – From Transformers to Titans3m

Chapter 2: Tokenization and Embeddings

0/5
2.1 Byte Pair Encoding (BPE), WordPiece, SentencePiece46m2.2 Training Custom Tokenizers for Domain-Specific Tasks49m2.3 Subword, Character-Level, and Multimodal Embeddings89mPractical Exercises – Chapter 24mChapter 2 Summary3m

Chapter 3: Anatomy of an LLM

0/5
3.1 Multi-Head Attention, Rotary Embeddings, and Normalization Strategies78m3.2 Transformer Depth vs Width, Position Encoding Tricks (ALiBi, RoPE)54m3.3 Advanced Architectures: SwiGLU, GQA, Attention Sparsity39mPractical Exercises – Chapter 38mChapter 3 Summary – Anatomy of an LLM3m

Chapter 4: Training LLMs from Scratch

0/6
4.1 Data Collection, Cleaning, Deduplication, and Filtering136m4.2 Curriculum Learning, Mixture Datasets, and Synthetic Data44m4.3 Infrastructure: Distributed Training, GPUs vs TPUs vs Accelerators85m4.4 Cost Optimization & Sustainability in Large-Scale Training76mChapter 4 Summary – Training LLMs from Scratch3mPractical Exercises – Chapter 46m

Chapter 5: Beyond Text: Multimodal LLMs

0/5
5.1 Text+Image Models (LLaVA, Flamingo, GPT-4o, DeepSeek-VL)100m5.2 Audio & Speech Integration (Whisper, SpeechLM)77m5.3 Video and Cross-Modal Research Directions89mChapter 5 Summary – Beyond Text: Multimodal LLMs3mPractical Exercises – Chapter 57m

Quiz

0/2
Questions3mAnswers2m

Project 1: Build a Toy Transformer from Scratch in PyTorch

0/8
0. Setup4m1. Tiny Dataset & Character Tokenizer5m2. Model Components14m3. The Tiny GPT-Style Model5m4. Training Loop (Causal LM)5m5. Text Generation5m6. Where to go next (your experiments)3mLearning outcomes1m

Project 2: Train a Custom Domain-Specific Tokenizer (e.g., for legal or medical texts)

0/11
0. Setup3m1. Gather a Representative Mini-Corpus3m2. Train a BPE Tokenizer (🤗 tokenizers)6m3. Train a SentencePiece Tokenizer (Unigram or BPE)5m4. Wrap Your Tokenizer for Transformers3m5. Evaluate Tokenizer Quality5m6. Add a User Vocabulary (optional but powerful)4m7. Save, Load, and Version3m8. Plug Into a Small Model (sanity run)4mPitfalls & Tips3mLearning outcomes1m