Library
Cuantum trackFull access

Under the Hood of Large Language Models

8 chapters and 47 canonical sections synced from the Cuantum content database.

Author
Cuantum Tech.
Chapters
8
Reading time
~ 9h
Level
Professional
Language
English
Edition
2025
Your progress0%
Start reading

Chapters & sections

8 chapters - 47 sections
Chapter 01

Chapter 1: What Are LLMs? From Transformers to Titans

0/5
11.1 From GPT to LLaMA, Claude, Gemini, Mistral, DeepSeek12m21.2 Decoder-Only vs Encoder-Decoder vs Mixture-of-Experts (MoE)12m31.3 Scaling Laws: Kaplan, Chinchilla, and Data–Model Trade-Offs12m5Chapter 1 Summary – From Transformers to Titans12m4Practical Exercises – Chapter 112m
Chapter 02

Chapter 2: Tokenization and Embeddings

0/5
12.1 Byte Pair Encoding (BPE), WordPiece, SentencePiece12m22.2 Training Custom Tokenizers for Domain-Specific Tasks12m32.3 Subword, Character-Level, and Multimodal Embeddings12m5Chapter 2 Summary12m4Practical Exercises – Chapter 212m
Chapter 03

Chapter 3: Anatomy of an LLM

0/5
13.1 Multi-Head Attention, Rotary Embeddings, and Normalization Strategies12m23.2 Transformer Depth vs Width, Position Encoding Tricks (ALiBi, RoPE)12m33.3 Advanced Architectures: SwiGLU, GQA, Attention Sparsity12m5Chapter 3 Summary – Anatomy of an LLM12m4Practical Exercises – Chapter 312m
Chapter 04

Chapter 4: Training LLMs from Scratch

0/6
14.1 Data Collection, Cleaning, Deduplication, and Filtering12m24.2 Curriculum Learning, Mixture Datasets, and Synthetic Data12m34.3 Infrastructure: Distributed Training, GPUs vs TPUs vs Accelerators12m44.4 Cost Optimization & Sustainability in Large-Scale Training12m5Chapter 4 Summary – Training LLMs from Scratch12m6Practical Exercises – Chapter 412m
Chapter 05

Chapter 5: Beyond Text: Multimodal LLMs

0/5
15.1 Text+Image Models (LLaVA, Flamingo, GPT-4o, DeepSeek-VL)12m25.2 Audio & Speech Integration (Whisper, SpeechLM)12m35.3 Video and Cross-Modal Research Directions12m4Chapter 5 Summary – Beyond Text: Multimodal LLMs12m5Practical Exercises – Chapter 512m
Chapter 06

Quiz

0/2
2Answers12m1Questions12m
Chapter 07

Project 1: Build a Toy Transformer from Scratch in PyTorch

0/8
10. Setup12m21. Tiny Dataset & Character Tokenizer12m32. Model Components12m43. The Tiny GPT-Style Model12m54. Training Loop (Causal LM)12m65. Text Generation12m76. Where to go next (your experiments)12m8Learning outcomes12m
Chapter 08

Project 2: Train a Custom Domain-Specific Tokenizer (e.g., for legal or medical texts)

0/11
10. Setup12m21. Gather a Representative Mini-Corpus12m32. Train a BPE Tokenizer (🤗 tokenizers)12m43. Train a SentencePiece Tokenizer (Unigram or BPE)12m54. Wrap Your Tokenizer for Transformers12m65. Evaluate Tokenizer Quality12m76. Add a User Vocabulary (optional but powerful)12m87. Save, Load, and Version12m98. Plug Into a Small Model (sanity run)12m11Learning outcomes12m10Pitfalls & Tips12m