Chapter 3 — The family tree: AI, ML, deep learning, LLMs

In one sentence

They are nested boxes: LLMs are a kind of deep learning, which is a kind of machine learning, which is one approach to AI.

The map

+---------------------------------------------------------------+
| ARTIFICIAL INTELLIGENCE                                        |
| "make machines do things that seem to need intelligence"       |
|                                                                |
|  +----------------------+  +--------------------------------+  |
|  | SYMBOLIC AI          |  | MACHINE LEARNING               |  |
|  | (hand-written rules, |  | "learn the rules from data"     |  |
|  |  logic, search)       |  |                                |  |
|  |                      |  |  +--------------------------+  |  |
|  | expert systems       |  |  | CLASSICAL ML             |  |  |
|  | chess minimax        |  |  | linear/logistic regr.,   |  |  |
|  | planners             |  |  | decision trees, SVM,     |  |  |
|  | Part 2 of this guide |  |  | k-means, n-grams         |  |  |
|  +----------------------+  |  | Part 3 of this guide     |  |  |
|                            |  +--------------------------+  |  |
|                            |                                |  |
|                            |  +--------------------------+  |  |
|                            |  | DEEP LEARNING            |  |  |
|                            |  | many-layered neural nets |  |  |
|                            |  | that LEARN their own     |  |  |
|                            |  | features   (Part 4)      |  |  |
|                            |  |                          |  |  |
|                            |  |   +------------------+   |  |  |
|                            |  |   | TRANSFORMERS     |   |  |  |
|                            |  |   |   (Part 6)       |   |  |  |
|                            |  |   |  +------------+  |   |  |  |
|                            |  |   |  |   LLMs     |  |   |  |  |
|                            |  |   |  | (Part 8+)  |  |   |  |  |
|                            |  |   |  +------------+  |   |  |  |
|                            |  |   +------------------+   |  |  |
|                            |  +--------------------------+  |  |
|                            +--------------------------------+  |
+---------------------------------------------------------------+

The one-line definitions

Term	Definition
AI	Any technique that makes a machine do something that looks intelligent. Includes rules, search, and learning.
Machine Learning (ML)	Programs that improve at a task by processing data, rather than being explicitly programmed.
Neural network	A model made of layers of simple units, loosely inspired by neurons.
Deep learning	Neural networks with many layers, which learn their own features from raw-ish data.
Transformer	A specific neural-network architecture (2017) built around "attention".
LLM (Large Language Model)	A very large Transformer trained on huge amounts of text to predict the next word, then tuned to follow instructions.
Generative AI	Any model that produces new content (text, images, audio) rather than just classifying. LLMs are generative.

Two more distinctions you'll hear

Discriminative vs generative:

Discriminative models answer "which category?" — "is this spam?"
Generative models can produce new examples — "write me an email".
An LLM is generative, but you get classification free by asking it a question.

Narrow vs general:

Everything that exists today is narrow AI: extremely good at a bounded set of tasks. LLMs are unusually broad narrow AI, but they have no goals, no persistent memory between conversations, and no direct contact with the world unless you build that around them.
AGI (artificial general intelligence) is a hypothetical system matching humans across essentially all cognitive work. It does not exist, and this guide makes no claims about when or whether it will.

The historical arc (the spine of this guide)

1950s-80s   RULES            humans write the knowledge      -> Part 2
1990s-2000s STATISTICS       humans design features,          -> Part 3
                             machine learns weights
2010s       DEEP LEARNING    machine learns features too      -> Part 4-5
2017        TRANSFORMER      an architecture that scales      -> Part 6
2018-2020   PRETRAINING      learn from unlabelled text       -> Part 8
2022-now    INSTRUCTION      make it a helpful assistant      -> Part 8
            TUNING + RLHF

Each step specified less and learned more, paying for it with more data and more computing power.

Common confusions

"AI" and "ML" are not synonyms. A chess engine using minimax is AI but not ML. A spam filter using logistic regression is ML.
Deep learning is not "better" than classical ML. For spreadsheet-style tabular data, gradient-boosted trees usually beat neural networks. Deep learning wins on images, audio, and text.
LLM ≠ chatbot. The chatbot is a product built on top of an LLM plus a system prompt, tools, safety layers, and a UI.

Check yourself

Is every AI system a machine-learning system? Give a counterexample.
What makes deep learning "deep"?
Where do LLMs sit in the family tree?