"NLP," "neural NLP," "Transformer," "foundation model," and "LLM" get used almost interchangeably in casual conversation. That looseness costs you the moment you need to reason about what a specific system can and can't do. This section draws the map those five terms actually sit on -- each one nested inside the last, not a synonym for it. Then it names, plainly, what this course itself adds once the map reaches its final stop.
Five words, one nested picture. Think of it as zooming in, not five different neighborhoods.
NLP is the whole field. Any time a computer does something with human language, that's NLP -- the field goes back to the 1950s. The earliest NLP systems had no learning in them at all. A programmer wrote rules like "if the sentence contains this pattern, do that." That's like a recipe: the steps are written out in advance, not discovered by the cook.
Neural NLP zooms in one level. It's NLP done by a neural network
that learns its own patterns from examples, instead of a programmer
writing rules by hand. This course assumes you already know "neural
network" from deep-learning. It just means the layered,
weight-and-bias system that chapter builds from scratch -- here, trained
specifically on text.
Attention and Transformers zoom in again. A Transformer is a
particular kind of neural network. Attention is the specific trick that
lets it read a whole sentence at once, instead of one word at a time.
Sections 5-6 of this chapter walk through that trick by hand. An older
kind of network -- a recurrent network, deep-learning chapters 18-19
-- had to read one word at a time.
A foundation model is a Transformer trained once, on a massive, broad pile of text, built to be a general-purpose starting point. Think of it as one very well-read intern. That one round of training makes the intern useful across dozens of different jobs, instead of training a fresh intern for each job separately. An LLM (large language model) is what this course calls that intern, once they're big enough to follow instructions and handle topics nobody specifically taught them. "Large" here really means billions of individually-learned numbers -- not a figure of speech.
Here's the part that matters for this course. Everything above -- NLP, neural NLP, attention, Transformers, foundation models, LLMs -- is about building the model. This course spends almost no time building one. Instead, it takes an LLM that already exists and is already trained, and turns it into a real piece of software. That means giving it facts it was never trained on, letting it call a weather API or run a calculation, tuning its behavior for one company's support tickets, and checking whether it's actually any good before shipping it. That's the "wiring" this course's own title names. It starts the moment this chapter ends.
Natural language processing (NLP) is the broad field of getting a computer to do anything useful with human language. That includes parsing a sentence's grammar, translating text, answering a question, or flagging spam. NLP existed for decades before neural networks entered the picture. Early systems were rule-based: hand-written grammars and dictionaries that encoded a linguist's own knowledge directly into code. They learned nothing from data at all.
Neural NLP solves NLP by training a neural network on data, instead
of hand-writing rules. Word embeddings (deep-learning chapter 17) and
recurrent networks (deep-learning chapters 18-19) were neural NLP's
first successful tools. Each one learns a task's patterns from a
training corpus, rather than from an engineer's grammar rules.
Attention (deep-learning chapter 20) and the Transformer
architecture it enables (deep-learning chapter 21) are a later, more
specific advance inside neural NLP. A Transformer reads an entire
sequence at once, letting every position weigh every other position
directly. That replaces the recurrent network's one-token-at-a-time
bottleneck. Sections 5-7 of this chapter recall exactly how.
A foundation model is a large model pretrained once, on broad data. It's general enough to become the starting point for many different uses, rather than a model trained from scratch for one task. Pretraining a foundation model is expensive, and happens once. Adapting it -- by prompting or fine-tuning -- is comparatively cheap. Many different users adapt the same underlying model, many times over.
A large language model (LLM) is the specific kind of foundation model this course is about. It's a Transformer with billions of parameters, pretrained on a huge amount of text to predict the next token (Section 2 makes this precise). An LLM is general enough to follow instructions, answer questions on topics nobody explicitly labeled during training, and write working code. Those abilities emerge from scale and broad pretraining -- not from being narrowly trained for any one task.
Everything through "LLM" is architecture and training. deep-learning
chapters 17-21 already cover it, and this chapter's remaining sections
recall it at working depth. This course's own job starts one step
further. It takes a pretrained LLM and wires it into a system that
does something useful: giving it outside knowledge it wasn't trained on
(retrieval), letting it take real actions (tools and agents), adapting
its behavior for a specific job (fine-tuning), and measuring and
shipping the result responsibly (evaluation, security, production).
None of that is a property of the Transformer architecture itself. All
of it is engineering built on top of one.
Which stage does each system belong to?
Click a system, then click the stage of the map it best fits.
Loading interactive…
Worked example
Basic: place a system on the map
-
1
A team publishes a paper. It describes a network that reads product reviews one word at a time. As it reads, it updates a running summary vector, to predict whether the review is positive. No attention mechanism is involved. Where does this system sit on the map?
-
2
It's a neural network trained on text, so it's past the rule-based NLP stage. But it processes one word at a time, with a running summary -- it doesn't let every word attend to every other word directly.
-
3
\boxed{\text{Neural NLP (specifically, a recurrent network, before attention)}} -- this belongs to
deep-learningchapters 18-19's territory. It's one stage before attention enters the picture.
Worked example
Typical: foundation model or not?
-
1
Two systems. (a) A Transformer pretrained once on a broad, web-scale corpus, later adapted by different teams for translation, summarization, and chat. (b) A Transformer trained from scratch on nothing but one company's support tickets, to do exactly one job: classify ticket urgency. Which one is a foundation model?
-
2
The defining property of a foundation model isn't its architecture -- both are Transformers. It's whether one broad pretraining run becomes the shared starting point for many different uses.
-
3
\boxed{\text{(a) is a foundation model; (b) is not}} -- (b) may be a perfectly good Transformer, but it was trained for one task from the start. It was never meant as a general-purpose starting point for others.
Worked example
Recognition: what makes it an LLM specifically, not just a foundation model
-
1
Every LLM is a foundation model. But not every foundation model is "large" enough to be called an LLM -- a foundation model trained for image classification, say, isn't a language model at all. What extra condition does an LLM add on top of "foundation model"?
-
2
"Foundation model" only requires broad pretraining, meant as a reusable starting point. It says nothing about scale, or what kind of data the model was trained on.
-
3
\boxed{\text{An LLM is specifically a foundation model that: (1) works on language/text, and (2) is large enough (billions of parameters, huge training corpus) that broad capabilities emerge}} -- a small model trained the same way, on the same kind of data, is still language-model shaped. It just doesn't earn the "large" in LLM.
Who uses this
Where each stage of the map shows up in production today
Rule-based NLP still runs inside spell-checkers and basic input validation -- cheap and reliable. Neural NLP without attention (word embeddings, RNNs) still powers small, on-device typing helpers. A full Transformer would be too slow or too big to run there. Attention and Transformers are the architecture behind almost every modern search-ranking system and translation service. Foundation models are what Hugging Face's model hub mostly hosts. One pretrained checkpoint gets shared and reused in thousands of different ways. LLMs specifically sit behind a chat assistant, a coding assistant, or a document-summary API. Building the system around one of those is this course's entire subject, starting Chapter 1.
Check your understanding (counts toward your progress)
A course lists 7 stages on its field map, from rule-based NLP through this course's own applications. If "attention" and "Transformers" are counted as one combined stage (as in this section's own figure) rather than two separate ones, how many DISTINCT stages does the map actually have?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
The figure lists exactly 7 boxes: NLP, neural NLP, attention, Transformers, foundation models, LLMs, and apps -- seven stages, with attention and Transformers already drawn as two separate boxes in that count.
Correct answer
Also accepted:
Why
Count the boxes in this section's own field-map figure directly.
According to this section, where does this course's own subject matter actually begin?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Building the architecture is deep-learning's job -- this course assumes a pretrained Transformer already exists.
Pretraining a foundation model is expensive and specialized; this course starts after that step is done.
Rule-based NLP is the oldest, pre-neural stage of the map -- not where this course begins.
Why
This course's subject starts after architecture and pretraining are done: retrieval, tools, agents, fine-tuning, evaluation, and production -- wiring a pretrained LLM into something useful.
Which pair correctly matches an example to its map stage?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
A fixed dictionary lookup involves no learning and no Transformer -- it's the pre-neural, rule-based stage.
word2vec has no attention mechanism -- it's an earlier, pre-attention neural NLP method.
BERT is a Transformer-based model, not a hand-written rule system.
Why
A fixed-dictionary lookup with no learning involved is the defining example of the oldest, rule-based stage of the map.
A model reads an entire input sentence at once and lets every word attend to every other word directly, with no recurrence involved. Which stage of the map does this describe?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Rule-based systems have no learned 'attending' behavior at all -- this is describing a specific neural mechanism.
Pre-attention neural NLP (word embeddings, recurrent networks) processes a sequence one token at a time, not all at once.
Only the attention/Transformer stage is defined by every position attending directly to every other position.
Why
Letting every position attend to every other position directly, with no step-by-step recurrence, is specifically what attention and the Transformer architecture add.
The phrase "every word attend to every other word directly" names one specific mechanism from this section's map.
A company fine-tunes a large pretrained Transformer on its own support-ticket data, then deploys it inside a customer-facing chat widget. Which map stage or stages does this involve?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Fine-tuning and deployment are real steps, and this course's own subject matter covers exactly that territory.
A fine-tuned Transformer-based chat system is not a rule-based, scripted system.
Fine-tuning doesn't remove the "LLM" label -- it's still a large, Transformer-based language model, just adapted.
Why
The pretrained starting point is a foundation model / LLM; fine-tuning it and deploying it into a real product is exactly the wiring work this course's own chapters build.
Separate what happened to the MODEL (pretraining) from what happened AFTER (fine-tuning, deployment).
Which of these best describes the relationship between NLP and neural NLP?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
They aren't unrelated -- neural NLP is a specific approach taken within the broader field of NLP.
NLP predates neural networks by decades; rule-based systems came first, not neural ones.
NLP is still the name of the whole field today, including its neural and Transformer-based methods.
Why
NLP is the broad field; neural NLP names the specific approach of solving NLP tasks by training a neural network on data instead of hand-writing rules.
Practice: Placing systems on the NLP-to-LLM map (5 problems · doesn’t affect your progress)
Which stage of the map is defined by having NO learning from data at all?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
Rule-based NLP predates neural networks entirely -- a programmer writes the rules directly, with no training from data.
Rule-based NLP
A translation model reads an entire sentence and lets every word attend to every other word at once. This is past which stage, at minimum?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
Letting every word attend to every other word directly is specifically the attention/Transformer stage's own mechanism.
Attention / Transformers
A team pretrains one large Transformer on broad web text, then different groups adapt it for chat, code completion, and summarization. What best names this pretrained starting point?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
One broad pretraining run, reused as the starting point for many different downstream uses, is the definition of a foundation model.
The key detail is that many DIFFERENT teams adapt the SAME pretrained model for different jobs.
A foundation model
What is this course's own job, according to Section 1, relative to everything deep-learning already teaches?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
This course starts after architecture and pretraining are done -- retrieval, tools, agents, fine-tuning, evaluation, and production are its own subject matter.
Wiring an already-pretrained LLM into a system that does something useful
A small on-device keyboard uses word embeddings and a recurrent network for next-word suggestions, with no attention mechanism and no Transformer anywhere in it. Where does it sit on the map?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
A neural network trained on data, but without attention or a Transformer, sits at the neural-NLP stage -- past rule-based, short of attention.
Check off each defining feature in turn: is it neural? Does it use attention?
Neural NLP, before attention