A language model small enough to read in one sitting

Every explanation of a language model starts in the same place: it predicts the next token. I've read that sentence more times than I can count and nodded at it every time.

Then I tried to explain to myself why a model starts repeating itself when you turn the temperature down, and found I could recite the answer without being able to derive it.

That's usually a sign I've memorised the shape of an explanation rather than the mechanism under it. Which, as it turns out, is what this post ends up being about.

So I built the smallest model that has the same problem. It's in the figure at the bottom of this page, it runs entirely in your browser, and it's about eighty lines of counting.

The whole model is one pass of counting

Walk through a text one word at a time. At every position, record three things:

• which word followed the two words before it — the little → cat
• which word followed the one word before it — little → cat
• which word this is, with no context at all

Those three tables are the entire model. No weights, no gradients, nothing that keeps changing once the pass is done. Training is a single loop over the text, which is why the figure below can retrain on every keystroke you type into it.

Predicting is a lookup. Try the most specific context first, the two-word row. If the model has never seen it, fall back to the one-word row. If that's missing too, fall back to plain word frequency.

Backoff is the part I had never actually pictured

I knew the word. I had not pictured it happening.

The figure labels which table answered and on what context, so you can watch it degrade. Type a phrase the text contains and you get a trigram answer with two or three candidates and a sharp distribution. Change one word and it drops to the bigram row, where the candidate list is suddenly much longer and much flatter. Push it further and it lands on unigram — which is just "here are the common words" — and the output stops being about your prompt at all.

That gradient is what I'd never seen laid out: specific and confident, then general and hedged, then meaningless. A real model doesn't back off in discrete steps like this, and I don't want to overclaim the analogy. But the direction it degrades in is recognisable. When there's nothing specific to draw on, output goes generic before it goes wrong.

There is no correct temperature

Temperature is one line. Take each count, raise it to the power of 1/T, normalise. Low T sharpens the distribution around the top count. High T flattens it until rare words get a real chance.

What the slider makes obvious is that both ends are broken, in opposite directions:

• At 0.05 the model always takes the highest count. Sample repeatedly and it walks into a cycle and stays there. It isn't confused. It's doing exactly what you asked, which is to always take the safest available step.
• At 1.8 a word the text contains once gets picked nearly as often as one it contains ten times. Every individual choice is legal and the paragraph is nonsense.

I'd half-assumed low temperature was the accurate setting and high temperature was the creative one. It's better described as how much of the distribution you're willing to use, and the useful range sits in the middle for reasons that have nothing to do with accuracy.

The term is borrowed from physics on purpose — the same shape of equation controls how often a system visits higher-energy states in the Boltzmann distribution. That's the kind of detail I'd have skimmed past a week ago.

The toy model is the memorisation failure mode

This is the part that made it worth the evening.

Give the counting model a large enough corpus and its trigram table gets very good at that corpus and learns nothing it could carry anywhere else. It can't generalise, because there's nothing in it that could. It's a lookup table wearing a probability distribution.

I wrote about this at a completely different scale a few days ago: what happens when you score an agent loop instead of reviewing it, and the loop starts optimising the eval rather than the problem. A system that scores perfectly on the set it was tuned against and falls over the moment the surface changes — that's the failure I was describing there, and it's what this eighty-line model is permanently and by construction.

Seeing it in a form small enough to read made the larger version much less abstract. The probe gap I use on agent runs is, in one sentence, a test for whether the thing became an n-gram model of my eval set. I didn't have that sentence before I built the toy.

What this isn't

Being explicit about the gap, because this genre has a habit of glossing it:

• No attention. This model can see two words back, ever. The 2017 transformer paper exists because that limit is fatal, and because the sequential alternatives that fixed it were slow to train.
• No learning. Nothing is fitted to anything. Counting is not training, even though I keep calling it that.
• No tokenizer. I split on whitespace and called the pieces words. Real models don't, and that turns out to matter more than I expected.

None of this is new ground either. Andrej Karpathy's makemore walks the same road with more depth, and I'd send anyone there first. I built my own anyway, because reading an implementation and writing one are different activities, and only one of them told me I didn't understand temperature.

Next

Tokenization, same treatment: implement byte-pair encoding, put it in the page, and find out what it costs to write in a language the merge loop never saw much of.

After that I'd like to train something real and small, and point the scoring apparatus from how I run agents at it — the vocabulary turns out to be shared, and I want to know whether the instincts are.

I'd have guessed the thing I was missing would be attention. It turned out to be temperature. If you've built one of these yourself, what did writing it tell you that reading it hadn't?