Training is learning to read. Inference is reading a sentence. The first took years and changed you permanently; the second takes a moment and changes nothing about you at all. Technically, inference is a single forward pass: your words are broken into tokens, those tokens flow once through the model's fixed parameters, and out the other side comes a probability distribution over what should come next. One token is chosen, added to the end, and the whole journey runs again from the beginning. A model writing a paragraph of a hundred words has made that journey a hundred times.
The consequence that surprises people most is that each pass is memoryless. The model does not remember your last message, or even the last sentence it wrote, unless that text is handed back to it as part of the input — which is precisely what a context window is, and why it has a limit, and why a long conversation grows more expensive with every turn rather than less. What feels like a model remembering you is a model re-reading you. It is an honest enough illusion, but it is worth knowing which one you are dealing with.
Inference is cheap once and ruinous in aggregate. A training run is a single bill, however large. Inference is a bill that arrives every time anyone asks anything, forever, and it grows with success rather than shrinking. By most accounts the compute now spent serving models has overtaken the compute spent building them — a reversal that happened quietly, and accelerated once reasoning models began spending far more compute per answer than the generation before them. This is why so much engineering effort goes into making a single response cheaper: distilling large models into small ones, caching whatever can be cached, storing the numbers at lower precision. A fraction of a cent, multiplied by a few billion, is a serious number.
The human translation is the working day. Education is expensive, finite, and behind you; the work is cheap by the hour, does not end, and is where the value actually shows up — for you and for everyone paying you. We have spent a decade fascinated by how these things get built. The bill, and most of the consequences, arrive on the other side.
