A mini tour of how language models write — one token at a time.
Large language models don't write whole replies in one go. They pick one token — a word or word-piece — at a time, each time choosing from a ranked list of candidates. Try it below: you are the model.
Ranked by probability. The top option builds KatGPT's most likely reply — pick lower ones to see the model branch into alternatives.
✨ KatGPT has finished writing.