Context window

A context window is the maximum number of tokens a language model can hold at once: the prompt, the conversation history, and its own generated output. Once a conversation runs past that limit, the oldest tokens fall out of scope and the model can no longer see them.

Windows are measured in tokens, and one token runs close to four characters of English text. Llama 2 launched in 2023 with a 4,096 token window, roughly six pages of text. Llama 3.1 raised that to 128,000 tokens across all its sizes, closer to two hundred pages in a single pass.

Every token in the window sits in GPU memory as a key-value cache alongside the model's weights. A longer window therefore raises the VRAM a card needs on top of the weights themselves. A GPU sized for the model alone can still run out of memory once a long conversation fills the window.

Sources

Last verified August 29, 2026.