What it means
The context window is the maximum amount of text, measured in tokens, that a language model can consider at one time, including the instructions, any documents provided and the conversation so far. Anything outside the context window is not "seen" by the model when it generates a response.
Real-world example
A legal department wants an assistant to compare a new contract against a 300-page framework agreement. If the documents exceed the model's context window, the system must either split them and process parts separately or retrieve only the relevant sections, and in both cases important clauses could be missed.
Why it matters for AI adoption
Larger context windows allow more material per request, but they increase cost and do not guarantee the model will pay equal attention to everything in a long input. System designers need to decide what information goes into the context and how, which is a key reason techniques like retrieval matter.
Questions to ask
- How much text does a typical task require, and does it fit within the model's context window?
- If not everything fits, how does the system decide what to include?