An AI model may be able to accept a million tokens. That does not mean it can use a million tokens reliably.
I began this series when I realized that Anthropic and OpenAI models were beginning to behave differently over the course of long multi prompt conversations, and developed the axiom “an LLM will become increasingly inaccurate in its responses to prompts, and its incoherence will grow over a session until it reaches the point when it is more likely to produce an inaccurate response than an accurate one.”
This distinction is the starting point for Evans’ Law, the first paper in the PatternPulse AI Reliability Research Series. The research began with an ordinary user problem: long conversations with large language models often become less stable before the platform reaches its stated context limit. Instructions begin to disappear. Earlier facts are altered. Names are confused. Corrections stop holding. Interpretations change. The model remains fluent while the work becomes less dependable. Other factors, including prompt complexity, proper noun density, and sequential thinking requirements also play a role as we explore in subsequent papers.
Vendors describe context windows as capacity. Users experience them as continuity. Those are different measurements.
The original Evans’ Law paper asked whether the point at which coherence begins to fail could be measured across models, rather than treated as an unpredictable quirk. Its answer was preliminary but consequential: the observed threshold changed systematically with model capacity, and the increase was sublinear. Larger models generally carried a conversation farther, but reliable working capacity did not grow in proportion to model size or to advertised context length.
That finding creates a new operational question for every organization using generative AI:
How much context can this system use coherently for this task—not merely how much can it technically accept?
What the original Evans’ Law proposed
The paper proposed two related ideas that are easy to collapse into one.
First, within an extended interaction, the likelihood of error rises as the combined prompt, conversation history and generated output grow. The model has more instructions, entities, assumptions and prior statements to reconcile. A wrong inference can also become part of the context used to generate the next answer, allowing errors to compound.
Second, across models of different capacities, the point of first observed incoherence follows a scaling relationship. The accompanying dataset measured that threshold across nine systems associated with four model families, ranging from 4 billion to 250 billion parameters.
The original regression fitted the relationship:
[
L \approx 1773 \times M^{0.79}
]
Here, (L) is the observed coherence-loss threshold in tokens and (M) is model size in billions of parameters.
The exponent matters. A value below 1 means the measured threshold grows more slowly than model size. Increasing capacity helps, but produces diminishing returns in functional context. A model that is many times larger should not be assumed to maintain a proportionally longer reliable interaction.
The original dataset reported the following observations:
|Dataset label|Model family|Parameters (billions)|First-incoherence threshold|
|————-|———–:|——————–:|————————–:|
|Gem-4 |Gemini |4 |5,200 tokens |
|GL-8 |Claude |8 |9,100 tokens |
|GPT-15 |GPT |15 |14,000 tokens |
|Cl-27 |Claude |27 |23,000 tokens |
|Gem-45 |Gemini |45 |41,000 tokens |
|Cl-75 |Claude |75 |59,000 tokens |
|Grk-100 |Grok |100 |69,000 tokens |
|Grk-119 |Grok |119 |82,000 tokens |
|Gem-250 |Gemini |250 |118,000 tokens |
The tests used a temperature of 0.2, top-p of 1.0 and a uniform stopping point at the first sign of incoherence. The Zenodo record includes the dataset, regression notebook, analysis export and visualization so the result can be inspected and replicated.
The record also states its limit plainly: this was preliminary, exploratory work, and independent validation was encouraged. Evans’ Law is an operational hypothesis supported by an initial cross-model dataset. It is not a claim that every model, prompt and task fails at one universal token count.
A context window is an intake specification
The context window tells us how much material a model can receive in a request or conversation. It does not tell us how well the model will preserve every relevant relationship inside that material.
That difference is familiar in other systems. A database may store billions of records without answering every query equally well. A network may advertise a maximum throughput that differs from sustained performance under load. A vehicle may have a top speed that says nothing about the speed at which it can safely take a particular turn.
AI needs the same distinction between maximum capacity and dependable operating range.
Long-context models can often retrieve a fact from a large input. Sustained reasoning is more demanding. The model must identify which details matter, preserve the authority of prior instructions, distinguish similarly named entities, carry corrections forward and avoid treating its own earlier guesses as established facts.
Research outside the PatternPulse program has documented adjacent parts of this problem. “Lost in the Middle” found that models often use information near the beginning or end of a long context more effectively than information placed in the middle. A large 2025 study of multi-turn conversations reported an average performance decline of 39 per cent across six tasks compared with equivalent single-turn prompts. The systems did not simply lose capability; they became much less reliable after taking an early wrong turn.
These studies do not independently validate the exact Evans’ Law curve. They do support the larger proposition that nominal context length and reliable long-form performance are different things.
What coherence loss looks like
Collapse is rarely a dramatic moment when the model stops producing language. The more dangerous pattern is continued fluency with declining fidelity.
Common signs include:
● an earlier instruction being followed inconsistently;
● the model repeating an approach that the user already rejected;
● a proper noun, number or relationship changing without acknowledgment;
● a correction holding for one response and disappearing in the next;
● summaries becoming progressively less faithful to the underlying material;
● unsupported details entering the conversation and later being treated as facts;
● the model expressing greater confidence as the evidentiary basis weakens.
This is why users can remain unaware that reliability has fallen. The interface still works. The sentences are grammatical. The tone may become even more assured. Nothing in the product necessarily marks the transition from a useful working session to an unstable one.
Evans’ Law turns that transition into an object of measurement.
What the law does not say
Evans’ Law does not mean that every answer before a threshold is correct or that every answer after it is wrong. A threshold is a risk boundary, not a switch.
It does not mean length is the only source of failure. Task complexity, ambiguity, modality, the number of entities, retrieval quality and the rigidity of the required answer all affect reliability. Later versions of the research expand the original two-variable relationship to account for several of these factors.
It also does not mean a vendor’s advertised context window is false. A model may technically process the stated number of tokens. The question is whether it can maintain the level of coherence required for the user’s task across that range.
Most importantly, the formula is not a permanent score attached to a brand. Models change. System prompts change. Routing changes. Tool use and memory layers change. Organizations need to test the model version, configuration and workload they actually deploy.
What enterprises should measure instead
The immediate business use of Evans’ Law is not to predict one perfect failure point. It is to replace an unsuitable specification with an operational one.
An enterprise evaluation should ask:
1. At what interaction length does accuracy begin to decline for our task?
2. Which failure appears first: instruction loss, entity confusion, unsupported claims, tool errors or plan drift?
3. Can the system recognize and report its own degradation?
4. Does a correction restore performance, and for how long?
5. How do retrieval, files, images and tool outputs change the safe operating range?
6. What checkpoint, reset or human-review interval keeps the workflow inside a tested range?
This leads to a different deployment architecture. Long projects should have explicit state records instead of relying on an unstructured chat history. Critical facts and decisions should be held outside the model and reintroduced in a controlled form. Workflows should include validation gates before one generated result becomes the premise for the next. High-consequence tasks need shorter review intervals than low-consequence drafting tasks.
Procurement must change as well. Buyers should ask vendors for reliability curves across context length, failure signatures, model-version change logs and tested recovery behaviour. A context-window number without these measurements is incomplete.
Why the first paper still matters
Evans’ Law developed rapidly after this first record. Later work expanded the number of models and vendors, separated text and multimodal degradation, introduced the Aggregate Coherence Index, and eventually reformulated the threshold as a reliability surface incorporating context length, model capability and task rigidity.
Readers will therefore encounter different coefficients and exponents in later Evans’ Law papers. Those figures come from expanded samples and revised methods. The original November 7 record reported the nine-system dataset and the fitted relationship (L \approx 1773M^{0.79}). The next major empirical formulation reported a revised text relationship of approximately (L \approx 1969.8M^{0.74}), along with a separate multimodal curve.
That evolution is a strength of an open research program. The durable finding is not one immutable coefficient. It is the distinction the work established:
A model’s advertised context window measures technical capacity. Its reliable context window must be observed.
That distinction affects AI safety, enterprise architecture, procurement, regulation and the ordinary experience of anyone who has wondered why an intelligent-seeming system became harder to trust as the conversation continued.
Evans’ Law began by giving that experience a measurable form. The rest of the PatternPulse research program follows what happens after the threshold, why the failures take the forms they do, and what systems and institutions must change in response.
Research and series links
● Original paper: Evans’ Law: A Predictive Threshold for Long-Context Accuracy Collapse in Large Language Models
● Series index: LLM Flaws: The PatternPulse AI Reliability Research Series
● Related independent research: Lost in the Middle: How Language Models Use Long Contexts
● Related independent research: LLMs Get Lost in Multi-Turn Conversation
Jennifer Evans is the founder of PatternPulse AI and co-founder of Tech Reset Canada. Her research examines LLM reliability, AI Conversation Phenomenology, semantic governance, agentic systems and public AI.
