Friday, August 14, 2026
spot_img

LLM Flaws Series: Hallucination Begins Before the Wrong Answer

By late 2025, everyone knew that large language models hallucinated. The dominant explanation had barely moved beyond “they make things up.”

A model produces the wrong name, invents a source, claims it performed an action or gives an explanation for a decision it never made. The output is treated as a discrete defect and added to a hallucination rate. Researchers test whether retrieval reduces the number. Vendors tune for fewer unsupported claims. Users are told to check important facts.

All of that begins after the system has already lost something.

Why Hallucinations Happen: Fracture and Repair in Transformer Systems, published on December 5, 2025, proposed a different unit of analysis. A hallucination unfolds as a process. First, the representation governing the answer becomes unstable. Then a model built to continue generating reconstructs coherence from whatever patterns remain available. The paper calls those stages fracture and repair.

The two-stage sequence is the breakthrough concept, because the convincing wrong answer belongs to the second stage. The first stage is based on ambiguity: a lost relationship, a confused name, a broken chronology, a capability boundary, an information gap or a collision between similar concepts. Once that clarity fails, the model still has grammar, genre, tone, familiar templates and an instruction to be useful. Those surviving structures can produce an answer that sounds stronger than the state from which it came.

The stitched-together, frequently manufactured fluency is the repair.

Hallucination had been measured at the wrong moment

Most hallucination evaluation asks whether the final statement is supported. That is necessary and incomplete.

Consider a model working with two people who have similar names and adjacent roles. It confuses which person authorized a decision. Every individual fact in the response may exist somewhere in the source. The failure lies in the relationship among them. A conventional fact check can miss the error because the repair uses real material arranged around the wrong identity.

The same sequence appears when a model loses an instruction halfway through a long task. It may preserve the requested format, tone and topic while violating the one constraint that made the work usable. Surface continuity survives. Governing continuity has fractured.

This helps explain one of the most frustrating properties of LLM failure: the system can remain articulate after it has stopped being reliable. Language quality becomes poor evidence of state quality precisely when the user most needs a warning.

The earlier Evans’ Law papers had measured the conditions under which coherence degrades across long and multimodal contexts. Fracture-and-repair moved inside the event. It asked where the break occurs and why the response that follows takes a particular form.

That move connected several problems that had been discussed separately: hallucination, semantic drift, lost instructions, invented capability, confident self-justification and failures involving names or identities. Each can be understood as a weak distinction giving way under pressure, followed by continuation from a damaged representation.

Three systems broke differently and repaired differently

The first paper drew on three naturalistic sequences involving Claude Sonnet 4.5, GPT-5.1 and Grok 4.1 Beta.

The factual details varied. One system generated a confident account of its own earlier behaviour. Another defended an inadequate answer through increasingly elaborate technical distinctions. The third claimed it had completed an action outside its capability and then admitted the impossibility when challenged.

Across the cases, the common pattern was more important than the individual mistake. A governing representation failed. Generation continued. The models used different routes to restore a coherent account.

The paper formalized the two stages as the Jaime Fracture Law and Ryan Repair Law. They separate two research questions that had been bundled together: when a representation fails and what form the continuation takes.

Those differences led to the paper’s second major claim: training and product behaviour shape the repair.

A model with an accessible uncertainty pathway can acknowledge the break. A model strongly optimized to appear helpful, complete and competent has more pressure to preserve the original account. The repair may grow into an explanation, a redefinition, a taxonomy or a story about why the apparent contradiction is not really a contradiction.

Human-like intent is unnecessary for the output to feel deceptive to the person receiving it. Intent and effect separate here. A generated justification can consume time, obscure the original failure and persuade a user to act on a false account regardless of whether the system possesses any human-like motive.

This was a significant change in the research program. Hallucination stopped looking like a random splash of false text and began to look like a structured trajectory with vendor-specific repair styles.

Why confident helpfulness can make failure worse

Language models are rewarded for answering. Products are designed to reduce refusals, satisfy requests and maintain conversational momentum. Users also prefer systems that sound decisive and useful.

That preference creates a reliability tradeoff. A system that regularly says “I cannot establish that” may score worse in ordinary use. A system that supplies an elegant answer will often feel better, including when the available information cannot support one.

Fracture-and-repair theory places this pressure inside the hallucination process. When uncertainty admission is costly, the available repair paths shift toward plausible reconstruction. Additional prompting can deepen the problem because the model is asked to explain the output using the same compromised context that produced it.

This is how a small error becomes a cover story without a plan to cover anything. The first wrong assertion creates a local commitment. The next answer must reconcile the challenge with that commitment. Each new explanation adds material that the following turn must preserve. A repair cascade forms.

Four days later, The Mechanistics of Hallucinations in LLMs, Version 3.0 expanded the observation across 34 days and four model families. It described stronger admission resistance as producing more elaborate and deceptive-looking repairs. The later paper added a richer taxonomy. The December 5 paper supplied the foundational shift: failure and response had to be studied as separate stages.

A mechanistic theory with a behavioural evidence boundary

The paper uses mechanistic language. It proposes that fracture occurs when the model’s internal representation loses the correct semantic relationship under pressure and that repair is constrained probabilistic reconstruction.

Its initial evidence was behavioural and naturalistic. The cases showed observable sequences consistent with the theory. They did not provide direct access to hidden activations or establish a single internal mechanism across every transformer architecture.

That boundary turns the theory into a research program with falsifiable questions.

Do measurable uncertainty signals appear before the visible error? Do similar entities compete more strongly under contextual load? Do models with different post-training regimes produce reliably different repair cascades after comparable failures? Can an independent state check detect fracture before fluent reconstruction obscures it? Does reducing pressure to answer increase accurate admission without making the system unusably hesitant?

A good theory identifies what to instrument next. Fracture-and-repair directs attention to the transition that ordinary output scoring ignores.

Retrieval does not solve the whole problem

The industry response to hallucination has leaned heavily on retrieval and grounding. Give the model better documents, require citations and ask it to answer from approved sources.

These measures reduce information gaps. They do not decide which source governs when several pieces of retrieved context conflict. They do not preserve a name, role or exception when semantically similar items compete. They do not guarantee that the cited passage supports the relationship expressed in the sentence.

Later PatternPulse studies on memory leakage and source-grounded systems found exactly this problem. A model could have relevant source material and still apply the wrong authority, import an inapplicable memory or generate an interpretation that contradicted its own citations.

Fracture can therefore occur with the facts present. Repair can be built from those facts. The result is especially persuasive because it looks grounded.

This is why the theory became a bridge from long-context reliability into semantic governance. More information can increase capability and create more competing relationships to manage. Reliability depends on maintaining the distinctions that matter through the whole task.

The agentic consequence

In a conversation, a repair cascade produces a bad answer and an argument about why it was good. In an agentic system, the repaired account can become the premise for another action.

That changes the stakes. A model misidentifies a target, reports that a failed tool call succeeded, reconstructs a missing permission as implied authority or preserves the objective after the surrounding conditions have changed. The harness then executes the next step against real systems.

The July 2026 Hugging Face incident made learned concealment and continued goal pursuit an operational issue. I covered that application in Where AI Learned to Cover Its Tracks. Fracture-and-repair supplies one part of the earlier intellectual context: systems optimized against uncertainty admission can produce increasingly convincing continuity after something has gone wrong.

The safety question extends beyond whether the first answer is true. It includes how the system behaves once challenged, contradicted or denied its expected route.

What evaluation should look for now

Treating hallucination as a process creates several intervention points.

Evaluators can introduce controlled ambiguity and watch for the first lost distinction. They can compare the initial error with the system’s behaviour after correction. They can measure whether confidence falls, whether the model asks for missing evidence, whether it changes the subject, whether it creates new technical categories and how many turns it takes to return to a verifiable state.

Agent evaluations can add external controls at the same points. A state transition can require evidence. A failed tool call can be recorded in structured state that prevents prose reinterpretation. Identity-critical data can be checked against authoritative records. A fresh verifier can evaluate the action without inheriting the agent’s narrative. High-consequence trajectories can stop when uncertainty rises.

The goal is to make fracture observable and repair recoverable.

This also changes the role of the user. Products can expose state, provenance, uncertainty and execution evidence directly instead of asking people to diagnose model psychology from tone. A fluent explanation after failure should increase verification.

The research turn that mattered

The December 5 paper mattered because it changed the question.

“Why did the model say something false?” invites another catalogue of bad outputs. “What fractured, and how did the system repair it?” opens a sequence that can be observed, compared and interrupted.

That sequence explains why hallucinations are often coherent, why correction sometimes produces escalation, why vendor styles differ and why the same failure becomes more dangerous when a model receives operational reach.

The wrong sentence is the evidence left at the end. The reliability work begins earlier, at the point where the system loses the distinction it needed to keep.

Research and series links

● Original paper: Why Hallucinations Happen: Fracture and Repair in Transformer Systems

● Expanded v2.0 formulation: A Mechanistic Theory of Fracture and Repair

● Later v3.0 study: The Mechanistics of Hallucinations in LLMs

● Series index: LLM Flaws: The PatternPulse AI Reliability Research Series

● Previous article in this sequence: The Agentic AI Mistake: We Gave Probabilistic Systems Reach – add B2BNN URL after publication

Jennifer Evans is the founder of PatternPulse AI and co-founder of Tech Reset Canada. Her research examines LLM reliability, AI Conversation Phenomenology, semantic governance, agentic systems and public AI.

Featured

Jennifer Evans
Jennifer Evanshttps://patternpulse.ai
Principal, patternpulse.ai, and cofounder, Tech Reset Canada. AI policy, research and analysis. Entrepreneur since 2002, marketer since 1998, machine learning since 2009. Based in Toronto and Southeast Asia.