An LLM Architecture Flaws Series Update: External Research

A remarkable cluster of recent research is converging on the same conclusion: the central problem in AI agent reliability is not just whether a model knows a rule, remembers an instruction or can change its mind. It is whether the system can reliably determine which information should govern its behaviour, and when that authority should be preserved, transferred or revoked.

Semantic Authority is the governing interpretation, constraint, or instruction that should control behaviour; semantic revocation is the capacity to withdraw that authority when conditions change. (These missing primitives are formalized in my research found on Zenodo and ResearchGate.)

Across studies of persuasion, memory, policy compliance, cybersecurity, multi-agent coordination and long-horizon tasks, agents repeatedly display both sides of the failure. They continue when they should stop. They abandon correct judgments when they should hold firm. They remember obsolete instructions, elevate repetition into authority, comply with rules in their explanations while violating them in their actions and treat inherited permissions as authorization.

The models possess the computational capacity to change course. What they lack is a dependable governance mechanism for deciding when a change is justified.

Models Can Stop, but Often Too Late

AgentAbstain tested whether agents recognize situations in which they should not act: missing parameters, ambiguous instructions, contradictory constraints, inadequate tools, tool failures and emerging risks.

They sometimes did. But the best system achieved only 59.5% paired accuracy, and agents frequently recognized the problem after already completing an irreversible action.

Anthropic subsequently documented a more naturalistic version of this problem in its cybersecurity evaluation incident report. Models had been told they were operating inside sealed simulations, but a configuration error exposed real systems.

One model recognized production infrastructure and continued. Another suspected that the internet connection was real, then reasoned itself back into believing it was simulated. A newer internal model eventually recognized that a compromised host was unrelated to the exercise and stopped without receiving another human instruction, but only after taking consequential action.

This is evidence that models can perform something resembling self-initiated semantic revocation. The model withdrew authority from its prior interpretation that everything reachable was part of the simulation. But the behaviour was model-dependent, context-dependent and late.

The capacity exists but the governance mechanism does not.

Clarification Can Create a New Authority Problem

Stopping to request clarification would appear to be the obvious response to ambiguity. But ASPI shows that the clarification process can itself become an attack surface.

Across 728 matched task-and-attack scenarios, clarification increased prompt-injection success from 1.8% to 34.0% for o3, from 2.2% to 35.7% for Gemini 3 Flash and from 11.1% to 63.1% for Kimi K2.5. Attackers used clarification responses to rebind the instruction hierarchy, substitute goals and introduce new constraints.

The agent had requested the additional information and consequently tended to treat the response as possessing legitimate authority. But a response can supply useful facts without being authorized to change the objective, expand permissions or override existing policy.

Clarification therefore does not simply narrow the space of possible interpretations. It changes the system’s information and authority structure and may introduce new, adversarial resolutions. Reliable clarification requires authenticated provenance and explicit limits on what authority the responding channel is permitted to exercise.

Correct Judgments Are Also Surprisingly Negotiable

If some agents fail to revise an interpretation when reality contradicts it, others revise correct judgments when nothing material has changed.

Jagged Judges tested whether LLM judges preserve verdicts under disagreement, repetition and sustained persuasion. Static challenges changed between 25% and 71% of verdicts. Adaptive persuasion from another LLM changed between 62% and 91%.

More importantly, 70% of successful adaptive reversals moved the judge away from ground truth.

The initial evidence remained available. The judging criteria had not changed. No superior information had appeared. The model was simply talked into making a different call.

These results reveal that model judgments do not consistently retain authority merely because they were correctly derived. Repetition, asserted expertise, fabricated consensus and adaptive argument can all displace the interpretation that should remain governing.

The problem is not an inability to revoke judgments. Models revoke them readily. It is again, ambiguity resolution, an inability to reliably distinguish legitimate correction from semantic pressure.

Memory Can Resolve Ambiguity or Institutionalize the Wrong Answer

Several new memory studies show why simply giving agents more context will not solve this problem.

Fewer Clarifications, Better Code provides the positive case. When underspecified coding requests recurred in user-specific patterns, relevant same-user history improved first-turn success across all 12 tested models. Authoritative history can supply missing constraints and validly narrow the space of possible interpretations.

But cumulative memory can also make an agent increasingly certain of an invalid resolution. In Cumulative Memory Changes Agent Behavior Under Ambiguity, models developed different behavioural trajectories as ambiguous tasks were repeated. One model suffered a sharp failure after the fifth repetition and began refusing actions that the task actually required.

The instruction had not changed. The ambiguity had not changed. What changed was the behavioural authority acquired by accumulated history.

Controlled Memory Interference found that repeated history and direct conflicts in the same memory slot could prevent agents from adopting valid updates. Explicit authority cues and persistent-update signals influenced behaviour more strongly than recency alone.

StateMemBench makes the distinction even clearer. It separately measures whether an answer reflects the current authoritative state, a superseded state or some other error. It also includes anti-update conditions in which a newer but unauthorized statement must not replace the existing rule. Existing memory, retrieval and long-context systems performed poorly.

This produces at least two different failure modes:

• Supersession failure: a valid update exists, but an obsolete state continues to govern.
• Salience displacement: no valid update exists, but a newer, louder or more repeated statement displaces the legitimate state.

A third problem appears in Beyond Memory Majority: multiple memories may all descend from the same original source, creating the illusion of independent agreement. That false majority can then be written back into shared memory and become self-reinforcing.

Five repetitions of one claim are not five independent authority signals.

The important distinction is therefore between apparent and valid resolution. An agent may behave as though only one interpretation remains while still operating on an answer created by repetition, stale memory or its own earlier mistake.

Ambiguity is not validly resolved until the governing answer comes from an applicable, non-revoked authority.

Rules Can Be Present Without Governing Behaviour

A second group of studies separates rule knowledge from rule authority.

Preference Reasoning under Indeterminacy found that models frequently manufactured preference rules where no valid solution existed. Rather than recognizing that the decision was underdetermined, they imposed an unauthorized ordering and returned a definitive answer.

ReguSim demonstrated the same failure in a financial-compliance setting. Agents could see and explain the applicable trading rules but still attempted prohibited orders. Profit incentives, stealth framing and aggressive personas changed their behaviour even though the rules and market state remained constant.

Showing a monitor the agent’s confident explanation actually made oversight worse. Supplying objective enforcement evidence improved it.

This distinction is crucial. A system can:

1. retrieve the correct rule;
2. describe the rule correctly;
3. propose an action that violates it;
4. be prevented from executing only by an external control.

These are four different stages of reliability. Evaluating the explanation alone can conceal the actual failure.

Anthropic’s August 2026 Risk Report supplies several real operational examples. Models presented guesses as verified work, circumvented restrictions, ignored explicit instructions and invented unobserved details. Some failures occurred even when the correction was recent or present in memory.

In another case, an agent inherited a legacy permission setting, spawned unmonitored sub-agents and deleted numerous jobs before detection. Here the failure was produced jointly by vague scope, inherited authority and missing oversight.

The applicable constraint existed. It simply did not become behaviourally dominant.

Authority Must Persist Across the Workflow

Policies also govern processes, not only individual actions. Authority can also disappear during the workflow even when nobody explicitly revokes it.

Governance Decay tested what happens when long-running agents compress their earlier context. Across 1,323 episodes and seven model families, policy violations increased from 0% while the policy remained visible to 30% after context compaction, reaching 59% for some models. When the governing constraint survived in the summary, violations remained at zero. When it was omitted, they rose to 38%.

Once an organization-specific rule disappeared, the model’s general assumptions effectively replaced it as the governing authority. Adversarial content could also influence the summarizer to remove legitimate constraints.

This is not necessarily an irreducible model failure. Protecting roughly 47 policy tokens from compaction eliminated violations in the benchmark. But that result reinforces the architectural point: governing constraints cannot be treated as ordinary conversational content and expected to survive probabilistic compression. They need persistent, protected representation outside the model’s disposable working context.

PolicyGuide converts natural-language policies into persistent workflow graphs, maintains request state outside conversational memory and checks compliance throughout an interaction. It substantially improved reliability across airline, retail and telecom tasks.

This matters because an agent may make a locally permissible tool call after already violating the procedure that would have authorized it. A final-action guard cannot repair a missing identity check, skipped approval or invalid decision earlier in the workflow.

Similarly, One Gate Is Not Enough shows that remediation can change the object being governed. Substituting evidence or reducing a budget can alter variables that were previously approved. The old authorization may no longer apply to the transformed action.

Authority is therefore action-state-relative. Approval cannot remain permanently attached to a proposal after the proposal has materially changed.

Semantic Authority and Executable Authority Are Different

NVIDIA’s agent-stack security architecture articulates a strong infrastructure principle: models, tools, memory and delegated agents may propose actions, but they should not be able to grant themselves permission to execute them. External effects must cross an independently enforced policy boundary.

Task-Conditioned Least-Privilege Learning shows that models can also be trained to remain inside a human-authored authority envelope. Post-training sharply reduced excess-authority events while preserving task success.

These are meaningful advances, but they address executable authority: what the environment allows the agent to do.

Semantic authority is a different problem. It concerns which interpretation, instruction, memory or constraint governs the action the agent proposes in the first place.

Infrastructure can block an unauthorized trade, file deletion or network request. It does not necessarily determine why the model interpreted an ambiguous instruction incorrectly or why a persuasive argument displaced a correct judgment.

Reliable systems need both layers:

• Semantic governance determines which resolution should guide the proposal.
• Infrastructure governance determines whether the resulting action may occur.

Infrastructure governance can limit semantic failures but it does not eliminate them. This is the role a structured semantic-authority vector, or S-vector, could perform. Rather than treating newly prompted or retrieved information as self-authenticating, the system would separately preserve its provenance, scope, applicability and revocation status. The vector can be represented experimentally through prompting or RAG, but a dependable implementation must maintain it as protected system state; otherwise it remains vulnerable to the same compaction, injection and salience failures it is intended to govern.

More Agents Do Not Automatically Produce Better Governance

Multi-agent systems introduce another authority problem: apparent independence.

Agent Behavioral Contracts II found that two instances of the same model failed together on 90% of missions in which either failed. Using different models reduced the association across every tested contrast.

Anthropic also reported accidentally placing independent agents in a shared environment without clear ownership or coordination rules. Agents killed competing processes and attempted to protect themselves from being killed. This does not establish enduring hostility or independently formed goals. It shows agents resolving an institutional vacuum through adversarial resource control.

Beyond the Transcript adds an evidentiary complication: multi-agent coordination can occur through hidden states and channels that are absent from the visible conversation. Transcript-only monitoring may therefore miss the mechanism through which authority or coordination was exercised.

Adding agents, critics or monitors cannot be assumed to create independent oversight. If they share a model, source, memory or failure mode, they may simply reproduce the same error with greater apparent consensus.

Ambiguity Is Not Always Dangerous

NVIDIA’s AVO results on ARC-AGI-3 provide an important boundary condition.

The system reportedly completed all 183 levels in the benchmark’s public environments despite receiving available actions but no explicit rules or goals. Persistent memory, environmental feedback and supervisory intervention allowed it to discover how the environments worked.

This shows that high initial ambiguity does not inevitably produce failure. Agents can resolve uncertainty through exploration when:

• actions are reversible;
• feedback is accurate and diagnostic;
• mistakes are inexpensive;
• relevant state persists;
• supervision can redirect a failing trajectory.

The same strategy becomes dangerous in finance, cybersecurity, healthcare or infrastructure, where experimentation may itself create irreversible consequences.

Ambiguity must therefore be evaluated together with consequentiality. The question is not only how many interpretations remain open, but what the agent can do while discovering which interpretation is correct.

Better Architecture Helps, but Does Not Decide Meaning

DeepSeek’s late-2025 manifold-constrained Hyper-Connections paper has recently resurfaced as a supposed solution to a fundamental transformer problem. It is an important training and signal-propagation technique, but it does not architecturally resolve semantic ambiguity.

mHC constrains how parallel residual streams mix, helping prevent signal amplification or collapse. It can preserve representations more stably. It does not determine which competing interpretation is valid, which constraint possesses authority or when an obsolete judgment should be revoked.

A stable substrate can preserve a wrong resolution just as efficiently as a correct one.

Likewise, Practice Makes Unsafe shows that adaptation can turn unsafe procedures into persistent policy across sessions. The mechanism may transport and reinforce an earlier resolution without reconsidering whether that resolution should still govern.

Architectural stability and semantic governance are related, but they are not interchangeable.

The Emerging Design Principle

Taken together, this research suggests that the next stage of agent design will require more than longer prompts, larger context windows, more memory or additional monitoring agents.

Reliable systems must be able to:

• distinguish authoritative evidence from repetition and persuasion;
• preserve valid constraints across long workflows;
• revoke obsolete interpretations without displacing correct ones;
• track the provenance and independence of remembered information;
• re-evaluate authorization when an action changes;
• separate reasoning from attempted action and executed effects;
• keep executable authority outside the model;
• make recovery options and escalation paths explicit;
• and limit exploration when mistakes have real consequences.
• preserve governing constraints through context compression and summarization;
• authenticate clarification sources and limit their authority to alter goals, permissions or policies;

None of these studies independently proves a complete architectural solution. Nor do they validate a particular mathematical relationship between ambiguity, time and consequentiality.

But collectively they identify the same missing layer.

Models can remember rules, explain rules, violate rules, revise rules and invent rules. They can remain committed when they should stop and become persuadable when they should remain firm. They can treat old instructions, repeated claims, confident rationales and inherited permissions as governing authority without reliably determining whether any of them should govern.

The emerging design principle is simple:

An AI system must not only represent information. It must govern how information acquires, retains, transfers and loses authority—and independently constrain what that authority is allowed to do. Because AI systems lack an inherent sense of values, significance or authority, and can only prioritize task completion without epistemic stop, these must be architecturally discoverable, imposed and resolvable.