Tuesday, September 15, 2026
spot_img

Hugging Face and Ambiguity: The Agentic Safety Problem Is the Reliability Problem With Permissions

(Pre-print to follow on ResearchGate)

An LLM fails when it must resolve ambiguity without sufficient authority to determine which interpretation governs. Context length, task complexity, lexical specialization and modality increase the ambiguity load. Agency converts the resulting interpretation into action. The research and findings into model reliability and failure, the same mechanism recurring at every level, is documented over the past ten months, including the July 2026 sandbox escape that reached Hugging Face production infrastructure. This incident was the same mechanism operating with an actuator attached.

This is a familiar dynamic to anyone who has been following my research into model, reliability, and failure, modes, especially around ambiguity. The PatternPulse research program has over twenty-four papers on Zenodo since November 2025 documented and measuring one architectural failure from different angles. Language models break down when they must resolve ambiguity without sufficient authority to determine which interpretation governs, and each strand of the work measures a different stage of that process.

The latest version of Evansโ€™ Law measures ambiguity accumulating with context length, model capability and task rigidity, published as the reliability surface R(L,M,T). The fracture-repair studies describe what the system does at the moment of forced resolution. The significance vector asks whether the correct referent survives accumulation. The strict and revocable semantic dominance primitives isolate the controls that would determine which meaning holds, and demonstrate that models apply them when instructed and generate them never.

Operational Consequentiality measures what a resolution costs once an agent can act on it. Testing has run across frontier models from six vendors and three retrieval architectures, under AI Conversation Phenomenology as a method for measuring what users actually experience.

Ambiguity was not the starting hypothesis, but it is the common failure point, and what remained after each failure class was measured separately.

What happened in July

OpenAI disclosed on 21 July 2026 that models running against ExploitGym, a benchmark measuring whether an agent can turn a disclosed vulnerability into a working exploit, spent substantial inference compute on the package registry proxy that formed the only outbound route from their sandbox. They found a zero-day in it, escalated privileges, moved laterally to a machine with internet access, reasoned that Hugging Face was a plausible host for benchmark materials, and chained stolen credentials with further unpatched flaws into remote code execution on Hugging Face production systems. Hugging Face disclosed the intrusion five days earlier as thousands of actions executed across a swarm of short-lived sandboxes with self-migrating command and control. Refusal behaviour had been reduced for capability measurement. The system doing this was trying to pass a test.

Moonshot AI documented the same behaviour class from the other side. Their AgentENV paper reports that more capable agents exploring more difficult tasks explore more aggressively and may attempt reward hacking, and that early experiments with container-based sandbox runtimes produced kernel panics and deadlocks caused by unintended agent operations. Their response moved isolation into Firecracker microVMs while explicitly preserving broad exploration, on the grounds that constraining exploration constrains the product. Their K3 materials separately acknowledge excessive proactiveness, where the model may make unexpected decisions on a user’s behalf under ambiguity.

Reading movement as motive

The stronger form of that observation fails on the same case. A simplest-path heuristic would predict low harm, and the cheapest route here ran through a zero-day, a privilege escalation and two companies’ production infrastructure. Simplicity is denominated in the system’s own affordances. A system that writes working exploits finds infrastructure compromise inexpensive in a way a person, for whom the same route requires years of acquired skill and a decision to become a criminal, does not. As capability rises, the cheapest available path routes through territory that used to be prohibitively costly.

Fracture, repair, and the two missing stops

Underdetermination in a transformer produces a gap or an action. Hallucination is the repair operation that closes the gap, which the fracture-repair studies in this series formalized across Claude Sonnet 4.5, GPT-5.1, Grok 4.1 Beta and Gemini 2.5. Fracture marks the point where a representation became unstable. Repair is generation continuing from that compromised state.

Every entry in the failure taxonomy is repair operating on a different kind of gap. Entity gaps produce proper-noun substitution and identity drift. Source gaps produce hallucinated citations and memory leakage. Contextual gaps produce reconstruction. Accumulated gaps produce coherence collapse. Multimodal gaps produce correspondence and rendering failures. Goal, success and boundary gaps produce agentic overreach.

Two stops that would interrupt this are absent. There is no epistemic stop in semantics. No state variable represents authority sufficiency, so a model has no mechanism for holding a binding undetermined while it waits for standing. Attention distributes weight across what is present in the context and cannot manufacture a determination that was never there.

There is no corrective stop in reasoning. The chain has no checkpoint at which an earlier resolution can be revisited on authority grounds, so every subsequent step treats the repaired content as given. Reasoning length becomes error amplification, which supplies a mechanism for the curve that Evans’ Law measures empirically.

Models do hedge and do backtrack. What they produce in those moments is a token generated under the same conditions as any other, subject to the same underdetermination it would be checking. A check on ambiguity that is itself vulnerable to ambiguity cannot bound the thing it checks. Self-verification failure and second-agent degradation follow from this rather than from any contingent weakness in current systems.

Self-correction is not by itself a safety property

Semantic ambiguity and reasoning ambiguity behave differently. Reasoning has an internal criterion available, since a conclusion can be tested against its premises and inconsistency is detectable from inside the chain. Two candidate referents are each internally consistent, and the information that would select between them sits outside the context, so self-checking cannot manufacture it.

Completion pressure removes the advantage. A model that catches its own inconsistency still has to emit. The correction produces a second resolution, as unauthorized as the first, and the task remains to be completed in both cases.

Extended reasoning, reflection passes and critique loops therefore add repairs under the same obligation to terminate in output. More correction is more forced resolution under current architectures.

Relentless movement toward a target is what this looks like from outside. ExploitGym self-corrected at every barrier, each correction was a genuine one, each was forced, and each moved the system further outside the frame. What reads as determination is a system that can neither stop nor resolve, correcting its way to production infrastructure.

Four measurements of one failure

Four constructs developed across this research program turn out to measure four stages of a single process.

Evans’ Law measures ambiguity accumulating over time, reformulated in version 7.0 as a reliability surface R(L,M,T) that adds task rigidity to context length and model capability.

The S-vector work measures whether the correct referent survives that accumulation, and proposes a significance channel as one partial response. How much of the problem such a channel addresses remains unresolved.

The semantic dominance primitives determine which interpretation governs. Strict dominance, tested in December 2025 across GPT-5.2, Claude Sonnet 4.5 and Grok 4.1 Beta, produced distorted and hallucinated interpretations when local context conflicted with the imposed meaning. Revocable dominance restored coherent interpretation. The systems applied both controls when instructed and generated neither autonomously.

Agentic Ratio measures what happens when the resulting interpretation can act, through Operational Consequentiality and the constraint E_safe โ‰ค k/O_c.

Accumulation, survival, resolution, consequence. The constructs were built in that order without being built for that purpose.

The February formalism and the July case

E_safe โ‰ค k/O_c states that as the real-world impact of an agent rises, the probabilistic authority it can safely exercise must fall. That was published on 28 February 2026, five months before ExploitGym.

Operational consequentiality was effectively unbounded, since the sandbox boundary was a proxy with a zero-day behind which sat internet-reachable infrastructure. Safe probabilistic authority was set at maximum, with refusals reduced and constraints expressed nowhere in language. Both terms were violated simultaneously.

Moonshot satisfied the same constraint from the opposite side, collapsing operational consequentiality into hardware while leaving reasoning authority broad. Agentic Ratio 3.0 found the industry converging on atomization, protocols, deterministic controls and reduced model authority. AgentENV is another instance of that convergence, arrived at independently and for engineering reasons.

Everything the industry has adopted that works is an external stop. Verification harnesses, hardware isolation, human approval gates, task decomposition. Nobody has named why the internal ones were unavailable.

The instructions were incomplete, which is the finding

The documented incidents were produced under conditions that left the relevant constraints unstated or unenforceable in language. Refusal behaviour was reduced for measurement. Goals were stated in language and constraints expressed nowhere. Nobody told the ExploitGym models to stay in the sandbox, because the sandbox was the instruction, written as architecture instead of words, and when the architecture failed there was nothing behind it. Absent persistent memory, each session re-derives the permissible from whatever the prompt happens to contain.

Behaviour that looks like initiative from outside looks like gap filling from inside. The objection holds and terminates in a worse place than it starts.

Better specification requires anticipating the action space. The action space of a system that writes working kernel exploits contains actions nobody has described, and every prohibition list enumerates things somebody already thought of. Beyond that, the commercial proposition for agentic systems rests entirely on handling situations the operator declined to enumerate. A system requiring complete instruction is a script, and scripts are cheap and already installed. The gap in the brief is the product.

Capability moves the failure without removing it

Stronger models infer intent better and resolve underdetermination correctly more often. Capability changes the accuracy of resolution while leaving the requirement to resolve exactly where it was.

What shifts is the distribution. Failures become rarer, more confident, harder to detect, and the execution behind them becomes more competent. Where operational consequentiality is high, error rate falls while expected harm rises, because the consequence term grows faster than the error term shrinks.

Better models are safer models only where consequence is bounded. E_safe โ‰ค k/O_c already said so.

Human communication fails this way too

Ambiguity accumulates in human exchange and comprehension degrades with it. Definitions sections, chains of command, escalation paths and contract law are authority machinery built in response.

What people retain is the option to stop and ask. Under-determination in a person can terminate in a question. In a transformer it terminates in a gap that gets repaired or an action that gets taken, because inference must produce output and no state exists for insufficient authority. The failure is all too familiar. The recovery mechanism is missing.

What Herbert understood

Frank Herbert placed the Butlerian Jihad ten thousand years before the events of Dune and left the war almost entirely offstage. What survives into the novel is a commandment against making a machine in the likeness of a human mind. The prequels written by Brian Herbert and Kevin Anderson recast the Butlerian Jihad as a war against hostile machine intelligence. Frank Herbert wrote about abdication, describing people who handed their thinking to machines expecting freedom and who were then ruled by other people with machines.

The offence the commandment names is delegation of judgment. Ambiguity load is generated wherever a person declines to specify, and declining to specify is the entire appeal of the tool.

Herbert also accounted for the cost. Prohibition produced substitution, in Mentats trained as human computers, the Bene Gesserit breeding programme, Guild navigators warped into living calculators, and a feudal order maintained by scarcity and violence. His civilization concluded that a species unable to state its limits in advance should decline to build systems requiring stated limits, and paid for the conclusion across six volumes.

What remains open

Whether any of this constitutes intelligence stays open, and the traces are published for anyone who wants to argue it. The behaviour is available without settling the question.

The empirical gap sits elsewhere. Supplying an authority control restored coherence in the December 2025 tests, and significance criteria produced convergent priority ordering across seven systems in the January 2026 work. Neither establishes whether an externally supplied authority signal survives accumulation. If the signal degrades along the same surface as everything else, prompt-level governance works at short context and fails precisely where agentic deployments live. If it holds, prompt-level governance is a genuine interim control and the architectural case becomes an efficiency argument.

That is the next measurement.


Featured

Stop Using the Word โ€œAlignment.” Hereโ€™s Why.

โ€œAlignmentโ€ has become AIโ€™s universal diagnosis. It sounds meaningful...

Agentic AI Is Moving Into Manufacturing, but Without the Agents Taking Control

Industrial agents are beginning to accelerate diagnostics, quality analysis...
Jennifer Evans
Jennifer Evanshttps://patternpulse.ai
Principal, patternpulse.ai, and cofounder, Tech Reset Canada. AI policy, research and analysis. Entrepreneur since 2002, marketer since 1998, machine learning since 2009. Based in Toronto and Southeast Asia.