On July 15, Fouad Bousetouane of ProofAgent.ai and the University of Chicago posted a paper to arXiv with a title that doubles as a thesis: “AI Agents Do Not Fail Alone: The Context Fails First.”⁠ The paper presents context-engineering quality as an independent leading indicator of agent reliability, measured through an open-source harness that scores agent contexts across seven criteria before behavior is evaluated. It is an independent empirical result that lands precisely where the Agentic Ratio (ResearchGate, also on Zenodo) predicts: what the industry calls agent behavior cannot be attributed to the model alone. It is powerfully shaped by the engineered system surrounding it.

What the study did

Bousetouane’s design holds the model backbone fixed and varies the operating context: instructions, guardrails, tool schemas, grounding, role definitions and injection hardening. He then measures the resulting behavior across customer support, healthcare claims triage and legal contract drafting.

The study comprises 300 multi-turn evaluations and 7,500 evaluated agent turns using GPT-5.5 and Claude Opus 4.8 as the model backbones. Contexts were divided into three conditions: poor, structured and hardened.

The largest behavioral change occurred between poor and structured context. The overall behavioral score rose from 3.15 to 5.49. Hallucination resistance increased from 3.21 to 5.61, tool-use performance rose from 3.46 to 6.25 and critical failures fell from 4.11 to 1.33 per evaluation.

The relationship was also criterion-specific. Grounding quality correlated with hallucination resistance. Guardrail coverage correlated with manipulation resistance. Instruction consistency correlated with instruction following. Tool-schema quality correlated with tool use.

The context itself is scored through multi-juror, consensus-based evaluation. Those scores are kept outside the calculation of behavioral scores and release outcomes. That separation prevents the central result from becoming circular: the harness is not declaring that context predicts behavior because context quality has already been included in its behavioral score.

Instead, it evaluates an engineering artifact and then asks whether that evaluation carries predictive information about what the resulting system does.

The scaffolding thesis, partially tested

My paper, Agentic Ratio 3.0, argued that observed agentic behavior arises overwhelmingly from deterministic engineering rather than model-driven autonomy. The Evans Ratio, E = Cp/Cd, quantifies the relationship between the probabilistic contribution and the deterministic contribution to a system’s behavior.

Cross-vendor measurement in the v6.0 study found production systems operating at E below 0.3, meaning deterministic scaffolding contributes between three and ten times more to system behavior than the probabilistic core. The Brock Threshold marks E = 1, the boundary at which genuine model-driven agency would begin. No commercial system measured approaches it.

A skeptic could reasonably ask how the underlying mechanism might be tested experimentally.

A clean first test would hold the model backbone stable, vary the engineered context and determine whether behavior moves systematically with the layer being changed. That would not by itself estimate the Evans Ratio. Measuring the relative contributions of Cp and Cd would require varying both the model and the scaffolding, then comparing their effects through a factorial design or variance decomposition. It would, however, establish whether the deterministic layer has the causal leverage the framework assigns to it.

That is what Bousetouane’s experiment does. He arrived at it from the engineering side, asking a practical question about preflight signals for reliability. As far as I can determine, he did so without awareness of the Agentic Ratio, the Evans Ratio or the Brock Threshold.

The result is not a measurement of E. It is independent evidence for one of the mechanisms on which E depends. When the model backbone remains fixed and engineered context changes, behavior changes with it, often dramatically and along the predicted dimensions.

The ProofAgent harness is, in effect, an instrument for measuring one large and inspectable component of Cd.

The harder result is not monotonic

One of the most interesting results is that more scaffolding did not always produce better aggregate behavior.

Moving from poor to structured context created the largest reliability gains. Moving from structured to hardened context raised the context-engineering score from 8.08 to 8.68, but the final behavioral score declined slightly from 5.49 to 5.16. Safety and hallucination resistance also fell modestly, while critical failures increased from 1.33 to 1.56.

Bousetouane attributes this to conservatism introduced by additional refusal conditions, escalation thresholds, injection separation and confirmation requirements. A system can become more extensively governed while becoming less capable of completing borderline tasks.

This is important because it shows that scaffolding is not merely an accumulation of rules. Its components interact. Adding deterministic controls can improve one property while degrading another. Guardrails do not simply make an agent “better.” They alter the distribution of actions the assembled system is permitted or encouraged to take. The layer requires measurement rather than assumption.

Atomization, arriving on schedule

Agentic Ratio 3.0 made a second claim, structural rather than quantitative. As agentic systems scale in enterprise environments, agency is being decomposed into separable, independently governed components. The industry says it is building autonomous agents. Its architectures say it is disassembling agency.

Bousetouane extends that decomposition into evaluation itself. He positions context engineering as an auditable layer of agent evaluation and governance, scored separately, validated separately and inspected before release.

The context becomes a governed artifact with its own measurement regime, distinct from the model and distinct from observed behavior. That is atomization applied to oversight: one more component of “the agent” split off, named and placed under independent examination.

Even the paper’s title contains the structural insight. An agent that does not fail alone is not being treated as a unitary actor. The assembled system fails, and the failure can originate in a specific, measurable layer of the assembly.

Naming context as the layer that fails first embeds this anti-unitary insight in the paper’s failure model. The “agent” is being treated not as a singular autonomous entity, but as a composite system whose components can be separated, measured and governed.

What convergence is and is not

This is independent convergence. It is not a replication of Agentic Ratio 3.0. Bousetouane does not engage the Evans Ratio, the Brock Threshold or the atomization argument. His experiment demonstrates the causal leverage of engineered context; it does not establish how much of total system behavior is attributable to scaffolding relative to the model.

The study also measures context in regulated domains without measuring operational consequentiality, leaving the quadrant structure of Agentic Ratio 3.0 untested.

Its individual context criteria correlate with their expected behavioral outcomes, but several dimensions of context were changed together across the three experimental conditions. The experiment therefore does not independently isolate the causal effect of every criterion.

The context and behavioral scores are separated computationally, but both are produced through the ProofAgent evaluation infrastructure using LLM jurors. This is an initial validation of the harness by its creator, published as a version-one arXiv preprint. Independent reproduction, cross-vendor calibration and resistance to score gaming remain necessary before the measurement can function as a certification standard.

The temporal dimension also remains open. The study uses 25-turn evaluations, but it does not analyze how error probability changes as reasoning accumulates across those turns.

Evans’ Law (related, but separately) concerns that trajectory: the probability of an incorrect response grows with reasoning length until error becomes more likely than accuracy (a paper now with over 3000 interactions on Zenodo) Bousetouane measures the quality of the operating environment in which a run begins and continues. He does not test whether a strong context changes the rate at which reliability degrades as reasoning accumulates.

The two frameworks therefore address different parts of the system’s reliability profile. Context scoring asks how well the reasoning environment has been constructed. Evans’ Law asks what happens as probabilistic reasoning proceeds inside it. Convergence of this kind is worth more than citation. A researcher who reads work and confirms it may have been persuaded by the framework. A researcher who never encountered it and arrives at the same structure was pushed there by the phenomenon itself.

Governance in the preflight era

Agent reliability now has the beginnings of a measurable preflight signal: an assessment that can be conducted before deployment, attached to evidence and potentially reproduced by third parties.

A vendor claiming agentic capability can be asked to disclose the context artifact and produce a structured assessment of it. Procurement teams can examine whether roles are explicit, instructions conflict, tool boundaries are defined, grounding is sufficient and untrusted inputs are separated from authoritative instructions.

This is not yet the equivalent of a building inspection certificate. Bousetouane explicitly describes the context grade as diagnostic rather than certification, and it does not replace adversarial testing, red teaming, trace audits or human review.

It does, however, provide the beginnings of something enterprise governance has lacked: a common, inspectable object against which context quality can be evaluated before deployment. The measurement proposal exists, an initial controlled validation exists and the infrastructure is open source. This changes the governance conversation.

Behavioral evaluation asks what the agent did after the fact, under conditions that may never recur. Context evaluation asks what the system was built from, in advance, using terms that can eventually be compared across vendors.

Regulators, procurement officers and insurers work naturally with the second kind of evidence. They govern components, controls and documented conditions more readily than emergent claims about autonomy.

The deterministic layer was always where much of the system’s reliability lived. Bousetouane’s work gives enterprises a way to begin seeing it before the failure arrives, and places accountability in the same engineered layer from which so much behavior originates.

Jen Evans is Principal of Pattern Pulse AI and co-founder of Tech Reset Canada.