The Authority Problem: What Enterprise AI Gets Wrong About Safety

For the past two years, much of the AI risk conversation has been framed as a choice between two futures (excluding the existential one): either models become more accurate and alignment improves, or agentic systems proliferate and risk explodes. My own work, and Google DeepMind’s new paper on distributional AGI safety, arrive independently at the same conclusion: this framing is wrong. The real failure mode is neither “bad answers” nor “too much autonomy.” It is a systemic absence of
authority and revocation mechanisms (“governance primitives”) that allow authority to be assigned, held, and revoked as context changes.

In my paper, I argue that contemporary language-model-based systems lack primitives for semantic dominance and revocable dominance, and that this absence manifests not at one level, but across three: semantic interpretation, operational procedure, and entity persistence. Google’s paper reaches a parallel diagnosis from a different direction. Rather than focusing on language or meaning, it examines how capability may emerge distributionally across networks of agents, tools, and incentives, producing AGI-like behavior without a single, centralized intelligence. What connects these two perspectives is the same underlying problem: authority fragments faster than systems can govern it.

At the semantic level, my work documents a well-known but poorly formalized issue. Models often fail to select and hold a dominant interpretation when ambiguity is present, or they continue operating under an interpretation after the evidence supporting it has collapsed. This is not a simple hallucination problem. It is a governance problem. The system lacks a mechanism to say, “this interpretation now has authority,” and just as importantly, “that authority is revoked.” As a result, meaning remains probabilistic long after action requires commitment.

In an enterprise context, this is equivalent to an executive continuing to approve purchases from a vendor that has already been delisted. The decision process still “works.” The forms route correctly. The approval completes. What’s missing is not information, but authority revocation. The system has no mechanism to recognize that the conditions granting authority no longer apply, so it proceeds as if nothing changed.

At the operational level, the same failure reappears in a different form. Multi-step workflows break not because individual steps are incoherent, but because procedural state is not governed. Models lose track of which phase they are in, which constraints still apply, or which assumptions were provisional. The system continues executing as if authority were still valid, even when the conditions that granted it no longer hold. This is why enterprise workflows can look locally correct while producing globally nonsensical or risky outcomes.

At the entity layer, the failure becomes materially dangerous. Proper nouns, named systems, policies, and organizations drift, genericize, or silently substitute. A system may speak fluently about “the platform,” “the regulation,” or “the customer,” while the referent has changed underneath. This passes surface content checks but breaks real-world correctness. Once again, the problem isn’t ignorance. It is the absence of a primitive that enforces identity persistence and revocation when identity is no longer justified.

Google’s distributional AGI paper describes the same pattern at a higher scale. Instead of semantic interpretations or workflow phases, it examines authority emerging across agentic ecosystems: tool-using agents, markets, planners, routers, and evaluators interacting in parallel. Their core warning is that safety research has largely assumed a monolithic AGI, while real systems will be patchworks. In these systems, no single component holds complete authority, but collective behavior can exceed any component’s capability. When authority is distributed, revocation becomes harder, slower, and more opaque.

The correspondence between these two analyses isn’t superficial. My semantic layer failures mirror Google’s concern about local optimization without global coherence. My operational layer failures map directly onto their discussion of runaway coordination and cascading actions. My entity layer failures align with their warnings about identity, access, and incentive manipulation in multi-agent environments. In both cases, the danger is greater than the system making a mistake. The danger is that it keeps going.

This is why both papers independently converge on the importance of circuit breakers. In my framework, revocable dominance is a circuit breaker for meaning, procedure, and identity. It is the ability to halt, reset, or re-evaluate before authority propagates further downstream. In Google’s framework, circuit breakers appear at the system level: market design, oversight mechanisms, gated access, and intervention points that prevent emergent behavior from compounding into irreversible harm. Different layers, same necessity.

Crucially, neither paper suggests that scaling capability solves this problem. Google explicitly warns that as transaction costs fall and agentic density increases, risk accelerates faster than traditional safety controls can adapt. My work shows the same dynamic internally: more capable models can generate more fluent, confident continuations of invalid authority. Without governance primitives, intelligence amplifies instability.

For enterprise leaders, this reframes the buying and building question. It doesn’t matter whether a model is accurate, aligned, or powerful. It matters where authority lives, how it is granted, and how it is revoked across semantic interpretation, operational execution, and entity persistence. Tool permissions, audit logs, and access controls are necessary but insufficient if the system cannot revoke meaning or phase or identity upstream.

The uncomfortable implication is that many failures attributed to “hallucination,” “agentic risk,” or “model unreliability” are symptoms of the same missing infrastructure. Enterprises are already deploying distributional systems: copilots calling tools, agents routing work, retrieval layers grounding responses, planners sequencing actions. These systems behave exactly as Google predicts and exactly as my tests demonstrate: locally coherent, globally fragile.

What does governance-as-primitive actually require? Start with procurement: instead of asking “what is the model’s accuracy on our benchmark?” ask “can this system revoke an interpretation mid-chain when evidence changes?” Ask “does this tool provide a mechanism to invalidate procedural state, or does it only allow workflows to complete or error out?” Ask “can we programmatically enforce that entity references remain stable, or are we trusting the model to maintain coherence?” Most vendors cannot answer these questions because the primitives do not exist in their architecture.

The audit logs show what the system called, but not why it believed it still had authority to do so.

The retrieval layer shows what was retrieved, but not whether the system revoked prior context when contradictory information arrived.

The planning module shows what steps were executed, but not whether phase transitions were governed or simply assumed.

These are not missing features—they are missing categories. A system built without revocable dominance cannot be patched into safety after deployment. This is why the current generation of enterprise guardrails functions as documentation: they describe what should happen, but they do not enforce the authority structure that would make it so. Real governance means the system can stop itself because it detected an error *and* because the conditions that authorized continuation no longer hold.

The future of enterprise AI will be secured by more than better answers alone. It will be secured by governance architectures that treat authority as a first-class primitive at every layer where action emerges. Without that, we will continue to build systems that look intelligent right up until the moment they fail—and then cannot explain why they didn’t stop.