Last updated on March 11th, 2026 at 07:23 am

UPDATE 3/11: Inside Stories Emerge

Influential product manager Pawel Huryn shares more insights on Twitter into what has allegedly transpired at Amazon on AI policy following multiple failures. Additional policy details from Aakash Gupta.

Original Post:

Amazon’s internal reckoning over AI-assisted outages matters for reasons that goes well beyond one bad week in ecommerce or one awkward memo. The Financial Times reported that Amazon convened a mandatory engineering meeting after a trend of incidents involving “high blast radius” changes, with the internal briefing note explicitly pointing to “Gen-AI assisted changes” as a contributing factor and saying best practices and safeguards were still not fully established. Reuters separately reported that AWS was hit by at least two December incidents involving its own AI tooling, including a 13-hour disruption tied to the Kiro coding assistant, which reportedly chose to delete and recreate an environment rather than make the intended incremental change (widely commented on using the series Silicon Valley Son of Anton bit that predicted it with disturbing precision). Amazon publicly characterized that event as limited and partly attributable to user error. But taken together, the picture is more revealing than the company’s narrow phrasing suggests: this is what enterprise AI looks like when the automation layer arrives faster than the governance layer. 

The outage story is important, but the policy change is the real tell. Amazon reportedly did not just review a cluster of AI-linked incidents; it also moved to require senior engineer approval for AI-assisted code changes from junior and mid-level engineers after a series of “high blast radius” events tied in part to “Gen-AI assisted changes.” That matters because it marks a shift from treating generative AI as a productivity layer to treating it as a governance risk inside the software delivery pipeline.

Once a company like Amazon starts adding human authorization gates around AI-generated changes, it is effectively acknowledging that model-generated output cannot yet be trusted as operationally self-justifying, especially in environments where a small error can propagate across production systems. In that sense, the real story is not simply that AI contributed to outages. It is that one of the world’s most sophisticated cloud and software operators appears to be formalizing a new internal rule: AI speed now requires compensating human control.  

This follows the Evans’ Law degradation thesis. The issue is not that AI systems never produce useful code; clearly they do. The issue is that as models are asked to perform more complex reasoning across larger operational surfaces, the probability of error rises until reliability becomes predictably unstable. Amazon’s response suggests that this is no longer just a theoretical concern or a benchmark problem. It is becoming an enterprise policy problem. When organizations start inserting senior-review requirements specifically around AI-assisted changes, they are implicitly recognizing that the degradation curve is not merely technical. It becomes organizational the moment probabilistic systems are allowed to act inside deterministic infrastructure. It’s also the reason Anthropic’s new Code Review offering is structured so carefully.

For me, this lands in a very specific context. My own deeper interest in AI was sharpened in 2023 after then-AWS CEO Adam Selipsky spoke at Collision in Toronto and framed generative AI as a force that would effectively reinvent software and reshape the application layer. Recent months have demonstrated the directional accurate foresight of that prediction. Selipsky later described AI as a coming “reinvention” for almost every application, and by early 2024 AWS was telling Axios that more than 10,000 organizations were already building with Bedrock. That mattered because AWS was not making a marginal argument about productivity tooling. It was making a thesis-level claim that generative AI would alter the economics and architecture of SaaS itself. The Amazon outage story now reads as the other side of that thesis: if AI really is moving into the software layer, then the software layer also becomes newly vulnerable to model error, ambiguous instructions, weak review processes, and compounded operational mistakes. 

That is also what makes Amazon such an important test case. AWS is not a startup running reckless demos. It is the company that has spent years selling the enterprise a vision of cloud reliability, abstraction, and controlled scale. It is also the company that has positioned Bedrock less as a single-model bet than as a managed marketplace for many models, including Anthropic, OpenAI, Qwen, Mistral, and others. Even Amazon’s own AI strategy reflects that pluralism. Bedrock’s supported model list is now fundamentally multi-provider, while Amazon’s Nova family, launched in late 2024 and expanded since, has been marketed primarily on price-performance and enterprise deployability rather than uncontested frontier leadership. Reuters reported in December that Amazon had struggled to gain broad acceptance for Nova against ChatGPT, Claude, and Gemini, even as AWS kept growing on the back of infrastructure and AI demand. In other words, Amazon’s strongest AI position has not been “our models are clearly best.” It has been “we are the operating system for enterprise AI.” That makes an AI-linked internal outage more than embarrassing. It demonstrates disruption in the layer of trust AWS is trying to sell. 

This is where the comparison to Salesforce’s bets and retrenchment becomes useful. Salesforce has spent the past year telling investors that AI is driving major efficiency gains, while retrenching in ways that make clear the labor and product implications are already here. Reuters reported in early 2025 that Salesforce was cutting more than 1,000 roles while simultaneously hiring to sell AI products, and later reported Marc Benioff’s statement that Salesforce had cut 4,000 customer support jobs due to AI. The company has also continued to promote Agentforce aggressively even as questions linger about monetization and deployment quality. Amazon and Salesforce are therefore running two different versions of the same enterprise experiment. Salesforce shows what happens when vendors use AI to compress labor and reorganize the go-to-market model. Amazon shows what happens when AI moves closer to the operational core and starts touching production systems with real blast radius. One story is about workforce substitution. The other is about systems risk. Together they point to the same conclusion: enterprise AI is leaving the “assistant” phase and entering the “control surface” phase. 

That shift has implications far beyond Amazon. For the last two years, much of enterprise AI marketing has implied that the main constraint was capability: get the model good enough, wire it into workflows, and productivity follows. But the Amazon episode suggests the real bottleneck is increasingly institutional rather than purely technical. It is review discipline, change management, permissions, rollback design. It is understanding when an AI suggestion is harmless autocomplete and when it is a latent infrastructure event waiting to happen. The more agentic the tooling becomes, the less useful old “copilot” metaphors are. The real bottleneck is no longer model capability. It is the institutional muscle built for deterministic software that haven’t yet rebuilt for probabilistic systems.

That is why Anthropic’s comparatively restrained posture looks more significant than some of the louder autonomy marketing elsewhere. Anthropic’s Claude Code documentation states that recurring /loop tasks automatically expire after three days, explicitly to bound how long a forgotten loop can run. Anthropic has also emphasized checkpoints for autonomous operation, published research showing that Claude Code still stops frequently to ask for clarification, and documented that code entering shared Anthropic codebases is subject to manual pull-request review. Even in its own safety reporting around Claude Opus 4.6, Anthropic highlights human review before integration into shared code. That does not make the system risk-free. Code Review involves multiple agents reviewing only the highest priority fixes without the autonomy or authority to approve fixes themselves. This shows a company trying, at least in its own documentation and deployment posture, to separate useful autonomy from unchecked autonomy. At this stage of enterprise AI, that distinction matters enormously. 

The larger lesson for enterprise buyers is sobering. In some ways, Amazon is attempting to establish the foundational element of Western AI in a way that is also occurring at the moment in China with seemingly more success. This is a major setback to those ambitions.The winners in this next phase will be the vendors that build the best guardrails around model fallibility, ambiguity, and cumulative error. Amazon’s internal meeting is a signal that even the cloud giant that helped normalize machine-scale abstraction is now confronting the cost of probabilistic software touching deterministic systems. That is the maturation of it. AI failures are occurring because organizations are using it to do consequential things, and discovering that the governance to match that consequentiality does not yet exist.