Doomsday, AI, and a Response to Dario Amodei

Must the AI frontier slow down? Perhaps. That’s not the right question. We should ask what, precisely, must slow; what risk that restraint would reduce; who could enforce it; and what would happen if careful firms paused while less careful actors did not.
Dario Amodei has published one of the most consequential essays yet written by a frontier AI company leader. He is candid, and he is reconsidering an earlier, more benign, position. But his argument is flawed: a model’s capability is not an application or an outcome.
He says the latest models can offer dramatically good or dramatically bad outcomes. On the one hand, AI may cure most major diseases within five to ten years. But it may also become an agent swarm capable of seizing the internet within six to twelve months.
Both are wrong. These extremes are oversimplifications and can undermine both AI’s benefits in many areas and our ability to regulate and monitor it appropriately. Simply saying “boo” and hoping everyone else is frightened isn’t an effective, comprehensive strategy for addressing a serious issue.
Any good or bad from an AI model only appears when that model is part of a system that enters the world through software, capital, organizations, machines, biological systems, and public institutions. Understanding this lets us unleash AI’s potential for good while also building the systems and processes to protect society.

The Application Layer

Artificial intelligence is a stack: energy, silicon, cloud, models, and applications. Each has its own economics, competitive dynamics, and challenges. Mistaking one layer for the whole industry causes confusion, misrepresentation, bad decisions, and misguided capital allocations. The infrastructure builders enable the platform; the application builders capture the value. The question is now, what value does all this deliver? Energy, silicon, cloud, and models only serve to deliver that product. There is a robust argument that we are at the beginning of an unprecedented value-creation curve. Built on the infrastructure and services provided by the other layers of the stack, the AI application layer will be globally transformative and disruptive. The constraints are imagination, execution, and the willingness to rebuild how work is done.

Capturing AI

AI models produce raw intelligence. They generate tokens. But tokens are an intermediate good, not a finished product. What customers actually pay for is legal work completed, code shipped, claims processed, research synthesized, and decisions supported.

They pay for refined output.

Attention has focused on the infrastructure layer — the frontier labs, the compute stack, and the data centers. That attention is not misplaced, but it overlooks a structural shift already underway. Once you understand the model as an intermediate good rather than the end product, the center of gravity moves. The decisive question is no longer who can produce intelligence, but who can turn it into something usable, trusted, repeatable, and economically defensible.

In other words, who can refine it into a usable product?

At the base of the chain sit the token producers — OpenAI, Anthropic, Google DeepMind, Meta, DeepSeek, and Qwen. They produce raw capability. This layer is expensive to build, technically formidable, and still moving fast. But crude oil is not gasoline.

Enterprises and consumers pay for gasoline.