Sutton and the Bitter Lesson

“The Bitter Lesson,” which Richard Sutton published on his blog incompleteideas.net in March 2019, ran one paragraph as its central claim: “general methods that leverage computation are ultimately the most effective, and by a large margin.” Search and learning, Sutton argued, are the two methods that scale arbitrarily. The essay closed with a sentence that has since been printed onto investor decks and quoted on podcasts: “We want AI agents that can discover like we can, not which contain what we have discovered.”

The line is now a creed for those who read it as a license to scale the transformer. Sutton reads it as a warning the field walked straight past. Six years after the essay, in a body of writing that runs from a research blueprint through a peer-reviewed result in Nature to a manifesto co-written with DeepMind’s David Silver, he has been making one argument: that pouring human knowledge into a model is the recurring mistake the 2019 essay named, and that pretraining a next-token predictor on human text is its latest, largest instance. Sutton has believed for thirty years that scaling gets there. His quarrel is over what gets scaled: compute spent learning from experience, or compute spent absorbing what people have already written.

The essay never changed its mind. Its author has only said, more loudly each year, what he meant by it. In August 2026, on Sequoia’s Training Data podcast, he added a turn: language models are both a positive and a negative example of the lesson, systems that scaled computation and then ran into the limits of finite human data. Scaling them on synthetic data, he said, would be “a big mistake.”

The writings, as one argument

Sutton’s case is one argument unfolding across consecutive publications, and reading them in order is the clearest way to see how old it is.

“The Bitter Lesson” (2019) supplies the frame: across seventy years of AI research, methods that build in human knowledge are eventually overtaken by methods that scale search and learning. The Alberta Plan for AI Research, posted to arXiv in August 2022 by Sutton with Michael Bowling and Patrick Pilarski, turns the frame into a program: a twelve-step research blueprint for an agent that learns continually from its own stream of experience.

The 2024 Nature paper “Loss of plasticity in deep continual learning,” with Shibhansh Dohare and four other co-authors, supplies the empirical lever: standard deep networks, trained continually, silently degrade until they learn no better than shallow ones. The paper proposes a remedy, an algorithm called continual backpropagation, but the finding is the point. Continual learning, the thing Sutton says real intelligence requires, is precisely the thing today’s deep-learning stack cannot yet do without intervention. Then “Welcome to the Era of Experience,” circulated as a preprint chapter in April 2025 with David Silver, states the thesis plainly: “the knowledge extracted from human data is rapidly approaching a limit.” Imitation, the authors write, may reproduce many human capabilities to a competent level, but “this approach in isolation has not and likely cannot achieve superhuman intelligence.” The era of human data, they argue, has to give way to an era of experience.

Reinforcement learning, the field Sutton helped found at the University of Alberta and the subject of the textbook he wrote with Andrew Barto, has been his axis the whole way. An agent that acts, observes the consequence, and updates is the unit he has spent a career scaling. That commitment runs deep enough that in 2021 Sutton co-authored a paper titled “Reward is Enough,” arguing that the maximization of a single scalar reward, pursued in a rich environment, is sufficient on its own to produce the full range of abilities associated with intelligence — perception, language, memory, planning.

The claim is the inverse of the program the field’s largest labs now run: pretrain a large model on the written record, train it further with reinforcement learning, and spend more computation when it answers. Set against Sutton’s axis, a language model is a system trained to predict the next human-written token, which is to say a more elaborate way of pouring human prior knowledge into the machine, exactly the move the 2019 essay said the field keeps making and keeps losing with. The 2024 ACM Turing Award, announced in March 2025 and shared with Barto for “developing the conceptual and algorithmic foundations of reinforcement learning,” recognized the axis, not the detour. The corpus does not retract the 2019 essay. It is the essay’s author spending six years insisting the field read it the way he wrote it.

The Dwarkesh reading

The argument turned public, and combative, in a podcast studio. On September 26, 2025, Dwarkesh Patel released his interview with Sutton under the headline “Father of RL thinks LLMs are a dead-end.” That phrase is Patel’s, not Sutton’s: Sutton retweeted the episode neutrally as “a frank exchange of views,” and his actual position is more precise than the headline and harder to wave off as a generation-gap reflex. He does not argue that scaling has run out of road. He argues that the field is scaling the wrong thing.

Sutton put it most cleanly to Patel:

Things that used human knowledge were eventually superseded by things that just trained from experience and computation. … People get locked into the human knowledge approach… get their lunch eaten by the methods that are truly scalable. … The large language models are learning from training data. It’s not learning from experience.

His positive criterion for an intelligence runs three ways: it has a goal in the sense John McCarthy, the AI pioneer, used the term; it predicts the environmental consequences of its own actions; and it can be surprised when those predictions fail. A model trained to continue human text fails the second test, and so the third. “To mimic what people say is not really to build a model of the world at all,” Sutton said. Large language models “have the ability to predict what a person would say. They don’t have the ability to predict what will happen.”

When Patel pressed on reinforcement learning atop a language model reaching International Mathematical Olympiad gold-medal math, Sutton declined to treat it as a counterexample on his own terms: a string of math tokens is not an action that draws a real-world response, so the system “will not be surprised by what happens next.” The dividing line is what the compute is in contact with.

OaK: the constructive program

The harder question is the one a critique invites — if not the language-model paradigm, then what? Sutton’s answer has a name. The OaK architecture, “Options and Knowledge,” is the positive bet, and he frames it without modesty: in his 2025 talks it is “a vision of superintelligence from experience,” a different thing to scale.

OaK is a model-based reinforcement-learning agent that builds its own model of the world and improves it by acting. Its engine is an open-ended cycle Sutton abbreviates FC-STOMP — Feature Construction, then SubTask, Option, Model, Plan — by which the agent invents its own abstractions, sets itself subgoals, learns reusable skills, models their consequences, and plans with them, then constructs new features and runs the loop again. Nothing in the cycle depends on a corpus of human text.

Two supports hold it up. The first is continual learning, which is where the 2024 Nature loss-of-plasticity result does its work: OaK needs deep networks that keep learning indefinitely, and that capability does not yet reliably exist, which is why Sutton calls it the missing piece rather than a solved one. The second is the Big World Hypothesis: the world is, and always will be, vastly larger than any agent, so useful knowledge has to be approximated at runtime, in contact with the environment, not packed in at design time.

Sutton presented OaK as an invited talk at the Reinforcement Learning Conference in August 2025 and again at NeurIPS in December. Its through-line is the one the Alberta Plan drew in 2022: an agent that learns continually from its own experience. Patel, summarizing Sutton’s view after their interview, drew the conclusion: once such an agent exists, “we won’t need a special training phase,” and the new paradigm “will render our current approach with LLMs obsolete.” OaK is the constructive end of the argument the 2019 essay began.

What it means for the scaling debate

The reading Sutton is arguing against is the one that put his sentence on the slides in the first place. Dario Amodei, the Anthropic chief executive, holds the sharpest version of it on the record: on Patel’s podcast in February 2026 he said his “top level model of scaling is the same as it was in 2017,” and placed 90% confidence on a “country of geniuses in a datacenter” arriving within a decade. His model counts reinforcement learning on top of pretraining: “We’re seeing the same scaling in RL that we saw for pre-training.”

Sam Altman, the OpenAI chief executive, went further in his February 2025 essay “Three Observations,” writing that “the intelligence of an AI model roughly equals the log of the resources used to train and run it,” and that “you can spend arbitrary amounts of money and get continuous and predictable gains.” Both read “general methods that leverage computation” as: scale the pretrained transformer, with the reinforcement learning and inference-time computation built on it. Sutton’s reply is structural. The essay singled out two methods that scale — search and learning — against a long history of clever priors that turned out to be dead weight. The relevant axis, in his frame, is what kind of compute is being scaled, and pretraining on human text is the wrong kind however much of it you buy.

That is Sutton’s case. There are real reasons to doubt it. Patel’s own follow-up, “Some thoughts on the Sutton interview,” assembled them. Pretraining might be a fossil fuel — non-renewable, but enough to bootstrap a successor that learns from experience, the line Ilya Sutskever, the Safe Superintelligence co-founder, staked at NeurIPS 2024 when he called data “the fossil fuel of AI.” Imitation might be continuous with reinforcement learning, not opposed to it. Test-time fine-tuning might give a language model an LLM-native version of the continual learning Sutton says it lacks.

Demis Hassabis, then Google DeepMind’s chief executive, told the Axios AI+ Summit in December 2025 that “the scaling of the current systems, we must push that to the maximum… It could be the entirety of the AGI system” — though he pairs that with a demand for explicit world models and architectural breakthroughs Sutton would also insist on. What the keep-scaling reading hears from Sutton’s sentence is an instruction to make the model bigger. What Sutton hears is an agent that has experience, has goals, and can be surprised. The two readings run in the same direction for one more clause and then part, and the line they part on is what counts as learning: token-prediction loss over a fixed corpus, or the updates an agent makes from a stream of consequences. Anyone quoting the 2019 essay without the 2025 corpus, on Sutton’s account, is quoting a reading its author never held.

Sources

• “The Bitter Lesson,” Richard Sutton, March 13, 2019
• Dwarkesh Patel interview, September 26, 2025
• Patel follow-up, “Some thoughts on the Sutton interview”
• Silver and Sutton, “Welcome to the Era of Experience,” April 2025
• Silver, Singh, Precup, and Sutton, “Reward is Enough,” Artificial Intelligence, 2021
• “OaK” at NeurIPS 2025
• 2024 ACM Turing Award announcement
• Sutton X post on the Dwarkesh framing
• Dario Amodei, Dwarkesh Patel interview, February 13, 2026
• Sam Altman, “Three Observations,” February 9, 2025