Carmack and Sutton’s bet against simulator-only RL

John Carmack, the id Software co-founder and former Oculus CTO who has spent the last several years on AGI research, founded Keen Technologies in Dallas in August 2022, with $20 million in seed funding from Nat Friedman, Daniel Gross, Patrick Collison, Tobi Lütke, Sequoia, Capital Factory, and Jim Keller. TechCrunch’s coverage on August 19, 2022 called it Carmack’s pivot to artificial general intelligence. Sutton’s name was not yet attached. Carmack’s framing in a Dallas Innovates interview that month was characteristic: he believed the algorithm for AGI may be small enough that “one person could write” it — an existence-proof argument from the small information content of DNA.

On September 25, 2023, at the Alberta Machine Intelligence Institute in Edmonton, Carmack and Richard Sutton announced their formal partnership. Sutton retained his University of Alberta professorship, his Amii Fellow role, and his Canada CIFAR (Canadian Institute for Advanced Research) AI Chair. The ACM citation for his 2024 Turing Award, announced March 5, 2025, listed him also as a research scientist at Keen Technologies. Sutton’s role at Keen ended in June 2026, when he and Khurram Javed left; on July 13, Javed announced Oak Lab, the startup the two co-founded to pursue the same experiential-learning program.

Keen Technologies has remained small. As of Upper Bound 2025, Carmack’s keynote slide listed six researchers: Richard Sutton; Joseph Modayil; Khurram Javed, a Keen research scientist and Sutton’s former PhD student; John Carmack; and Gloria Kennickell and Lucas Nestler, both researchers. Keen raised $20 million in August 2022, in a round led by Nat Friedman and Daniel Gross. There is no Series B, no foundation-model release, no consumer product, no announced commercial roadmap. Carmack told D Magazine in November 2025: “We’re trying to learn fundamental things about architecture and learning that nobody knows right now.”

The stated goal, per the September 2023 partnership press release: develop a “genuine AI prototype by 2030… establishing, advancing and documenting AGI signs of life.”

The affirmative case Keen is making

The small team is itself part of the bet. Carmack’s Dallas Innovates founding aesthetic — that the algorithm for AGI may be small enough that “one person could write” it — is methodological, not just rhetorical. A research-only operation with no commercial roadmap (Keen’s 2025 roster listed six people) selects for first-principles work: long-horizon experiments on a small set of environments, a small handful of algorithms, full ownership of the codebase, no quarterly product cycle pulling effort sideways. The frontier-lab consensus has moved in the opposite direction over the same period — toward thousand-researcher organizations, multi-billion-dollar training runs, and product-team integration. Carmack’s bet is that on a research question whose answer no one currently has, organizational scale is a different kind of variable than compute scale.

The deliberate non-LLM choice is the methodological signal that follows from the same logic. Sutton’s own view of language models is less dismissive than Carmack’s. In August 2026 he called them a scientific breakthrough and put them at roughly a quarter of intelligence, faulting how they are frozen at deployment. Carmack’s Upper Bound 2025 keynote slide titled “Where I thought I was going” explicitly listed: “Not LLMs. LLMs can know everything without learning anything. Learn from experience, not an IID blender.” Keen is not an RL-on-top-of-LLMs operation in the OpenAI o-series mold. Pretrained transformer weights, in this frame, are not a useful prior for an agent meant to learn from environmental consequences; they are a different kind of artifact altogether.

The “Physical Atari” demo, shown at Upper Bound 2025 in May, is the existence proof. A robot points a real camera at a real Atari console television, controlling a servo-actuated joystick to play Atari games. The reinforcement-learning loop runs online, in real time, on an ASUS ROG Strix SCAR 16 laptop with a 16GB 4090 GPU. Carmack’s slide called it “Kind of a stunt, but novel AFAIK… We will Open Source.” Atari is a well-worn benchmark: DeepMind’s deep Q-network paper made it a landmark in 2015, learning 49 games from screen pixels and reaching a professional human tester’s level on many of them. The novelty is that the agent operates in physical reality, with camera lag, pixel noise, mechanical joystick latency, and lighting variation. Carmack’s framing slide read: “Reality is not a turn-based game.”

Khurram Javed, Sutton’s former PhD student and his Oak Lab co-founder, frames the program on his GitHub README: “I make systems that learn in real-time from experience… driven by the “Big World Hypothesis” (Javed and Sutton, 2024).” The “Big World Hypothesis” — that the world is, and always will be, vastly larger than the agent — supplies the theoretical justification for prioritizing runtime adaptation over design-time pre-loading.

What the bet commits to, and what it forecloses

The public artifacts that would normally accompany a frontier AI research program — a foundation model release, a benchmark sweep, a published scaling-law paper — had not appeared from Keen by mid-2026. The team has published peer-reviewed work in adjacent areas: “SwiftTD” (Javed, Sharifnassab, Sutton; RLC 2024) and “Reward Centering” (Naik, Wan, Tomar, Sutton; RLC 2024). The “loss of plasticity” paper appeared in Nature in August 2024 with Sutton as senior author. None of these are commercial artifacts.

Keen has not announced a customer, a product, a Series B, or a foundation-model release. The bull-case operations — Anthropic at $14 billion run-rate revenue as of February 12, 2026 (per Anthropic’s Series G press release) and, by the company’s count, above $30 billion by April 6, 2026, OpenAI’s hyperscaler-anchored compute commitments, the roughly $720–745 billion of combined 2026 capital spending that Microsoft, Alphabet, Amazon, and Meta guided to in July 2026 (total capex, each company’s own definition, Oracle excluded) — are running a different kind of bet. Keen is running on $20 million and a 2030 target.

The leading labs held the strongest results on every widely used benchmark through 2025–26, among them Google DeepMind’s IMO gold, certified on July 21, 2025. Gundlach and colleagues found that models in the top GPQA Diamond performance bin fell in price about 31-fold a year (“The Price of Progress,” arXiv 2511.23455, revised March 2026). Sam Altman’s February 9, 2025 “Three Observations” 10x-per-year cost claim has been broadly validated by Epoch AI, the AI-research analytics group, and by Gundlach and colleagues. Keen has not yet demonstrated artifacts that surpass simulator-trained Atari agents. Carmack’s bet is that the comparison is on the wrong axis.

Gary Marcus’s transformer-limits position would read the Keen direction as confirmation that the LLM paradigm is approaching limits and that hybrid or non-LLM approaches deserve serious investment. Marcus’s December 30, 2024 wager with Miles Brundage on a ten-task AGI bet — 10:1 odds: Marcus donates $2,000 if AI clears eight of ten by end of 2027, Brundage donates $20,000 if it does not, tracked on Metaculus question 31246 — is the active formal stake. Marcus’s framing of the August 2025 GPT-5 launch as more incremental than paradigm-shifting reads the GPT-5 router architecture as a signal that monolithic-model scaling has plateaued.

The succession framing

Sutton’s WAIC Shanghai keynote on September 11, 2025 argued that fears of AI bias, unemployment, and extinction are exaggerated and incentivized. The Dwarkesh interview crystallized his four-part argument: no single global organization can coordinate humanity; researchers will figure out intelligence; it will not stop at human-level intelligence; the most intelligent things gain resources and power. “Put all that together and it’s sort of inevitable.” The phrase “AI succession” is Sutton’s own; the WAIC Shanghai talk is posted on Sutton’s site.

Keen Technologies was the operational expression of this view, and Oak Lab, which Sutton co-founded after leaving Keen in June 2026, is its successor; neither is a hedged AGI lab. The view is that AGI is coming, that the path runs through experiential reinforcement learning rather than pretrained transformers, that the destination is a kind of intelligence that supersedes human intelligence rather than a kind that augments it, and that the appropriate response is to do the engineering work rather than to delay it.

Safety researchers run in a different direction: Yoshua Bengio, the Université de Montréal professor who founded Mila; Stuart Russell, the UC Berkeley computer scientist and author of Human Compatible; and, among the labs, Anthropic with its responsible-scaling framework. Sutton’s “AI succession” framing — at WAIC Shanghai, in the Dwarkesh interview — is in tension with this part of the field, and Sutton has not engaged the safety critique on the technical level on the public record.

The charitable reading is that Sutton’s WAIC framing puts the safety question on a different time scale than the Bengio/Russell program: he is arguing about the long-arc inevitability of human-to-AI succession; the safety researchers are arguing about near-term controllability of specific systems. The two sides are not yet asking the same question on the same horizon.

Keen listed six researchers in 2025 and has raised $20 million. It has no commercial roadmap, a 2030 target for AGI signs of life, and a deliberate refusal to use LLM pretraining as the substrate. The bet is that experiential reinforcement learning, scaled the way the bull case scales pretraining, gets to the same destination through a different door.

The scale of the bet against the rest of the field is asymmetric. In a November 25, 2025, interview with Dwarkesh Patel, Ilya Sutskever said that “based on what people say on Twitter, they spend more compute on RL than on pre-training at this point” — Sutskever’s own hedge attributing the claim to Twitter rather than first-hand observation. That compute is being spent inside an LLM substrate. Keen’s compute, what little of it the team commands, is being spent on agents in real environments. Whether the asymmetry resolves in the bull case’s favor through two or three more pretraining doublings, or in Sutton’s favor through a continual-learning architecture that the LLM stack does not yet have, is the open question.

Sources

• Carmack and Sutton partnership announcement, September 25, 2023
• TechCrunch, Carmack AGI Keen $20M raise, August 19, 2022
• Carmack Upper Bound 2025 slides
• D Magazine Carmack interview, November 2025
• ACM 2024 Turing Award announcement, March 5, 2025
• Sutton, Bowling, and Pilarski, “The Alberta Plan for AI Research,” arXiv 2208.11173
• Anthropic Series G announcement, February 12, 2026
• Sutskever on Dwarkesh Patel, November 25, 2025
• Marcus and Brundage wager, December 30, 2024
• Mnih et al., “Human-level control through deep reinforcement learning,” Nature, February 25, 2015