Scaling isn’t enough

In July 2017, François Chollet, then a Google Brain engineer best known as the author of Keras, published two essays on the Keras blog: “The Limitations of Deep Learning” on July 17 and “The Future of Deep Learning” the following day. Together they ran perhaps 8,000 words. They were not formal papers. They circulated quietly inside the field and barely outside it.

Read nine years later, they hold up better than most commentary written before the LLM era.

The diagnosis was specific. Chollet argued that deep neural networks, as then conceived, performed what he called “local generalization.” They were extraordinary at interpolating within the distribution of their training data and weak at extrapolating beyond it. The future of the field, he argued, would not lie in scaling the same architectures larger but in combining what he called a “blend of formal algorithmic modules” for reasoning and abstraction with “geometric modules” for pattern recognition. The combination covered a family of approaches that would later be called neuro-symbolic, hybrid, or program-synthesis-augmented.

The word “transformer” does not appear in either essay. Ashish Vaswani and seven coauthors had introduced the architecture in the June 12, 2017 preprint “Attention Is All You Need.” Chollet’s argument addressed the broader limits of deep learning and proposed combining learned representations with explicit algorithmic reasoning.

In 2019 he formalized the position in “On the Measure of Intelligence,” the paper that introduced the Abstraction and Reasoning Corpus. The paper distinguished between skill, performance on a specific task, and intelligence, which Chollet defined as “skill-acquisition efficiency,” the rate at which an agent could acquire new skills from limited data and prior knowledge.

For the next five years, ARC was the benchmark frontier language models could barely move on. GPT-3, GPT-4, Claude 3 Opus, the original Gemini lineup — all of them performed somewhere between essentially zero and the low single digits. The result was used, by Chollet and others, as evidence that the LLM trajectory was not, in fact, on a curve toward general intelligence.

Then in December 2024, OpenAI announced o3-preview results, and the curve broke.

ARC Prize reported 75.7 percent on ARC-AGI-1’s Semi-Private evaluation at low compute and 87.5 percent at high compute. Chollet’s reaction on X was to call it a “significant breakthrough.” The systems had moved by something the o-series had been built to do, which was to spend test-time compute searching over its own reasoning. By the conventional reading of Chollet’s 2017 essays, this should not have been the path. On that reading, deeper, larger models, even with chain-of-thought reasoning, would still be doing local generalization.

The diagnosis, that pure scaling of base LLMs would not produce systems that generalize like humans, has aged well. Whether the o-series contradicted it or illustrated its limits is contested, including by Chollet, who in September 2026 described o3 as a qualitative leap to “solid fluid intelligence.” The prediction, that the LLM trajectory would plateau and that hybrid architectures would be required to make further progress on something like ARC, has not yet been confirmed. OpenAI cleared ARC-AGI-1 with o3, a scaled reasoning model, not the hybrid the 2017 essays described.

Chollet has acknowledged this in public. Calling o3 a breakthrough in December 2024 was not a hedge. It was a concession that the curve had moved further than the prediction implied it would. He has not conceded the prediction outright. In September 2026 he said his views had shifted after both o3 and GPT-6 Astra, calling Astra a “step-function change” and a “major breakthrough,” without abandoning his view that scaling base models alone does not produce fluid intelligence. His position had been that o3-class progress on ARC-AGI-1 reflected a specific kind of scaffolding around base LLMs and that the harder versions of the benchmark, ARC-AGI-2 and ARC-AGI-3, would continue to expose the limits the original diagnosis identified.

The evidence has moved. On ARC-AGI-2’s Semi-Private set, production o3 at medium reasoning scored 2.9 percent in April 2025, and Anthropic’s Opus 4.5 (Thinking, 64K) reached 37.6 percent by December 2025. ARC-AGI-3, whose environments human testers had all solved, opened in March 2026 with frontier AI at 0.51 percent on its action-efficiency score; on September 2, 2026, ARC Prize reported GPT-6 Astra at 62.7 percent on its Semi-Private set under the Standard harness at max reasoning effort.

On this record, Chollet’s diagnosis was sharper than his prediction. The diagnosis identified a real and persistent gap. The prediction said that the gap would not close without architectural change. Astra’s ARC-AGI-3 result is the strongest pressure on that prediction yet; whether it holds turns on what ARC-AGI-4, which Chollet said in September 2026 is in development for the first quarter of 2027, exposes.

The other major AGI skeptic in the field, Yann LeCun, agrees with Chollet on the diagnosis and disagrees on the remedy. LeCun has spent the past several years arguing that LLMs are missing world models, that text alone cannot produce the kind of grounded prediction that intelligence requires, and that a different architecture — JEPA, the Joint Embedding Predictive Architecture — would be the path. Chollet’s bet is different. Where LeCun emphasizes representational grounding and prediction, Chollet emphasizes program synthesis: the ability to compose discrete symbolic operations under neural guidance, which his 2017 essays sketched and which Ndea, founded in January 2025, is now pursuing under the explicit banner of merging deep learning with program synthesis.

Gary Marcus, the third name in the usual cohort, has the diagnosis without the constructive bet and without a benchmark of his own. He has been more publicly combative than either Chollet or LeCun. His best-known line from the week of o3’s ARC result, “goal post shifting,” was aimed at readers who said o3 refuted his own “wall” prediction, not at ARC. The asymmetry is structural: Marcus does not run a lab racing against ARC, while Chollet has both built the test and named the alternative.

Sam Altman runs the company that has, on present evidence, moved ARC-AGI-1 the furthest. Dario Amodei runs the company whose Opus 4.5 (Thinking, 64K) reached 37.6 percent on ARC-AGI-2’s Semi-Private set in late 2025. LeCun argues for a different architecture from AMI Labs, the startup he founded after leaving Meta at the end of 2025. Marcus argues from outside any architecture. Chollet argues for the same constructive thesis as in 2017, now from inside Ndea, with the additional commitment that the benchmark by which the thesis will be judged is also his.

Chollet wrote the 2017 essays while working at Google. In 2023 they looked partly wrong, as GPT-4 and Claude 2 surpassed what they anticipated, and in 2024 partly right again, as ARC-AGI exposed what the same models could not do. In 2026 their long-term hold on the field turns on whether frontier-model progress keeps moving across ARC variants and whether Ndea, or some lab pursuing a similar architectural bet, produces a system that registers on ARC by doing the thing the essays predicted would be required.

The diagnosis is the more durable side. The prediction is the side Chollet’s company exists to make true.