Turing Award winner Richard Sutton is pushing back on a common reading of his influential “Bitter Lesson”: that adding enough compute to current methods will produce intelligence. In a new post, he says AI science still needs fundamental algorithmic ideas and is “far from finished.”
Sutton’s 2019 essay argued that general methods able to exploit growing computation eventually outperform systems packed with human expertise. His latest point is that the method still matters. He has criticized relying only on larger language models and synthetic data, arguing that the physical world contains far more complexity than a human-built simulation or a fixed text corpus can capture.
His preferred direction is agents with world models that keep learning from experience without destroying knowledge acquired earlier. That is a different problem from pretraining a model once and then freezing its parameters. Former OpenAI researcher Jerry Tworek similarly argues that some algorithms scale while others reach dead ends, and that transformer success encouraged researchers to treat one recipe as universal.
The position remains disputed. OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei argue that language-model scaling still has substantial room, pointing to continuing gains on mathematics and other tasks. Sutton is not claiming that computation has stopped helping. He is arguing that compute amplifies the learning process it is given, making better algorithms—not compute alone—the unresolved research bottleneck.