Recent claims about AI systems hacking companies, solving longstanding mathematics problems and approaching self-improvement deserve more scrutiny than their initial headlines received, two researchers argue in MIT Technology Review. Their central concern is that company framing often gives software human-like agency while obscuring ordinary engineering decisions and weak controls.
The authors point to security incidents that were described as models going rogue. Cybersecurity experts later emphasized failures to use established safeguards around the systems. They also note that mathematicians challenged the novelty and attribution of some highly publicized results after reviewing them more closely.
This does not show that language models lack useful capabilities. Coding and mathematics are especially attractive testing grounds because proposed answers can often be checked automatically. But that same property can encourage companies to optimize for impressive demonstrations and present them as evidence of broader intelligence.
The authors endorse a statement from mathematicians asking policymakers to consult domain experts rather than rely on corporate announcements. Their practical recommendation is to focus accountability on the organizations building and deploying AI, and to demand methods, evidence and independent review before treating a striking demonstration as a scientific breakthrough.