A new benchmark called DecoyCount tests whether multimodal AI systems can count real objects without being fooled by reflections, pictures, screens and other visual decoys. The accompanying paper reports that every tested frontier model scored below 50 percent, with GPT-6 Astra leading that group at 43.17 percent.
The benchmark was co-released by visual-model startup Elorian and data company Mercor. It uses 1,660 items spanning seven failure mechanisms intended to separate basic object recognition from physical reasoning. A model must understand, for example, that one apple beside a mirror is still one physical apple rather than two.
Elorian says its new Glimpse model reached 50.48 percent on an internal run and that evaluation images and text were withheld from training. That is not a fully equivalent comparison: the six models in the paper were averaged across three runs under reported settings, while Glimpse’s number came from one run supplied by the company.
The result is therefore evidence of a difficult test, not proof that one system has solved visual reasoning. Even the reported leader was wrong on nearly half the items. Independent reruns and analysis of contamination, scoring stability and real-world transfer will be needed before the benchmark can support broader capability claims.