Google is piloting what it calls the first double-blind evaluation of a proprietary frontier-class AI model. A Gemini Flash Lite model will be tested against confidential benchmarks inside a privacy-preserving environment, with neither the model developer nor the outside evaluator receiving the other side’s protected material.
The project involves the Singapore AI Safety Institute, OpenMined, AVERI and MLCommons. Its goal is to reduce benchmark contamination, which happens when test questions enter training or development workflows and make strong scores less meaningful. Cryptographic controls keep evaluation prompts inside a protected “box” while still allowing the model’s performance to be measured.
The pilot does not prove that every benchmark result is reliable or that the model is safe. It tests a process for preserving the independence of external evaluations while companies keep model weights and system details private. If the approach works at scale, safety institutes could run more credible checks without forcing either side to disclose its most sensitive assets.