Microsoft’s multi-agent Diagnostic Orchestrator reached about 83% accuracy on difficult cases drawn from The New England Journal of Medicine, compared with roughly 36% for generalist physicians allowed to use references and 22% for those without them.
The system uses several AI agents to review symptoms and medical history, choose tests and build a diagnosis, resembling a panel of specialists. On a separate emergency-department dataset from Beth Israel Deaconess Medical Center, it scored 80% against an average 47% for physicians. Microsoft also estimated that it reached conclusions at lower test cost.
The peer-reviewed Science paper used GPT-5 and found that orchestration outperformed individual frontier models. However, it did not include several newer models, and every evaluation used past cases rather than live clinical care. Microsoft plans to open-source its SDBench benchmark, which it assembled for the work.
The central limitation is clinical validation. Microsoft intends to begin hospital testing with Beth Israel Deaconess, but has not committed to a launch schedule or commercial product. Real patients, incomplete records and treatment consequences introduce conditions that retrospective benchmarks cannot reproduce, so the results do not justify replacing doctors.