Microsoft researchers have developed an artificial intelligence tool that diagnoses patients step by step, the way doctors do, and it proved more than twice as accurate as general physicians at solving complex medical cases, according to a study published on Thursday in the journal Science. The work targets diagnosis itself — an area long identified by patient-safety researchers as a persistent source of preventable harm and one of the most closely watched frontiers for medical AI.

Rather than processing all of a patient's information at once — the approach taken by most existing medical AI models, which examine symptoms, lab results, and imaging studies together to generate a diagnosis — the new system works sequentially. It relies on an “orchestrator” that organizes how the AI tackles a problem by employing a simulated panel of clinical experts collaborating to solve diagnostic cases, the report said.

With the orchestrator coordinating the steps toward a diagnosis, the model first decides which questions to ask about the patient's history and what findings matter in the physical exam. Based on what it learns, it determines which tests to order, evaluates the results, and requests further testing as appropriate. Throughout the process, it updates its working hypotheses and plans, the research leaders said in an interview.

Tasked with diagnosing more than 300 highly unusual and confusing medical cases, the model using the diagnostic orchestrator was more than twice as accurate at “working up” patients and determining the diagnosis as general physicians using textbooks and the internet. The tool also outperformed human physicians when presented with scenarios drawn from a major database of emergency room admissions.

The researchers noted that conventional assessments comparing older AI systems of this type to human doctors typically rely on multiple-choice questions. “AI has become very good at getting a 100% pass rate on those,” said Dr. Dominic King, corporate vice president of the health unit at Microsoft AI, whose team is developing the new technology. “But of course, that's not real-world. No patient ever came to see me with a nice summary.” The findings feed an ongoing debate in the AI field over whether strong exam-style benchmark scores translate into safe, useful performance in messy real-world clinical settings.

The study also found that diagnostic processes cost less when the orchestrator designs the work-ups than when human doctors do — a notable result, since test ordering is a recognized driver of health-care spending. “We're not just measuring the accuracy of the systems. We're also measuring how judiciously and carefully it goes through a diagnostic work-up,” said Harsha Nori, head of AI at Microsoft AI Health.

Next, in collaboration with Beth Israel Deaconess Medical Center in Boston, the researchers will test the tool's ability to triage patients in emergency department waiting rooms. That trial will indicate whether the lab results hold up in live clinical workflow, where triage decisions carry immediate stakes, and any move toward routine clinical use would draw attention to regulatory review, as AI tools that inform diagnoses typically fall under medical-device oversight in the United States.

— Reuters