Microsoft's Diagnostic AI Got 83% of Difficult Cases Right

Microsoft has published research on MAI-DxO, its diagnostic AI system, in the journal Science. The paper was published on October 22. According to the company, the system correctly diagnosed roughly 83% of patient cases that are considered difficult to crack. Microsoft also released SDBench, an evaluation tool that tests how an AI asks questions, orders tests, and works its way toward a diagnosis.

“Twice as good as doctors” - but what was compared to what?

In the same evaluation, generalist physicians were accurate 36% of the time when they could consult reference materials, and 22% when they could not. The “more than double” framing comes from comparing the AI’s 83% with the doctors’ 36% on difficult cases drawn from medical journals. In a separate evaluation based on data from US emergency rooms, the company said the AI scored 80% against the doctors’ 47%.

MAI-DxO is not a brand-new system. In 2025, Microsoft introduced a system in the same family and reported accuracy as high as 85.5% on difficult cases. What is new this time is the additional evaluation results and the Science paper.

The AI asks its own questions and picks its own tests

MAI-DxO assigns several AI agents the role of virtual physicians, has them propose candidate diagnoses, and then has them critique one another’s assessments. As new information arrives, they update their judgments. SDBench likewise withholds the full patient record at the outset. Information becomes available only when the system asks a question or orders a test, so the benchmark grades not just the final answer but the process of gathering information.

When choosing a test, the system weighs the expected cost against the likelihood that the result will help with the diagnosis. Microsoft said that even in a configuration tuned to prioritize cost, accuracy held at 81%. In that setting, it said, estimated testing costs came to less than half of the comparison baseline. Microsoft has released code for a subset of SDBench cases so other researchers can run the same evaluation.

Will it hold up in actual clinical care?

Google is also working on diagnostic AI that gathers information by questioning patients. Its AMIE system was used in a small study in which it spoke with patients ahead of real appointments and then passed a summary to their doctors. The MAI-DxO figures released here come from an evaluation in which the system works through past patient cases step by step.

When MAI-DxO will become available remains unclear. Microsoft has not disclosed availability dates or pricing for general users or healthcare institutions, nor a Korean launch date or plans for Korean-language support. The company said it will run a study involving real patients in partnership with a healthcare institution in Boston.

This research looks beyond raw accuracy to examine the questions asked, the tests chosen, and estimated testing costs when evaluating diagnostic AI. The next step is finding out whether it actually helps in a real clinical setting.