Hello, this is Tak@. As a system integrator, I usually help clients solve complex problems with IT and create new value. As someone passionate about developing web services with generative AI, the possibilities of AI surprise me daily.

Recently, I came across some data in the field of medical AI that overturned conventional wisdom, and I was incredibly excited.

It was the shocking fact that AI surpassed experienced doctors, achieving more than four times the correct diagnosis rate for the most complex diseases, while also reducing diagnostic costs. The very nature of healthcare, which supports our lives, might be about to undergo a massive transformation.

Is AI Changing the Norms of Medical Diagnosis?

In the medical world, accurate diagnosis is paramount as it directly impacts patients' lives and their quality of life. However, diagnosis requires specialized knowledge, extensive experience, and keen insight to find appropriate clues from a vast amount of medical information.

Moreover, this process is constantly time-sensitive, and patient financial burdens must also be considered.

In recent years, while global medical demand has expanded, healthcare costs continue to rise, leaving many people facing situations where they cannot access high-quality medical care. Against this backdrop, society has a strong demand for more accurate, rapid, and efficient diagnostic methods.

It is Microsoft's medical AI system, MAI-DxO (Microsoft AI Diagnostic Orchestrator), that is showing the potential to meet this pressing need.

What is MAI-DxO: Replicating the Diagnostic Process with AI

MAI-DxO is a next-generation medical diagnostic support AI system developed by Microsoft's AI team. This system is not merely a simple tool that pulls medical knowledge from a database.

It aims to replicate the complex thought process of skilled doctors who come together to discuss difficult cases and arrive at a diagnosis.

Here's how it works specifically:

First, initial symptoms and basic patient information are presented. From there, MAI-DxO, just like a human doctor, asks itself questions like "What additional questions should be asked?" and "What tests should be ordered next?" and then refines its diagnostic hypotheses based on the results.

Finally, when it reaches a confident stage, it provides a definitive diagnosis. This series of self-directed diagnostic cycles, much like a human "thinking," is what sets MAI-DxO apart from conventional medical AI.

MAI-DxO aims not just for correct answers, but to deliver "outcomes that provide value," emphasizing diagnoses that are relevant in real clinical settings.

Diagnostic Accuracy Surpassing Experienced Doctors and Its Implications

To evaluate MAI-DxO's performance, the research team set a very stringent benchmark: actual records from "Clinicopathological Conferences (CPCs)" published in the New England Journal of Medicine (NEJM), a globally authoritative medical journal.

NEJM CPC cases are highly complex medical reasoning challenges, where diagnoses are extremely difficult and often require collaboration among multiple specialists to reach a final conclusion.

Unlike multiple-choice questions such as those in the "US Medical Licensing Examination (USMLE)" often used to evaluate traditional medical AI, NEJM cases follow a "Sequential Diagnosis" format, similar to clinical practice. This involves starting with fragmented information, requesting additional questions and tests as needed, and progressively collecting information to narrow down the diagnosis.

This is a more realistic and rigorous evaluation method that tests the AI's actual clinical reasoning ability, not just its knowledge base.

Under these challenging conditions, MAI-DxO correctly diagnosed up to 85.5% of NEJM cases. The impressiveness of this figure becomes clearer when compared to the average correct diagnosis rate of only 20% for a group of 21 experienced physicians performing diagnoses under the same conditions.

This calculates to MAI-DxO outperforming humans by more than four times.

I have witnessed numerous instances where AI surpassed humans in specific tasks, but I am genuinely shocked by such overwhelming results in the complex and life-critical field of medical diagnosis.

I was reminded of the importance of AI evolving at a speed far beyond our imagination, focusing not just on diagnostic results as mere "deliverables," but on the "value" that the diagnosis brings to the patient.

This also aligns with the "outcome-oriented" approach emphasized in modern project management.

Why Is MAI-DxO So Accurate? Delving into Its Ingenuity

The high diagnostic accuracy and cost-efficiency achieved by MAI-DxO stem not from relying on a single high-performance AI model, but from a unique approach called "orchestration," which links multiple AI models, and a strategy that meticulously mimics the specialized thought processes of doctors.

From a system design perspective, this is very interesting, and I feel it holds many clues for building complex modern systems.

A Virtual Team of Doctors for Meticulous Collaboration: The Art of System Linkage

One of MAI-DxO's most unique features is its mechanism, which functions like a "virtual team of doctors" where specialists from different fields gather to examine difficult cases.

This virtual team is formed by a single powerful language model (LM) playing five distinct "personas," each with a different role. Each persona provides specific functions and perspectives in the diagnostic process, functioning as a "system" that interacts and collaborates.

• Dr. Hypothesis: Constantly maintains up to the top three most probable disease candidates and logically updates their probabilities based on Bayesian theory whenever new patient information or test results come in. This allows the AI to consider multiple possibilities without losing diagnostic direction, promoting actions that account for diagnostic "uncertainty."
• Dr. Test-Chooser: Responsible for selecting up to three of the most effective and efficient tests to narrow down diagnostic hypotheses. It doesn't just order many tests, but determines which tests are most helpful to definitively rule out or confirm current hypotheses.
• Dr. Challenger: During the diagnostic process, rigorously checks whether the AI is falling into an "anchoring bias" by sticking to a particular hypothesis, or overlooking contradictory evidence. Like a lawyer conducting a cross-examination, it proposes weaknesses in the current diagnosis or suggests tests that could refute it, thereby improving diagnostic accuracy and mitigating risks.
• Dr. Stewardship: Promotes cost-conscious diagnosis. If tests have equivalent diagnostic value, it proposes cheaper alternatives or rejects expensive, unnecessary tests with poor cost-effectiveness. This enables the AI to make judgments that consider not only diagnostic accuracy but also the patient's financial burden.
• Dr. Checklist: Functions as an internal quality control officer. It quietly verifies that the names and formats of tests and information generated by the AI are correct, and that the overall reasoning process of the virtual doctor team is consistent, ensuring overall system reliability.

These specialized AIs internally collaborate, sometimes debating, to correct cognitive biases that human doctors can fall into, allowing for a more comprehensive and cost-effective diagnostic process.

I found this similar to my experience in system development, where complex problems are unravelled by simple questions. This collaborative approach is a major factor in MAI-DxO's high-precision diagnosis.

Structured Thinking Process and Pursuit of Cost-Effectiveness: The Secret to Adaptability

MAI-DxO not only collects information but also has the ability to estimate cumulative costs during diagnosis and operate within budget constraints. This allows the AI to make judgments that incorporate a cost-effectiveness perspective—"How much should be spent on this information gathering?"—rather than blindly pursuing every possibility and ordering tests.

This aligns with the concept of "tailoring" in project management, which means flexibly adjusting the approach according to the situation.

This design philosophy directly addresses the problem of "excessive diagnostic tests," a contributing factor to rising diagnostic costs in general medical settings. By performing necessary and sufficient information gathering with cost awareness, MAI-DxO achieves both diagnostic accuracy and economic efficiency.

In practice, while an off-the-shelf AI model (OpenAI's o3) achieved 78.6% accuracy at a cost of $7,850, MAI-DxO (integrated with o3) achieved 79.9% accuracy for $2,397. Furthermore, with settings aimed at higher accuracy, it reached 85.5% accuracy for $7,184.

These figures are an excellent example of how structured reasoning can resolve the trade-off between cost and accuracy.

MAI-DxO can also be said to implement "Hybrid" thinking, combining the benefits of both predictive and adaptive approaches. This enhances its ability to deal with "uncertainty" in the diagnostic process.

AI's Power Also Contributes to Cost Reduction

Modern healthcare systems face significant challenges worldwide due to high costs, driven by advancements in medical technology and an aging population. In the United States, in particular, nearly 20% of the gross domestic product (GDP) is spent on healthcare, with up to 25% of that estimated to be wasted.

In such a situation, MAI-DxO's potential for cost reduction offers great hope for the medical field. This will be an important clue for utilizing limited medical resources more effectively.

AI's Smart Decisions Reduce Unnecessary Tests

MAI-DxO, with its advanced diagnostic capabilities, not only improves diagnostic accuracy but also has the ability to significantly reduce unnecessary tests. Conventional AI models, and sometimes even humans, tend to order many tests "just in case" to rule out every conceivable possibility.

However, MAI-DxO, through the "Dr. Stewardship" function of its virtual medical team, focuses only on tests that are truly necessary and provide high informational value.

This smart decision-making allowed MAI-DxO to reduce costs by an average of 20% compared to diagnoses made by experienced physicians. Furthermore, it succeeded in reducing diagnostic costs by up to 70% compared to certain off-the-shelf AI models.

This also has the potential to reduce diagnostic delays, patient discomfort, and risks associated with unnecessary medical procedures, giving it meaning beyond mere cost reduction. AI demonstrates the potential to improve "Quality" in healthcare while simultaneously reducing "Cost," truly offering two birds with one stone.

The Value of a Design Philosophy that Emphasizes Cost-Effectiveness

The MAI-DxO development team built the system based on a design philosophy that emphasizes cost-effectiveness throughout the entire process, not just the AI providing correct diagnostic results.

To prevent situations where the AI system orders tests without limit or cost awareness, it can be operated with clear budget constraints.

This approach of pursuing cost-effectiveness will be especially valuable in regions with underdeveloped medical infrastructure or environments with insufficient medical access.

AI systems like MAI-DxO show the potential to contribute to achieving two seemingly conflicting yet crucial goals: providing high-quality medical care while simultaneously reducing economic burdens.

I am confident that this is a very positive step towards realizing a more equitable and sustainable healthcare system.

Furthermore, this is also deeply related to the concept of "Value Realization" in modern project management. It emphasizes not just completing a project, but what concrete value it brings to the organization and society.

Concrete Example: Moments When AI Shows "Intelligence"

A major strength of MAI-DxO is that its diagnostic capability is universal and not dependent on a specific AI model. In fact, when MAI-DxO was applied to various foundational models (language models) such as GPT, Llama, Claude, Gemini, Grok, and DeepSeek, it improved diagnostic accuracy by an average of 11% across all models.

This means that medical systems can leverage a wide range of technological assets and improve overall diagnostic capabilities without being tied to the performance of a specific AI model. As a system integrator, I consider this "model-agnostic versatility" a very important point for system implementation and operation.

Let's look at a specific diagnostic example.

In one complex case, a patient was hospitalized for alcohol withdrawal symptoms but had actually accidentally ingested hand sanitizer. The conventional off-the-shelf AI model (OpenAI's o3) became fixated on the patient's initial symptoms and biased towards a hypothesis of "antibiotic poisoning."

As a result, it repeatedly ordered unnecessary and costly imaging tests, such as expensive brain MRIs and EEGs, and ultimately delivered an incorrect diagnosis. The diagnostic cost amounted to $3,431.

In contrast, MAI-DxO, by having its virtual medical team collaborate throughout the diagnostic process, yielded completely different results.

First, Dr. Hypothesis pointed out that the possibility of toxic exposure during hospitalization should be considered early on. Then, Dr. Stewardship urged the team to consider cheaper and more direct ways to obtain information before ordering expensive tests.

As a result, the AI asked the patient a simple yet pertinent question: "Did you ingest any hand sanitizer?"

This question led to the crucial information that the patient had accidentally ingested hand sanitizer, which then guided the process towards a targeted toxic alcohol test (confirming elevated acetone levels).

Ultimately, MAI-DxO was able to accurately diagnose this complex case at a very low cost of just $795.

This example demonstrates how important it is for AI to not only possess vast knowledge but also to be able to choose appropriate questions and flexibly change its thinking direction based on the situation, just like a human.

This is an indispensable element for AI to be truly "intelligent" and "practical." From this detailed case, I myself have once again deeply felt the importance of "contextual understanding" and "decision-making processes" in AI design.

It seems to me that modern system design philosophy, where systems should not just execute requested processes but also have the ability to "Recognize, Evaluate, and Respond" to situations, is embodied here.

Discussion: The Future Woven by Medicine and AI

The Relationship Between AI and Doctors: Augmenting Human Capabilities, Not Replacing Them

MAI-DxO's remarkable achievements might fuel the common anxiety that "AI will take away doctors' jobs." However, Microsoft's AI team has expressed a very clear view on this point.

They strongly assert that AI will not completely replace doctors; rather, it is a powerful tool that will "augment" their capabilities.

The field of medicine is incredibly vast and deep, making it practically impossible for one person to cover all specialized knowledge and experience and perfectly diagnose complex cases. While skilled general practitioners handle a wide range of cases, specialists possess deep knowledge specific to certain diseases or organs.

AI possesses unique characteristics that are difficult for humans, combining both "broad knowledge (Generalist)" and "deep expertise (Specialist)." In other words, AI like MAI-DxO can cover diverse cases like a general practitioner and simultaneously support detailed and advanced medical reasoning like a specialist.

A doctor's role extends beyond diagnosis. They build trust with patients and their families, show empathy, listen to anxieties, and make human decisions in ambiguous situations, performing many crucial roles beyond diagnosis.

AI will complement these uniquely human roles, enabling doctors to focus on more "human-centered care" and "complex judgments" through early disease detection, personalized treatment plans, and automation of routine tasks. This will reduce the burden on doctors and improve the overall quality of medical care.

Furthermore, the evolution of AI also relates to the concept of "leadership" in project management. By supporting the diagnostic process, AI will enable doctors to gain a broader perspective and exercise "leadership" in managing complex projects (the overall patient treatment).

The Future of AI and the Evolution of Healthcare: Equal Access and Adaptability

Such advancements in AI will bring immeasurable changes to the medical field. Patients may gain a deeper understanding of their health conditions through AI and improve their self-management capabilities.

Moreover, doctors will be able to handle complex cases that were previously challenging with greater confidence by leveraging advanced diagnostic support systems like MAI-DxO.

Particularly in regions with underdeveloped medical infrastructure and a shortage of doctors or medical resources, cost-effective AI systems like MAI-DxO have the potential to significantly expand access to high-quality medical services and improve healthcare equity.

For example, a future where simple consumer-向けの diagnostic tools, accessible via smartphones, allow people to take the first step in diagnosis from home, something previously only possible by visiting a hospital, is not just a dream.

In this rapidly changing era, it is crucial for medical AI to possess "Adaptability and Resiliency."

MAI-DxO's design demonstrates "adaptability" by flexibly responding to new information and changes in circumstances, and adjusting its diagnostic process. This will be a strength in dealing with unexpected situations and complexities faced in medical settings.

Challenges and Future Outlook: The Path to Practical Application and Expectations from SIers

Despite MAI-DxO's astonishing achievements, there are several significant challenges that must be overcome before this system can be widely used in actual clinical settings.

First and foremost, the most important aspect is the thorough assurance of safety and reliability.

This research is merely an initial stage of concept validation, and rigorous safety testing, verification through large-scale clinical trials, and strict approval processes by national regulatory authorities are essential. Since healthcare is a field concerning human lives, meticulous measures are required to minimize the risks posed by potential AI misdiagnosis.

Next, since NEJM cases are highly complex and specialized, further verification and adjustments are needed to see how MAI-DxO performs against more common and routine medical conditions (e.g., a common cold or minor sprain).

Furthermore, during diagnosis, in addition to test costs, factors such as patient discomfort, waiting times for test results, the types of tests available at the current location, and even insurance coverage, which are economic and ethical constraints, must be considered. The extent to which AI can appropriately judge these multifaceted elements will be a future research topic.

However, from my perspective as a system integrator, it is precisely because of these challenges that I find great appeal in the process of AI being implemented as a technology that truly contributes to society.

I strongly feel the desire to continue exploring and observing the development of technology and how it connects with the complex needs of society, especially in the crucial field of medicine.

Nevertheless, the "Model-Agnostic Orchestrator" system design adopted by MAI-DxO offers a significant advantage: as high-performance AI models from various companies like OpenAI and Google emerge one after another, healthcare systems can flexibly incorporate the latest and best AI models without being tied to a specific vendor's technology.

This is a very sensible approach when considering the long-term evolution and stable operation of medical AI. I have great expectations that AI will be a "catalyst for Change" in shaping the future of healthcare and will support us in realizing our "Envisioned Future State."

Conclusion: A New Form of Healthcare Envisioned by AI

The fact that AI has demonstrated a higher correct diagnosis rate and cost-effectiveness than experienced doctors in complex medical diagnosis holds the potential to significantly change our perception of healthcare.

AI systems like MAI-DxO do not merely mimic human intelligence; rather, they function as powerful tools that maximize its potential and create new forms of healthcare. This directly overlaps with the goal of building a "Value Delivery System" in modern project management.

By improving diagnostic accuracy and reducing unnecessary medical expenses, AI will enable doctors to focus on areas that AI cannot replicate, such as patient dialogue, empathy, and more complex, human-centric decision-making.

This should alleviate the burden on healthcare professionals and be a major step towards realizing higher quality, more accessible, and sustainable healthcare services for everyone. AI will bring "System Thinking" to the medical field, helping to reveal previously obscure interactions.

So, if an AI like MAI-DxO were introduced into your local healthcare setting, what kind of changes do you think it would bring? And in the future that these changes bring, how should we collaborate with AI and leverage its potential for our health and society?

I myself intend to continue contributing to solving various social issues through the development of tools utilizing generative AI. Let's imagine and create the future that technology brings, together.

Follow me!