AI Is Changing Diagnostics and Test Interpretation, Despite Hallucinations
Doctors note progress in GPT-5 for advisory support, but warn: current neural networks make >50% errors when interpreting tests.
Artificial intelligence in medicine: a doctor's assistant or a risk to the patient?
Introduction
Artificial intelligence has firmly entered medical practice. Neural networks already read X-rays, write epicrises, and help interpret CT, MRI, and ECG results. It seems that the future described in science fiction novels has arrived: algorithms promise to speed up diagnostics, reduce the burden on doctors, and make medicine more accessible.
However, along with the first successes come the first questions. How much can you trust an AI assistant? What happens when a modern language model takes on diagnosis? In 2026, a series of high-profile studies provided an alarming, though not final, answer to these questions. The breakthrough capabilities of AI in "routine" tasks are combined with shocking incompetence in the main thing—clinical thinking.
Event Details and Timeline
First-Hand Data: Failure in 80% of Cases
In April 2026, the authoritative journal JAMA Network Open published the results of a large-scale study conducted by specialists at Mass General Brigham, one of the largest scientific and medical systems in the United States.
The scientists tested 21 modern language models (including GPT-5, Gemini 3.0 Flash, and Grok 4) on 29 standardized clinical scenarios. The evaluation covered all stages of clinical thinking: from collecting complaints to the final diagnosis. The results were paradoxical.
On one hand, when the models were provided with a complete clinical picture (test results, imaging data), they demonstrated impressive accuracy, making the correct final diagnosis in more than 90% of cases. This creates an illusion of AI's power.
On the other hand—and this is the most important—at the stage of initial differential diagnosis, when information is still scarce and the doctor has to consider possible options, all models suffered a crushing failure. They failed to formulate a correct differential diagnosis in more than 80% of cases.
"These models are great at naming the final diagnosis when the data is already complete, but they have serious difficulties at the initial, open-ended stage when information is still scarce," explained Aria Rao, lead author of the study.
Google AdInline article slot
Hallucinations and Systemic Bias
In parallel, in April 2026, another alarming study was published in the journal npj Digital Medicine (Nature), focusing on a detailed analysis of GPT-5's performance. The scientists evaluated not only accuracy but also the model's safety.
The findings were disheartening. First, the rate of "hallucinations" (when the model confidently outputs false information, mistaking fiction for fact) increased. When processing clinical scenarios with deliberately introduced errors, GPT-5 "swallowed" false data in 65% of cases (compared to 53% for GPT-4o).
Second, the model demonstrated clear sociodemographic bias. With identical symptoms, patients from LGBTQ+ communities, homeless individuals, and ethnic minorities received recommendations for unjustified psychiatric hospitalization in 100% of cases.
Impact and Significance
For Healthcare: A New Tool with Old Risks
These studies deal a serious blow to the concept of the "AI diagnostician." They show that modern algorithms lack clinical thinking—they cannot reason under uncertainty, weigh probabilities, or doubt. As Dr. Mark Succi noted, differential diagnosis is "the art of medicine" that AI is not yet able to replicate.
In practice, this means that patients and doctors who trust the "opinion" of a neural network in the early stages risk going down the wrong path. An incorrect preliminary diagnosis can lead to:
- Prescribing unnecessary and traumatic procedures
- Delaying critically important treatment
- Increasing patient anxiety ("cancer" instead of "mole")
As study co-author Mark Succi warns: "Even if we eventually arrive at the correct answer, an incorrect differential diagnosis can lead to delays in treatment, unnecessary procedures with complications, and high costs."
For Doctors: A Reason to Reassess Their Role
At the same time, it is clear that AI successfully handles routine analytics. In medical imaging (image interpretation) and pathology (analysis of histological specimens), neural networks show high results and are already being implemented in practice.
Thus, the main takeaway of 2026 is a clear division of responsibilities: AI can be a lightning-fast calculator and assistant, but decisions and diagnoses must be made exclusively by humans. At the current stage, artificial intelligence does not replace the doctor, but only provides a "second opinion."
Reaction of Key Players
The professional community's reaction to the published data was immediate.
Researchers at Mass General Brigham urge caution and a rejection of "marketing illusions." "Despite constant progress, large general-purpose language models are not yet ready for autonomous use in the clinic," said Mark Succi. The researchers emphasize that modern AI lacks the clinical reasoning mechanisms necessary for safe operation in the healthcare system.
Russian experts are also actively commenting on the situation. Marina Lyashenko, Vice President of the Insurance House VSK, emphasizes the key principle: "trust but verify." In her opinion, critical thinking and professional oversight by the doctor remain key decision-making factors.
Yevgeny Mukhametshin, Deputy Chief Physician of the federal optics chain "Schastlivy Vzglyad," adds a regulatory aspect: for safe implementation of AI in medicine, clear frameworks are needed—certification of digital solutions, transparency of algorithms, and mandatory indication that AI use is advisory in nature.
Interestingly, the technologists themselves do not dispute these findings. The study showed that a simple "mitigation prompt" reduces GPT-5's hallucination rate from a catastrophic 65% to an acceptable 7.67%. The problem is not that AI is "stupid," but that it is being used incorrectly by default.
Forecast and Conclusions
Where the Industry Is Heading
The main conclusion of 2026 is harsh but sobering: selling language models as full-fledged "clinical diagnosticians" is premature and dangerous for now.
Nevertheless, AI has a future in medicine, and it lies not in replacing the doctor, but in augmenting them. The most likely scenarios for the coming years:
- Strict regulation. The market entry of AI systems must be accompanied by mandatory certification and oversight similar to medical devices. Transparency of algorithms (fighting "black boxes") will become a mandatory requirement.
- "Protected" environments. Medical AI systems will operate within limited frameworks—not as open chatbots, but as narrow-purpose tools for a specific task (e.g., detecting tumors on CT scans).
- Assistant, not replacement. The role of AI will be reduced to routine analytics, checking the correctness of filling out charts, and quickly searching databases, leaving complex diagnostic reasoning to humans.
As one of the study's reviewers rightly noted, even if technologies learn to correctly answer most queries, a statistically significant volume of errors will remain. AI "hallucinations" in a field where the cost of error is a patient's life are unacceptable. Therefore, despite the "breakthrough" in data processing, diagnosis will remain a human prerogative for a long time.
— Editorial Team
