Science: AI Surpasses Physicians in Clinical Reasoning on Real Emergency Cases for the First Time
A study in Science showed that OpenAI's o1 large language model outperformed board-certified physicians in diagnosis, especially in emergency triage with incomplete data. In a double-blind test on real patient histories, AI accuracy was higher, and physicians could not distinguish its conclusions from human ones.
The article in Science from April 30, 2026, is not just a benchmark. It is a legal precedent wrapped in a scientific protocol. Broder and Manrai's group did not just prove that a machine diagnoses better than a doctor; they created an evidence base that, within 5–7 years, will change the standards of medical malpractice and—ironically—make the use of AI by physicians not a right but a duty.
The Core: What Is Really Happening
The study, published in Science, compared OpenAI o1 with hundreds of live physicians in a blind test on real emergency cases from Beth Israel Deaconess Medical Center. The result: at the triage stage, when data is minimal, the model's accuracy was 67.1% versus 55.3% and 50.0% for two attending physicians. Reviewers guessed whether the conclusion was written by a human or a machine only 15% of the time.
But the key is not even the numbers. The o1-preview model scored 89% on Grey Matters tests for management tactics, while physicians with access to GPT-4 scored 41%, and physicians with standard tools scored 34%. This means that having a live specialist between the patient and the algorithm did not improve but worsened the outcome. The human factor—overconfidence, cognitive biases, fatigue—acted not as a filter but as an obstacle.
Timeline and Context
The path to this publication took 65 years. In 1959, Ledley and Lusted published an article in Science proposing that complex clinical cases be the gold standard for evaluating medical computing systems. Since then, every technology—from naive Bayes to symptom checkers—has been tested on NEJM tasks.
Until 2023, machines lost to physicians so obviously that human control groups were unnecessary. GPT-4 first showed results requiring comparison with humans. In 2025, OpenAI released o1-preview—a model with step-by-step logical reasoning (chain-of-thought)—and in early 2026, the full o1. Researchers from Harvard managed to conduct experiments in the narrow window between the model's release and its obsolescence: today, labs are already testing o3 and GPT-5.3 with multimodal capabilities.
An important chronological detail: the manuscript was accepted in April, and in May the world discusses it a technological generation late. What we are reading now is yesterday's news in the industry.
Who Wins and Who Loses
Winners:
OpenAI. The study was published in Science, not in a narrow medical journal. This gives the company not a clinical but a reputational asset: the narrative "the model thinks like a doctor" turns into "thinks better than a doctor." OpenAI's estimated valuation after closing its round in May 2026 approaches $400 billion, and each such publication adds 0.2–0.3x to the multiplier.
Medical startups building "second opinion" as a service. Before the publication, their product was an optional tool. Now they have a quote from Science where AI without a doctor outperforms a doctor with internet access. This opens a B2B sales market to hospitals worth $8.4 billion.
Patients with rare and complex diagnoses. The model on CPC cases included the correct diagnosis in the differential list in 78.3% of cases, and with "close" diagnoses considered, in 97.9%. This means that a second opinion from AI could cost $14 per inference instead of $2,500 for a consultation at the Mayo Clinic.
Losers:
Primary care and emergency physicians. The study systematically dismantles the defensive thesis "AI is an assistant, not a replacement." If the machine surpasses humans precisely in conditions of incomplete information—the main argument for human intuition—then the value of the physician shifts from diagnostician to communicator and procedure operator. This reduces the market value of cognitive labor in medicine over a 10-year horizon.
Developers of traditional CDS systems. The result of 34% for physicians with standard tools versus 89% for o1-preview is a death sentence for knowledge bases like UpToDate and Isabel Healthcare if they do not integrate reasoning models into their interface within the next 18 months.
Manufacturers of specialized medical software. When a general-purpose model outperforms experts on their own turf, niche solutions lose their purpose. Why buy software for $120,000 per year if an API subscription for $800 per month gives a better result?
What the Media Are Not Saying
First: the study authors have conflicts of interest that are not articulated in headlines. Adam Rodman is a visiting researcher at Google DeepMind. Eric Horvitz works at Microsoft. This does not discredit the results but explains why the study tested OpenAI o1 specifically, not competing models. No publication of this level is neutral.
Second: the model did not show superiority in the most important metric—identifying "diagnoses that must not be missed." In NEJM Healer tests, o1-preview identified life-threatening conditions with a median of 0.92, but the difference with physicians was not statistically significant. This is a time bomb: AI may be more accurate overall, but if it misses meningitis or aortic dissection as often as a human, its implementation in emergency care creates a risk of catastrophic errors.
Third: inference cost. The researchers mention that processing one CPC case on o1-preview costs between $3.20 and $42.10 depending on complexity. If an emergency department sees 200 patients per day, running the model on each would cost $640–$8,420 daily, or $230,000–$3 million per year. This is comparable to the salary of one or two physicians, and hospital administrators will calculate exactly that.
Fourth: the study tested text, but medicine is multimodal. An editorial comment in Science directly states that GPT-5.3 and Gemini 3.1 Pro already process text, images, audio, and video. This means the next publication—likely in 6–8 months—will show multimodal models surpassing physicians in tasks requiring interpretation of images and physical exam data. Broder's results are the lower bound of AI capabilities, not the ceiling.
Forecast: Next 30 Days and 90 Days
30 days (by June 5, 2026):
Medical insurance companies, including UnitedHealth Group and Aetna, will begin internal testing of LLM second opinions. Goal: to understand if costs for unnecessary specialist consultations can be reduced. Pilot projects will launch in 3–5 large networks, each with a budget of $2–4 million.
JAMA and NEJM will publish editorials urging caution. The rhetoric will be: "impressive results, but prospective trials are needed." This is a standard defensive reaction from the medical establishment, which sees AI as a threat to its expert monopoly.
OpenAI and Google will intensify negotiations with the FDA on a regulatory framework for SaMD (Software as a Medical Device) based on reasoning models. The question is not whether such systems will be approved, but what risk class they will be assigned—II or III. This determines the cost of market entry: $2 million versus $15 million for clinical trials.
90 days (by August 5, 2026):
The first preprints with multimodal models in clinical tasks will appear. It is expected that GPT-5.3 with access to CT scans and audio recordings of breathing will surpass pulmonologists in diagnosing interstitial lung diseases.
Quiet integration of LLM second opinions will begin in emergency departments of five leading US academic centers. Formally as research protocols, but effectively as a working tool. Physicians will use it like they use UpToDate today, but without disclosing it to patients.
The key legal precedent has not yet occurred, but its contours are already visible. The first lawsuit where a lawyer asks a physician: "Why did you not use an AI second opinion, if published data in Science shows its superiority in emergency diagnosis?" will change the standard of care faster than any clinical guidelines. Broder and Manrai's study will become a cited document in court—not as a scientific paper, but as evidence that the standard of care has evolved.
— Editorial Team
