How Accurate Is an AI Doctor? What the Clinical Studies Actually Show

There is no single accuracy percentage for an ai doctor — it ranges from roughly 52% average diagnostic accuracy in strict clinical studies to about 76% on everyday health questions, and 85–95% in image-based tasks like reading a skin lesion or scan. According to a 2025 meta-analysis of 83 studies published in npj Digital Medicine, generative AI’s pooled diagnostic accuracy sits at 52.1%, with a 95% confidence interval of 47.0–57.1%. «Accuracy» is not one number; it depends entirely on what is being measured and how.

This article is honest about both sides: an AI doctor is a genuinely useful triage and reference tool, and it is not a replacement for a licensed physician. The last section explains exactly how to use one safely — all figures below come from peer-reviewed research or major health-system reporting, cited inline.

⚠️ Medical disclaimer. This article is for general information only and is not medical advice. An AI doctor is not a substitute for a licensed physician, diagnosis, or treatment. If you have a medical emergency, call 911 (US). If you are in a mental-health crisis or thinking about suicide, call or text 988 (Suicide & Crisis Lifeline). Always consult a qualified healthcare professional about your specific situation.

The short answer: there is no single accuracy number

Accuracy depends entirely on the task an AI symptom checker or LLM is being asked to perform, and comparing numbers across studies without noting that context is misleading.

Accuracy depends entirely on the task

The average diagnostic accuracy of generative AI is 52.1% (95% CI 47.0–57.1%), pooled from 83 studies in a 2025 meta-analysis published in npj Digital Medicine. On everyday health questions — the kind people actually type into a chat window — large language models answer correctly 76.2% of the time, according to Penn State Health’s «Diagnose-a-thon» (FAccT 2026). In image-based tasks such as radiology or dermatology, AI diagnostic accuracy climbs to 85–95%. Symptom checkers specifically range from 30% to 96% depending on how «correct» is counted — the reasons for that spread are explained further down.

The gap between these numbers usually comes down to a handful of factors:

• What is being measured — a single top diagnosis, or any correct answer within the top three
• Whether the test used real patients or written clinical vignettes
• How narrow or common the condition is (imaging and frequent symptoms score higher)
• The quality and risk of bias of the underlying study

A quick reference table

AI doctor vs. a real doctor: who is more accurate?

The comparison shifts sharply depending on whether the AI doctor is measured against a specialist, a generalist, or a narrow clinical task.

Here is how those head-to-head numbers actually break down against each type of clinician.

Against expert physicians, AI still trails

Per the npj Digital Medicine meta-analysis (2025), AI is 15.8% less accurate than expert physicians (95% CI 4.4–27.1, p=0.007), but shows no statistically significant difference compared with non-expert physicians (a gap of just 0.6%, p=0.93). In practice, an AI doctor performs closer to a general practitioner than to a specialist.

In some narrow tasks AI matches or beats doctors

The picture is not one-sided. In rheumatology, the Ada Health symptom checker correctly listed the top diagnosis 70% of the time versus 54% for physicians (p=0.002; Rheumatology International, 2022). Babylon Health’s triage was rated safer than human doctors’ — 97.0% versus 93.1% — in a study published in Frontiers. In April 2026, an OpenAI reasoning model matched or outperformed two experienced internal-medicine physicians on diagnosis from electronic health records, in a study conducted at Harvard Medical School and Beth Israel Deaconess Medical Center and published in the journal Science. These results came from narrow, controlled conditions — the AI worked from text records only, without a physical exam.

Where AI is clearly weaker

Context changes everything. The same Ada symptom checker managed only 30% top-1 accuracy versus 47% for physicians in a 2023 emergency-department study — a much harder, higher-stakes environment than a routine visit. In mental health, licensed therapists responded appropriately in 93% of cases compared with under 60% for AI chatbots, in a University of Minnesota study.

Why do the accuracy numbers vary so much?

The wide range of reported figures usually comes down to how «correct» is defined, not how good the underlying model is.

Top-1 vs. top-3 counting changes the headline number. Babylon Health’s symptom checker scored 70% when only the single top diagnosis counted, but 96.7% when a correct answer anywhere in the top three counted, using the Semigran 30-vignette benchmark reported in Frontiers (2020). The same tool can look «accurate» or «inaccurate» purely based on which metric a study chooses to headline.

Recall and precision pull in opposite directions. Babylon’s recall — the ability to not miss a real condition — was 80.0%, while its precision — not flagging conditions that weren’t actually present — was only 44.4%. A high-recall, low-precision system will look impressively «safe» in one framing and unreliably «noisy» in another.

Study quality varies a lot, and that matters. In the npj Digital Medicine meta-analysis, 76% of the underlying studies carried a high risk of bias, and the analysis found statistically significant signs of publication bias (p=0.045). Many tests use written clinical vignettes rather than real patients, which tends to inflate results compared with messier real-world conditions.

Where AI doctors are strong — and where they fail

Strong: pattern recognition and common presentations. AI performs best where large, labeled datasets exist. Across multiple imaging studies — radiology, dermatology, ophthalmology — accuracy has been reported in the 85–95% range, and a meta-analysis of 11 prospective dermatology studies on melanoma detection specifically found AI sensitivity of 80.9% and specificity of 75.6%, comparable to dermatologists (78.6% and 75.2%). AI doctors also handle frequent, well-documented symptom patterns well:

• Common, well-studied conditions with large training datasets
• Image-based diagnosis (skin lesions, radiology scans)
• Structured triage questions with clear urgency thresholds
• Everyday health questions with a single, well-defined answer

Weak: rare disease, missing context, and the physical exam. The weak points are consistent across studies: rare conditions (too little training data), no physical examination or lab work, and heavy dependence on how precisely a user describes symptoms. Two failure modes compound this — hallucination, where the model states something incorrect with full confidence, and automation bias, where a clinician or patient trusts an AI output too readily. A 2021 randomized controlled trial found that giving physicians an AI differential-diagnosis list did not improve their overall accuracy (57.4% vs. 56.3%, p=0.91). But when the correct diagnosis was actually present in that AI-generated list, physician accuracy jumped to 79.0% versus 34.7% when it wasn’t (adjusted OR 7.68). The tool helps — but only when it gets the answer right in the first place, which is documented in the differential-diagnosis RCT on NCBI/PMC.

As the researchers behind that trial put it:

Overall diagnostic accuracy was not significantly different between groups that used the AI differential-diagnosis generator and those that did not.

NCBI/PMC — AI Differential-Diagnosis List RCT

How to use an AI doctor safely

Used deliberately, an AI doctor is a second opinion, not a diagnosis. Here is how to get useful answers without over-relying on the tool:

1. Use it to understand a medical term, lab value, or diagnosis you already received.
2. Ask it to help you prepare specific questions before a real appointment.
3. Use it to gauge how urgent a symptom might be — not to confirm it is nothing.
4. Treat any answer as a starting point for ai doctor accuracy research, not a final word.
5. Write specific, detailed prompts — Penn State Health found that prompts between roughly 60 and 250 characters produced more accurate AI responses than vague one-line questions.
6. Cross-check anything alarming or unusual with a licensed clinician before acting on it.
7. Never use it as the sole basis for starting, stopping, or changing a treatment.

Treat it as a second opinion, not a diagnosis

An AI doctor is well suited to explaining a term or test result, helping you prepare for an appointment, or offering an early read on urgency. It is not suited to delivering a final diagnosis, prescribing treatment, or handling emergencies. That caution is reflected in patient behavior too: 52% of patients still say they prefer a human doctor over an AI one, according to research from University of Arizona Health Sciences.

Good uses versus poor uses break down clearly:

• Good: explaining a diagnosis or lab result, drafting questions for an appointment, gauging urgency
• Good: general health education and understanding medication interactions
• Poor: final diagnosis without a physical exam or lab work
• Poor: emergencies, mental-health crises, or anything requiring immediate care

Red flags — when to stop and call a professional

Some situations call for a professional immediately, not a chat window:

• Sudden or severe symptoms — chest pain, trouble breathing, signs of stroke
• Rapid worsening of any condition over hours
• Infants, pregnant patients, or elderly patients with new symptoms
• Existing chronic conditions with unusual changes

For a genuine emergency, call 911. For a mental-health crisis or thoughts of suicide, call or text 988. More detailed, specific prompts do produce better AI answers, but that never replaces an in-person evaluation when any of the above applies.

When symptoms are severe or moving fast, treat the urgency as real and get a human opinion immediately — an AI doctor is the wrong tool for an emergency.

FAQ

Related guides

• What Is an AI Symptom Checker? How It Works, Accuracy & Safety
• How Does an AI Doctor Work? A Plain-English Look Under the Hood
• AI Doctor vs Real Doctor: What Each Can (and Can’t) Do for You