BRIDGE: A Benchmark for Understanding Real-World Clinical Text

DeepSeek-R1 scored 92 out of 100 on the US medical licensing exam. That's an impressive result. But give the same model a real clinical note, the kind full of shorthand and abbreviations a doctor typed in a rush, and its score drops to 44 out of 100.

That single gap is why BRIDGE exists. Researchers from Harvard, MIT, Stanford and Mayo Clinic built the largest test yet of how AI performs on real medical text, not textbook questions.

They call it, in their own words:

“The largest benchmark for LLM in medicine to date”

The Problem With Exam-Style Testing

Most AI health benchmarks use medical exam questions or clean journal text. Useful for a quick check, but nothing like a real hospital record.

Real clinical notes are messy:

• Full of abbreviations and shorthand
• Inconsistent in structure
• Written for other clinicians, not for a test

A model that has memorised enough textbooks to pass an exam isn't automatically a model that can read a real discharge summary correctly. BRIDGE was built to test that gap directly.

So, What Is BRIDGE?

BRIDGE is a large group of tasks built from real clinical notes. These are not from clean notebooks or exam banks.

• BRIDGE contains 87 expert made task drawn from real clinical data sources
• The benchmark tests the model’s ability to read through 9 unique languages from English and Chinese to Norwegian and Portuguese.
• BRIDGE contains over 1.4 million samples, it includes messy clinical notes, case reports and consultations.

BRIDGE has currently ran 52 models, from proprietary, privatised models like ChatGPT and Gemini to Open-source and medical specialist models. Over 13,000 experiments ran in all. These tasks include diagnoses, triages, spotting symptoms and something simple as summarising a note

Fig. 1 — The BRIDGE process at a high level: real clinical text sourced from hospitals and consultations is turned into 87 tagged tasks, tested against 52 models, and scored into a leaderboard.

The Headline Result

With the benchmark built, the next step was finding out how best to use these models on it. The researchers tested all 52 models three different ways: asked with no examples, asked to reason step by step first, and shown 5 worked examples before answering.

Without any examples to learn from, the top models were nearly tied. Asking them to reason step by step barely helped, and in some cases made results slightly worse. What did help, was showing the model a handful of examples first: scores jumped across nearly every model tested.

This matters because prompting strategy is often the one lever a hospital or developer can actually pull without retraining. Knowing that a few good examples beat an elaborate reasoning prompt is a simple, low-cost finding with real practical value.

Some key findings:

• Open-source models are closing the gap with proprietary ones.
• Giving a model a handful of examples helped far more than asking it to reason step by step.
• No model scored above 55.5 out of 100 anywhere on the benchmark, even under the best conditions.
• Models fine-tuned specifically for medicine often underperformed general-purpose models of a similar size.

As the researchers put it:

"Open-source LLMs can achieve performance comparable to proprietary models."

Fig. 2 — Overall score of leading models across three prompting strategies: zero-shot, chain-of-thought, and five-shot. For each model, three bars are shown in that order.

Where Models Still Struggle

Not every task was equally hard. Some patterns stood out:

• Easy: sorting text into a fixed category, like flagging a likely diagnosis
• Hard: pulling detailed information out of a messy note
• Hard: matching text to official medical codes
• Hard: writing an accurate, human-quality summary

Language mattered too. Some models stayed strong across all nine languages, while others, especially medical models trained mostly on English, dropped off sharply once the language changed. The same was true across medical specialties. No single model won everywhere.

The Number That Says It All

92 out of 100 on a standard medical exam. The same model, 44 out of 100 on real clinical text, by DeepSeek-R1. Overall, the best performing model was Gemini-1.5-pro with a 5-shot prompt strategy, achieving 55.5 our of 100 in BRIDGE.

Knowing medical facts and being able to work with real, messy clinical documents are clearly two different skills. Right now, AI is far better at the first than the second.

A Practical Use Case

These findings make for a really great guide to use this kind of AI in healthcare today. The main takeaway, use AI, not as an autonomous decision-maker, but as a drafting assistant built around the benchmark's limits rather than in spite of them.

Picture an AI tool that helps clinicians draft discharge summaries and flag likely diagnoses from consultation notes. Built on what BRIDGE shows, that tool would focus on four things:

• Accuracy: Use a few-shot approach with real, vetted examples from the hospital's own notes, rather than relying on raw zero-shot output or an elaborate reasoning prompt. Choose the specific model using BRIDGE's leaderboard results for that task and specialty, since no single model wins everywhere.
• Privacy: Clinical notes are sensitive. BRIDGE shows us that open-source models can match proprietary ones, hosting an open-source model inside the hospital's own infrastructure keeps patient data from ever leaving the building, rather than sending it to a third-party API.
• Multilingual support: If the hospital serves patients in more than one language, the tool should be tested and scored separately in each language actually used, not assumed to carry over from its English performance.
• Human review: With the best score on the whole benchmark still only 55.5 out of 100, every AI-drafted note needs a clinician's sign-off before it reaches a patient record, especially for harder tasks like summarisation and medical coding, where scores were lowest.

Used this way, AI becomes a tool that saves a clinician time on the first draft, while the clinician stays responsible for what the final note says. The Aim is to not replace humans but to expedite their processes.

What This Means Going Forward

For hospitals, clinicians and AI developers, BRIDGE offers a few clear lessons:

• Test AI on real documents, not exam questions, before trusting it with real work
• A few good examples beat asking a model to "think step by step"
• Bigger isn't always better if the underlying model is outdated
• A model that works well in English cannot be assumed to work well elsewhere

As AI keeps moving into healthcare, this kind of honest, real-world testing matters more than another leaderboard win. The goal isn't a model that can pass an exam. It's one that can be trusted with the real, complicated work clinicians do every day.