The field of artificial intelligence is undergoing unprecedented evolution, with large language models (LLMs) becoming capable of performing complex cognitive tasks in medicine. From accurately answering multiple-choice medical questions to developing advanced differential diagnosis, AI is now demonstrating its potential in assisting with medical diagnosis.
According to statistics, diagnostic errors result in the death or permanent disability of approximately 795 patients each year in the United States alone. Due to the high cost of misdiagnosis or delays, artificial intelligence could play a vital role in improving diagnostic accuracy and reducing medical errors. Over the past two years, proprietary AI models, such as OpenAI's GPT-000, have excelled in diagnosing complex medical cases, gradually becoming integrated into healthcare applications.
But open source models had not previously achieved this level of performance.
However, a study conducted by researchers at Harvard Medical School reveals that recent open source models, such as Llama 3.1 developed by Meta, are able to compete with proprietary models. The study, published on March 17, 2025 in the JAMA Health Forum, showed that Llama 3.1 achieved promising results in advanced clinical tests, comparable to some of the best closed-end models for diagnosing complex cases.
Part of the Llama series, Llama 3.1 is offered by Meta as an open-source alternative that allows researchers to use, adapt, and train it freely, without commercial restrictions. Open-source models also have the advantage of being customizable for specific needs (e.g., training on internal data while preserving patient confidentiality), unlike closed-source models, which are often limited to external servers and proprietary use.
Diagnosis of diseases
AI models, whether open or closed, rely on massive databases (medical literature, clinical studies, patient data) to analyze and make a diagnosis. When presented with a new case, they compare the information received with their accumulated knowledge to generate diagnostic hypotheses.
In this study, Llama 3.1 was tested on 70 complex clinical cases and 22 unpublished cases.
results:
70% overall accuracy, compared to 64% for GPT-4;
41% of correct diagnoses on the first suggestion, compared to 37% for GPT-4;
In new cases, accuracy rose to 73%, with 45% of first-choice answers correct.
Researchers' opinions
Dr. Arjun Manrai, lead author of the study and a professor of biomedicine at Harvard, says that artificial intelligence, if properly integrated into healthcare systems, could become a reliable assistant for doctors.
In a newspaper statement Asharq Al-Awsat, he points out that open source models allow data to be kept local (in hospitals), thus ensuring confidentiality, while closed models require sending data to external servers, which raises security concerns. He adds that open source models are more flexible and less expensive, therefore more suitable for institutions with limited budgets.
However, he also notes challenges to overcome, including the need for technical skills for maintenance, the difficulty of integration with existing hospital systems (unlike more "turnkey" closed solutions), and the need for additional clinical trials to ensure reliability, safety, and reduction of bias.
Open source platforms in health
Several open source platforms are available today:
Azure AI Foundry (Microsoft): Offers AI models to analyze medical images and automatically generate radiology reports.
Among its tools:
MedImageInsight: image classification and anomaly detection,
MedImageParse: tumor and organ delineation,
CXRReportGen: Chest X-ray analysis and report generation.
MONAI: Developed by Nvidia in collaboration with King's College London, this platform supports the creation and training of AI models for medical imaging, with a focus on privacy and diagnostic accuracy.
OpenMEDLab: Launched by researchers at Cornell University in March 2024, this open-source platform develops multimodal models for health, leveraging Google's Gemini in particular, and aims to stimulate innovation in the medical sector.
