Loading...
Publication
Zero-shot learning for clinical phenotyping: comparing LLMs and rule-based methods
| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 5.17 MB | Adobe PDF |
Advisor(s)
Abstract(s)
Background: Phenotyping, the process of systematically identifying and classifying conditions within clinical data, is a crucial first step in any data science work involving Electronic Health Records (EHRs). Traditional approaches require extensive manual annotation efforts and face challenges with scalability. Methods: We investigated the use of Large Language Models (LLMs) for zero-shot phenotyping of 20 prevalent chronic conditions based on synthetic patient summaries generated from real structured EHRs codes. We evaluated the performance of multiple LLMs, including GPT-4o, GPT-3.5, and LLaMA 3 models with 8-billion, 70-billion, and 405-billion parameters, comparing them against traditional rule-based methods. For the analysis we used a dataset of 1,000 patients from Hospital da Luz Lisboa. Results: GPT-4o outperformed both traditional rule-based methods and alternative LLMs, achieving superior recall (0.97) and macro-F1 score (0.92). Rule-based phenotyping, while highly precise (0.92), showed lower recall (0.36). The integration of rule-based methods with LLMs optimized phenotyping accuracy by targeting manual annotation efforts on discordant cases. Conclusion: Zero-shot learning with LLMs, particularly GPT-4o, offers a powerful and efficient approach for phenotyping chronic conditions from EHRs, significantly reducing the need for extensive labeled datasets while maintaining high accuracy and interpretability.
Description
Keywords
Large language models Multimorbidity Phenotyping Zero-shot learning
Pedagogical Context
Citation
Neves, B., Moreira, J. M., Gonçalves, S., & Cerejo, J. et al. (2025). Zero-shot learning for clinical phenotyping: comparing LLMs and rule-based methods. Computers in Biology and Medicine, 192, Article 110181. https://doi.org/10.1016/j.compbiomed.2025.110181