Zero-shot learning for clinical phenotyping: comparing LLMs and rule-based methods

Neves, BernardoMoreira, José MariaGonçalves, SimãoCerejo, JorgeSilva, Nuno A. daLeite, FranciscaSilva, Mário J.2025-08-132025-08-132025-06-01Neves, B., Moreira, J. M., Gonçalves, S., & Cerejo, J. et al. (2025). Zero-shot learning for clinical phenotyping: comparing LLMs and rule-based methods. Computers in Biology and Medicine, 192, Article 110181. https://doi.org/10.1016/j.compbiomed.2025.1101810010-48251903ac83-59bd-4d5e-98e0-cd0ae07e7fe4http://hdl.handle.net/10400.14/54550Background: Phenotyping, the process of systematically identifying and classifying conditions within clinical data, is a crucial first step in any data science work involving Electronic Health Records (EHRs). Traditional approaches require extensive manual annotation efforts and face challenges with scalability. Methods: We investigated the use of Large Language Models (LLMs) for zero-shot phenotyping of 20 prevalent chronic conditions based on synthetic patient summaries generated from real structured EHRs codes. We evaluated the performance of multiple LLMs, including GPT-4o, GPT-3.5, and LLaMA 3 models with 8-billion, 70-billion, and 405-billion parameters, comparing them against traditional rule-based methods. For the analysis we used a dataset of 1,000 patients from Hospital da Luz Lisboa. Results: GPT-4o outperformed both traditional rule-based methods and alternative LLMs, achieving superior recall (0.97) and macro-F1 score (0.92). Rule-based phenotyping, while highly precise (0.92), showed lower recall (0.36). The integration of rule-based methods with LLMs optimized phenotyping accuracy by targeting manual annotation efforts on discordant cases. Conclusion: Zero-shot learning with LLMs, particularly GPT-4o, offers a powerful and efficient approach for phenotyping chronic conditions from EHRs, significantly reducing the need for extensive labeled datasets while maintaining high accuracy and interpretability.engLarge language modelsMultimorbidityPhenotypingZero-shot learningZero-shot learning for clinical phenotyping: comparing LLMs and rule-based methodsresearch article10.1016/j.compbiomed.2025.11018110500321828440273817