Large Language Models for Detecting Extraprostatic Extension in Free-Text Prostate MRI Radiology Reports

Authors

• Nikhil Goyal, MD, Thomas Jefferson University HospitalFollow
• Satvik Tripathi, University of Pennsylvania
• Krishnan R. Patel, MD, MHS, The University of Texas MD Anderson Cancer Center
• Hyun Tae Kim, MD, MPH, University of Pennsylvania
• Tessa S. Cook, MD, PhD, University of Pennsylvania

Abstract

Extraprostatic extension (EPE) is a prognostically important pathologic feature of prostate cancer that is difficult to predict preoperatively. Because EPE may be microscopic, it may not be detected with conventional imaging. Several magnetic resonance imaging (MRI) features associated with EPE risk are routinely described in radiology reports and may be detectable by large language models (LLMs). This study evaluated the performance of five commercial LLMs (GPT-3.5-turbo, GPT-4, GPT-4o-mini, GPT-4o, and o3-mini) in classifying pathologic EPE from free-text prostate MRI reports for patients with confirmed prostate cancer. Among the models tested, o3-mini achieved the highest accuracy (66.3%) and positive predictive value (55.6%), while GPT-4 achieved the highest sensitivity (78.9%) and negative predictive value (77.1%). GPT-3.5-turbo had the lowest accuracy (56.8%), and GPT-4o-mini the lowest sensitivity (45.5%). These findings suggest that LLMs may serve as useful tools for classifying pathologic EPE, potentially aiding radiologists and clinicians in interpreting prostate MRI reports.

Recommended Citation

Goyal, MD, Nikhil; Tripathi, Satvik; Patel, MD, MHS, Krishnan R.; Kim, MD, MPH, Hyun Tae; and Cook, MD, PhD, Tessa S. (2026) "Large Language Models for Detecting Extraprostatic Extension in Free-Text Prostate MRI Radiology Reports," The Medicine Forum: Vol. 27, Article 24.
Available at: https://jdc.jefferson.edu/tmf/vol27/iss1/24