Impact of Large Language Model–Assisted Reporting Tools on Radiology Reporting Efficiency and User Experience

Background

Increasing radiology workloads have driven interest in artificial intelligence (AI) tools designed to improve reporting efficiency. Large language model (LLM)-based applications may facilitate report generation; however, evidence regarding their impact in real-world, Hebrew-language radiology practice remains limited.

Materials and Methods

Eleven attending radiologists participated in a retrospective before-and-after workflow study. Examinations interpreted after implementation of an LLM-assisted reporting tool were compared with examinations of the same type interpreted by the same radiologist before implementation. Report generation time was extracted from the radiology information system (RIS). For radiologists reporting more than one examination type, each examination type was analyzed separately. Reporting times before and after LLM implementation were compared for each radiologist–examination type combination using the Mann–Whitney U test, with p<0.05 considered statistically significant. Following the study period, participants completed a questionnaire assessing usability, satisfaction, and perceived effects on reporting workflow and fatigue.

Results

A total of 485 examinations were analyzed across 11 radiologists and 13 examination types. Statistically significant reductions in report generation time following LLM implementation were observed for one or more examination types in six of the 11 radiologists, with the magnitude of the effect varying across radiologists and examination types. Questionnaire responses indicated an overall favorable perception of LLM-assisted reporting: 10 of 11 radiologists perceived shorter reporting times, 9 of 11 reported reduced fatigue (scores 4–5 out of 5), and all participants indicated that they would recommend LLM-assisted reporting.

Conclusion

Implementation of LLM-assisted reporting was associated with reduced report generation time in selected radiologist–examination type combinations and was well received overall. Variability in the observed effect across radiologists and examination type suggests that the efficiency gains of LLM-assisted reporting may depend on both individual reporting patterns and examination characteristics. These findings support further evaluation of LLM-based reporting tools in real-world radiology practice.