ISRA May 2022

Using Deep Learning and Natural Language Processing to Automate the Identification and Extraction of Follow-Up Recommendations

Hannah Silverman 1 Omri Sivan 2 Tzvi Shir 2 Ma'ayan Tropper-Wachtel 2 Timor Marom 2 Michal Meiri 2
1Medical Imaging, Hadassah University Medical Center, Israel
2Computer Science, Agamontech Technologies LTD., Israel

Background: Radiology reports often contain follow-up recommendations for further imaging or other procedures or consultations. On average around 50 percent of recommendations are not acted upon, meaning that many patients do not receive the recommended follow-up care needed. Miscommunication about recommendations or failure to follow-up at all can lead to delayed treatment, poor patient outcomes, unnecessary testing, lost revenue, and legal liability.

Purpose: We describe an evaluation of a Natural Language Processing system at the Department of Radiology at a big hospital system. The purpose of this system is to automate the identification, extraction, verification and communication of follow-up recommendations in radiology reports so that healthcare organizations can close the loop on these recommendations and provide better care for their patients.

Methods: Over a period of six months from 2019 to 2020 a Deep Learning Natural Language Processing language model specifically trained on millions of radiology reports. Qualified follow-up recommendations are then communicated to the referring physician who ordered the study.

Results: The use of Deep Learning Natural Language Processing (NLP) language model specifically trained on millions of radiology reports. This baseline model is then used by two long short-term memory (LSTM) intermediate models that are further fine-tuned for specific tasks. A threshold for the intermediate models was chosen so that the final model, which is a function of the two intermediate models, would have precision above 95%. The system achieves over 96% precision, with the 95% confidence interval of the precision lying between 94.7 and 97.6 percent. Evaluation of these efforts has shown a substantial increase in the percentage of patients who completed their follow-ups, from 36% to 87%.

Conclusion: The work described herein is continued, with the goal of automating follow-up recommendation identification, extraction, verification and communication, effectively closing the loop to ensure better clinical care for patients.