PURPOSE: The purpose of this study was to evaluate the influence of AI-based prioritization on RTAT (report turnaround time, the most frequently used quality metric in radiology) on prioritized and non-prioritized studies containing positive findings.
Primary endpoint: The RTAT distribution for prioritized workflow queue with various detection accuracy parameters, compared to FIFO (First in, first out) workflow baseline.
METHOS: In this analysis, we developed a model to describe the workflow queue of a single radiologist. We used this model to study the effect of various accuracy parameters of the prioritization algorithm on the RTAT using a Monte-Carlo simulation. The results of the simulation were analyzed to determine the effect of the queuing method on the distribution of RTAT of medical images containing positive findings. The trivial queuing method of FIFO (first in first out) was used as a baseline to compare to AI-prioritization based queuing with various sensitivity and specificity parameters for the AI performance.
RESULTS: The results showed that RTAT of head and c-spine CT scans containing abnormal hyperdense and neck bone hypodense findings improved significantly when utilizing Aidoc’s prioritization algorithm, in comparison to FIFO, with all performance parameters tested. Specifically, the “low” accuracy prioritization algorithm (70% sensitivity, 70% specificity) resulted in over 46% RTAT reduction and the “perfect” accuracy algorithm (100% sensitivity, 100% specificity) resulted in 67.3% reduction in RTAT. When evaluating the algorithm’s detection parameters demonstrated in the pivotal clinical trial (~90%, 90% sensitivity, specificity) the results were very similar to those of the “perfect” algorithm, with 60.3% improvement in RTAT. Additionally, the average RTAT of all medical images without findings remained essentially unchanged.
Figure 1: Average RTAT for Positive and Negative Studies for AI-Based Prioritized Queue vs. Baseline (Using Aidoc’s 90% Sensitivity and Specificity Performance)
CONCLUSION: These results demonstrate the potential clinical benefit of Aidoc’s prioritization algorithm for improving the timeliness of positive findings reporting to the ordering physician, in accordance with one of The Joint Committee (TJC)’s identified goals for patient safety. The results showed that RTAT of head and c-spine CT scans containing positive findings improved with all performance parameters of the prioritization algorithm. Moreover, the average RTAT of all medical images containing positive findings was shown to somewhat improve as well, with no significant impact on the RTAT of negative studies.
