AI for Evaluating Residual Calcifications in Post-Lumpectomy Pre-Radiation Mammography
2Gray School of Medicine, Tel-Aviv University, Israel
Purpose: Commercial mammography AI systems are widely adopted owing to excellent screening performance; their utility for evaluating residual calcifications in the post-lumpectomy breast, before radiation therapy, remains unclear.
Materials and Methods: In this retrospective single-center study, the post-lumpectomy arm comprised 156 pre-radiation mammographic examinations performed to assess residual calcifications in 148 women (median age 59 years; five contributed more than one examination); 43 underwent stereotactic vacuum-assisted biopsy (VAB). The control arm comprised 131 of 425 consecutive stereotactic VABs of calcifications from the same period, selected by case-by-case review excluding prior breast malignancy, breast surgery, or radiation and examinations without a pre-biopsy AI result. AI (Transpara, ScreenPoint Medical) output was positive when the examination-level category was Elevated or Intermediate and negative when Low; the continuous score (0–100) was recorded. Post-lumpectomy examinations without biopsy-proven malignancy were considered negative; radiologist interpretation was recorded in both arms. Wilson CIs, Fisher exact and exact McNemar tests, and ROC AUC were used.
Results: VAB yielded 13 malignancies in the post-lumpectomy arm. AI detected 6/13 (sensitivity 46.2%; 95% CI 23.2–70.9), all seven misses categorized Low (scores 28–38), whereas radiologists detected 13/13 (100%; McNemar P=.016), with specificity 78.3% versus 65.4% for AI. In the control arm (30 malignant, 85 benign, 16 high-risk [B3] lesions), AI performed as expected: sensitivity 93.3% (28/30; 95% CI 78.7–98.2; P=.001), median malignant score 83 versus 38 (P<.001), and AUC 0.800 versus 0.578. Restricted to biopsy-proven benign calcifications, specificity was 11.8% (10/85) in controls versus 63% (17/27) post-lumpectomy (P<.001).
Conclusion: AI underperformed in post-lumpectomy mammography for residual calcifications, relative to radiologists and to its performance in non-operated breasts, indicating globally attenuated response in the operated breast: the setting, not the algorithm, underlies the failure. This highlights the need for caution when applying AI in clinical settings outside its validated domain.