Comparative Evaluation of Machine Learning and Specialist Physicians in Breast Care Triaging: A Real-World Observational Study.

Abstract

Objective: To evaluate the diagnostic accuracy and efficiency of a proprietary breastspecific machine learning (ML) model—built upon the open-source Open Triage platform—in comparison to specialist physicians, using standardized real-world clinical data for breast referral triaging. Materials and Methods: A retrospective observational study was conducted using 174 standardized breast cases obtained from proprietary industry datasets, spanning 46 disease types including 23 cancers. The cohort ranged from 19 to 75 years (mean: 39.4 ± 12.0). Physicians and an ML model each generated three diagnostic predictions per case. Both modalities were subsequently compared after benchmarking their predictions against a gold standard diagnosis established through imaging and biopsy. Performance was evaluated using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and receiver operating characteristic (ROC) analysis. Time efficiency was also assessed to compare diagnostic turnaround across physicianand ML-generated predictions. Results: The ML model demonstrated superior diagnostic accuracy (100%) compared to physicians (83.9%), with higher sensitivity (0.947 vs. 0.826) and PPV (0.500 vs. 0.442). Both groups achieved comparable specificity and NPV values. ROC analysis showed an AUC of 0.91 for the 1st ML prediction versus 0.83 for the 1st prediction of the doctor, indicating superior predictive power for the ML model. Conclusion: The ML model demonstrated comparable or superior diagnostic accuracy to physicians while significantly reducing time requirements. These findings suggest that AI-powered triaging tools could enhance efficiency and standardization in breast triaging.