Objectives To evaluate the diagnostic value of computer-aided diagnosis (CAD) software on ultrasound in distinguishing benign and malignant breast masses and avoiding unnecessary biopsy. Methods This prospective, multicenter study included patients who were scheduled for pathological diagnosis of breast masses between April 2019 and November 2020. Ultrasound images, videos, CAD analysis, and BI-RADS were obtained. The AUC, accuracy, sensitivity, specificity, PPV, and NPV were calculated and compared with radiologists. Results Overall, 901 breast masses in 901 patients were enrolled in this study. The accuracy, sensitivity, specificity, PPV and NPV of CAD software were 89.6%, 94.2%, 87.0%, 80.4%, and 96.3, respectively, in the long-axis section; 89.0%, 91.4%, 87.7%, 80.8%, and 94.7%, respectively, in the short-axis section. With BI-RADS 4a as the cut-off value, CAD software has a higher AUC (0.906 vs 0.734 vs 0.696, all p < 0.001) than both experienced and less experienced radiologists. With BI-RADS 4b as the cut-off value, CAD software showed better AUC than less experienced radiologists (0.906 vs 0.874, p < 0.001), but not superior to experienced radiologists (0.906 vs 0.883, p = 0.057). After the application of CAD software, the unnecessary biopsy rate of BI-RADS categories 4 and 5 was significantly decreased (33.0% vs 11.9%, 37.8% vs 14.5%), and the malignant rate of biopsy in category 4a was significantly increased (11.6% vs 40.7%, 7.4% vs 34.9%, all p < 0.001). Conclusions CAD software on ultrasound can be used as an effective auxiliary diagnostic tool for differential diagnosis of benign and malignant breast masses and reducing unnecessary biopsy.