ObjectiveAccurate identification of testicular tumors through better lesion characterization can optimize the radical surgical procedures. Here, we compared the performance of different machine learning approaches for discriminating benign testicular lesions from malignant ones, using a radiomics score derived from magnetic resonance imaging (MRI).MethodsOne hundred fifteen lesions from 108 patients who underwent MRI between February 2014 and July 2022 were enrolled in this study. Based on regions-of-interest, radiomics features extraction can be realized through PyRadiomics. For measuring feature reproducibility, we considered both intraclass and interclass correlation coefficients. We calculated the correlation between each feature and the predicted target, removing redundant features. In our radiomics-based analysis, we trained classifiers on 70% of the lesions and compared different models, including linear discrimination, gradient boosting, and decision trees. We applied each classification algorithm to the training set using different random seeds, repeating this process 10 times and recording performance. The highest-performing model was then tested on the remaining 30% of the lesions. We used widely accepted metrics, such as the area under the curve (AUC), to evaluate model performance.ResultsWe acquired 1,781 radiomic features from the T2-weighted maps of each lesion. Subsequently, we constructed classification models using the top 10 most significant features. The 10 machine-learning algorithms we utilized were capable of diagnosing testicular lesions. Of these, the XGBoost classification emerged as the most superior, achieving the highest AUC value of 0.905 (95% confidence interval: 0.886-0.925) on the testing set and outstripping the other models that typically scored AUC values between 0.697-0.898.ConclusionPreoperative MRI radiomics offers potential for distinguishing between benign and malignant testicular lesions. An ensemble model like the boosting algorithm embodied by XGBoost may outperform other models.