Background The management of diabetes mellitus-induced erectile dysfunction (DMED) is progressively becoming tricky due to the surge in the number of patients and the poor efficiency of phosphodiesterase type 5 inhibitors in DMED. Paeonol (Pae), as a traditional Chinese medicine, has been more and more widely used in the treatment of diabetic complications. However, whether Pae could be a potential therapeutic drug of DMED needs to be further evaluated. Objectives To investigate the pharmacological effect and possible mechanism of Pae in the treatment of DMED. Methods Intraperitoneal streptozotocin injection and an apomorphine test were used to construct the model of DMED. Seventeen DMED rats were divided into two groups: DMED group (n = 8) and DMED+Pae group (Pae; 100 mg/kg/d; oral administration; n = 9). In addition, there were still 10 normal age-matched male rats as control group. Four weeks later, the cavernous nerve electric stimulation was carried out to measure the erectile response. Moreover, the corpus cavernosum smooth muscle cells (CCSMCs) were primarily isolated and exposed to high glucose (HG) stimulation, Pae treatment and glycyrrhizin (GL; the selective inhibitor of HMGB1). After an incubation for 1 week, the CCSMCs were harvested for detection. Results The impairment of erectile function was observed in DMED rats compared with control samples, accompanied by the upregulation of HMGB1/RAGE/NF-kappa B Pathway. The lower nitric oxide and cGMP level and the higher level of inflammation, fibrosis, and apoptosis were also observed in DMED rats. It showed contrast that Pae treatment could improve the erectile function, as well as histologic alteration and related molecular changes. In addition, Pae could downregulate the HMGB1/RAGE/NF-kappa B pathway to regulate the apoptosis and inflammation levels of CCSMCs in high-glucose conditions, which is similar to the results of GL treatment. Conclusion Pae alleviated ED in DMED rats, likely by inhibiting HMGB1/RAGE/NF-kappa B Pathway, inflammatory, apoptosis, and fibrotic activity, and moderating endothelial dysfunction. Our study provide evidence for a potential new therapy for DMED.