Objectives To investigate the diagnostic value of contrast-enhanced computed tomography (CECT) histogram analysis in predicting the World Health Organization (WHO) grade of rectal neuroendocrine tumors (R-NETs). Materials and Methods A total of 61 (35 G1, 12 G2, 10 G3, and 4 NECs) patients who underwent preoperative CECT and treated with surgery to be confirmed as R-NETs were included in this study from January 2014 to May 2019. We depicted ROIs and measured the CECT texture parameters (mean, median, 10th, 25th, 75th, 90th percentiles, skewness, kurtosis, and entropy) from arterial phase (AP) and venous phase (VP) images by two radiologists. We calculated intraclass correlation coefficient (ICC) and compared the histogram parameters between low-grade (G1) and higher grade (HG) (G2/G3/NECs) by applying appropriate statistical method. We obtained the optimal parameters to identify G1 from HG using receiver operating characteristic (ROC) curves. Results The capability of AP and VP histogram parameters for differentiating G1 from HG was similar in several histogram parameters (mean, median, 10th, 25th, 75th, and 90th percentiles) (all p < 0.001). Skewness, kurtosis, and entropy on AP images showed no significant differences between G1 and HG (p = 0.853, 0.512, 0.557, respectively). Entropy on VP images was significantly different (p = 0.017) between G1 and HG, however, skewness and kurtosis showed no significant differences (p = 0.654, 0.172, respectively). ROC analysis showed a good predictive performance between G1 and HG, and the 75th (AP) generated the highest area under the curve (AUC = 0.871), followed by the 25th (AP), mean (VP), and median (VP) (AUC = 0.864). Combined the size of tumor and the 75th (AP) generated the highest AUC. Conclusions CECT histogram parameters, including arterial and venous phases, can be used as excellent indicators for predicting G1 and HG of rectal neuroendocrine tumors, and the size of the tumor is also an important independent predictor.