Mesial temporal lobe epilepsy (MTLE), one of the most common types of refractory focal epilepsy, has shown white matter abnormalities both within and beyond the temporal lobe. In particular, the white matter abnormalities in the ipsilateral hemisphere are more obvious than those in the contralateral hemisphere in MTLE, that is, the abnormalities present asymmetrical characteristics. However, very few studies have characterized the white matter microstructure asymmetry in MTLE patients specifically. Thus, we performed diffusion tensor imaging (DTI) to investigate the white matter microstructure asymmetries of patients with MTLE with unilateral hippocampal sclerosis (MTLE-HS). We enrolled 25 MTLE-HS (left MTLE-HS group, n = 13; right MTLE-HS group, n = 12) and 26 healthy controls (HC). DTI data were analyzed by tract-based spatial statistics (TBSS) to test the hemispheric differences across the entire white matter skeleton. We also conducted a two-sample paired t-test for 21 paired region of interests (ROIs) parceled on the basis of the ICBM-DTI-81 white-matter label atlas of bilateral hemispheres to test the hemispheric differences. An asymmetry index (AI) was calculated to further quantify the differences between the left and right paired-ROIs. It was found that the asymmetries of white matter skeletons were significantly lower in the MTLE-HS groups than in the HC group. In particular, the asymmetry traits were moderately reduced in the RMTLE-HS group and obviously reduced in the LMTLE-HS group. In addition, AI was significantly different in the RMTLE-HS group from the LMTLE-HS or HC group in the limbic system and superior longitudinal fasciculus (SLF). The current study found that the interhemispheric white matter asymmetries were significantly reduced in the MTLE-HS groups than in the HC group. The interhemispheric white matter asymmetries are distinctly affected in left and right MTLE-HS groups. The differences in AI among RMTLE-HS, LMTLE-HS, and HC involved the limbic system and SLF, which may have some pragmatic implications for the diagnosis of MTLE and differentiating LMTLE-HS from RMTLE-HS.