Introduction: Premature ventricular contractions (PVCs) were now well recognized to carry the risk of inducing left ventricular (LV) enlargement and were closely related to the cardiac autonomic nervous activity quantified by heart rate variability (HRV) analysis. However, the relationship between LV enlargement and HRV in patients with frequent PVCs is still unclear. This study aimed to investigate the risk factors and HRV for LV enlargement in patients with frequent PVCs. Methods: Patients with frequent PVCs (n = 571), whose PVC burden counts >10,000/24 h or PVC burden >10%, were recruited. Patients were divided into LV enlargement group (n = 161), defined as female left ventricular end-diastolic diameter (LVEDD) >49.8 mm or male LVEDD >54.2 mm, and LV normal group (n = 410). Two groups were compared on their clinical, electrocardiographic, and HRV parameters. Logistic regression analysis was used to predict the risk factors of LV enlargement in patients with frequent PVCs. The parameters of echocardiography, Holter monitoring, and HRV were collected after ablation. Results: There were significant differences between the patients with left enlargement and with normal LV structure, in terms of sex, left ventricular ejection fraction (LVEF), level of N-terminal pro-brain natriuretic peptide (NT-proBNP), 24-h PVC burden, with nonsustained ventricular tachycardia, multifocal PVCs, QRS duration of PVC, and values of very low-frequency power of HRV parameter (all p < 0.05). Multivariate analysis showed that female gender (odds ratio [OR] = 2.753, p < 0.001), increased NT-proBNP (OR = 1.011, p = 0.022), increased LVEF (OR = 0.292, p < 0.001), increased 24-h PVC burden (OR = 1.594, p < 0.001), increased standard deviation of all NN intervals (SDNN) (OR = 1.100, p = 0.003), increased the proportion of consecutive NN intervals that differ by more than 50 ms (pNN50) (OR = 0.844, p = 0.026) were predictors for LV enlargement in patients with frequent PVCs. 84.4% (54/64) of patients with LV enlargement at baseline had normalized their LV structure after ablation. The values of SDNN, standard deviation of the averages of NN intervals in all 5-min segments, the square root of the mean of the sum of the squares of differences between adjacent NN intervals, pNN50, low-frequency power (LF), LF/high-frequency power ratio of patients were significantly decreased after ablation (all p < 0.05). Conclusion: Female gender, increased level of NT-proBNP, lower LVEF, higher PVC burden, increased sympathetic parameters SDNN, and reduced parasympathetic parameters pNN50 were the independent risk factors of LV enlargement in patients with frequent PVCs. LV enlargement induced by PVCs can be reversible after PVC elimination by ablation. The activities of sympathetic and parasympathetic were reduced after ablation.