Background: This study aimed to evaluate the influences of SARS-CoV-2 infection on semen parameters and investigate the impact of the infection on in vitro fertilization (IVF) outcomes. Methods: This retrospective study enrolled couples undergoing IVF cycles between May 2020 and February 2021 at Tongji Hospital, Wuhan. Baseline characteristics were matched using propensity score matching. Participants were categorized into an unexposed group (SARS-COV-2 negative) and exposed group (SARS-COV-2 positive) based on a history of SARS-CoV-2 infection, and the populations were 148 and 50 after matching, respectively. IVF data were compared between the matched cohorts. Moreover, semen parameters were compared before and after infection among the infected males. The main measures were semen parameters and IVF outcomes, including laboratory and clinical outcomes. Results: Generally, the concentration and motility of sperm did not significantly differ before and after infection. Infected males seemed to have fewer sperm with normal morphology, while all values were above the limits. Notably, the blastocyst formation rate and available blastocyst rate in the exposed group were lower than those in the control group, despite similar mature oocytes rates, normal fertilization rates, cleavage rates, and high-quality embryo rates. Moreover, no significant differences were exhibited between the matched cohorts regarding the implantation rate, biochemical pregnancy rate, clinical pregnancy rate, or early miscarriage rate. Conclusions: The results of this retrospective cohort study suggested that the semen quality and the chance of pregnancy in terms of IVF outcomes were comparable between the males with a history of SARS-CoV-2 infection and controls, although a decreased blastocyst formation rate and available blastocyst rate was observed in the exposed group, which needs to be reinforced by a multicenter long-term investigation with a larger sample size.