Background: Coronavirus disease 2019 (COVID-19) has become a pandemic. Few studies have explored the role of chest computed tomography (CT) features and severity scores for prognostic prediction. In this study, we aimed to investigate the role of chest CT severity score and imaging features in the prediction of the prognosis of COVID-19 patients. Methods: A total of 134 patients (62 recovered and 72 deceased patients) with confirmed COVID-19 were enrolled. The clinical, laboratory, and chest CT (316 scans) data were retrospectively reviewed. Demographics, symptoms, comorbidities, and temporal changes of laboratory results, CT features, and severity scores were compared between recovered and deceased groups using the Mann-Whitney U test and logistic regression to identify the risk factors for poor prognosis. Results: Median age was 48 and 58 years for recovered and deceased patients, respectively. More patients had at least one comorbidity in the deceased group than the recovered group (60% vs. 29%). Leukocytes, neutrophil, high-sensitivity C-reactive protein (hsCRP), prothrombin, D-dimer, serum ferritin, interleukin (IL)-2, and IL-6 were significantly elevated in the deceased group than the recovered group at different stages. The total CT score at the peak stage was significantly greater in the deceased group than the recovered group (20 vs. 11 points). The optimal cutoff value of the total CT scores was 16.5 points, achieving 69.4% sensitivity and 82.2% specificity for the prognostic prediction. The crazy-paving pattern and consolidation were more common in the deceased patients than those in the recovered patients. Linear opacities significantly increased with the disease course in the recovered patients. Sex, age, neutrophil, IL-2, IL-6, and total CT scores were independent risk factors for the prognosis with odds ratios of 3.8 to 8.7. Conclusions: Sex (male), older age (>60 years), elevated neutrophil, IL-2, IL-6 level, and total CT scores (>= 16) were independent risk factors for poor prognosis in patients with COVID-19. Temporal changes of chest CT features and severity scores could be valuable for early identification of severe cases and eventually reducing the mortality rate of COVID-19.