Esophageal cancer is a highly invasive tumor with a poor prognosis. Lymphocytes play an important role in systemic immune responses, but their role in cancers varies depending on the specific tumor microenvironment. The aim of this study was to provide evidence for tumor-infiltrating lymphocytes (TILs) as a prognostic biomarker in esophageal squamous cell carcinoma. TIL analysis was retrospectively performed on full-face hematoxylin and eosin-stained sections from 127 patients. A majority (92.6%) of tumors had at least 10% stromal TILs (sTILs) (range, 10%-90%), and 84.3% of cancers had at least 10% intraepithelial TILs (iTILs) (range, 10%-40%). Multivariate analysis showed progressively better overall survival (P < 0.001, hazard ratio = 0.968, 95% confidence interval 0.955-0.981) and disease-free survival (P = 0.005, hazard ratio = 0.982, 95% confidence interval 0.970-0.995) in patients with higher sTILs. Marginal increases in overall survival and disease-free survival were found in the higher iTILs cohort versus the lower iTILs cohort, but the difference was not significant. In conclusion, in addition to tumor stage increasing stromal lymphocytic infiltration is an independent prognostic factor for esophageal squamous cell carcinoma treated by radical resection.