Objective To determine the impact of smoking on disease-specific health care utilization and medical costs in patients with chronic non-communicable diseases (NCDs). Methods Participants were middle-aged and elderly adults with chronic NCDs from a prospective cohort in China. Logistic regressions and linear models were used to assess the relationship between tobacco smoking, health care utilization and medical costs. Results Totally, 1020 patients with chronic obstructive pulmonary disease (COPD), 3144 patients with coronary heart disease (CHD), and 1405 patients with diabetes were included in the analysis. Among patients with COPD, current smokers (beta: 0.030, 95% CI: -0.032-0.092) and former smokers (beta: 0.072, 95% CI: 0.014-0.131) had 3.0% and 7.2% higher total medical costs than never smokers. Medical costs of patients who had smoked for 21-40 years (beta: 0.028, 95% CI:-0.038-0.094) and >= 41 years (beta: 0.053, 95% CI: -0.004 beta 0.110) were higher than those of never smokers. Patients who smoked >= 21 cigarettes (beta: 0.145, 95% CI: 0.051-0.239) per day had more inpatient visits than never smokers. The association between smoking and health care utilization and medical costs in people with CHD group was similar to that in people with COPD; however, there were no significant associations in people with diabetes. Conclusion This study reveals that the impact of smoking on health care utilization and medical costs varies among patients with COPD, CHD, and diabetes. Tobacco control might be more effective at reducing the burden of disease for patients with COPD and CHD than for patients with diabetes.