As a component of collagen II, glycosaminoglycan (GAG) has a relatively close relationship with bone metabolism. GAG and collagen II have been proven to promote connection of the bone trabecular structure. However, the exact mechanism remains unknown. In this study, we aimed to determine the concrete effect and the mechanism of GAG and collagen II on glucocorticoid-induced osteoporosis. We implanted prednisolone pellets subcutaneously in mice to mimic glucocorticoid-induced osteoporosis. GAG was administered intragastrically every day for 60 days. The results demonstrated a protective effect of GAG and collagen II on glucocorticoid-induced osteoporosis. Trabecular number and connection density increased after treatment with GAG and collagen II. We generated bone marrow-derived macrophages to explore the effect of GAG and collagen II on osteoclast differentiation. We collected cell protein and RNA in the presence of macrophage colony-stimulating factor (M-CSF) and receptor activator for nuclear factor-kappa B ligand (RANKL) and found that GAG and collagen II inhibited the NF-kappa B and MAPK pathways, thereby down-regulating osteoclast differentiation molecules such as matrix metallopeptidase 9 (MMP 9) and nuclear factor of activated T-cells, cytoplasmic 1 (NFATc-1). Our findings suggest that GAG and collagen II may have therapeutic potential of patients with glucocorticoid-induced osteoporosis in clinical settings.