Sorting nexin17 (SNX17) is a member of the sorting nexin family, which plays a crucial role in endosomal trafficking. Previous research has shown that SNX17 is involved in the recycling or degradation of various proteins associated with neurodevelopmental and neurological diseases in cell models. However, the significance of SNX17 in neurological function in the mouse brain has not been thoroughly investigated. In this study, we generated Snx17 knockout mice and observed that the homozygous deletion of Snx17 (Snx17-/-) resulted in lethality. On the other hand, heterozygous mutant mice (Snx17+/-) exhibited anxiety-like behavior with a reduced preference for social novelty. Furthermore, Snx17 haploinsufficiency led to impaired synaptic transmission and reduced maturation of dendritic spines. Through GST pulldown and interactome analysis, we identified the SRC kinase inhibitor, p140Cap, as a potential downstream target of SNX17. We also demonstrated that the interaction between p140Cap and SNX17 is crucial for dendritic spine maturation. Together, this study provides the first in vivo evidence highlighting the important role of SNX17 in maintaining neuronal function, as well as regulating social novelty and anxiety-like behaviors.© 2023. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.