Objective We aim to examine changes in sexual activity and function among younger breast cancer survivors who were sexually active before diagnosis and to investigate risk factors for negative impacts on them. Methods An observational cohort study enrolled 304 premenopausal and sexually active women diagnosed with early stage breast cancer. Questionnaires were completed, and sexual activity was measured at two time points: after surgery, to assess sexual activity and function before diagnosis, and then at least 12 months after the completion of chemotherapy or endocrine therapy. For each domain of the Female Sexual Function Index, a score below 3 was classified as indicative of a sexual problem. Each sexual problem was considered to be dysfunctional if it was associated with distress. Results The median age at the last survey was 46.0 years (range: 23-57). Of the participants, 35 (11.5%) became sexually inactive after treatment. Among the 269 women who remained sexually active, 31.6% were currently experiencing sexual dysfunction, which was significantly higher compared with the frequency before diagnosis. In the multivariate logistic regression model, chemo-related menopause, thyroid dysfunction, and depression were independent risk factors for sexual inactivity. Chemo-related menopause was a significant risk factor for sexual dysfunction. Conclusions Chemo-related menopause was significantly associated with both sexual inactivity and dysfunction after treatment. Thyroid dysfunction and depression were risk factors for sexual inactivity in younger breast cancer survivors.