Background: We tested our hypothesis that the white matter network might mediate the effect of amyloid and small vessel disease (SVD) on cortical thickness and/or cognition. Methods: We prospectively recruited 232 patients with cognitive impairment. Amyloid was assessed using Pittsburgh compound B-PET. SVD was quantified as white matter hyperintensity volume and lacune number. The regional white matter network connectivity was measured as regional nodal efficiency by applying graph theoretical analysis to diffusion tensor imaging data. We measured cortical thickness and performed neuropsychological tests. Results: SVD burden was associated with decreased nodal efficiency in the bilateral frontal, lateral temporal, lateral parietal, and occipital regions. Path analyses showed that the frontal nodal efficiency mediated the effect of SVD on the frontal atrophy and frontal-executive dysfunction. The temporoparietal nodal efficiency mediated the effect of SVD on the temporoparietal atrophy and memory dysfunction. However, Pittsburgh compound B retention ratio affected cortical atrophy and cognitive impairment without being mediated by nodal efficiency. Conclusions: We suggest that a disrupted white matter network mediates the effect of SVD, but not amyloid, on specific patterns of cortical atrophy and/or cognitive impairment. Therefore, our findings provide insight to better understand how amyloid and SVD burden can give rise to brain atrophy or cognitive impairment in specific patterns.