Background: Conflicting data exist regarding the association of body mass index (BMI) changes with all-cause and cardiovascular (CV) mortality. The current study investigated the association between changes in BMI and all-cause, CV, and non-CV mortality in a large cohort of middle-aged adults. Methods: A total of 379,535 adults over 40 years of age without pre-existing CV disease or cancer at baseline were enrolled to undergo a series of at least three health examinations of biennial intervals. Changes in BMI between baseline, midpoint follow-up, and final health examination during mean 9.3 years were defined according to the pattern of BMI change as follows: stable, sustained gain, sustained loss, and fluctuation. The relationship between BMI change category and mortality was assessed by multivariate Cox regression reporting hazard ratio (HR) with 95% confidence interval (95% CI). Results: During a mean follow-up of 10.7 years for mortality, 12,378 deaths occurred from all causes, of which 2,114 were CV and 10,264 were non-CV deaths. Sustained BMI gain was associated with the lower risk of all-cause (HR 0.89, 95% CI: 0.83–0.95), CV (HR 0.84, 95% CI 0.72–0.98), and non-CV mortality (HR 0.90, 95% CI 0.84–0.96) compared with stable BMI. Conversely, sustained BMI loss (HR 1.25, 95% CI 1.19–1.32) and fluctuation (HR 1.13, 95% CI 1.08–1.19) displayed a higher risk of all-cause mortality compared with stable BMI, which was mainly attributable to the increase in non-CV mortality. Conclusion: Sustained BMI gains were associated with reduced risk of all-cause, CV, and non-CV mortality in middle-aged healthy adults.