Using density functional theory slab calculations we have investigated the effect of surface passivation on vacancy and interstitial annihilation on the Si(001) surface. We find that interstitials and vacancies are both stabilized at the clean and H-/Cl-terminated surfaces, with an energy gain of about 2-3 eV relative to those at the fifth subsurface layer where the surface effect becomes insignificant. This suggests that the Si surface is an effective sink for vacancies and interstitials, irrespective of surface passivation. However, our calculations show that the stability of vacancies and interstitials within the topmost three subsurface layers is greatly influenced by surface passivation, due to (i) strong interaction with surface Si dangling orbitals and (ii) rearrangement of surface atoms in the clean surface case. The chemical effect of adsorbates itself appears to be unimportant in the defect surface annihilation.