Purpose: The purpose of this study was to estimate the value of addition of liver imaging to initial rectal magnetic resonance imaging (MRI) for detection of liver metastasis and evaluate imaging predictors of a high risk of liver metastasis on rectal MRI. Methods: We enrolled 144 patients who from October 2010 to May 2013 underwent rectal MRI with T2-weighted imaging (T2WI) and diffusion-weighted imaging (DWI) (b values = 50, 500, and 900 s/mm2) of the liver and abdominopelvic computed tomography (APCT) for the initial staging of rectal cancer. Two reviewers scored the possibility of liver metastasis on different sets of liver images (T2WI, DWI, and combined T2WI and DWI) and APCT and reached a conclusion by consensus for different analytic results. Imaging features from rectal MRI were also analyzed. The diagnostic performances of CT and an additional liver scan to detect liver metastasis were compared. Multivariate logistic regression to determine independent predictors of liver metastasis among rectal MRI features and tumor markers was performed. This retrospective study was approved by the Institutional Review Board, and the requirement for informed consent was waived. Results: All sets of liver images were more effective than APCT for detecting liver metastasis, and DWI was the most effective. Perivascular stranding and anal sphincter invasion were statistically significant for liver metastasis (p = 0.0077 and p = 0.0471), while extramural vascular invasion based on MRI (mrEMVI) was marginally significant (p = 0.0534). Conclusion: The addition of non-contrast-enhanced liver imaging, particularly DWI, to initial rectal MRI in rectal cancer patients could facilitate detection of liver metastasis without APCT. Perivascular stranding, anal sphincter invasion, and mrEMVI detected on rectal MRI were important imaging predictors of liver metastasis.