Background The aim of this study was to investigate the metabolism of the spleen, bone marrow (BM), and liver from preoperative F-18 FDG PET/CT scans for the prediction of recurrence in breast cancer. Methods We retrospectively included 153 patients diagnosed with invasive ductal carcinoma (IDC) of the breast who underwent preoperative F-18 FDG PET/CT scan and a curative operation. The mean standardized uptake value (SUV mean ) of the spleen, liver, and BM and maximum SUV (SUV max ) of primary tumors were measured. The relationships between spleen, BM, and liver metabolism and clinicopathologic parameters were evaluated, and possible prognostic parameters predicting recurrence were assessed using disease-free survival (DFS). Results Spleen SUV mean was significantly correlated with primary tumor SUV max , pathologic T (pT) stage, and histologic grade of primary tumor. BM SUV mean also showed a positive correlation with primary tumor SUV max . Spleen SUV mean were significantly associated with recurrence from binary logistic regression analysis (P = 0.004). Spleen, BM, liver, and primary tumor SUVs were all significant prognostic factors for DFS in univariate Cox regression analysis (all P<0.024). Among all PET parameters analyzed, spleen SUV mean 2.21 (P = 0.032) was in the multivariable analysis the powerful poor prognostic factor predicting DFS that was independent of other clinicopathological features like T stage (pT >2; P = 0.009) and estrogen receptor (ER) status (ER negativity; P = 0.001). Conclusion Splenic metabolism together with pT stage and ER status was an independent prognostic factor for predicting recurrence in breast cancer. Metabolic activity of reticuloendothelial system such as spleen, liver or BM on preoperative F-18 FDG PET/CT can be a meritorious imaging factor for discriminating patients with IDC that require adjunctive therapy to prevent recurrence.