Serum inflammatory markers are used in the prognostication of colorectal cancer (CRC); however, the corresponding role of positron emission tomography (PET)-derived inflammatory markers remains unclear. This study aimed to investigate the prognostic value of 18F-fluorodeoxyglucose (FDG) uptake in the bone marrow and spleen of patients with CRC and evaluate the relationship between FDG uptake estimates in these organs and serum inflammatory markers. In total, 411 patients who underwent preoperative FDG PET/computed tomography (CT) within 1 month of surgery were enrolled. The mean standardized uptake values of the bone marrow and spleen were normalized to the value of the liver, thereby generating bone marrow-to-liver uptake ratio (BLR) and spleen-to-liver uptake ratio (SLR) estimates. The value of BLR and SLR in predicting overall survival (OS) was assessed using the Cox proportional hazards model. The correlation between BLR or SLR and neutrophil-to-lymphocyte ratio (NLR) was evaluated. The predictive accuracy of BLR alone and in combination with SLR was compared using the integrated area under the receiver operating characteristic curves (iAUC). In the univariate analysis, BLR (> 1.06) and SLR (> 0.93) were significant predictors of OS. In the multivariate analysis, BLR was an independent predictor of OS (hazard ratio = 5.279; p < 0.001). Both BLR and SLR were correlated with NLR (p < 0.001). A combination of BLR and SLR was better than BLR alone at CRC prognostication (iAUC, 0.561 vs. 0.542). FDG uptake estimates in the bone marrow and spleen may be useful imaging-derived biomarkers of systemic inflammation, supporting CRC prognostication.