Magnetic resonance imaging (MRI) is routinely used for preoperative tumor staging and to assess response to therapy in rectal cancer patients. The aim of our study was to evaluate the accuracy of MRI based restaging after neoadjuvant chemoradiotherapy (CRT) in predicting pathologic response. This multicenter cohort study included adult patients with histologically confirmed locally advanced rectal adenocarcinoma treated with neoadjuvant CRT followed by curative intent elective surgery between January 2014 and December 2019 at four academic high-volume institutions. Magnetic resonance tumor regression grade (mrTRG) and pathologic tumor regression grade (pTRG) were reviewed and compared for all the patients. The agreement between radiologist and pathologist was assessed with the weighted k test. Risk factors for poor agreement were investigated using logistic regression. A total of 309 patients were included. Modest agreement was found between mrTRG and pTRG when regression was classified according to standard five-tier systems (k = 0.386). When only two categories were considered for each regression system, (pTRG 0-3 vs pTRG 4; mrTRG 2-5 vs mrTRG 1) an accuracy of 78% (95% confidence interval [CI] 0.73-0.83) was found between radiologic and pathologic assessment with a k value of 0.185. The logistic regression model revealed that "T3 greater than 5 mm extent" was the only variable significantly impacting on disagreement (OR 0.33, 95% CI 0.15-0.68, P = .0034). Modest agreement exists between mrTRG and pTRG. The chances of appropriate assessment of the regression grade after neoadjuvant CRT appear to be higher in case of a T3 tumor with at least 5 mm extension in the mesorectal fat at the pretreatment MRI.