The association of radiotherapy and immunotherapy has recently emerged as an exciting combination that might improve outcomes in many solid tumor settings. In the context of breast cancer, this opportunity is promising and under investigation. Given the heterogeneity of breast cancer, it might be meaningful to study the association of radiotherapy and immunotherapy distinctly among the various breast cancer subtypes. The use of biomarkers, such as tumor infiltrating lymphocytes, which are also associated to breast cancer heterogeneity, might provide an opportunity for tailored studies. This review highlights current knowledge of the association of radiotherapy and immunotherapy in the setting of breast cancer and attempts to highlight the therapeutic opportunities among breast cancer heterogeneity.