Cytotoxic CD8 T cells mediate immunity to pathogens and they are able to eliminate malignant cells. Immunity to viruses and bacteria primarily involves CD8 T cells bearing high affinity T cell receptors (TCRs), which are specific to pathogen-derived (non-self) antigens. Given the thorough elimination of high affinity self/tumor-antigen reactive T cells by central and peripheral tolerance mechanisms, anti-cancer immunity mostly depends on TCRs with intermediate-to-low affinity for self-antigens. Because of this, a promising novel therapeutic approach to increase the efficacy of tumor-reactive T cells is to engineer their TCRs, with the aim to enhance their binding kinetics to pMHC complexes, or to directly manipulate the TCR-signaling cascades. Such manipulations require a detailed knowledge on how pMHC-TCR and co-receptors binding kinetics impact the T cell response. In this review, we present the current knowledge in this field. We discuss future challenges in identifying and targeting the molecular mechanisms to enhance the function of natural or TCR-affinity optimized T cells, and we provide perspectives for the development of protective anti-tumor T cell responses.