Memory and effector T cells have the potential to counteract cancer progression, but often fail to control the disease, essentially because of three main stumbling blocks. First, clonal deletion leads to relatively low numbers or low-to-intermediate T cell receptor (TCR) affinity of self/tumor-specific T cells. Second, the poor innate immune stimulation by solid tumors is responsible for inefficient priming and boosting. Third, T cells are suppressed in the tumor microenvironment by inhibitory signals from other immune cells, stroma and tumor cells, which induces T cell exhaustion, as demonstrated in metastases of melanoma patients. State-of-the-art adoptive cell transfer and active immunotherapy can partially overcome the three stumbling blocks. The reversibility of T cell exhaustion and novel molecular insights provide the basis for further improvements of clinical immunotherapy.