Despite a significant amount of data on incidence and therapy of immune-related adverse events affecting virtually all organ systems, the potential impact of immune checkpoint inhibitors (ICIs) on gonadal function has not been sufficiently studied. The limited evidence available suggests that ICI-related primary hypogonadism due to orchitis as well as secondary hypogonadism due to hypophysitis are a potential risk for infertility. A systematic investigation of gonadal function under ICIs is warranted given the increasing application of ICIs in the adjuvant setting, among young adults and children and the possible influence of sex hormone levels on the efficacy and toxicity of ICIs.