Semen quality represents a compelling factor for fertility, and delineating the normal values has proven difficult. In the last four decades, several authors have reported a noticeable decline in sperm parameters. Also, studies investigating 'time to pregnancy' have shown that fecundity begins to be reduced when sperm numbers decrease below 30 million, even though according to the 6th edition of the WHO manual, the normal value is currently 16 million/mL or 39 million per ejaculate. There exists sufficient data to suggest a decline in sperm counts over time, even though the clear reason for this adverse trend is not well established, but some associations have been hypothesised, such as maternal smoking during pregnancy. Additional potential factors have yet to be fully illustrated but involve poor diet, increased obesity, and exposure to environmental toxins. Moreover, the change in environmental conditions and more common exposure to endocrine-disrupting chemicals (EDCs), such as pesticides and herbicides, as well as bisphenol A, phthalates, polychlorinated biphenyls, and heavy metals, starting from prenatal life and continuing into adulthood, may exhibit probable features explaining the reduction in sperm parameters. Therefore, the main goal of this narrative review is to furnish an overview of the possible effects of exposure to EDCs on testicular function and spermatogenesis and, also, to summarise the evidence regarding a decrease in sperm quality and examine its potential consequences.