Glaciers and ice sheets erode mountains and produce vast quantities of sediments that are delivered to rivers and oceans, impacting global sediment and biogeochemical balances. Therefore, understanding how the production of sediments by glacial erosion has evolved in the past, and will evolve in a changing climate, is increasingly important. In this Review, we examine the processes that control the magnitude and timing of glacial erosion of mountains, and how models can be used to reconstruct processes during the development of mountains. Field observations reveal the important role of sliding on the erosion rate, which provide an empirical basis to explain the glacial buzzsaw and the impact of late Cenozoic cooling on erosion rates. Glacial erosion is also expected to evolve in the context of anthropogenic climate warming, as both glacier sliding and the input of meltwater related to thinning and retreat of ice will change, with large effects on downstream ecosystems and global biogeochemical cycles. Thus, the mechanics and impacts of glaciers on sediment production warrant more research, especially in regions experiencing rapid warming. Above all, there is a need for better monitoring of how erosion rates changed over the last decades and will evolve in the future.