We investigated genome folding across the eukaryotic tree of life. We find two types of three-dimensional (3D) genome architectures at the chromosome scale. Each type appears and disappears repeatedly during eukaryotic evolution. The type of genome architecture that an organism exhibits correlates with the absence of condensin II subunits. Moreover, condensin II depletion converts the architecture of the human genome to a state resembling that seen in organisms such as fungi or mosquitoes. In this state, centromeres cluster together at nucleoli, and heterochromatin domains merge. We propose a physical model in which lengthwise compaction of chromosomes by condensin II during mitosis determines chromosome-scale genome architecture, with effects that are retained during the subsequent interphase. This mechanism likely has been conserved since the last common ancestor of all eukaryotes.