Multi-subunit SMC complexes control chromosome superstructure and promote chromosome disjunction, conceivably by actively translocating along DNA double helices. SMC subunits comprise an ABC ATPase "head" and a "hinge" dimerization domain connected by a 49 nm coiled-coil "arm." The heads undergo ATP-dependent engagement and disengagement to drive SMC action on the chromosome. Here, we elucidate the architecture of prokaryotic Smc dimers by high-throughput cysteine cross-linking and crystallography. Co-alignment of the Smc arms tightly closes the interarm space and misaligns the Smc head domains at the end of the rod by close apposition of their ABC signature motifs. Sandwiching of ATP molecules between Smc heads requires them to substantially tilt and translate relative to each other, thereby opening up the Smc arms. We show that this mechanochemical gating reaction regulates chromosome targeting and propose a mechanism for DNA translocation based on the merging of DNA loops upon closure of Smc arms.