Expanding and intensifying agriculture has led to a loss of soil carbon. As agroecosystems cover over 40% of Earth’s land surface, they must be part of the solution put in action to mitigate climate change. Development of efficient management practices to maximize soil carbon retention is currently limited, in part, by a poor understanding of how plants, which input carbon to soil, and microbes, which determine its fate there, interact. Here we implement a diversity gradient by intercropping undersown species with barley in a large field trial, ranging from one to eight undersown species. We find that increasing plant diversity strengthens positive associations within the rhizosphere soil microbial community in relation to negative associations. These associations, in turn, enhance community carbon use efficiency. Jointly, our results highlight how increasing plant diversity in agriculture can be used as a management strategy to enhance carbon retention potential in agricultural soils.