When making a decision, solitary animals often adjust to local conditions by using flexible evaluation and decision criteria, even though these may occasionally lead to irrationality. By contrast, collective decision making in large animal groups-such as, nest choice by emigrating ant colonies-is usually considered to rely on robust, fixed preference rules and to be immune to irrationality. Here, we show that familiarization with available nest sites prior to emigration can lead to flexible collective decisions in the house-hunting ant Temnothorax albipennis. Colonies allowed to inspect a mediocre nest site while their home nest is still intact usually develop an aversion toward that nest. We found that aversion strength was not determined by the quality of the familiar nest only but was also influenced by the quality of the home nest. As a result, nest choice in later emigrations depended strongly on the quality of the previously experienced home nest, allowing colonies to adjust to the local quality of available sites. Additionally, we found that in a worst-case scenario where the only alternatives are of even lower quality, developing an aversion toward a mediocre nest can occasionally lead to poor collective decisions. We discuss whether the observed flexibility in collective choices necessarily requires experience-dependent changes in individual decision criteria and develop a new analytical model of nest choice in house-hunting ants showing that a fixed-threshold decision strategy at the individual level can lead to experience-dependent, flexible decisions at the colony level.