: The presence of growing fungi in the indoor environment has been associated with the development of respiratory problems such as asthma or allergic rhinitis, as well as the worsening of respiratory pathologies. Their proliferation indoors could be a result of water leakage or inadequate ventilation. Although the factors promoting mould growth have been widely investigated in traditional dwellings, little work has been done in energy efficient dwellings. Here, the effectiveness of ventilation type, i.e., natural or mechanical, in influencing mould development was estimated in 44 recent and 105 retrofitted energy efficient dwellings. Fungi growing on surfaces were investigated in the dwellings situated in rural, peri-urban, and urban regions of Switzerland. The presence of these fungi was also investigated in bedroom settled dust. Information on building characteristics and owners’ lifestyle were collected. Significant associations were found with the level of urbanisation, the location of mouldy area in dwellings, and the diversity of fungal taxa. Dwellings in peri-urban zones showed the most frequent fungal contamination in the owners’ bedroom and the highest diversity of fungal genera among dwellings. While the urbanisation level or the ventilation type favoured no specific genus, we found marked disparities in the diversity of fungi growing on surfaces in naturally ventilated versus mechanically ventilated dwellings. Aspergillus, in particular, was a frequent surface contaminant in bedrooms with natural ventilation, but not in those mechanically ventilated. We observed a strong association between fungal growth on surfaces and the number of fungal particles counted in the settled dust of owners’ bedrooms. These results demonstrate the importance of ventilation systems in energy efficient dwellings in controlling fungal proliferation in living areas.