The literature concerning the settlement choices of wealthy taxpayers has emphasised differences in tax rates and proximity to other wealthy individuals respectively. In this article, we explore the significance of these explanations through a mapping of wealthy Swedes living in Switzerland in the 1960s–1980s period. In the decades after the Second World War, Switzerland became a popular destination for foreign tax emigrants, while Swedish taxation on income and wealth became more progressive. We base our study on never-used-before records of all Swedes living in Switzerland during this period, combined with archive material and other sources on the tax regimes of the Swiss cantons. Our findings suggest that both the tax rates and the proximity to other wealthy individuals—and opportunities for social and business networking—can explain the settlement of wealthy Swedish expatriates in Switzerland. Overall, a large proportion of the Swedish emigration to Switzerland was related to tax rates. More specifically, Swedes settled in cantons with comparatively low-tax rates and/or with lump-sum taxation of wealthy foreigners. Furthermore, the wealthy Swedes were clustered in a few particular cantons in which a large proportion of well-off people lived, which suggests that they also favoured living close to other wealthy expatriates. The study contributes to the research on the settlement choices of the wealthy and on tax avoidance and evasion.