Job loss is widely known to lead to a substantial decrease in workers' subjective well-being. Functionalist theories explain this fact by arguing that the fundamental needs that work fulfills are absent during unemployment. Recent evidence from longitudinal studies however contradicts this approach, showing that workers who find a new job do not fully regain their former level of well-being upon reemployment. Therefore other mechanisms must be at work. We suggest that changes in social or economic domains of workers' lives - triggered by job displacement - lead to the observed changes in well-being. Drawing on a unique data set from a survey of workers displaced by plant closure in Switzerland after the financial crisis of 2008, our analysis confirms the previous result that finding a job after displacement does not completely restore workers' pre-displacement level of well-being. The factors that best explain this outcome are changes in social domains, notably changes in workers' job - related social status and their relationships to friends. This result provides valuable insights about the long lasting scars job displacement leaves on workers' lives.