Evidence shows that people’s preference for risk changes considerably when measured using different methods, which led us to question whether the common practice of using a single behavioural elicitation method (EM) reflects a valid measure. The present study addresses this question by examining the across-methods consistency of observed risk preferences in 1,507 healthy participants using six EMs. Our analyses show that risk preferences are not consistent across methods when operationalized on an absolute scale, a rank scale or the level of model parameters of cumulative prospect theory. This is at least partly explained by the finding that participants do not consistently follow the same decision strategy across EMs. After controlling for methodological and human factors that may impede consistency, our results challenge the view that different EMs manage to stably capture risk preference. Instead, we interpret the results as suggesting that risk preferences may be constructed when they are elicited, and different cognitive processes can lead to varying preferences.