How much a randomized controlled trial (RCT) sample is representative of or differs from its source population is a challenging question, with major implications for generalizability of results. It is particularly crucial for freely-available web-based interventions tested in RCTs since they are designed to reach broad populations and could increase health disparities if they fail to reach the more vulnerable individuals. We assessed the representativeness of a sample of participants in a primary/secondary prevention web-based brief intervention RCT in relation to its source population. Then we compared those recruited to those not recruited in the RCT.
There is a mandatory army recruitment process in Switzerland at age 19 for men. Between August 2010 and July 2011, 12,564 men (source population) attended two recruitment centers and were asked to answer a screening questionnaire on alcohol use. Among 11,819 (94.1%) who completed it, 7027 (59.5%) agreed to participate in a longitudinal cohort study with regular assessments. In 2012, these participants were invited to a web-based brief intervention RCT. Participation was not dependent on the presence or quantity of alcohol use. We assessed the representativeness of the RCT sample in relation to the source population and compared participants recruited/not recruited in the RCT with respect to education level and alcohol use.
The RCT sample differed from the source population: individuals 20 and over were significantly less represented (34.3% vs 37.9%, p = 0.006), as were those with lower education level (58.6% vs 63.0%, p = 0.0009). The prevalence of any alcohol use was higher in the RCT population (92.3% vs 90.6%, p = 0.03) but unhealthy alcohol use was less represented (37.1% vs 43.2%, p < 0.0001). Differences on alcohol use measures and education were similarly found when those recruited in the RCT were compared to those who were not, including in a multivariable model, showing independent associations between less unhealthy alcohol use and higher education and recruitment in the RCT.
RCT participants differed from other members of the source population, with those participating in the RCT having higher prevalence of any alcohol use but lower levels of consumption and lower prevalence of indicators of unhealthy alcohol use. Individuals with higher education were overrepresented in the RCT sample. Selection bias may exist at both ends of the drinking spectrum and individuals with some indicators of greater vulnerability were less likely to participate. Results of web-based studies may not adequately generalize to the general population.Trial registration: The trial was registered at current controlled trials: ISRCTN55991918.