Online panels based on probability samples have typically relied on costly recruitment procedures involving interviewers, but, especially for short-term panels, the costs of using face-to-face or telephone recruitment may be prohibitive. On the other hand, non-probability based online panels, while much less costly, raise concerns regarding how well they represent the general population and hence, the accuracy of the estimates they produce.
One possibility for reducing the costs and complexity of recruiting a probability-based panel is to use mail as the only mode of contact. The main advantage of this approach are lower costs and much less requirements in terms of infrastructure and personnel. This also allows institutions to conduct surveys in-house to save costs and to have control over the whole procedure. The main disadvantage is that as there is no interviewer present to convince people to join or address concerns of respondents. The content and design of contact letters and incentives thus become the fundamental elements determining whether a person joins and remains in the panel. However, past research is lacking regarding how to design these elements to ensure the success of this type of survey.
To address these shortcomings, the aim of the current study is to evaluate the effects of both letter wording and incentives on participation in the first wave of the survey, enrolling in the panel, participating in subsequent waves, and possible biases in sociodemographics and key estimates. The focus is on how to inform participants about the panel component and the addition of a conditional incentive to the existing prepaid incentive.
We use data collected in a three-wave pilot experiment conducted in the context of the 2015 Swiss Electoral Study (Selects). The sample consisted of 2,700 German-speaking Swiss citizens 18 years or older living in Switzerland and included the names and addresses of the individuals, as well as basic sociodemographics. The experimental design consisted first of two groups with a different wording of the letters, one in which the panel aspect was described in detail and another where the question was addressed more vaguely. Each group was then separated into three groups with different conditional incentives. In addition to a 10CHF prepaid incentive, respondents would either be promised another 10CHF, participate in a raffle of five iPads, or receive no additional incentive as a control group.
We compare the response rates in the first wave, the proportion enrolling in the panel (i.e. providing a valid e-mail address), and attrition among the different groups. We then study the determinants of these aspects as well as effects of the different conditions on the representativity of the sample. Finally, we provide an analysis of costs regarding the different options and provide a discussion and recommendations for conducting similar surveys as well as avenues for further research.