The wave 14 refreshment sample was part of a larger pilot in preparation for recruiting a refreshment sample for the main Understanding Society study. The pilot included a clustered sample that was added to the Innovation Panel data as a new refreshment sample (identified by hhorig=19), and an unclustered sample whom we do not plan to contact for future waves. The pilot was conducted as a web survey and included a series of experiments with the protocols for recruiting a new sample for a web survey. The experimental allocation variables are included in the wave 14 Innovation Panel data, although the data exclude the unclustered part of the pilot sample. The randomised allocation variables for these experiments are set to Inapplicable (-8) for the continuing samples, as they only applied to the wave 14 refreshment sample. For more details and results of the experiments from the full pilot sample see (Williams et al 2022).
Prenotification letters and reminders
This experiment varied whether sample households were sent a prenotification letter before the letter inviting them to the survey, and whether they were sent two or three reminders if they did not complete the survey. The variable n_ff_prenotcon on record n_hhsamp_ip controls this experiment:
1 Prenotification, invitation letter, first reminder, second reminder
2 Prenotification, invitation letter, first reminder, second reminder, third reminder
3 Invitation letter, first reminder, second reminder, third reminder
Logo/branding on outside of envelope
This experiment tested the use of different logos on the envelopes sent to the refreshment sample households. A third of households received envelopes with the logo of the Institute for Social and Economic Research (ISER) as well as the logo of the University of Essex; a third received the University of Essex logo only; and the control group received a blank envelope without logos. This experiment is controlled by the variable n_ff_logocon on record n_hhsamp_ip:
1 Logo is ISER AND University of Essex
2 Logo is University of Essex only
3 Blank (No logo)
This experiment tested the effect of including a gift incentive in the invitation letter. The gift consisted of branded Post-it notes that featured the Understanding Society logo as well as branded Post-it notes that included the Understanding Society logo and an encouraging message (“Help shape the decisions that affect people like you… take part in the study today.”). The control group received no gift incentive. 1/3 of households were allocated to each group. This experiment is controlled by the variable n_ff_giftcon on record n_hhsamp_ip:
1 Receive Post-its with invitation letter with message and logo
2 Receive Post-its with invitation letter with logo only
3 No Post-its
Explain the longitudinal nature of the survey
Half the sample were informed of the longitudinal nature of the study in the leaflet sent with the invitation letter (“The study involves one survey every 12 months. Your information is most valuable when we learn how it changes over time, so we will ask you to complete the study every year until you choose to leave.”). The other half were told at the end of their interview. This experiment is controlled by the variable n_ff_hhoriglong on record n_hhsamp_ip:
1 Informed it is a longitudinal survey in information leaflet
2 Not informed it is a longitudinal survey in information leaflet
Early-bird’ incentive and deadline
Half the sample were offered £30 if they completed the survey online within a five-week deadline. The other half were offered £20 for completing the survey, plus £10 if they completed it within two weeks. The ‘early bird’ deadline was mentioned in all mailings. This experiment is controlled by the variable n_ff_earlybird on record n_hhsamp_ip:
1 Two week time frame to complete (£20 plus £10 if early)
2 Five week time frame to complete (£30 at end)
This experiment tested different ways of informing the refreshment sample that the survey requires involvement from the whole household. Half of the sample were told up front in the letter that the study is a household survey. The other half were not told in the letter but informed after the first person in the household logged into the survey and completed the household grid. This experiment is controlled by the variable n_ff_toldallad on the record n_hhsamp_ip:
1 Not told in letter (only in HH grid)
2 Told in letter
Collecting email addresses of household members
This experiment tested asking the first person who completed the survey, and was therefore asked to complete the household grid, for the email addresses of all other adult household members. This would enable us to contact the other household members by email as well as by letter. The emails collected in this way were, however, not used; this was to test the feasibility of requesting these contact details. This experiment is controlled by the variable n_ff_ohhemail on the record n_hhsamp_ip:
1 Do not ask for email addresses of other HH members
2 Do ask for email addresses of other HH members
Include encouraging messages during the web survey to signal progress
Half the sample were shown encouraging messages part-way through the survey, the other half were not shown these messages. The following messages were used for all respondents in the treatment group and inserted in different locations in the questionnaire: “We’re going to ask you some important questions about money. We know it’s a sensitive subject so we want to remind you that all your answers will be kept safe and anonymous.” “Thank you for all the information you’ve shared so far. We will keep it safe.” “We’re going to ask some sensitive questions, so we just want to remind you that your responses will be anonymised, and no one will know it’s you.” “We appreciate some of those questions may have been difficult to answer, so thank you for sharing. Now keep going – you’re not too far from the end of the survey.” This experiment is controlled by the variable n_ff_encourmes on record n_hhsamp_ip:
1 Exposed to the encouraging messages
2 Not exposed to the encouraging messages
