Elevated levels of item non-response in web surveys are a concern. To see whether the incidence of item non-response can be reduced, Wave 6 contained an experiment on 6 questions which showed relatively high levels of item non-response in past waves. Wave 6 web respondents were assigned to one of two experimental treatments, or to a control group. The control group received the standard non-response protocol. The routine approach for accessing non-response codes is to not present the codes on the screen presenting the question to respondents but then if the respondent attempts to skip the question without answering codes for “Don’t know” and “Rather not answer” appear in blue with a message stating “You forgot to answer this question”. Respondents must select an answer at this point before moving to the next screen.
The first experimental treatment received a re-active prompt for item non-response which altered the non-response message. The altered message read, “If possible, please provide an answer to this question as this is one of the key questions in this study. Please be assured that the information you give us will be treated confidentially.” The same set of response categories appeared, but without specific item non-response codes. If the respondent attempted to skip the question, the item non-response codes appeared, and respondents were forced to pick an answer.
The second experimental treatment replicated the standard non-response protocol. However, respondents who provided a “Don’t know” or “Refused” code at the tested items received a set of follow-up questions at the end of the questionnaire. These questions enquired about any difficulties in answering the questionnaire, confirming who answered the questionnaire, and re-presented the questions to which the respondent answered “Don’t know” or “Refused”. At this re-presentation of items, respondents were thanked for their participation, informed that they had not answered certain key questions, told the importance of these questions, told that these are the last questions and asked to complete them before finishing.
Households within PSUs were randomly allocated to the two treatments and control in equal proportion, one-third in each group. However, the experiment only applied to respondents interviewed via web and this mode control was implemented in question filtering within the survey instrument.
The controlling variable is item f_ff_itemnonw6 on record f_hhsamp_ip. The groups are:
1 Standard procedure
2 Prompt for item non-response
3 Follow-up questions at end of questionnaire
Affected questionnaire items are f_plbornuk, f_mstatsam, f_paygl, f_basrate_cawi_1 to f_basrate_cawi_3, f_jsprf, f_fiyrdia, f_paygl, f_webexp1_code to f_webexp3_code, f_webwho1 to f_webwho16, f_webnrnum, f_paygl_fu, f_payglwc_fu, f_fiyrdia_fu, f_jsprf_fu, f_plbornuk_fu, f_basrate_fu, f_mstatsam_fu, and f_webincwhy on record f_indresp_ip.
The variables F_variable_CAWI_1, F_variable_CAWI_2, F_variable_CAWI_3 and F_variable_FU correspond to the serial order of attempts asking the question, such that F_variable_CAWI_3 is the final time for the initial query, and the only attempt with opt-out options available for each question. These variables correspond to the responses in the three conditions in the following way:
f_variable_cawi_1= The first time the question was asked in the motivational statement condition.
f_variable_cawi_2= The second time the question was asked in the motivational statement condition and the first time the question was asked in the control and follow-up conditions.
f_variable_cawi_3 =The third time the question was asked in the motivational statement condition and the second time the question was asked in the control and follow-up conditions. The only time the opt-out options are available.
The only way to identify this is to control using f_ff_itemnonw6 on file f_hhsamp.
If the respondent does not answer or selects an opt-out option, this response is coded in an entirely different variable (i.e. not f_variable_cawi_3). These are all prefixed with DKRF, but have numbers that follow that identify it for specific variables. These are as follows
| Basrate= | DKRF34 |
| FiYrDIA= | DKRF41 |
| JsPrf= | DKRF8 |
| Mstatsam= | DKRFY1 |
| Paygl= | DKRF31 |
| Paygwc= | DKRF31 |
| Plbornuk= | DKRF13 |
Note that these DKRF variables are coded as the positive values 1 and 2 instead of -1 and -2 for “Don’t Know” and “Refused”, respectively.
Respondents in the follow-up condition who have a value in DKRF variables are asked the question in the follow-up section, coded in f_variable_fu. However, unlike f_variable_cawi_3, missing values can be recorded in this variable.
Using the experimental variables the main variable is coded. Important to note is that the default for all variables is -2 (“Refused”) and only changed if some other answer is given. The backcoding to the main variable is done following this logic:
f_variable = -2
IF DKRF[N]=1 THEN f_variable=-1
ELSE IF f_variable _CAWI_3>0 THEN f_variable= f_variable _CAWI_3
ELSE IF f_variable _CAWI_2>0 THEN f_variable= f_variable _CAWI_2
ELSE IF f_variable _CAWI_1>0 THEN f_variable= f_variable _CAWI_1
f_variable_fu is not backcoded to the main variable, so the main variable only constitutes the first set of queries.
Error in f_fiyrdia and f_fiyrdia_fu:
An error occurred in coding of f_fiyrdia and the implementation of f_fiyrdia_fu. This occurred because of the above logic. In the above logic the experimental variables are backcoded to the main variable only if the response is greater than 0.
However, the majority of respondents has and gave a zero value to this question. Hence these answers were not backcoded and responses remained incorrectly coded as -2, the default.
Due to this, everyone in the follow-up condition who gave an answer of zero were also coded as a refusal, and asked f_fiyrdia_fu even though they provided a substantive answer.
These variables can be corrected by using the provided experimental variables, correcting the above logic to greater or equal to 0.
