In Wave 7, this set of experiments reflects two different, though related, research questions. The first, most general, research question is whether and how the direction of a response scale affects survey responses. If a scale runs from positive to negative (e.g., “strongly agree” to “strongly disagree” or “excellent” to “poor”) versus from negative to positive (e.g., “strongly disagree” to “strongly agree” or “poor” to “excellent”), should we expect different responses from survey respondents? The empirical evidence is mixed. A second research question builds on this first and is more specific about the theoretical underpinnings. That is, on social surveys researchers routinely ask questions on sensitive topics: e.g. health and wellbeing, drug and alcohol consumption, sleeping problems, to name a few. Survey questions of this kind are prone to social desirability bias. In a recent study, in which respondents’ eye-movements were traced and respondents were found to not read response options presented at the bottom of the scale (Kaminska and Foulsham 2016). The second research question is therefore: can we improve survey answers to sensitive questions if the scale starts with socially unpleasant response options which might otherwise appear at the bottom of response option lists?
Households within IP7 PSUs were allocated to treatment such that all individuals in the household received the same version of questions, as did all split-off households.
In Wave 8, both motivational message and response option reversal experiments are repeated. However for the response option experiment in Wave 8, half of households will be randomly re-allocated the opposite treatment.
Controlling variables on record hhsamp:
Table: Control variable for scale ordering experiment
| ff_reversew7 | ff_reversew8 |
| 1 = Version A | 1 = Version A |
| 2 = Version B | |
| 2 = Version B | 1 = Version A |
| 2 = Version B |
The variables affected in the Wave 7 and Wave 8 questionnaires are: job satisfaction (jbsat_a7, jbsat_b7), general health (scsf1_a, scsf1_b), all items in the self-completion GHQ general health module (scghqa_a, scghqa_b,…, scghql_a, scghql_b), and all items in the self-completion satisfaction module (sclfsat1_a, sclfsat1_b, …, sclfsato_a, sclfsato_b).
This experiment was combined with a motivational message read or displayed to a random half of respondents. The motivational message read
“In order for your answers to be most helpful to us, it is important that you try to be as thoughtful as you can. Since we need complete and accurate information from this research, we hope you will think hard to provide the information we need.”
The variable controlling who received the motivational message is ff_motivw7 on record hhsamp:
Group 1 – Receives message
Group 2 – Does not receive message
