This experiment contained a number of questions originally described and analyzed by Schuman and Presser (1981), as well as some new variants. The purpose of these questions was to examine whether the experiments can be replicated decades later and across countries. The differences across countries may include varying cultural and conversational norms, which may lead to different response distributions. Respondents were asked a total of 20 questions in this experiment. Each question has between 2 – 4 versions, controlled by the variables below. Each question set has its own controlling variable which is independent of all other experiments including each other.
Each individual was allocated to unique treatment groups for each set of response order question experiments with controlling variables listed below under “Controlling variables”. Unlike other experiments — where allocation to treatment normally occurs in advance of fieldwork — allocation to experimental treatment occurred post enumeration via “SPSS Data Collection”, the software used by TNS BMRB to do computer assisted interviewing. All controlling variables were returned on the data as ordinary survey variables.
Table: Variables used in response order experiment
| Controlling variables (on record indresp): | Variables affected in the IP7 questionnaire: all variables in the “Response Option” module on record indresp: |
| oilsupw7 – Coded 1-4 : Version A through D | oilsup_a, oilsup_b, oilsup_c, oilsup_d |
| oilcompw7 – Coded 1-4 : Version A through D | oilcomp_a, oilcomp_b, oilcomp_c, oilcomp_d |
| adqhousw7 – Coded 1-4 : Version A through D | adqhous_a, adqhous_b, adqhous_c, adqhous_d |
| insocondw7 – Coded 1-4 : Version A through D | insocond_a, insocond_b, insocond_c, insocond_d |
| jobsw7 – Coded 1-4 : Version A through D | jobs_a, jobs_b, jobs_c, jobs_d |
| womenpolw7 – Coded 1-4 : Version A through D | womenpol_a, womenpol_b, womenpol_c, womenpol_d |
| saygovw7 – Coded 1-2 : Version A and B | govcomp_a, govcomp_b |
| churchw7 – Coded 1-2 : Version A and B | churches_a, churches_b |
| avtempw7 – Coded 1-2 : Version A and B | avtemp_a, avtemp_b |
| noopinw7 – Coded 1-4 : Version A through D | courts_a, courts_b, courts_c, courts_d, ldsmrt_a, ldsmrt_b, ldsmrt_c, ldsmrt_d, ldcrkd_a, ldcrkd_b, ldcrkd_c, ldcrkd_d |
| fuelshw7 – Coded 1-4 : Version A through D | fuelsh_a, fuelsh_b, fuelsh_c, fuelsh_d |
| unionsw7 – Coded 1-4 : Version A through D | unions_a, unions_b, unions_c, unions_d |
| abortw7 – Coded 1-2 : Version A and B | abortgen_a, abortgen_b, abortdef_a, abortdef_b |
| spendw7 – Coded 1-2 : Version A and B | unispend_a, unispend_b, busspend_a, busspend_b |
| inequalw7 – Coded 1-2 : Version A and B | inequal_a, inequal_b |
