Showcards vs. no showcards

There were experiments involving showcards in Waves 1-3. Face-to-face interviews often rely heavily on the use of showcards to enhance measurement.  However, showcards are not necessarily available in a mixed mode approach to data collection particularly when the use of telephone interviewing is included in the mode mix.

Experimentation in Wave 1 examined whether there may be a primacy effect (selecting the first listed choice) of visual cues when a showcard is used or a recency effect (selecting the last listed choice) if the list is read. Also there was experimentation with different methods of obtaining information on unearned income sources without showcards. The question topics are about labour force status and unearned income.

For the labour force status experiment, there was random allocation of households within PSUs. All interviewed adults within households received the same experimental treatment. The groups are:

Group 1 –  Question asked using a showcard

Group 2 –  Question asked as a “read out” without a showcard

The variable for allocation to treatments is a_group2 on the record a_hhsamp_ip.  The variable a_jbstat on the record a_indresp_ip contains the substantive information.

The experiment also compares three methods of obtaining measures of unearned income sources.  These sources include benefits paid by the government as well as money from loans, rents, private grants, and money transfers from private individuals. The British Household Panel Survey (BHPS) protocol for enumerating unearned income sources has relied heavily on showcards to remind respondents of all potential sources in order to obtain more accurate reporting. Such reliance on showcards may not be feasible in a mixed modes approach. The UK Labour Force Survey uses an enumeration protocol that does not rely on showcards but a complex array of filter questions. A three-way split-ballot (different wordings) experiment contrasts the BHPS approach to unearned income enumeration with alternative “no showcard” designs.

Within PSUs, households were randomly allocated to treatments and all adult respondents within households were treated the same:

Group 1 – Original British Household Panel Study version with showcard enumeration

Group 2 – Adapted Labour Force Survey approach without showcards, no filter

Group 3 – Adapted Labour Force Survey without showcards, with two initial filter questions

The variable in the data that controls allocation to treatments is a_group3 on the record a_hhsamp_ip.  A list of items about substantive benefit and payment sources for the three experimental groups follows:

Table: Variables measuring unearned income sources

Group 1 – showcard	Group 2 – no showcard, no filter question	Group 3 – no showcard, filter question
a_benpeng11 to a_benpeng196 a_bendisg11 to a_bendisg196 a_bensupg1 to a_bensupg196 a_benpayg11 to a_benpayg196 a_nfa_g1 a_nfb_g1 a_nfc_g1 a_benalg11 to a_benalg196 a_nfe_g11 to a_nfe_g13 a_nff_g1 a_nfg_g1 a_f2_g1 a_niserp	a_btypeg21 to a_btypeg296 a_benunempg21 to a_benunempg296 a_bendisg21 to a_bendisg296 a_bendlag21 to a_bendlag296 a_benpeng21 to a_benpeng296 a_niserps2 a_benctcg2 a_benfamg21 to a_benfamg296 a_bentaxg21 to a_bentaxg296 a_benhoug21 to a_benhoug296 a_benstag21 to a_benstag296	2 initial filter questions: a_benefit_g3 a_payment_g3a_btypeg31 to a_btypeg396 a_benunempg31 to a_benunempg396 a_bendisg31 to a_bendisg396 a_bendlag31 to a_bendlag396 a_benpeng31 to a_benpeng396 a_niserps3 a_benctcg3 a_benfamg31 to a_benfamg396 a_bentaxg31 to a_bentaxg396 a_benhoug31 to a_benhoug396 a_benstag31 to a_benstag396

At Wave 2, the showcards study was expanded to incorporate the entire interview, though the experiment to measure unearned income sources was discontinued.  CAPI respondents were randomly allocated into groups interviewed with showcards and those interviewed without showcards. The showcard experiment was independent of all other experiments, including the mixed modes experiment carried at Wave 2. Thus, all households were allocated to experimental treatments. If a telephone case was transferred to a face-to-face interviewer for follow-up, then they received their allocated showcard treatment.

Allocation was at the PSU level so that interviewers would be either with or without showcards for all of their interviews to avoid contamination through the inadvertent use of visuals within face-to-face interviews. The experimental allocation is as follows:

Group 1 – Showcards

Group 2 – No showcards

The controlling variable on the data is b_ff_showcardw2 on the record b_hhsamp_ip.  The experiment applies to all items in the questionnaire which are indicated as having a showcard for use with face-to-face interviewing.

At Wave 3, the showcard experiment was repeated. As in Wave 2, manipulations were done at the PSU level such that each interviewer either used or did not use showcards. Note, at Wave 2, the showcard experiment was confounded with the showcard treatments of other experiments. At Wave 3, these are not confounded.  However, the use of showcards was still consistent across all other experiments except for the life satisfaction and job satisfaction experiment where a special showcard was required for all interviewers. There is further information about this implementation problem below. See the satisfaction experiment implementation notes (below) for further details. Showcards were used extensively throughout all portions of the questionnaire. The questionnaire indicates which questions use a showcard.

The controlling variables are c_ff_showcardsw2 and c_ff_showcardsw3 on record c_hhsamp_ip. note thatc_ff_showcardsw3 represents a rotation from the Wave 2 allocation. The rotation pattern is as follows:

Table: Rotation pattern for showcard experiment, waves 2 and 3

Values for c_ff_showcardsw2 and c_ff_showcardsw3 showing the allocation rotation
Wave 2	Wave 3
1	1	Showcards both waves
1	2	Showcards at Wave 2 & no showcards at Wave 3
2	1	No showcards at Wave 2 & showcards at Wave 3
2	2	No showcards at Wave 2 and Wave 3

Note that there was an error in the implementation of the IP3 showcard experiment, which meant that the treatments were not necessarily implemented as allocated. For details see the section detailing Known data issues.