Changes to the Questionnaire and questions

We collect information from our sample members each year to build a picture of how society is changing over time or remaining the same.

By asking the same questions of the same set of people each year it allows researchers to measure change over time. However, there are several reasons why a question or its response options may change over time or why a question may not be asked every year.

Changes to questionnaire content are never made lightly and content is only updated if it is of clear benefit to the Study and enriches the data for research. When questions are reviewed, relevant topic experts and data users are consulted on proposed changes and ensure longitudinal compatibility for researchers.

Questions included in the questionnaires/survey instruments undergo thorough testing and validation and are based on robust methodological research. Questions from other surveys are used where appropriate. All survey instruments/questionnaires are tested so that any issues with question wording and routing, interview flow and timings can be identified, and appropriate changes made before the survey is implemented or fielded.

Question changes to reflect societal change

As society changes, some questions need reviewing to align with current practices or changes in policy. For example since the Study began the way we work has changed and reviewing these questions means we can incorporate those changes. To capture information on respondents current employment and those with multiple jobs, changes to the questionnaire were first made in Wave 13 to the Currentemployment module. Previously questions had focussed on the assumption that people have one paid job with a ‘standard’ working week. These questions were redesigned to better cater for respondents with multiple jobs taking into account non-standard working, including portfolio working, where people have lots of smaller jobs rather than one main job.

Some questions become unnecessary or obsolete such as whether you have a ‘colour television’ or ‘video recorder/DVD player’ Cduse. These were replaced with ‘television set’ and ‘DVD/Blu-Ray player’ respectively onwards from Wave 8 (2016-2018).

This can also work in reverse with the introduction of previously non-existent items such as iPads, electric cars and changes in behaviour such as meeting friends online and owning smartphones smartmob. Variable Mobcomp introduced in Wave 5 is now a core question asked every year asking if respondents have a tablet or iPad. To reflect changes in how young people feel and interact socially, changes in questions asked to 10-15-year-olds were introduced in Wave 12 asking how often they get together with friends in person or online Ypfndonl, as well as a question about how often they feel lonely Yplonely.New questions can respond to an increase in awareness and inclusivity within society, such as how participants self-identify their sex and gender and how this might have changed from that described at birth. Wave 12 variables Birthsex and Genderself were asked of young adults aged 16-21 in the self-completion module Scasexandgender. Questions can also respond to changes in the economy such as the questions added in Wave 11 to measure the experience of gig economy type of jobs (see module Gigeconomy).

Changes in policy

Changes in policy can result in new response options. For example, within Jbstat, following the introduction of shared parental and adoption leave. In Wave 13 the Study introduces more comprehensive questions about parental leave. Question and response wording also change with the introduction of new qualifications or state benefits. For example, GCSE grades changing from letters to numbers in England and Wales in 2017 (GCSE5). Introduced in Wave 11, GCSEatoceq asked new entrants to the Study the number of GCSEs or equivalent qualification names and/or grades to accommodate those devolved nations who did not change qualification name and/or grades. Wave 14 (2022-2024) sees the introduction of new options for income variable Ficode, asking about income from new Government payment schemes for Child, Adult and Pension age disability as well as the Scottish child payment scheme.

Changes in the political landscape can also prompt additional modules such as voting behaviour in the devolved elections in Scotland, Wales and Northern Ireland asked in Wave 12 in modules scadevolvedScotland, Scadevolvedwales and Scadevolvedni.

Enriching the data collected

Response options can respond to changes in education pathways e.g., with the addition of two new degree apprenticeship options in variable Apprent in Wave 12. Similarly, responses can be aligned with source data to enable comparisons with other studies, such as the extra questions aligned with the Gender and Generation survey for child responsibilities in the Domesticlabour module in Wave 12.

Additional questions can also enrich the data in understanding people’s income. Wave 12 captured payments for student loans within the household income for the first time in the Studentloans module. Altering the universe can enrich the data too: those without a job are asked if they were looking for a job in the last 4 weeks (julk4). From onwards, Wave 14 (2022-2024) this question also asks those in employment as well. To define missing or inapplicable values, questions previously asked only to the extra 5 minute sample in the Harassment module (attacked) were asked to the entire universe from Wave 11 onwards.

Improving the questionnaire

Sometimes changes are made to the questionnaire across waves when routing errors only become known after data collection has been completed. (e.g., in Wave 1 only the proxy interview included the question for whether or not a respondent had access to a car (w_drive) and from Wave 2 (2010-2012) onward this information is available for both adult and proxy respondents (those with missing information from Wave 1 were asked again).

Introduction of new modes of data collection

The switch in mode from paper self-completion to the CASI on the lap-top in Wave 4 meant that for some questions the response options were presented differently between waves. For example, response options were arrayed horizontally in the paper self-completion (e.g., satisfaction questions), and vertically in CASI. There is some evidence that the change in the way the response options were presented may affect how some people respond to the question (Budd, Gilbert et al. 2012).

From Wave 7 (2015-2017) onwards an increasing proportion of the sample were offered the opportunity to complete the questionnaire via the web. To adjust for the change in mode and reduce mode effects, some question wording and structure changed for the web mode with interviewer instructions and help tips, usually read by the interviewer, included in the question text. For example, in web mode, the question ‘In whose name is this accommodation rented?’ (Rentp) includes the extra wording ‘This is the person responsible to the landlord for the rent. If there is a joint tenancy, record all those responsible.’ Similarly questions with showcards give further clarification. For example in the face-to-face mode, after asking the question about consumer durables in the accommodation (Cduse), interviewers present the showcard to respondents and ask the numbers that apply. In the web mode, the question text includes this additional clarification “If you have a combined TV/Video select ‘Colour television and ‘Video recorder/DVD player’. If you have a combined washer drier select ‘Washing machine’ and ‘Tumble Drier’.”

Within wave changes

On rare occasions question text changes within a wave, or questions are dropped within a wave. For example, at the end of the first six months of data collection in Wave 1, multiple variables were dropped because of the length of the interview to reduce the burden on respondents, e.g., employment history module, parents’ educational qualification questions. The rest of the sample were asked these questions in subsequent waves (parents’ educational qualification questions were asked in Wave 2 and employment history in Wave 5). Similarly, with the start of the Covid19 pandemic in March 2020, questions about furlough and the coronavirus symptoms were added to Wave 11. These were asked of respondents who were interviewed from onwards April 2020.

Variable notes

To help researchers identify variables where the question text may have changed over time, variable notes are being created and added to the Variable search on the Understanding Society website. Other fieldwork materials such as showcards, advance letters and interviewer instructions are also on the website.