The UKHLS dataset is designed to be used with weights
The weights adjust for unequal selection probabilities, differential nonresponse, and potential sampling error. Unweighted analysis does not correctly reflect the population structure as some groups may be over-represented in the sample by design (e.g., over sampling of ethnic minorities in the EMBS) or due to nonresponse as certain types of people are more likely to respond than others. If a variable or characteristic differs by these groups which are over/under represented in the sample, then estimates of that variable based on this sample will be biased. For example, if the wages of some ethnic minority groups are lower in the population than the white British population and as there is a higher proportion of ethnic minorities in this sample than in the population, unweighted estimates of UK wages will be downwardly biased. A weighted analysis will adjust for the higher sampling fraction in Northern Ireland and for different probabilities of selection in the EMB and IEMB samples, as well as for response rate differences between subgroups of the sample. An unweighted analysis does not correctly reflect the population structure unless the assumptions below are true. It is suggested that researchers publishing or presenting unweighted estimates make these assumptions explicit.
In this podcast Olena Kaminska, Survey Statistician talks about maintaining a representative sample using weights to correct for nonresponse and attrition within Understanding Society.
Weighting:
What happens if I don’t use a weight?
You implicitly assume that sample members have equal probabilities of selection and of response. This is not true.
If no weighting is used, an analysis of Understanding Society data assumes that all estimated parameters (means, measure of dispersion, model coefficients, etc) do not differ between:
- Northern Ireland and the rest of the UK
- People of ethnic minority origin and people of white British origin
- Recent immigrants to UK and people who have stayed in the country longer
- People who live at an address with more than three dwellings or more than three households and those who don’t
- People who responded at Wave 1 and those who did not
- People who continued to respond at later waves and those who did not
- People who responded to each particular instrument used in the analysis (individual interview, self-completion questionnaire etc.) and those who did not, see Lynn, Burton et al. (2012)
An unweighted analysis of the former-BHPS sample assumes that estimated parameters do not differ between:
- Each of England, Scotland, Wales and Northern Ireland
- People who live at an address with more than three dwellings or more than three households and those who don’t
- People who responded at Wave 2 of Understanding Society in 2010 and those who may have become non-respondents at any time since Wave 1 of BHPS in 1991
- people who keep responding in later waves of Understanding Society and those who stopped responding at any point of time between 1991 and the last year in your analysis
We therefore strongly suggest conducting weighted analyses of the Understanding Society data.
Weights are constructed by combining (i) design weights which adjust for unequal selection or sampling fraction and (ii) nonresponse correction which adjusts for differential nonresponse and attrition at various stages (household level, within household at individual level, whether adult respondent completed self-completion questionnaire or not). Nonresponse occurs when some people/households respond while others do not. Differential response occurs when response is related to a particular characteristic/variable of interest.
