Inconsistent answers on health – do they matter?

Socioeconomic inequalities in health are central to the policy agenda and are seen as inequitable if they’re avoidable and go against principles of social justice. There is growing evidence that repeated measures of self-assessed health from the same respondents within a short time interval, but with different survey modes, can result in inconsistent responses. These inconsistencies are not random, and are driven by differences in respondents’ cognitive skills, education, income and age.

We wanted to explore whether inconsistency between responses to identical questions about self-assessed health, when the time frame is too short for a genuine change in people’s health, could affect estimates of socio-economic inequality in health in large social science surveys. It’s important that surveys such as Understanding Society can rely on their measures of self-assessed health, and be sure that any influences that tend to make measurements different (such as interview mode) have limited impact.

Using the data 

We were able to investigate this question using Understanding Society, because the Study measured each respondent’s self-assessed health twice within the same interview in Waves 2-5, with two different collection modes: speaking to their interviewer and a self-completion questionnaire. The question was asked twice with identical wording and ordering of the health categories: “In general, would you say your health is: Excellent, Very Good, Good, Fair or Poor?”.

Key findings

A bubble plot of pooled responses to self-completion versus open interview mode shows considerable agreement in responses to both measures (the main diagonal), but a sizeable proportion of respondents (as shown by the size of the bubbles above and below the main diagonal) reported their self-assessed health inconsistently within the same interview.

Fig 1. Self-completion versus open interview SAH measures

We also produced dissimilarity indexes for cumulative levels of self-assessed health over the four waves based on open and self-completion interview mode. A higher dissimilarity index is a sign of larger socioeconomic inequality (Fig 2). We found systematic, statistically significant socioeconomic inequalities in all our health measures, both for health outcomes based on open interview and self-completion. However, these differences were relatively small.

Fig 2. Dissimilarity indexes for socioeconomic inequality in self-assessed health

We also analysed of what lies behind the socio-economic inequalities in health (Fig 3). This showed that the relative contribution of age, education, income, parental education, parental job status, and household characteristics (i.e, household size and urban residency) to socio-economic inequality in self-assessed health was larger towards the top end of the distribution – i.e. the better the respondent’s self-assessed health was. However, the contribution of jobs status and marital status decreases. We didn’t see any major differences in these contributions across the two measures.

Fig 3. Shapley decomposition

Implications of the findings

There is some existing evidence that the ‘micro-social environment’ might affect responses during household interviews – in other words, the presence of other adults or children affecting people’s answers. However, we found less pronounced evidence that they play a systematic role in reporting inconsistencies in self-assessed health.

Overall, our evidence suggests that reporting inconsistency in self-assessed is not purely random measurement error. People with certain characteristics are on average less likely to report their self-assessed health consistently in different modes of the survey (self-completion v. interview) – and socioeconomic status is one of the characteristics which plays an important role.

Despite the observed differences in reporting inconsistency to  self-reported health measures by individuals’ socioeconomic status, we found small differences in the estimated levels of socio-economic inequality in self-reported health and their underlying sources between different survey modes.

Given the importance for government policy of health measures collected in social science surveys, these results are generally reassuring as far as socioeconomic inequalities in health are concerned. 
 

Authors

Health and wellbeingSurvey methodology