How can we measure disability in research related to the COVID-19 response?

Equity Depth Local Relevance Feasibility
Download brief
pdf
419 KB

Question & problem

There is growing evidence that COVID-19 is disproportionally impacting the lives of people with disabilities. This includes evidence of the increased risk of severe outcomes of contracting COVID-19 amongst people with existing health conditions, including many people with disabilities. It also includes a wide range of other potential impacts such as: reductions or disruptions in non-COVID-19 health or rehabilitation services, the effects of shielding on isolation and mental health, the implications of social distancing on people who require carer support, and the impact on poverty, participation and wellbeing due to disrupted disability-inclusive development programmes.

Measurement of disability in research has historically been contested and a number of different tools exist. Clear guidance is needed on how to determine which tool to use to understand the situation of people with disabilities in different settings, and plan responsive and inclusive COVID-19 programmes and policies to support their needs. Good quality, comparable data on disability is essential for tracking the impact of the COVID-19 pandemic, as well as prevention and mitigation interventions, amongst people with disabilities. Such evidence is also imperative for tracking progress towards the Sustainable Development Goals, and UNCRPD compliance.

Quote

Collecting evidence on the impact of COVID-19 on people with disabilities and their inclusion in the COVID-19 pandemic response is crucial to ensure they are not left behind.
Sightsavers, 2020

Recommendations

Use the Washington Group Enhanced Short Set (WGES), supplemented by other tools as relevant

Key recommendation

Use the Washington Group Enhanced Short Set (WGES) to measure disability in global health research, supplemented by other tools as relevant

Action

Access the WGES directly from the Washington Group Website and embed it into all research that seeks to understand the situation of people with disabilities. 

Dedicate sufficient resources to translation and training

Key recommendation

Dedicate sufficient resources for accurate translation and back translation of questionnaires, and training of data collection teams

Action

Draw on guidance available from the Washington Group and other disability data users such as Sightsavers, Humanity & Inclusion and BOND on how to prepare for data collection. Use mobile data collection tools to minimise error and maximise efficiency, and budget accordingly.

Follow guidance to analyse, disaggregate, interpret and present data

Key recommendation

Use established cut-offs to analyse, interpret and present data, and disaggregate outputs by age group, sex, functional limitation type and socio-economic position when sample size allows

Action

Invest in good statistical support to ensure you have a big enough sample size to answer the questions you have and can present your data following best-practice principles. Involve stakeholders and people with disabilities in interpreting outputs

Where insufficient resources exist to embed the WGES, consider alternatives but invest in pilot testing first

Key recommendation

Where insufficient resources exist to embed the WGES, consider a reported global assessment indicator such as the Global Activity Limitation Instrument (GALI) but invest sufficient resources into pilot testing this

Action

Engage with disability data users to learn from previous research using these tools. For example, via EUROSTAT or the UK Office for National Statistics. Follow cognitive testing principles when using these tools in new settings for the first time.

Include assessment of impairments or health conditions depending on the research question and resource availability

Key recommendation

Include assessment of impairments or health conditions depending on the research question and resource availability

Action

Leverage technology and mobile tools to capture data on impairments and health conditions, following guidance from global health researchers

Invest in further research to strengthen identification tools for people with disabilities

Key recommendation

Invest further in identifying how and why outputs from different tools differ, and how these can be compared

Action

Further research should be conducted on the  comparability of outputs from different tools, or different ways of analysing them. In particular, this includes further research on the utility of concise tools such as the GALI, when it is not possible to include longer tools

Challenges

Challenge #1: Various approaches to measuring disability exist, reflecting different models and definitions of disability

  • Both the International Classification of Functioning, Disability and Health (ICF) and the UN Convention on the Rights of People with Disabilities (UNCRPD) describe disability as an interaction between impairments and various barriers, that causes or risks participation restriction.
  • Not all available tools that are commonly used for measuring disability are compatible with this definition and amongst those that are, many measure only one component of the interaction, e.g. self-reported functioning, participation restrictions, environmental barriers or impairments separately, leading to difficulties comparing estimates
  • In countries such as the United Kingdom and elsewhere in Europe, where the definition of disability in legislation includes participation restrictions, different tools are used that incorporate both self-reported functioning and participation restrictions together
  • Question variations of “do you have a disability” will only identify people with the most substantial functioning limitations and can be stigmatising, yet are still commonly used

Challenge #2: Use of the Washington Group Questions is common, but the pros and cons of the different modules is not always clear

  • The most commonly used tool is the Washington Group Short Set (WGSS), which aims to identify the majority of people at risk of participation restrictions because of a health condition or impairment. It focuses on six basic functional domains: seeing, hearing, walking, cognition, communicating and self-care.
  • The WGSS are widely used and recommended, including for SDG indicator disaggregation. They are simple, quick, reproducible and easy to translate into different languages, allowing results to be compared over time and between countries. However, the WGSS provides limited data on other components of disability, such as participation restrictions or impairments. It also excludes psychosocial functioning and mental health.
  • The Extended Set of Washington Group questions (WGExS) contains up to thirty-five questions, and can be used in surveys where more time is available. They capture a more complete picture of disability but take longer to complete. When using the Extended set, prevalence tends to be much higher, due to high rates of pain and fatigue. The Washington Group have also collaborated with UNICEF to develop an extended set of questions on functioning for children aged 2 to 17, and the International Labour Organisation to develop a livelihoods module.
  • The Washington Group “Enhanced Short Set” (WGES) is a hybrid tool which includes the 6 questions of the Short Set, plus six additional questions related to upper body strength and mental health.

Challenge #3: Tools are translated, delivered, analysed, interpreted and presented in different ways, making comparison challenging

  • Rigour in translating and testing tools to measure disability is mixed, with anecdotal evidence that cognitive testing (testing whether key concepts are understood) is not regularly completed
  • The way that questions are delivered – for example whether to the individual or a proxy via household head or caregiver report, and whether or not response options are read aloud to participants, affect the outputs
  • The most common tool, the WGSS, can be analysed in different ways including considering anyone who reports “a lot of difficulty” or higher in any domain (or “daily” and “a lot” for either of the psychosocial functioning questions) as having a disability; or considering anyone who reports “some” or greater in any domain as having a disability
  • Different analytical and theoretical approaches are applied to data to assess the impact on participation or wellbeing of people with disabilities, that can be complex or poorly explained.
  • Disaggregation of data is important, as age group, gender and type of limitation can add additional layers of barriers or exclusion amongst people with disabilities, but often sample sizes are too low to disaggregate data meaningfully, and data is often presented for “all people with disabilities” only.
  • Some tools, such as the Model Disability Survey, require more complex statistical analyses to categorise people as having a disability or not

Challenge #4: Stakeholders may refuse to include tools with multiple questions on account of the number of questions included

  • Tools such as the UK Equality Act Disability Definition (EADD) and Global Activity Limitation Instrument (GALI) provide holistic information on health conditions that cause participation restriction and are more commonly used in High Income settings
  • Because they do not ask about specific functions, they are shorter than the WGES. However, this means they also do not collect data on the type of limitation the participant has, or whether or not they have considered the use of assistive devices (e.g. glasses) in their response

Challenge #5: Research questions related to health and rehabilitation would benefit from information on impairments and health conditions

  • Impairments or health conditions are components of disability that can be measured directly using objective testing. These data are important for planning appropriate health and rehabilitative services amongst those who would benefit from these (e.g. provision of cataract surgery, hearing aids, mobility devices). This may be particularly needed in low resource settings where inadequate access to health care is closely related to disability
  • However, impairment and health condition testing in isolation does not consider the effect on the individual’s level of functioning or participation. This type of assessment may also be more resource intensive to undertake than gathering self-reported information.

Finding the answers

This evidence brief synthesises findings from a scoping review of ICF-compatible tools to measure disability in population-based surveys with a focus on LMICs (2018), protocols and research outputs from seven population-based surveys of disability across Asia, Africa and the Pacific, secondary analyses of the South African Census, US National Health Interview Survey and three Demographic and Health Surveys, reflections from global stakeholders in disability measurement (including the UN Flagship Report on Disability), and evidence compiled for the upcoming Global Disability Research Massive Open Online Course at the London School of Hygiene & Tropical Medicine.

Recommendations & actions

Use the Washington Group Enhanced Short Set (WGES), supplemented by other tools as relevant

Key recommendation

Use the Washington Group Enhanced Short Set (WGES) to measure disability in global health research, supplemented by other tools as relevant

Action

Access the WGES directly from the Washington Group Website and embed it into all research that seeks to understand the situation of people with disabilities. 

Dedicate sufficient resources to translation and training

Key recommendation

Dedicate sufficient resources for accurate translation and back translation of questionnaires, and training of data collection teams

Action

Draw on guidance available from the Washington Group and other disability data users such as Sightsavers, Humanity & Inclusion and BOND on how to prepare for data collection. Use mobile data collection tools to minimise error and maximise efficiency, and budget accordingly.

Follow guidance to analyse, disaggregate, interpret and present data

Key recommendation

Use established cut-offs to analyse, interpret and present data, and disaggregate outputs by age group, sex, functional limitation type and socio-economic position when sample size allows

Action

Invest in good statistical support to ensure you have a big enough sample size to answer the questions you have and can present your data following best-practice principles. Involve stakeholders and people with disabilities in interpreting outputs

Where insufficient resources exist to embed the WGES, consider alternatives but invest in pilot testing first

Key recommendation

Where insufficient resources exist to embed the WGES, consider a reported global assessment indicator such as the Global Activity Limitation Instrument (GALI) but invest sufficient resources into pilot testing this

Action

Engage with disability data users to learn from previous research using these tools. For example, via EUROSTAT or the UK Office for National Statistics. Follow cognitive testing principles when using these tools in new settings for the first time.

Include assessment of impairments or health conditions depending on the research question and resource availability

Key recommendation

Include assessment of impairments or health conditions depending on the research question and resource availability

Action

Leverage technology and mobile tools to capture data on impairments and health conditions, following guidance from global health researchers

Invest in further research to strengthen identification tools for people with disabilities

Key recommendation

Invest further in identifying how and why outputs from different tools differ, and how these can be compared

Action

Further research should be conducted on the  comparability of outputs from different tools, or different ways of analysing them. In particular, this includes further research on the utility of concise tools such as the GALI, when it is not possible to include longer tools

Policy priorities

Advocate for the inclusion of the WGES in official data collection efforts such as the census or national population-based surveys, to provide comparative population-data for research outputs. Disaggregate all global health data by disability status to determine the situation of people with disabilities and plan responsive programmes and policies to support their full inclusion and meet their needs.

Conclusion

There are different approaches to the measurement of disability, which have both advantages and disadvantages. Using a commonly used tool and definitions such as the type offered by the WGES, and where appropriate using a combined approach of both self-report and assessment of impairments or participation is recommended.

Gaps & research needs

There is one scoping review of ICF-compatible tools to measure disability, and no known reviews of the comparability of disability measurement outputs in the literature. This brief was  prepared using findings from numerous population-based surveys, reflections from global stakeholders in disability measurement and evidence compiled for the upcoming Global Disability Research MOOC at LSHTM. Further research to replicate studies that have sought to compare  different tools in the same populations are recommended, to reinforce guidance on appropriate methodologies.

Acknowledgements

Publication details: © London School of Hygiene & Tropical Medicine, August 2020

Suggested citation: Islay Mactaggart, Hannah Kuper. Evidence Brief: How should we measure disability in global health research? Disability Evidence Portal, 2020

Disclaimer: The views expressed in this publication are those of the author/s and should not be attributed to the Disability Evidence Portal and/or its funders