Systematic reviews

Clarifying the review question leads to specifying what type of studies can best address that question and setting out criteria for including such studies in the review. This is often called inclusion criteria or eligibility criteria. The criteria could relate to the review topic, the research methods of the studies, specific populations, settings, date limits, geographical areas, types of interventions, or something else.

Systematic reviews address clear and answerable research questions, rather than a general topic or problem of interest. They also have clear criteria about the studies that are being used to address the research questions. This is often called inclusion criteria or eligibility criteria.

Six examples of types of question are listed below, and the examples show different questions that a review might address based on the topic of influenza vaccination. Structuring questions in this way aids thinking about the different types of research that could address each type of question. Mneumonics can help in thinking about criteria that research must fulfil to address the question. The criteria could relate to the context, research methods of the studies, specific populations, settings, date limits, geographical areas, types of interventions, or something else.

Examples of review questions

1. Needs - What do people want? Example: What are the information needs of healthcare workers regarding vaccination for seasonal influenza?
2. Impact or effectiveness - What is the balance of benefit and harm of a given intervention? Example: What is the effectiveness of strategies to increase vaccination coverage among healthcare workers. What is the cost effectiveness of interventions that increase immunisation coverage?
3. Process or explanation - Why does it work (or not work)? How does it work (or not work)?  Example: What factors are associated with uptake of vaccinations by healthcare workers?  What factors are associated with inequities in vaccination among healthcare workers?
4. Correlation - What relationships are seen between phenomena? Example: How does influenza vaccination of healthcare workers vary with morbidity and mortality among patients? (Note: correlation does not in itself indicate causation).
5. Views / perspectives - What are people's experiences? Example: What are the views and experiences of healthcare workers regarding vaccination for seasonal influenza?
6. Service implementation - What is happening? Example: What is known about the implementation and context of interventions to promote vaccination for seasonal influenza among healthcare workers?

Examples in practice: Seasonal influenza vaccination of health care workers: evidence synthesis / Loreno et al. 2017

Research question: What are the views and experiences of UK healthcare workers regarding vaccination for seasonal influenza?

• Inclusion criteria:
  • Population: healthcare workers, any type, including those without direct contact with patients.
  • Context: seasonal influenza vaccination for healthcare workers.
  • Study design: qualitative data including interviews, focus groups, ethnographic data.
  • Date of publication: all.
  • Country: all UK regions.
• Exclusion criteria:
  • Studies focused on influenza vaccination for general population and pandemic influenza vaccination.
  • Studies using survey data with only closed questions, studies that only report quantitative data.

It is important to consider the reasons that the research question is being asked. Any research question has ideological and theoretical assumptions around the meanings and processes it is focused on. A systematic review should either specify definitions and boundaries around these elements at the outset, or be clear about which elements are undefined. 

For example if we are interested in the topic of homework, there are likely to be pre-conceived ideas about what is meant by 'homework'. If we want to know the impact of homework on educational attainment, we need to set boundaries on the age range of children, or how educational attainment is measured. There may also be a particular setting or contexts: type of school, country, gender, the timeframe of the literature, or the study designs of the research.

Example

Research question: What is the impact of homework on children's educational attainment?

• Scope: Homework - Tasks set by school teachers for students to complete out of school time, in any format or setting.
• Inclusion criteria:
  • Population: children aged 5-11 years.
  • Outcomes: measures of literacy or numeracy from tests administered by researchers, school or other authorities.
  • Study design: Studies with a comparison control group.
  • Context: OECD countries, all settings within mainstream education.
  • Date Limit: 2007 onwards.
• Exclusion criteria:
  • Any context not in mainstream primary schools.
  • Non-English language studies.

Some mnemonics that sometimes help to formulate research questions, set the boundaries of question and inform a search strategy.

Intervention effects

PICO Population – Intervention– Outcome– Comparison

Variations: add T on for time, or ‘C’ for context, or S’ for study type,

Policy and management issues

ECLIPSE: Expectation – Client group – Location – Impact ‐ Professionals involved – Service

Expectation encourages reflection on what the information is needed for i.e. improvement, innovation or information. Impact looks at what you would like to achieve e.g. improve team communication.

Analysis tool for management and organisational strategy

PESTLE: Political – Economic – Social – Technological – Environmental ‐ Legal

An analysis tool that can be used by organizations for identifying external factors which may influence their strategic development, marketing strategies, new technologies or organisational change.

Service evaluations with qualitative study designs

SPICE: Setting (context) – Perspective– Intervention – Comparison – Evaluation

Perspective relates to users or potential users. Evaluation is how you plan to measure the success of the intervention.

Read more about some of the frameworks for constructing review questions: