Systematic reviews

On this page:

Studies that are used in a review are described in a standardised way that is suitable for each review.  The detail provided facilitates transparency in how each study contributes to the overall findings of the review, and the overall reliabillity of the review.

There are three key reasons for describing (or coding) the studies in a systematic review.

1. To know more about the included studies. Studies may be described on characteristics such as the aims, methods, or particular elements to describe the research sample and any outcomes measured.  Historically such descriptions about the studies may have been limited to basic information such as authors names, place of publication, and research methods. It is now recognized that information about what has or has not been studied is a useful product in its own right, as highlighted in the box on systematic evidence maps.
2. To identify the research findings of the individual studies to be synthesized
3. To identify the methods used in each study, so the study can be critically appraised for trustworthiness and relevance to the review. Methodological aspects of each study might be described in terms of how sampling was undertaken, recruitment of the sample, data collection methods, data analysis methods.

Critical appraisal involves checking the quality, reliability and relevance of the studies in the review in relation to the review question. It appraises each study in terms of the following aspects:. 

1. Is the study relevant to the research question?
2. Is the study valid? E.g. Were the study methods applied appropriately?
3. Were appropriate methods used in relation to the review question?

In addition, the studies are collectively appraised in terms of how they support the review findings and evidence claims of the review. For example, if the research evidence comprises of studies that have wide variation of findings, this reduces the strength of the evidence claims.

There are many standardised tools available for critical appraisal depending on the study design and the type of review. The approach to critical appraisal and the appraisal decisions for each study should be reported.

Commonly-used tools for appraising research evidence in reviews:

It is important for users of systematic reviews to consider the quality of the whole review. There are three separate elements that contribute an appraisal:

1. the quality and relevance of the methods used to address the review questions;
2. the quality and relevance  of the methods used by the individual studies included in the review;
3. the nature and extent of the total evidence from studies included in the review.

There are tools to help with the appraisal of a whole review. Some of these are specific to certain types of reviews, and others are more generic. 

Some tools focus only on appraising the methods of specific types of reviews:

Further reading: