The challenges of screening and synthesising qualitative research in a mixed-methods systematic review. The case of the impact of agricultural certification schemes

The number of mixed-methods systematic reviews in international development is growing in recent years. By recognising the value of qualitative research in providing valuable evidence on causal mechanisms, barriers, facilitators and the importance of context, mixed-methods systematic reviews go beyond the ‘what works’ question. However, appropriate methods to screen and synthesise qualitative evidence in these reviews are still in a development phase, and the methodological literature dealing with reviewing qualitative evidence in the field of development studies is scarce and under-developed. This paper aims to contribute to this gap by discussing the methodological and practical challenges of including qualitative evidence in a mixed-methods systematic review in international development. In particular, this article makes a contribution in terms of offering reviewers and users of systematic reviews a full account of the process of screening and synthesising a very large volume of heterogeneous qualitative studies. Using as an example a review on the effects of certification schemes for agricultural production, we report on each reviewing step, describing the problems encountered and solutions found. The paper proposes ways of extracting a large volume of data and integrating the qualitative synthesis with the evidence from the related quantitative effectiveness review.