Barriers and facilitators to model replication within health economics

Emma McManus, David Turner, Ewan Gray, Haseeb Khawar, Toochukwu Okoli, Tracey Sach

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)
33 Downloads (Pure)

Abstract

Objective:
Model replication is important as it enables researchers to check research integrity, transparency and, potentially, to inform the model conceptualisation process when developing a new or updated model. The aim of this study was to evaluate the replicability of published decision analytic models, and to identify the barriers and facilitators to replication.
Methods:
Replication attempts of five published economic modelling studies were made. The replications were conducted using only publicly available information within the manuscripts and supplementary materials. The replicator attempted to reproduce the key results detailed in the paper, for example the total cost, total outcomes and if applicable, the incremental cost effectiveness ratio reported. Whilst a replication attempt was not explicitly defined as a success or failure, the replicated results were compared in terms of percentage difference to the original results.
Results:
In conducting the replication attempts, common barriers and facilitators emerged. For the majority of the case studies, the replicator needed to make additional assumptions when recreating the model. This was often exacerbated by conflicting information being presented in the text and the tables. Across the case studies, the variation between original and replicated results ranged from -4.54% to 108.00% for costs and -3.81% to 0.40% for outcomes.
Conclusion:
This study demonstrates that whilst models may appear to be comprehensively reported, it is often not enough to facilitate a precise replication. Further work is needed to understand how to improve model transparency and in turn to increase the chances of replication, thus ensuring future usability.
Original languageEnglish
Pages (from-to)1018-1025
Number of pages8
JournalValue in Health
Volume22
Issue number9
Early online date16 Jul 2019
DOIs
Publication statusPublished - Sept 2019

Keywords

  • Replication
  • economic models
  • health economics

Cite this