Sequential biases in accumulating evidence

Elena Kulinskaya, Richard Huggins, Samson Henry Dogo

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)
5 Downloads (Pure)


Whilst it is common in clinical trials to use the results of tests at one phase to decide whether to continue to the next phase and to subsequently design the next phase, we show that this can lead to biased results in evidence synthesis. Two new kinds of bias associated with accumulating evidence, termed "sequential decision bias" and "sequential design bias" are identified. Both kinds of bias are the result of making decisions on the usefulness of a new study, or its design, based on the previous studies. Sequential decision bias is determined by the correlation between the value of the current estimated effect and the probability of conducting an additional study. Sequential design bias arises from using the estimated value instead of the clinically relevant value of an effect in sample size calculations. We considered both the fixed effect and the random effects models of meta-analysis, and demonstrated analytically and by simulations that in both settings the problems due to sequential biases are apparent. According to our simulations, the sequential biases increase with increased heterogeneity. Minimisation of sequential biases arises as a new and important research area necessary for successful evidence-based approaches to the development of science.
Original languageEnglish
Pages (from-to)294–305
Number of pages12
JournalResearch Synthesis Methods
Issue number3
Early online date1 Dec 2015
Publication statusPublished - Sep 2016


  • accumulating evidence
  • cumulative meta-analysis
  • sequential meta-analysis
  • sequential bias

Cite this