Replication of Quantitative Work in Development Studies: Experiences and Suggestions

Replication1 is seen as a key characteristic of natural science (Collins, 1985; Jasny et al., 2011); observations, especially those employing complex instruments, and experiments need to be repeated, and statistical analyses be scrutinized, before results gain credibility. This is not the case in social sciences; social science data are seldom re-produced,2 or re-analysed to check the original calculations, or analysed using alternative perspectives or frameworks.3 Hence, it is not clear that quantitative social science can advance in the same way as natural science. Nevertheless, there have often been calls among quantitative social scientists for replication (Frisch, 1933; Dewald et al., 1986; King, 1995; Gleditsch and Metelits, 2003; McCullough and Vinod, 2003; Pesaran, 2003; Bernanke, 2004; Freeze, 2007; McCullough and McKitrick, 2009; Burman et al., 2010), especially of computational4 studies (Peng, 2011). The proposed benefits include: full understanding of the computations5 and estimations, including for pedagogy; credibility; a basis for further work either assessing the robustness of the study to alternative variable constructions, model specifications, estimation methods or software, or extending or building on it; and an audit function — to identify and deter fraud and/or over-interpretation of the data (McCullough et al., 2006).