Estimating the reproducibility of experimental philosophy

Florian Cova; Brent Strickland; Angela Abatista; Aurélien Allard; James Andow; Mario Attie; James Beebe; Renatas Berniūnas; Jordane Boudesseul; Matteo Colombo; Fiery Cushman; Rodrigo Diaz; Noah N’Djaye Nikolai van Dongen; Vilius Dranseika; Brian Earp; Antonio Gaitán Torres; Ivar Hannikainen; José V. Hernández-Conde; Wenjia Hu; François Jaquet; Kareem Khalifa; Hanna Kim; Markus Kneer; Joshua Knobe; Miklos Kurthy; Anthony Lantian; Shen-yi Liao; Edouard Machery; Tania Moerenhout; Christian Mott; Mark Phelan; Jonathan Phillips; Navin Rambharose; Kevin Reuter; Felipe Romero; Paulo Sousa; Jan Sprenger; Emile Thalabard; Kevin Tobia; Hugo Viciana; Daniel Wilkenfeld; Xiang Zhou

doi:10.1007/s13164-018-0400-9

Estimating the reproducibility of experimental philosophy

Florian Cova, Brent Strickland, Angela Abatista, Aurélien Allard, James Andow, Mario Attie, James Beebe, Renatas Berniūnas, Jordane Boudesseul, Matteo Colombo, Fiery Cushman, Rodrigo Diaz, Noah N’Djaye Nikolai van Dongen, Vilius Dranseika, Brian Earp, Antonio Gaitán Torres, Ivar Hannikainen, José V. Hernández-Conde, Wenjia Hu, François JaquetKareem Khalifa, Hanna Kim, Markus Kneer, Joshua Knobe, Miklos Kurthy, Anthony Lantian, Shen-yi Liao, Edouard Machery, Tania Moerenhout, Christian Mott, Mark Phelan, Jonathan Phillips, Navin Rambharose, Kevin Reuter, Felipe Romero, Paulo Sousa, Jan Sprenger, Emile Thalabard, Kevin Tobia, Hugo Viciana, Daniel Wilkenfeld, Xiang Zhou

Research output: Contribution to journal › Article › peer-review

39 Downloads (Pure)

Abstract

Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.

Original language	English
Pages (from-to)	9-44
Number of pages	36
Journal	Review of Philosophy and Psychology
Volume	12
Early online date	14 Jun 2018
DOIs	https://doi.org/10.1007/s13164-018-0400-9
Publication status	Published - Mar 2021

Access to Document

10.1007/s13164-018-0400-9

Accepted manuscriptAccepted author manuscript, 1.17 MB

Cite this

Cova, F., Strickland, B., Abatista, A., Allard, A., Andow, J., Attie, M., Beebe, J., Berniūnas, R., Boudesseul, J., Colombo, M., Cushman, F., Diaz, R., van Dongen, N. ND. N., Dranseika, V., Earp, B., Torres, A. G., Hannikainen, I., Hernández-Conde, J. V., Hu, W., ... Zhou, X. (2021). Estimating the reproducibility of experimental philosophy. Review of Philosophy and Psychology, 12, 9-44. https://doi.org/10.1007/s13164-018-0400-9

@article{9716fab652b54cb3868ad1306052d30b,

title = "Estimating the reproducibility of experimental philosophy",

abstract = "Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.",

author = "Florian Cova and Brent Strickland and Angela Abatista and Aur{\'e}lien Allard and James Andow and Mario Attie and James Beebe and Renatas Berniūnas and Jordane Boudesseul and Matteo Colombo and Fiery Cushman and Rodrigo Diaz and {van Dongen}, {Noah N{\textquoteright}Djaye Nikolai} and Vilius Dranseika and Brian Earp and Torres, {Antonio Gait{\'a}n} and Ivar Hannikainen and Hern{\'a}ndez-Conde, {Jos{\'e} V.} and Wenjia Hu and Fran{\c c}ois Jaquet and Kareem Khalifa and Hanna Kim and Markus Kneer and Joshua Knobe and Miklos Kurthy and Anthony Lantian and Shen-yi Liao and Edouard Machery and Tania Moerenhout and Christian Mott and Mark Phelan and Jonathan Phillips and Navin Rambharose and Kevin Reuter and Felipe Romero and Paulo Sousa and Jan Sprenger and Emile Thalabard and Kevin Tobia and Hugo Viciana and Daniel Wilkenfeld and Xiang Zhou",

year = "2021",

month = mar,

doi = "10.1007/s13164-018-0400-9",

language = "English",

volume = "12",

pages = "9--44",

journal = "Review of Philosophy and Psychology",

issn = "1878-5158",

publisher = "Springer",

}

Cova, F, Strickland, B, Abatista, A, Allard, A, Andow, J, Attie, M, Beebe, J, Berniūnas, R, Boudesseul, J, Colombo, M, Cushman, F, Diaz, R, van Dongen, NNDN, Dranseika, V, Earp, B, Torres, AG, Hannikainen, I, Hernández-Conde, JV, Hu, W, Jaquet, F, Khalifa, K, Kim, H, Kneer, M, Knobe, J, Kurthy, M, Lantian, A, Liao, S, Machery, E, Moerenhout, T, Mott, C, Phelan, M, Phillips, J, Rambharose, N, Reuter, K, Romero, F, Sousa, P, Sprenger, J, Thalabard, E, Tobia, K, Viciana, H, Wilkenfeld, D & Zhou, X 2021, 'Estimating the reproducibility of experimental philosophy', Review of Philosophy and Psychology, vol. 12, pp. 9-44. https://doi.org/10.1007/s13164-018-0400-9

TY - JOUR

T1 - Estimating the reproducibility of experimental philosophy

AU - Cova, Florian

AU - Strickland, Brent

AU - Abatista, Angela

AU - Allard, Aurélien

AU - Andow, James

AU - Attie, Mario

AU - Beebe, James

AU - Berniūnas, Renatas

AU - Boudesseul, Jordane

AU - Colombo, Matteo

AU - Cushman, Fiery

AU - Diaz, Rodrigo

AU - van Dongen, Noah N’Djaye Nikolai

AU - Dranseika, Vilius

AU - Earp, Brian

AU - Torres, Antonio Gaitán

AU - Hannikainen, Ivar

AU - Hernández-Conde, José V.

AU - Hu, Wenjia

AU - Jaquet, François

AU - Khalifa, Kareem

AU - Kim, Hanna

AU - Kneer, Markus

AU - Knobe, Joshua

AU - Kurthy, Miklos

AU - Lantian, Anthony

AU - Liao, Shen-yi

AU - Machery, Edouard

AU - Moerenhout, Tania

AU - Mott, Christian

AU - Phelan, Mark

AU - Phillips, Jonathan

AU - Rambharose, Navin

AU - Reuter, Kevin

AU - Romero, Felipe

AU - Sousa, Paulo

AU - Sprenger, Jan

AU - Thalabard, Emile

AU - Tobia, Kevin

AU - Viciana, Hugo

AU - Wilkenfeld, Daniel

AU - Zhou, Xiang

PY - 2021/3

Y1 - 2021/3

N2 - Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.

AB - Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.

UR - http://www.scopus.com/inward/record.url?scp=85131297329&partnerID=8YFLogxK

U2 - 10.1007/s13164-018-0400-9

DO - 10.1007/s13164-018-0400-9

M3 - Article

VL - 12

SP - 9

EP - 44

JO - Review of Philosophy and Psychology

JF - Review of Philosophy and Psychology

SN - 1878-5158

ER -

Estimating the reproducibility of experimental philosophy

Abstract

Access to Document

Other files and links

Correction to: Estimating the reproducibility of experimental philosophy

Cite this