We won’t be needing ‘that’ sample: An overview of the replication crisis in psychology
So…you’re studying psychology and you’ve been up to your eyeballs in coursework, reading journal article after article. You come across the phrase “failed to replicate” in the results of a paper. Hmm... In psychology, researchers strive to replicate studies for confirmation of reliable and valid results (Maxwell et al., 2015). But when would studies not be replicated? Researchers know that significant results are favoured more over non-significant findings, as they gain a better reputation, with increases in citations and h-index (Song et al., 2013).
Since researchers know this is the case… they may fabricate data, brush off non-significant findings or not publish their research (Ioannidis, 2005). This is otherwise known as publication bias. The main reasons behind non-publication are lack of time researchers have, unimportant results and journal rejection (Song et al., 2013).
Thinking point : As researchers are being rewarded for productivity, rather than whether their research is replicable or of a good quality, how can the publication process be made more stringent (Moylan, 2015) ?
What can be done to solve publication bias? Some solutions proposed by Carroll et al., (2017) are to adopt a two-stage review process and make peer-review training mandatory for researchers. Moylan (2015) advocated the idea of preclinical registration for animal research, which has proven important for ethical and moral practise. However, these are just a few ways of resolving publication bias, some of which only fit academics and the public sector..
Thinking point: So, how can students fight back ? Perhaps ‘being the voice of reason’ on social media or on campus is tricky to sustain on your own. A more suitable approach would be to collaborate with fellow psychology students, grouping ideas and putting them forward to your department, societies, social media and further afield (if you are feeling ambitious!). The importance of resolving publication bias is not only at the best of interests of aspiring psychologists and scientific researchers... but in the public eye too. As the public are increasingly relying upon the media for the latest scientific research, the repercussions of misinformation will sooner or later, backfire. After all, who knows, someone could be scrutinizing your ground-breaking research someday.
Retrieved from imgflip.com
Written by Evelyn Antony
“Hello! My name is Evelyn and I recently graduated from the University of Edinburgh with a Master of Arts (with Honours) degree in Psychology. My undergraduate dissertation leveraged data from the UK Millennium Cohort Study, where I investigated the role of emotional dysregulation in explaining the association between ADHD and internalising emotional problems amongst young children. Throughout my degree, I have acquired valuable and meaningful experiences working with young people that have profound mental health issues and additional needs, as well as working in research and events management within higher education settings. Collectively my work experiences and undergraduate degree in psychology has led me to pursue postgraduate studies, in educational psychology, which commences later in 2021.”
Connect with Evelyn on Linkedin and through her personal blog.
References
Carroll, H. A., Toumpakari, Z., Johnson, L., & Betts, J. A. (2017). The perceived feasibility of methods to reduce publication bias. PloS one, 12(10). https://doi/10.1371/journal.pone.0186472
Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLOS Medicine,
2(8), e124. https://doi.org/10.1371/journal.pmed.0020124
Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean?. American Psychologist, 70(6), 487. https://psycnet.apa.org/doi/10.1037/a0039400
Moylan, E. (2015). Reducing publication bias in animal research. Retrieved from: http://blogs.biomedcentral.com/bmcblog/2015/02/27/reducing-publication-bias-animal-research/
Song, F., Hooper, L., & Loke, Y. (2013). Publication bias: what is it? How do we measure it? How do we avoid it?. Open Access Journal of Clinical Trials, 2013(5), 71-81. https://doi.org/10.2147/oajct.s34419