Monday, December 24, 2012

Recommendations for improving psychology

In a new blog post in The New Yorker,  makes some excellent suggestions for improving the state of psychology publishing. I personally prefer not to cluster fraud cases together with results that do not replicate. The former are revealed only via whistle blowing or investigation. The later inspire the "science is self correcting" refrain—later work can "fix" claims that don't hold up. 

The broader problem, though, is that science often is not self correcting. Science can't correct itself if nobody tries to replicate published claims or if those replications are relegated to the file drawer. The bias against publishing negative findings provides a disincentive for attempting direct replications in the first place. And, even when replication attempts are published, they often are ignored both in the literature and in the public eye (just as newspaper errata are buried). 

The field needs to change the incentive structure that governs scientific publishing in order to encourage and support direct replications. Initiatives like 's reproducibility project get us part of the way there, but only when the academic societies and journals encourage direct replication can the field really change. I'm involved in a project to implement just such a change, and I'll be writing more about it soon.