Wednesday, August 7, 2013

Stop the presses

Yesterday I encountered something I've never seen before: a formal press release from an academic society (SPSP) about a conference presentation of unpublished research:
http://www.eurekalert.org/pub_releases/2013-08/sfpa-vgb080213.php
A friend of mine forwarded it to me because it makes claims about the cognitive benefits of video game training, an area fraught with methodological problems that my colleagues and I have written about extensively (e.g, here's a recent blog post about a recent critique of such interventions). My guess is that the design shortcomings we discussed in that paper undermine the claims that these authors are making. But, I have no way to know. The actual research isn't available.

 Why does this work merit a press release now, before the research has been published? 

 The purpose of a press release is to draw public (and media) attention to a new finding, but in this case, the press release effectively is the finding because nobody can access the actual research. Journalists or science writers covering this study will have no more information than is available in the release itself, so they cannot verify that the research actually shows what the release claims that it does. In other words, the press release encourages churnalism rather than science reporting.

 In my view, academic societies should not be encouraging media coverage of research until the actual research is available for popular consumption. Doing so risks misleading the public. For this particular release, if the studies suffer from the problems we discussed in our recent article, then the conclusions might be unjustified and there would be a reasonable chance that the research would not survive the peer review process (I can only hope that reviewers would nix publication if the claims aren't justified). If that happened, then the press release would have hyped vapor-findings, claims that lack any underlying support. How does that benefit the popular appraisal of our field?

 Journalists and bloggers are free to discuss research they learn about at conferences, of course. And they typically do a good job in noting when findings are tentative (or giving enough details that others can evaluate the claims). But a formal press release from an academic society about unpublished research that is not available seems to me to be a different beast.

 Are there cases in which an academic society should issue a press release based on a conference presentation? Do you think this sort of press release is acceptable? I'd be curious to hear the perspectives of other scientists and science writers. Let me know what you think?