The association between exaggeration in health related science news and academic press releases: retrospective observational study

Sumner,P., Vivian-Griffiths,S., Boivin,J., et al, (2014) BMJ 349:g7015 10.1136/bmj.g7015

OBJECTIVE: To determine how often news contains claims or advice from health related research that goes beyond those in the peer reviewed journal articles and to identify the likely source of such exaggerations (e.g. press releases or news). Also, to test whether exaggerations in press releases were associated with a higher likelihood of news coverage, compared with press releases without exaggeration.

DESIGN: Press releases issued in 2011 by the 20 leading UK research universities, that were based on published studies with relevance for human health were identified from publicly accessible university repositories. For the 462 relevant press releases the associated peer reviewed journal article was sourced along with print or online news stories (n=668) from national press using the Nexis database, BBC, Reuters and Google. Broadcast news was excluded.

Recognising that peer reviewed publications may themselves contain exaggeration, they were nevertheless taken as the baseline, and cases were sought where news stories offered advice to readers, made causal claims or inferred relevance to humans beyond or different from that stated in the associated peer reviewed paper. Reference was then made back to the press releases to check whether discrepancies were already present in the corresponding press release.

SETTING: N/A

PARTICIPANTS: N/A

MAIN OUTCOME MEASURES: Discrepancies between peer-reviewed papers, press releases and new stories in relation to exaggeration of findings, advice to the public, causal statements from correlational results, conclusions for humans from non-human studies.

RESULTS: 40% of press releases contained more direct or explicit advice than the relevant journal article. For correlational results, 33% of primary claims in press releases were more strongly deterministic than those present in the original journal article. For non-human studies, 36% of press releases inflated inference to humans compared with the original article. In news stories giving advice to the public, 36% of items contained more direct or explicit advice than did the journal article. For correlational results, 39% of news statements were more strongly deterministic than those in the original journal article. For non-human studies 47% of news items contained inflated inference to humans.

For advice to the public, the odds of exaggerated advice appearing in the news was 6.5 times higher when the press release contained exaggerated advice than when it did not. For correlational results the odds of exaggerated statements in the news was 20 times higher when the press release contained exaggerated statements than when they did not. For non-human studies the odds of exaggeration in the news were 56 times higher when press release statements were exaggerated than when they were not.

Contrary to expectations for press releases with or without exaggeration, there was no significant difference in the proportion with at least one associated news story, though it was not possible to assess uptake for identical press releases with/without exaggeration. Further there was no statistical support for the idea that when press releases do successfully generate news, exaggeration would be linked with more associated news stories.

CONCLUSIONS: Most of the inflation of information detected did not occur de novo in the media but was already present in the text of related press releases produced by academics and their establishments, with 33-40% containing exaggerated statements compared with the corresponding peer reviewed journal articles. When press releases contained exaggeration it was likely that news stories would too, but when they did not contain exaggeration, news stories were much less likely to contain exaggerated information. Taking journal articles as the baseline, which may themselves contain exaggeration, means that the measured level of within university exaggeration is likely to underestimate the extent of the problem.