Little evidence for effectiveness of scientific peer review

Vaccination News Home Page                                            subscribe Vaccination NewsLetter

http://bmj.com/cgi/content/full/326/7383/241/a?etoc

BMJ Intended for Health Professionals in the US
 

Home Help Search/Archive Feedback Table of Contents

PDF of this article
Email this article to a friend
Respond to this article
Read responses to this article
PubMed citation
Related articles in PubMed
Download to Citation Manager
Search Medline for articles by:
White, C.
Alert me when:
New articles cite this article
 
Collections under which this article appears:
Peer Review

BMJ 2003;326:241 ( 1 February )

News

Little evidence for effectiveness of scientific peer review

Caroline White, London

Despite its widespread use and costs, little hard evidence exists that peer review improves the quality of published biomedical research, concludes a systematic review from the international Cochrane Collaboration.

Yet the system, which has been used for at least 200 years, has only recently come under scrutiny, with its assumptions about fairness and objectivity rarely tested, say the review authors. With few exceptions, journal editors---and clinicians---around the world continue to see it as the hallmark of serious scientific endeavour.

Published last week, the review is the third in a series from the Cochrane Collaboration Methods Group. The other reviews look at the grant application process and technical editing.

Only the latter escapes a drubbing, with the reviewers concluding that technical editing does improve the readability, accuracy, and overall quality of published research.

The Cochrane reviewers based their findings on 21 studies of the peer review process from an original trawl of only 135. These were drawn from a comprehensive search of biomedical print and online databases, and information received from bodies such as the World Association of Medical Editors.

Almost half of the available research focused on the effects of concealing the identity of reviewers and/or authors, which, the Cochrane authors conclude, has little impact on quality. Few studies assessed the impact of peer review on the importance, usefulness, relevance, or quality of research. Only one small study tested the validity of the peer review procedure itself.

On the basis of the current evidence, "the practice of peer review is based on faith in its effects, rather than on facts," state the authors, who call for large, government funded research programmes to test the effectiveness of the system and investigate possible alternatives.

"As the information revolution gathers pace, an empirically proven method of quality assurance is of paramount importance," they contend.

Professor Tom Jefferson, who led the Cochrane review, suggested that further research might prove that peer review, or an evolved form of it, worked. At the very least, it needed to be more open and accountable.

But he said that there had never even been any consensus on its aims and that it would be more appropriate to refer to it as "competitive review."

Not only did peer review pander to egos and give researchers licence to knife each other in the back with impunity, he said, but it was also "completely useless at detecting research fraud" and let editors off the hook for publishing poor quality studies.

In the latest report from the Committee on Publication Ethics, Professor Peter Lachmann, until recently president of the UK Academy of Medical Sciences, comments: "Peer review is to science what democracy is to politics. It's not the most efficient mechanism, but it's the least corruptible."

Footnotes

The report can be accessed from the National electronic Library for Health (www.nelh.nhs.uk)

 


© 2003 BMJ Publishing Group Ltd

Rapid responses:

Read all Rapid responses

Oh, Darn...
Ron Law
bmj.com, 30 Jan 2003 [Full text]
Scientific Peer Review
N R Panikker
bmj.com, 30 Jan 2003 [Full text]
arXiv.org has the solution???
Brian T Foley
bmj.com, 31 Jan 2003 [Full text]
Peer-Review leads to 'Status Quo', not to 'Solutions'
Trevor G Marshall, et al.
bmj.com, 31 Jan 2003 [Full text]
The Anonymous Expert Referee
harry morrow brown, et al.
bmj.com, 2 Feb 2003 [Full text]
Yes - not anonymous
Sinisa Lj. Maricic
bmj.com, 3 Feb 2003 [Full text]



 

PDF of this article
Email this article to a friend
Respond to this article
Read responses to this article
PubMed citation
Related articles in PubMed
Download to Citation Manager
Search Medline for articles by:
White, C.
Alert me when:
New articles cite this article
 
Collections under which this article appears:
Peer Review


 

 


Home Help Search/Archive Feedback Table of Contents

BMJ Intended for Health Professionals - Click here for further information
© 2003 BMJ Publishing Group Ltd

 

Vaccination News Home Page

ALL INFORMATION, DATA, AND MATERIAL CONTAINED, PRESENTED, OR PROVIDED HERE IS FOR GENERAL INFORMATION PURPOSES ONLY AND IS NOT TO BE CONSTRUED AS REFLECTING THE KNOWLEDGE OR OPINIONS OF THE PUBLISHER, AND IS NOT TO BE CONSTRUED OR INTENDED AS PROVIDING MEDICAL OR LEGAL ADVICE.  THE DECISION WHETHER OR NOT TO VACCINATE IS AN IMPORTANT AND COMPLEX ISSUE AND SHOULD BE MADE BY YOU, AND YOU ALONE, IN CONSULTATION WITH YOUR HEALTH CARE PROVIDER.