Sense about Science - What is Peer Review?

Chris Street edits in bold (partly) reposted from: Sense about Science - download pdf
"I don't know what to believe" - an article on Peer Review

SUMMARY
•Science has a system for assessing the quality of research before
it is published. This system is called peer review.
•Peer review means that other scientific experts in the field
check research papers for validity, significance and originality –
and for clarity.
•Editors of scientific journals draw on a large pool of suitable
experts to scrutinise papers before deciding whether to publish
them.
•Many of the research claims you read in newspapers and
magazines, find on the internet, or hear on television and the
radio are not published in a peer-reviewed journal.
•Some of this research may turn out to be good but much of it is
flawed or incomplete. Many reported findings, such as claims
about “wonder cures” and “new dangers”, never come to anything.
•Unpublished research is no help to anyone. Scientists can’t repeat
or use it and as a society we can’t base decisions about our public
safety – or our family’s health for example – on work that has a
high chance of being flawed.
•So, no matter how exciting or compelling new scientific or medical
research is, you must always ask…
Is it peer reviewed? If not, why not?
If it is peer reviewed, you can look for more information on what
other scientists say about it, the size and approach of the study and
whether it is part of a body of evidence pointing towards the same
conclusions.

HOW SHOULD YOU MAKE SENSE OF SCIENCE STORIES?
Peer review can help you make sense of science stories as it tells you that the
research has passed the scrutiny of other scientists and is considered valid,
significant and original.
Peer review means that statements made by scientists in scientific journals are
critically different from other kinds of statements or claims, such as those made by
politicians, newspaper columnists or campaign groups. Science is therefore more than
just another opinion.

A SHORT EXPLANATION OF PEER REVIEW
When a researcher, or team of researchers, finishes a stage of work, they usually write
a paper presenting their methods, findings and conclusions. They then send the paper
to a scientific journal to be considered for publication.
If the journal’s editor thinks it is suitable for their journal they send the paper
to other scientists who research and publish in the same field asking them to:
• Comment on its validity – are the research results credible;
are the design and methodology appropriate?
• Judge the significance - is it an important finding?
• Determine its originality - are the results new?
Does the paper refer properly to work done by others?
• Give an opinion as to whether the paper should be published,
improved or rejected (usually to be submitted elsewhere).
This process is called peer review. The scientists (peers)
assessing the papers are called referees or reviewers.

Scientists never draw firm conclusions from just one paper or set of results.
They consider the contribution it makes in the context of other work and their
own experience. It usually takes more than one research paper for results to be
seen as good evidence or accepted as a public truth.

THE SCIENCE PUBLISHING SCENE
For scientific knowledge to progress scientists need to share their research findings
with other scientists. The main way they do this is by publishing their research in
scientific journals – periodical publications intended to further the development of
science by reporting new research.
Journal editors receive many more papers than they can publish, so they use a twostep
selection process. First, they consider whether the paper is a ‘fit’ for their
journals. For example, some journals only publish research papers that are
groundbreaking; others only publish research in a specific area, such as microbiology.
If a journal editor decides that a paper is right for their journal, they send it for peer
review to check whether the research findings are valid, significant and original.

Did you know? There are around 21,000 scholarly and scientific journals that use the peer-review system. A high proportion of these are scientific, technical or medical journals, publishing over 1 million research papers each year.

You are most likely to hear about
new research from the daily news
media, where there is not space or
interest in full references. Good
journalists usually indicate whether
research has been published and
mention the name of the journal.

The more we ask, ‘is it peer
reviewed?’ the more obliged
reporters will be to include this
information.

There is no definitive list of peer reviewed
journals but you can look
up the names of selected peer reviewed
journals online at the
science news service EurekAlert!

SO SCIENTISTS USE PEER
REVIEW, SO WHAT?
When research findings have been peer
reviewed and published in a scientific
journal, this indicates that they are
sufficiently valid, significant and original
to merit the attention of other scientists.
Peer review is an essential dividing line
for judging what is scientific and what is
speculation and opinion. Most scientists
make a careful distinction between their
peer-reviewed findings and their more
general opinions.

Sounds good, but what happens
next?
Publication of a peer-reviewed paper is
just the first step: findings, and theories
about them, must go on to be re-tested
and judged against other work in the
same area. Some papers’ conclusions will
be disputed or further research will show
that they need to be revised as more data
are gathered.

SOURCES FOR FURTHER INFORMATION
Sense About Science: To find out more about peer review you can visit the Sense About Science website where there is a section dedicated to it.

Association of Medical Research Charities: The AMRC has a page, for medical research charities, on the peer review of research grant applications:

Committee on Publication Ethics: COPE provides a sounding board for journal editors struggling with how to deal with breaches in research and publication ethics:

The National Electronic Library for Health: The NELH has a ‘Hitting the Headlines’ archive,
which looks at medical news stories and provides the research evidence on which they are
based

The Science Media Centre: The SMC has published a leaflet, Peer Review in a Nutshell, a guide for scientists preparing for a news interview.