conflicting opinion drives science & limitation of peer reviews?

Does anyone want to comment on this article from UK's Financial Times (
<http://www.ft.com> www.ft.com) on limitations of peer review?

............................................................................
...........................
John Kay: Conflicting opinion drives science
By John Kay
Published: May 22 2006 19:37 | Last updated: May 22 2006 19:37

The Royal Society, Britain's scientific establishment, has just released a
report on public communication of scientific findings. Journalists in search
of stories and scientists anxious for publicity and research funding issue
early, oversimplified or downright misleading accounts of research.
Unsubstantiated claims of a link between immunisation and autism have caused
distress to millions of British parents. Korea's progress in stem cell
research seems to have been won at the expense of truth and ethics.

The Society's answers are self-restraint and peer review. Peer review is the
process by which professions review their own work. Articles submitted to
journals receive critical assessment from referees experienced in the field.
Peer review is a bulwark against cranks, crooks and incompetents. But too
much reliance on peer review carries its own dangers. Every profession
defines its own concept of excellence in inward-looking ways.

Successful academics learn how to trigger the buttons that win the approval
of referees. The physicist, Alan Sokal, demonstrated this by the submission
of a spoof article to the cultural studies journal Social Text in 1996. The
content was nonsense, but the form and jargon corresponded so closely to
reviewers' expectations that the contribution was accepted. Professor
Sokal's purpose was to demonstrate that standards were lower and more
subjective in softer subjects than in more scientific ones and, while he was
right, the problem identified was more general. All subjects, from
architecture to physics, from literary criticism to economics, develop what
Thomas Kuhn called paradigms - assumptions common to all practitioners and
assumed to represent universal truth until a new paradigm displaces the old.
A further step down a well defined road wins easier acceptance than a
deviation from the beaten track. Most academic research is therefore boring,
and more so as scholarship has become more professional, eccentricity less
tolerated and peer review multiplied through processes of grant awards and
research assessment. The latest idea in Britain is to make these processes
routine by shifting from the costly and fallible exercise of subjective
judgment to a cheaper and objective system of quantitative metrics. This can
only aggravate the problems.
Big advances come through the paradigm shifts and peer review makes this
difficult. The line between the crank and the genius is sometimes a fine one
and may only be apparent after time has elapsed. Many Nobel Prize winners
had difficulty securing early recognition. The world of today favours the
competent professional - as judged by the standards of other competent
professionals. In a sense this self-reference is right: the people to decide
whether astrology is good astrology are other astrologers. But they are not
the people to decide whether astrology itself is any good. Judgment of the
rigour and relevance of professional standards and scholarly research can
never be left to professionals and scholars alone.

The Tacoma Narrows Bridge, an elegant suspension bridge in Washington State,
carried traffic for four months in 1940. In a high wind, the flat deck
acquired a beautiful wave pattern. The oscillations grew larger and larger
until the roadway finally disintegrated into Puget Sound.

The trade newspaper, Engineering News-Record, was forced to retract its
suggestion that the designer, Leon Moisseiff, might have been responsible.
The editors apologised for any inference drawn by "the casual reader" that
"the modern bridge engineer was remiss".
But the perspective of "the casual reader", though not a substitute for peer
review, is as essential as the contribution of the little boy who pointed
out that the emperor had no clothes. Any form of censorship, including
self-censorship and censorship by fellow professionals, encourages
complacency and discourages innovation. The history of modern scholarship is
that, more slowly than we would wish, truth and new knowledge emerge only
from a cacophony of conflicting opinions.

The writer's new book, The Hare and the Tortoise, is published on June 6

 
Received on Thu May 25 17:33:02 2006