Re: [asa] Nature: Peer Review Trial and Debate

There are certainly debatable aspects of the peer review system for scientific publications. They are being discussed by scientists in those publications themselves.
E.g., there has been ongoing discussion in Physics Today (which is not itself a peer reviewed journal but is an official publication of the American Institute of Physics which published a number of such journals).

My comment here is simple. Anyone who has not been involved in the process, both as an author of such papers and as a referee for them, probably doesn't know enough about it to be able to comment on it intelligently.

Shalom
George
http://web.raex.com/~gmurphy/
  ----- Original Message -----
  From: Janice Matchett
  To: Rich Blinne
  Cc: asa@calvin.edu
  Sent: Thursday, June 15, 2006 10:40 AM
  Subject: Re: [asa] Nature: Peer Review Trial and Debate

  At 09:01 AM 6/14/2006, Rich Blinne wrote:

    On 6/13/06, Janice Matchett <janmatch@earthlink.net> wrote:

      Much that passes for "peer review" today is merely political payoff, so it's going to be fun to watch those who reject the "consensus" opinions being allowed to open the curtain, let the sunshine in, and start spraying the BSR into every nook and cranny. ~ Janice :).

    I believe a more open process is a good thing in order to show that Janice's perception is what is B.S. Let's take the following example ....So, the "consensus" is challenged. Let's look at the peer review. I'll take reviewer 2 as an example of how a well-written paper can sucessfully challenge the conventional wisdom and how the interaction between author and reviewer improve the paper and is not some political payoff. Janice, where is the political payoff or BSR here? [snip] ~ Rich

  @ My! That's a very "scientific" way to prove your case - just provide anecdotal stories / examples. :)

  National Academy of Sciences (PNAS) 3/16/06 http://www.pnas.org/cgi/content/abstract/0600591103v1
  Excerpt:
   "..We analyzed a very large set of molecular interactions that had been derived automatically from biological texts. We found that published statements, regardless of their verity, tend to interfere with interpretation of the subsequent experiments and, therefore, can act as scientific "microparadigms," similar to dominant scientific theories [Kuhn, T. S. (1996) The Structure of Scientific Revolutions (Univ. Chicago Press, Chicago)].

  Using statistical tools, we measured the strength of the influence of a single published statement on subsequent interpretations. We call these measured values the momentums of the published statements and treat separately the majority and minority of conflicting statements about the same molecular event.

  Our results indicate that, when building biological models based on published experimental data, we may have to treat the data as highly dependent-ordered sequences of statements (i.e., chains of collective reasoning) rather than unordered and independent experimental observations.

  Furthermore, our computations indicate that our data set can be interpreted in two very different ways (two "alternative universes"): one is an "optimists' universe" with a very low incidence of false results (<5%), and another is a "pessimists' universe" with an extraordinarily high rate of false results (>90%).

  Our computations deem highly unlikely any milder intermediate explanation between these two extremes. ~ [end excerpt]

  No!! They couldn't possibly be saying that scientists are human and can sometimes be subject to "crowd psychology"? And here comes this fly in the ointment just when some were getting comfortable relying on what they thought were safeguards against collective error (peer review, etc.).

  Newton's first law of motion is often stated as: An object at rest tends to stay at rest and an object in motion tends to stay in motion with the same speed and in the same direction unless acted upon by an unbalanced force."

  Applying that to social science, we could say that scientific consensus tends to stay locked into immobility and when it comes up for discussion, its trajectory remains uniform and linear unless and until force is applied.

  I interpret the squealing I hear from you as a reflection of even the mere threat of force being applied. :)

  ~ Janice .... who knows that the analogy breaks down at the word, "unbalanced", so don't attempt to use that subjective word (when it comes to "social science") to justify your objections.

To unsubscribe, send a message to majordomo@calvin.edu with
"unsubscribe asa" (no quotes) as the body of the message.
Received on Thu Jun 15 11:17:08 2006