Re: Peer review [ was: Re: [asa] Atheist finds God thru Behe's books....]

Hi all,

One of the things that should not be overlooked in regards to peer review is that it is a varied, multi-stage and open ended process - in particular, "passing peer review" ISN'T synonymous with getting an article published in a journal even if it often is used in that sense.

Expanding on this a little;

Varied: Peer-review means different things to a post-grad student submitting a lab report to his supervisor than it does to a philosopher publishing a complex argument in book form.

Multi state: Once the above post-grad student has his work accepted by his supervisor, or the above philosopher has her book published, they are BOTH subject to a wider circle of scrutiny. NEITHER can be said to "pass peer review" simply because their work jumped the first hurdle.

Open ended: Even IF every contemporary of the above displays overwhelming acceptance of their work, the reality is that critical scrutiny doesn't stop there. One might have been forgiven if, in the late 1800's, one was to argue that Isaac Newton's work in mechanics had "passed peer review" - but, of course, Newton's 20th century peers had different ideas. Indeed, I'm not aware of any historical figure - from Aristotle to Galileo to Einstein - whose work can be said to be beyond critical scrutiny. Nobody, really, ever "passes peer review" except in a restricted sense.

So, I think it should be kept in mind that "peer-review" isn't remotely a simple idea.

Consequently, I wonder if part of Cameron's objection to peer review doesn't arise precisely because people have a tendency to use the term in a rather tendentious manner, viz; 'I define "peer review" as meeting criteria X, theorem Y meets criteria X, therefore theorem Y "passes peer review," therefore you can't argue against theorem Y' (call this "the tribe has spoken gambit").

In light of this I don't think it bad practice to ask ourselves two questions;

(1) What ARE the criteria X had in mind might when somebody speaks of "peer review" and are they the best criteria for assessing the merits of a theorem

and

(2) Just because a theorem is thought to meet criteria X at any given time does it follow that further critical scrutiny of that theorem after that time is somehow invalid

And reflecting on (2) we can see an interesting implication in the argument that "this theorem has 'passed peer-review' [so don't dare question it!]" - namely that the questioner has no right to a dissenting opinion. Which could only be the case if the questioner is not, in fact, a "peer" of the person putting the theorem.

So we get the further issue; Just who counts as a "peer"? And who says?

Ultimately I think what really matters is whether persons have good arguments for or against a theorem/thesis and speaking of "peer review" is (or should be!) just shorthand for describing what happens when those persons most informed on a matter assess a theorem/thesis according to their best lights.

Should THEY decide that a theorem/thesis "passes" then this tells us ONLY that such-and-such group of people, having such-and-such qualifications, have assessed, to the best of their ability, the strength of such-and-such a theorem/thesis, according to such-and-such criteria - and, of course, have not found it wanting.

I think Cameron quite rightly points out that there are certain limitations to this process - it is a fine process as far as it goes, but I don't think it can rightly be appealed to as a decisive argument against a dissenting opinion - i.e. "the tribe has spoken gambit" simply has to be seen as a polemical ploy which is, technically, a form of logical fallacy.

So appeals to peer review should not, I think, carry much weight unless one happens to know just what persons, with what qualifications, using what criteria, have stated that a theorem Y has "passed peer-review" whilst knowing also what persons, with what qualifications, using what criteria, have stated that it does not.

Blessings,
Murray

To unsubscribe, send a message to majordomo@calvin.edu with
"unsubscribe asa" (no quotes) as the body of the message.
Received on Mon Oct 19 14:58:15 2009