Re: [asa] Sternberg quote

One thing to consider is the alternative to a traditional peer review
system. One maybe promising alternative is the open online commenting
system some journals have been experimenting with.

But here is a perhaps cautionary tale from legal academia. The major
outlets for legal academics are "law reviews" published by the law
schools. NONE of these journals are peer reviewed. Rather, student
editors -- third year law students who have been selected as editors
because of their grade or through writing competitions -- choose the
articles that will get published in upcoming volumes.

The peer review process actually occurs through informal networking
and working conference presentations. Once an author has a paper
worked into a reasonable draft, h/she sends it around to trusted
colleagues and established experts for comment. Usually it's also
wise to worskhop a paper at one's home institution and at
work-in-progress conferences elsewhere. This at least tends to expose
glaring problems with a draft (like: "um, the Supreme Court decided
that question last year....")

In addition, the electronic Social Science Research Network (SSRN)
database has become very important in recent years. It's important to
post a work-in-progress to SSRN as soon as that can be done without
major embarassment. SSRN distributes emails with links to work in
progress under various speciality headings. This is the first way an
idea gets into the community, and it can produce valuable feedback.
Also, believe it or not, the number of times an author's paper has
been downloaded from SSRN has become an important metric for one's
status as a scholar (!?).

Once an author feels an article is ready for publication, submissions
to law reviews are not blind, nor are they exclusive. Articles are
never returned with comments for resubmission. Instead, twice a year
-- in fall, when students return to school, and in spring, when new
law review editorial boards are chosen -- authors broadcast their
manuscripts to 50 or more law reviews.

When an editor makes an offer of publication, the author "shops" that
offer to higher-ranked law reviews -- a process called "expediting."
If the author gets an offer from a higher-ranked law review, h/she
then expedites that offer further up the chain. At some point,
usually a few weeks into the process, the author decides that the
article has placed as high as it is likely to go, and accepts the
offer and withdraws the article from further consideration by other
journals. (Or, in the unhappy event that the article doesn't place in
a credible journal, the author agonizes over the tenure clock and
reworks it for the next submission season).

Needless to say, this is an idiotic system. Most of the "top" law
reviews are inacessible to the average scholar -- the student editors
generally will only read articles submitted by established "big name"
scholars or that have been referred to them by such after being
noticed on the conference / SSRN circuit. Third year law students
pretty much have no clue whether a submission is truly an excellent
piece of scholarship or just recycled schlock.

On 3/28/07, David Campbell <pleuronaia@gmail.com> wrote:
> I'm negatively impressed with the quality of Meyer's arguments in the
> paper, but I'm also negatively impressed with the quality of some
> things that make it through peer review in other places, including
> major journals. Is Sternberg overstating the degree of approval the
> re-examined process received? Are his opponents overstating the
> degree of deviation from the normal process? Probably yes to both.
> For example, few molecular biologists and only some evolutionary
> biologists actually know much about the Cambrian radiation, one of
> Meyer's main mishandled topics. Thus, the claim that the reviewers
> were all competent evolutionary and molecular biologists doesn't prove
> that they would have spotted the flaws, just as most papers invoking a
> molecular clock would be different if they had been reviewed by
> someone knowledgeable about paleontology or statistics.
>
> I had one paper published despite a rather unhelpful review (by
> someone whose system I was suggesting needed some changes). Editors
> do have some capacity to spot such reviewer bias, though of course
> having an editor with a particular agenda could be an issue.
>
> I recently reviewed a paper by someone with unconventional views. I
> noted that this was an issue affecting review that the editor should
> take into consideration for evaluating my review and tried to suggest
> some important points to develop that were novel.
>
> It's true that peer review tends to favor the status quo over novelty.
> However, this is often a good thing-there are plenty of crank ideas
> out there. Some process is needed to sort out what has credibility or
> not.
>
> The invocation of work of Newton, etc. as non-peer reviewed is silly.
> True, there was not much peer review back then, although
> well-established review procedures are in place at least by the early
> 1800's. However, there are plenty of other publications
> contemporaneous with Newton et al. that are rightly dismissed as
> baloney (whether reflecting mistaken premises prevalent in their day
> or just individual notions). The works that are now regarded as great
> works of science from the time before extensive peer review are
> regarded as great because they have held up well under subsequent
> review.
>
> Also, novel ideas have made it into the scientific mainstream through
> peer review. Enough evidence eventually wins out over tradition.
> Complaining that your ideas are being rejected due to the hidebound
> establishment or claming that they are sure to succeed because other
> novel ideas have succeeded raises the question of why you aren't
> gathering more convincing data instead of campaigning.
>
> --
> Dr. David Campbell
> 425 Scientific Collections
> University of Alabama
> "I think of my happy condition, surrounded by acres of clams"
>
> To unsubscribe, send a message to majordomo@calvin.edu with
> "unsubscribe asa" (no quotes) as the body of the message.
>

To unsubscribe, send a message to majordomo@calvin.edu with
"unsubscribe asa" (no quotes) as the body of the message.
Received on Wed Mar 28 13:20:50 2007