[asa] Associated Press Reporter Does Science and Peer Review

Irrespective of what side many of us find ourselves in the many scientific
controversies of the day, what we can agree on is that science journalism is
for the most part poor. Oftentimes, the story devolves into a he said/she
said "balanced" piece with equal times for both views. This makes it easier
on the journalist because it does not require him or her to have a deep
knowledge of the material. You write the "conflict" narrative first and then
find quotes to fill it in. Because of this, it should be noted when there is
an exception and I found one here:
http://www.ajc.com/news/nation-world/ap-impact-statisticians-reject-174088.html

One controversy of the day is whether the global temperature is warming or
cooling. The problem is what cognitive psychologists call motivated
reasoning. We see what we want to see. Thus, the same data produces two
grossly different interpretations. Instead of recycling the old conflict
narrative Seth Borenstein of the Associated Press wanted to see which
interpretation was correct. So he designed a clever combination of a
single-blind protocol and peer review. Namely, he took the labels off and
sent the raw data from both the surface data (which the climate change
believers prefer) and the satellite data (which the climate change skeptics
prefer) to statistics experts for analysis.

Their uniform answer was a warming trend.

Statisticians who analyzed the data found a distinct decades-long upward
> trend in the numbers, but could not find a significant drop in the past 10
> years in either data set. The ups and downs during the last decade repeat
> random variability in data as far back as 1880.
>

Their comments are illustrative. For example:

Saying there's a downward trend since 1998 is not scientifically legitimate,
> said David Peterson, a retired Duke University statistics professor and one
> of those analyzing the numbers.
>

and

"If you look at the data and sort of cherry-pick a micro-trend within a
> bigger trend, that technique is particularly suspect," said John Grego, a
> professor of statistics at the University of South Carolina.
>

and

Grego produced three charts to show how choosing a starting date can alter
> perceptions. Using the skeptics' satellite data beginning in 1998, there is
> a "mild downward trend," he said. But doing that is "deceptive."
>
The trend disappears if the analysis starts in 1997. And it trends upward if
> you begin in 1999, he said.
>

The best-selling authors of Freakonomics have also pushed the cooling
narrative. How did these guys miss this? They admitted they eyeballed the
graph rather than subjecting it to a rigorous statistical analysis.

A line in [their] book says: "Then there's this little-discussed fact about
> global warming: While the drumbeat of doom has grown louder over the past
> several years, the average global temperature during that time has in fact
> decreased."
>
> That led to a sharp rebuke from the Union of Concerned Scientists, which
> said the book mischaracterizes climate science with "distorted statistics."
>
> [Freakonomics author] Levitt, a University of Chicago economist, said he
> does not believe there is a cooling trend. He said the line was just an
> attempt to note the irony of a cool couple of years at a time of intense
> discussion of global warming. Levitt said he did not do any statistical
> analysis of temperatures, but "eyeballed" the numbers and noticed 2005 was
> hotter than the last couple of years. Levitt said the "cooling" reference in
> the book title refers more to ideas about trying to cool the Earth
> artificially.
>
Face it Professor Levitt, you got pwned. (http://en.wiktionary.org/wiki/pwn)
Huzzah to Mr. Borenstein. This piece should be submitted to the Pulitzer
Committee. What this shows is you don't need a fancy degree to do science.
You just need to follow the method. It's also not limited to areas of study
that traditionally are considered "science". I will be addressing this topic
in a future post about a fascinating paper in last week's *Science*.

Rich Blinne
Member ASA

To unsubscribe, send a message to majordomo@calvin.edu with
"unsubscribe asa" (no quotes) as the body of the message.
Received on Tue Oct 27 11:15:40 2009