From: Brian D Harper <bharper@postbox.acs.ohio-state.edu>
>At 03:09 PM 6/13/00 +0100, Richard wrote:
>>So how do you determine which is the "correct" explanation, without
>>considering parsimony? You can't. For any set of observations, there are
an
>>infinite number of theories that could explain them. For example, consider
>>fitting a curve to a set of data points. There are an infinite number of
>>different polynomials that will fit, not matter how many data points you
>>have. But, we would tend to reject higher order polynomials on the grounds
>>of parsimony. In fact, knowing that there are likely to be random errors
in
>>the data, we would probably accept a simple curve which gives an imperfect
>>fit, in preference to a high order polynomial that gives a perfect fit,
>>because the latter seems ad hoc.
>
>Let me ask a question that I recently encountered in a book by Rene Thom.
>The question is related to the above, except I want to first divorce it
from
>mere curve fitting. IMHO, curve fitting tells you very little about the
>"correctness" of a model.
Agreed. That was my point. The curve that fits the data best isn't
necessarily the best model.
>I recall one of my professors saying that all data becomes linear
>if you take enough logarithms :). So, let's suppose we have two different
>models.
>Preliminary experiments have been used to determine all free parameters.
>Now we need some "model verification" experiments. Experiments of a
>fundamentally
>different nature than those used to characterize the model. Prediction of
these
>experiments with no additional "tweaking" of parameters gives some strong
>support for the model.
Yes, predictive success is a major factor in supporting a model. I forgot to
mention that! Probably because it's not so useful when we're dealing with a
theory about past events. We can make predictions about what evidence (e.g.
fossils) will be found in the future, but inevitably prediction plays less
of a role when we're dealing with historical theories.
>OK, so we compare the predictions of the two models for the verification
>experiments. One model does a tremendous job "fitting" the experiment
>by any quantitative measure, such as square mean error, but does a
>really lousy job matching the overall "shape" of the data. The other model
>predicts the shape very well but is way way off quantitatively.
I'm not sure I understand you. How can a curve which gives you a much
smaller mean square error be a poorer fit to the shape?
>What would parsimony have to say in this situation?
Well now you can compare the two models on the basis of predictive success
*and* parsimony (simplicity). We would hope that the most parsimonious model
will have had the most predictive success, in which case the choice is easy.
Otherwise, we have to make a judgment about the relative weight of the two
criteria.
There have been some significant developments recently in this area of
statistics, but I'm afraid I'm not up-to-date (I studied statistics over 20
years ago). For more info, try taking a look at:
http://philosophy.wisc.edu/forster/220/simplicity.html and
http://philosophy.wisc.edu/simplicity/
Richard Wein (Tich)
This archive was generated by hypermail 2b29 : Thu Jun 15 2000 - 10:33:00 EDT