Re: Information: Brad's reply (was Information: a very

Greg Billock (billgr@cco.caltech.edu)
Wed, 1 Jul 1998 09:41:31 -0700 (PDT)

Brad Jones:

> > The question is whether it is meaningful to consider DNA a channel. This
> > is a question to be answered theoretically, not by checking definitions.
> > (BTW, most channels considered in information theory are memoryless, as
> > Glenn has been saying. And channels don't 'output' information;
> > information
> > comes *through* them.)
>
> You mean does it help your position to consider DNA as a channel. Not is it
> meaningful to consider it as a channel. DNA is a device which contains
> stored information, therefore it is meaningful to consider it as a channel.
> (even if it doesn't help your position)

That stone is likely to blow up: it could as easily be argued that
you are so insistent in (incorrectly) treating it as a channel because
that makes it seem more dramatic that there are errors in it. The
meaninglessness of applying the model there doesn't faze you, because
you're ideologically disposed to want the results of the analysis to
'prove' that DNA has to 'degenerate' in information.

But I hope that we can avoid this sort of long-distance psychoanalysis
and stick to the point.

> > The problem is that you are considering DNA a channel. If you aren't
> > willing to reconsider that (as well as firming up your language a bit),
> > we'll probably have to declare the discussion over.
>
> I will not reconsider looking at DNA as a channel because it is one. On the
> other hand if you pretend it is a source you can easily find that it has
> zero information (which is what any channel will turn out to be).

Good day, then. A couple more comments:

> A source *creates* information. DNA does not create information, it stores
> it. You were arguing that the information is created by random mutations
> were you not? By your own argument random mutations are the source and DNA
> is the channel. why is this so hard?

Please look at the diagram in Shannon's paper. There is a source,
a transmitter, and a channel (amidst other things). The source is
easily modelled as a store of information; the transmitter 'reads' the
source and sends the symbols through the channel. I can't think of
an easier way to explain it, so if this doesn't do the trick, I'm giving
up.

> > to say it.
> >
> > > the question is: Does noise have more information than any other signal?
> >
> > A noise source will generate maximal information, yes.
>
> No. A noise source generates no information. A source that has equiprobable
> symbols does generate the most information, but it is not random. do you get
> this yet?

I emailed Dr. Togneri about the question you posted. Here is a fragment
of his response:

Yes you are right a noise source represents maximum information even
though it is meaningless.

So it seems the information theory community has more solidarity than
you thought. :-) I'm sorry your comprehension of the class was less
than it should have been; I admit to having wondered if perhaps the
instructor was less than clear, but having talked to him, it seems
the fault lies either in the channel or the consumer. :-)

[more confusion]

Shannon didn't work, I didn't work, Togleri didn't work. I'm giving
up.

> "Disagree! Info theory is not nonsense. The person listening to radio static
> is nonsense. Info theory quantifies the information content of a source, it
> does not ascribe any meaning to the source"
>
> I hope you will think long and hard about this statement. It fits both what
> I have been saying and also your quotes from Shannon.

It does not fit in the slightest degree the misunderstandings you are
calling 'information theory.' Perhaps seeing that Prof. Togleri has the
appropriate view of what information theory relates to will help you
see how you might improve your understanding.

[...]

> Incidentally this is way off track with respect to the creation of
> information which was what was originally being debated.

Without a shared comprehension of what information theory is *about*,
there is little hope of having a complex discussion of its application
to biology.

-Greg