RE: Information: Brad's reply (was Information: a very

Brad Jones (bjones@tartarus.uwa.edu.au)
Fri, 3 Jul 1998 17:22:01 +0800

Greg,

>
> > > The question is whether it is meaningful to consider DNA a
> channel. This
> > > is a question to be answered theoretically, not by checking
> definitions.
> > > (BTW, most channels considered in information theory are
> memoryless, as
> > > Glenn has been saying. And channels don't 'output' information;
> > > information
> > > comes *through* them.)
> >
> > You mean does it help your position to consider DNA as a
> channel. Not is it
> > meaningful to consider it as a channel. DNA is a device which contains
> > stored information, therefore it is meaningful to consider it
> as a channel.
> > (even if it doesn't help your position)
>
> That stone is likely to blow up: it could as easily be argued that
> you are so insistent in (incorrectly) treating it as a channel because
> that makes it seem more dramatic that there are errors in it. The
> meaninglessness of applying the model there doesn't faze you, because
> you're ideologically disposed to want the results of the analysis to
> 'prove' that DNA has to 'degenerate' in information.

No, I treat it as a channel because that is the only model of random changes
to information that exists in information theory. Note I say changes to
information, that happens in channels not sources which create information.

If you or Glenn can point out where in any text a source has its information
randomly changed without a channel then I'll concede this point. But of
course you cannot because it doesn't work that way.

If this is to be modeled then a channel MUST be used, you and Glenn refuse
to do this and that invalidates your position.

>
> But I hope that we can avoid this sort of long-distance psychoanalysis
> and stick to the point.

Sure show me cases where random changes to information are done outside of a
source in information theory.

Do you want me to show you examples of where random noise reduces the
information content of a signal? Of course you don't because it would
disprove your argument. That is why you and Glenn refused to accept my model
of the CD. Incidentally Glenn himself posted a quote where Yockey stated
that a tape machine was a good example.

"Thus the DNA-mRNA-protein system is discrete, memoryless and
unconstrained. It transmits the same message to the destination
repeatedly, as in a tape recorder."
-- Yockey as quoted by Glenn.

Now ANY engineer in this field will tell you that randomly changing the
information on a tape will destroy it. That is because any engineer will
know that it is the meaning on the tape which is important.

So the argument that "it isn't a channel" is simply wrong because a tape
machine sure is a channel. And Yockey himself compares them, so if you want
to argue that "tape machines are not best modeled as a channel" then ok,
but....

Definition of a channel:

"
An information channel is described by giving an input alphabet A; an output
alphabet B; and a set of conditional probabilties P(bj/ai). These
probabilties are that an output symbol bj will be recived if the input
symbol ai is sent.

Note: The actual symbols tranmitted will usually be the same as those
recieved. However our channel also models the noise which can:
- change the received symbols or introduce new ones.
- change the information content of what is being transmitted.

"
-- Dr Togneri, Lecture notes, information theory and coding, 1998.

Do you see that this is an EXACT model of information being changed by
random noise, that is what is being discussed. This is explicitly stated
that a channel should be used for modeling random noise.

It explicitly states that channels are used to model the effect that noise
has on information content. I cannot think of a clearer explanation than
this. Post this to Dr Togneri and ask his opinion if you like *ROFL*

I conclude from this that Dr Togneri is more likely to agree with my
position. I probably will ask him at some stage though (about the whole
series of posts) just to clarify some points.

So Please explain how random changes to information in DNA are not best
modeled as a channel?

Please post the definition of a source that makes it a better model than the
channel as given above.

>
> > > The problem is that you are considering DNA a channel. If you aren't
> > > willing to reconsider that (as well as firming up your
> language a bit),
> > > we'll probably have to declare the discussion over.
> >
> > I will not reconsider looking at DNA as a channel because it is
> one. On the
> > other hand if you pretend it is a source you can easily find that it has
> > zero information (which is what any channel will turn out to be).
>
> Good day, then. A couple more comments:

Hmm, I think I've shown this one by now. yes?

>
>
> > A source *creates* information. DNA does not create
> information, it stores
> > it. You were arguing that the information is created by random mutations
> > were you not? By your own argument random mutations are the
> source and DNA
> > is the channel. why is this so hard?
>
> Please look at the diagram in Shannon's paper. There is a source,
> a transmitter, and a channel (amidst other things). The source is
> easily modelled as a store of information; the transmitter 'reads' the
> source and sends the symbols through the channel. I can't think of
> an easier way to explain it, so if this doesn't do the trick, I'm giving
> up.

Hmm, please look at the definiton of a channel given above. There is no
easier way to explain it than the quote gives. And it is not "just my
opinion" although it is my opinion also.

I know a source CAN be used to model a storage device, but if you want to
see the effect that noise has on the information in such a device, then a
channel MUST be used. I've been repeating this for a while now.

If you model a storage device as a source then you have some restrictions.
One of these is that a source ALWAYS puts out the correct symbol, sources do
not make mistakes (its not part of the model). If you want to add random
changes to information then a channel is used.

> > > to say it.
> > >
> > > > the question is: Does noise have more information than any
> other signal?
> > >
> > > A noise source will generate maximal information, yes.
> >
> > No. A noise source generates no information. A source that has
> equiprobable
> > symbols does generate the most information, but it is not
> random. do you get
> > this yet?
>
> I emailed Dr. Togneri about the question you posted. Here is a fragment
> of his response:
>
> Yes you are right a noise source represents maximum information even
> though it is meaningless.
>
> So it seems the information theory community has more solidarity than
> you thought. :-) I'm sorry your comprehension of the class was less
> than it should have been; I admit to having wondered if perhaps the
> instructor was less than clear, but having talked to him, it seems
> the fault lies either in the channel or the consumer. :-)

Hmm, well I'd sure like to see the question and his whole response. I think
if he understood what you were proposing then he would disagree with you.
I'll probably take these posts to him at some stage (Uni is on break right
now, maybe next week) and see what he thinks. But until then I would
appreciate it if you could mail me your question and his response.

>
> [more confusion]
>
> Shannon didn't work, I didn't work, Togleri didn't work. I'm giving
> up.
>
> > "Disagree! Info theory is not nonsense. The person listening to
> radio static
> > is nonsense. Info theory quantifies the information content of
> a source, it
> > does not ascribe any meaning to the source"
> >
> > I hope you will think long and hard about this statement. It
> fits both what
> > I have been saying and also your quotes from Shannon.
>
> It does not fit in the slightest degree the misunderstandings you are
> calling 'information theory.' Perhaps seeing that Prof. Togleri has the
> appropriate view of what information theory relates to will help you
> see how you might improve your understanding.

I am using information theory as it should be used. As a tool to help
analyse information systems. REAL information systems, systems that require
the efficient storage, transmission and preservation of information which
has meaning.

Nobody would store information that has no meaning and DNA is no exception.
In the real world noise is the enemy of meaningful information. DNA creates
very complex cells, that requires the information to have meaning, random
noise will destroy that meaning.

>
> [...]
>
> > Incidentally this is way off track with respect to the creation of
> > information which was what was originally being debated.
>
> Without a shared comprehension of what information theory is *about*,
> there is little hope of having a complex discussion of its application
> to biology.
>

I treat information theory as a tool to help me do a job which I expect
people will pay for. You use it out of context to try to prove your personal
beliefs on how life developed on this planet.

The correct model is a channel and it shows information loss due to random
noise.

--------------------------------------------
Brad Jones
3rd Year BE(IT)
Electrical & Electronic Engineering
University of Western Australia