[...]
>
>Let me try to explain what all this means. Mathematics cannot determine what
>two people have decided. In information theory information is determined
>not by what 2 people agree culturally or privately is information.
>Information is measured by compressibility of the sequence. The alphabet
>even minus the L has more information in it that does the sequence
>isisisisisisisisis. This is highly compressible.
>
I'm really kicking myself that I didn't see Burgy's message.
I noticed the L was missing and even said to myself no L,
but didn't make the connection. Interestingly, I did see
the theological twist James mentioned but figured this
couldn't possibly be the hidden message Burgy was talking
about :).
Anyway, just wanted to say I'm in agreement with what Glenn
has written. It's essential to understand that information
theory does not address the meaning of a message. There's a
simple reason for this. In designing a system for transmitting
information one doesn't want the channel to be passing
judgement on the transmitted message. The channel has to
transmit all information even if its junk. This doesn't
mean that the messages have no meaning, only that info
theory doesn't judge whether messages are meaningful or
not. Similarly, the genetic information system is also able
to transmit meaningful as well as junk messages.
When reading the newsgroup bionet.info-theory awhile back
I came across a really useful suggestion for interpretting
information content which I will apply to the present situation.
The basic idea is to associate information with surprise.
Imagine that you are receiving the message ABCD... one
character at a time. The amount of information contained
in the message can be roughly related to how surprised you
are with each new character. Take Glenn's example. After
receiving isis you are not surprised to see the next i and
you become less and less surprised with each repeat of is.
This is closely related to compressibility since your lack
of surprise comes from having noticed a pattern. The existence
of a pattern guarantees compressibility. One has to be careful
here that the patterns one observes are structural patterns
as opposed to patterns you might get by guessing the meaning
of the message.
Based on the above I would argue that the sequence ABCD...
contains little information since it is an ordered pattern.
After ABCDEF we would not be too surprised to see a G pop
out next. But after ABCDEFGHIJK we are really really expecting
to see an L. When M arrives instead we are surprised and begin
to doubt the pattern we thought we saw. After MNOPQR the
pattern re-establishes itself and we start to anticipate the
next charater. Thus, I would say Burgy's sequence had more
information than had he simply typed the entire alphabet
without missing a character.
Brian Harper
Associate Professor
Applied Mechanics
The Ohio State University
"... we have learned from much experience that all
philosophical intuitions about what nature is going
to do fail." -- Richard Feynman