04/18/23Andrew Frank 1

Communication: Information Content

Andrew U. FrankGeoinformationTU Viennafrank@geoinfo.tuwien.ac.at

Overheads at:http://www.geoinfo.tuwien.ac.at/presentations/frank.htm

04/18/23Andrew Frank 2

Overview

Case: gamesCase: Real World – DrivingConclusions

04/18/23Andrew Frank 3

Question:

How to measure information?Shannon & Weaver:

1 bit = 1 binary decision (e.g. head or tail)

Works for data.How to apply to information?

04/18/23Andrew Frank 4

Definitions

Data : (machine readable) signs

Information: answers to questionsDescriptions of real world situationsUsed to decide on actions

Information is derived from data.

04/18/23Andrew Frank 5

Case Board Games

e.g. checkers (Dame)Differentiate:

The game of checkers (game)The game I played last Wednesday evening (play)

We analyzed the game by XX in 1995 (match)

04/18/23Andrew Frank 6

Game as algebra:

The game with rules is an algebra:class BoardGames g where

initialize:: g

move :: Player -> Position -> Position -> g -> g

isFinished:: g -> Bool

winner:: g -> Maybe Player

This signature seems to fir for most board games. The difference is in the instantiation, where e.g. ‘move’ checks for legality of a move according to the rules.

04/18/23Andrew Frank 7

Play as a sequence of operations

Playing a game is carrying out a sequence of operations:My partner and I moved yesterday evening alternatively pieces on my board.

Algebraically:g0 = initialize

g2 = move Black A2 A3. Move White A7 A6 $ g0

…

g17 =move …. g16

w2 = winner g17

04/18/23Andrew Frank 8

Match as an abstract description of a play separated from instantiationWeiß: Kasparov Gary (2595)Schwarz: Pribyl Josef (2395)1.d4 Sf6 2.c4 g6 3.Sc3 d5 4.cxd5 Sxd5 5.e4 Sxc3 6.bxc3 Lg7 ...

04/18/23Andrew Frank 9

Descriptions of a Match:

Representation as observable physical phenomena(including mental representation)

Information is linked to a representation,Information cannot exist independent of a

representation

04/18/23Andrew Frank 10

Alternative Descriptions of a Match- Natural Language- Alternative Formal Descriptions

Different encoding

- A play (instantiation)- …All are equivalent (in terms of the

algebra) They describe the same abstract match.

04/18/23Andrew Frank 11

Information as an Equivalence ClassAll descriptions which instantiate to the

same play are equivalent.

The information is the equivalence classhomomorphism between the representations

04/18/23Andrew Frank 12

Measure of the Information

Measure a representation (data)

How many binary decisions are necessary for the representation?=Logarithm base 2 of the number of different messages

Information content of all non-redundant representations must be equal.

04/18/23Andrew Frank 13

Chess:

How many different messages in a ‘move’ operation:

Player -> Position -> Position -> …2 * 64 * 64 = 32 768 = 2**77 bits per move

This encodes legal and illegal situations; it is the information content of arbitrary positioning and moving of pieces on a board.

04/18/23Andrew Frank 14

Information Content

Depends on the algebra underlying.The 7 bit calculation is the free algebra:All possible moves are legal

Real play: many moves are illegal,-> algebra with axioms-> less possible cases-> less informationMany moves are stupid; good chess players consider only reasonable moves

-> less information again

04/18/23Andrew Frank 15

Redundancy

Representations can contain more bits than the minimum necessary for a specific game.

The additional information can help to guard against transmission errors (but is not necessarily effective)

Better: use non-redundant representation and add redundancy systematically

04/18/23Andrew Frank 16

Conclusions:

Information is the content of the class of equivalent representations – equivalent with respect to the algebra of the game.

The equivalence class contains only the non-redundant information (with respect to a given algebra).

04/18/23Andrew Frank 17

Real World Situation: Driving in CityA friend gives me driving instructions:

Follow Rechte Wienzeile, turn into Schleifmuehlgasse …

I check the webRechte Wienzeile 0.7 kmSchleifmuehlgasse 0.1 km…

Or get a map:..

04/18/23Andrew Frank 18

Driving as an Algebra

Driving in a city is like a game and formalized as an algebra:

class Driving d where

startAt :: Location -> Driver -> d -> d

move :: Driver -> Location -> d -> d

isAt :: Driver -> d -> Location

This is a different model than the one used in the driving instructions.

04/18/23Andrew Frank 19

Equivalence of Instructions

The instructions are equivalent if they lead me correctly along the same path.

This is equivalence with respect to the algebra defined.

Information content can be measured the same way than for a a game!

04/18/23Andrew Frank 20

Different drivers – different algebra:Possible instructions:move :: Location -> Location -> d -> d

Drive to intersection Rechte Wienzeile/Schleifmuehlgasse,

Drive to intersection Schleifmuehlgasse/Margaretenstrasse

…

04/18/23Andrew Frank 21

Alternatives

Web instructions:move :: Street -> Distance -> d -> d

Rechte Wienzeile 0.7 kmSchleifmuehlgasse 0.1 km…Oral instructionsmove :: Location -> Location -> d -> dturn :: Direction -> d -> d

Follow Rechte Wienzeile to the intersection with Schleifmuehlgasse, turn right

Follow Schleifmuehlgasse to the intersection with Margaretenstrasse, turn right

04/18/23Andrew Frank 22

Ontological commitment:

Information is linked to a representation,Information cannot exist independent of a

representation

04/18/23Andrew Frank 23

Information content

Relative to an algebra:Minimal representation of description of a

sequence of actions.

04/18/23Andrew Frank 24

Redundancy

A representation can contain more data than necessary for the algebra, but not more information.

04/18/23Andrew Frank 25

Different algebra – different information content (from the same data)

Information content is different relative to the algebra:

Different people play different games; their algebras are different (e.g. more knowledge available)

04/18/23Andrew Frank 26

Information for sender and receiver is not always the same

What is redundant for one may be necessary for the other.

Information + Redundancy = constant

04/18/23Andrew Frank 27

Conclusion

No simple answer to ‘how much information’

Algebra gives framework in which questions can be posed and answered for specific cases.

Insight gained corresponds with experience.