IST 4 Information and Logic - Paradiseparadise.caltech.edu/ist4/lectures/lect 0614 p.pdf ·...

IST 4Information and Logic

mon tue wed thr fri31 M1 1= todayT

7 M1

14 1 2M2x= hw#x out

= todayT

T

oh oh

oh oh

21 228 M2

x= hw#x due

idt

oh oh

h h28 M2

5 312 3 4Mx= MQx out

midtermsoh

oh oh

oh = office hours oh

12 3 419 4 526

Mx= MQx out

Mx= MQx dueoh oh

ohoh

26

2 5oh

oh

oh

oh

Last lecture: Associative Memories, Yue Li

William Scoville(1906 – 1984)

Brenda Milner(1918 - )

Henry Molaison or H. M. (1926-2008)Suzanne Corkin

Alexander Luria(1902 – 1977)

J h FSarnoff A. Mednick

Joshua Foer

MemoryMQ2• You are invited to write short essay on the topic of theM t Q ti

QDeadline Thursday 4/29/2014 at 10pm

Magenta Question.• Recommended length is 3 pages (not more)• Submit the essay in PDF format to [email protected] the essay in PDF format to [email protected]

file name lastname-firstname.pdf• No collaboration. No extensionsGrading of MQ:3 points (out of 103)

50% for content quality, 50% for writing quality

Some students will be given an opportunityto give a short presentation for up to 3 additional points

A word that is associated with the following?with the following?

Life: Unique Origin – 3.5 Bya

Origins ??Memory

DNA

Novelty Novelty Selection?Editing? Mutation?g

New species?

Human brain: Unique Origin:2 000 l

Origins2,000 people in Africa – 60 Kyag

Memoryy

m yLanguages

Novelty g g

Novelty Interaction, editing, selection , g,

New ideas/languages

Funes the MemoriousWill be posted on the class website

Funes the MemoriousA short story by: Jorge Luis Borges

1899-19861899 1986

“With no effort, he had learned English French Portuguese andEnglish, French, Portuguese andLatin. I suspect, however, that he was not very capable of thought.”

“To think is to forget differences, generalize make abstractions Ingeneralize, make abstractions. Inthe teeming world of Funes, there were only details, almost immediate in th i ”their presence.”

Babylonian Clay Tablets G k P fGreek Proofs...

Memoryyof mathematical knowledge

MCMXX = 1930MCMXX = 1930

Some History on Roman Numeralsy

Origins of roman numerals are believed to be Origins of roman numerals are believed to be in the form of notches on tally sticks, such as those used by European shepherds

The Roman Numeral Puzzle: Very slow impact of the

Babylonians???Babylonians???Positional number

Babylonia 5,000 yasystem influence

Greek, India, Chinese 2,500 ya

Persia, Arabs 1,500 ya

Europe 500 ya

The Syntax of Roman Numerals

A Refresher on the Roman Numeral SystemRoman Numeral System

Roman Numeral Number Large

Roman NumberI 1V 5

NumeralsV 5,000

X 10L 50

X 10,000L 50,000C 100 000C 100

D 500M 1000

C 100,000D 500,000M 1 000 000M 1000 M 1,000,000

LCD Monitor

The Abacus: It’s all About SyntaxIt s all About Syntax

VLDVIIIII V VLDVIIIII

XXXXX

V

L

CCCCC D

MMMMM V

IXCM

LL C

VV X

LL

DD

C

M

The Abacus: It’s all About Syntax

VLDVIIIII V

It s all About SyntaxWhat is the number?

VLDVIIIII

XXXXX

V

L

CCCCC D

MMMMM V

IXCM

LL C

VV X

LL

DD

C

M


VLDVIIIII V


VLDVIIIII

XXXXX

V

L

CCCCC D

MMMMM V

IXCM

LL C

VV X

LL

DD

C

MDCI 601


VLDVIIIII V

It s all About Syntax

Touch the middle: Yes or NoVLDVIIIII

XXXXX

V

Lit is a binary mechanism

CCCCC D

MMMMM V

IXCM

LL C

VV X

LL

DD

C

MDCI 601

The AbacusCalculating Machine are Based on Syntax

The first actual calculating mechanism known to us is th b s hi h is th ht t h b n in nt d

Calculating Machine are Based on Syntax

the abacus, which is thought to have been invented by the Babylonians sometime between 1,000 BC and 500 BC

The original concept referred to a flat stone covered with sand or dust with pebbles being placed on lines with sand or dust, with pebbles being placed on lines drawn in the sand

Source: Wikipedia

The AbacusCalculating Machine are Based on Syntax

The original concept referred to a flat stone covered

Calculating Machine are Based on Syntax

The original concept referred to a flat stone covered with sand or dust, with pebbles being placed on lines drawn in the sand ?

?

In Phoenician the word abak means sand

In Hebrew the word abhaq ָאָבק means dust

Calculus is Latin for pebble

Source: Wikipedia


VLDVIIIII V


VLDVIIIII

XXXXX

V

L

CCCCC D

MMMMM V

IXCM

LL C

VV X

LL

DD

C

MDCI 601


VLDVIIIII V


VLDVIIIII

XXXXX

V

L

CCCCC D

MMMMM V

IXCM

LL C

VV X

LL

DD

C

MCCCCLXXXVIIII 489


VLDV


VLDV

IXCM

CCCCLXXXVIIII 489DCI 601 +


VLDV


What is the decimal VLDVWhat is the decimal representation?

MLXXXX ??

IXCM



VLDV


What is the decimal VLDVWhat is the decimal representation?

MLXXXX 0901

IXCMWhat’s wrongwith this picture?


Roman Numerals and Base 10 Systemsy

VLDVMLXXXX VLDVMLXXXXRoman numerals

The representation in the abacusis a positional base 10 representation

0901m um

used for numberRepresentation

is a positional base 10 representation

For calculation:we used the abacus

IXCMwe used the abacus

From Physical (abacus) From Physical (abacus) to Symbols

Algorizmsg

Algorizmig

A positional number system Operations are done on syntaxp yis a key enabler for efficient arithmetic operations

Operations are done on syntax

Muhammad ibn Mūsā al-Khwārizmī

الخوارزمي موسى بن محمد 780 850ADي رز و ى و 850AD-780 بن

A Persian mathematician, who wrote on Hindu-Arabic numerals and was among the first to use zero as a place numerals and was among the first to use zero as a place holder in positional base notation. The word algorithm derives from his name. His book Kitab al-jabr w'al-muqabala gives us the word algebra

Source: Wikipedia

The Beginning of the “Algebra Book ” by “Algorizmi ”

Everything requires computation...

The Beginning of the “Algebra Book ” by “Algorizmi ”

Positional: order is important; from 1 to infinity...

Example from the “Algebra Book ” by “Algorizmi ”computation = syntax manipulation

????

It is rhetorical (words) no symbols

Algorithms and Algebra in Europeg g pLeonardo Fibonacci1170-1250AD Leonardo was born in Pisa his father directed Leonardo was born in Pisa, his father directed

a trading post in Bugia, a port east of Algiers in North Africa, as a young boy Leonardo traveled there to help him This is where he learned about there to help him. This is where he learned about the Arabic numeral system

Perceiving that arithmetic with Arabic numerals is simpler and more efficient than with Roman numerals, Fibonacci traveled ,throughout the Mediterranean world to study under the leading Arab mathematicians of the time, returning around 1200. t me, return ng around 00. In 1202, at age 32, he published what he had learned in Liber Abaci, or Book of Calculation

Source: Wikipedia

Liber Abaci – First ChapterL F p

Introduction of the syntax; from 1 to infinity...

Liber Abaci – First ChapterL F p

Positional: order is importantPositional: order is important

al-Khwarizmi Dear Caltech students al Khwarizmi780-850AD an algorizm is a procedure

for syntax (language) i ( iti )processing (composition)

Dear Mr. Algorizmi, how would you define an algorithm??? Thank you!

In first grade:We use our BRAIN for remembering We use our BRAIN for remembering

Algorizmi’s syntax

In first grade:We use our BRAIN for remembering We use our BRAIN for remembering

Algorizmi’s syntax

Avoiding headaches!U b f b i Use syntax-boxes for remembering

Algorizmi’s syntaxg y

Syntax BoxesSyntax Boxescomposition and magiccomposition and magic

Syntax Manipulation with Boxesy p

ba b

inputs

a b o

S-Box

outputso

p


b0 0

inputs

a b o

0 0 0

outputs0

p


b0 1

inputs

a b o

0 0 010 1

outputs1

p


b0 2

inputs

a b o

0 0 010

0 212

outputs2

p


b1 2

inputs

a b o

0 0 010

0 212

21 2

outputs

21 2

2p


b2 1

inputs

a b o

0 0 010

0 212

21 2

outputs

2

1

1

2

2

22p 12 2


bCan we ‘compute’ max (a,b,c) with this s-box?

a binputs

a b o

0 0 010

0 21 0

1211 0

12

11

112

?outputs

201

122

222o

p 1222

22

o = max (a,b)


o = max (a,b)Can we ‘compute’ max (a,b,c) with this s-box?

a b


Can we ‘compute’ max (a,b,c) with this s-box? o = max (a,b)

a b Composition:build big s-boxes

c

build big s boxes from small s-boxes

c


bCan we ‘compute’ zwith the max s-box?

a ba b o

0 0 0

with the max s box?

?10

0 21 0

121

a b z

0 0 0 1 012

11

112

0 010

1 0

011 2

01

122

222

1 011

11

o 1222

22

o = max (a,b)YES


bCan we ‘compute’ wwith the max s-box?

a ba b o

0 0 0

with the max s box?

?10

0 21 0

121

a b w

0 0 0 1 012

11

112

0 010

1 0

011 2

01

122

222

1 011

10

o 1222

22

o = max (a,b)NO Why not?


Can we ‘compute’ wwith the max s-box? Composition:

a b

with the max s box?

?

pbuild big s-boxes from small s-boxes

a b

0 0 0

The output of every small s-box is bigger or equal to its inputs

w

Big s-box0 0

101 0

011

or equal to its inputs

1 011

10 The output of the big

s-box must be bigger l t it i t

w

or equal to its inputsNO Why not?

fi it i lIs there a finite universal set of building blocks?

‘ thi ’Can construct ‘everything’.The most important idea in InformationThe most important idea in Information

DNAABCDE DNAABCDE...

A Magic (Universal) Boxg ( )

A binary s box that can A binary s-box that can compute any binary s-box?

a ba b m

p y y

a b m

0 010

1110

1 011

110

HW#1

11 0m

A Magic Boxg

Can you compute the min(x,y)

Can you compute the following with the magic box?

a ba b m x y oa b m

0 010

11

x y

0 010

o

0010

1 011

110

101 0

11

00111 0

m11 1

A Magic BoxHint 1: g

Can you compute the

Hint 1: min(x,y)


a ba b m x y oa b m

0 010

11

x y

0 010

o

0010

1 011

110

101 0

11

00111 0

m11 1

A Magic BoxHint 2: g

Can you compute the

Hint 2: min(x,y)


a ba b m x y oa b m

0 010

11

x y

0 010

o

0010

1 011

110

101 0

11

00111 0

m11 1

A Magic Boxg

x y

min(x,y)

x y

a b m x y o1a b m

0 010

11

x y

0 010

o

00

1

101 0

11

110

101 0

11

00111 0 11 1

o0

A Magic Boxg

x y

min(x,y)

x y

a b m x y o0a b m

0 010

11

x y

0 010

o

00

0

101 0

11

110

101 0

11

00111 0 11 1

o1

A Magic BoxHINT

g

Can you compute the

HINT max(x,y)

Can you compute the following with the m-box?

a ba b m x y oa b m

0 010

11

x y

0 010

o

0110

1 011

110

101 0

11

11111 0

m11 1

A Magic Boxg

x y

max(x,y)

x y

a b m x y oa b m

0 010

11

x y

0 010

o

0110

1 011

110

101 0

11

11111 0 11 1

o

A Magic Boxg

x y

max(x,y)

x y0 0

a b m x y oa b m

0 010

11

x y

0 010

o

0110

1 011

110

101 0

11

111

11

11 0 11 1

o0

A Magic Boxg

x y

max(x,y)

x y0 1

a b m x y oa b m

0 010

11

x y

0 010

o

0110

1 011

110

101 0

11

111

01

11 0 11 1

o1

A Magic Boxg

x y

max(x,y)

x y1 1

a b m x y oa b m

0 010

11

x y

0 010

o

0110

1 011

110

101 0

11

111

00

11 0 11 1

o1

m-Box: A two input binary syntax nbox that can compute any (two

input) binary syntax box?input) binary syntax box?How many differentbinary 2 input How will you prove it?

a ba b m x y o

binary 2-input s-boxes?

How will you prove it?24 = 16

a b m

0 010

11

x y

0 010

o

**10

1 011

110

101 0

11

***11 0

m

11

S nt x B xSyntax Boxesproof of universalityproof of universality

4 Useful Boxes

min(x,y) 1-y 1

x y o

0 0 1

x y o

0 0 1 0 010

1 0

111

0 010

1 0

101

max(x,y)

1 011

11

1 011

10

1-y

yy

a b m x y oa b m

0 010

11

x y o

0 010

1010

1 011

110

101 0

11

010

o

11 0 11 0

1

y 1-y

a b m x y oa b m

0 010

11

x y o

0 010

1110

1 011

110

101 0

11

111

o

11 0 11 1

4 Useful Boxes

min(x,y) 1-y 1

( ) So what?max(x,y)

Need to prove:So what?

Need to prove:any (t i t) bin nt x any (two input) binary syntax box can be computed by box can be computed by the 4 Useful Boxes

An Arbitrary Two Input Box

x y oTwo 1-input boxes!

0 010

**

1 011

**

x= 0 then

x= 1 then x= 1 then

What are the possible values of p

00

01

10

110 1 0 1


min(x,y) 1-y 1

( )max(x,y)

y

What are the possible values of

y

x y op

00

01

10

11

0 010

**0 1 0 1

Can we compute it with the m-box?1 0

11**


x y o

0 0 *

x= 0 then

1 th 0 010

1 0

***

x= 1 then How can you compute this box?

1 011 *

x

o

x= 0 then

x= 1 then

1-x x1 x x

min(a,b) min(a,b)

max(a,b)

o

x= 0 then

x= 1 then

0 10 1

min(a,b) min(a,b)

0

max(a,b)

0

o

x= 0 then

x= 1 then

1 01 0

min(a,b) min(a,b)

0

max(a,b)

0

o

QED

Does the magic continue?D m g u

Given a 2-input binary box that can compute any 2-input binary box

Can it compute any 3-input binary box?

a b x zy

m-box

o o

3-input binary s-box

x y ozHow many differentbinary 3-input s boxes?

0 010

**

00

s-boxes?

28 = 2561 0

11**

0 0 *

0010 0

101 0

***

1111 0

11**

11

3-input binary s-box

x y ozTwo 2-input boxes!

0 010

**

00

1 011

**

0 0 *

001

z= 0 then

z= 1 then 0 010

1 0

***

111

z= 1 then

1 011

**

11

x y o

0 0 *

z

0x y x y

0 010

1 0**

000

11 *0 0 *

01

101 0

**

11

11 *1

z

z= 0 then o z= 1 then

x y o

0 0 *

z

0z= 0 then

0 010

1 0**

000

???? z z= 1 then

11 *0 0 *

01

o

101 0

**

11

z1-z

11 *1min(a,b) min(a,b)

m x( b)max(a,b)

o

x y o

0 0 *

z

00 010

1 0**

00001

11 *0 0 *

01

101 0

**

11

min(a,b) min(a,b)

0 11 *1max(a,b)

0

o

x y o

0 0 *

z

00 010

1 0**

00010

11 *0 0 *

01

101 0

**

11

min(a,b) min(a,b)

0 11 *1max(a,b)

0

o

Does the magic continue?

G l b f 2 b b

D m g u

Given a magical box for any 2-input binary box

We proved that it is magical for any 3-input binary box!p g f y p y

Is it magical for any n-input binary box????YES!!!!Proof by induction on the number of inputs

a bProof by induction on the number of inputs

m-box

....

o o

z= 0 then

1 th Are tables with n 1 variablesz= 1 then

1

Are tables with n-1 variables

z1-z

min(a,b) min(a,b)

max(a b)max(a,b)

o

We need a language forS boxes!!S-boxes!!

Questions about building blocks?Feasibility

Questions about building blocks?

Given a set of building blocks: What can/cannot be constructed?

Efficiency and complexity

Given a set of building blocks and a description of a structure:Given a set of building blocks and a description of a structure:

Size: If feasible, how many blocks are needed?

Time: How long will it take to complete the construction?

A word that is associated with the following?with the following?

FaceFace

You have one week!You have one week!

Diff i i ti 7 di it b 60Difference in approximation - 7 digits base 60

IST 4 Information and Logic - Paradiseparadise.caltech.edu/ist4/lectures/lect 0614 p.pdf ·...

Documents

Transcript of IST 4 Information and Logic - Paradiseparadise.caltech.edu/ist4/lectures/lect 0614 p.pdf ·...