Post on 18-Mar-2020
IST 4Information and Logic
mon tue wed thr fri31 M1 1= todayT
7 M1
14 1 2M2x= hw#x out
= todayT
T
oh oh
oh oh
21 228 M2
x= hw#x due
idt
oh oh
h h28 M2
5 312 3 4Mx= MQx out
midtermsoh
oh oh
oh = office hours oh
12 3 419 4 526
Mx= MQx out
Mx= MQx dueoh oh
ohoh
26
2 5oh
oh
oh
oh
Last lecture: Associative Memories, Yue Li
William Scoville(1906 – 1984)
Brenda Milner(1918 - )
Henry Molaison or H. M. (1926-2008)Suzanne Corkin
Alexander Luria(1902 – 1977)
J h FSarnoff A. Mednick
Joshua Foer
MQ2Q
MemoryMQ2• You are invited to write short essay on the topic of theM t Q ti
QDeadline Thursday 4/29/2014 at 10pm
Magenta Question.• Recommended length is 3 pages (not more)• Submit the essay in PDF format to ta4@paradise.caltech.eduSubmit the essay in PDF format to ta4@paradise.caltech.edu
file name lastname-firstname.pdf• No collaboration. No extensionsGrading of MQ:3 points (out of 103)
50% for content quality, 50% for writing quality
Some students will be given an opportunityto give a short presentation for up to 3 additional points
A word that is associated with the following?with the following?
Life: Unique Origin – 3.5 Bya
Origins ??Memory
DNA
Novelty Novelty Selection?Editing? Mutation?g
New species?
Human brain: Unique Origin:2 000 l
Origins2,000 people in Africa – 60 Kyag
Memoryy
m yLanguages
Novelty g g
Novelty Interaction, editing, selection , g,
New ideas/languages
Funes the MemoriousWill be posted on the class website
Funes the MemoriousA short story by: Jorge Luis Borges
1899-19861899 1986
“With no effort, he had learned English French Portuguese andEnglish, French, Portuguese andLatin. I suspect, however, that he was not very capable of thought.”
“To think is to forget differences, generalize make abstractions Ingeneralize, make abstractions. Inthe teeming world of Funes, there were only details, almost immediate in th i ”their presence.”
Babylonian Clay Tablets G k P fGreek Proofs...
Memoryyof mathematical knowledge
MCMXX = 1930MCMXX = 1930
Some History on Roman Numeralsy
Origins of roman numerals are believed to be Origins of roman numerals are believed to be in the form of notches on tally sticks, such as those used by European shepherds
The Roman Numeral Puzzle: Very slow impact of the
Babylonians???Babylonians???Positional number
Babylonia 5,000 yasystem influence
Greek, India, Chinese 2,500 ya
Persia, Arabs 1,500 ya
Europe 500 ya
The Syntax of Roman Numerals
A Refresher on the Roman Numeral SystemRoman Numeral System
Roman Numeral Number Large
Roman NumberI 1V 5
NumeralsV 5,000
X 10L 50
X 10,000L 50,000C 100 000C 100
D 500M 1000
C 100,000D 500,000M 1 000 000M 1000 M 1,000,000
LCD Monitor
The Abacus: It’s all About SyntaxIt s all About Syntax
VLDVIIIII V VLDVIIIII
XXXXX
V
L
CCCCC D
MMMMM V
IXCM
LL C
VV X
LL
DD
C
M
The Abacus: It’s all About Syntax
VLDVIIIII V
It s all About SyntaxWhat is the number?
VLDVIIIII
XXXXX
V
L
CCCCC D
MMMMM V
IXCM
LL C
VV X
LL
DD
C
M
The Abacus: It’s all About Syntax
VLDVIIIII V
It s all About SyntaxWhat is the number?
VLDVIIIII
XXXXX
V
L
CCCCC D
MMMMM V
IXCM
LL C
VV X
LL
DD
C
MDCI 601
The Abacus: It’s all About Syntax
VLDVIIIII V
It s all About Syntax
Touch the middle: Yes or NoVLDVIIIII
XXXXX
V
Lit is a binary mechanism
CCCCC D
MMMMM V
IXCM
LL C
VV X
LL
DD
C
MDCI 601
The AbacusCalculating Machine are Based on Syntax
The first actual calculating mechanism known to us is th b s hi h is th ht t h b n in nt d
Calculating Machine are Based on Syntax
the abacus, which is thought to have been invented by the Babylonians sometime between 1,000 BC and 500 BC
The original concept referred to a flat stone covered with sand or dust with pebbles being placed on lines with sand or dust, with pebbles being placed on lines drawn in the sand
Source: Wikipedia
The AbacusCalculating Machine are Based on Syntax
The original concept referred to a flat stone covered
Calculating Machine are Based on Syntax
The original concept referred to a flat stone covered with sand or dust, with pebbles being placed on lines drawn in the sand ?
?
In Phoenician the word abak means sand
In Hebrew the word abhaq ָאָבק means dust
Calculus is Latin for pebble
Source: Wikipedia
The Abacus: It’s all About Syntax
VLDVIIIII V
It s all About Syntax
VLDVIIIII
XXXXX
V
L
CCCCC D
MMMMM V
IXCM
LL C
VV X
LL
DD
C
MDCI 601
The Abacus: It’s all About Syntax
VLDVIIIII V
It s all About SyntaxWhat is the number?
VLDVIIIII
XXXXX
V
L
CCCCC D
MMMMM V
IXCM
LL C
VV X
LL
DD
C
MCCCCLXXXVIIII 489
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
VLDV
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
VLDV
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
VLDV
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
VLDV
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
VLDV
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
VLDV
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
VLDV
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
VLDV
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
VLDV
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
VLDV
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
What is the decimal VLDVWhat is the decimal representation?
MLXXXX ??
IXCM
CCCCLXXXVIIII 489DCI 601 +
The Abacus: It’s all About Syntax
VLDV
It s all About Syntax
What is the decimal VLDVWhat is the decimal representation?
MLXXXX 0901
IXCMWhat’s wrongwith this picture?
CCCCLXXXVIIII 489DCI 601 +
Roman Numerals and Base 10 Systemsy
VLDVMLXXXX VLDVMLXXXXRoman numerals
The representation in the abacusis a positional base 10 representation
0901m um
used for numberRepresentation
is a positional base 10 representation
For calculation:we used the abacus
IXCMwe used the abacus
From Physical (abacus) From Physical (abacus) to Symbols
Algorizmsg
Algorizmig
A positional number system Operations are done on syntaxp yis a key enabler for efficient arithmetic operations
Operations are done on syntax
Muhammad ibn Mūsā al-Khwārizmī
الخوارزمي موسى بن محمد 780 850ADي رز و ى و 850AD-780 بن
A Persian mathematician, who wrote on Hindu-Arabic numerals and was among the first to use zero as a place numerals and was among the first to use zero as a place holder in positional base notation. The word algorithm derives from his name. His book Kitab al-jabr w'al-muqabala gives us the word algebra
Source: Wikipedia
The Beginning of the “Algebra Book ” by “Algorizmi ”
Everything requires computation...
The Beginning of the “Algebra Book ” by “Algorizmi ”
Positional: order is important; from 1 to infinity...
Example from the “Algebra Book ” by “Algorizmi ”computation = syntax manipulation
????
It is rhetorical (words) no symbols
Algorithms and Algebra in Europeg g pLeonardo Fibonacci1170-1250AD Leonardo was born in Pisa his father directed Leonardo was born in Pisa, his father directed
a trading post in Bugia, a port east of Algiers in North Africa, as a young boy Leonardo traveled there to help him This is where he learned about there to help him. This is where he learned about the Arabic numeral system
Perceiving that arithmetic with Arabic numerals is simpler and more efficient than with Roman numerals, Fibonacci traveled ,throughout the Mediterranean world to study under the leading Arab mathematicians of the time, returning around 1200. t me, return ng around 00. In 1202, at age 32, he published what he had learned in Liber Abaci, or Book of Calculation
Source: Wikipedia
Liber Abaci – First ChapterL F p
Introduction of the syntax; from 1 to infinity...
Liber Abaci – First ChapterL F p
Positional: order is importantPositional: order is important
al-Khwarizmi Dear Caltech students al Khwarizmi780-850AD an algorizm is a procedure
for syntax (language) i ( iti )processing (composition)
Dear Mr. Algorizmi, how would you define an algorithm??? Thank you!
In first grade:We use our BRAIN for remembering We use our BRAIN for remembering
Algorizmi’s syntax
In first grade:We use our BRAIN for remembering We use our BRAIN for remembering
Algorizmi’s syntax
Avoiding headaches!U b f b i Use syntax-boxes for remembering
Algorizmi’s syntaxg y
Syntax BoxesSyntax Boxescomposition and magiccomposition and magic
Syntax Manipulation with Boxesy p
ba b
inputs
a b o
S-Box
outputso
p
Syntax Manipulation with Boxesy p
b0 0
inputs
a b o
0 0 0
outputs0
p
Syntax Manipulation with Boxesy p
b0 1
inputs
a b o
0 0 010 1
outputs1
p
Syntax Manipulation with Boxesy p
b0 2
inputs
a b o
0 0 010
0 212
outputs2
p
Syntax Manipulation with Boxesy p
b1 2
inputs
a b o
0 0 010
0 212
21 2
outputs
21 2
2p
Syntax Manipulation with Boxesy p
b2 1
inputs
a b o
0 0 010
0 212
21 2
outputs
2
1
1
2
2
22p 12 2
Syntax Manipulation with Boxesy p
bCan we ‘compute’ max (a,b,c) with this s-box?
a binputs
a b o
0 0 010
0 21 0
1211 0
12
11
112
?outputs
201
122
222o
p 1222
22
o = max (a,b)
Syntax Manipulation with Boxesy p
o = max (a,b)Can we ‘compute’ max (a,b,c) with this s-box?
a b
Syntax Manipulation with Boxesy p
Can we ‘compute’ max (a,b,c) with this s-box? o = max (a,b)
a b Composition:build big s-boxes
c
build big s boxes from small s-boxes
c
Syntax Manipulation with Boxesy p
bCan we ‘compute’ zwith the max s-box?
a ba b o
0 0 0
with the max s box?
?10
0 21 0
121
a b z
0 0 0 1 012
11
112
0 010
1 0
011 2
01
122
222
1 011
11
o 1222
22
o = max (a,b)YES
Syntax Manipulation with Boxesy p
bCan we ‘compute’ wwith the max s-box?
a ba b o
0 0 0
with the max s box?
?10
0 21 0
121
a b w
0 0 0 1 012
11
112
0 010
1 0
011 2
01
122
222
1 011
10
o 1222
22
o = max (a,b)NO Why not?
Syntax Manipulation with Boxesy p
Can we ‘compute’ wwith the max s-box? Composition:
a b
with the max s box?
?
pbuild big s-boxes from small s-boxes
a b
0 0 0
The output of every small s-box is bigger or equal to its inputs
w
Big s-box0 0
101 0
011
or equal to its inputs
1 011
10 The output of the big
s-box must be bigger l t it i t
w
or equal to its inputsNO Why not?
fi it i lIs there a finite universal set of building blocks?
‘ thi ’Can construct ‘everything’.The most important idea in InformationThe most important idea in Information
DNAABCDE DNAABCDE...
A Magic (Universal) Boxg ( )
A binary s box that can A binary s-box that can compute any binary s-box?
a ba b m
p y y
a b m
0 010
1110
1 011
110
HW#1
11 0m
A Magic Boxg
Can you compute the min(x,y)
Can you compute the following with the magic box?
a ba b m x y oa b m
0 010
11
x y
0 010
o
0010
1 011
110
101 0
11
00111 0
m11 1
A Magic BoxHint 1: g
Can you compute the
Hint 1: min(x,y)
Can you compute the following with the magic box?
a ba b m x y oa b m
0 010
11
x y
0 010
o
0010
1 011
110
101 0
11
00111 0
m11 1
A Magic BoxHint 2: g
Can you compute the
Hint 2: min(x,y)
Can you compute the following with the magic box?
a ba b m x y oa b m
0 010
11
x y
0 010
o
0010
1 011
110
101 0
11
00111 0
m11 1
A Magic Boxg
x y
min(x,y)
x y
a b m x y o1a b m
0 010
11
x y
0 010
o
00
1
101 0
11
110
101 0
11
00111 0 11 1
o0
A Magic Boxg
x y
min(x,y)
x y
a b m x y o0a b m
0 010
11
x y
0 010
o
00
0
101 0
11
110
101 0
11
00111 0 11 1
o1
A Magic BoxHINT
g
Can you compute the
HINT max(x,y)
Can you compute the following with the m-box?
a ba b m x y oa b m
0 010
11
x y
0 010
o
0110
1 011
110
101 0
11
11111 0
m11 1
A Magic Boxg
x y
max(x,y)
x y
a b m x y oa b m
0 010
11
x y
0 010
o
0110
1 011
110
101 0
11
11111 0 11 1
o
A Magic Boxg
x y
max(x,y)
x y0 0
a b m x y oa b m
0 010
11
x y
0 010
o
0110
1 011
110
101 0
11
111
11
11 0 11 1
o0
A Magic Boxg
x y
max(x,y)
x y0 1
a b m x y oa b m
0 010
11
x y
0 010
o
0110
1 011
110
101 0
11
111
01
11 0 11 1
o1
A Magic Boxg
x y
max(x,y)
x y1 1
a b m x y oa b m
0 010
11
x y
0 010
o
0110
1 011
110
101 0
11
111
00
11 0 11 1
o1
m-Box: A two input binary syntax nbox that can compute any (two
input) binary syntax box?input) binary syntax box?How many differentbinary 2 input How will you prove it?
a ba b m x y o
binary 2-input s-boxes?
How will you prove it?24 = 16
a b m
0 010
11
x y
0 010
o
**10
1 011
110
101 0
11
***11 0
m
11
S nt x B xSyntax Boxesproof of universalityproof of universality
4 Useful Boxes
min(x,y) 1-y 1
x y o
0 0 1
x y o
0 0 1 0 010
1 0
111
0 010
1 0
101
max(x,y)
1 011
11
1 011
10
1-y
yy
a b m x y oa b m
0 010
11
x y o
0 010
1010
1 011
110
101 0
11
010
o
11 0 11 0
1
y 1-y
a b m x y oa b m
0 010
11
x y o
0 010
1110
1 011
110
101 0
11
111
o
11 0 11 1
4 Useful Boxes
min(x,y) 1-y 1
( ) So what?max(x,y)
Need to prove:So what?
Need to prove:any (t i t) bin nt x any (two input) binary syntax box can be computed by box can be computed by the 4 Useful Boxes
An Arbitrary Two Input Box
x y oTwo 1-input boxes!
0 010
**
1 011
**
x= 0 then
x= 1 then x= 1 then
What are the possible values of p
00
01
10
110 1 0 1
An Arbitrary Two Input Box
min(x,y) 1-y 1
( )max(x,y)
y
What are the possible values of
y
x y op
00
01
10
11
0 010
**0 1 0 1
Can we compute it with the m-box?1 0
11**
An Arbitrary Two Input Box
x y o
0 0 *
x= 0 then
1 th 0 010
1 0
***
x= 1 then How can you compute this box?
1 011 *
x
o
x= 0 then
x= 1 then
1-x x1 x x
min(a,b) min(a,b)
max(a,b)
o
x= 0 then
x= 1 then
0 10 1
min(a,b) min(a,b)
0
max(a,b)
0
o
x= 0 then
x= 1 then
1 01 0
min(a,b) min(a,b)
0
max(a,b)
0
o
QED
Does the magic continue?D m g u
Given a 2-input binary box that can compute any 2-input binary box
Can it compute any 3-input binary box?
a b x zy
m-box
o o
3-input binary s-box
x y ozHow many differentbinary 3-input s boxes?
0 010
**
00
s-boxes?
28 = 2561 0
11**
0 0 *
0010 0
101 0
***
1111 0
11**
11
3-input binary s-box
x y ozTwo 2-input boxes!
0 010
**
00
1 011
**
0 0 *
001
z= 0 then
z= 1 then 0 010
1 0
***
111
z= 1 then
1 011
**
11
x y o
0 0 *
z
0x y x y
0 010
1 0**
000
11 *0 0 *
01
101 0
**
11
11 *1
z
z= 0 then o z= 1 then
x y o
0 0 *
z
0z= 0 then
0 010
1 0**
000
???? z z= 1 then
11 *0 0 *
01
o
101 0
**
11
z1-z
11 *1min(a,b) min(a,b)
m x( b)max(a,b)
o
x y o
0 0 *
z
00 010
1 0**
00001
11 *0 0 *
01
101 0
**
11
min(a,b) min(a,b)
0 11 *1max(a,b)
0
o
x y o
0 0 *
z
00 010
1 0**
00010
11 *0 0 *
01
101 0
**
11
min(a,b) min(a,b)
0 11 *1max(a,b)
0
o
Does the magic continue?
G l b f 2 b b
D m g u
Given a magical box for any 2-input binary box
We proved that it is magical for any 3-input binary box!p g f y p y
Is it magical for any n-input binary box????YES!!!!Proof by induction on the number of inputs
a bProof by induction on the number of inputs
m-box
....
o o
z= 0 then
1 th Are tables with n 1 variablesz= 1 then
1
Are tables with n-1 variables
z1-z
min(a,b) min(a,b)
max(a b)max(a,b)
o
We need a language forS boxes!!S-boxes!!
Questions about building blocks?Feasibility
Questions about building blocks?
Given a set of building blocks: What can/cannot be constructed?
Efficiency and complexity
Given a set of building blocks and a description of a structure:Given a set of building blocks and a description of a structure:
Size: If feasible, how many blocks are needed?
Time: How long will it take to complete the construction?
A word that is associated with the following?with the following?
FaceFace
A word that is associated with the following?with the following?
FaceFace
You have one week!You have one week!
Diff i i ti 7 di it b 60Difference in approximation - 7 digits base 60