Post on 02-Dec-2014
description
Yann-Gaël Guéhéneuc
This work is licensed under a Creative Commons Attribution-NonCommercial-
ShareAlike 3.0 Unported License
Some Theory and Practice on Patterns – In Theory
NII, Tokyo, Japan12/02/14
2/216
3/216
Did you know that…
4/216
Vision is our dominant sense?
5/216
6/216
7/216
How does vision work?
8/216
Outline
Visual systemMemory modelsMental models Schema theory Software patterns Empirical studies Conclusion
9/216
Outline
Visual systemMemory modelsMental models Schema theory Software patterns Empirical studies Conclusion
10/216
Visual System
Vision Science: Photons to Phenomenology by Stephen E. Palmer
11/216
Visual SystemRetinalImage
Image-basedStage
Surface-basedStage
Item-basedStage
Category-basedStage
WorkingMemory
12/216
Visual SystemRetinalImage
Image-basedStage
Surface-basedStage
Item-basedStage
Category-basedStage
WorkingMemory
RegionAnalysis
Figure/GroupDistinction
VisualInterpolation
ItemsGrouping
13/216
Visual SystemRetinalImage
Image-basedStage
Surface-basedStage
Item-basedStage
Category-basedStage
WorkingMemory
RegionAnalysis
Figure/GroupDistinction
VisualInterpolation
ItemsGrouping
Comparison Decision
14/216
Visual SystemRetinalImage
Image-basedStage
Surface-basedStage
Item-basedStage
Category-basedStage
WorkingMemory
RegionAnalysis
Figure/GroupDistinction
VisualInterpolation
ItemsGrouping
Comparison Decision
Items(Visuo-spatial scratchpad)
ArticulatoryLoop
CentralExecutive
Long-termMemory
15/216
Visual SystemRetinalImage
Image-basedStage
Surface-basedStage
Item-basedStage
Category-basedStage
WorkingMemory
RegionAnalysis
Figure/GroupDistinction
VisualInterpolation
ItemsGrouping
Comparison Decision
Items(Visuo-spatial scratchpad)
ArticulatoryLoop
CentralExecutive
Long-termMemory
16/216
Visual SystemRetinalImage
Image-basedStage
Surface-basedStage
Item-basedStage
Category-basedStage
WorkingMemory
RegionAnalysis
Figure/GroupDistinction
VisualInterpolation
ItemsGrouping
Comparison Decision
Items(Visuo-spatial scratchpad)
ArticulatoryLoop
CentralExecutive
Long-termMemory
17/216
Visual System
Retinal image
18/216
Visual System
Retinal image
19/216
Visual System
Region analysis– Distinguish items based on their regions
20/216
Visual System
Region analysis– Distinguish items based on their regions
Zhuolin Jiang and Larry S. Davis ; Submodular Salient Region Detection ;Conference on Computer Vision and Pattern Recognition, IEEE, 2013
21/216
Visual System
Items grouping– Gestalt theory
• Gestalten: to put in form, to give structure
Fritz Perls*Berlin, July 8, 1893
†Chicago, March 14, 1970
22/216
Visual System
23/216
Visual System
24/216
Visual System
25/216
Visual System
26/216
Visual System
Items grouping– Laws of the Gestalt
27/216
Visual System
Items grouping– Laws of the Gestalt
• Good form
28/216
Visual System
Items grouping– Laws of the Gestalt
• Good form
29/216
Visual System
Items grouping– Laws of the Gestalt
• Good form
• Continuity
30/216
Visual System
Items grouping– Laws of the Gestalt
• Good form
• Continuity
31/216
Visual System
Items grouping– Laws of the Gestalt
• Good form
• Continuity
• Proximity
32/216
Visual System
Items grouping– Laws of the Gestalt
• Good form
• Continuity
• Proximity
33/216
Visual System
Items grouping– Laws of the Gestalt
• Good form
• Continuity
• Proximity
• Similitude
34/216
Visual System
Items grouping– Laws of the Gestalt
35/216
Visual System
Items grouping– Laws of the Gestalt
• Common destiny
36/216
Visual System
Items grouping– Laws of the Gestalt
• Common destiny
37/216
Visual System
Items grouping– Laws of the Gestalt
• Common destiny
• Closure
38/216
Visual System
Items grouping– Laws of the Gestalt
• Common destiny
• Closure
Completion
39/216
Visual System
Items grouping– Involvement of the memory
40/216
Visual System
Items grouping– Involvement of the memory
41/216
Visual System
Comparison– Involvement of the memory
42/216
Visual System
Comparison– Involvement of the memory
• Spatial
43/216
Visual System
Comparison– Involvement of the memory
• Spatial• Temporal
44/216
Outline
Visual systemMemory modelsMental models Schema theory Software patterns Empirical studies Conclusion
45/216
Memory Models
One modelItems
(Visuo-spatial scratchpad)Articulatory
Loop
CentralExecutive
Long-termMemory
46/216
Memory Models
One modelItems
(Visuo-spatial scratchpad)Articulatory
Loop
CentralExecutive
Long-termMemory
EpisodicMemory
ProceduralMemory
SemanticMemory
47/216
Memory Models
One modelItems
(Visuo-spatial scratchpad)Articulatory
Loop
CentralExecutive
Long-termMemory
EpisodicMemory
ProceduralMemory
SemanticMemory
ObjectsRepresentations
CategoriesRepresentations
48/216
Memory Models
One model
EpisodicMemory
ProceduralMemory
SemanticMemory
ObjectsRepresentations
CategoriesRepresentations
49/216
Memory Models
Another model
50/216
Memory Models
Yet another model
51/216
Memory Models
Yet another model
52/216
Memory Models
Yet, yet another model
53/216
Memory Models
Miller, 1956
54/216
Memory Models
Miller, 1956
M R B X Q V L W Q K
55/216
Memory Models
Miller, 1956
?
56/216
Memory Models
Miller, 1956
?
57/216
Memory Models
Miller, 1956
The Magical Number Seven, Plus or Minus Two:Some Limits on Our Capacity for Processing Information
?
58/216
Memory Models
Brooks, 1978
59/216
Memory Models
Brooks, 1978– Problem domain
60/216
Memory Models
Brooks, 1978– Problem domain– Executing program
61/216
Memory Models
Brooks, 1978– Problem domain– Executing program
– Knowledge domains• Hypothesis-and-verify process• Importance of documentation
62/216
Memory Models
Brooks, 1978– Problem domain– Executing program
– Knowledge domains• Hypothesis-and-verify process• Importance of documentation
63/216
Memory Models
Soloway, 1983– Experts’ knowledge that novices do not have
• Programming plans– RUNNING TOTAL LOOP PLAN– ITEM SEARCH LOOP PLAN– …
• Rules of programming discourse– An IF should be used when a statement body is guaranteed
to be executed only once and a WHILE used when a statement body may need to be repeatedly executed
– …
64/216
Memory Models
Soloway, 1983
65/216
Memory Models
Soloway, 1983An IF should be used when a statement body is guaranteed to be executed only once and a WHILE used when a statement body may need to be repeatedly executed
66/216
Memory Models
Soloway, 1983An IF should be used when a statement body is guaranteed to be executed only once and a WHILE used when a statement body may need to be repeatedly executed
67/216
Memory Models
Soloway, 1983An IF should be used when a statement body is guaranteed to be executed only once and a WHILE used when a statement body may need to be repeatedly executed
68/216
Memory Models
Soloway, 1983An IF should be used when a statement body is guaranteed to be executed only once and a WHILE used when a statement body may need to be repeatedly executed
69/216
Memory Models
Pennington, 1987– Multiple abstractions for computer programs
(COBOL but may be true for other languages)• Abstraction of function• Abstraction of data-flow• Abstraction of control-flow• Abstraction of conditionalised actions
70/216
Memory Models
Pennington, 1987
71/216
Abstraction of function
72/216
Abstraction of data-flow
73/216
Abstraction of control-flow
74/216
Abstraction of conditionalised actions
75/216
76/216
Memory Models
Pennington, 1987– Programming knowledge
• Text structure knowledge– Sequences– Iterations– Conditionals
• Plan knowledgePlans are intermediate-level programming concepts
– Summing– Hashing– Counting– …
77/216
Memory Models
Pennington, 1987– Which knowledge and what abstraction?
78/216
Memory Models
Pennington, 1987– Which knowledge and what abstraction?
Text structure
79/216
Abstraction of control-flow
80/216
Memory Models
Van Mayrhauser, 1995– From previous works by
• Pennington• Soloway et al.• Rist• Van Mayhauser and Vans• Letovsky• Brooks• Schneiderman and Mayer
81/216
Memory Models
Van Mayrhauser, 1995
82/216
Memory Models
Van Mayrhauser, 1995
Integrated model
83/216
84/216
85/216
86/216
87/216
Outline
Visual systemMemory modelsMental models Schema theory Software patterns Empirical studies Conclusion
88/216
Mental Models
“The image of the world around us, which we carry in our head, is just a model. Nobody in his head imagines all the world, government or country. He has only selected concepts, and relationships between them, and uses those to represent the real system.”
—Jay Wright Forrester, 1971
89/216
Mental Models
90/216
Mental Models
Mental models
91/216
Mental Models
Mental models Formal rules of inference
92/216
Mental Models
Mental models Formal rules of inference Domain-specific rules of inference
93/216
Mental Models
Mental models Formal rules of inference Domain-specific rules of inference Probabilities
94/216
Mental Models
Mental models Formal rules of inference Domain-specific rules of inference Probabilities…
95/216
Mental Models
96/216
Memory Models
97/216
Mental Models
98/216
Mental Models
Context matters!
99/216
Mental Models
Context matters!
100/216
Mental Models
Context matters!
101/216
Mental Models
Context matters!
102/216
Mental Models
Context matters!
103/216
Mental Models
Minsky, 1974– Frames
• “data-structure for representing a stereotyped situation”• “network of nodes and relations” and “slots”
– Frame-systems• Important actions are transformations between the
frames of a system • Different frames of a system share the same terminals
(and slots)– Matching process
104/216
Mental Models
Minsky, 1974– Vision– Speech– Knowledge– Control
105/216
Mental Models
Minsky, 1974– Vision– Speech– Knowledge– Control
106/216
Mental Models
Minsky, 1974– Vision
• A frame, once evoked on the basis of partial evidence or expectation, would first direct a test to confirm its own appropriateness
• It would request information needed to assign values to those terminals that cannot retain their default assignments
• If informed about a transformation (e.g., an impending motion), it would transfer control to the appropriate other frame of that system
107/216
Mental Models
Minsky, 1974– Vision– Speech– Knowledge– Control
108/216
Mental Models
Minsky, 1974– Knowledge
• Similarity networks
109/216
Mental Models
Minsky, 1974– Knowledge
• Similarity networks
110/216
Mental Models
Minsky, 1974– Vision– Speech– Knowledge– Control
111/216
Mental Models
Minsky, 1974– Control
• TOP-DOWN or LATERAL: Should one make a pass over all the terminals first or should one attempt a complete, detailed instantiation of some supposedly most critical one?
• CENTRAL CONTROL: Should a frame, once activated, "take over" and control its instantiation, or should a central process organize the operation
112/216
Mental Models
Soloway, 1986– In the context of analyzing problems and
constructing programs– Goals
• Requirements• Bugs
– Plans• “Canned” solutions• With “meaning”
113/216
Mental Models
Soloway, 1986“Master and novice chess players were shown a meaningful chess board, and asked to recall the pieces […]. The masters recalled more of the pieces than did the novices. Next, the masters and novices were shown a board in which the chess pieces were placed randomly on the board. […] The performance of the masters was the same as that of the novices.”
—Chase and Simon, 1973
114/216
Mental Models
Soloway, 1986– Micro-strategies
• Ask questions• Seek answers• RepeatDifficult with delocalised plans
– Macro-strategies• Systematic (scalability?)• As-needed (correctness?)
115/216
Mental Models
Rich and Waters, 1990(IBM’s Harlan Mills idea of “chief programmer teams”)
“[T]o provide software engineers with intelligent assistance.” (p. 2)
“The intended interaction between a software engineer […] is modeled after interaction with a human assistant.” (p. 2)
116/216
Mental Models
Rich and Waters, 1990
“Between the engineer and the Apprentice […], the key to effective cooperation is shared knowledge.” (p. 4)
– Shared knowledge• About the system being worked on• About concepts of software engineering
Clichés
117/216
Mental Models
Rich and Waters, 1990“[An algorithmic] cliché contains both fixed parts and parts that vary from one occurrence of the cliché to the next […], [it] may also include constraints […].” (p. 12)
“Non-algorithmic clichés are represented […] using standard frame-based knowledge representation techniques […].” (p. 21)
118/216
Mental Models
Rich and Waters, 1990– Key properties of algorithmic clichés
• Canonical form• Language independence• Convenient manipulation• Expressiveness
– A representation of clichés The Plan Calculus
119/216
Mental Models
Rich and Waters, 1990“The Plan Calculus is intended to be a blueprint language for software.” (p. 29)
• Formal representation for– Programs– Algorithmic clichés
Subroutines Program generators
MacrosFlowcharts Program schemas
Flowchart schemas Logic
Program transformation Data abstractionFormal grammars
Plan calculus
120/216
Mental Models
Rich and Waters, 1990
if:negative
T F
then:negate
end:join
T F
121/216
Mental Models
Rich and Waters, 1990
if:negative
T F
then:negate
end:join
T F
absolute-value
122/216
Mental Models
Rich and Waters, 1990
next:apply-function
next:
input:
input:continue:
generation
generation
generate
output:*
123/216
(On Gaming…)
Visual system Memory models Mental models
http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/
124/216
(On Gaming…)
Visual system Memory models Mental models
http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/
125/216
(On Gaming…)
Visual system Memory models Mental models
http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/
126/216
(On Gaming…)
Visual system Memory models Mental models
“[G]amers [were] better at picking out details in a cluttered scene”
http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/
127/216
(On Gaming…)
Visual system Memory models Mental models
“[G]amers [were] better at picking out details in a cluttered scene”“[G]amers were able to produce accurate estimates [of relative counts]”
http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/
128/216
(On Gaming…)
Visual system Memory models Mental models
“[G]amers [were] better at picking out details in a cluttered scene”“[G]amers were able to produce accurate estimates [of relative counts]”“Gamers found it easier to perform mental rotations of geometric shapes”
http://arstechnica.com/science/2014/02/how-action-games-can-improve-our-visual-skills/
129/216
Outline
Visual systemMemory modelsMental models Schema theory Software patterns Empirical studies Conclusion
130/216
Schema Theory
“The captain asked the passengers to fasten the seat belts. They were ready to take off.”
—Kohls and Scheiter, 2008
131/216
Schema Theory
132/216
Schema Theory
“This comprehensive situation is an interrelation of events and entities, and is stored in an internal data structure that can be activated by recognizing its typical features. Such data structures, or schemas, are mental representations in an individual’s mind.”
—Kohls and Scheiter, 2008
133/216
Schema Theory
“This comprehensive situation is an interrelation of events and entities, and is stored in an internal data structure that can be activated by recognizing its typical features. Such data structures, or schemas, are mental representations in an individual’s mind.”
—Kohls and Scheiter, 2008
134/216
Schema Theory
“Human brains operate fundamentally in terms of pattern recognition rather than of logic. They are highly constructive in settling on given schema and at the same time are constantly open to error.”
—Edelman, 2006(With minor adaptation)
135/216
Schema Theory
Piaget, 1969– Child development theory– Schema and associated processes
136/216
Schema Theory
Piaget, 1969– Child development theory– Schema and associated processes
137/216
Schema Theory
Piaget, 1969– Schema and associated processes
• From Plato and Aristotle: essential commonalities, basic properties that are distinctive of an object
To Kant: link from stimuli to “pure” concepts– Perceived chair vs. “ideal” chair– No a-priori knowledge
• Gick and Holyoak: learning by examples and problem solving (like in mathematics)
138/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Abstract representation of multiple instances of the same kinds of
– Concept– Situation– Plan– Behaviour– …
to recognise similar / discriminate dissimilar experiences, access common concepts, draw inferences, create goals, develop plans, use skills
139/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Variables (or slots or attributes)
• (Constrained) Ranges of values– People’s experience– Constants – Optional or mandatory
• Constraints among variables and their values
140/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Variables (or slots or attributes)
• (Constrained) Ranges of values– People’s experience– Constants – Optional or mandatory
• Constraints among variables and their values
141/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Variables (or slots or attributes)
• (Constrained) Ranges of values– People’s experience– Constants – Optional or mandatory
• Constraints among variables and their values
142/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Variables (or slots or attributes)
• (Constrained) Ranges of values– People’s experience– Constants – Optional or mandatory
• Constraints among variables and their values
143/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Prototypical object
• Object space and (mental) object generator
144/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Prototypical object
• Object space and (mental) object generator
145/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Prototypical object
• Object space and (mental) object generator
146/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Prototypical object
• Object space and (mental) object generator
147/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Prototypical object
• Object space and (mental) object generator
148/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Part-of
• Coupling
• Is-a
149/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Part-of
• Coupling
• Is-a
150/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Part-of
• Coupling
• Is-a
151/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Part-of
• Coupling
• Is-a
152/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Hierarchy– Contexts– Forces– Probabilities
153/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Hierarchy– Contexts– Forces– Probabilities
154/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Hierarchy– Contexts– Forces– Probabilities
155/216
Schema Theory
Piaget, 1969– Schema and associated processes
• Use
156/216
157/216
(Schema Theory Programming)
Papert worked with Piaget from 1958 to 1963 and afterwards– Constructionism
• Learning as a reconstruction rather than as a transmission of knowledge
• Learning is most effective when part of learner’s activity to build a meaningful product
http://www.lispcast.com/programming-language-stages-of-development
158/216
(Schema Theory Programming)
Papert developed the Logo programming language within the constructionism theory circa. 1967
159/216
(Schema Theory Programming)
Explanations of Logo programming using a simplified cognitive development modelCorporal Visual Symbolic
– The child learns to see her own perspective from the outside in the turtle
– The child sees the results of her commands immediately (cf. chick sexing)
– The child could think of the result of her commands before their execution
160/216
(Schema Theory Programming)
Explanations of object-oriented program-ming using the same simplified modelCorporal Visual Symbolic
– The ability to program from the perspective of a single object at a time
– The ability to think in terms of visual, archetypal interactions (client-server, MVC, etc.)
– The ability to build abstractions with simple yet powerful rules and meta-programming
161/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Comprehension: pattern matching of memorised schema that cooperates and competes
162/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Comprehension: pattern matching of memorised schema that cooperates and competes
163/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Comprehension: pattern matching of memorised schema that cooperates and competes
• Animal face• human face• Bob’s face
164/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Comprehension: pattern matching of memorised schema that cooperates and competes
• Animal face• human face• Bob’s face
165/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Comprehension: pattern matching of memorised schema that cooperates and competes
• Animal face• human face• Bob’s face
166/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Comprehension: pattern matching of memorised schema that cooperates and competes
• Animal face• human face• Bob’s face
167/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Comprehension: pattern matching of memorised schema that cooperates and competes
• Animal face• human face• Bob’s face
168/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Comprehension: pattern matching of memorised schema that cooperates and competes
• Animal face• human face• Bob’s face
169/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Accretion: specific situation or experience stored in schemas used to reconstruct original experience
– Episodic memory– Specific details
170/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Accretion: specific situation or experience stored in schemas used to reconstruct original experience
– Episodic memory– Specific details
곰세마리가한집에있어아빠곰엄마곰애기곰아빠곰은뚱뚱해엄마곰은날씬해애기곰은너무귀여워
171/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Accretion: specific situation or experience stored in schemas used to reconstruct original experience
– Episodic memory– Specific details
Why/How is specific information forgotten/replaced by generalised knowledge?
곰세마리가한집에있어아빠곰엄마곰애기곰아빠곰은뚱뚱해엄마곰은날씬해애기곰은너무귀여워
172/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Tuning: modifications to existing schemas– Variables– Ranges– Probabilities
173/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Tuning: modifications to existing schemas– Variables– Ranges– Probabilities
174/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Restructuring: ex nihilo / copy of existing schemas– Schema generation by analogies with existing schema, is-a– Schema induction from existing schema, has-a
175/216
Schema Theory
Rumelhart and Norman, 1978– Models of learning
• Restructuring: ex nihilo / copy of existing schemas– Schema generation by analogies with existing schema, is-a– Schema induction from existing schema, has-a
176/216
Outline
Visual systemMemory modelsMental models Schema theory Software patterns Empirical studies Conclusion
177/216
Software Patterns
Schemas– Variables
– Ranges of values• Forces
– Constraints among variables and their values
178/216
Software Patterns
Schemas– Variables
– Ranges of values• Forces
– Constraints among variables and their values
What class plays this role?
179/216
Software Patterns
Schemas– Variables
– Ranges of values• Forces
– Constraints among variables and their values
Possible classes playing this roleplus their relations, implementation…
180/216
Software Patterns
Schemas– Variables
– Ranges of values• Forces
– Constraints among variables and their values
Expected relations, interactions…
181/216
Software Patterns
Schema theory
“The pattern is an attempt to discover some invariant features, which distinguishes good places from bad places with respect to some particular system of forces.”
—Christopher Alexander, 1979
182/216
Software Patterns
Knowledge and–or procedure
“The solution part of a good pattern describes both a process and a thing: the ‘thing’ is created by the ‘process’.”
—Christopher Alexander, 1979
“Furthermore, a pattern tells about a form not only what it is but also what it does.”
—Christopher Alexander, 1964
183/216
Software Patterns
Space and generation
“A pattern describes a coherent yet infinite design space, not a finite set of implementations in that space.”
—Frank Buschmann, Kevlin Henney, and Douglas C. Schmidt, 2007
184/216
Software Patterns
Use design patterns– Composite– Decorator– Observer– …
to generate the architecture of the program
185/216
Software Patterns
Wholeness– Quality Without a Name
• Forces
“Having an ideal concept of a thing lets one judge the beauty of it”
—Kohls and Scheiter, 2008
186/216
Software Patterns
187/216
Software Patterns
“Important assumptions– That patterns can be codified in such a way that
they can be shared between different designers– That reuse will lead to “better” designs. There is
an obvious question here of what constitutes “better”, but a key measure is maintainability”
—Zhang and Budgen, 2012 (With minor adaptations)
188/216
Outline
Visual systemMemory modelsMental models Schema theory Software patterns Empirical studies Conclusion
189/216
Empirical Studies
“Eye movements are uniquely poised between perception and cognition. They are central to the function of the visual system, but for such scanning to be efficient, it cannot be simply a random sample of the visual world. To be useful, eye movements must be related to an organism’s memories, expectations and goals. Consequently, eye movements are driven equally by bottom-up perceptual properties of the world and top-down cognitive processes.”
—Richardson, Dale, and Spivey, 2013
190/216
Empirical Studies
“Eye movements are uniquely poised between perception and cognition. They are central to the function of the visual system, but for such scanning to be efficient, it cannot be simply a random sample of the visual world. To be useful, eye movements must be related to an organism’s memories, expectations and goals. Consequently, eye movements are driven equally by bottom-up perceptual properties of the world and top-down cognitive processes.”
—Richardson, Dale, and Spivey, 2013
191/216
Empirical Studies
“Eye movements are uniquely poised between perception and cognition. They are central to the function of the visual system, but for such scanning to be efficient, it cannot be simply a randomsample of the visual world. To be useful, eye movements must be related to an organism’s memories, expectations and goals. Consequently, eye movements are driven equally by bottom-up perceptual properties of the world and top-down cognitive processes.”
—Richardson, Dale, and Spivey, 2013
192/216
Empirical Studies
“Eye movements are uniquely poised between perception and cognition. They are central to the function of the visual system, but for such scanning to be efficient, it cannot be simply a randomsample of the visual world. To be useful, eye movements must be related to an organism’s memories, expectations and goals. Consequent, eye movements are driven equally by bottom-up perceptual properties of the world and top-down cognitive processes.”
—Richardson, Dale, and Spivey, 2013
193/216
Empirical Studies
“Eye movements are uniquely poised between perception and cognition. They are central to the function of the visual system, but for such scanning to be efficient, it cannot be simply a randomsample of the visual world. To be useful, eye movements must be related to an organism’s memories, expectations and goals. Consequent, eye movements are driven equally by bottom-upperceptual properties of the world and top-downcognitive processes.”
—Richardson, Dale, and Spivey, 2013
194/216
195/216http://web-tan.forum.impressrd.jp/e/2009/04/02/5280
196/216http://another-voice.jp/jp/katsu/professional_view/01/index.shtml
197/216http://another-voice.jp/jp/katsu/professional_view/01/index.shtml
198/216
Empirical Studies
Data– Fixations– Saccades– Scan path
Analyses– Centres of interests– Effort– Duration– …
199/216
Empirical Studies
200/216
201/216
202/216
Empirical Studies
SR Research Eye-link II– www.sr-research.com– Intrusive– Two infra-red cameras and one sensor– Error rate < 0.5º– Sampling rate of 500Hz– Access eye position < 3.0 ms.– 0.01º RMS resolution (tilt)
EyeLink Data Viewer
203/216
204/216
205/216
Empirical Studies
Tobii 1750 eye-tracker – www.tobii.se– Non-intrusive (no head mount)– Two cameras built into a 17 in flat screen– Records audio and video of a session– Error rate < 0.5º– Sampling rate of 50Hz
ClearView software
206/216
207/216
208/216
Empirical Studies
Seeing Machines FaceLab 5– www.seeingmachines.com– Non-intrusive– Two infra-red cameras– Error rate < 1º– Sampling rate of 60Hz– Complete head-tracking ±90º (y), ±45º (x) – Complete eye-tracking ±45º (y), ±22º (x)
GazeTracker
209/216
Outline
Visual systemMemory modelsMental models Schema theory Software patterns Empirical studies Conclusion
210/216
Conclusion
“That practices can be codified in such a way that they can be shared between different designers”
—Zhang and Budgen, 2012
What format? How memorised? How acquired?
211/216
Conclusion
“That practices can be codified in such a way that they can be shared between different designers”
—Zhang and Budgen, 2012
What format? Schema theory How memorised? How acquired?
212/216
Conclusion
“That practices can be codified in such a way that they can be shared between different designers”
—Zhang and Budgen, 2012
What format? Schema theory How memorised? Long-term memory How acquired?
213/216
Conclusion
“That practices can be codified in such a way that they can be shared between different designers”
—Zhang and Budgen, 2012
What format? Schema theory How memorised? Long-term memory How acquired? Gestalt theory
214/216
Conclusion
Software patterns– Schema theory– Knowledge and–or procedure– Space and generation– Wholeness
215/216
Conclusion
Empirical studies– Patterns use and usefulness
• Faults• Changes• Comprehension• …
– Many opportunities to understand program comprehension better!
216/216
Conclusion
Empirical studies– Patterns use and usefulness
• Faults• Changes• Comprehension• …
– Many opportunities to understand program comprehension better!