Oh, no! validation bingo!. algorithm complexity analysis.
-
Upload
amberlynn-tate -
Category
Documents
-
view
216 -
download
0
Transcript of Oh, no! validation bingo!. algorithm complexity analysis.
![Page 1: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/1.jpg)
oh, no! validation bingo!
![Page 2: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/2.jpg)
oh, no! validation bingo!
• algorithm complexity analysis
![Page 3: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/3.jpg)
oh, no! validation bingo!
• algorithm complexity analysis• field study with target user population
![Page 4: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/4.jpg)
oh, no! validation bingo!
• algorithm complexity analysis• field study with target user population• implementation performance (speed, memory)
![Page 5: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/5.jpg)
oh, no! validation bingo!
• algorithm complexity analysis• field study with target user population• implementation performance (speed, memory)• informal usability study
![Page 6: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/6.jpg)
oh, no! validation bingo!
• algorithm complexity analysis• field study with target user population• implementation performance (speed, memory)• informal usability study• laboratory user study
![Page 7: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/7.jpg)
oh, no! validation bingo!
• algorithm complexity analysis• field study with target user population• implementation performance (speed, memory)• informal usability study• laboratory user study• qualitative discussion of result pictures
![Page 8: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/8.jpg)
oh, no! validation bingo!
• algorithm complexity analysis• field study with target user population• implementation performance (speed, memory)• informal usability study• laboratory user study• qualitative discussion of result pictures• quantitative metrics
![Page 9: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/9.jpg)
oh, no! validation bingo!
• algorithm complexity analysis• field study with target user population• implementation performance (speed, memory)• informal usability study• laboratory user study• qualitative discussion of result pictures• quantitative metrics• requirements justification from task analysis
![Page 10: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/10.jpg)
oh, no! validation bingo!
• algorithm complexity analysis• field study with target user population• implementation performance (speed, memory)• informal usability study• laboratory user study• qualitative discussion of result pictures• quantitative metrics• requirements justification from task analysis• user anecdotes (insights found)
![Page 11: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/11.jpg)
oh, no! validation bingo!
• algorithm complexity analysis• field study with target user population• implementation performance (speed, memory)• informal usability study• laboratory user study• qualitative discussion of result pictures• quantitative metrics• requirements justification from task analysis• user anecdotes (insights found)• user community size (adoption)
![Page 12: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/12.jpg)
oh, no! validation bingo!
• algorithm complexity analysis• field study with target user population• implementation performance (speed, memory)• informal usability study• laboratory user study• qualitative discussion of result pictures• quantitative metrics• requirements justification from task analysis• user anecdotes (insights found)• user community size (adoption)• visual encoding justification from theoretical principles
![Page 13: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/13.jpg)
split threats to validity into four levels
![Page 14: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/14.jpg)
split threats to validity into four levels
domain problem characterization
• wrong problem• they don’t do that
![Page 15: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/15.jpg)
split threats to validity into four levels
domain problem characterization
data/operation abstraction design
• wrong problem• they don’t do that
• wrong abstraction• you’re showing them the wrong thing
![Page 16: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/16.jpg)
split threats to validity into four levels
domain problem characterization
data/operation abstraction design
encoding/interaction technique design
• wrong problem• they don’t do that
• wrong abstraction• you’re showing them the wrong thing
• wrong encoding/interaction technique• the way you show it doesn’t work
![Page 17: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/17.jpg)
split threats to validity into four levels
domain problem characterization
data/operation abstraction design
encoding/interaction technique design
algorithm design
• wrong problem• they don’t do that
• wrong abstraction• you’re showing them the wrong thing
• wrong encoding/interaction technique• the way you show it doesn’t work
• wrong algorithm• your code is too slow
![Page 18: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/18.jpg)
validate according to threatthreat: wrong problem threat: bad data/operation abstraction threat: ineffective encoding/interaction technique threat: slow algorithm
implement system
![Page 19: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/19.jpg)
validate according to threatthreat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique threat: slow algorithm
implement system validate: observe adoption rates
![Page 20: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/20.jpg)
validate according to threatthreat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique threat: slow algorithm
implement system validate: test on target users, collect anecdotal evidence of utility validate: field study, document human usage of deployed system validate: observe adoption rates
![Page 21: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/21.jpg)
validate according to threatthreat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique validate: justify encoding/interaction design threat: slow algorithm
implement system validate: qualitative/quantitative result image analysis [test on any users, informal usability study] validate: lab study, measure human time/errors for operation validate: test on target users, collect anecdotal evidence of utility validate: field study, document human usage of deployed system validate: observe adoption rates
![Page 22: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/22.jpg)
validate according to threatthreat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique validate: justify encoding/interaction design threat: slow algorithm
validate: analyze computational complexity implement system validate: measure system time/memory validate: qualitative/quantitative result image analysis [test on any users, informal usability study] validate: lab study, measure human time/errors for operation validate: test on target users, collect anecdotal evidence of utility validate: field study, document human usage of deployed system validate: observe adoption rates
![Page 23: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/23.jpg)
threat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique validate: justify encoding/interaction design threat: slow algorithm
validate: analyze computational complexity implement system validate: measure system time/memory validate: qualitative/quantitative result image analysis [test on any users, informal usability study] validate: lab study, measure human time/errors for operation validate: test on target users, collect anecdotal evidence of utility validate: field study, document human usage of deployed system validate: observe adoption rates
don’t do them all in any single paper
![Page 24: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/24.jpg)
threat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique validate: justify encoding/interaction design threat: slow algorithm
validate: analyze computational complexity implement system validate: measure system time/memory validate: qualitative/quantitative result image analysis [test on any users, informal usability study] validate: lab study, measure human time/errors for operation validate: test on target users, collect anecdotal evidence of utility validate: field study, document human usage of deployed system validate: observe adoption rates
match up with level of contributions!
![Page 25: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/25.jpg)
threat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique validate: justify encoding/interaction design threat: slow algorithm
validate: analyze computational complexity implement system validate: measure system time/memory validate: qualitative/quantitative result image analysis [test on any users, informal usability study] validate: lab study, measure human time/errors for operation validate: test on target users, collect anecdotal evidence of utility validate: field study, document human usage of deployed system validate: observe adoption rates
match up with level of contributions!
![Page 26: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/26.jpg)
threat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique validate: justify encoding/interaction design threat: slow algorithm
validate: analyze computational complexity implement system validate: measure system time/memory validate: qualitative/quantitative result image analysis [test on any users, informal usability study] validate: lab study, measure human time/errors for operation validate: test on target users, collect anecdotal evidence of utility validate: field study, document human usage of deployed system validate: observe adoption rates
match up with level of contributions!
![Page 27: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/27.jpg)
threat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique validate: justify encoding/interaction design threat: slow algorithm
validate: analyze computational complexity implement system validate: measure system time/memory validate: qualitative/quantitative result image analysis [test on any users, informal usability study] validate: lab study, measure human time/errors for operation validate: test on target users, collect anecdotal evidence of utility validate: field study, document human usage of deployed system validate: observe adoption rates
match up with level of contributions!
![Page 28: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/28.jpg)
threat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique validate: justify encoding/interaction design threat: slow algorithm
validate: analyze computational complexity implement system validate: measure system time/memory validate: qualitative/quantitative result image analysis [test on any users, informal usability study] validate: lab study, measure human time/errors for operation validate: test on target users, collect anecdotal evidence of utility validate: field study, document human usage of deployed system validate: observe adoption rates
avoid deadly mismatches
![Page 29: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/29.jpg)
threat: wrong problem validate: observe and interview target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique validate: justify encoding/interaction design threat: slow algorithm
validate: analyze computational complexity implement system validate: measure system time/memory validate: qualitative/quantitative result image analysis [test on any users, informal usability study] validate: lab study, measure human time/errors for operation validate: test on target users, collect anecdotal evidence of utility validate: field study, document human usage of deployed system validate: observe adoption rates
avoid deadly mismatches
![Page 30: Oh, no! validation bingo!. algorithm complexity analysis.](https://reader035.fdocuments.net/reader035/viewer/2022070401/56649f165503460f94c2c560/html5/thumbnails/30.jpg)
examples! nuances! 4:15 tomorrow
genealogical graphs
justify encoding/interaction design
qualitative result image analysistest on target users, collect anecdotal evidence of utility
MatrixExplorerjustify encoding/interaction design measure system time/memory
qualitative result image analysis
observe and interview target users
LiveRACjustify encoding/interaction design
field study, document usage of deployed system
qualitative result image analysis
observe and interview target users
LinLog qualitative/quantitative result image analysis