Design and Implementation of Interactive Data Visualization Method in Practical Usesインタラクティブな情報可視化⼿法の実利⽤に向けた設計と実装

M. C. 2nd yearHe ChenKitamura lab

1

Background

Digital Content in Practical Uses

Lack of ❖a comprehensive

visualization for diverse content

❖flexibility in interactive arrangement

❖ context adaptive visualization

2

Background

Demand

❖An interactive content visualization framework1. diverse content2. flexible interaction3. multiple interactive scenarios

3

Related Work(1/2)❖ Interactive By-example Design of Artistic Packing

Layouts [SIGGRAPH Asia 2013]

4

Related Work(1/2)❖ Benefit

❖ Infer packing layouts by examples❖ Limitation in flexibility

❖ High dimensional metadata

5

Related Work(2/2)❖ The Bohemian Bookshelf [CHI 2012]

6

❖ Serendipitous discovery❖ Flexible visual pathways❖ Highlighting adjacencies❖ A playful exploration

❖ Limitation in flexibility❖ Transition between open-ended and targeted

browsing

7

Related Work(2/2)

Possible Foundation D-Flip: Dynamic & Flexible Interactive PhotoShow

❖ Dynamically show overview & detail of photos without overlaps

❖ Flexible in arranging photos based on its metadata

❖ Playful experience

8

Chi Thanh Vi, Kazuki Takashima, Hitomi Yokoyama, Gengdai Liu, Yuichi Itoh, Sriram Subramanian, Yoshifumi Kitamura, A dynamic & flexible interactive display method of digital photographs, Entertainm. Comput. (2014)

Overview of This Research

9

2. flexible interaction

3. multiple interactive scenarios

1. diverse content

An Interactive Content Visualization Framework

D-Flip -- Foundation�

Original

+Photo ❖ Shooting time & location

❖ Tagged RIO ❖ Color

Extension Examples

+

+

+

Music❖ Composer ❖ Lyricist ❖ Singer ❖ Genre

Movie❖ Title ❖ Producer ❖ Director ❖ Actor

10

❖ A comprehensive framework❖ High dimensional metadata

Extension

Purpose: 1. Diverse Content

11

❖ Allow & entice user exploration

Extension

Purpose: 2. Flexible Interaction

12

❖ A wide range of usage contexts

Extension

Purpose: 3. Multiple Interactive Scenarios

Original Scenario ❖ Desktop display ❖ Mouse interaction

Extension Examples ❖ Multi-touch display ❖ Whole body interaction

Design Concepts of Extension

1. Diverse Content2. Flexible Interaction

❖ wall label❖ clustering❖ interactive venn diagram❖ dynamic sorting

3. Multiple Interactive Scenarios❖ multi-touch interaction❖ whole body interaction

13

Design Concepts of Extension

1. Diverse Content2. Flexible Interaction

❖ wall label❖ clustering❖ interactive venn diagram❖ dynamic sorting

3. Multiple Interactive Scenarios❖ multi-touch interaction❖ whole body interaction

14

Diverse Content

Metadata Management❖ Relational Database❖ XML❖ Flat File

15

Diverse Content

Metadata Management❖ Relational Database❖ XML❖ Flat File

16

a MySQL example

Diverse Content

Metadata Management❖ Relational Database❖ XML❖ Flat File

17

Diverse Content

Metadata Management❖ Relational Database❖ XML❖ Flat File

18

An example of configuration:filePath | fileName | shooting time | creation time | tags | color

Diverse Content

Metadata Management❖ Easy to maintain

❖ Relational Database❖ Easy to extend

❖ XML❖ Easy to implement

❖ Flat File

19

Design Concepts of Extension

1. Diverse Content2. Flexible Interaction

❖ wall label❖ clustering❖ interactive venn diagram❖ dynamic sorting

3. Multiple Interactive Scenarios❖ multi-touch interaction❖ whole body interaction

20

2. Flexible Interaction(1/4)

Wall Label❖ Spacial proximity

21

Content Metadata

Artwork

TitleArtistMuseumCountry

Flash Card

EnglishJapaneseChineseKorean

22

❖ Relevancy: 0 / 1

Selected item (tagA, tagB)item(tagA)

item(tagB)item(tagA, tagB)

item(tagC)

2. Flexible Interaction(2/4)

Clustering

23

❖ Relevancy: 0 ~ 1

Selected itemitem(0.9)

item(0.7)

item(0.8)item(0.5)

2. Flexible Interaction(2/4)

Clustering

24

❖ a natural way to visualize intersections❖ targeted browsing

Metadata A Metadata B

item (A, B)

item(B)item

(A)

2. Flexible Interaction(3/4)

Interactive Venn Diagram

25

❖ Value❖ Time❖ Season❖ Distance❖ …

2. Flexible Interaction(4/4)

Dynamic Sorting

Design Concepts of Extension

1. Diverse Content2. Flexible Interaction

❖ wall label❖ clustering❖ interactive venn diagram❖ dynamic sorting

3. Multiple Interactive Scenarios❖ multi-touch interaction❖ whole body interaction

26

3. Multiple Interactive Scenarios

Multi-Touch Interaction

27

❖ Individual / Collaborative work❖ Intuitive, fast, enjoyable to interact with

3. Multiple Interactive Scenarios

Whole Body Interaction❖ Kinect + Wall display❖ Shadow metaphor connecting private & shared space❖ Promising for public settings

28

Applications

❖ Flexible Interaction❖ Paintings for art appreciation

❖ Multiple interactive scenarios❖ Public walk-up-and-use display

29

Application (1/2)

Paintings for Art Appreciation

Motivation:❖ Visualize a collection of paintings in a

playful way❖ Entice engaging interaction for art

appreciation

30

Application (1/2)

Paintings for Art Appreciation

31

Application (2/2)

Public Walk-up-and-use Display

Motivation:❖ Evaluate this visualization in public

settings❖ Observe people’s behavior towards this

interactive public display system

32

Application (2/2)

Public Walk-up-and-use Display

33

RIEC Introduction Content

34

❖Experiment Purpose❖Experiment Settings❖Results

❖ Behavioral Counting Data❖ Questionnaire ❖ Observed Gesture

❖Experiment Conclusion

Application (2/2)

Public Walk-up-and-use Display

35

❖ Observe how people ❖ notice❖ approach ❖ interact

with our multi-touch public display system❖ Explore design factors for such system

Application (2/2)

Experiment Purpose

Application (2/2)

Experiment Settings❖ Time: During RIEC Open Day, 10. 4 - 10. 5, 2014. ❖ Place: RIEC building #1, 1F lobby❖ RIEC Introduction Content

❖ 60 photos and text images ❖ 16 short video clips

36

• 4K 50inch TV display• Max 10-point multi-touch screen

Application (2/2)

Results: Behavioral Counting Data

37

interacted 115stopped 176looked 414

passed by 1115official data: 585 visitors in two days

0%

25%

50%

75%

100%

Passed-By Looked Stopped Interacted

10.3%15.7%

37.1%

100%

user experience

38

❖ 49 Questionnaires: 33 male, 16 female❖ positive feedback

Application (2/2)

Results: Questionnaire

39

❖ want to use this in public places❖ restaurant, museum, theater, etc

❖ need enlarge the image❖ want to know detail of the content (link,

webpage)

Application (2/2)

Results: Questionnaire — Comments

40

Tap

Drag

Enlarge

Flick

Continuous touch. Probable explanation: system reaction is slower than expected

Long time press. Probable explanation: touch takes effects after finger leaves

Application (2/2)

Results: Observed Gesture

Application (2/2)

Experiment Conclusion

❖ Promising as interactive public display❖ Touch experience can be further improved,

e.g., enlarge, flick❖ Item’s detailed information can be

provided, e.g., link

41

Summary❖ extended the original framework in three aspects

❖ diverse content❖ flexible interaction to entice user’s exploration❖ multiple interactive scenarios: individual/

collaborative work❖ applied design concepts

❖ paintings ❖ public walk-up-and-use display

42

Future Work1. Diverse Content

❖ Metadata source, e.g., community website, image recognition

2. Flexible Interaction❖ Immersive

e.g., Visualization in 3D3. Multiple Interactive Scenarios

❖ Gaze interface, tangible interaction, etc43

Thank you for your attention !

Q & A

44