HAL Id hal-01138024httpshalinriafrhal-01138024

Submitted on 2 Apr 2015

HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents whether they are pub-lished or not The documents may come fromteaching and research institutions in France orabroad or from public or private research centers

Lrsquoarchive ouverte pluridisciplinaire HAL estdestineacutee au deacutepocirct et agrave la diffusion de documentsscientifiques de niveau recherche publieacutes ou noneacutemanant des eacutetablissements drsquoenseignement et derecherche franccedilais ou eacutetrangers des laboratoirespublics ou priveacutes

Design Considerations for Composite PhysicalVisualizations

Mathieu Le Goc Pierre Dragicevic Samuel Huron Jean-Daniel Fekete

To cite this versionMathieu Le Goc Pierre Dragicevic Samuel Huron Jean-Daniel Fekete Design Considerations forComposite Physical Visualizations Proceedings of the CHI Workshop on Exploring the Challenges ofMaking Data Physical Apr 2015 Seoul South Korea hal-01138024

Design Considerations forComposite Physical Visualizations

Mathieu Le GocINRIAmathieule-gocinriafr

Pierre DragicevicINRIApierredragicevicinriafr

Samuel HuronUniversity of Calgary amp IRIsamuelhuroncybunkcom

Jean-Daniel FeketeINRIAjean-danielfeketeinriafr

Paste the appropriate copyright statement here ACM now supports threedifferent copyright statementsbull ACM copyright ACM holds the copyright on the work This is the historicalapproachbull License The author(s) retain copyright but ACM receives an exclusivepublication licensebull Open Access The author(s) wish to pay for the work to be open accessThe additional fee must be paid to ACMThis text field is large enough to hold the appropriate release statementassuming it is single spacedEvery submission will be assigned their own unique DOI string to be includedhere

AbstractWhile physical visualizations have existed for many yearsmost of them remain monolithic and static We identify apromising category of physical visualizations we callcomposite physical visualizations Composite physicalvisualizations are combinations of multiple physicalobjects and can be designed to better leverage bothhuman and technological capabilities We show that twoimportant properties have to be considered whendesigning such visualizations their level of actuation andtheir manipulability Through examples we illustrate thetradeoffs between these two dimensions and identify theneed for more research in this particular area

Author KeywordsPhysical Visualizations

ACM Classification KeywordsH5m [Information interfaces and presentation (egHCI)] Miscellaneous

IntroductionPhysical visualization has existed for thousands of yearsyet the Information Visualization community is juststarting to study it [9] Many current physicalvisualizations (eg [3]) are monolithic static and notinteractive Some of them are made of multiple individual

objects that can be rearranged in order to represent avariety of informative configurations We call themcomposite physical visualizations A major benefit of suchvisualizations is that they support modularity andupdatability but their design space is not well understood

Figure 1 Visualization usingcolored wooden tiles

Figure 2 Activity loggingvisualization built by MichaelHunger out of LEGO bricks

In this paper we show that composite physicalvisualizations can be classified according to twoorthogonal dimensions i) their level of actuation and ii)their manipulability Among existing systems some havea high manipulability but no support for actuation(eg [8]) while others are fully actuated but notmanipulable (eg [1]) Only a few systems are combiningboth qualities and none supports both full manipulabilityand full actuation We discuss the tradeoffs between thesetwo dimensions and identify the opportunities andchallenges for future research and design

Manually arranged visualizationsAn easy way to build a composite physical visualization isto arrange multiple objects manually in order to createvisual patterns representing data This type of compositephysical visualization is fully manipulable but notactuated at all Such visualizations have been studied byeg Huron et al [8] In their study users were askedusing square wooden tiles of various colors (see figure 1)to build representations of a given dataset

Figure 3 BMW Kineticsculpture

Other examples include an activity logging visualizationbuilt by Michael Hunger [7] which uses stacks of LEGObricks of different colors to represent activities carried onduring each day (see figure 2) and Jacques Bertinrsquosphysical matrices [4]

Such manually arranged physical visualizations provide thebenefits of being highly flexible and requiring littleexpertise [8] However constructing and updating them

can be tedious and time consuming when manipulatedobjects are numerous

Actuated visualizationsBy introducing automatic actuation and computation it ispossible to make composite physical visualizationsdynamic It is then possible to automatically rearrange theobjects to reflect changes in data For exampleART+COM built a series of kinetic sculptures made ofobjects attached to winch-controlled wires [1] The heightof each object can be accurately controlled (see figure 3)Even though this allows visual representations to beupdated dynamically users are not able to manipulate theobjects directly This limitation is due to the wires used tocontrol the objects and preventing them from beingmanipulated

Some actuated composite physical visualizations can bemanipulable but the level of actuation is limited Forinstance Durrell Bishoprsquos Marble Answering Machine [2]uses physical tokens to represent incoming voicemessages For each new message a new token rolls downautomatically from a storage container to a presentationcontainer (see figure 4) To listen to the message theuser places the token in a specific spot However thesystem cannot be considered as fully actuated Forexample once heard a message token has to berepositioned in the storage container

More recently Follmer et al [5] developed inFORM adynamic shape display which using a large collection ofmoving vertical bars can change shape (see figure 5)This device is fully actuated in the sense that each of itsobjects (ie the vertical bars) can be movedcomputationally It is also manipulable as each object canreact to usersrsquo gestures However each object cannot be

considered fully mobile as it can move in only onedimension Moreover many of the supported gestures arenot direct-manipulation gestures Thus this system is notfully manipulable

Figure 4 Durrell Bishop MarbleAnswer Machine

Figure 5 inFORM a DynamicShape Display

ChallengesAs we can see through the previous examples selectedamong many others many different types of compositephysical visualizations exist However none of them is yetable to combine two properties that we considerimportant for an ideal composite physical visualizationfull actuation and full manipulability

Technological considerationsOne technological challenge is to support full actuationwhile ensuring the full mobility of objects Asdemonstrated by the kinetic sculpture (figure 3) and theinFORM system (figure 5) one cannot fully manipulateobjects that are physically constrained By contrastDurrell Bishoprsquos marbles are not constrained in any wayallowing users to grasp marbles and manipulate several ofthem at the same time However supporting full objectmobility will in many cases make the design more complexif computation or actuation features have to be embeddedin the objects Different approaches are possible [9] andcan be classified into extrinsic (eg magnetic fields [10])and intrinsic actuation (eg self-propulsion [6]) Intrinsicapproaches seem more realistic as they scale up toarbitrary numbers of objects and they do not require acontrolled environment to operate

Assuming that technology will soon make it to possible tobuild composite physical visualizations providing these twoproperties designing them will remain a challengeDesigners will need to consider many aspects to buildeffective and usable composite physical visualizations

Physical Object DesignOne aspect to consider is physical object design Toencourage users to take advantage of manipulation thedesign of the objects is crucial as it will constrain thepossible interactions Even for visualizations that aredynamic and updatable objects have to be carefullydesigned The choice of the form factor will impactpossible manipulations such as grouping stacking orassembling For instance square objects like LEGO brickscan be assembled and stacked easily while round objectscannot The right form factor is highly context dependentfor example it is often desirable to have objects with a flatbase to insure stability but the roundness of objects canbe also be exploited to ease actuation like in figure 4 Thesize and the material will also affect manipulationMedium sized objects are easy to handle but users cannotmanipulate many (dozens) at a time Furthermore lowfriction can make objects slippery and difficult to controlwhile heavy material can make manipulation tiresome

ConclusionWhile many current physical visualizations are monolithicand static we believe that physical visualizations made ofmultiple objects can better leverage both human andtechnological capabilities We called such visualizationscomposite physical visualizations and showed that theycan be usefully classified according to two dimensionstheir level of actuation and their manipulability Only afew systems are combining both characteristics and nonesupports both full manipulability and full actuationThrough examples we illustrated the tradeoffs betweenthese two dimensions and identified opportunities andchallenges for future research and design in this domainWe are aware that this position paper rises more questionsthan solutions but we hope it will lead to interestingdebates and discussions during this workshop

References[1] ART+COM Kinetic sculpture the shapes of things

to come https

artcomdeenprojectkinetic-sculpture2008

[2] Bishop D Durrell bishop marble answer machinehttpvimeocom19930744 1992

[3] Boleslavsk A Google eyehttpid144orgprojectsgoogle-eye 2012

[4] Charles P Pierre D and Jean-Daniel F BertifierNew interactions for crafting tabular visualizationsIn IHMrsquo14 26e conference francophone surlrsquoInteraction Homme-Machine (2014) 16ndash17

[5] Follmer S Leithinger D Olwal A Hogge A andIshii H inform dynamic physical affordances andconstraints through shape and object actuation InUIST (2013) 417ndash426

[6] FutureLab A E Spaxelshttpwwwaecatspaxels 2012

[7] Hunger M On lego powered time-tracking my dailycolumn httpjexpdeblog200808

on-lego-powered-time-tracking-my-daily-column2008

[8] Huron S Jansen Y and Carpendale SConstructing Visual Representations Investigatingthe Use of Tangible Tokens IEEE Transactions onVisualization and Computer Graphics 20 12 (Aug2014) 1

[9] Jansen Y Dragicevic P Isenberg P AlexanderJ Karnik A Kildal J Subramanian S andHornbaek K Opportunities and challenges for dataphysicalization In Proc CHIrsquo2015 (to appear) ACM(2015)

[10] Lee J Post R and Ishii H Zeron mid-airtangible interaction enabled by computer controlledmagnetic levitation In Proceedings of the 24thannual ACM symposium on User interface softwareand technology ACM (2011) 327ndash336

objects that can be rearranged in order to represent avariety of informative configurations We call themcomposite physical visualizations A major benefit of suchvisualizations is that they support modularity andupdatability but their design space is not well understood

Figure 1 Visualization usingcolored wooden tiles

Figure 2 Activity loggingvisualization built by MichaelHunger out of LEGO bricks

In this paper we show that composite physicalvisualizations can be classified according to twoorthogonal dimensions i) their level of actuation and ii)their manipulability Among existing systems some havea high manipulability but no support for actuation(eg [8]) while others are fully actuated but notmanipulable (eg [1]) Only a few systems are combiningboth qualities and none supports both full manipulabilityand full actuation We discuss the tradeoffs between thesetwo dimensions and identify the opportunities andchallenges for future research and design

Manually arranged visualizationsAn easy way to build a composite physical visualization isto arrange multiple objects manually in order to createvisual patterns representing data This type of compositephysical visualization is fully manipulable but notactuated at all Such visualizations have been studied byeg Huron et al [8] In their study users were askedusing square wooden tiles of various colors (see figure 1)to build representations of a given dataset

Figure 3 BMW Kineticsculpture

Other examples include an activity logging visualizationbuilt by Michael Hunger [7] which uses stacks of LEGObricks of different colors to represent activities carried onduring each day (see figure 2) and Jacques Bertinrsquosphysical matrices [4]

Such manually arranged physical visualizations provide thebenefits of being highly flexible and requiring littleexpertise [8] However constructing and updating them

can be tedious and time consuming when manipulatedobjects are numerous

Actuated visualizationsBy introducing automatic actuation and computation it ispossible to make composite physical visualizationsdynamic It is then possible to automatically rearrange theobjects to reflect changes in data For exampleART+COM built a series of kinetic sculptures made ofobjects attached to winch-controlled wires [1] The heightof each object can be accurately controlled (see figure 3)Even though this allows visual representations to beupdated dynamically users are not able to manipulate theobjects directly This limitation is due to the wires used tocontrol the objects and preventing them from beingmanipulated

Some actuated composite physical visualizations can bemanipulable but the level of actuation is limited Forinstance Durrell Bishoprsquos Marble Answering Machine [2]uses physical tokens to represent incoming voicemessages For each new message a new token rolls downautomatically from a storage container to a presentationcontainer (see figure 4) To listen to the message theuser places the token in a specific spot However thesystem cannot be considered as fully actuated Forexample once heard a message token has to berepositioned in the storage container

More recently Follmer et al [5] developed inFORM adynamic shape display which using a large collection ofmoving vertical bars can change shape (see figure 5)This device is fully actuated in the sense that each of itsobjects (ie the vertical bars) can be movedcomputationally It is also manipulable as each object canreact to usersrsquo gestures However each object cannot be

considered fully mobile as it can move in only onedimension Moreover many of the supported gestures arenot direct-manipulation gestures Thus this system is notfully manipulable

Figure 4 Durrell Bishop MarbleAnswer Machine

Figure 5 inFORM a DynamicShape Display

ChallengesAs we can see through the previous examples selectedamong many others many different types of compositephysical visualizations exist However none of them is yetable to combine two properties that we considerimportant for an ideal composite physical visualizationfull actuation and full manipulability

Technological considerationsOne technological challenge is to support full actuationwhile ensuring the full mobility of objects Asdemonstrated by the kinetic sculpture (figure 3) and theinFORM system (figure 5) one cannot fully manipulateobjects that are physically constrained By contrastDurrell Bishoprsquos marbles are not constrained in any wayallowing users to grasp marbles and manipulate several ofthem at the same time However supporting full objectmobility will in many cases make the design more complexif computation or actuation features have to be embeddedin the objects Different approaches are possible [9] andcan be classified into extrinsic (eg magnetic fields [10])and intrinsic actuation (eg self-propulsion [6]) Intrinsicapproaches seem more realistic as they scale up toarbitrary numbers of objects and they do not require acontrolled environment to operate

Assuming that technology will soon make it to possible tobuild composite physical visualizations providing these twoproperties designing them will remain a challengeDesigners will need to consider many aspects to buildeffective and usable composite physical visualizations

Physical Object DesignOne aspect to consider is physical object design Toencourage users to take advantage of manipulation thedesign of the objects is crucial as it will constrain thepossible interactions Even for visualizations that aredynamic and updatable objects have to be carefullydesigned The choice of the form factor will impactpossible manipulations such as grouping stacking orassembling For instance square objects like LEGO brickscan be assembled and stacked easily while round objectscannot The right form factor is highly context dependentfor example it is often desirable to have objects with a flatbase to insure stability but the roundness of objects canbe also be exploited to ease actuation like in figure 4 Thesize and the material will also affect manipulationMedium sized objects are easy to handle but users cannotmanipulate many (dozens) at a time Furthermore lowfriction can make objects slippery and difficult to controlwhile heavy material can make manipulation tiresome

ConclusionWhile many current physical visualizations are monolithicand static we believe that physical visualizations made ofmultiple objects can better leverage both human andtechnological capabilities We called such visualizationscomposite physical visualizations and showed that theycan be usefully classified according to two dimensionstheir level of actuation and their manipulability Only afew systems are combining both characteristics and nonesupports both full manipulability and full actuationThrough examples we illustrated the tradeoffs betweenthese two dimensions and identified opportunities andchallenges for future research and design in this domainWe are aware that this position paper rises more questionsthan solutions but we hope it will lead to interestingdebates and discussions during this workshop

References[1] ART+COM Kinetic sculpture the shapes of things

to come https

artcomdeenprojectkinetic-sculpture2008

[2] Bishop D Durrell bishop marble answer machinehttpvimeocom19930744 1992

[3] Boleslavsk A Google eyehttpid144orgprojectsgoogle-eye 2012

[4] Charles P Pierre D and Jean-Daniel F BertifierNew interactions for crafting tabular visualizationsIn IHMrsquo14 26e conference francophone surlrsquoInteraction Homme-Machine (2014) 16ndash17

[5] Follmer S Leithinger D Olwal A Hogge A andIshii H inform dynamic physical affordances andconstraints through shape and object actuation InUIST (2013) 417ndash426

[6] FutureLab A E Spaxelshttpwwwaecatspaxels 2012

[7] Hunger M On lego powered time-tracking my dailycolumn httpjexpdeblog200808

on-lego-powered-time-tracking-my-daily-column2008

[8] Huron S Jansen Y and Carpendale SConstructing Visual Representations Investigatingthe Use of Tangible Tokens IEEE Transactions onVisualization and Computer Graphics 20 12 (Aug2014) 1

[9] Jansen Y Dragicevic P Isenberg P AlexanderJ Karnik A Kildal J Subramanian S andHornbaek K Opportunities and challenges for dataphysicalization In Proc CHIrsquo2015 (to appear) ACM(2015)

[10] Lee J Post R and Ishii H Zeron mid-airtangible interaction enabled by computer controlledmagnetic levitation In Proceedings of the 24thannual ACM symposium on User interface softwareand technology ACM (2011) 327ndash336

considered fully mobile as it can move in only onedimension Moreover many of the supported gestures arenot direct-manipulation gestures Thus this system is notfully manipulable

Figure 4 Durrell Bishop MarbleAnswer Machine

Figure 5 inFORM a DynamicShape Display

ChallengesAs we can see through the previous examples selectedamong many others many different types of compositephysical visualizations exist However none of them is yetable to combine two properties that we considerimportant for an ideal composite physical visualizationfull actuation and full manipulability

Technological considerationsOne technological challenge is to support full actuationwhile ensuring the full mobility of objects Asdemonstrated by the kinetic sculpture (figure 3) and theinFORM system (figure 5) one cannot fully manipulateobjects that are physically constrained By contrastDurrell Bishoprsquos marbles are not constrained in any wayallowing users to grasp marbles and manipulate several ofthem at the same time However supporting full objectmobility will in many cases make the design more complexif computation or actuation features have to be embeddedin the objects Different approaches are possible [9] andcan be classified into extrinsic (eg magnetic fields [10])and intrinsic actuation (eg self-propulsion [6]) Intrinsicapproaches seem more realistic as they scale up toarbitrary numbers of objects and they do not require acontrolled environment to operate

Assuming that technology will soon make it to possible tobuild composite physical visualizations providing these twoproperties designing them will remain a challengeDesigners will need to consider many aspects to buildeffective and usable composite physical visualizations

Physical Object DesignOne aspect to consider is physical object design Toencourage users to take advantage of manipulation thedesign of the objects is crucial as it will constrain thepossible interactions Even for visualizations that aredynamic and updatable objects have to be carefullydesigned The choice of the form factor will impactpossible manipulations such as grouping stacking orassembling For instance square objects like LEGO brickscan be assembled and stacked easily while round objectscannot The right form factor is highly context dependentfor example it is often desirable to have objects with a flatbase to insure stability but the roundness of objects canbe also be exploited to ease actuation like in figure 4 Thesize and the material will also affect manipulationMedium sized objects are easy to handle but users cannotmanipulate many (dozens) at a time Furthermore lowfriction can make objects slippery and difficult to controlwhile heavy material can make manipulation tiresome

ConclusionWhile many current physical visualizations are monolithicand static we believe that physical visualizations made ofmultiple objects can better leverage both human andtechnological capabilities We called such visualizationscomposite physical visualizations and showed that theycan be usefully classified according to two dimensionstheir level of actuation and their manipulability Only afew systems are combining both characteristics and nonesupports both full manipulability and full actuationThrough examples we illustrated the tradeoffs betweenthese two dimensions and identified opportunities andchallenges for future research and design in this domainWe are aware that this position paper rises more questionsthan solutions but we hope it will lead to interestingdebates and discussions during this workshop

References[1] ART+COM Kinetic sculpture the shapes of things

to come https

artcomdeenprojectkinetic-sculpture2008

[2] Bishop D Durrell bishop marble answer machinehttpvimeocom19930744 1992

[3] Boleslavsk A Google eyehttpid144orgprojectsgoogle-eye 2012

[4] Charles P Pierre D and Jean-Daniel F BertifierNew interactions for crafting tabular visualizationsIn IHMrsquo14 26e conference francophone surlrsquoInteraction Homme-Machine (2014) 16ndash17

[5] Follmer S Leithinger D Olwal A Hogge A andIshii H inform dynamic physical affordances andconstraints through shape and object actuation InUIST (2013) 417ndash426

[6] FutureLab A E Spaxelshttpwwwaecatspaxels 2012

[7] Hunger M On lego powered time-tracking my dailycolumn httpjexpdeblog200808

on-lego-powered-time-tracking-my-daily-column2008

[8] Huron S Jansen Y and Carpendale SConstructing Visual Representations Investigatingthe Use of Tangible Tokens IEEE Transactions onVisualization and Computer Graphics 20 12 (Aug2014) 1

[9] Jansen Y Dragicevic P Isenberg P AlexanderJ Karnik A Kildal J Subramanian S andHornbaek K Opportunities and challenges for dataphysicalization In Proc CHIrsquo2015 (to appear) ACM(2015)

[10] Lee J Post R and Ishii H Zeron mid-airtangible interaction enabled by computer controlledmagnetic levitation In Proceedings of the 24thannual ACM symposium on User interface softwareand technology ACM (2011) 327ndash336

References[1] ART+COM Kinetic sculpture the shapes of things

to come https

artcomdeenprojectkinetic-sculpture2008

[2] Bishop D Durrell bishop marble answer machinehttpvimeocom19930744 1992

[3] Boleslavsk A Google eyehttpid144orgprojectsgoogle-eye 2012

[4] Charles P Pierre D and Jean-Daniel F BertifierNew interactions for crafting tabular visualizationsIn IHMrsquo14 26e conference francophone surlrsquoInteraction Homme-Machine (2014) 16ndash17

[5] Follmer S Leithinger D Olwal A Hogge A andIshii H inform dynamic physical affordances andconstraints through shape and object actuation InUIST (2013) 417ndash426

[6] FutureLab A E Spaxelshttpwwwaecatspaxels 2012

[7] Hunger M On lego powered time-tracking my dailycolumn httpjexpdeblog200808

on-lego-powered-time-tracking-my-daily-column2008

[8] Huron S Jansen Y and Carpendale SConstructing Visual Representations Investigatingthe Use of Tangible Tokens IEEE Transactions onVisualization and Computer Graphics 20 12 (Aug2014) 1

[9] Jansen Y Dragicevic P Isenberg P AlexanderJ Karnik A Kildal J Subramanian S andHornbaek K Opportunities and challenges for dataphysicalization In Proc CHIrsquo2015 (to appear) ACM(2015)

[10] Lee J Post R and Ishii H Zeron mid-airtangible interaction enabled by computer controlledmagnetic levitation In Proceedings of the 24thannual ACM symposium on User interface softwareand technology ACM (2011) 327ndash336