ESWC Summer School 2016 Understanding and...

ESWCSummerSchool2016UnderstandingandCommunicating

withData

Dr.ChrisPhetheanUniversityofSouthampton

AboutMe

• Undergraduate,postgraduateandPhDfromUniversityofSouthampton• ComputerScience->WebScience->DataScience• Now• ResearchFellow• PrimarilyworkingontheEuropeanDataScienceAcademyproject• TeachonMScDataSciencecourse

Data…there’salotoutthere!

•AccordingtoIBM:•2.5Quintillionbytesofdataarecreatedeachday•90%ofthedataintheworldhasbeencreatedinthelastTWOyears.

http://www-01.ibm.com/software/data/bigdata/what-is-big-data.html

https://www.linkedin.com/pulse/4-big-data-sources-can-used-commercial-lending-risk-modeling-azalone

Communicatingwithdata

• Manypeoplestruggletoconveythevalueoftheirdata• Asthecomplexityofdataincreases,sotoodoesthecomplexityoftheanalysisdonewithit• Theseresultsthenneedtobecommunicatedtoarangeofaudiences

• “Juststatisticsdoesnotgivethefullpicture”– ClaudiaonMonday

• Alongsidethiswecanusedatavisualisation togetabetterideaofwhatthedataissaying.

DataVisualisation is…

• “TheVisualDisplayofQuantitativeInformation”• Thetitleofthisbookprovidesagoodstartingdefinition• WewillseelaterthatTufte isabignameintheworldofvisualisation

WhataboutWikipedia?

• “tocommunicateinformationclearlyandefficientlyviastatisticalgraphs,plotsandinformationgraphics”• https://en.wikipedia.org/wiki/Data_visualization

• Keypoints• Communicateinformation• Clearly• Efficiently• Usingstatisticalgraphs,plots,informationgraphics

Anexample– Detroit(1)

Anexample– Detroit(2)

https://www.motorcitymapping.org/#t=parcels&s=detroit&f=all&x=preset2

Whydatavisualisation?

• Easiertospotkeyfeaturesinthedata• Actonemergingtrends

• Explainthestorythatthedatarepresents• Explorewhatthedatashowstofindinterestinginsights• Easiertocommunicatefindingsthanwithaspreadsheet

Dataisn’tnew:we’vealwaysneededtounderstandandcommunicateinsights

• Londoncholeraoutbreak,1854• JohnSnowinvestigatedtheoutbreak,observingclustersaroundBroadStreet• Usingamap-basedvisual,hetalliedhowmanycasewerereportedineachlocation

CholerainLondon,1854

JohnSnow

• Uponfurtherinvestigation,henoticedthatmanycaseswereclusteredaroundtheBroadStreetwaterpump.• Heusedthemaptoconvincetheauthoritiesandorderedthatthehandlewasremovedandthepumpinvestigated,andcasessubsequentlyreduced.• Thishelpedtochangescientificunderstandingofhowdiseasesarespread.

TheFirstBarChart:Anapology

15

Readmore:Beniger,JamesR.,andDorothyL.Robyn.1978.“QuantitativeGraphicsinStatistics:ABriefHistory”.TheAmericanStatistician 32(1).[AmericanStatisticalAssociation,Taylor&Francis,Ltd.]:1–11.doi:10.2307/2683467.

TypesofvisualisationExploreorExplain?

TypesofVisualisation

• Datavisualisationscangenerallybesplitintotwomaincategories:

Exploratory Explanatory

‘Explore’yourdata ‘Explain’yourfindings

Provideaway toexamineyourdatafromdifferentviewpointsorwithdifferentfilters.

You’vefoundtheinterestingpointsinthedata,nowyouwant toexplainorcommunicatethemtoothers

Generallyallowsthereadertodrivetheexperience:Letthemgainasenseofwhatthedataisaboutandwhatitissaying,sothatyou/theycanfindtheinterestingpoints.

Generally‘author-driven’: yourmainfocusisontellingthestorythatyou’vefound,soyouwantto presentitasclearlyaspossible.

ExploratoryVisualisation– Wherecanyouaffordtolive?

http://www.bbc.co.uk/news/business-23234033

ExplanatoryVisualisation

http://wheredoesmymoneygo.org/bubbletree-map.html#/~/total

Storytelling

Humanshavebeentellingstoriesforcenturies…

https://annadoherty.wordpress.com/2012/10/06/history-of-animation-cave-paintings/

Storytellingwithdatavisualisation

• Storytelling:“Theconveyingofeventsinwords,soundand/orimages,oftenbyimprovisationorembellishment.”(Wikipedia)

• Recallthattheaimofvisualisationistoeffectivelyconveysomeinformationfromdata.• Youwantdecisionmakerstobeabletoseetheresults ofyouranalysis.• Theseshouldbeconveyedinamannerthatissuitablefortheintendedaudiencetounderstand.

Considerationsforthestory

• ClaudiacoveredtheseabitonMonday

• Whoareyouraudience?• Whatisthestory?• Whatdoyouwantpeopletodoafterseeingyourvisualisation?

1.Youraudience

•Whatpriorknowledgedotheyposses?• Aretheyexpertsinthefield?Orgeneralpublic?

•Howmuchtime dotheyhave?•Willtheybeabletoexplorethevisualisationindepth,orjustglanceatit?

•Whatformat aretheycomfortabledigestinginformation?• Charts?Infographics?Interaction?

25

2.Thestory

•Whatisitthatthedatashows?•Whatisyourkeyresult?Themessage.• Notthedataitself.• Northemethod.Theresult.

•Thestoryisvital– thisiswhatyouwillbe‘telling’inyourvisualisation.•Whydoyouwantpeopletolookatyourvisualisation?

26

2.Thestory(2)

•Youreallyhavetoknowthestoryyourselfinorderto‘sellit’.• “Simplify,thenexaggerate”.• GeoffreyCrowther,TheEconomist• ReferringtojournalismtechniquesthathavebeenfollowedatTheEconomistsincethe1950s

• Important:thisdoesnotmeanlying!

27

3.Theaction

•Whatdoyouwantpeopletodo afterwards?• Thisdependsontheaudience,andcontext.

•Changebehaviour?• E.g.,• Implementanewpolicy• Allocateresources• Campaignforchange• Changeeatinghabits• ...

28

Structuringyourstory:author- orreader-driven?

• Thetypeofstorywilldeterminethetypeofvisualisationyouneedtocreate.• Explanatoryvisualisationsaretypically‘author-driven’.• Anauthor-drivenstoryhasalinearordering,whichyouwalkthereaderthrough,withlittleornointeractivity.

• Exploratoryvisualisationsaremorelikelytobe‘reader-driven’.• Forreader-drivenstories,thereisnoprescribedorderingtothecontent,allowingthereadertointeractandfindtheirownstory.

Balancingauthor- andreader-drivenstories

• Asacompromisebetweenthetwoapproaches,wecanmergethemboth1. Initialauthor-drivensequence,thatopensupintoaninteractive

narrativeforexploration.

Oftendescribedasthe‘martiniglass’.


• Asacompromisebetweenthetwoapproaches,wecanmergethemboth2. Author-drivensequencewithinteractionallowed‘mid-narrative’– user

canexploreaparticularpointmorebeforemovingahead.

Oftendescribedasthe‘interactiveslideshow’.


• Asacompromisebetweenthetwoapproaches,wecanmergethemboth3. Generalthemepresentedallowingreader-driveninteraction,butthey

canthenchooseparticularelementstorevealadditionalauthor-drivendetails.

Oftendescribedasthe‘drill-downstory’.

Findinga‘compelling’narrative

•Thenarrativeshouldlinkthefactsthatyou’vefoundtogetherinanengaging way.• Ifyoucan’tfindanarrativestructure,thenyourfindingsmaynotbeasgoodasyouthink• Oryourdatamaybemoresuitedtoanexploratorydataanalysis(reader-driven) styleofvisualisation,lettingtheviewerfindtheirownstory.

33

https://hbr.org/2013/04/how-to-tell-a-story-with-data/

Communicatingyourmessage

•Twomaintechniquesforensuringyoueffectivelyconveyyourstory:•Highlightandemphasise• Showtheimportantinformationthatiskeytothestory,withoutdistractions.

•Organise• Structureinformationtheclearlyguidereaderstoproducethemostunderstanding.

Structurewithoutastory

35

https://contently.com/strategist/2015/11/25/why-good-infographics-are-more-than-just-pictures-and-numbers/

HighlightingandEmphasisingTufte’s theoriesonchartjunk

Emphasisingkeyinformationforthemind(1)

• Makinganeffectivegraphiciseasierifweknowhowthemindworks• Asanexample,lookatthisimageandseehowmany‘3’sthereare:

198356425378452384684796121263489654231986354871315546821357912349876543212345

Emphasisingkeyinformationforthemind(2)

• Wecanmakeitaloteasiertoperceivebychangingthingsslightly

• Shadevariations=>Easytoperceive• Shapevariations=>Difficulttoperceive

1983 564253 784523 84684796121263 48965423 19863 548713 155468213 579123 49876543 2123 45

Chartjunk

•Chartjunk,accordingtoTufte,isanythingthatcanberemovedfromachartwithoutchangingitsmeaning• Itobscuresthetruemeaningandstory• Imageryisnotinformation• Imagerydrawsattentionawayfromdata• Short-termmemoryresourcesareusedtoidentifytheimagesratherthanunderstandthechart’smeaning

Chartjunk Example

0

1000

2000

3000

4000

5000

6000

Year 1 Year 2 Year 3 Year 4 Year 5

Number of customers

Number of customers

Necessary?

Necessary?

Necessary?

Chartjunk Example- Fixed

0

1000

2000

3000

4000

5000

6000

Year 1 Year 2 Year 3 Year 4 Year 5

Number of customers

Let’sfixthishorribleexample(2)

43

https://datavizblog.com/category/chartjunk/

‘Wedges’oftimeaddnothing–they’renotevenproportionallysized!

Pencils…they’renotdata

Whosearmisthis?!

Isthissayingsomeoneisworkinglateatnight,orontheircoffeebreak?

0 10 20 30 40 50 60 70 80

FreeTimeandRest

Sportsand/orHobby

TimeWithFriends

TimewithFamily

Community&VolunteerWork

FurtherEducation

Nothing

%ofentrepreneurs

Whatentrepreneurssacrificedtostarttheirbusiness

44

Afterremovingallthe‘junk’andkeepingonlythe‘data’,weget:

Thegraphicwemadeissimple,butwillitreallyinterestpeople?Willtheyrememberit?

TheBeautyParadox

• Howcomplexshouldagraphicbe?• Howmuchshouldyoushow?

• Tufte emphasisesminimalism• Communicateasmuchinformationfromasfewpixelsaspossible

• Dependingontheaudience,thismayormaynotbethebestapproach• Whencreatingavisualisation,youmustbalancetheneedtofocusonthestoryofthedata,withartanddesigndecisionstohelpyouraudienceengagewithandrememberthevisual.

OrganisingandStructuringInformationVisualperceptionandgestalttheory

Howisdatadisplayed?

• Weturndataintoagraphicalrepresentationusing‘visualencodings’• Dependingonthetype ofdatayouhave,differentencodingswillbesuitable• E.g.ordereddata(discreteorcontinuous):• Size – thelargerthecircle,themoreofsomething• Orientation/rotation• Colour (saturation)

• E.g.nominaldata(categories):• Colour (hue)

• RedcirclestorepresentManchesterUnitedsupporters• BluecirclestorepresentChelseasupporters

Ordereddata

SIZE ORIENTATION / ROTATION COLOUR

Unordereddata

ResultingDisappointment

PriorOptimism

DifferencesinEnglandandWalessupporters’feelingsbeforeandafterEuro2016

Unordereddata

ResultingDisappointment

PriorOptimism

DifferencesinEnglandandWalessupporters’feelingsbeforeandafterEuro2016

Walessupporters

Englandsupporters

RankingVisualEncodings

• In1985,ClevelandandMcGillpublishedintheJournaloftheAmericanStatisticalAssociation

• ‘GraphicalPerceptionandGraphicalMethodsforAnalyzing ScientificData’

• Theyproposedbasicguidelinesforchoosinganappropriategraphicform(oranappropriateencoding)

• Thepaperlistsandranks10‘elementaryperceptualtasks’

ClevelandandMcGill

• Rankofgraphicpropertiesbasedonhumanabilitytounderstandinformation• Theseareparticularlyrelevantfordetectingdifferencesandmakingcomparisons• Positionalongacommonscale• Positiononidenticalbutnonalignedscales• Length• Angle,slope• Area• Volume,density,coloursaturation• Colourhue

• Foraccuratecomparisons,graphicalformsfromthetopofthislistshouldbeused.

ChoosingaGraphicforVisualPerception

53

• Question:• Foraprecisecomparison,whichtypeofchartwouldbemostaccurate?

• Barchart?• Bubblechart?• Heatmap?

ClevelandandMcGill- Example

0

0.5

1

1.5

2

2.5

3

3.5

0 0.2 0.4 0.6 0.8

0 0.2 0.4 0.6 0.8

ItiseasierforhumanstoperceivethedifferencesthevaluesforX=0.2andX=0.4usingpositionalongacommonscale(onthescatterplot),thanbyusingarea(inthebubblechart).

ChoosingaGraphicforVisualPerception

55

• Whenprecisionislessimportant,lowerrankedvisualformscanbeusedwell• E.g.,spottinglargerpatterns

Butreallydifficulttocomparetwosimilarareas!

56

Modernversion:https://en.wikipedia.org/wiki/Charles_Joseph_Minard#/media/File:Minard_map_of_napoleon.png

Howmanydimensionscanyoufindbeingrepresentedonthismap?

PatternRecognition

• Aswellastheencodingused,human’salsoperceiveinformationbasedonhowitisorganised• AccordingtoGestaltTheory,thebrainandvisualsystemfollowanumberofprinciplesforperceptualorganisation• Howthebraingroupselementsinto‘patterns’

• ThisemergedasaschoolofthoughtfromGermanpsychologistsin1930s/40s

GestaltTheory:PrinciplesofOrganisation

• Proximity– objectsthatarecloseareperceivedtobenaturalgroups

580

1

2

3

4

5

6

GestaltTheory:PrinciplesofOrganisation(2)

• Similarity– identicalorsimilarobjectsbelongtoagroup

590

1

2

3

4

5

6


• Connectedness– linkingusingalineorsimilar

60


• Continuity:Smoothcontoursareeasiertoperceivethansharpangles

61

Person1 Person2 Person3 Person1 Person2 Person3


• Closure:boundedareasindicateagrouping

6200.51

1.52

2.53

3.54

4.55

GestaltTheory:TakeawayMessage

• Graphicscanbemademorefunctionalusingsimpletechniquesaroundorganisationandlayoutofcomponents• Conversely,notusingGestaltprinciplescorrectlymightmisleadusersastheywillmakewrongassumptionsaboutthecomponentsyoudisplay

63

Deceivingyourbrain

• Aswefindoutmoreabouthowthebrainworks,andhowitmakesassumptionsaboutwhatitalreadyknows,wecanlearnhowtolietoit• Ormoreimportantly,howtoavoidlyingtoit

• E.g.Whatdoyousee?

Whosawawhitesquareoverfourblackcircles?

65

YourBrainisDeceivingYou!

• Thereisnowhitesquare.• Removethe4‘pacman’shapes,andthereisnothingthere!

• Yourbrainismakingassumptionsbasedonwhatitknowsabouttheworldalready• “Emptyareas”don’texist

• Thebrain‘perceives’morethanwhattheretinasinyoureyesactuallysee

66

67

http://thebrainbank.org.uk/wp-content/uploads/2012/08/Ponzo_illusion.gif

Whichyellowlineislonger?

68

http://thebrainbank.org.uk/wp-content/uploads/2012/08/Ponzo_illusion.gif

Whichyellowlineislonger?(2)

Don’tstretchthetruth!Tufte’s LieFactor

Warning!Thefollowingslidescontaindetailsonhowtodo“bad”things.Usethisknowledgeforgood,andavoidthesemistakes!

“Withgreatpowercomesgreatresponsibility”

BeObjective

• Tufte createdaformulatocalculatethe‘liefactor’orhow‘misleading’agraphicis

• Liefactor=

• whereEffectSize =

• Areliablegraphicshouldbebetween0.95and1.05.

EffectSizeInGraphic / EffectSizeInData

(value2− value1) / value1

LieFactor:AnExample

10200

10400

10600

10800

11000

11200

11400

11600

11800

12000

CandidateA CandidateB

Votes

• ThissuggeststhatAreceivedaroundthreetimesasmanyvotesasB.

LieFactor:AnExample

0

2000

4000

6000

8000

10000

12000

14000

CandidateA CandidateB

Votes • Inreality,thingsaremuchcloser.

• Theactualvaluessuggestaround11,750forA,and10,800forB– About13.2%difference

• Theoriginalgraphhowever,makesthedifferenceseem61.3%betweenthetwovalues

• 0.613/0.132=4.64

• Theoriginalchartthereforehadahighliefactorandisoverstatingtheeffect.– Wellover1.05!

Piecharts+3Dareparticularlybad

• Tocommunicateamessageclearlyandeffectively.• Agraphorchartshouldclarifywhatnumbersinatablearesaying.• 3DdoesNOThelpwiththis.

75

VotesABCDEF

Piecharts+3Dareparticularlybad

• Tocommunicateamessageclearlyandeffectively.• Agraphorchartshouldclarifywhatnumbersinatablearesaying.• 3DdoesNOThelpwiththis.

76

VotesABCDEF

VotesABCDEF

Party VotesA 60B 195C 60D 78E 265F 80

Whatmakesavisualisationa‘bad’visualisation?

78

EXERCISEInpairs

Discusswhetheryouthinkthevisualisationisgoodorbad?

Why?Whatisthestorythatitistryingtotell?

Whatwouldyouchangeinordertoenhancethestory?

79http://www.businessinsider.com/gun-deaths-in-florida-increased-with-stand-your-ground-2014-2?IR=T

The“fixed”version…

80https://rstudio-pubs-static.s3.amazonaws.com/70948_b5daeeeecf824047b580c1a101407a98.html

Let’sfindallthethingswrongaboutthisvisualisation

81

• Whatproblemdoesthissolve?• Isthisthebestwaytodisplaytheinformation?• Doeseverything addvalue?• Arethereclearlabels?

82

ThankstoThierryDriver(Tableau)forthisexample

Titleisclearandaddressesthequestion

Barchartsallowforaclearercomparisonofvalues

Labelsaresimplerandmoreconsistent

Sortingofbarsaddsanothervisualisationelement

DataVisTechnologies

Webtechnologiesandtheirroleinvisualisations

• Evenifyou’reusingahigh-level,onlinetoolthathandlesthecodeforyou,itispertinenttounderstandtheunderlyingtechnologiesoftheWeb.• Thesewillbe‘manipulated’tocreatemanytypesofgraphics.• Coveringthesetechnologiesindepthisnotthefocusofthecourse.Thefollowingisa(very)briefintroduction.

84

85Imagefromhttp://blog.teamtreehouse.com/wp-content/uploads/2014/11/progressive-enhancement.png

Hypertext

• Hypertext:textwhichcontainslinkstoothertexts.• HTML(HypertextMarkup Language)• AlanguagethatdefinesthebasicstructureandcontentofaWebpageusingmarkup ‘tags’.• Eachtagdescribesadifferenttypeofcontent• E.g.,• <html></html>Foranhtmlpage• <body></body>Forthebodyofanhtmlpage• <p></p> Aparagraph• <ahref=“www.bbc.co.uk”>TheBBC</a>AlinktotheBBC

• ReadbyaWebBbrowser,whichdecideshowtodisplaythecontent.

86

CSS(CascadingStyleSheets)

• DefinethedesignandpresentationofanHTMLdocument.• TellstheWebbrowsermoreinformationaboutthelookandfeelofeachHTMLelement– howshoulditbedisplayedonthescreen?• E.g.,

p{font-family:“TimesNewRoman”;color:blue;font-size:12px;

}

“MakeallparagraphsuseTimesNewRomanfont,size12,andcolourthetextblue.”

87

Javascript

• ThethirdessentialtechnologybehindtheWWW.• TheprogramminglanguageoftheWeb.• Programsthe‘behaviour’oftheWebpage.• Client-sidelanguage(runsontheuser’smachine,notontheserver).• Definehowapageshouldbehavewhenthereisauserevent(e.g.,clickingonsomething),

88

Javascript withHTML

• WhenthebrowserdisplaysanHTMLpageitcreatesahierarchicalstructureoftheHTMLelements.• ThisiscalledtheDocumentObjectModel(DOM).

• Javascript canaccessthissothatitcanchangespecificstylesofparticularelements• TheinteractionwiththeDOMiskeytohowD3 isusedtomakevisualisations.

89

D3.js(Data-DrivenDocuments)

• Javascript library(“.js”).• BasedonstandardWebtechnologies.• UsesHTML,SVG,andCSStocreatevisualisationsfromdata.• Drives theconnectionbetweendata (providedbyuser)anddocuments(renderedbytheWebbrowser).

90

Generalsteps

• Loadingthedataintobrowser’smemory.• Bindingthedatatoelementswithinthedocument.• Transformingelementsbysettingvisualpropertiesbasedonthebounddata.• Transitioningelementsbetweenstates• Responsetouserinput.

91

D3

• DynamicallyaccesstheDOMbehindawebpagetoapplystyles• DOM:• DocumentObjectModel• HierarchicalStructureofHTMLelements• Allowsthingslike• d3.select(“body”).append(“p”).text(“NewParagraph”);

92

D3:ChainSyntax

• Using“.”stochainmethodstogether• Performseveralactionsinasinglelineofcode• Fastandeasy,butcancausedebuggingproblemslater• Javascript doesn’tcareaboutwhitespaceorlinebreaks• d3.select(“body”)

.append(“p”)

.text(“NewParagraph”);

93

D3:ChainSyntax

• d3.select(“body”).append(“p”).text(“NewParagraph”);• Firstpasses‘select’aCSSselector– “body”,returningthefirstelementinDOMthatmatches.• ThencreatesanewDOMelement“p”andappendsittotheendofthepreviouslyselectedelement.Soaparagraphisappendedtothebodyelementofthedocumentinthiscase.Thenewelementispassedoninthechain.• Tothethetext()functionwhichtakesastringandinsertsitintothecurrentlyselectedelement(the“p”elementasthiswaspassedfromappend).

94

D3Example• http://d3.artzub.com/wbca/• InteractivevisualisationofWorldBankcontracts

95

Tableau

• Forthehands-onsessionafterlunchwewilluseTableau• Draganddropvisualanalyticspackage• Connecttodatafrommultiplesourceswithoutanycode• Quicklybuildvisualstoexplorethedataandspottrends• Thecreatedashboardsandstoriestopresentyourdataandtellcompellingnarratives

• Thereisalicensekeyforyoualltouse,andyoucanuseTableaufortherestoftheweekinyourprojects• Or,ifyouwanttouseD3thereareawiderangeoftutorialsavailableonline

TableauPublic

• TableauDesktopletsyoupublishyourworkbookto‘TableauPublic’• Hereyoucanbeaddedtotheirgalleryofprojects,withyourvisualisations hostedliveonline• Remainconnectedtothedatasotheyarefullyinteractive• WorkexactlyastheydoinTD• Cancreatecompellingdatastoriesandsharethemwithyouraudiencesimplyandeasily

Summary

• Thepointofdatavisualisationistocommunicatewhatisimportantinyourdata,clearlyandeffectively• Youmayhaveaparticularstorytotell,oritmaybethatyouwanteachreadertofindtheirownstoryinthedata• Regardless,youwanttohighlightthekeyelementsofthedata,andnotdistractthereaderwithchartjunk orunnecessary‘art’• Wecanuseknowledgeabouthowthebrainworkstoorganisecontentonthepage/screeneffectivelytoenhanceunderstanding• Agoodvisualisationshouldnotlie.• Thereareahugerangeofvisualisationtechnologies.D3ispopularforcodingthem.Tableauisapopularhigh-leveldesktoptool.

FurtherReading

• Tufte,Edward.“TheVisualDisplayofQuantitativeInformation”2ndEdition• Cleveland,WilliamS.,andRobertMcGill."Graphicalperceptionandgraphicalmethodsforanalyzing scientificdata."Science 229.4716(1985):828-833.• Murray,Scott.InteractivedatavisualizationfortheWeb."O'ReillyMedia,Inc.",2013.• D3Tutorialssuchashttps://www.dashingd3js.com/table-of-contents• TheD3galleryforadaptableexamples:• https://github.com/mbostock/d3/wiki/Gallery

• Cairo,Alberto.TheFunctionalArt:Anintroductiontoinformationgraphicsandvisualization.NewRiders,2012

99

ESWC Summer School 2016 Understanding and...

Documents

Transcript of ESWC Summer School 2016 Understanding and...