The Guide to Researching Audiences Concise edition · TV only), with an emphasis on ensuring that...
Transcript of The Guide to Researching Audiences Concise edition · TV only), with an emphasis on ensuring that...
The Guide to Researching AudiencesConcise edition
Dr Rachel Quirk, Martin Olver, Dr Max Hammond & Dr Claire Davies
JISC Content April2009
JISC, British Library, BBC, National Health Service, Becta, and Museums, Libraries and Archives Council working together to fully realise the potential of e-content for all users. For more information on the Strategic Content Alliance, please visit:
www.jisc.ac.uk/contentalliance
Produced and funded by JISC
Curtis+Cartwright Consulting LtdMain Office: Surrey Technology Centre, Surrey Research Park, Guildford, Surrey GU2 7YG T: +44 (0)1483 685020 E: [email protected] W: www.curtiscartwright.co.uk
This document is available in alternative formats which can be found at: www.jisc.ac.uk/contentalliance
Contents PAGE 3
ContentsOverviewofthisguide 4
1. Introduction 51.1 Theconciseedition 51.2 What,why,who…? 51.3 Usingthisconciseedition 71.4 Supportingdocuments 7
2. Describing and defining the target audience 92.1 Theimportanceofthetargetaudience 92.2 Theconceptofthetargetaudiencein
servicedevelopment 102.3 Definingthetargetaudience 102.4 Describingthetargetaudience 11
3. Planning audience research 123.1 Introduction 123.2 Objectivesofaudienceanalysis 123.3 Choosingthemethodology 153.4 Researchingnon-usage 163.5 Recruitingsubjectsforaudienceresearch 163.6 Othersourcesofaudiencedata 17
4. Collecting audience data 184.1 Undertakingresearch:in-houseorthirdparty? 184.2 Overviewofdatacollectionmethods 19
5. Modelling the audience 225.1 Introduction 225.2 Analysingquantitativedata 235.3 Makinguseofqualitativedata 235.4 Personasanduserprofiles 24
6. Making use of audience research 256.1 Introduction 256.2 Audienceresearchincontext 26
Glossary 27
The Guide to Researching AudiencesConciseedition
PAGE 4 Overview of this guide
Overview of this guide
PLANNING AUDIENCE RESEARCH
COLLECTION OF AUDIENCE
DATA
MODELLING THE AUDIENCE
MAKING USE OF AUDIENCE
RESEARCH
Section 3
Setting clear objectives for research activities, and planning activities to meet these objectives
Section 5
Analysing and interpreting the data collected to build up a picture of the audience
Section 4
The methods available for collecting audience data, and deciding how to conduct the research
Section 6
Using the research to meet the needs, wants and expectations of the audience
DESCRIBING AND DEFINING THE TARGET
AUDIENCE
Section 2
Defining who you would like your audience to be,and how to approach segmenting the audience
GLOSSARY
INTRODUCTION
SUPPORTING INFORMATIONCORE GUIDANCE
Section 1
An introduction to the toolkit, including an overview of how it might be used in practice
Annex A
A glossary of the key terms used in the toolkit (eg ethnographic research)
1. Introduction PAGE 5
1. Introduction
1.1 The concise edition1.1.1 Thisisthe‘conciseedition’of‘TheGuidetoResearchingAudiences’(‘theGuide’).1Itisashorterversion
ofthemainGuide,designedtoprovideaneasilyaccessiblesummaryofthekeyprinciplesofaudienceresearchandsomepracticalinformation.
1.1.2 ThefullversionoftheGuidecontainsmoredetailedinformationthanthisconciseedition,buttheyarelaidoutinexactlythesamewaytomakeiteasyforyoutofollow.
1.1.3 Thisconciseeditionalsoincludessnapshotsofthecasestudieswhichsupport‘TheGuidetoResearchingAudiences’(seesub-section1.4below).
1.1.4 TheGuideandsupportingdocumentshavebeenproducedbyCurtis+CartwrightConsultingLtdonbehalfoftheStrategicContentAlliance.
1.2 What, why, who…?
What is in The Guide to Researching Audiences?1.2.1 TheGuidesetsoutthebasicprinciplesofaudienceresearch.Thesecanbefollowedregardlessofthetype
ofserviceoraudience,andwillhelpyoutoconductaudienceresearchmoreeffectively(betterresults)andefficiently(lowereffort),withfewerproblemsandunforeseencomplications.Theyprovidethebuildingblockstoenableyoutodesign,conductandapplyyourownaudienceanalysisresearch.WhatthisGuidewillnotprovideyouwithisaready-madeaudienceanalysisprogrammespecificallydesignedforyourservice.
1.2.2 Itisaguidetocurrentgoodpracticeandastartingpointforfurtherreading.ThereisnothingveryradicalintheGuide–therearemanyotherresourcesavailableonaudienceanalysisandmodelling,someofwhicharereferencedinAnnex D (Bibliography) of the full version of the Guide.
1.2.3 TheGuidehasaspecificfocusononlinedigitalservices.Thedigitalrevolutionhasresultedinanexplosionofpossibilities:newandinnovativeservices;adelugeofcontent;availabilityofadditionalresearchtechniques(egwebstatistics);andpotentialnewaudiences.Relationshipsbetweendigitalcontentandnon-digitalcontent,betweendigitalservicesandnon-digitalservices,andbetweena‘digitalaudience’anda‘non-digitalaudience’areanewandinterestingconsiderationforserviceprovidersandarediscussedinAnnex C (Digital Audiences and Digital Services) of the full version of the Guide.
1 The Guide to Researching Audiences, Curtis+Cartwright Consulting Ltd, v1.0, 5 December 2008.
The Guide to Researching AudiencesConciseedition
PAGE 6 1. Introduction
Why do audience research?1.2.4 Itisincreasinglyimportantforpublicsectorservicesandresourcestobeabletodemonstratethatthey
areusedandvaluedbyanappropriateaudience.TheusesofaudienceresearcharediscussedindetailatAnnex B (Making Use of Audience Research) of the full version of the Guide,andextendfarbeyondsimplydemonstratingacertainnumberofvisitorsorwebsitehitstotickaboxonaservicelevelagreement.Audienceresearchcanbeusedto:
�� Developaproductorservice
�� Evaluatetheservice(egaudiencesatisfaction)
�� Demonstrateaccountabilitytofunders(egvalueformoney,highlyvaluedbyanicheaudience)
�� Informlong-termplanning
�� Attractsponsorshipandmarketing
�� Informbusinessplanning(egtofeedintoabusinesscase)
�� Informthemanagementofcontractualrelationships
�� Supportprofessionaldevelopment
Case study extract… Design of the BBC iPlayer2
Using audience research to design, develop and test a new version of a service
In2008theBBCdecidedtolauncharedesignedversionoftheiriPlayer,tobeknownasBBCiPlayer2,inordertointegratethedeliveryofon-demandTVandradio.Thisredesignreliedonextensiveuserengagementtogaugeuserperceptionsandattitudes–itwasimportanttocheckthatexistingusersofbothserviceswouldnotreactnegativelytothenewcombinedservice.ItwasalsoimportanttofindouthowperceptionsofthedesignofiPlayer2mightaffecttheaudience’sattitudestowardstheBBCasanorganisationandcontentproviderinthe21stcenturydigitalworld.
Separateresearchprojectswereconductedtoinvestigatedesignandusability.Twoqualitativemethodswerechosentoinvestigateaudienceperceptionsofthedesign–moderated‘audiencelabs’atwhichparticipantswereabletotryouttheiPlayer2andsharetheirresponsesandreactionswithotherparticipants,andin-depthindividualinterviews.ParticipantswererecruitedbasedontheirTVandradioconsumptionhabits(egTVonly),withanemphasisonensuringthatradiousersreflectedthefullspectrumofgenres(music,drama/entertainment,comedy,news,documentariesetc).
Whilstanumberof‘radioonly’userswereconcernedabouttheimplicationsthesedesignfeatureshadforthefunctionalityandservicethatradiolistenerswouldreceive,theresponsetothedesignwasgenerallypositive(egthegenerallookofthehomepagewaswellreceived:userslikedthe‘cinematic’feelofthewidescreen).
TheBBCisnowcollectingandanalysingdatafromserverlogfiles,BBCmessageboards,blogshostedelsewhereontheinternet,andisusingaudiencesurveystomonitorthesuccessoftheiPlayer2.Thisdatawillbeusedtoinformfurtherservicedevelopment.
1.2.5 Audienceresearchshouldbeseenasanongoingprocessratherthananoccasional,one-offevent.Evenasmallaudienceresearchprojectisworthwhile.Manytechniquescanbeimplementedquitecheaplyoradaptedtoashoestringbudget.Afrequentlyexpressedconcernisthatapieceofaudienceresearchwon’tbeanyusebecauseitisn’tgoodenough.Someinsightintoyouraudienceisvastlybetterthannoneatallandtheseconcernsshouldnotdeteryoufromundertakingaudienceanalysis.
1.2.6 AglossaryofthekeytermsusedintheGuideisprovidedatAnnex A.
Who is the Guide for?1.2.7 TheGuidehasbeenwrittenforpeopleintheUKpublicsectordeliveringonlinedigitalservices,whowishto
researchtheiraudiences.2
2 Although it is focused on online digital services (ie those accessed via the internet), it will also be useful for researching audiences reached via other channels.
The Guide to Researching AudiencesConciseedition
1. Introduction PAGE 7
1.2.8 Itisprimarilyaimedatnon-expertswhodonothavemarketresearchskillsthemselves,andwhodonothaveaccesstodedicatedmarketresearchteams.Itisthereforelikelytobemostusefulforpeopleinsmallorganisations/services,orthoseconductingsmallprojectsinlargerorganisations.Itmayalsobeusefulforpeopleprocuringserviceswhowanttoknowwhatguidelinestoprovide,andforthosesettingpolicies.
1.2.9 Audienceanalysisresearchshouldbedonewithcommitmentandsupportfromseniormanagement–withoutit,thefullbenefitsoftheresearcharelesslikelytoberealised.Shoulditberequired,thisGuideprovidesyouwithsomeofthebasicinformationneededtowriteaninformalbusinesscaseforaudienceresearchtofacilitatebuy-infromseniormanagement.
1.3 Using this concise edition1.3.1 Thebasicprinciplesarestructuredasstep-by-stepactivitiesinthelifecycleofaudienceresearchandare
representedinFigure1-1onthefollowingpage.3
1.3.2 Howyouchoosetousetheguidancetoplanandconductaprogrammeofaudienceresearchwillbeinfluencedbymanyfactors,includingthematurityoftheservice,thebudgetavailable,theamountofdigitalcontentheldandthebreadthofthetargetaudience.
1.4 Supporting documents1.4.1 TheGuideissupportedbyaseparatedocumentwhichprovidesreal-world case studiesdemonstrating
goodpracticeinaudienceresearch.4Thecasestudiesarefromboththepublicandprivatesectors,andboththeUKandabroad,andareusefultogainapracticalunderstandingofwhatcanbeachievedwhenaudienceanalysisisdonewell.
1.4.2 Anillustrative case studypresentingahypotheticalprogrammeofaudienceresearchtoputthisGuideincontextisavailableasaseparatedocument.5
1.4.3 Anumberofsector-specificandrole-specificbriefingpapersonaudienceresearcharealsoavailablefor:
�� Peoplesettingpolicies
�� Peoplemanagingprogrammesofresearchandservices
�� Peopleprocuringdigitalservicesintheculturalsector
�� Peopleprocuringdigitalservicesintheeducationsector
�� Peopleprocuringdigitalservicesinthehealthsector
�� Peopleexperimentingwithdigitalmedia
3 Note that although these activities are presented as separate activities in a linear process, in practice this is not likely to be the case – it is likely to be an iterative process.
4 The Guide to Researching Audiences: case studies, Curtis+Cartwright Consulting Ltd. v1.0. 5 December 2008.5 The Guide to Researching Audiences: illustrative case study. Curtis+Cartwright Consulting Ltd. v1.0. 5 December 2008.
The Guide to Researching AudiencesConciseedition
PAGE 8 1. Introduction
Figure 1‑1: an outline of the step‑by‑step activities, and structure and content of the Guide
SECTION 3
Planning audience research
SECTION 4
Collection of audience data
SECTION 5
Modelling the audience
SECTION 6
Making use of audience research
SECTION 2
Describing and defining the target
audience
This discusses the importance of knowing who you would like your audience to be, and provides guidance about how to describe and segment it. It will be especially relevant if you are currently designing a new service. However, it will also be relevant if you would like to start from the basics when planning research activities for services that are already in production and have an established audience.
This is the place to start if you have already defined your target audience and wish to start planning your audience research. It addresses the importance of setting clear objectives for audience research activities, and things to consider when planning activities to meet these objectives. It also sets out example questions about the audience which can drive your research.
This provides an overview of the various audience analysis methods available (eg focus groups, web analytics). It also provides some information about the pros and cons of conducting the research in-house and contracting the work out to a market research company to help you decide the best route for your organisation/service.
This focuses on how to analyse and interpret the data that has been collected to address the specific research questions and build up a better understanding of the audience and their interactions with their service.
This briefly discusses using the information to build the service around the needs, wants and expectations of the audience.
2. Describing and defining the target audience PAGE 9
2. Describing and defining the target audience
This chapter discusses the importance of the target audiences and looks at ways of defining the target audiences, and approaches to audience segmentation
Knowing who you would like your audience to be and what service you intend to offer is a vital part of business planning and service development.
Bothformalandinformalapproachestodefiningthetargetaudiencearepossible.
Segmentingtheaudienceintodifferentcategoriescanbehelpfulwhendescribingthetargetaudience.
2.1 The importance of the target audience2.1.1 Knowingwhoyouwouldlikeyouraudiencetobeandwhatservicesyouintendtoofferisvitalforany
service.Itmayappearobvious,butitshouldbepartofbusinessplanningasthereareimplicationsforfundingandpotentialroutestosustainabilityforadigitalservice.
2.1.2 Thecharacteristicsofthetargetaudienceshouldinfluencethedeliveryofaservice,thefunctionalityprovided,meansofaccessetc.Itisimportanttorecognisethataservicemayhavemorethanonedistinctaudience.Needsandexpectationsofthedifferentaudiencesarelikelytobeonlypartlyoverlappingandtheimplicationsthishasforservicedeliveryshouldbeconsidered.
Case study extract… oldbaileyonline
A good illustration of audience-focused service development
AprojectwasfundedtodigitisetheOld Bailey Proceedingstoprovideaccesstothetextandscannedimagesoftrialrecords.Theprojectteamwasdeterminedtocreatearesourcethatwouldreflecttheneedsofthedifferenttargetaudiencesegments,andconsultationswiththetargetaudienceprovidedevidenceforitsbidforfunding,guidedthedesignandfeaturesofthewebsite,andguidedthetestingoftheprototypewebsiteandlaterredesign.
ThetargetaudienceswereidentifiedonthebasisofthecontentoftheProceedingsandtheprojectteam’sprofessionalexperienceofworkingintheLondonArchives,forexample‘communitygroups’wereidentifiedasanimportantaudiencesegment.
Followingconsultationwiththetargetaudience,thedesignspecificationincludedfeaturesaimedatmeetingtheneedsofdifferentaudiencesegments.Forexample,itwasdiscoveredthatsomeofthetargetaudienceswerenotfamiliarwith18thcenturyhistory,andthatinthecommunitygroupssegmentschoolswerelikelytobekeyusers.Accordingly,alotofbackgroundmaterialwasprovidedanddedicated‘ForSchools’pageswereincludedtoprovideguidanceforteachersonwaysofusingtheProceedingsinteaching,andresourcestailoredtoschoolstudentsincludingatimeline,glossary,tasksandprimarysource.
Thisaudience-focusedapproachwassuccessful–thesizeoftheaudienceforoldbaileyonlinehasexceededexpectationsandtheserviceisclearlywidelyusedandappreciated.Forfurtherinformationontheoldbaileyonline,seethewebsite(www.oldbaileyonline.org).
The Guide to Researching AudiencesConciseedition
PAGE 10 2. Describing and defining the target audience
2.1.3 Understandingtheaudiencecanbevitaltogiveasenseoffocusandidentitytoaservice.Tryingtobeallthingstoallpotentialusersisdifficult.Differentgroupsofusersmayhaveincompatibleneeds(egexperiencedacademicresearcherswillprioritiseheavyweightsearchandanalysistoolswhenusinganarchive;leisureuserswillexpectaveryuser-friendlyinterfacethatletsthemquicklyexplorethearchiveandgetaflavourofthecontents).Thisdoesn’tmeanthatadigitisedarchiveshouldn’taimtoappealtobothaudiences,butunderstandingthatthesearedistinctaudienceswithdifferentneedsshouldinformthedesignanddevelopmentoftheservice.
2.2 The concept of the target audience in service development2.2.1 Servicedevelopmentmaybeeitherresource-driven(egbecausesomeonehasagreatidea)oraudience-
driven(egifanorganisationwantstoextenditsaudience).Bothapproachesarevalid,butdetailedservicedevelopmentshouldtakeintoaccounttheintendedaudiencefortheservice.Theservicemustappealtothem,sotheirhabits,needsandexpectationswillhaveanimpactonthenatureoftheserviceandthewaysinwhichitispromoted.
Case study extract… Digitisation of the John Johnson Collection
A good example of engaging the target audience to focus service development
TheJohnJohnsonCollection,housedattheUniversityofOxford,iswidelyrecognisedasoneofthemostimportantcollectionsofprintedephemeraintheworld.Tobroadenaccesstothisarrayofrareoruniquearchivalmaterial,aprojectwasconductedbytheBodleianLibrary(UniversityofOxford)andProQuesttoconserve,catalogueanddigitisemorethan65,000itemsoftheCollection.
TheprojectteamconductedsomeinitialaudienceresearchtosupporttheirsuccessfulbidforfundingthroughtheJISCDigitisationProgramme.ThetargetaudienceforthedigitisedCollectionwasdefinedandtheprojectteamdemonstratedtherelevanceoftheCollectiontotheseaudiences.Forexample,lettersofsupportweresolicitedfrompotentialusersinUKuniversitiesandinternationalacademicswithaninterestinephemera,andtheprojectteaminvestigatedwhichhighereducationcoursemodulesmightusethisephemera.
Toensurethatthedigitisationwouldrealisemaximumbenefitforusers,audienceresearchwasanimportantpartofthedesign,developmentandtestingoftheonlinedigitisedCollection.Forinstance,toassistindecidingwhichephemeratodigitise,statisticsforphysicaluseofthevariouscomponentsoftheCollectionoverthepreviousthreeyearswerecompiled.
SomepartsoftheCollectionhavenowbeendigitisedandarefreelyavailabletoallUKuniversities,furthereducationinstitutions,schoolsandpubliclibraries.Uptakeanduseofthesecollectionsarebeingmonitoredusingfocusgroupsandsurveys(usersatisfaction),webstatistics(useractivity),andassessmentofuser-generatedcontentandtagging(userengagement).Thisresearchformsanimportantpartoftheevaluationstrategyforthisproject.
ForfurtherinformationontheJohnJohnsonCollection,seethewebsite(www.bodley.ox.ac.uk/johnson).SomeareasofthewebsitearerestrictedtomembersoftheHEcommunity.
2.2.2 Understandingthetargetaudiencewillbecriticaltothesuccessoftheservice.Audienceresearchwillfocusonfindingoutthesortsofresourcesandservicesthatappealtothetargetaudience,howtheywouldprefertoaccesstheservice,andthetechnologiestheyuse.Backgroundknowledgeaboutattitudes,lifestyle,socialinteractionsetcisalsolikelytobeusefuliftheresourceistobeprimarilyforleisureuseratherthanforformallearning.
2.3 Defining the target audience2.3.1 Definingyourtargetaudiencedoesn’tneedtobeadifficulttechnicalexercise.Agoodstartingpointmay
betopictureanidealaudiencemember–oratypical,actualaudiencemember.Howwouldyoudescribethisperson?Thedescriptorsyouusearegoodstartingpointsforthinkingaboutcategoriesforaudiencesegmentation.
The Guide to Researching AudiencesConciseedition
2. Describing and defining the target audience PAGE 11
2.3.2 Amoreformalapproachtodefiningthetargetaudiencewouldbetodevelopaseriesofuserprofilesorfullyworkedpersonasasanaidtoservicedesignanddevelopment(seesub-section).Negativeuserdescriptions,profilesorpersonascanalsobehelpful:thekind(s)ofpeopleyourserviceisnotintendedfor.
2.4 Describing the target audience2.4.1 Thecharacteristicsofthetargetaudienceshouldinfluencethedeliveryofaservice,thefunctionality
provided,meansofaccessetc.Thefeaturesincludedinadescriptionofthetargetaudiencewillvaryaccordingtotheservicebutmayincluderesidentialaddress,occupation,gender,interests,technologicalexpertiseetc.Atargetaudiencemaybeveryinclusive(egallthepeopleinManchester)orhighlyspecific(egqualifiedmedicalprofessionalswitharesearchinterestinOsgood-Schlatter’sdisease).
Audience segmentation2.4.2 Audiencesareoftensegmentedintodifferentcategories.Publisheddata(egtelevisionviewingfigures,
newspaperreadership)oftensegmentstheaudienceonthebasisofdemographiccategories.Academiclibrariesaremorelikelytocategoriseusersaccordingtotheirrolewithinacademia.Themostappropriatewaytodefineatargetaudienceandsegmenttheactualaudiencewilldependontheserviceandtheobjectivesoftheaudienceanalysiswork,andseveraldifferentapproachestosegmentationarepossible.
PAGE 12 3. Planning audience research
3. Planning audience research
This chapter looks at defining research objectives, selecting and combining appropriate methods, researching non-usage and recruiting subjects
Research need not be perfect to be useful – the key to good audience analysis is setting out clear objectives and planning appropriate research to achieve the objectives. Engaging non-users is important but presents special problems.
Objectivescanbebroadlydividedintotwocategories:assessmentofreachanduptake,andassessmentofqualityandimpact.
Startingpointsforplanningresearchinclude:
�� Definingservicedevelopmentobjectives.
�� Definingquestionsaboutthenatureandbehaviouroftheaudienceandnon-users.
Differentaudienceresearchtechniquesaresuitedtodifferentobjectivesandmethodscanbecombined.
3.1 Introduction3.1.1 Planningaudienceresearchshouldstartwiththebasicquestion:What do you want to know about
your audience?Consideringwhy the knowledge is neededandhow it will be put to useisalsoimportant.Understandingtheaudienceisnotanendinitself,butanintegralpartofdevelopingandmaintainingasuccessfulandsustainableservice.Understandingthedriversfor,andusesof,audienceanalysisworkwillhelptoensurethatitisalignedwithwiderservicevisionandobjectives,andmayhelptoobtainbuy-infromseniormanagement.
3.2 Objectives of audience analysis3.2.1 Thekeytogoodaudienceanalysisissettingoutclearobjectivesandplanningappropriateresearch
activitiestomeettheseobjectives.Thereislittlepointinspendingtime,moneyandstaffeffortcollectingdataunlessitisgoingtobeuseful.
Case study extract… London Museums Hub
A good example of setting clear objectives to focus research activities
TheLondonMuseumsHubisapartnershipoffourofthe‘bigplayer’non-nationalmuseumsinLondon:theMuseumofLondon(leadpartner),theLondonTransportMuseum,theGeffryeMuseumandtheHornimanMuseum.TheHubconductedaresearchprojecttoinvestigatethepotentialforajoined-upapproachtoonlineaudiences.
The Guide to Researching AudiencesConciseedition
3. Planning audience research PAGE 13
Theprojecthadawell-definedbrieffortheresearch,whichstatedthat‘theHublacksacoordinatedstrategicapproachtoqualitativeonlineaudienceprofilingandresearch…weaimtobuildamorecompleteunderstandingoftheHub’sonlineaudiences,andusethisknowledgetotakeamorestrategicuser-focusedapproachtoonlinedelivery’.Thisbriefwasunderpinnedbyspecificresearchquestionsabouttheironlineaudiences:Whoareouraudiences?Whydotheyuseourwebsites?Whataretheirexpectationsofthewebsites?Towhatextentaretheseexpectationscurrentlybeingmet?
Theprojectusedamulti-strandedapproachtoaudienceresearch(egcombiningonlinequestionnaires,face-to-faceinterviews,focusgroupsandwebstatistics)togatherinformationtoanswerthesequestions.Thefindingsfromthisprojectwerethenusedtotakeamorestrategicuser-focusedapproachtoonlinedelivery.Forexample,theresearchconfirmedthattheoverwhelmingmajorityofwebsiteusagewasforplanningavisittoamuseum.However,thewebsitewasnotakeyfactorinthedecisiontovisit–thereputationofthemuseuminaparticularfield,locationetcwerethemainfactors.
Followingthisexploratoryresearch,concretestepshavebeenagreedthatwillenabletheHubtomovetowardsacommonapproachtocollectingdataonhowonlineservicesarereceivedbytheaudienceandthebroaderimpactthatonlineserviceshaveonorganisationsasawhole.FormoreinformationontheLondonMuseumsHub,seetheMLALondonwebsite(www.mlalondon.org.uk).
3.2.2 Audienceanalysiscanbedividedintotwobroad,interdependentcategories:assessmentofreach and uptake,andassessmentofquality and impact.Inplanningaudienceresearchitishelpfultodecidehowimportanteachaspectoftheanalysisis.
3.2.3 Researchobjectivesmustbesetaccordingtoindividualserviceprioritiesandcontext.Startingpointsforidentifyingbroadaimsandframingobjectivesincludeservice development objectives(eghowcanweextendthereachofourservice?)andquestions about the audience and non-users(egwhoareouraudience?howdotheyuseourservice?).SeeFigure3-1andFigure3-2forexamples.
Figure 3‑1: understanding non‑users
Usage
Ex-users
Never
NON-USERS
WHY are they not using our service?
Awareness
Not aware
Aware
Reason
Not interested
Happy with alternative
Too slow
WHY do they no longer use our
service?
WHO are they?Which segments of the target
audience are under-represented?
Target audience – existing audience = non-users
WHAT did they think of our
service?
HOW could we make them more
aware?
HAVE they ever used our service?
WHY do they not use our service?
Reason
Not interested
Happy with alternative
Poor public image
WHY are the alternatives preferred?
Reason
Better quality
Better coverage
Easier to access
Audience satisfaction
Aware but cannot accessWHY can they not
access it?
Reason
Technological
Authentication required
Disabilities
Marketing
The Guide to Researching AudiencesConciseedition
PAGE 14 3. Planning audience research
Figure 3‑2: questions about the audience
WHO
are
they
?
EXIS
TIN
G
AU
DIE
NC
E
WH
Y do
they
use
ou
r ser
vice
?
WH
O is
not
usi
ng
our s
ervi
ce?
DOES
the
serv
ice
mee
t the
ir ne
eds
and
expe
ctat
ions
?
Acce
ss
Hom
ePu
blic
Mob
ile
Freq
uenc
y
Regu
lar
Occ
asio
nal
One
-off
Com
petit
ors
Publ
ic
Priv
ate
Inte
rnat
iona
l
Wha
t for
Con
tact
sCo
llect
ions
How
to fi
nd u
s
Firs
t use
Sear
ch e
ngin
eRe
com
men
datio
n
Ref
errin
g si
te
Subs
eque
nt
use
Dire
ct
Sear
ch e
ngin
e
Ref
errin
g si
teD
irect
Rea
son
Qua
lity
Auth
ority
Com
preh
ensi
ve
Clar
ityNo
alte
rnat
ives
Alte
rnat
ives
Dig
ital
Non-
digi
tal
Dem
ogra
phic
Func
tion
Goa
lTe
chno
logy
Do
mai
n
Age
Gen
der
Soci
oeco
nom
ic
cate
gory
Loca
tion
Prof
essi
onal
Rese
arch
er
Acci
dent
al/
impu
lse
Leis
ure
Brow
sing
Defin
ed
obje
ctiv
eSy
stem
atic
ex
plor
er
Expe
rtC
onfid
ent
Nov
ice
Expe
rtSo
me
know
ledg
eNo
kno
wle
dge
HO
W d
o th
ey u
se
our s
ervi
ce?H
OW
do
they
re
ach
our
serv
ice?
Sepa
rate
dia
gram
Audi
ence
sat
isfa
ctio
n
WH
AT a
re th
e lo
nger
term
effe
cts
of th
e se
rvic
e?
Impa
ct a
naly
sis
Qua
lity
Rel
iabi
lity
Con
veni
ence
Opi
nion
Impo
rtan
ce
The Guide to Researching AudiencesConciseedition
3. Planning audience research PAGE 15
3.3 Choosing the methodology
Different approaches to audience analysis3.3.1 Audienceanalysismethodscanbecategorisedindifferentways,butthedistinctionbetweenquantitative
andqualitativemethodsisanimportantone.Quantitativemethods(egsurveys,webanalytics)areusuallymoreappropriatewhenstatisticallyvalidconclusionsneedtobedrawnaboutthesize,compositionandotherpropertiesofanaudience.Qualitativetechniques(egfocusgroups,one-to-oneinterviews)maybemoreappropriateforexploringattitudesandmotivationsindepth,discoveringpatternsofuserbehaviouranddevelopingarelationshipwithaudiencemembers.
3.3.2 Ethnographicorobservationaltechniques(egdiarystudies,accompaniedbrowsing)areincreasinglyusedforstudyingusabilityandpatternsofserviceuse.Ethnographicdataisoftenverydetailedandmayincludecomputercollecteddatawhichisamenabletostatisticalanalysisaswellasaqualitativerecordofuserbehaviour.
Representative data3.3.3 Samplesaresaidtoberepresentativewhenthecompositionofthesampleaudience–withrespecttoall
relevantattributes–isthesameasthatoftheaudienceasawhole.Insomeinstancesitisimportanttoobtaindatafromarepresentativesampleoftheaudience(egtoassessthelevelofuptake,toprovideevidenceforattainmentofkeyperformanceindicators).However,representativedataisnotalwaysvital,particularlywhenresearchispreliminaryorexploratory.
Selecting appropriate methods3.3.4 Thekindofinformationyouwantabouttheaudienceshouldhelptodeterminethemethodsyouuse,but
pragmaticconsiderations(egtime,money,difficultyofrecruitingsubjects,staffavailabilityandexpertiserequired)willinevitablyplayapart.Itiscommontouseacombinationofmethodsinaudienceanalysisprojects,forexamplequantitativesurveysarefrequentlycombinedwithfocusgrouporinterviewwork.Table3-1providesaguidetothemethodsmostlikelytobeusefulinresearchingdifferentaspectsoftheaudience.
Case study extract… Europeana project
A good example of selecting appropriate research methods
Europeanaisa2-yearprojecttodesignandbuildawebportaltomakeavailabletousersawealthofdigitalobjects(egphotographicimages,sounds,paintings,maps,manuscripts,books)frommuseums,librariesandarchivesacrossdifferentEuropeancountries.
A‘maquette’–ananimateddemonstrationwebsite–wasinitiallydevelopedbasedonuserrequirementsanddeterminedinconsultationwithcontentprovidersandtechnicaldevelopers.Anaudienceresearchprojectwasconductedtotestwhetherthismaquetteliveduptotheexpectationsandwishesofrealend-usersintermsoffunctionality,theinterfaceandnavigation.ThiswasanimportantsteptowardsdevelopingafullyfunctionalprototypeEuropeanasite(launchedinNovember2008).
Althoughtheresponsetothedemonstrationsitewasgenerallypositive,anumberofcommonconcernsemerged.Forexample,itwasfoundthatthehomepagewasnotclearenoughanddidnotgiveagoodintroductiontothesite;thesearchresultspagelackedorderinresults,levelofdetailandpresentation;andtherewasconfusionabouthowthesitecouldbebrowsedandsearched.Theseresultswerefedbacktothetechnicaldevelopmentteamworkingonthefullyfunctionalprototype.Forfurtherinformation,pleaseseetheEuropeanawebsite(www.europeana.eu).
The Guide to Researching AudiencesConciseedition
PAGE 16 3. Planning audience research
Table 3‑1: selecting appropriate methods (Key: ‑ unlikely to be suitable; * may be suitable, but of limited use; ** often suitable)
Asp
ect o
f au
dien
ce
Qua
ntit
ativ
e-su
rvey
Web
sta
tist
ics
Ethn
ogra
phic
st
udie
s
Focu
s gr
oups
Indi
vidu
al
inte
rvie
w
Info
rmal
Oth
er
appr
oach
es
Size ** ** - - - - -
Composition ** * - - - * -
Motivations ** * - ** ** * -
Attitudes ** * - ** ** - -
Satisfaction ** * - ** ** - -
Prevalence of non-usage ** - - * - -
Reasons for non-usage - - - ** ** - -
Patterns of usage - ** ** * * - -
Tasks service is used for ** ** **usertracking
** ** - -
Lifestyle * - ** * ** - -
Service usability * * ** * * - Heuristicevaluation6
User requirements - - ** ** ** * -
3.4 Researching non-usage6
3.4.1 Understandingnon-useofaserviceisimportant:publiclyfundedservicesmayhavearesponsibilitytoserviceaparticularaudience(egseniorcitizens,ethnicminoritycommunities).
3.4.2 Engagingwithnon-usersmaybedifficult,time-consumingandpotentiallyexpensive,butitshouldbeviewedasanecessarypartofaudiencedevelopmentwork.Buildingrelationshipswithnon-userswillpaydividends:7
�� Theviewsofnon-usersarenotrepresentedinusersurveysandthedigitalserviceisnotmeetingtheirneeds;thismaybeanimportantfailing
�� Non-userscanprovidevaluableinsightsforservicedevelopment
�� Engagingnon-userscanhelpdemonstrateacommitmenttothataudiencesegmentandincreaseknowledgeaboutmarketingandcommunicationstrategiesthatwillbeeffectiveforthatsegment
3.4.3 Sub-section3.5ofthefull-versionoftheGuideprovidesmoreinformationontheimportanceof,difficultieswith,andpossiblestrategiesforresearchingnon-users.
3.5 Recruiting subjects for audience research3.5.1 Recruitingsubjectsisaperennialproblemandtheeasiesttechniquemaynotalwaysbethebest,andmay
leadtoanunrepresentative,biasedsample.Incentivescanbeusedtoaidrecruitment,butcareneedstobetakentoensurethattheincentiveisappropriatetotheaudienceyouseektorecruit.
6 Heuristic evaluation is where expert evaluators assess the service’s user interface against recognised usability criteria. This method falls outside the scope of this guide but is described briefly in the full-version of the Guide because of its utility in the early stages of service development.
7 Responses to a survey may not provide very much information. Engaging more deeply with non-users through focus groups, discussion fora and interviewing will be necessary to build a richer, more meaningful understanding of non-usage.
The Guide to Researching AudiencesConciseedition
3. Planning audience research PAGE 17
3.5.2 Recruitmentwillbemoredifficultifabigtimecommitmentisrequiredortheresearchisgoingtobeinconvenientforsubjects.Itisalsoimportanttobearinmindhowyourinformationcollectiontechniqueswillinfluenceresponseratesandtherobustnessofthedatacollected.
3.5.3 Userregistrationcanalsobeexploitedeasilybydigitalservicesforaudienceanalysis.8Thisallowstheservicetoassociateusercharacteristicswithuserbehaviour.Registeredusersmaybemorefrequent,moreloyalormorecommittedtotheserviceandthereforemorewillingtoparticipateinresearch.
3.5.4 Anextensionofthisideaistosetupresearchpanels–asampleofaudiencememberswhohavebeenrecruitedspecificallyforthepurposeofregularly(ieonanongoingbasis)respondingtoqualitativeand/orquantitativesurveys.
3.6 Other sources of audience data3.6.1 Don’tneglectthepossibilityofincreasingyourunderstandingofyouraudiencebylookingatresearchdone
byotherserviceprovidersontheiraudienceorbyusingdatafrompopulationsurveyscarriedoutbymarketresearchorganisations,governmentagenciesoracademicbodiesegtheOxfordInternetInstitutebiannualsurveyofinternetusage.9
3.6.2 Inaddition,exploitinginformalknowledgeaboutyouraudience(egobservationsbystaffwhointeractdirectlywiththeaudience,technicalsupportemails)isasmartthingtodoandcanbeparticularlyvaluableifresourcesforaudienceanalysisarelimited.
8 Subject to the provisions of the data protection legislation. 9 Oxford Internet Institute surveys, www.ox.ac.uk/microsites/oxis
PAGE 18 4. Collecting audience data
4. Collecting audience dataThis chapter provides an overview of different methods for collecting audience data and considers outsourcing audience research
The pros and cons of conducing research in-house and commissioning an independent market research agency to undertake it on your behalf are discussed.
Datacollectionmethodscoveredinclude:
�� Quantitativesurveys
�� Interviews
�� Focusgroups
�� Webstatistics
�� Ethnographic(observational)techniques
4.1 Undertaking research: in-house or third party?4.1.1 Technologicaladvancesmakeitincreasinglytemptingtoconductresearchona‘DIY’basis,ratherthan
commissioninganindependentmarketresearchagencytoundertakeitonyourbehalf.However,itisworthconsideringanumberoffactorsbeforedecidingifthein-houseorcommissionedresearchrouteismostappropriate.
Table 4‑1: critique of approaches to undertaking research
Approach Pros Cons
In-house research
�� Canoftenbetheleastexpensiveoption–especiallyifexpertin-houseresourcesareavailable.�� Knowledgeoftheaudience:expertknowledgeand/orexperiencecanhelpinplanningandconductingresearch.�� Greaterlevelofcontrol:amarketresearchagencymighthavedifferentideasabouthowtoresearchanaudience.
�� Lackofexperienceinconductingresearchcanleadtopoorresearchdesignandotherproblemsthatanexperiencedresearchagencywouldhaveanticipatedandplannedfor.�� Questionnairedesignissurprisinglydifficulttogetright.Poorlydesignedquestionsresultinambiguousormisleadingdata.Experienceofquestionnairedesignisimportant.�� Similarly,unlessknowledgeofstatisticalandsamplingtechniquesisavailablein-house,mistakescanbemade–bothattheplanningandattheanalysisstages.
Commission market research company
�� Ifcarefullyselected,aresearchagencywillbringexpertiseandexperiencetobear,resultinginamoreefficient(possiblyevenmorecost-effective)surveyandgreaterclarityofunderstanding.�� Theinvolvementofarespected,independentthird-partybringscredibilitytotheresearch.�� Respondentsaregenerallymorelikelytorespondopenlyandhonestlytoathirdparty.�� Marketresearchagencieshaveaccesstothespecialistresourcesrequiredtoconductresearch(egrecruitersandsurveytools).
�� Commissionedresearchcanbeexpensive.�� Theprocessofidentifyingpotentialresearchagencies,writingaresearchbriefandbriefingtheselectedagencytakestime.�� Evenmarketresearchagenciesexperiencedinyoursectorwilllackyourdetailedknowledge.Bepreparedtospendtimeensuringthattheagencyunderstandsthenatureofyouraudienceandwhatyouaretryingtofindoutaboutit.
The Guide to Researching AudiencesConciseedition
4. Collecting audience data PAGE 19
4.1.2 Ifyoudodecidetouseamarketresearchcompany,‘TheResearchBuyer’sGuide’10publishedbytheMarketResearchSocietyhassomeusefulinformationbothonthetypeofmarketresearchcompanytochoosefromandalsoonthequestionstobearinmindwhenselectingacompany.Itincludesadviceonwhethertosub-contractsomeelementsofaresearchprojecttoconsultants(egquantitativeinterviewingorrecruitmentoffocusgroups),orbuyinthefullservice(whereprofessionalswillhelptodesigntheresearchandcollect,analyseandevaluatedata).
Case study extract… Teens Music and Technology: BBC research
An example of using a research agency and managing the relationship
TheBBCconductedaprojecttoinvestigatetherolethatmusicplaysinthelivesofyoungpeopleandtheextenttowhichyoungpeopleusetechnologytoaccessandperhapssharemusicwiththeirfriends.ThehopewasthataninsightintothebehavioursofthisagegroupwouldleadtonewideasabouthowtheBBCshouldbedevelopingmobileproductsandservicesforthisaudiencegroup.
AresearchagencywascommissionedbytheBBCtoundertakethiswork.TheBBChadveryspecificrequirementsofthework–itwasimportantthattheprojecthadmusicatitscentreratherthanusingtechnologyasthefocus,theresearchmust‘getbeneaththeskin’ofyoungpeopleandofferarealinsightintothepartmusicplaysintheirlives,andtheoutputneededtobea‘designtool’toinspireandstimulatediscussionandsparkideas.Becauseofthesespecificrequirements,itwasnecessaryfortheBBCtoworkverycloselywiththeresearchagencytoensurethattheresearchmettheirrequirements.Acontinualdialoguewasessential,beginningatthepointthattheideafortheresearchwasconceivedandcontinuingrightthroughtofinalreporting.
Anethnographicapproachwasadoptedasthishasthepotentialtoofferrichdetailanddeepinsightsintothebehaviours,practicesandeverydaylivesofpeople.Theresearchagencydesignedthefinalreporttobevisuallystimulating,fullofimagesofthedifferentlifestylesandinterestsofeachoftheparticipatingteenagersandthegroupstheybelongto,andmostimportantly,adocumentthatinvitedreaderstoimmersethemselvesintheworldoftheseteenagers.Thereportcentredonthepersonasoffourfriendshipgroups,representingdifferentinterests,lifestylesandimportantly,tastesinmusic.Becausetheformatofthereportwascriticaltokick-startdiscussionamongstdesignersandproducers,thisledtocloseconsultationwiththeresearchagencyoniterativedraftsofthereporttoensureitlentitselftothispurpose.
Todate,theoutputoftheresearchhasbeenusedinaninformalway.However,itisplannedthataseriesofcreativeworkshopswillbeorganised,challengingdesignerstoreacttothefindingswithnewideas,solutionstosomeoftheissuesthatyoungpeoplehaveidentified,andgenerallyengageinconstructivedebateaboutthewayforwardfortheBBCinthisarea.
4.2 Overview of data collection methods4.2.1 Informationandacritiqueondifferentmethodsofcollectingaudiencedataissummarisedinthefollowing
table.Morepracticalguidanceforusingthesemethodsisprovidedinthefull-versionoftheGuide.Thesetofmethodscoveredisnotintendedtobedefinitiveanddetailedinformationonhowtoimplementeachmethodisnotprovided.
4.2.2 Mostaudienceanalysismethodscanbeusedoradaptedsuccessfullybynon-specialists;however,therearesomecircumstancesunderwhichitmaybeappropriatetoconsideroutsourcingoneormoreelementsofanaudienceresearchproject(seeabove,section4.1).
10 The Research Buyer’s Guide, UK and Ireland (2007); London: Market Research Society.
The Guide to Researching AudiencesConciseedition
PAGE 20 4. Collecting audience data
Case study extract… Arthritis Source
A good example of using complementary research methods to gather information
ArthritisSourceisaninformationalwebsite,developedandhostedbytheUniversityofWashington,providingaccurateandauthoritativeinformationaboutarthritisandrelatedconditions.Afterbeinginserviceforfouryears,aprojectwasconductedtocharacterisetheaudienceoftheserviceandtoredesignthewebsitetomeettheirneeds.
TheresearchprojectusedthreeresearchmethodstogathercomplementaryinformationabouttheaudienceforArthritisSource.Logfileanalysiswasusedtoexplorepatternsofusage–whichpageswerebeingaccessedmostfrequently,andwhenweretheybeingaccessed?Anonlinesurveyofuserswasconductedtocollectabroadrangeofdataaboutusers–whytheyusedthewebsite,whattheirrolewaswithrespecttoarthritis(egmedicalstudent),demographicsandITexpertise.Telephoneinterviewswerethenconductedwithselectedrespondentsoftheonlinesurveytogathermoredetailedinformationaboutusers’knowledgeofarthritisandtheirinteractionswithArthritisSource.
InformationabouttheiraudiencesandtheuseofresourceswasusedbytheteamattheUniversityofWashingtontodiscoverandprioritiseredesignobjectives.Forexample,theresearchrevealedthatnearly20%ofuserswerebasedoutsideNorthAmerica,sobasicchangesweremadetointernationalisethecontent,egbothgenericandbrandnamesfordrugswereincluded.Furthermore,somecommonmisconceptionsaboutarthritisamongthewebsiteaudiencewererevealed,andanonlinequizaboutcommonmythsandmisconceptionswasdesignedtohelpovercomethesemisconceptions.
AudienceanalysisisnowroutineforArthritisSourceandthedesignistweakedwhenthereisevidenceofachangeinuserneeds,ortotakeadvantageofchangesinthetechnologicallandscape,egthesitenowincludesmorevideoclipsandpodcastsashigh-speedinternetaccesshasbecomemorewidelyavailable.Thewebsite’scommitmenttoassessingandrespondingtoaudienceneedsissignalledtotheaudience:webstatisticsarepubliclyavailableandveryshortonlinequizzeshavebeenplacedonmanyofthepagesaskinguserstoratetheirusefulness.
Forfurtherinformation,seetheArthritisSourcewebsite(www.orthop.washington.edu/uw/tabID__3370/Default.aspx).
Table 4‑2 (part 1 of 2): overview of data collection methods
Method Information
Focus groups Description:Focusgroupscanbeorganisedinmanywaysandinmanyformats.However,allformatsarebasedonthepremiseofasmallgroupofrespondentsdiscussingissuesandbeingguidedbyanexperiencedfacilitator(sometimescalledamoderator).
Uses: Forexploringattitudesandmotivationsandforgeneratingnewideas.Focusgroupsareoftenusedastheprecursortosomeformofquantitativesurveyresearch.
Pros: Usefulforcreativediscussionandnewideas,relativelyshorttimescales,well-suitedtointroducingandgainingreactionstostimulusmaterials.
Cons:Reliesonsufficientnumbersofaudiencebeingclusteredinageographicalarea,respondentsmightnotbepreparedtodiscusspersonally,commerciallyoracademicallysensitivesubjectswithothers,requiresexperiencedfacilitators.
One-to-one in-depth interviews
Description:In-depthinterviewsconductedonaone-to-onebasis,bytelephoneorinperson.
Uses:Forunderstandingtherangeofattitudesandmotivationswhichmightexistwithinanaudience.
Pros:Canbeusedtocollectqualitativeorquantitativedata,goodqualitydata,morein-depthinformationcanbecollected,stimulusmaterialcanbeshown.
Cons:Relativelyexpensive,requireahighlevelofrespondentcooperationandcommitment,time-consuming.
The Guide to Researching AudiencesConciseedition
4. Collecting audience data PAGE 21
Method Information
Quantitative surveys
Description:Quantitativesurveysinvolveusingasampleofanaudience/populationasthebasisforproducingastatisticallyreliablepictureofthewhole.Includespostal,email,telephone,andinternetsurveys.
Uses: Forprovidingquantifiableinformationaboutanaudience.Forexample,itsprofileanditspatternsofusageandbehaviour.
Pros: Varydependingonmethod.Ingeneral:cost-effective,offeralargeproportionofrelevantaudienceachancetorespond,lowlevelofcommitmentfromrespondents,internetandemailsurveyscanprovideveryimmediatefeedback.
Cons:Varydependingonmethod.Ingeneral:sampleisoftenself-selecting,noguaranteesofresponserates,methodshouldbechosencarefully(ienotallpotentialrespondentswillhaveinternetaccessorwillbecomfortableusingit),interpretationofquestionscanbesubjective.
‘Traditional’ web statistics11
Description:Webserversgeneratelogfileswhentheyservearequest,whichistypicallydirectlyfroma‘useragent’–usuallyabrowsersuchasInternetExplorerorFirefox.Theselogfilesrecordtherequestwhichwasmade,thestatusoftheresponse(success,failureetc)andsomebasicinformationabouttheuseragent.Thismethodconsidersrelativelysimplewaysinwhichtheselogfilescanbeprocessedtogeneratequantitativeinformation(webstatistics)regardingtheuseoftheservice.
Uses:Forunderstanding,onaggregate,thewaysinwhichanonlineresource(orcollectionofresources)isused,forexampletherelativeusageofdifferentpartsofasite,andhowtheusagevariesovertime.
Pros:Lowcostintimeandeffort,informationcanbeavailableimmediately,providesan‘at-a-glance’viewofhowbusyasiteis,andwhichsectionsofthesitearemostused.
Cons:Limitedinsightintouserbehaviour,verycrudemeasuresofdemographicofaudience,temptingtoinfertoomuchfromtheinformationwhichisavailable.
Web analytics Description:Webanalyticsisamoreadvancedmethodforinvestigatingtheactionsofusers.Webanalyticswillincludetheinformationavailablefromtraditionalwebstatistics,butwhereaswebstatisticsisfocusedontheresource,webanalyticsisfocusedonindividualusers.Theactionsoftheseuserscanbecombinedtoprovidebulkinformationonaservice,butatafarhigherlevelofdetailthanthatavailablefromtraditionalwebstatistics.
Uses:Webanalyticscanbeparticularlyhelpfultosupportsitedesigndecisions,byunderstandingthedecisionsthatuserstakewhenusingaresource.
Pros: Informationcanbeavailableimmediately,providesextensiveinformationonuserbehaviour,canbeusedtosegmentusersbasedonarangeofbehavioursandpossiblyondemographics.
Cons:Commercialwebanalyticspackagesareexpensive,settingupreportingistime-consuminganddemandsgoodbusinessandtechnicalknowledge,insightintouserbehaviourfromwebstatisticsmustbevalidated.
User observation techniques (ethnography)
Description:Ethnographyisatermusedtoencompassawiderangeoftechniquesandapproaches.Atitscoreisadesiretounderstandhumanculturebyobservingbehaviour,butthisbroadprincipleisincreasinglyusedincommercial,publicandsocialresearchcontexts.Itincludesusertrackingstudies(egshadowing,diarystudies)andusetesting(egtask-focusedactivitysessions).
Uses: Ethnographyandobservationaltechniquescanofferaninsightintohowaudiencesuseandinteractwithservicesandresources.Fordigitalservicesverydetaileddataaboutusagecanoftenbecapturedunobtrusivelybycomputerloggingtechniques.
Pros: Varydependingonmethod.Example–usetestingisanexcellentwayofidentifyingproblemswithusability.
Cons:Varydependingonmethod.Generallytime-consumingandresourceintensive.
11
11 In this document, we have divided the measurement of online resources into ‘traditional’ web statistics and web analytics. This is a somewhat artificial boundary, but one that is necessary to consider the full range of opportunities. Traditional web statistics consider bulk information on a resource (ie looks at usage from the service’s perspective), whereas web analytics considers the actions and pathways chosen by individual users (so looks at usage from a user’s perspective).
PAGE 22 5. Modelling the audience
5. Modelling the audienceThis chapter deals with analysis and interpretation of audience data to address specific objectives and improve understanding of the audience
Datafromanaudienceanalysisprojectshouldimproveunderstandingoftheaudienceinseveralways:
�� Answerstospecificresearchquestions–orfurtherevidence
�� Newknowledgeabouttheaudienceanditsinteractionswiththeservice
�� Objectivesforfutureaudienceresearch
Topicscoveredinthischapterincludeapproachestoanalysingquantitativeandqualitativedata,potentialproblemswithdatainterpretationanddevelopingpersonasoruserprofilesforservicedevelopment.
5.1 Introduction5.1.1 Thissectiondealswiththeissuesinvolvedinanalysingandinterpretingthedatatoaddressthespecific
researchquestionsandbuildabetterunderstandingoftheaudienceanditsinteractionswiththeservice.
5.1.2 Theaudiencedatamustbeanalysedappropriately:thetechniquesusedwilldependonthetypeofdataandtheaimsoftheresearch.Thedatashouldprovideevidencethatcanbeinterpretedtodrawconclusionsrelevanttothespecificobjectivesoftheresearch.Itisimportanttobesurethatthedatareallydoprovideevidenceforanystatementsyoumakeandthatyouhaveconsideredallthepossibleinterpretationsofthedata.Fullanalysisandinterpretationofthedatashouldalsoprovidewiderinsightsintotheaudience,anditsbehaviourandrelationshipwiththeservice.
Case study extract… JISC national e-books observatory project
A good example of analysing research data to understand user behaviour
TheJISCnationale-booksobservatoryprojectisa2-yearprojectledbyJISCCollections,whichisinvestigatinguserbehavioursandthedemand,usage,deliveryandbusinessmodelsforcoursetexte-books.Theoutputsofthisresearchwillsupportthefutureprocurementofe-booksinthesector.
Thisprojectisusinganumberofcomplementaryresearchmethodsincludingwebanalytics,userinterviewsandsurveystounderstandhowusersinteractwithelectronicbooks.Forexample,theprojectisusingweblogdatatoprovidequantitativemeasuresofusageandsuggestanumberofpossibleinterpretationsofuserbehaviour.However,thisdatamustbetreatedwithcare(egdoesalownumberofsearchesindicatethatauserhasquicklyfoundtheresourcetheywanted,orthattheyhavegivenupafteroneattempt?).Moredetailedanalysisofusagedata(iewebanalytics)maybeinformative,butinterviewsandsurveysarebeingconductedtohelpvalidateassumptionsandinferencesfromtheweblogdata,andtodevelopanunderstandingofuserbehaviour.
Forfurtherinformation,pleaseseetheprojectwebsite(www.jisc-collections.ac.uk/catalogue/jiscebooksproject).
The Guide to Researching AudiencesConciseedition
5. Modelling the audience PAGE 23
5.1.3 Analysingthedatafromaudienceresearchwillbeeasieriftheresearchwaswellplannedandconducted.Manyproblemsinanalysisandinterpretationcanbeavoidedifdatacollectionproceduresareselectedandimplementedwiththeanalysisinmind.Sometimesatrade-offneedstobemadebetweensimplicity,convenienceandreliabilityofrecording,andconvenienceandpowerforanalysis.Paper-baseddatacollectionmightbeeasiesttoadminister,butdatawillhavetobecodedandenteredontoacomputerbeforeanalysis.
5.1.4 Analysisofthedatafromanaudienceresearchprojectshouldimproveunderstandingoftheaudienceinseveralways:
�� Answerstothespecificresearchquestions–oratleastrelevantevidence
�� Newknowledgeabouttheaudiencemoregenerally
�� Furtherquestionsabouttheaudiencetobeaddressedinfuturework
5.2 Analysing quantitative data5.2.1 Quantitativedatacanbeanalysedstatisticallytoexplorerelationshipsbetweenvariables.Forexample,
toanswerquestionssuchas‘Areregisteredusersmorelikelytodownloadfilesfromaservice?’andtouncoverhomogeneoussegmentsoftheaudience.
5.2.2 Techniquesincluderegression and correlation analysis(fordefiningrelationshipsbetweenvariables)andfactor and cluster analysis(tolookatrelationships amongvariables).
5.2.3 Potentialproblemswithinterpretationofquantitativedatainclude:
�� Poor questionnaire design.Decisionsaboutquestionnairedesigncanhaveimplicationsfortheinterpretationoftheresults,forexampletheexactwordingofstatementsaboutsatisfactionwiththeservicecouldaffecttheproportionofthesamplewhoappeartobecontentwiththeservice
�� Instructions and/or responses are misunderstood.Pilotingaquestionnaireonasmallsampleorusinginterviewerstocollectsurveydatacanhelpavoidtheseproblems
�� The sample of respondents is unrepresentative.Obtainingarepresentativesampleisparticularlydifficultwhensurveysaredistributedtoanunknownsetofpotentialrespondents.Ifthesampleislikelytobeunrepresentativeitisimportanttoconsidertheimplicationsforinterpretationofthedataandcaveattheconclusions
5.3 Making use of qualitative data5.3.1 Interpretingqualitativedataisnecessarilyasubjectiveprocess,soifresourcespermititmaybeworth
involvingmorethanonepersoninanalysisandpresentationofthedata.
5.3.2 Presentationsofqualitativedatawillusuallyattempttosummarisethecontentandimportofthematerial,andtypicallyincludequotationsfromthematerial.Foragroupdiscussionthiswouldusuallyincludeajudgementaboutvariationsofopinionbetweengroupmembersandcommentsonpossiblereasonsfordifferencesofopinion.Abriefsummaryoftheflowofdiscussioncanbehelpfultoelucidatethecontextfortheviewsandattitudesexpressedbygroupmembers.
5.3.3 Itispossibletocodeorcategorisequalitativedatasothatquantitativetechniquescanbeapplied.Codedqualitativedatacanbepresentedintabularorgraphicalformatasasupplementtotextualdigestsofthematerial.
The Guide to Researching AudiencesConciseedition
PAGE 24 5. Modelling the audience
5.4 Personas and user profiles5.4.1 Apersonaisanarchetype,withthepersonaltraits,lifestyle,aspirations,beliefs,needsandgoalsetctypical
ofasegmentoftheaudience.Personascanhelptoprovidefocusforservicedevelopmentworkthathasfairlyhigh-level,genericobjectives,egtomakethesiteeasiertouseor‘makethesitemoreappealingtoschoolagechildren’.Theprocessofdevelopingapersonaisoftenquiteinvolved,includingdetailedinterviewswithpotentialusers,ethnographicstudies,novelactivities–anythingwhichwillhelptofosteragoodunderstandingoftheaudience.
5.4.2 Ifresourcesdonotpermitthecreationoffullyfledgedpersonas,itmaybepossibletoachievesomeofthesamebenefitsbydevelopinguser profiles–descriptionsoftypicalusers,orientedtotheserviceunderconsiderationandfocusingoninteractionswiththeservice,needs,expectationsandattitudestowardstheservice.
6. Making use of audience research PAGE 25
6. Making use of audience research
This chapter reinforces the message that audience analysis should be seen in the context of the service as a whole
Audienceanalysiscanprovidemanybenefits,butanydecisionsonuseofthedatawilldependonthewiderservicecontextandpriorities.
Astrategyforrealisingthefullvalueofaudienceanalysisworkisdiscussed.
6.1 Introduction6.1.1 Audienceresearchisatoolforservicedevelopmentandcanbeusedtoprovidethebenefitsdiscussedin
Section1;however,audienceresearchisanaidtodecision-makingandnotasubstituteforit.Evidencefromaudienceresearchcanhelpserviceprovidersmakeinformeddecisionsaboutspendingondigitisation,newservices,supportfordifferentplatforms,promotingtheservicetonewaudiencesetcbutitisnotasubstituteforbusinessplanningorthepoliticaldecisionsthatneedtobemadeaboutprioritiesforpubliclyfundedservices.
Case study extract… Penn Data Farm
Using audience research to monitor resource usage and organisational performance
TheUniversityofPennsylvaniadevelopedarepositoryofquantitativeinformation,thePennDataFarm,toaidthemeasurementandassessmentoflibraryresourceuseandorganisationalperformance.
TheultimateaimoftheDataFarmistosupportexcellent‘customer’managementandtooptimisethedesignanddeliveryoflibraryresourcesandservices.ExamplesofhowstaffhaveusedtheDataFarmtoachievethisaimincludeanalysinginformationontheuseofelectronicresources(includingfrequencyofusage,costperuseandstatusofusers)todeterminetheoptimumallocationofmoneyforelectronicresources–thisisparticularlyimportantaslibrarybudgetsaresqueezed.TheDataFarmhasalsobeenusedtotracktheuseofresourcesproducedin-houseforUniversityofPennsylvaniamembers(egcourseguides,teachingmaterials,guidestolibraryservicesforasubjectetc)torefinetheirunderstandingoftheUniversitymembers’needsandworkouthowtomeetthem.
TheteamattheDataFarmarealwayslookingforwaystocollectandexploitmoreaudiencedata,buttheyremainveryconsciousofthelimitationsoftheirdata–itisoftendifficulttoknowwhatwebusagestatisticsreallymean,andusabilityanalysisisanimportantcomplementwhendesigningnewtoolsetcforadigitalservice.Forfurtherinformation,seethePennDataFarmwebsite(http://metrics.library.upenn.edu/prototype/datafarm).
6.1.2 Becausedistinctaudiencesegmentshavedifferentrequirements,itisimportanttounderstandtherelationshipbetweenthemandprioritiseallocationofresourcesappropriately.Audienceanalysiscanprovideinformationaboutaudiencesegments,theirrelativesize,andhowtheyuse,valueandaccessaservice,butdecisionsaboutresourceallocationstilldependonservicepriorities.
The Guide to Researching AudiencesConciseedition
PAGE 26 6. Making use of audience research
6.1.3 Decisionsabouthowtodevelopandpromotetheservicearestillbusinessdecisions,notleastbecauseuseoftheservicewilldependonthequalityoftheresource,andthesuccessofthemarketingstrategy.Ofcourse,audienceanalysisshouldinformboththeseactivities.
6.2 Audience research in context6.2.1 Attheconclusionofanaudienceresearchproject,whenthedatahasbeencollected,analysedand
interpreted,andconclusionshavebeendrawn,itisimportanttomakesurethatthefullvalueoftheresearchisrealised.Forexample:
�� Theresultsoftheresearchshouldbedisseminatedtostakeholdersandotherinterestedparties
�� Everyonewhohelpedwiththeprojectshouldbethanked;thisincludesstaffwhohelpedtocarryouttheresearch,participantsandanyoneelsewhocontributed
�� Considertheimplicationsofthefindingsforservicedevelopmentandaudiencedevelopment
�� Lookattheconclusionsoftheresearch–Werethespecificobjectivesmet?Whatlessonscanbelearnedforfuturework?
�� Planfollow-upandfutureresearch
�� Assessthewiderimplicationsofthework(egarethereanyimplicationsforlong-termstrategy?)
6.2.2 Itmaybeappropriatetosharesomeofyourfindingswithpartnersinthesamesector,otherpublicsectororganisationsservingasimilaraudienceetc.
Case study extract… Archival Sound Recordings
Using audience research to develop a service and monitor its use
TheBritishLibrary’sArchivalSoundRecordingswebsite,conceivedasawayofincreasingaccesstotheSoundArchive’sextensivecollection,waslaunchedin2006.
Audienceresearchplayedacentralroleinthedevelopmentofthewebsite.Theinitialphaseofworkrelatedtothechallengeofdevelopingawebsitethatmettheneedsofitsaudienceintermsofcontent,usabilityandaccessibility.Consequently,aspecificationforthewebsitewasdevelopedandthisbecamethefocusoftheinitialaudienceresearchwork.Auserpanelconsistingof10-12individualsfromavarietyofHEandFEinstitutions(amixofacademics,librariansandresearchers)wassetuptorepresenttheneedsofthetargetaudiencethroughoutthedevelopmentandrefinementofthewebsite.Thedemonstratorversionofthewebsitewasalsotestedonnewpeoplewhowerefreshtothesite.Testingwasconductedinadedicated‘usabilitylab’,andledbytrainedfacilitatorswhoshadowedtheusers.
SubsequenttothelaunchoftheArchivalSoundRecordingswebsite,audienceresearchwasusedasawayofensuringthatthewebsitewasfulfillingtheroleforwhichitwasintendedintheHEandFEcommunities.Inaddition,aprogrammeofaudienceanalysiswhichhasexaminedhowandwhenthewebsiteisusedhasbeenundertaken.TheBritishLibraryteamalsoengagedwithuserstodevelopcasestudiesonwaysofmakinguseoftheArchivalSoundRecordings,whichwillbeusedtopromotetheresourcetotheacademiccommunity.
FormoreinformationontheBritishLibrary’sArchivalSoundRecordings,seethewebsite(http://sounds.bl.uk).
6.2.3 Itshouldalsobenotedthataudienceresearchprojectswilloftenresultinchange–whetheritbedirectlytotheserviceoffered,ortotheorganisationitself(egasaresultofachangeincompanystrategy).Changemustbecarefullymanaged,andwillrequirethoughtfulplanningandsometimessensitiveimplementation,andaboveallengagementwiththepeopleaffectedbythesechanges.UsefulfurtherreadingonthemanagementofchangeinaprogrammaticcontextisManagingSuccessfulProgrammes(MSP),OfficeofGovernmentCommerce(OGC),2007edition.
Glossary PAGE 27
GlossaryAudience development
Activitiescarriedouttoincreasetheaudienceforaserviceorresource.
Audience satisfaction Establishingiftheaudienceishappywiththeservicethatisbeingprovided.Forexample,isithappywiththequality/content/accessibility/comprehensivenessoftheservice?Interrogationofaudiencesatisfactionisoftendoneintandemwithaudienceanalysis.
Digital service and digital audience
Inthisguide,digitalisusedtorefertoresourcesand/orservicesexploitingdigitisedcontentordeliveredviaadigitalchannel.Inmostinstancesthismeansaservicewhichisavailableonline,butitalsoincludesdigitisedresourcesdistributedonCDandcouldincludedigitaltelevisionserviceswhichsharesomeofthecharacteristicsofonlinedigitalservices.Digital audiencereferstotheaudienceforadigitalserviceorresource.
Ethnographic research
Researchbasedonobservationsofhumanbehaviour,carriedoutina‘naturalistic’setting.Thiscaninclude‘observation’viaautomatedrecordingdevicesandissometimesextendedtoincludeobservationsofsubjectsmadeinalaboratory–thecrucialpointisthatbehaviourinresponsetotherelevantstimulishouldbenormalandthatthepresenceoftheresearcherortheobservationalequipmentshouldnotinterferewithoraffectthesubject’sbehaviour.
Formative research Inthecontextofservicedevelopmentanddelivery,thistermcoversresearchcarriedoutduringservicedevelopmentwork,aspartofthedevelopmentprocess,toprovidefeedbackonthework.
Front-end research Inthecontextofservicedevelopmentanddelivery,thistermcoversresearchcarriedoutintheplanningandpreparationofaservice.
Focus group Agroupofindividualsselectedandassembledbytheresearchertodiscussandcommentonthetopicunderconsideration.Subjects’personalexperienceofthetopicandinteractionsamongstgroupmembersarethedistinguishingfeaturesofthetechnique.
Impact analysis Analysisofthelonger-termeffectsofaservice(orgroupofservices)onitsaudience.Longer-termevaluationmethodsarerequiredtodetermineiftheimpactofaserviceislasting.
Reach Inthisguide,reachisusedtorefertothenumberofpeopletowhomaserviceorresourceisavailable.Differentfromuptake,whichreferstousageoftheserviceorresource.
Summative research Inthecontextofservicedevelopmentthisreferstoevaluativeresearchassessingtheextenttowhichaservicehasmetspecificcriteriaforsuccess.
Target audience Theintendedaudienceforaserviceorresource.
Segmentation Theprocessofdividinganaudienceintodifferentcategoriesorsegmentsonthebasisofcriteriarelevanttotheserviceorresourceunderconsideration.Segmentsarerelativelyhomogeneousanddistinctfromothersegmentswithrespecttothechosencharacteristics.
Usability Ameasureofthequalityoftheuserexperienceofaserviceorresource.Itgenerallyencompassesfactorssuchaseaseoflearning,efficiencyofuse,memorability,errorfrequencyandseverity,andsubjectivesatisfactionwiththeinterface.
User-centred design Adesignphilosophywhichplacesthegoals,expectationsandskillsofusersatthecentreofthedesignprocess:theserviceorproductisdesignedtofittheuser.Typically,usersareinvolvedthroughoutthedesignanddevelopmentprocess,includingusertestingoftheserviceorproductwhilstitisunderdevelopment
This work is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 2.0 UK: England & Wales License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/2.0/uk/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA
This document is available in alternative formats. For more information: www.jisc.ac.uk/contentalliance
Document No: 547a Version 3.1, December 2009
The Guide to Researching AudiencesConcise edition
Further information about JISC:Web: www.jisc.ac.ukEmail: [email protected]: +44 (0)117 331 0789