Methodologies

Setting up a Study

Review

Problem Statement Problem Originality Direction Significance

Literature Review & Theoretical Framework Logical Structure Research Questions Population Sampling

Methodologies

Specific systems/ tools/ approaches to gathering and analyzing data.

Structure of the research Builds on and draws from previous sections:

problem statement, research questions, literature review, theoretical framework

Influences analysis How to choose an appropriate method?

Methodologies: Options

Quantitative Numbers, statistics

Qualitative Words, narrative

Mixed Methods Both Quantitative and

Qualitative Triangulation

More than one method, may be two of the same type

Methods: Options

Quantitative Qualitative Mixed

PredeterminedInstrument-based questionsPerformance data, attitude data, observational data, and census dataStatistical analysis

EmergingOpen-ended questionsInterview data, observation data, document data, and audiovisual dataText and image analysis

Predetermined and emergingOpen- and close-ended questionsMultiple forms of data Statistical and textual analysis

-Creswell, J.W.

Methodologies: Options Quantitative

Surveys Structured Interviews Questionnaires Bibliometrics Transaction Log Analysis

Qualitative Case Studies No Contact

Examining Documents Citation Analysis Diaries

One-to-one Unstructured Interviews Think-Aloud Ethnographic methods Observations

Group Interaction Focus Groups Delphi Method Concept Mapping

Quantitative: Bibliometrics

Quantitative study of literature Patterns of publishing within a field or body of

literature Quantitative study of information associated

with published works: authorship, publishers, citations

Quantitative: Transaction Log Analysis Quantitative study of user behavior as

exhibited through computer logs Domains (.gov, .edu, .com) “Hits” Paths followed Time spent (searching, viewing, downloading) Errors Applications used

Qualitative: Case Study

In-depth review of a single situation: program, process, phenomenon

Qualitative: Examining Documents

Systematic review of text/ images: content analysis

Classifies textual or visual material Uses analytical constructs or rules to draw

inferences about recurring aspects of text.

Types of Documents?

Policy Manuals Digital reference transcripts Comment/ complaint cards Job ads Published Literature Blogs, listserv postings, etc. Open-ended responses

Evaluation Research

Published Books Journal articles Conference proceedings Theses/dissertations Web ALA. ACRL. CIP Notes CLIR.

www.clir.org/pubs/pubs.html ARL Other

Peer ReviewPeer Review

Unpublished Internal reports Local use only reports

Qualitative Analysis: Examining Documents Focus on:

Word choice Word frequency Word sequence Intensity of feeling/ expression (how to

measure?) Key word in context

Typology of concepts/ categories?

Example

Regional Accreditation Organizations’ Treatment of Information Literacy

Qualitative Study: Citation Analysis

Systematic review of bibliographies/ references within published literature.

Focus on Authorship Form of publication (periodical, monograph,

etc.) Class of material (primary, secondary, etc.) Language Subject Currency

Citation Analysis

What can citations tell us? How scholarly is the cited literature? How current are the citations? How research-oriented is it? How interdisciplinary is it? How writes the literature? How collaborative? Where does the literature appear?

Citation Analysis

Growth of literature on a subject Dispersion of writings on a subject across

form and journals Obsolescence of literature Scholarly networks: who cites whom? Publishing productivity

Citation Analysis

Advantages Shows what is cited Does not involve

interaction with subjects

Profiles a literature Shows changes in a

field over time?

Disadvantages How complete is the

work from which citations are drawn?

How accurate are citations?

Are all materials cited? Choosing easy to find/

retrieve items over better quality?

How easily retrievable are works?

Citation Analysis: Issues

From where are citations drawn? How far can you generalize findings?

Does citation reflect use? Self-citations and/ or gratuitous citations Half-life Impact factor (to what extent are recent articles in

journals cited?) Calculated annually: divide the number of current

citations a journal receives to articles published in the previous two years by the number of articles published in those same years

Web Citations

How “prestigious” are different online sources- ejournals, open access, etc.

References to and from a Web site Retrievability

Qualitative Study: Diaries

Participants record activities, thoughts, reactions, etc. daily (weekly, etc.) over a set period of time.

Blogs- equivalent? When to use? Issues

How faithful are participants? How can you help ensure full participation?

How much guidance to give? Too much/ too little detail.

Advantages and disadvantages?

Students’ World

Nicole Henning/ Photo Diary Study (MIT) 16 students tracked their information seeking

behavior for one week. Used diaries and screen shots to record their

thoughts and actions

Qualitative Study: Interviews

Attempt to gain in-depth knowledge on a topic In a less structured format, interviewer may

act more as facilitator, asking open ended questions and drawing the participant out.

Structured, semi-structured, or unstructured? Format- in person, telephone, email?

Interviews

Types of questions: Experience/ behavior Feeling Opinion/ value Knowledge Sensory Demographic

Interviews

Issues Logistics Building trust Delicate questions/ situations- ensuring

honesty Accuracy of transcripts

Advantages and disadvantages? Appropriate uses?

Qualitative Study: Think-Aloud

Asks respondent to verbalize their thoughts while performing an assigned activity or task

Attempt to gain more insight into thought processes

Correct for mistakes/ assumptions of observer

Think Aloud

Requires participants that are highly verbal in nature

Adding this layer of feedback may affect their cognitive processes and behaviors- more “self-concious.”

How to transcribe while observing? Accuracy?

Appropriate use of clarifying questions? Appropriate uses?

Qualitative Study: Ethnographic

Tools developed in the field of cultural anthropology

Attempt to better understand people/ behavior by observing it within natural setting

Can study observable material items, individual behaviors and performances, or ideas

Requires careful attention to detail within cultural context

Ethnographic

Possible Tools: Observation in context Key informant interviewing Drawing pictures Taking photos Using maps to track activities Videotaping

Ethnographic: Examples

Susan Gibbons and Nancy Fried Foster of University of Rochester: Understanding Users to Develop Better Library Services (ACRL/NEC 2006) User-centered studies of library use… Mapping Diaries Photo Elicitation Interviews

Photo Elicitation

Mapping Diaries

Sweeping Studies

A type of spatial data analysis Useful for mapping out the physical spaces of

a library and investigating how people use those spaces

Qualitative Study: Observation

Attempt to understand activities/ behaviors Obtrusive or Unobtrusive Role of the observer:

Complete observer Observer and participant Participant with individual being observed

Qualitative Study: Focus Groups

Attempt to learn about the attitudes/ beliefs/ feelings of groups and how those influence behavior

Why groups? One individual’s comments can trigger important responses from others.

Can explore large ranges of topics

Focus Groups

Issues How structured? Importance of facilitator’s role- how well

trained? Building trust Finding reliable representative volunteers Ensuring accuracy of transcription- audio/

video tape? Outside observation? Facilities

Qualitative Study: Delphi Method

A “systematic interactive forecasting method.” Involves interviewing/ surveying of experts

within a field- generally 9 to 99 Experts are provided with hypotheses, trigger

statements, scenarios, etc. and asked to respond.

First round responses are shared with the group anonymously, so participants can revise earlier statements, react to responses.

Delphi Method

After several rounds, hypotheses may be refined, group may reach “consensus”

Facilitator reviews responses to each round- can filter out irrelevant content, choose presentation of information, ask questions.

Advantages and disadvantages?

Qualitative Study: Concept Mapping

“Any process that represents ideas in pictures of maps.”

A method of organizing the ideas and thoughts of a group to form a common framework

Can be used to integrate ideas from less structured activities such as brainstorming

Concept Mapping

Typical steps: Focus: determine desired outcomes/ questions to be

addressed Generating ideas: Brainstorm, use trigger statements,

ask questions Analysis: Sort ideas into large sets

Unstructured idea sorting: ask individuals to sort ideas into groups and label

Sorting by stakeholders: organize ideas by group that generated those ideas

Ratings: assign values to ideas (importance, feasibility, etc.)

Concept Mapping

Map analysis: generate map based on idea sorting (could use statistical software, or do by hand)

Interpret map: Share with other groups to obtain understanding

Example: IR and IL