LibQUAL+LibQUAL+®® in South Africa in South Africa

Sherrie Schmidt, Arizona State and ARL PresidentSherrie Schmidt, Arizona State and ARL President Martha Kyrillidou, ARL Martha Kyrillidou, ARL

Stellenbosch, South AfricaAugust 13, 2007

old.libqual.org

old.libqual.org

OverviewOverview

• Introduction and Overview

• LibQUAL+® in South Africa

• The importance of LibQUAL® for ARL Statistics and Measurement Program

• LibQUAL+® in the UK

• LibQUAL+® at Arizona State

• LibQUAL+® at the U. of Virginia

• Share Fair

• What have we learned?

old.libqual.org

old.libqual.org

old.libqual.org

old.libqual.org

World World LibQUALLibQUAL++®® Survey Survey

old.libqual.org

Mission: Shaping the future of research libraries in the changing environment of public policy and scholarly communication.

Members: 123 major research libraries in North America.

Ratios: 4 percent of the higher education institutions providing 40 percent of the information

resources.

Users: Three million students and faculty served.

Expenditures: 37 percent is invested in access to electronic resources.

old.libqual.org

ARL Mission

• Non-profit organization of the libraries of research institutions in North America

• Forum for exchange of ideas

• Agent for collective action

old.libqual.org

. . . chart a course for every endeavor that we take the people's money for, see how well we are progressing, tell the public how we are doing, stop the things that don't work, and never stop improving the things that we think are worth investing in.

– President William J. Clinton, on signing the Government Performance and Results Act of 1993

Assessment – the Imperative

old.libqual.org

Assessment – the ChallengeAssessment – the Challenge

“The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.”

Sarah Pritchard, Library Trends, 1996

old.libqual.org

Library Assessment at ARL

What are some of the developments with library assessments efforts?

ARL StatsQUAL™

E-Metrics

LibQUAL+®

DigiQUAL™

MINES for Libraries™

Where are the most critical needs and opportunities?

What are the lessons learned?

old.libqual.org

LibQUAL+ LibQUAL+ TM Brief History Brief History

• Experience with SERVQUAL in many libraries in the 1990s• Texas A&M SERVQUAL assessment in the 1990s• New Measures Initiative called a meeting of interested ARL libraries

(ALA Midwinter 2000)• Pilot with 12 ARL libraries (spring 2000)• External funding through FIPSE, U.S. Department of Education

(September 2000)• Participation and endorsement by consortia (Greater Midwestern Big

12 Plus Consortium, OhioLINK, etc.)• Consortia and related associations interest• 2003 UK participation• 2005 South Africa• More than 200 institutions every year

old.libqual.org

Rapid GrowthRapid Growth

• Languages– American English– British English– Chinese (2007)– French– Dutch– Swedish– Norwegian– Finnish– Danish

• Consortia– Each may create 5 local questions

to add to their survey

• Countries– Australia, Canada, Denmark,

Egypt, Finland, France, Hong Kong, New Zealand, the Netherlands, Norway, South Africa, Sweden, Switzerland, UAE, U.K., U.S.

• Types of Institutions– Academic Health Sciences– Academic Law– Academic Military– College or University– Community College– Electronic– European Business– Family History– FFRDC – High School (2007)– Hospital– National Health Service England– Natural Resources– New York Public– Public– Smithsonian– State– University/TAFE

old.libqual.org

250

218

307308

204

164

13

43

152,111

176,360

151,460

113,480

78,863

4,407

20,416

128,958

0

50

100

150

200

250

300

350

2000 2001 2002 2003 2004 2005 2006 2007

Survey Year

Number ofInstitutions

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

180,000

200,000

Number ofResponses

Number of Institutions

Number of Responses

LibQUALLibQUAL++®® Participants Participants

* 2007 data reflects Session I data only

old.libqual.org

LibQUALLibQUAL++®® Languages Languages

American English

Dutch EnglishFrench Canadian DutchSwedish

Swedish(British English)

Afrikaans

DanishFinnishGerman Norwegian

British English

Continental French

old.libqual.org

Why LibQUAL+?

• To base decisions on data– Internally– Externally– Comparative– Longitudinally

• Cost effective• Timely data• Support available• Community building• Sharing of experiences• Learning and improvement

old.libqual.org

13 LibrariesEnglish LibQUAL+™ Version

4000 Respondents

QUAL

QUAN

QUAL

QUAL

QUAN

QUAL

PURPOSE DATA ANALYSIS PRODUCT/RESULTDescribe library environment;build theory of library service quality from user perspective

Test LibQUAL+™ instrument

Refine theoryof service quality

Refine LibQUAL+™ instrument

Test LibQUAL+™ instrument

Refine theory

Unstructured interviewsat 8 ARL institutions

Web-delivered survey

Unstructured interviews at Health Sciences and the Smithsonian libraries

E-mail to surveyadministrators

Web-delivered survey

Focus groups

Content analysis:(cards & Atlas TI)

Reliability/validityanalyses: CronbachsAlpha, factor analysis,SEM, descriptive statistics

Content analysis

Content analysis

Reliability/validity analyses including Cronbachs Alpha,factor analysis, SEM, descriptive statistics

Content analysis

VignetteRe-tooling

Iterative

Emergent2000

2004315 Libraries English, Dutch, Swedish,

German LibQUAL+™ Versions160,000 anticipated respondents

LibQUAL+LibQUAL+®® Project Project

Case studies1

Valid LibQUAL+™ protocol

Scalable process

Enhanced understanding of user-centered views of service quality in the library environment2

Cultural perspective3

Refined survey delivery process and theory of service quality4

Refined LibQUAL+™ instrument5

Local contextual understanding of LibQUAL+™ survey responses6

old.libqual.org

Multiple MethodsMultiple Methodsof Listening to Customersof Listening to Customers• Transactional surveys*• Mystery shopping• New, declining, and lost-customer surveys• Focus group interviews• Customer advisory panels• Service reviews• Customer complaint, comment, and inquiry capture• Total market surveys*• Employee field reporting• Employee surveys• Service operating data capture

*A SERVQUAL-type instrument is most suitable for these methods

Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).

Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.

old.libqual.org

old.libqual.org

Premises

Three Seminal Quotations

old.libqual.org

PERCEPTIONS SERVICE “….only customers judge quality;

all other judgments are essentially

irrelevant”

Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.

LibQUAL+™ Premise #1

old.libqual.org

LibQUAL+™ Premise #2

“Il est plus nécessaire d'étudier

les hommes que les livres”

—FRANÇOIS DE LA ROCHEFOUCAULD

old.libqual.org

“We only care about the things we measure.”

--Bruce Thompson, CASLIN, 2006

LibQUAL+™ Premise #3

old.libqual.org

Extended GAPS ModelOrganizational Barriers to SQ Customers’ Assessment of SQ

Poor UpwardCommunication

Poor HorizontalCommunication

Poor Tech - JobFit

Perception ofInfeasibility

GAP 1

GAP 2

GAP 3

GAP 4

GAP 5

Reliability

Responsiveness

Assurance

Empathy

Tangibles

old.libqual.org

DimensionsDimensions

2000 2001 2002 2003-200741 items 56 items 25 items 22 items

Affect of Service Affect of Service Service Affect Service Affect

Library as Place Library as Place Library as Place Library as Place

Reliability Reliability Personal ControlInformation Control

Provision of Physical Collections

Self-RelianceInformation Access

Access to Information

Access to Information

old.libqual.org

Survey Structure Survey Structure (Detail View)(Detail View)

old.libqual.org

Interpreting Service Quality Data

Three Interpretation Frameworks

old.libqual.org

Benchmarking Against Peer Institutions

--1,000,000 Users; 1,000 Institutions!

NORMS! NORMS! NORMS!

Interpretation Framework #1

old.libqual.org

Score Norms

• Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample.

• LibQUAL+™ norms have been created at both the individual and institutional level

old.libqual.org

Benchmarking Against Self, Longitudinally

“Nobody is more like me than me!”--Anonymous

Interpretation Framework #2

old.libqual.org

Interpreting Perceived Scores Against Minimally-Acceptable and Desired

Service Levels (i.e., “Zones of Tolerance”)

Interpretation Framework #3

old.libqual.org

old.libqual.org

old.libqual.org

“22 Items and The Box….”

Why the Box is so Important– About 40% of participants provide open-

ended comments, and these are linked to demographics and quantitative data.

– Users elaborate the details of their concerns.

– Users feel the need to be constructive in their criticisms, and offer specific suggestions for action.

old.libqual.org

“…and Five Ancillary Items”

Either Zero or Five Ancillary items are selected to address local or consortial concerns

– Items from the initial LibQUAL+TM item pool.

– Items written by previous consortial groups.

old.libqual.org

LibQUAL+LibQUAL+®® Resources Resources

• LibQUAL+® Web site:http://old.libqual.org

• Publications:http://old.libqual.org/publications

• Events and Training:http://old.libqual.org/events

• Gap Theory/Radar Graph Introduction:http://old.libqual.org/Information/Tools/libqualpresentation.cfm

• LibQUAL+® Procedures Manual:http://old.libqual.org/Publications/index.cfm