Cooperative Virtual Reference Assessment: Service Process, Service Quality, and User Perception
1
Cooperative Virtual Reference Assessment: Service Process, Service Quality, and User Perception Christine Tobias, Reference & Technology Librarian, Michigan State University Libraries Background and Objectives Michigan State University participates in virtual reference with two cooperatives: Research Help Now (RHN) • Academic libraries in Michigan (2 universities and 11 community colleges). • More information available at www.researchhelpnow.org. QuestionPoint 24/7 Academic Cooperative • Academic libraries utilizing OCLC’s QuestionPoint software to conduct virtual reference nation-wide. • More information available at www.questionpoint.org. This poster shares the methodology and preliminary results of a three-part quantitative assessment of cooperative virtual reference conducted in the areas of Service Process, Service Quality, and User Feedback. Service Process What are the information needs of MSU patrons in virtual reference? (i.e. – What types of questions are being asked?) Service Quality What is the overall quality of service provided to MSU patrons in virtual reference? Are cooperative librarians helpful to MSU patrons? User Perception Methodology Service Process To develop a better understanding of the information needs of MSU patrons in virtual reference, a systematic sample of 343 chat and instant message (IM) transcripts from the academic year, 2009-2010, were assigned up to three customized descriptive codes. Service Quality To evaluate the quality of service provided to MSU patrons in cooperative virtual reference, the systematic sample of 343 chat and IM transcripts used to determine the Service Process were scored using a Virtual Reference Service Quality Instrument, designed to assess tone, reference interview quality, provision of information, and closing. User Feedback To gain a user perspective on the effectiveness and quality of the cooperative virtual reference service, responses from 209 user surveys submitted between September 2010 – June 2011 were reviewed and scored. Service Process How are MSU patrons using the virtual reference service? (i.e. – Proportion of web chat vs. Qwidget (IM)) 1. Research Question (27.4 %): “How do I find research materials on my specific topic?” 2. Specific Journal Article (19.5 %): “I have a citation for this journal article. How do I find it?” 3. Electronic Resource (18.6 %): “How can I find online journal articles/e-books?” 4. Specific Book/Document (10.9 %): “Do you have this book in the library?” 5. Databases (7.1 %): “Can you help me use [Name of Database]?” Types of Questions Asked in Virtual Reference Service Quality Scores: Average Percentage by Section Proportion of Questions Answered Per Librarian Group Service Quality Scores: Average Percentage by Librarian Group Type of Virtual Reference Service Used: Web Chat vs. Qwidget (IM) Service Quality Is cooperative reference an efficient and/or effective service? From the users’ perspective, is cooperative virtual reference effective? What are the average scores of responses on user surveys? Contact Information: Christine Tobias Reference & Technology Librarian Michigan State University Libraries Email: [email protected] Twitter: @ctobias27 Phone: 517-884-0897 Was the librarian helpful? 2.88 Neutra l = 2 Yes = 3 Was the service easy to use? 2.88 Neutral = 2 Yes = 3 Did you feel the librarian fully understood your question? 2.83 Neutr al = 2 Yes = 3 Were you satisfied with the answer to your question? 2.78 Neutra l = 2 Yes = 3 The quality of the library staff answering your question was…? 3.65 Good = 3 Excellent = 4 Will you use this service again? 2.82 Maybe = 2 Very likely = 3 Preliminary Results Service Process Cooperative virtual reference is a valid research service point! • Most frequently asked type of question is a research question. • Web chat used much more frequently than Qwidget (IM). • Possibly due to order of presentation of services on Ask A Librarian page. • Qwidget (IM) not a 24/7 service in 2009-2010; Hidden from public view when no MSU librarians were available. Service Quality Participation in cooperative virtual reference is efficient and effective! • MSU patrons are serviced mostly by cooperative librarians in virtual reference. • Research Help Now and 24/7 librarians scored higher in service quality than MSU librarians. • Each group of cooperative VR librarians scored highest in Provision of Information. • Each group of cooperative VR librarians scored lowest in Tone and Closing. • Each group of cooperative VR librarians could benefit from enhancing their soft skills/communication etiquette skills. User Perception Like! Like! Like! • Users of cooperative virtual reference provide mostly positive feedback about the quality of service, level Copyright 2011 Christine L. Tobias. All rights reserved.
-
Upload
christine-tobias -
Category
Technology
-
view
595 -
download
1
description
Poster presented at Michigan Library Association Annual Conference - October 26, 2011. Presents the methodology and preliminary results of a three-part quantitative assessment of the virtual reference service at Michigan State University Libraries.
Transcript of Cooperative Virtual Reference Assessment: Service Process, Service Quality, and User Perception
Cooperative Virtual Reference Assessment: Service Process, Service Quality, and User Perception
Christine Tobias, Reference & Technology Librarian, Michigan State University Libraries
Background and ObjectivesMichigan State University participates in virtual reference with two cooperatives:
Research Help Now (RHN)
• Academic libraries in Michigan (2 universities and 11 community colleges).
• More information available at www.researchhelpnow.org.
QuestionPoint 24/7 Academic Cooperative
• Academic libraries utilizing OCLC’s QuestionPoint software to conduct virtual reference nation-wide.
• More information available at www.questionpoint.org.
This poster shares the methodology and preliminary results of a three-part quantitative assessment of cooperative virtual reference conducted in the areas of Service Process, Service Quality, and User Feedback.
Service ProcessWhat are the information needs of MSU patrons in virtual reference? (i.e. – What types of questions are being asked?)
Service QualityWhat is the overall quality of service provided to MSU patrons in virtual reference?
Are cooperative librarians helpful to MSU patrons?
User Perception
Methodology
Service ProcessTo develop a better understanding of the information needs of MSU patrons in virtual reference, a systematic sample of 343 chat and instant message (IM) transcripts from the academic year, 2009-2010, were assigned up to three customized descriptive codes.
Service QualityTo evaluate the quality of service provided to MSU patrons in cooperative virtual reference, the systematic sample of 343 chat and IM transcripts used to determine the Service Process were scored using a Virtual Reference Service Quality Instrument, designed to assess tone, reference interview quality, provision of information, and closing.
User FeedbackTo gain a user perspective on the effectiveness and quality of the cooperative virtual reference service, responses from 209 user surveys submitted between September 2010 – June 2011 were reviewed and scored.
Service Process
How are MSU patrons using the virtual reference service? (i.e. – Proportion of web chat vs. Qwidget (IM))
1. Research Question (27.4 %):“How do I find research materials on my specific topic?”
2. Specific Journal Article (19.5 %):“I have a citation for this journal article. How do I find it?”
3. Electronic Resource (18.6 %): “How can I find online journal articles/e-books?”
4. Specific Book/Document (10.9 %):“Do you have this book in the library?”
5. Databases (7.1 %):“Can you help me use [Name of Database]?”
Types of Questions Asked in Virtual Reference
Service Quality Scores: Average Percentage by Section
Proportion of Questions Answered Per Librarian Group
Service Quality Scores: Average Percentage by Librarian Group
Type of Virtual Reference Service Used: Web Chat vs. Qwidget (IM)
Service QualityIs cooperative reference an efficient and/or effective service?
From the users’ perspective, is cooperative virtual reference effective?
What are the average scores of responses on user surveys?
Contact Information:
Christine TobiasReference & Technology LibrarianMichigan State University Libraries
Email: [email protected]: @ctobias27Phone: 517-884-0897
Was the librarian helpful?
2.88Neutral = 2
Yes= 3
Was the service easy to use?
2.88Neutral = 2
Yes= 3
Did you feel the librarian fully understood your question?
2.83Neutral = 2
Yes= 3
Were you satisfied with the answer to your question?
2.78Neutral = 2
Yes= 3
The quality of the library staff answering your question was…?
3.65Good = 3
Excellent = 4
Will you use this service again?
2.82Maybe = 2
Very likely = 3
Preliminary Results
Service ProcessCooperative virtual reference is a valid research service point!• Most frequently asked type of question is a research question.• Web chat used much more frequently than Qwidget (IM). • Possibly due to order of presentation of services on Ask A Librarian page.• Qwidget (IM) not a 24/7 service in 2009-2010; Hidden from public view
when no MSU librarians were available.
Service QualityParticipation in cooperative virtual reference is efficient and effective!• MSU patrons are serviced mostly by cooperative librarians in virtual reference.• Research Help Now and 24/7 librarians scored higher in service quality than MSU
librarians.• Each group of cooperative VR librarians scored highest in Provision of
Information. • Each group of cooperative VR librarians scored lowest in Tone and Closing. • Each group of cooperative VR librarians could benefit from enhancing their
soft skills/communication etiquette skills.
User PerceptionLike! Like! Like!
• Users of cooperative virtual reference provide mostly positive feedback about the quality of service, level of librarian skill, and satisfaction in answer.
• Users frequently report that they would use the virtual reference service again.
Copyright 2011 Christine L. Tobias. All rights reserved.
