Management Metrics That Work - Home - Special Libraries ... · PDF file• Steeper learning...
Transcript of Management Metrics That Work - Home - Special Libraries ... · PDF file• Steeper learning...
Management Metrics That Work
Karen White USAID
Kris Vajs
Federal Reserve Board [email protected]
Karen Krugman
Export-Import Bank of the US [email protected]
April 9, 2013
Agenda
Advantages of Management Reporting
Collecting, Storing, & Analyzing Metrics
Presenting & Packaging Metrics
Management Reporting at Our Organizations
Questions & Answers
Why Report to Your Management?
3
To communicate the library’s value and demonstrate return-on-investment Reason 1
To generate support for continued or even increased staff and resources Reason 2
To demonstrate how the library supports the organization’s mission and strategic goals Reason 3
To demonstrate transparency and accountability in how resources are used Reason 4
To show how library contributions have changed over time Reason 5
Metrics: What Categories To Measure?
4
Collect This Data: • Library Size and Patron Base • Transactions: Circulations and Reference • Budget, Resource Allocation, and Value of Print &
Digital Materials • Facilities and Physical Foot Traffic • Virtual Foot Traffic • Education and Outreach • Operations
To Measure These Areas:
• Customer Satisfaction • Return on Investment • Effectiveness of Patron Outreach & Education • Impact
• Comparability to Like Libraries
Sample Metrics
5
Library Size and Patron Base
• Percentage of actual patrons to potential patrons
• Percentage of patrons seeking assistance in person, via e-mail, via phone, or via web
• Percentage of patrons by patron type
Transactions: Circulations &
Reference
• Total number of checkouts per year
• Percentage of checkouts by patron type
• Percentage inbound vs. outbound ILL requests
Budget, Resources Allocation, & Value
of Materials
• Percentage increase or decrease in overall budget over time
• Change in print vs. digital resource budget allocation over time
• Average cost per full-text article downloaded
See handout for more examples.
Sample Metrics (continued)
6
Facilities & Physical Foot
Traffic
• Number of patrons entering library, year over year
• Average duration of a library visit
• Average duration of use of various physical spaces
Virtual Foot Traffic
• Average number of library website visits per day
• Most used pages on library website
• Increase in database users over time
Education & Outreach
• Number of database trainings and attendees per year
• Increase in database users after database trainings
• Number of new library users after attending orientation
Operations
• Average minutes to process, catalog, and shelve an item
• Average daily library staff time spent on core work vs. non-core work (troubleshoot-ing computers, looking for passwords, etc.)
See handout for more examples.
Storing Your Metrics
7
Type Pros Cons
Microsoft Excel
• No cost, easy to setup • Can produce pie charts & graphs • Can export to Access and create
a database
• Inflexible • Not a true database
Relational Database (in-house or cloud-based)
• No cost, somewhat easy to setup • Consistent data that can be
searched, manipulated and reported
• Automatic calculations and reports
• Higher data storage capacity • Data exportable to multiple tools
• Steeper learning curve than Excel.
• Needs care in planning and setup
• Sometimes created for a different purpose
• If in-house, needs to interact with the network on an ongoing basis.
Virtual reference software (i.e., Altarama RefTracker, Springshare LibAnalytics)
• Collects & analyzes data for you • Reports & graphics on demand • Consolidates requests from chat,
email, phone & SMS text • Supports a searchable
knowledge base • Ability to tag individual metrics
• More expensive • Security concerns if hosted by
vendor • Possibility of vendor going out
of business
8
Decide what you’re trying to measure before you begin to collect metrics
Collect data proactively so you have them when needed
Collect quantitative and qualitative metrics. Collect data, stories and testimonials
Identify metrics from other libraries and organizations or industry standards that can be used to contextualize your metrics
Look to library vendor-supplied usage data as a resource
Limit the number of metrics to provide focus and to concentrate on the metrics with greatest impact
Collecting Your Metrics
9
Ways To Collect Metrics Data
Tally sheets/Forms
Surveys/Customer Feedback/Observations
Focus Groups/Interviews
Benchmarking
10
Interpreting Your Metrics
Number of library patrons Take simple measures
Percentage of actual patrons to potential patrons
Use ratios to compare two measures
Year-over-year increase or decrease in actual patrons to potential patrons
Use trends to see changes over time
Percentage of actual patrons to potential patrons vs. like libraries
Use benchmarks to compare to like libraries
Year-over-year increase or decrease in actual patrons to potential patrons vs. like libraries
Combine benchmarks and trends to compare to like libraries over time
See handout for more examples.
11
Presenting Your Metrics: Best Practices
Report on a regular basis: weekly, monthly, quarterly or annually
Use creative, single-use metrics to supplement your ongoing metrics program and illustrate unique points
Tailor your reporting to different audiences (executives, your supervisor, the public, your customers)
Compare metrics to past years, a small sample of similar libraries, or industry averages to create a frame of reference
Use a mixture of quantitative & qualitative data; if possible, have your report tell a story with words, pictures, and numbers.
Understand how your metrics relate to each other and be prepared to answer questions about “what it all means”
Share metrics and reports with your staff to foster an understanding of how they contribute to the library’s success
12
Presenting Your Metrics: Packaging Methods
Written Reports
Verbal Presentations
Dashboards
Infographics
Impact stories
13
Our Management Metrics: USAID
About the USAID Library
Official Department Name Knowledge Services Center
What the
Organization Does
Promotes economic prosperity; strengthens
democracy; improves global health, food
security, environmental sustainability &
education; provides humanitarian assistance.
Staff Size 11,000 Agency staff
Subject Matter International development
Our Customers USAID staff in Washington & overseas, U.S.
government agencies, the public
14
Our Management Metrics: USAID
Our Metrics Experience
How We
Collect the Data
• From our shared mailbox into SharePoint
• At the reference desk, saved in a Google Doc
• Customer feedback from emails into SharePoint
• Database usage from vendors saved in Excel
What Data
We Collect
• Type of request
• Type of client (public, partner, USAID bureau or mission)
• Customized research by sector and topic
• Door count, circulation, PC & WiFi use
• E-resource usage
• Results of outreach activities (new employee orientations,
office presentations, open houses) in terms of requests
• Use examples from customers
How We Interpret &
Report on the Data
• Weekly, monthly & annual reports
• Mixture of quantitative & qualitative information
• Trends analysis
• How we contribute to the “USAID Forward” goals
Our Management Metrics: Ex-Im Bank
About the Ex-Im Bank Library
Official Department Name The Research Library & Archives
What the
Organization Does
It backs the financing of US exports, mainly to
emerging markets.
Exports can be products (Boeing jets, oil rigs,
nail files, candy) or services (engineering,
architects)
Staff Size 400 FTE in agency; 7 staff in Research Library
Subject Matter International Business and Economics
Our Customers Chair & Board, Economists, Bankers, Lawyers;
Staff from Policy, Communications, Business
Development
15
Our Management Metrics: Ex-Im Bank
Our Metrics Experience
How We
Collect the Data
• Collected through Quickbase, a cloud-based relational
database.
• Library staff report data via dropdowns, checkboxes, and
written overviews of requests (mostly copied from e-mail)
• Anecdotal evidence collected through written and verbal
follow-up
What Data
We Collect
• Research, research, research:
• Who makes requests; purpose of requests; sources
used; time spent; % actual vs. potential users; highest
volume users
• To collect going forward: ROI of databases, gap analysis
How We Interpret &
Report on the Data
• When we report: weekly reports detailing research
requests; annual reports analyzing the year.
• Annual report contains different formats of information:
charts showing trends, pie charts showing concentrations,
tables showing impact, and bulleted lists detailing past
accomplishments & future goals.
16
17
Our Management Metrics: Federal Reserve Board
About the Board Research Library
Official Department Name Board Research Library
What the
Organization Does Provide the nation with a safer, more flexible,
and more stable monetary and financial system
Staff Size 1800 professional staff members; 13 staff in
Research Library
Subject Matter Economics, banking, and finance
Our Customers Economists, financial analysts, research
assistants, and other professionals. The Library
is open to all Board staff.
18
Our Management Metrics: Federal Reserve Board
Our Metrics Experience
How We
Collect the Data
• Service Interaction Access Database – used to collect
and generate data on customer use of library services
• Ongoing metrics are supplemented by “snapshot” data
demonstrating impacts. A snapshot looks at sample data
to make a specific point
• SharePoint site – money saved on data purchases
• SurveyMonkey – just beginning to use this to get
feedback from customers on projects and services
• Vendor statistics
What Data
We Collect
• Access database – customer name, division, library staff
member, type of request, length of time to craft response,
notes
• Our favorite metrics – market penetration and percentage
of repeat customers. We use repeat customers as an
indication of customer satisfaction.
• ROI – money saved and cost avoidance calculated for
data acquisitions work.
How We Interpret &
Report on the Data
• Annual budget presentation
19
More Resources To Get Started
• “5 Best Practices for Creating Effective Dashboards,” Tableau White Paper, August 2011. http://www.tableausoftware.com/learn/whitepapers/5-best-practices-for-effective-dashboards
• Ard, Constance. “Beyond Metrics: The Value of the Information Center,” Information Outlook (September 2012).
• Ard, Constance. Adding Value to Corporate Libraries and Information Services. London: Ark Group, 2012.
• Davis, Hilary. “Not Just Another Pretty Picture,” 2009. http://www.inthelibrarywiththeleadpipe.org/2009/not-just-another-pretty-picture/
• Dugan, Robert E., Peter Hernon, and Danuta A. Nitecki. Viewing library metrics from different perspectives: inputs, outputs, and outcomes. Santa Barbara, Calif.: Libraries Unlimited, 2009.
• Hales, Stuart. “Metrics for Special Libraries,” Information Outlook (September 2012).
• Hiller, Steve. “Performance Measurement in Libraries,” paper presented at the NISO Webinar on Measure, Assess, Improve, Repeat: Using Library Performance Metrics, September 8, 2010. http://www.niso.org/apps/group_public/download.php/4872/performance10web.pdf
20
More Resources To Get Started • Hiller, Steve. “What Are We Measuring, and Does it Matter?” Information
Outlook (September 2012). • Hiller, Steve. “Performance Measurement in Libraries,” paper presented at the
NISO Webinar on Measure, Assess, Improve, Repeat: Using Library Performance Metrics, September 8, 2010. http://www.niso.org/apps/group_public/download.php/4872/performance10web.pdf
• Kyrillidou, Martha. “Looking Ahead: The Future of Performance Metrics,” paper presented at the NISO Webinar on Measure, Assess, Improve, Repeat: Using Library Performance Metrics, September 8, 2010. http://www.niso.org/apps/group_public/download.php/4872/performance10web.pdf
• “More Alike than We Think,” Library Journal vol. 136 no. 8 (2011).
• Stevens, Kerrie, “Introduction to Library Metrics: Statistics, Evaluation, and Assesment, Australian and New Zealand Theological Library Association, unknown date. http://www.anztla.org/Conference10/postconf10/Stevens_1.pdf
• Strouse, Roger. Information management under fire: measuring ROI for
enterprise libraries. Outsell, Inc., Nov. 9, 2007. 11 p.
• Strouse, Roger. Using ROI to support the IM value proposition. Outsell, Inc., May 27, 2010. 18 p.