Return-on-Investment Studies: Using Production Data in Technical Services

Beth Picknally CamdenUniversity of Pennsylvania Libraries

bethpc@upenn.eduJune 2014

Business definition Library approach My approach

◦ Cost◦ Value

What is ROI?

Fast-track replacements Serials check-in Returns

Three studies

Fast-track replacement is a process where we purchase replacements for missing books before they are fully searched & declared lost.

Reason for study: ◦ time pressure; “niche” workflow◦ Staff reporting that many books were “found” by

the time replacement arrived◦ Growth of BD/EZB seen as replacing the need

Fast-track: Problem/question

Costs Sample of 75 replacements over 3 FYs Circulation data Discussions with bibliographers

Fast-track: Data gathering

Total spentNumber purchased

Average cost

FY08 $19,394.33 427 $45.42FY09 $23,953.85 478 $50.11FY10 $27,074.64 507 $53.40

     Total 08-10 $70,422.82 1,412 $49.87

Fast Track: Costs

 

Sample 

sizeStill

lost? %Aver circ orig copy

Aver circ repl copy

Aver circ BD/EZB

FY08 25 14 56.0% 5.12 2.64 0.68FY09 25 12 48.0% 3.64 0.96 2.4FY10 25 13 52.0% 3.04 1.44 0.16Overall 75 40 53.3% 3.93 1.68 1.08

Fast track: Sample

Couldn’t get report with all the factors that I wanted to consider

So used sample for a deep dive◦ Voyager searching◦ BD/EZB reports

Fast track: Missing data?

Nearly half of the sampled titles (46.7%) were no longer missing

Replacement copies circulated fewer times than originals

11 of the 75 titles (14.6%) had no charges for either the original or replacement copies

58 of the 75 titles were never borrowed BD/EZB

Bibliographers valued fast-track

Fast-track: Data summary

Continue Fast-track Change in timing (now biweekly) Considering another review

Fast-track: End result

Is periodical check-in worth the cost (staff time, etc.)?

Check-in: Problem/question

Cost Staff interviews Patron point-of-use survey Tracking patron questions

Check-in: Data gathering

Total salaries  $           77,216.00 current periodicals (est.) 5500Annual cost per title  $                  14.04 

Check-in: Cost

FY11 collection spending Monograph:  $      3,686,338.19  24.82%

 Serial:  $      1,507,453.46  10.15% Electronic:  $      9,031,362.90  60.80%

 General funds:  $         629,180.47  4.24%

2 minute paper survey The survey results indicated:

◦ Users found issues they needed on the shelves (71%)

◦ Users consulted catalog first (64%)◦ If not found: they equally either ask for help,

request an ILL, or use the online version

Check-in: User survey

Two week period tracked patron questions relating to current periodicals at various locations

Total (in-person; phone; IM/Chat): 1,011 Questions relating to current periodicals:

38 (3.75%)

Check-in: Tracking questions

Value of check-in◦ Collection integrity◦ Fiscal responsibility◦ Competence and service quality◦ Vendor reliability and accountability

Also gathered info about current WF problems

Check-in: Staff interviews

Voyager does not provide any data to track staff productivity; time spent on check-in; who did the check-in; etc.

Web analytics could not be gathered to track patron use of current issue information

Check-in: Missing data

Report recommended continuing check-in◦ Deciding factor: fiscal responsibility

Appendix documented a variety of workflow and timeliness problems. Many have been addressed.

Check-in: Results

This study looked at activity of returning books to vendors, for both approval plans and firm orders, and included a review of related activities including bibliographer review of approval books.

Returns: Problem/question

Departmental annual reports Voyager reports YBP GOBI reports

Returns: Data sources

Cost of returns   annual cost   cost per item Staff time to process returns  $ 14,528.80   $ 23.97 Return Postage  $    2,500.00   $   4.13 Total  $ 17,028.80   $ 28.10 

Cost of Returns

30+ approval plans 3 years of data:

◦ Number received (up slightly over 3 years)◦ Number returned (declining over 3 years)◦ Calculated % returned for each vendor

Overall return rate declining 4%2%

Returns: Percent Returned

06/07 07/08 08/09 09/10 10/11 11/12

All Approval Returns

1956 1877 1312 993 693 606

YBP Returns n/a n/a n/a 576 461 388

Returns: 6 year comparison

Approval Return Reasons damaged/defective 40 2%duplicate 564 29%no to series 13 1%poor quality 21 1%reprint 12 1%scope 656 34%textbook 3 0%too expensive 56 3%vendor error 1 0%wrong format 2 0%no reason 563 29%

Returns: Reasons

LC Class  Shipped Returned % Ret'd

A 12 0 0%B 1828 21 1%C 84 0 0%D 1299 6 0%E 637 1 0%F 374 1 0%G 449 28 6%H 2937 36 1%J 970 5 1%K 460 54 12%L  551 5 1%

LC Class  Shipped Returned % Ret'd

M 335 0 0%N 664 11 2%P 3839 12 0%Q 460 76 17%R  184 14 8%S 55 2 4%T 357 55 15%U 183 27 15%V 35 5 14%Z 47 1 2%Total 15760 360 2%

Returns by LC class

Eliminated plan with high rate of return Shared data with Coll Dev for updating

profiles Not returning books costing $25 or less Used data (reduction in returns) to justify

changing staff duties to union

Returns: Results

Time consuming Not all data available Unexpected findings about related WF Assumptions may/may not be proven Other factors beyond data can influence

decisions Determining the VALUE of a service is more

difficult than determining the COST

ROI Lessons learned

“Evidence-based decision-making allows libraries to take new strategic directions more decisively and develop innovative service offerings more effectively than would otherwise be possible.”

Driving With Data: A Roadmap for Evidence-Based Decision Making in

Academic Libraries – Marcum & Schonfeld