New Performance Measures

(and Rankings of Public Libraries in Ontario)Bob Molyneux

&Stephen Abram, Convenor

January 29, 2016

What do we normally use our data for? Budget justification Decision support Finding libraries like ours to compare our

experience to theirsWhich means they can be a directory to libraries

like mineComparing like with like is important

We will add trying to get a sense of the health and trends in Ontario’s public libraries

Library data have a long history Pergamom and Alexandria We have fragmentary numbers of collections of a

number of these libraries And like modern library numbers, we are not

always sure exactly what they mean Adriano Balbi, A Statistical Essay of the Libraries of

Vienna and the World [1835] First modern attempt at comparing libraries in

major European cities using published statistics about them

Balbi’s observations

“disheartened by a disparity of opinion…” “only approximate data” “exaggerated” numbers in pursuit of

prestige

Then a wonderful discussion of the problems of comparative library data

Types of library data

Balbi was dealing with reports by visitors to various libraries at different times who recorded estimates they heard from a variety of peopleOne-time studies done by different

methods Episodic surveys

Attitudinal surveys—particularly users and non-users

Data collection on fugitive or new subjects

The data we are going to discuss are systematically collected, annual data, professionally compiled from surveys of Ontario public libraries by the Ministry of Tourism, Culture, and Sport

Available from 1999-2014 in pdf

Now converted to csv files

The Ministry has reissued these data in csv (“comma-separated values”) which means they can be read into a spreadsheet program such as Excel or LibreOffice Calc readily.

In other words, there is not a chance of introducing error when you rekey data.

This is a tremendous boon to studying our libraries using these data.

What can we do with these data?

Too much for one slide! We can look at any variables we choose for

individual libraries in one year or all libraries in one year. For example: How big were the budgets of

Ontario libraries in 2013? With a good bit of work, we can rearrange the data

and look at the select variables through time—that is, trends. Say: What happened to their budgets from 2001-

2013?

We did a bit of both and more

The report is a sampler of what can be done with these kinds of data with the focus on a province-wide view, not individual libraries

We did not take all data from this series for our report Small number of variables In order to analyze trends properly, we only took

data from libraries which reported each year. For this study, that number is 301 libraries

We separated them into 9 “Bands”—8 by size plus the First Nations’ Libraries in a 9th Band. The Ministry did the same kind of thing This is common practice in this kind of

publication

Let’s take a tour through the report

First the Primer The big story is the consistent characteristic of the library

world that affects about everything:Skewed distribution: a few large libraries and many

small onesIn 2013, the 10 largest libraries (of 300+) had 60%

of the total circulations and 54% of the total expenditures.

We must take these characteristics into account in analyzing dataHence, our size “Bands” which echo Ministry practice

The Primer, then, informs the analysis in the FOPL Reports Given it is a sampler

We segment by size of library in “Bands” Same as those used by the Ministry with a difference: First

Nations’ Libraries are analyzed separately in those tables where we use Bands

Another common tool is what is called “Rank Order Tables” Sort libraries in order by their reported data. That is rank their

results by the reported data or statistics calculated from these data

Most commonly per capitas. Dividing, say, circulations, by the resident population served by the library

We combine this technique with analysis of Bands.

Exploratory Data Analysis

Our focus, primarily, is the state of Ontario’s libraries and trends affecting them

There are many other things you could do with these dataThis is a rich series

Key Ratios

2013 data per capita and per cardholder Thirteen ratios, all libraries The ratios are largely those we focus on in the rest

of the report. As the Primer showed, per capitas allow apples to

apples comparisons of libraries of vastly different sizes You may be small, but you may be doing a better

job with what you have than bigger libraries.

Annual Population and Circulation, 2001-2013 Trend analysis is a bit different Of all libraries which reported in any year,

301 reported in each year These tables are complex

We will see them again, so let’s take a look

Traditional library measures are steadyOTOH: New things are growing

Rank Order Tables Circulation per capita and per active

cardholder, 2013, by Bands

Rank Order Tables

Expenditures per capita and per active cardholders

Conclusions for now (then)

This is the beginning. A first shot based on best guesses of where to look.

There are other ways of studying libraries such as qualitative surveys of a library’s users and their non-users. Given the rapidly changing information

environment in libraries, quicker surveys likely will be a part of the future of data gathering for supporting decision making.

Developments since then

2014 data are coming!Updates to what we have done so

comments/suggestions/criticisms eagerly sought

Experimental FOPL Index

Towards a FOPL Index “Index” numbers

Consumer Price Indexes Stock price indexes Baltic Dry Index Etc.

Library Indexes ARL Library Index BIX (http://www.bix-bibliotheksindex.de/) Hennen American Public Library Rankings (HAPLR) (haplr-

index.com) LJ Index State Rank Order Tables

Development of the FOPL Experimental Index Based on commonalities in the other indexes

(except for the ARL Index)Largely follows the BIX and HAPLR while

including the LJ Index variables which overlap the Ministry’s data

We actually have constructed an assessment inventory

Being reviewed before distribution

Structure of the FOPL Experimental Index Four “dimensions” containing 16 variables

ServiceUsageEfficiencyDevelopment

Next?

2014 data will be analyzed and integrated The FOPL Index will be reviewed by a

committee of members and adjustedThen a decision will be made about

distributionIt will very likely include live spreadsheets so

you can experiment, too.Expect a year or two for the data to mature

Questions?

Bob Molyneux drdata@molyneux.com