Basics Terminology Nat Lib Stats Lib Meter Zbw Hh Workshop

LibMeterLibMeter

LibMeter - Library eMetrics

Basics & TerminologyWorkshop ZBW-HH Part 1-2

Nutzungsanalyse elektronischer Informationsdienstleistungen für das praktische Bibliotheks-Management

Veranstalter: Berufsverband Information

Bibliothek e. V. (BIB)Landesgruppe Hamburg

6. November 2009

Referent:Dr. Peter Ahrens

Freier Referent

Assistenz:Tanja Haberkorn

Beta 0.8 2009-11-01

© Peter Ahrens, Cologne, 2009

LibMeterLibMeter

Nutzungsanalyse elektronischer Informationsdienstleistungen für

das praktische Bibliotheks-Management

Grundlagen Standortbestimmung & Perspektiven

Vorstellung Programm Überblick 1. Grundlagen und Begriffe 2. DBS & BIX Vorstelllung praktischer Teil – BIX Übungen 3. Nationale eUsage Jahres-Statistiken 4. COUNTER, OpenURL & u.a. Monatsstatistiken 5.Alternativ-Blöcke 6. Diskussion & Feedback


LibMeterLibMeter

LibMeter Seminar

x

1. Intro


LibMeterLibMeter

LibMeter Seminar

x

Introduction


LibMeterLibMeter

Questions to StartAre you familiar with ? EZB / DBIS DBS / BIX Indikatoren-Raster COUNTER / Sushi OpenURL / SFX OPAC 2.0 Funktionen Web Analytics


LibMeterLibMeter

How Needs, Services, Usage & Community Value are related ?

C O M M U N I T Y with N E E DS E R V I C E S

S C

O P

E

U T

I L

I T Y

USA

BIL

ITY

ACCE

SSIB

ILIT

Y

VISI

BIL

ITY

AW

AR

ENES

S

USAGE ( = ACCEPTANCE)VALUE - Performance


LibMeterLibMeter

What Performance Indicators to measure ? Input

Resourcen: Finanzen, Räume, Personal, Bestände, PCs

Output Bestände, Angebote Besuche, Ausleihen, konv. Nutzung &

elektron. Outcome / Impact - Wirksamkeit

Z.B. Informationskompetenz, Wissen Z.B. Fähigkeiten / Verhaltensänderungen


LibMeterLibMeter

How to collect data about Libary‘s eService Usage with eMetrics?4 Dimensions of Assessment Census counts (full):

externally generated, Vendor usage data (eg COUNTER)

Locally/internally generated usage data, (eg OpenURL)

Sample counts (random or non-random): externally generated, web survey data Internally generated, web survey/usage data


LibMeterLibMeter

eMetrics and broader context of Library assessment not covered Integrated Assessment Instruments

LibQual / ServQual Measure users expectations and subjective

fullfillment of Library Services ISO 2789 und ISO 11620 Management Approaches

Balanced Scorecard Library Controlling Cycles


LibMeterLibMeter

How to look at any Statistic ? Motivation (Why) Methods / Tools / Technologies (How) Limitations (Scope, Issues, Problems) Observations (Figures / Results) Interpretation (Changes, Trends, Pittfalls) Consequences (Decisions, Actions)

And remember: Don‘t trust any statistic you did not fake yourself !


LibMeterLibMeter

Questions asked: How are we doing ? How high is a specific usage ?

(absolute/normalized) How does this compare to peer-

institutions? How does this compare to other services? What is the trend ?

Growth/Decline Acceleration, Slowdown Life-Cycle Phase


LibMeterLibMeter

Usage „Stats“ Terms &

Difinitions


LibMeterLibMeterBasic Terms I Log (item)

Record of Events Count = Value = Figure = Measure = Parameter

Numerical Quantity Data Variable (numerical) value; Here: Output Parameter!

Indicator Something giving information, indicating something

Statistics = „Stats“ Collection of numerical data Analysis and Interpretation of Statistical Data


LibMeterLibMeterBasic Terms II Logging

Storing and preserving a File-record with detailed technical Information about all occuring service events (Web)

Usage (figures or statistics) Quantitative parameters/values derived from filtering

and processing Log-Files (aggregating, relating, normalizing)

Usage monitoring = analysing (web) usage figures over time (longitudinal) or between services (transversal)

Usability Degree of suitability of (web) service for purpose


LibMeterLibMeterBasic Terms III Performance

Degree of Effectiveness (e.g. of work or a service) Ranking

Position in Relation to others / Working out Ranking Order

Rating classification of something on a scale according to

how much or how little of a quality it possesses Benchmarking

Comparing against a standard against which something can be measured

Best Practice


LibMeterLibMeter

Basic Terms IVDescriptive Statistics Minimum / Maximum

Smallest value / Highest value Average (Artihmetic Mean)

Sum of all values divided by total number Median

Value in the middle of the range

Standard Deviation (Measure for average Distance) Variation Coefficient (Measure for Width of Distribution) Quantil, Percentil, Quartil … (->Indikatoren-Raster) (Skewness, Kurtosis …)


LibMeterLibMeter

Basic Terms VUsage Counts and Derivatives Value

Example: Downloads (D) Normalized Value or Ratio (per external basis)

Example: Downloads per User (DpU) First Derivative (by time=rate) – per period

Example: Downloads per User per Year (DpUpY) Second Derivative –> Change Rate

Example: Growth/Decline=Change (CoDpUpY) Third Derivative - > Trend

Example: Acceleation/Slowdown (CoCoDpUpY)


LibMeterLibMeter

LibMeter Seminar

x

2. NatLibStats


LibMeterLibMeter

Comparing Usage at

different Types of Institutions


LibMeterLibMeter

2007 EZB Usage per Primary User at Universities versus Polytechs

EZB Statistics Universities (n=57)

Median = 6,2 Mean / Average = 6.9 Standard Deviation = 5.6 Variation Coefficient = 0.8

Polytechs (n=34) Median = 0,42 (sic!) Mean / Average = 2.0 Standard Deviation = 6.1 Variation Coefficient =3.0

Düsseldorf: Mean / Average 7.0 = 0.13 Std. Dev. above

Mean

While you may calculate Mean and St. Dev. for a population, make sure it is normally

distributed before drawing further conclusions (check Median) !


LibMeterLibMeterLooking annually at

1 Parameter (EZB)

at 1 time (DBS 2007)

for many libraries


LibMeterLibMeter

Comparing EZB Use among University Libraries in 2007

Compiled from DBS VA: http://www.bibliotheksstatistik.de/eingabe/dynrep/index.php [2009-04-15]

EZB (B+C only) call as per DBS 2007Standardized per Primary User per Year

http://www.bibliotheksstatistik.de/eingabe/dynrep/index.php


LibMeterLibMeter

Sample University Library Profile (DBS)Basics on Library-“Size“, -Input & Output

Actions Raw Numbers

Often very high values Go beyond imagination

Work Basis: User Numbers Different Types Can be combined to

Primary User NumberPrimary User Number Standardization

Key Step: Calculation of „Value Value per User per per User per YearYear“ = Normalization

Benefits Numbers become imaginable Numbers can be memorized Numbers become

comparable

Data downloaded from www.bibliotheksstatistik.de into PC-Spreadsheet Program, [2009-05-03]

http://www.bibliotheksstatistik.de/


LibMeterLibMeter

BIX Performance-Indicator „Virtual Library Use“ New (per 2008*) in German Library Performance Index

„BIX“ Normalized Combined Parameter: „Virtual Annual Use“ Definition: (OPAC & Homepage) / Number of

Primary Users Simple Interpretation

Year Source OPAC Homepage OPAC + HP Primary Users2007 DBS 684.097 3.477.400 4.161.497 16.991

HDM 1.116.511 2.360.890 3.477.401 16.991

Hits per User 65,7 138,9 204,7 = Virtual Use


LibMeterLibMeter

How SMART is „Virtual Use“ ?Specific: Quite general Measurable: YES Community Standard with help

of HDM ZählpixelAchievable and Attributable: Library is actively producing Service Today every library can produce #Relevant and Realistic: YES Users frequently go there Closely tied to core services How representative for total

spectrum of electronic Library Services ?

Time-bound, Timely, Trackable, and Targeted:

No log-files publicly available (yet?) Not available for all periods


LibMeterLibMeterGo to Screens List

Slides on DBS, BibS,BIX &

ARL(Start Present_26 Slide 07)


LibMeterLibMeter

German Library Statistics 10 Year Overview

2008

1999 Univ Lib Change +/- 0% -24% +35% +50% -7% +10% +29% +31% Price Index DEEducation

1999 to 2008> + 50 %


LibMeterLibMeter

„DBS Variable Analysis“ (Display!) Selection Dialogue


LibMeterLibMeter

DBS Variable AnalysisResult Table


LibMeterLibMeter

DBS – IndikatorenrasterBenchmark for Public Libraries Logical Framework

Public Libraries Only 1 of 5 Community Size

Classes 2007 (only as per 2009-10-10)

Data Shown 21 Indicators calculated

from Primary Measures Grouped in 4 Dimensions Minimum, Median,

Maximum 6 Percentile Ranges Positioning „own“ Library

A 5% B 25% C 50% D 75% E 95% F gültige Werte

Medien / EW 0,69 1,15 1,21 1,66 2,18 3,24 565

Erneuerungsquote (Zugang / Bestand in Prozent) 3,1 6,0 8,1 8,4 10,5 15,0 557

Neuerwerbungen: ME / EW 0,03 0,09 0,10 0,13 0,19 0,30 563

Veranstaltungen / 1000 EW 0,5 1,6 1,8 2,9 5,0 9,0 570

Jahresöffnungsstunden / 1000 EW 33,8 35,0 56,9 73,2 93,3 131,7 566

Publikumsfläche / 1.000 EW (qm) 7,44 14,82 21,87 25,18 32,16 56,55 560

Computerplätze/1000 EW 0,0 0,09 0,1 0,1 0,2 0,5 570

Anteil aktive Entleiher an EW 3,8% 7,6% 10,0% 11,2% 15,2% 26,0% 557

Besuche / EW 0,3 1,0 1,5 2,3 2,5 4,7 404

Umschlag (Entleihungen / ME) 1,0 2,0 2,7 3,7 4,8 5,3 565

Entleihungen / EW 1,0 2,8 4,5 5,5 7,1 11,6 567

Recherchen / EW 0,00 0,06 0,15 0,16 0,30 0,68 149

Mitarbeiter (VZÄ )/ 1000 EW 0,04 0,09 0,14 0,17 0,20 0,33 522

1000 Entleihungen / MA (VZÄ) 12,6 23,6 31,5 33,7 44,6 62,9 515

Jahresöffnungsstunden / MA (VZÄ) 193,4 255,8 392,0 524,5 727,1 1053,4 505

Fortbildungsstunden / MA (VZÄ) 0,00 0,00 7,22 9,88 20,00 46,91 483

Laufende Ausgaben / Besuch (€) 2,34 3,68 5,07 7,66 7,72 14,27 400

Laufende Ausgaben / EW (€) 2,56 5,03 8,45 12,13 17,31 20,07 560

Erwerbungsausgaben / EW (€) 0,24 0,68 1,04 1,14 1,76 2,89 568

Anteil Personalausgaben / Gesamtkosten 1% 43% 58% 66% 74% 83% 555

erwirtsch. Mittel + Fremdmittel / Gesamtausgaben 0,5% 1,0% 5,5% 9,9% 16,0% 42,5% 552

572 Bibliotheken in dieser Größenklasse Copyright © 2009 hbz.

Fina

nzen

Musterbibliothek - INDIKATORENRASTER Größenklasse: 10.000 bis 20.000 EW - Berichtsjahr: 2007

Ser

vice

Nut

zung

Per

sona

l


LibMeterLibMeter

DBS – Public Libraries Indicatorgrid „Indikatorenraster“


LibMeterLibMeter

German Library Statistics 2009New Online Exploration Tool - BibS


LibMeterLibMeter

Basic interactive Comparrison in Time Series


LibMeterLibMeter

BIX - 1-Level Univ. LibrariesRanking Result Table in Print 2006


LibMeterLibMeter

(Germany+) BIXLibrary Statistics Index - Overview

http://www.bix-bibliotheksindex.de/vergleich_wb/index.php?nID=19&updown=&sortby=rangNutzung&kategorie=&year=2009&bl=

http://www.bix-bibliotheksindex.de/vergleich_wb/index.php?nID=19&updown=&sortby=rangNutzung&kategorie=&year=2009&bl=


LibMeterLibMeter

(Germany+) BIXLibrary Statistics Index - Measures


LibMeterLibMeter

(Germany+) BIXMeasures over Years


LibMeterLibMeter

(Germany+) BIXCompare & Show „Usage“


LibMeterLibMeter

American Association of Research Libraries (ARL) – Interactive Statistics


LibMeterLibMeter

ARL – Interactive StatisticsExample: Library Service Hours


LibMeterLibMeter

ARL – Interactive StatisticsService Hours over time @ 9 Inst.

ARL – Interactive StatisticsExample: Library Service Hours


LibMeterLibMeter

National Statistics (e.g. BIX/DBS)Benefits & LimitsBenefits Broad Institutions Base Fairly Comprehensive Creates Awareness

Community Public

Online Data Warehouse Transparency Allows Meta Studies

Limits Limited Parameter Set Scope

i.e. more detailled services missing

Data completeness & quality Standardization Issues Time Lag, Annual FrequencyNational Differences International Comparability ? Interactivity/-Download Meta-Analysis Tools, Charts


LibMeterLibMeterDiskussionsfragenWann erste Ergebnisse ?Wie lange auf welche Aussagen

warten ?


LibMeterLibMeter

LibMeter Seminar

x

2. bÜbungen


LibMeterLibMeterGo to Screens List

Übungen hier


LibMeterLibMeterDiskussionsfragenWelche Komponenten können

Nationalstatistiken haben ?Möglichkeiten und Grenzen ?Wie SMART Virtual Use ?

Basics Terminology Nat Lib Stats Lib Meter Zbw Hh Workshop

Technology

Transcript of Basics Terminology Nat Lib Stats Lib Meter Zbw Hh Workshop