Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha...
-
date post
21-Dec-2015 -
Category
Documents
-
view
217 -
download
0
Transcript of Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha...
![Page 1: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/1.jpg)
Can we quantify the library’s influence?
Creating an ISO standard for impact assessment
Roswitha PollChair of ISO TC 46 SC 8: Quality – statistics
and performance evaluation
Roswitha PollChair of ISO TC 46 SC 8: Quality – statistics
and performance evaluation
PM 9
![Page 2: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/2.jpg)
ISO standards for quantity and quality in libraries
ISO Technical Information and committee 46: documentation
Subcommittee 8: Quality - statistics and performance evaluation
WG 2: International library statistics
ISO 2789 (2006) International library statistics,4th edition Revision started March 2010
WG 4: Performance indicators for libraries
ISO 11620 (2008) Library performance indicators, 2nd editionRevision started December 2010
WG 7: Quality measures for national libraries
ISO TR 28118 (2009) Performance indicators for national libraries
![Page 3: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/3.jpg)
WG 8: Statistical data for library buildings
ISO TR 11219 Qualitative conditions and basic statistics for library buildings (Voting ends August 25, 2011)
WG 9: Statistics and quality issues for web archiving
ISO TR 14783 Statistics and quality issues for web archiving Work started December 2009
ISO Technical Information and committee 46: documentation
Subcommittee 8: Quality - statistics and performance evaluation
![Page 4: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/4.jpg)
WG 10: Methods and procedures for assessing the impact of libraries
• 17 experts
• 13 countries
• 17 experts
• 13 countries
ISO 16439 Methods and procedures for assessing the impact of libraries
Work started December 2010
![Page 5: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/5.jpg)
Library services aim ultimately at effects that are difficult to identify, e. g.
Library services aim ultimately at effects that are difficult to identify, e. g.
Can the assessment of such effects be „standardised“? Can the assessment of such effects be „standardised“?
• social inclusion • knowledge, life-long learning• free access to information• cultural awareness• local culture and identity• individual well-being
„measuring the unmeasurable“
„nailing jelly to the wall“
„six impossible things bevor breakfast“
![Page 6: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/6.jpg)
ISO 16439 aims at
• standardising the terminology and definitions for impact assessment
• describing and harmonising the methods that have been tested and found to deliver meaningful results
ISO 16439 aims at
• standardising the terminology and definitions for impact assessment
• describing and harmonising the methods that have been tested and found to deliver meaningful results
A standard is not• a theoretical description of a topic,• a practical handbook with many examples.
![Page 7: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/7.jpg)
Terminology
output outcome impact value
benefit
Terminology
output outcome impact value
benefit
Outcomes Outcomes are the short- to medium- term results of applying are the short- to medium- term results of applying outputsoutputs Impact Impact is the effect of the is the effect of the outcomesoutcomes on the environment and is usually long-term on the environment and is usually long-term
(Sue Howley)(Sue Howley)
![Page 8: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/8.jpg)
inputcontribution of resources in support of a library (e. g. funding, staff, collections, space, equipment)
processset of interrelated or interacting activities which transforms inputs into outputs (e.g. cataloguing, lending, reference service)
outputproducts of library processes (e. g. number of titles catalogued, number of loans, number of information requests)
impactdifference or change in an individual or group resulting from the contact with library services
valuethe importance that stakeholders (funding institutions, politicians, the public) attach to libraries (monetary value may be included)
input
process
output
impact
value
outcomedirect, pre-defined effects of the output related to goals and objectives of the
library’s planning (e. g. number of users, user satisfaction levels)
![Page 9: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/9.jpg)
impact
difference or change in an individual or group resulting from the contact with library services
The change can be
• tangible or intangible,
• direct (immediate) or long-term,
• small or far-reaching,
• intended or unintended,
• actual or potential.
impact
difference or change in an individual or group resulting from the contact with library services
The change can be
• tangible or intangible,
• direct (immediate) or long-term,
• small or far-reaching,
• intended or unintended,
• actual or potential.
and hopefully beneficial
![Page 10: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/10.jpg)
impact
• on individuals - changes in skills and competences
- changes in attitudes and behaviour - higher success in research, study, or career - individual well-being
• social impact
- social inclusion - free access to information - education, life-long learning - local culture and identity - health care
• economic impact - ROI, cost-benefit - influence on economic life (local, regional, etc.)
impact
• on individuals - changes in skills and competences
- changes in attitudes and behaviour - higher success in research, study, or career - individual well-being
• social impact
- social inclusion - free access to information - education, life-long learning - local culture and identity - health care
• economic impact - ROI, cost-benefit - influence on economic life (local, regional, etc.)
![Page 11: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/11.jpg)
Data that can indicate an impact
1. Statistics (libraries, cultural statistics, etc.)
Example: Increase in attendants of training lessons
Data that can indicate an impact
1. Statistics (libraries, cultural statistics, etc.)
Example: Increase in attendants of training lessons
The data must be validated by other The data must be validated by other methodsmethods
2. Performance indicators (especially indicators concerning usage)
Example: Higher market penetration in a target group
2. Performance indicators (especially indicators concerning usage)
Example: Higher market penetration in a target group
3. Results of user satisfaction surveys (perceived quality of library services)
Example: High satisfaction with reference service
3. Results of user satisfaction surveys (perceived quality of library services)
Example: High satisfaction with reference service
![Page 12: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/12.jpg)
1. Solicited evidence (qualitative or soft methods)
surveys (in-house, telephone, mail, online)
street surveys
interviews
focus groups, discussion groups
self-assessment of users
collected anecdotal evidence
The results must be made quantifiable to show patterns of impact
1. Solicited evidence (qualitative or soft methods)
surveys (in-house, telephone, mail, online)
street surveys
interviews
focus groups, discussion groups
self-assessment of users
collected anecdotal evidence
The results must be made quantifiable to show patterns of impact
Methodsfor
assessing impact
![Page 13: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/13.jpg)
2. Observed evidence (quantitative methods)
1. library use compared to success
2. observation
3. data mining
4. tests for ascertaining an increase of skills and
competences (e. g. before and after user training)
Methods 1 to 3 do not always yield statistically valid results and can be affected by data protection regulations.
2. Observed evidence (quantitative methods)
1. library use compared to success
2. observation
3. data mining
4. tests for ascertaining an increase of skills and
competences (e. g. before and after user training)
Methods 1 to 3 do not always yield statistically valid results and can be affected by data protection regulations.
Methodsfor
assessing impact
![Page 14: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/14.jpg)
Economic impact
1. Library benefits expressed in economic terms
- Contingent valuation
- Surrogate prices - Time spent on library services (calculated in salaries)- Time saved by library services
2. Impact of the library on the economies of the
local or regional surroundings
Economic impact
1. Library benefits expressed in economic terms
- Contingent valuation
- Surrogate prices - Time spent on library services (calculated in salaries)- Time saved by library services
2. Impact of the library on the economies of the
local or regional surroundings
Methodsfor
assessing impact
![Page 15: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/15.jpg)
Problems of identifying the impactof libraries
The library’s influence is generally not the only and
possibly not the strongest one.
Impact of the same library service can vary in different user
populations and in different cultural and economic
surroundings.
The impact is for the most part intangible and difficult to
quantify.
Long-term effects cannot be ascertained if the users are
no more available.
The results of qualitative methods have a subjective bias.
Libraries are not familiar with the methods used for impact
assessment.
The expenditure of time and effort can be considerable
![Page 16: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/16.jpg)
Impact surveys
(examples in an Annex to the standard)
Surveys developed 2009 in the project “global statistics” (IFLA, ISO, UNESCO), separate for academic and public libraries
Tested in 2010 in 4 Italian public libraries and 1 Spanish academic library
Survey should be as short as possible, not more than 15 minutes to fill out
A simple survey with wide applicability
Impact surveys
(examples in an Annex to the standard)
Surveys developed 2009 in the project “global statistics” (IFLA, ISO, UNESCO), separate for academic and public libraries
Tested in 2010 in 4 Italian public libraries and 1 Spanish academic library
Survey should be as short as possible, not more than 15 minutes to fill out
A simple survey with wide applicability
![Page 17: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/17.jpg)
Introductory QuestionsIntroductory Questions
How often do you visit the library (or its online services)?
(Why do you not visit the library?)
What do you do when you visit the library (or its online services)?
![Page 18: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/18.jpg)
Have you benefited from visiting the library or using its online services?
Have you developed new skills? Have you obtained new ideas, new interests? Have you got helpful information for
- school and learning? - job seeking? - health and well-being? - business and commerce? - about your community? Has the library helped you to save time? Have you enjoyed the visits; have you felt comfortable? Have you experienced the library as safe and quiet place? Have you made contacts with other people?
Have you benefited from visiting the library or using its online services?
Have you developed new skills? Have you obtained new ideas, new interests? Have you got helpful information for
- school and learning? - job seeking? - health and well-being? - business and commerce? - about your community? Has the library helped you to save time? Have you enjoyed the visits; have you felt comfortable? Have you experienced the library as safe and quiet place? Have you made contacts with other people?
Public libraries
![Page 19: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/19.jpg)
Have you benefited from visiting the library or using its online services?
Have you developed new skills, e.g. in information seeking? Have you obtained new ideas, new interests? Has the library supported you in studying? Has the library supported you in your research? Has the library helped you to save time? Has the library helped you to be better in examinations? Has the library helped you to be better in your job? Have you experienced the library as safe and quiet place for
study and reading? Have you made contacts with other people?
Have you benefited from visiting the library or using its online services?
Have you developed new skills, e.g. in information seeking? Have you obtained new ideas, new interests? Has the library supported you in studying? Has the library supported you in your research? Has the library helped you to save time? Has the library helped you to be better in examinations? Has the library helped you to be better in your job? Have you experienced the library as safe and quiet place for
study and reading? Have you made contacts with other people?
Academic libraries
![Page 20: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/20.jpg)
If this library did not exist, could you have got the same information and help elsewhere?
From another library?
Via the Internet?
From teaching staff?
From colleagues, friends etc.?
In the media (newspapers, radio, television)?
Perhaps, but it would have taken more time and
effort
No, I do not think I would have got the same
information
From another library?
Via the Internet?
From teaching staff?
From colleagues, friends etc.?
In the media (newspapers, radio, television)?
Perhaps, but it would have taken more time and
effort
No, I do not think I would have got the same
information
All libraries
Time and effort saved can mean costs saved (very interesting for funding institutions)Time and effort saved can mean costs saved (very interesting for funding institutions)
![Page 21: Can we quantify the library’s influence? Creating an ISO standard for impact assessment Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d615503460f94a43c0d/html5/thumbnails/21.jpg)
Personal well-being will be furthered by areas for relaxation