NISO Apr 29 Virtual Conference: Assessing Game-Based Library Initiatives Kyle Felker, Digital...
-
Upload
national-information-standards-organization-niso -
Category
Education
-
view
525 -
download
4
Transcript of NISO Apr 29 Virtual Conference: Assessing Game-Based Library Initiatives Kyle Felker, Digital...
ASSESSING GAME-BASED PROGRAMMINGIssues, challenges, and models to follow
WHO I AM
Kyle Felker
Grand Valley State University Libraries
Digital Initiatives Librarian
Investigates, prototypes, integrates
[email protected], @gwydion9
STRUCTURE OF THIS TALK
What games can do for us
Basic assessment principles
Unique Challenges and approaches
LibraryQuest assessment program
WHY GAMES?
Greater, lengthier, or more focused engagement with facilities, staff, services, etc.
Reaching one or more learning objectives
GENERAL PRINCIPLE : SET CLEAR AND MEASURABLE GOALS
Engagement based-goal: Increase participation in our summer reading program by at least 50%
Learning goal: To teach incoming undergraduates how to locate books using the online catalog and how to find them on the shelf.
Skills-based vs Conceptual
GENERAL PRINCIPLE: CONTEXTUALIZE AND HARMONIZE GOALS
Should be in accord with institutional goals and objectives
Should not duplicate other programs
Should take into account national and institutional standards, frameworks, etc.
EVALUATION VS. ASSESSMENT
Evaluation: How can we improve this?
Assessment: Did this do what it was supposed to do?
Evaluation often easier, mistaken for assessment
Assessment often requires more effort and thought
GENERAL PRINCIPLE: BUILD ASSESSMENT INTO PROJECTS AT INCEPTION
Consider assessment plan as part of overall plan
Design, schedule, and plan opportunities for both evaluation and assessment
Secure resources such as staff time and money
Initial pilot: evaluation, subsequent pilot: assessment
GENERAL PRINCIPLE: INVOLVE OTHERS
What do you needs to know/do to fulfill your assessment plan?
Do you have all the necessary skills?
Who does, and can you get them to help you?
GAME-SPECIFIC CHALLENGES
The conundrum of “fun”
Asynchronous games
Complexity
INTEGRATING ASSESSMENT INTO GAMEPLAY
Can interactions include assessment components?
Can players complete assessment as part of gameplay?
What information can be collected unobtrusively? By staff? By the system?
ELECTRONIC GAMES AND METRICS
Number of players or interactions
Length of interaction
Path through the game or app
Privacy concerns
CASE STUDY: LIBRARYQUEST
LIBRARYQUEST
Asynchronous mobile game
Offers players “quests” or tasks
Complete for points, which can be turned in for prizes
Learn by doing: accomplishing quest = successfully using skill
Second pilot: Assessment focused
OBJECTIVES:
To teach incoming freshmen ten distinct skills and knowledge-based goals relating to library services and spaces (example: how to use the online catalog to locate a book).
To teach freshman that the library is a support service that exists to help them in their work.
To increase freshman sense of self-sufficiency and confidence in using the library.
INSTITUTIONAL AND NATIONAL CONTEXT
ACRL Information Literacy Standards: PK Morgan Critique “Pausing at the Threshold.”
Institutional IR assessment
Research on Student Retention
Emphasis on incoming freshmen
OUTSIDE COLLABORATORS
Head of Instructional Services
First-Year Experience Librarian
Statistical Consulting Center
ASSESSMENT COMPONENT: LEARNING GOALS/STUDENT CONFIDENCE
Student selects quest
Quest app assesses student ability through skills-based quiz
Exposes results to student
Quest asks student to rate confidence on a ten-point scale
Student performs quest
Quest app asks student to rate confidence once again using ten point scale
App rewards student
EXAMPLE QUESTIONS: CALL NUMBER QUESTWhich of the following is shelved first?
EA 301.E799 1990
E 302.E799 1989
E 302.A799 1999
Which part of this call number designates a general subject area?
HK
302.1
.E99
1990
On a scale of 0-9, rate how confident you are that you could find a book on the shelf using a call number:
Not Confident 0 1 2 3 4 5 6 7 8 9 Very Confident
ASSESSMENT: IDENTIFYING LIBRARY AS SUPPORT SERVICE
Pre and post-play questions:
Which of the following offer services that support you in completing your assignments?
Registrar’s office
Writing Center
Student life office
University Libraries
EVALUATIVE DATA
Statistics:
Number of players
Number of quests completed
Which quests completed and abandoned
What steps completed before abandoned
When completed
What rewards were selected (and how many)
EVALUATION COMPONENTS
Periodic surveys throughout the game on elements of gameplay:
“What is your favorite quest so far? Why?”
Post-game Interviews
“If you could tell the designer of this game any one thing, what would it be?”
Questions?