Post on 23-Dec-2015
OSP User Experience: Current Examples and Calls for Future
CollaborationTeggin Summers, Assistant Director ePortfolio Initiatives, Virginia Tech
Lynn Ward, Principle Systems Analyst, Indiana University Purdue University IndianapolisSteve Foehr, Director of Application Services, Rhode Island Network for Educational Technology
Janice Smith, Consultant, Three Canoes ConsultingTiffany Marra, Project Manager, Univeristy of Michigan
Noah Botimer, ePortfolio Technical Lead, University of Michigan
10th Sakai Conference - Boston, MA, U.S.A. 2
Introductions
July 2009
Teggin Summers Assistant Director of Electronic Portfolio Initiatives, Virginia Tech
Lynn Ward Principle Systems Analyst, Indiana University Purdue University Indianapolis
Steve Foehr Director of Application Services, Rhode Island Network for Educational Technology
Janice Smith Consultant, Three Canoes Consulting 2009 Sakai Fellow
Tiffany Marra Project Manager, University of Michigan
Noah Botimer ePortfolio Technical Lead, University of MichiganSakai Product Council, Member
10th Sakai Conference - Boston, MA, U.S.A. 3
Panel Objectives
• Share a wide variety of the user experiences we have encountered
• Start a conversation to discuss how we might broaden the OSP community to include more discussion of user experience (and perhaps teaching & learning?)
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 4
Brief History of OSP at Virginia Tech
• OSP tools integrated into our instance of Sakai (“Scholar”) since 2003
• Virginia Tech OSP system based on the University of Michigan Scaffolding Portfolio• However, we have extended the schemes,
layouts, navigation capability, and functions of the UMSCP to satisfy our local needs
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 5
Different Uses for our eP Tools
• eP@VT• Learning, Assessment, and Professional
Development• Training for Academic Support Groups• Co-curricular ePortfolios
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 6
eP@VT
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 7
eP@VT
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 8
eP@VT
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 9
Learning, Assessment, & Professional Development
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 10
Learning, Assessment, and Professional Development
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 11
Learning, Assessment, and Professional Development
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 12
Learning, Assessment, and Professional Development
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 13
Learning, Assessment, and Professional Development
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 14
Training for Academic Support Groups
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 15
Co-curricular ePortfolios
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 16
Challenges Our Users Have Encountered
• Definitely affected by the form slow down issue (with the help of Michigan, we found a solution to this)
• Multiple tools in multiple sites leads to confusion
• FCK Editor• “Submitting evidence” can be confusing
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 17
Additional Steps to Facilitate Use
• Documentation
• Videos• Coordination with our support groups
(i.e., Innovation Space)
• Channel student energy
• Website: http://eportfolio.vt.edu July 2009
10th Sakai Conference - Boston, MA, U.S.A. 18
OSP AND PROGRAM ASSESSMENT:LOWERING THE BAR TO PARTICIPATIONLynn Ward, Principle Systems AnalystIndiana University
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 19
OSP Use @IU
• > 20 active portfolio projects underway, many in professional schools
• Majority focus on outcomes assessment at the program level
• Some disciplines (Education, Nursing, Visual Communication) embrace portfolio pedagogies
• Others (Engineering, Computer Science) do not)
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 20
Barriers to Adoption @ IU
• Difficult to achieve faculty consensus and buy-in across entire program
• Perceptions that application is too complicated, difficult to use
• Concerns about faculty workload• Reports and templates require custom
programming
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 21
Removing the Barriers
• Difficult to achieve faculty consensus and buy-in across entire program
• Perceptions that application is too complicated, difficult to use
• Concerns about faculty workload• Reports and templates require custom
programming
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 22
Design and Development Goals
• Simplicity - simplify workflows for all roles, especially faculty and student participation in program assessment projects
• Flexibility - accommodate a wide range of approaches to portfolio use and assessment
• Standardization – define and implement standards to facilitate reporting
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 23
Simplified Workflowfor Program Assessment
Instructor and student never leave course site/tools:1. Coordinator associates course site(s) with portfolio
site2. Students in associated site automatically added to
portfolio site3. Instructors create and link assignments to matrix
cells4. Students submit linked assignments5. Evaluator(s) review and evaluate linked assignment
submissions.6. Report results
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 24July 2009
Associate Course Sites
1
2
10th Sakai Conference - Boston, MA, U.S.A. 25
Roster Syncing
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 26
Roster Sync Auto Groups
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 27
Synced Roster
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 28
Instructor Links an Assignment: 1
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 29
Instructor Links an Assignment: 2
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 30
Student Submits Assignment
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 31
Submitted Assignment in Matrix: 1
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 32
Submitted Assignment in Matrix: 2
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 33
Submitted Assignment in Matrix: 3
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 34
Evaluating Linked Assignments
• Add evaluator(s) to class group in portfolio site
• Convert cell to pending for entire class
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 35
Pending Cells in Evaluations Tool
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 36
Evaluations can be hidden…
from students
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 37
Evaluation can be hidden…
from other evaluators
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 38
Report and Analyze Results
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 39
Additional Workflow Support
• Participant can “request feedback”• Email notifications:
• For Reviewer when Participant requests feedback
• For Evaluator when Participant Submits for Evaluation
• For Participant when feedback or evaluation is added to cell.
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 40
Future Directions
• Evaluation directly within Assignments tool• Enable linking in other Sakai tools• Long term storage for all portfolio evidence, including
linked items• Improved scalability and performance • Advanced and custom reporting• Advanced multimedia support (including a turn-key
video solution) • Simplified site management and scaffolding authoring
tools.
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 41
RI Proficiency Based Graduation Requirements
• Standards Based• Applied Learning
• Problem solving, critical thinking, research, communication, and reflection/evaluation
• Student choice/Responsibility• Proficiency for Graduation
July 2009
10th Sakai Conference - Boston, MA, U.S.A. 42
RI Proficiency Based Graduation Requirements
• Standards Based
• Curriculum Development
• Student Artifacts
July 2009
Ballston Spa Central School DistrictOSP Implementation
Janice Smith Three Canoes Education Consultant
Ballston Spa Central School District
• BSCSD located near Albany, New York• Hands-on coordination by Superintendent
Joe Dragone, Ph.D.• Planning group includes:
• IT Coordinator• School Principals• Lead Teachers for Each Grade• Guidance Counselors
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 44
Project Background / History
• IBM Innovation Retreat in Albany in 2007
• October 22nd 2008 IBM Open Education Services Briefing
• December 4th 2008 OSP / Site Visit in RI (RINET)
• Spring 2009 Phase I Ballston Spa Competency Mapping
• Summer 2009 Phase II OSP Customization
• Future / TBD – OSP Training/Support & Staff Professional Development
BSCSD Competency Mapping ProjectHigh Level Project Plan
Mar - April Sept -Nov Jan - Mar April - TBD
Phase IPreparatory: a) gather information; b) determine work plan and responsibilities; c) perform mapping activities On-site Mapping Activity meetingsFollow-up activities for finalization and delivery of Competency Mapping Report
Phase IIRequirement GatheringOSP Design and Customization
Reports
Phase IIIDocumentation and Creation of Training Materials
On-site Function and Technical Training
Phase IVFollow-on Support
On-site Mapping Activity meetings
Project Months 2009 - 2010
Competency Mapping Project Phases
• Phase I Curriculum Mapping• Mapping BSCSD Graduate Competencies to
NYS Performance Indicators• Mapping NYS Performance Indicators to BSCSD
9th Grade ELA and 5th Grade curriculum • Phase II Sakai/OSP Customization
• 9th Grade ELA (English Language Arts)• 5th Grade (4 integrated content areas)
Phase I Deliverables: Curriculum Maps
9th Grade English Language Arts Curriculum Map9th Grade ELA NYS Performance Indicator Code
9th Grade ELA NYS Performance Indicator Text
Ballston Spa Central School District Graduate Competencies
Sample Topics / Activities
Sample ELA Assessments
Available Resources
5th Grade Curriculum Map5th Grade NYS Performance Indicator Code
5th Grade NYSPerformance Indicator Text
Ballston Spa Central School District Graduate Competencies
Sample Topics / Activities
Sample 5th Grade Assessments
Available Resources
Plus calendar component
Phase II Project Objectives
• Gather OSP requirements via Ballston Spa site visit
• Confirm OSP requirements via follow-up report
• Design OSP data structures according to requirements
• Set up OSP data structures and workflow on the Ballston Spa server (running rSmart Sakai CLE)
• Conduct on-site user acceptance testing
OSP Workflow Design
PortfolioSite
GoalSite
Assignments
Goal 2Portfolio
Report
Goal 1
Goal 3
Matrix ReflectionForm
GraduateCompetencies
EvaluationForm
NYS PerformanceIndicators
Portfolio and Goal Sites
Portfolio sites for each grade level course and class
Goal sites (using project sites) for each grade level
Assignments
• Assignments preloaded in draft form according to grade level curriculum maps
• Assignments linked to NYS Performance Indicators• Teachers select and flesh out assignments• Students complete and submit assignments
Matrices and Forms
• Submitted assignments appear in matrix cells according to BSCSD Graduate Competencies mapped to linked NYS Performance Indicators
• Students reflect upon their work for each Graduate Competency and submit reflection cells for evaluation
• Teachers use evaluation forms and evaluations tool to rate and comment on student work for each Graduate Competency
Portfolio Templates
• Students create a portfolio to share their work on BSCSD Graduate Competencies
• Exact design of portfolio template for each grade level to be determined• Students will be able to display their matrix reflections and evaluations in
relation to assignments and other artifacts for each Graduate Competency
Reporting Options
Plans for BSCSD Reporting• Fall 2009 – Community development work will allow
direct query of form information for reporting• Reports design will focus on student mastery of BSCSD
Graduate Competencies accessed via evaluation forms• Reports will include data on
• Individual students• Aggregation of all students within
• Each course• Each grade level within and across schools• Each school• Across the Ballston Spa Central School District
Administrative Concerns:
• My Workspace Tools• Fewer to begin
with• User account
features• No use of email
(point to nowhere to begin)
• Read-only account tool
• Teacher collaboration sites• Sharing of
assignments• Communication
tools
Looking to the Future
• Proof of concept project• Intention to expand to more grade levels• Importance of teacher and student buy-in
• 9th grade ELA• 5th grade
• Modeling use of Sakai/OSP for K-12• Making customized OSP data structures
available to similar K-12 projects
10th Sakai Conference - Boston, MA, U.S.A. 58
Contact Us
July 2009
Teggin Summers Assistant Director of Electronic Portfolio Initiatives, Virginia Techteggin@vt.edu
Lynn Ward Principle Systems Analyst, Indiana University Purdue University Indianapolis leward@iupui.edu
Steve Foehr Director of Application Services, Rhode Island Network for Educational Technologysteve@staff.ri.net
Janice Smith Consultant, Three Canoes Consulting 2009 Sakai Fellowjanice.smith@threecanoes.com
Tiffany Marra Project Manager, University of Michigantmarra@umich.edu
Noah Botimer ePortfolio Technical Lead, University of MichiganSakai Product Council, Memberbotimer@umich.edu