Design Implications of the xAPI (“Tin Can API”) mLearnCon, San Jose, CA, June 19, 2013 Mr. Jason Haag, Research Analyst, Mobile Learning Lead The Tolliver Group, Inc. SETA Support for ADL

2

‣  ADL Initiative was established in 1999 by Presidential Executive Order 13111. The first two ADL Co-Labs opened in Alexandria, VA and Orlando, FL that same year.

Advanced Distributed Learning Background and Vision

3

“Provide access to the highest quality education and training, tailored to individual needs, delivered cost effectively, anywhere and anytime.”

- ADL Vision

4

ADL Global Reach

Canada ADL Partnership Lab Kingston, ON, Canada

Norway ADL Partnership Lab

Oslo, Norway

United Kingdom ADL Partnership Lab

Buckinghamshire, United Kingdom

Korea ADL Partnership Lab

Seoul, Korea Romania ADL

Partnership Lab Bucharest, Romania

Latin America and Caribbean Regions ADL Partnership Lab Mexico City, Mexico

NATO ACT ADL Partnership Lab

Norfolk, VA

ADL Initiative

Orlando, FL

Academic ADL Co-Lab Madison, WI

ADL Initiative Alexandria, VA

ADL Center for Intelligent Tutoring Systems Research

and Development Memphis, TN

Poland ADL Partnership Lab Warsaw, Poland

Design Implications of the xAPI

‣  Background

‣  Introduction to xAPI & LRS

‣  Design Implications

‣  SCORM

‣  Instructional & Learning Design

‣  User Experience (UX) & Interface

‣  Reporting & Analytics

‣  Examples

Agenda

6

“So some things get invented because it is suddenly possible to invent them.”

- Vint Cerf

Background

‣  Sharable Content Object Reference Model (SCORM)

•  Created to support courseware interoperability and reusability within Learning Management Systems (LMS)s

•  Enables online instruction in a (desktop) browser only

•  Need for Mobile SCORM! (mLearnCon 2010-2011): Earliest attempts on mobile circa 2004 (PocketSCORM)

•  Works for desktop browsers (and will work in the future), but we now have many more use cases to support

What Are the Use Cases?

Next Gen “SCORM”

Personalized and Brokered Content • Just-in-Time • Just-for-You

Access From Any Device

Learn From: • Intelligent Tutors • Mentors and peers via social networks • Self-Discovery

Learn Using: • Games • Virtual Worlds • Intelligent content

SCORM “- ilities” Conceptual Starting Point (High-level requirements) ‣  Accessibility

•  Ability to locate and access instructional components from multiple locations and deliver them to other locations

‣  Interoperability •  Ability to take instructional components developed in one system and

use them in another system

‣  Durability •  Ability to withstand technology changes over time without costly redesign,

reconfiguration, or recoding

‣  Reusability •  Ability to use instructional components in multiple applications, courses,

and contexts

Background

‣  Result of Project Tin Can

‣  Tracks experiences, informal learning, real-world experiences (not just completions)

‣  Activities can be tracked at any level of granularity

‣  Allows data storage and retrieval (ex. 3rd party reporting and analytics tools)

‣  Enables tracking mobile, games, ITS, and virtual worlds experiences

‣  Collaboration of industry, government, and academia

‣  Developed by open source community

What is the Experience API?

Project Tin Can

‣  ADL BAA Funded Research Effort

‣  Code-named “Project Tin Can” Rustici Software

‣  Requirement to “modernize” SCORM communication framework (but created new opportunities)

‣  Community-derived use cases, interviews, over 100 white papers

Project Tin Can Top 5 Requirements

1.  Should support a variety of content types

2.  Should be simple to implement

3.  Should support offline or disconnected scenarios

4.  Portable content (interoperable across platforms)

5.  Improved access to run-time data

Experience Tracking

(xAPI)

Learner Profile

Content Brokering

Competency Networks

Comparing TLA and the SCORM

Run Time Environment (RTE)

Content Aggregation

Model (CAM)

Sequencing &

Navigation (S&N)

Training & Learning Architecture (TLA)

Any Learning Activity"On Any Platform

Web Browser-based Instruction

Comparing xAPI and the SCORM RTE

‣  Course tracking: Bookmarking, completion, time, pass/fail, scores

‣  Multiple scores per object, unlimited test results and interactions

‣  Secure

‣  No LMS required

Comparing xAPI and the SCORM RTE

‣  No web browser required

‣  Supports offline scenarios

‣  Control over your content

‣  Tracks web or native apps

‣  Tracks serious games, simulations, virtual worlds

‣  Tracks real-world, informal learning & performance

‣  Tracks team-based learning

How Do We Track All Of These Activities?

Credit: Liz Burow (@Burlix)

Experience API (xAPI)

‣  Transfer of data between learning systems

‣  Capture learning data from any system (other than LMS applications such as HR systems, etc.)

‣  Learning does not have to originate in an LMS (learning takes place often outside of the LMS on the web, with mentors, videos, classrooms, conferences, etc.)

‣  Correlation of general activity with learning data (job performance data with learning data)

‣  Specialized analytics (reports that weren't previously possible)

Key Enablers

19

Activity Streams ‣  http://activitystrea.ms

‣  Collaboration between Google, Facebook, Microsoft and others

‣  Allows reporting of experiences, not just completions

‣  Format: <Actor> <Verb> <Object> (I did this):

•  Simple Statement: I (actor) watched (verb) a video on protecting employee data (activity)

•  Complex: in the context of [information assurance certification training] with result [timestamp:2013-0618T18:30:32.360Z ].

Example (Formal) Learning Stream ‣  John Connor attempted “The War of 1812, Part 1”

‣  John Connor watched “The Battle of New Orleans Video”

‣  John Connor attempted “The War of 1812, Assessment”

‣  John Connor answered “Question 1” with “True”

‣  John Connor answered “Correctly”

‣  John Connor answered “Question 2” with “False”

‣  John Connor answered “Correctly”

‣  John Connor answered “Question 3” with “a”

‣  John Connor completed “The War of 1812, Assessment”

‣  John Connor scored “90%” on “The War of 1812, Assessment”

‣  John Connor satisfied objective “Battles of the War of 1812”

‣  John Connor mastered objective “The War of 1812” to level “1”

‣  John Connor earned “The War of 1812 – Level 1 Badge”

Example (Informal) Learning Stream ‣  Walter Isaacson authored [Steve Jobs Biography]

‣  David proofread [Steve Jobs Biography]

‣  Margaret approved [Steve Jobs Biography]

‣  Simon & Schuster published [Steve Jobs Biography]

‣  John purchased [Steve Jobs Biography]

‣  Ted read [Steve Jobs Biography]

‣  Philip shared [Steve Jobs Biography]

‣  Steven liked [Steve Jobs Biography]

‣  Harry reviewed [Steve Jobs Biography]

Where Is The Data Stored?

Learning Management System (LMS)

User Management

Course Management

Run-Time

Player

Scheduling

Grade Book

Search

Delivery

Learning Records

Statistics

Repository

Sequencing

Preferences Assessment Reports

What is a LMS?

23

Learning Management System (LMS)

What is a LRS?

24

Learning Record Store (LRS)

Learning Records

Learning Record Store (LRS)

Reporting Systems

Assessment Services

Semantic Analysis

Statistical Services

25

Act on It

LRS = Learning Record Store

Design Implications

‣  No. xAPI can modernize the SCORM RTE, but doesn’t replace all aspects of SCORM

‣  A SCORM xAPI verb profile is in the works!

‣  Implication: It can easily work with SCORM. It does not replace SCORM. New SCORM content developed could use the xAPI.

Does xAPI replace SCORM?

Design Implications

‣  No. SCORM is still useful for tracking web-based desktop courses.

‣  If your learners need mobile access, SCORM will not work alone. SCORM will continue to work (durability) in your LMS.

‣  Implication: Is your current investment in SCORM producing good results? Are you able to accurately measure your ROI with current LMS reports?

Does this mean all of the time and money we invested in SCORM content will be worthless?

Design Implications

‣  Only if there is a requirement.

‣  xAPI isn’t a new version of SCORM.

‣  Need to support mobile delivery of your course? Don’t convert. Think mobile first.

‣  Implication: Is there a need? You could easily make your SCORM content support the xAPI. But determine your needs. Is it for more robust reporting/analytics or mobile? If so you will need to setup a a LRS and also redesign your content user interface (if mobile).

Will we need to convert our training content from SCORM to the xAPI?

Design Implications

‣  Yes. The SCORM CAM and metadata can still be used.

‣  An xAPI Content Package Profile is on the roadmap. Tin Can demonstrated a draft profile.

‣  Implication: SCORM is not optimized for mobile so xml parsing of large manifests, metadata, etc may not be appropriate for other learning environments outside the LMS. Is Reusability and Portability a requirement?

Can SCORM content packaging work with xAPI?

Design Implications

‣  Correlation to score on an assessment or job performance

‣  Videos within a course (or independent of a course)

‣  Implication: SCORM CMI interaction data was limited and LMS applications didn’t support test item analysis consistently. Still requires a systematic design strategy for where your LRS(s) will reside and communicate, and which will be authoritative sources.

How can learner interactions and performance be analyzed?

Instructional Design

We are all experiencing a major paradigm shift to think not only about instructional design, but to consider the broader impact on performance and the learner.

Learning Design

‣  How might we redesign our approach to curriculum development and delivery to center around the needs of the learner?

‣  Combine the best attributes from disciplines of HCD, UX, ISD, etc.

33

Bloom’s Revised Taxonomy (Kathy Shrock)

Moving Away from Linear Thinking

The interlocking of the cognitive process

Design Thinking

‣  Term first emerged in 80s (rise of human-centered design)

‣  Ability to combine empathy for the context of a problem

‣  Creativity in the generation of insights and solutions

‣  Rationality to analyze and fit solutions to the context

‣  Divergent (ideate) and Convergent (realize)

Design Thinking ‣  7 Steps (non-linear: can occur simultaneously and

iterative)

•  Define, research, ideate, prototype, choose, implement, and learn

‣  Interaction between designers and stakeholders enhance alignment

‣  Feedback loops help inform intuition over time

Design Implications

‣  Visualize large datasets and make informed decisions

‣  Plan your learning according to previously travelled “expert” paths

‣  Track user progress, analyze, and group users to personalize instruction & learning

‣  Use live data to drive business decisions

‣  Report data which was not easily gathered previously

‣  GOAL: Improve PERFORMANCE

Use Data Analytics

Design Implications Think Outside the LMS Interface

Design Implications Browser Bookmarklet (formal, informal, nonformal)

‣  Tappestry (social, informal)

Design Implications Social & Informal (Float)

Design Implications Gamification – BlackBerry Gamification (OnPoint)

Design Implications Gamification – BlackBerry Gamification (OnPoint)

xAPI statements sent to partner

system

‣  Procedural tasks (guided response)

‣  Evaluation by instructor, mentor may be required for accurate assessment

‣  Mobile capture using camera?

‣  Using sensors to report xAPI Statements?

Design Implications Psychomotor Domain

Arduino + ElectricImp + xAPI, LRS

What’s next?

‣  Foster vendor/user community and tool ecosystem

‣  Attract early adopters to build tools & services

‣  Create open source software libraries, tools, best practices, and examples

‣  Write articles to illustrate new capabilities enabled by using the xAPI

‣  Produce prototypes & examples using xAPI

‣  Gather feedback – incorporate community input

‣  Version 1.0 released on 26 April 2013

xAPI Resources

‣  xAPI Design Implementer’s WG: xapi-design@adlnet.gov

‣  xAPI Resources: http://xapi.adlnet.gov

‣  ADL GitHub: http://www.github.com/adlnet

Related Projects

‣  The MoTIF Project

‣  ISD & Learning Design for Mobile

•  Alternative Approaches

•  Device Affordances

•  Workflow Process

‣  http://motif.adlnet.gov

‣  adlmobile@adlnet.gov

Wednesday, June 19, 2013 4:00 PM - 5:00 PM | Session #708

Your Design Challenges?

47

Let’s Connect!

@ADLmobile

Jason Haag Mobile Learning Lead ADL Technical Team The Tolliver Goup, Inc jason.haag.ctr@adlnet.gov Twitter: @mobilejson