w w w . h a m k . f i

Using Kinect to Promote Learning and Interaction

Leena Koskimäki 5.12.2013

Openness Accelerating Learning Networks

w w w . h a m k . f i

THE PROJECT

w w w . h a m k . f i

"Openness Accelerating Learning Networks“ project (2012-2014)

peer education collegial support mobile services

effective learning networks new learning environments

continous learning distance education sharing

new tools for participation and learning

augmented reality communality open source

co-creating virtual worlds open decision-making

Collaborative, participatory and networked work culture in educational institutions and organizations and among their stakeholders

w w w . h a m k . f i

Subproject: 3D and mobile environments for participation and learning

In cooperation with the University of Tampere, the Salpaus further education unit and the Educational Association of Citizens' Forum

The goal: to develop augmented and virtual reality for educational and participational use

w w w . h a m k . f i

The goals in HAMK University of Applied Sciences

To implement pilots to study the possibilities of new technologies and services as tools for learning and participation.

HAMK pilots implemented in an authentic learning environment - Virvelinranta Development and Resource Centre for the Disabled, Finland.

w w w . h a m k . f i

THE CRUCIAL QUESTION: HOW TO ENHANCE LEARNING IN THE WELFARE PROGRAM WITH THE NEWEST TECH AND VIRTUAL OR AUGMENTED REALITY?

w w w . h a m k . f i

KINECT

w w w . h a m k . f i

Kinect – motion sensing input device

• Launched by Microsoft in late 2010 as a bodily control for the Xbox game console

• A version for Windows on early 2012• Key features:

– Skeletal and facial tracking– Gesture recognition – Voice recognition

| http://www.microsoft.com/en-us/kinectforwindows/discover/features.aspx

w w w . h a m k . f i

Why Kinect?

• Visual, experimental environments for learning and participation in 3D

• The approachable technology facilitates the welfare sector students', mostly new to Kinect, orientation.

• All interaction at the user interface takes place in ways natural to human beings: gestures, facial expressions and speech.

• Suitable for various user groups who may not be able to use technically demanding user interfaces at all due to disabilities relating to their cognitive, motion, motor or sensory functions.

w w w . h a m k . f i

Kinect as the control unit in the augmented and virtual reality applications

• A Virvelinranta client bowling virtually.• The user sees digital data and content combined to user’s surrounding

environment.

w w w . h a m k . f i

PILOTS IN THE VIRVELINRANTA SERVICE CENTRE FOR DISABLED PEOPLE

w w w . h a m k . f i

The pilots in Virvelinranta resource centre

• Using Kinect seen as a method of practicing the directing skills and interaction– Welfare students in work placement in Virvelinranta and on

different courses directed game sessions for disabled clients

• Gathering knowledge of how the technology works and supports learning, participation and interaction– Commercially available games and tailor-made apps were

played

• Not all of the students were familiar with directing groups, with the disabled people or at least using Kinect

w w w . h a m k . f i

The progress of the pilots

Preparation

Directing acitivities,

observation

Surveys, feedback,

discussionsEvaluation

Development

w w w . h a m k . f i

”

” It was great to be a part of this pilot and see the joy and the empowerment of the clients after successing.

Ruut, crafts and recreation student

w w w . h a m k . f i

The goals of the IT-students' software development project

• To design and develop accessible, tailor-made apps, based on the feedback collected during the pilots - to create more functional apps to focus on the directing process

• To understand the needs of the app users (students, staff, customers)

• To understand the possibilities and possible restrictions of creating apps used with Kinect.

w w w . h a m k . f i

THE EXPERIENCE FROM THE PILOTS

w w w . h a m k . f i

More experience of

Directing Kinect-related acitivies

How does the client react on directing and

using Kinect?

Understanding the client’s needs

Understanding the varying abilities of

the clients

The interaction

The relationship between the student

and the client

How to communicate with the client?

Observing the fellow students and the directing process

The directing skills

Preparing the directing situation

Taking control of the situation

What kind of an impact the directing has on

client?

Evaluating the situation

w w w . h a m k . f i

The natural user interface (NUI)

Gesture controlled virtual and augmented reality

The possibilities for various user groups not able to use technically demanding interfaces?

No mouse, keyboard or handheld devices

needed

A technology easy to approach and learn

Augmented reality

Digital content brought to one’s

surrounding environment

How to support the user by bringing audio and visual elements to the

app?

Visual, experimental 3D environments

Virtual reality

Simulation

New experiences, memorizing the past

events

The possibilities of NUI simulations?What other elements

could be brought in?

Interaction within the simulation

w w w . h a m k . f i

The change

• New information about new possibilities of learning and participation for the target groups (teachers and students as well as service providers and their customers)

• Welfare sector actors familiarized with new technologies

• IT students learned about the interconnection between Kinect and the promotion of wellbeing but also the educational prospects.

• More possibilities for citizens' participation have emerged during the project – including those for special groups.

w w w . h a m k . f i

Read more:

• The project website at HAMK: http://www.hamk.fi/avo2/ • The brochure of the project at HAMK: http://bit.ly/18OM0AQ

Contact us: leena.koskimaki@hamk.fimerja.salminen@hamk.fi