Interactive technologies for children: new frontiers

Franca GarzottoMatteo Valoriani

Dipartimento di Elettronica ed Informazione – HOC LabPolitecnico di Milano

Outline

Who we are

Children and Technology

New frontiers: - Tangible Interaction- Touchless Gesture Based Interaction

Who we areHOC Lab A multidisciplinary research lab at the Dept. of Electronics and Information at Politecnico di MilanoAplication domain:• e-learning• e-culture• e-tourismTechnologies: web, mobile devices, tangibles, gesture-based http://hoc.elet.polimi.it/hoc/

Children and technology • A huge amount of digital technologies for

children

• Mostly for entertainment, education, and learning

• Children are a stimulating and wide target of users for current and new technologies

Example 1: tangible interaction• physical representation of data and functions• manipulation of digitally enriched physical objects• Embodiment and embeddedness in real space

I/O Brush- MIT Media Lab

I/O Brush- MIT Media Lab

I/O Brush- MIT Media Lab

Input and Control Device: Brush

I/O Brush- MIT Media Lab

Input: coloured balls

Input and Control Device: Brush

Paper-based Multimedia as Learning Tool for Disabled ChildrenTangible Interaction: Case

GoalTo support learning

in the school context

for children with severe cognitive,

linguistic and motor disabilities

in the long term and in a sustainable way

Profile of our disabled children• Severe impairments in body

movement and muscle coordination

• Limited cognitive functions• They understands from 40 to 70% of

speech and non verbal communication (i.e., conventional gestures and face expressions),

• They are not able to speak

• They normally communicate through face expressions, body movements, sounds, PCSs and portable VOCAs

Requirements on the UX• Use of PCS based Communication• Customization (“each child is unique”)• Tangible Interaction• Use of familiar, situated material• Consolidation• Enagagement• Concentration and Timing

Requirements on Technology• Open

• Flexible

• Easily adaptable

• “Construction Kit” and “End User Programming” paradigms

Physical visual content:- representations of concepts to learn

(e.g., PCSs, story elements, …)- affordances for interacting with digital

material

RFID technology

multimedia resourcessound, voice, music

images or videos (on the screen or on the wall)

tangible paper based visual content

Scenario 1: Enriching the Goal: Recognition of known concepts; Learning new PCS symbols

Materials: • Videos of school mates’ actions • PCS tagged and untagged action cards and

phrases; School mates’ tagged pictures; class poster

Activity:• Looking at videos corresponding to school mate

images• Given a PCS phrase, recognize which school mate

performs the corresponing action• Tag the class poster with proper PCS action

symbols

Scenario 2: Understanding more complex communication structures (storytelling)

Goals: Recognition of story scenes; building associations (scenes ->PCS phrases); aggregating scenes

Materials: • Story animations (whole story + single scenes)• PCS tagged and untagged phrases; • tagged story poster; story “path” poster

Activity:• Interact with the story (scenes) poster• Match a PCS phrase with the corresponding scene• Build the story poster

Scenario 3: Building “My” storyGoal: Understanding temporal concepts (“before”, “after”); develop elementary narrative skills

Materials: • Videos of child’s activities at school• Tagged pictures of child’s activities• Tagged PCS symbols

Activity: story building• Put images in the right temporal order• Build a proper PCS phrase for each moment• Associate phrases to images

Engaging the whole class

Non-disabled children: experience co-designers• Identifying what their disables mates like more• Starring actors in videos• Content authors (images, pictures, sounds)

the element project

Case study 2:

60M people in the world

∼3 xAustralian

population

70%increase in last

10ys

new kinect-based, playful learning application for autistic children with innovative cloud remote assistance

Autism Spectrum Disorders (ASDs) is very large and comprises different symptoms

problem & goals

motion

Autism Spectrum Disorders (ASDs) is very large and comprises different symptoms

interest communication relation

problem & goals

motion

Autism Spectrum Disorders (ASDs) is very large and comprises different symptoms

interest communication relation

create interest and motivation

1

problem & goals

motion

fosterphysical activity

2

Autism Spectrum Disorders (ASDs) is very large and comprises different symptoms

interest communication relation

create interest and motivation

1

problem & goals

motion

fosterphysical activity

2

Autism Spectrum Disorders (ASDs) is very large and comprises different symptoms

interest communication relation

improve communication

skills

3create

interest and motivation

1

problem & goals

motion

fosterphysical activity

2

Autism Spectrum Disorders (ASDs) is very large and comprises different symptoms

interest communication relation

improve communication

skills

3promote social interaction thru

teamwork

4create

interest and motivation

1

problem & goals

motion

fosterphysical activity

2

Autism Spectrum Disorders (ASDs) is very large and comprises different symptoms

interest communication relation

technology can’t heal, but can help people bypass disability effects

improve communication

skills

3promote social interaction thru

teamwork

4create

interest and motivation

1

problem & goals

our solution

our solution

customization

our solution

therapy centers

customization

our solution

therapy centers

customization

@home therapy

our solution

therapy centers

customization remote therapy

@home therapy

game patternsdifferent activities to address very different aspects of autism

Sequentialgames

Quizgames

Musicgames

Interactiongames

game patternsdifferent activities to address very different aspects of autism

Sequentialgames

Quizgames

Musicgames

Interactiongames

Focused on teaching a sequence of actions

game patternsdifferent activities to address very different aspects of autism

Sequentialgames

Quizgames

Musicgames

Interactiongames

Flexible and adaptable to many teaching activities

game patternsdifferent activities to address very different aspects of autism

Sequentialgames

Quizgames

Musicgames

Interactiongames

Leverage the emotion and the involvement of music

game patternsdifferent activities to address very different aspects of autism

Sequentialgames

Quizgames

Musicgames

Interactiongames

Focused on communication and social interaction

personalization

standard AAC

elementsdifferent

levels

different contents and tasks

journal & reportingteachers can insert private notes about child response and/or comments for the parentsparents can view these notes on a personal web journal

remote therapy

remote therapy

remote supervision

remote communication

remote interaction

remote customization

User testing

User testing

Thanks for your attentionFranca GarzottoMatteo Valoriani

Dipartimento di Elettronica ed Informazione – HOC LabPolitecnico di Milano

[garzotto, valoriani@polimi.it]