Atau TanakaSony CSL Paris

Malleable Mobile Music

•Deploying music on mobile infrastructures– Not for musicians…for music listeners– Go beyond Kazaa and iPod

•New modes of music listening using:– Social dynamic– Wireless networks– Sensor embedded devices

Mobile Music Making

Talk Overview

•Background

•The System

•Demo/video

Backgrounds:Music and Research

•Network music

Backgrounds:Music and Research

•Network music– Remote concerts

Backgrounds:Music and Research

•Network music– Remote concerts– Netmusic

Backgrounds:Music and Research

•Network music– Remote concerts– Netmusic– Installations

• Global String (Ars Electronica 2002)

Backgrounds:Music and Research

•Network music– Remote concerts– Netmusic– Installations

• Global String (Ars Electronica 2002)

•Research

Backgrounds:Music and Research

•Network music– Remote concerts– Netmusic– Installations

• Global String (Ars Electronica 2002)

•Research– P2P + adhoc

Backgrounds:Music and Research

•Network music– Remote concerts– Netmusic– Installations

• Global String (Ars Electronica 2002)

•Research– P2P + adhoc

– Bootstrapping Trust

Backgrounds:Music and Research

•Network music– Remote concerts– Netmusic– Installations

• Global String (Ars Electronica 2002)

•Research– P2P + adhoc

– Bootstrapping Trust

– Location Sensitive Audio

Objectives

•How to leverage wireless networks for music?– The infrastructure can do more than file sharing– Enhance sense of community via music

•Making a new music listening experience– Draw upon musician’s experience– Dynamic and new, yet familiar songs

•Could this point to new content models?

The System

•Mobile terminal– PDA– Wireless network

The System

•Mobile terminal– PDA– Wireless network– Sensor subsystem

The System

•Mobile terminal– PDA– Wireless network– Sensor subsystem• FSR• Accelerometer• GPS (in simulation)

The System

•Mobile terminal– PDA– Wireless network– Sensor subsystem• FSR• Accelerometer• GPS (in simulation)

•Music engine– Server-side– Network control in– Stream out

Malleable Music

•Take a hit song, and make it interactive

Malleable Music

•Take a hit song, and make it interactive– Bjork: I miss you (1995)

Malleable Music

•Take a hit song, and make it interactive– Bjork: I miss you (1995)– The Meters: Look-ka Py Py (1970)

Malleable Music

•Take a hit song, and make it interactive– Bjork: I miss you (1995)– The Meters: Look-ka Py Py (1970)

•Shaping– Mixing– filtering

•Familiar song, new experience– Instantaneous “versions” following your mood– Flexible time - 3 min hit song now

fits your train ride time

Malleable Music

•Familiar song, new experience– Instantaneous “versions” following your mood– Flexible time - 3 min hit song now

fits your train ride time

•Mobility and participatory listening

•Musical content reflectingsocial dynamic

Malleable Music Engine

•Time domain segmentation and reconstruction

•Generalized temporal reorganization model– Song structure (song level)– Variations (measure level)– Melodic/rhythmic (sub-measure level)

Malleable Social Dynamic

•Community Navi Re-mix– Group participation in a single musical piece– Collective Mobility - moving about town

•Intuitive, subconscious input – Actions while listening• swinging along• gripping the Walkman

– Remote yet collective

•Future content model?