Post on 20-Aug-2015
USING SMARTPHONES AS PERSONAL MONITORING TOOLS FOR THE ACOUSTIC
ENVIRONMENT!
João Cordeiro!Álvaro Barbosa!
School of Arts - Portuguese Catholic University – C. R. Porto!CITAR - Research Centre for Science And Technology of The Arts (Porto)!Faculty of Creative Industries - University of Saint Joseph – Macau, China!
Outline!
• Main goals!• Problem statement!• System design!• Experimental procedure!• Results!• Conclusions!• Future work!
Main goals!
Better qualityof life!
Achieve better soundscapes in
urban areas!
Increase awareness for the acoustic phenomenon!
Problem statement!
How to promote sound awareness? !
Education!Music > Sound !Sound walks!Sound design!
Public Politics!Campaigns !
Laws!Training!
Technology!Entertaining / Useful / Social!
Problem statement!Entertaining / Useful / Social MOBILE Technology!
!!Advantages!
!- People are keen to new technologies!!- Mostly mobile-based!!- Powerful computing processing!!- Feature several sensors (microphones, camera, accelerometer…)!!- Open to developers!!- …!
!Disadvantages!
!- Not rigorously calibrated!!- Not multitasking friendly!!- Power autonomy!!- Privacy issues !!!- Latency communication!!- …!
!
Problem statement!!!>> SOUND BASED <<!!!Audio Games!(ex: Murky Shooting)!!!
Entertaining / Useful / Social MOBILE Technology!
Problem statement!!!>> SOUND BASED <<!!!Audio Games!!!Sound Maps!(ex. Freesound uploads) !!!
Entertaining / Useful / Social MOBILE Technology!
Problem statement!!!>> SOUND BASED <<!!!Audio Games!!!Sound Maps!!!Sound-Based Social Networks!(ex. Hurly-Burly)!
Entertaining / Useful / Social MOBILE Technology!
Problem statement!
Can we use soundscapes to improve Online Social Networking?!(thus creating an engaging service based on sound)!
System design!
Hurly-Burly!(means: boisterous activity) !!
• senses the sound environment surrounding each user!• classifies it according to the sound sources!• measures the sound pressure level!• implements a online social network!• distributes the sound metadata through the social network!• keep a record of user’s soundscapes automatic metadata!
System design!
• iOS platform!• Senses the soundscapes using built
in microphone!
• Classification: Music, Speech, Environmental Sounds, Silence!
• Measures SPL – dB(A)Leq, 3s!
• Continuous Monitoring (background)!
• Online Social Network!
• Movement Sensing!
System design!
• MySQL database!• PHP web interface!
• Users’ soundscape logs!- timestamp !- soundscape type !- SPL!
• Relational table (who is friends with who)!
System design!
• Interactive timeline graph with users’ soundscapes and movement data!
• Affords horizontal zoom in/out and timeline “traveling”!
Experimental procedure!
• Test the system with 3 test groups!• Total 23 users + 2 control devices!• 3 days each test!• Surveys and data logs were collected!
Each group is a Social Network!with Cluster Coefficient = 1!
Results!
1
10
2
1
5
1
1 1
1
USJ UCP HB
Portuguese Philippine Chinese Israeli Spanish Brazilian Czech
Sample characterization | nationality !
Results!
Sample characterization | age distribution!
0
1
2
3
4
19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
Num
ber o
f sub
jects
Age range
USJ UCP HB
Results!
Sample characterization | frequency of Social Media usage !
18
2
2
1
Everyday
Twice a week
Once a month
I'm no sure
Results!
Sample characterization | favourite way to access Social Media!
35%
48%
13%
4%
Mobile devices (smartphones, iPods...)
Laptops
Tablets
Desktops
3
3
4
7
9
13
14
17
Who is with me
My Loca[on
What music Am I
Videos
Audio
Things I Like
Textual messages
Photos
Results!
Sample characterization | what users prefer to share on Social Media!
non-contextual cues !
Results!
Experiment characterization | users proximity with the device during exp.!
48% 52%
Yes, I carried the device always with me
Most of the [me, but not always
No, We were separate
Results!
Experiment characterization | devices used to run the application!
65%
18%
17%
iPhone
iPod
iPad
Results!
Experiment characterization | application performance per device!
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
iphone ipod ipad
Major Porblems
Minor Porblems
Excelent
Results!
User experience | what was the first reason to switch of the application!
1
10
3
0
0
0
All the above
Low ba_ery
Privacy issues
Keeps my device slow
I don't likethe red bar
I'm not sure
Results!
User experience | audio clips v.s. soundscapes automatic metadata!
3
7
9
4
I would use it, I think it increases the sense of realism of the social experience
I might use it with cau[on
I would use it only in very controlled situa[ons.
I would never use an applica[on like that, it violates my privacy
Results!
User experience | hurly-burly systems as environmental gauges!
2
7
14
I doubt I guess so Yes, definitely
Results!
User experience | choices according the knowledge of soundscapes !
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
I know More or Less
I Doubt
Maybe
Define_ly
Conclusions!• Mobile devices are a great opportunity for sensing the environment,
however, multitasking and power constraints may hinder this use!
• Soundscapes provide an interesting description of people’s activity, which can be applied to many areas where this information is relevant, for example Social Networking!
• Recording and sharing audio clips may hinder the use of sound in Social Networking, due to privacy issues. Classifying the sound with machine audition techniques represents a good option!
• There’s a relation between knowing the concept of soundscape and acknowledging it as an important resource. Thus an increase of education on this field will lead to more conscientious citizens about their sound environment!
Future work!
• Port the application for other platforms (Android)!
• Enrich the study with more users!
• Interleave the application with existent Online Social Networks (e.g. Facebook, Google +, Twitter…)!
• Developing other technological applications that may lead to an increase of the sound awareness among non-experts!
• …!