Creating an Entertaining and Informative Music Visualization
-
Upload
icchp2012 -
Category
Health & Medicine
-
view
810 -
download
8
description
Transcript of Creating an Entertaining and Informative Music Visualization
![Page 1: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/1.jpg)
By: Michael Pouris, BSc, Deborah I. Fels, PhD, P.EngIMDC, Ryerson University, Toronto, Canada
![Page 2: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/2.jpg)
1. Background information2. Research and Model
Purpose of the visualization
Past Industry attempts
The priorities
Psychological model
3. The implementation
Translating to a visual medium
4. Study results and discussion
![Page 3: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/3.jpg)
Music is a major art form Present in all cultures worldwide Transcends cultural and language boundaries
Portal to a cultural shared experience (McDermott, 2004)
Hard of hearing and deaf have limited access to a shared hearing experience
![Page 4: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/4.jpg)
Serve as a tool that depicts music visually for hard-of-hearing or deaf audiences
Sensory substitution is possible (Nanayakkara, Taylor, Wyse and Ong, 2009)
Research questions:
Can sensory substitution models be used to provide access to music using the visual channel?
What are user reactions to MusicViz?
![Page 5: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/5.jpg)
Industry attempts have failed
Does not use justified psychological model which describes auditory mappings to visual system
![Page 6: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/6.jpg)
Three primary goals are to develop a visualization that is:
1. Aesthetically pleasing
2. Emotionally moving
3. Adaptable
![Page 7: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/7.jpg)
Interpretation is to explain the meaning of something with subjective bias
Interpreting feelings of the music is not feasible
There is not one “feeling” for a musical piece
Everyone interprets music differently
Goal: Visualization is open to each individuals interpretation
![Page 8: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/8.jpg)
Translation is to change or convert to another form
Translate auditory cues to visual medium
Give enough information
Goal: Individual can interpret information that is translated
![Page 9: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/9.jpg)
Auditory System Visual System
Pitch height and Pitch differences • Higher pitches = smaller objects• Lower pitches = larger objects• Pitches = altitude
Volume changes • Bigger objects = more prominent
Tempo and Beats • Indicate BPM and rhythm (repeated pattern)
Auditory Visual
Based on Ilie & Thompson (2006)
![Page 10: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/10.jpg)
Issue between pitch and volume in visual system:
Higher pitch smaller object
Higher amplitude (volume) larger object
Problem: amplitude and frequency are integrated within visual system
Solution:
Auditory System Visual System
Pitch height and Pitch differences • Associate with altitude
Volume changes • Associate with size
![Page 11: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/11.jpg)
Implemented using JOGL 2.0 (Java OpenGL)
Is a wrapper class for C OpenGL calls
Uses GLUEGEN to bind Java to C
X-Axis: Instrument arrangementY-Axis: Pitch scaleZ-Axis: Time progression
![Page 12: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/12.jpg)
Pitch: Pitch height determines height of the pipe
Timbre: Different “sound families” are defined through color
![Page 13: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/13.jpg)
Volume: Depicted through using thickness of pipes
Tempo: Clusters of activity
![Page 14: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/14.jpg)
Rhythm: Visual depiction of beats and interactivity
![Page 15: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/15.jpg)
![Page 16: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/16.jpg)
12 participants 3 hearing, 3 hard of hearing and 6 deaf ASL users
Procedure: 6 one minute-songs, each from different genres in
random order
Genres: classical, country, jazz, pop, rap/hip-hop/R&B (RHRB), rock▪ Commonly used groupings in industry
Pre-study, post-song, post-study questionnaires Eye tracking data recorded with FaceLab 5 Data analysed with repeated measures ANOVA
![Page 17: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/17.jpg)
Feedback on enjoyment and emotional experience Enjoyment:
5 point Likert scale (1-not enjoyable at all to 5-enjoyable)
Emotions:
1. Valence and arousal model (Russell, 1972), overbearingness added
2. Discrete model (happy, sad, anger, fear) (Ekman, 1972)
Level of focus
![Page 18: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/18.jpg)
No statistical difference between genres Descriptive analysis:
All genres are rated between 4-somewhat enjoyable and 5-enjoyable
Rock is most enjoyable (M=4.5, SD=0.85)
RHRB (M=4.0, SD=1.054) and classical (M=4.0, 1.247) are least enjoyable
▪ Initially hypothesized to be the most enjoyable due to bass
![Page 19: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/19.jpg)
Overbearingness: 1-subtle to 9-overbearing:
Between pop (M=6.42, SD=1.084) and RHRB (M=4.92, SD=1.564)
Between pop and country (M=4.25, SD=1.603)
Trend: an overbearing song can still be happy.
Pop is most overbearing (M=6.42, SD=1.084), highest arousal (M=6.08, SD=2.021) and happiest (M=6.75, SD=2.179)
Valence from 1-unhappy to 9-happy: not significant
Arousal: 1-calm to 9-excited: not significant
![Page 20: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/20.jpg)
Four 7-point scales (1-weak to 7-strong) for:
No significant difference between genres
A song may have multiple emotions and people were not able to choose one with limited set of words
1. Happiness2. Sadness
3. Anger4. Fear
![Page 21: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/21.jpg)
5-point likert scale: “1-my mind wondered a lot” to “5-I was always focused on the visualization”
No significant difference between genres Trends:
Paid most attention to rock (M=4.91, SD=0.302), country (M=4.73, SD=1.206) and jazz (M=4.64, SD=0.505)
Paid least attention to pop (M=4.09, SD=1.378), RHRB (M=4.09, SD=1.578)
![Page 22: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/22.jpg)
Major limitation is the lack of participants Additional participants are needed to determine
whether trends observed are actual statistical differences
Additional participants are needed to explore the differences between the hearing statuses
Participants showed preference for rock, pop and classical genres
MusicViz provided enjoyment and information
![Page 23: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/23.jpg)
Funding provided by GRAND, CFI, NSERC Co-workers at the IMDC Study participants
![Page 24: Creating an Entertaining and Informative Music Visualization](https://reader034.fdocuments.net/reader034/viewer/2022042607/559b13ff1a28abf5238b458f/html5/thumbnails/24.jpg)
Questions?
Michael Pouris, Deborah FelsRyerson University, Toronto, Ontario, Canada