Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview...
Transcript of Recent CCRMA Research on Virtual Musical Instrumentsjos/pdf/AskenfeltJOS09.pdf · Overview Overview...
Julius O. Smith III Askenfelt60, KTH, Sweden – 1 / 23
Recent CCRMA Research on Virtual Musical Instruments
Julius O. Smith IIICCRMA, Stanford University
Askenfelt60, KTH, Sweden
October 21, 2009
Overview
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 2 / 23
Recent Research on Virtual Musical Instruments at CCRMA
Overview
• CCRMA
• Outline
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 3 / 23
CCRMA building: The Knoll, Stanford University
Outline
Overview
• CCRMA
• Outline
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 4 / 23
• Virtual Acoustic Guitar — Nelson Lee(Computer Science PhD student)
• Haptic Virtual Instruments — Ed Berdahl(Electrical Engineering PhD student)
• Virtual Harpsichord — Jack Perng(Physics PhD student)
• Acoustic Space Modeling — Consulting Professor JonathanAbel, Music PhD student Nick Bryan, EE graduate student TravisSkare, and others
• Special Issue on Virtual Analog Audio Effects & MusicalInstruments, for the IEEE Transactions on Audio, Speech, &Language Processing (ASLP) — edited by Valimaki, Fontana,Zolzer, & Smith
Virtual Acoustic Guitar Models
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 5 / 23
Coupled Strings Analysis and Synthesis
Overview
Acoustic Guitar Models
• Coupled Strings
• String Model
• Sound Examples
• Sound Examples
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 6 / 23
Submitted paper based on recent CCRMA/CS thesis by Nelson Lee :
“Analysis and Synthesis of Coupled Vibrating StringsUsing a Hybrid Modal-Waveguide Synthesis Model”
by Nelson Lee, Julius Smith, and Vesa Valimaki.Accepted for publication in the IEEE special issue onVirtual Analog Audio Effects and Musical Instruments,May 2010 (est.)
Overview of Nelson Lee String Model
Overview
Acoustic Guitar Models
• Coupled Strings
• String Model
• Sound Examples
• Sound Examples
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 7 / 23
• String excitation (for commuted waveguide synthesis) is highpassfiltered to avoid exciting first N partials
• Lowest N partials are replaced by fourth-order resonators (whichcan independently beat and give two-stage decay)
• Similar to Balazs Bank formulation which adds second-orderresonators to existing partials of the filtered-delay-loop
• New analysis methods (in thesis) for estimating partialparameters, as well as other results
Sound Examples of Individual Effects
Overview
Acoustic Guitar Models
• Coupled Strings
• String Model
• Sound Examples
• Sound Examples
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 8 / 23
From Nelson Lee’s thesis defense:
• Original waveform: (WAV) (MP3)• Simple lossless, reflectively terminated digital waveguide (DWG):
(WAV) (MP3)• Add loop filter: (WAV) (MP3)• Add interpolation filter: (WAV) (MP3)• Add excitation (ICMC07): (WAV) (MP3)• Add body response: (WAV) (MP3)• Add hybrid modal/waveguide model: (WAV) (MP3)• Exaggerate pitch glide due to tension modulation: (WAV) (MP3)
Virtual Acoustic Guitar Sound Examples
Overview
Acoustic Guitar Models
• Coupled Strings
• String Model
• Sound Examples
• Sound Examples
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 9 / 23
More Nelson Lee examples:
• Original 1: (WAV) (MP3)• Synthesized 1: (WAV) (MP3)
• Original 2: (WAV) (MP3)• Synthesized 2: (WAV) (MP3)
• Original 3: (WAV) (MP3)• Synthesized 3: (WAV) (MP3)
• Original 4: (WAV) (MP3)• Synthesized 4: (WAV) (MP3)
• Original 5: (WAV) (MP3)• Synthesized 5: (WAV) (MP3)
• Original 6: (WAV) (MP3)• Synthesized 6: (WAV) (MP3)
• Synthesized Chord Demo: (WAV) (MP3)
Haptic Feedback Control forVirtual Instruments
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 10 / 23
Haptic Virtual Musical Instruments
Overview
Acoustic Guitar Models
Haptic Instruments
• Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 11 / 23
CCRMA/EE PhD student Ed Berdahl is working on
Haptic Feedback Control for Virtual Instruments
Dissertation expected by the end of the year
Goals:
• Assist and/or augment gestures• Assist with accurate playing• Recent projects:
◦ Haptically plucked virtual string◦ Active drumhead (one-handed rolls, etc.):
http://ccrma.stanford.edu/~eberdahl/Projects/-HapticDrum/
Virtual Harpsichord
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 12 / 23
Harpsichord Modeling
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
• Harpsichord
• Harpsichord Jack
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 13 / 23
CCRMA/Physics PhD student Jack Perng is working on
1. Building a harpsichord jack and monochord (mostly done)2. Measuring position, velocity, etc. (getting started)3. Studying models of Giordano et al., Valimaki et al., and others
(well under way)4. Looking for ways to improve the model(s)
Prof. Tom Rossing collaborating
Harpsichord Jack and Monochord
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
• Harpsichord
• Harpsichord Jack
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 14 / 23
Acoustically Transparent andConfigurable Microphone Array
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 15 / 23
Microphone Array
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
• Mic Array
• Mic Array Paper
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 16 / 23
• Adustable geometry (software calibrated)• Sixteen microphones (Countryman B6 Omni Lavalier):
◦ 2 mm diameter capsules◦ 1 mm diameter flexible mounting wire◦ Acoustically transparent over most of the audio band
Upcoming Paper
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
• Mic Array
• Mic Array Paper
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 17 / 23
“A Configurable Microphone Array with Acoustically TransparentOmnidirectional Elements”
Jonathan Abel, Nicholas Bryan, Travis Skare, Patty Huang, DariusMostowfi, Miriam Kolar, and Julius Smith
AES-2009, New York
Current Application:
Recording and modeling acoustic properties of undergroundgalleries at pre-Inca archeological site Chavın de Huantar in Peru
Special Issue of the IEEE ASLP
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 18 / 23
IEEE ASLP Special Issue
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
• ASLP Special Issue
• Issue Overview
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 19 / 23
We have been working on a special issue devoted to
Virtual Analog Audio Effects and Musical Instruments
for the
IEEE Transactions on Audio, Speech, and LanguageProcessing (ASLP)
• Editors:
◦ Vesa Valimaki◦ Federico Fontana◦ Udo Zolzer◦ Julius Smith
• Eleven papers accepted so far, with more to come• Likely publication date: May 2010
Accepted Papers on Virtual Musical Instruments
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
• ASLP Special Issue
• Issue Overview
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 20 / 23
• “Tubular Bells — A Physical and Algorithmic Model” byRabenstein, Koch, and Popp
• “A block-based physical modeling approach to the soundsynthesis of drums” by Marogna and Avanzini
• “A Virtual Model of Spring Reverberation” by Bilbao and Parker
• “Analysis and Synthesis of Coupled Vibrating Strings Using aHybrid Modal-Waveguide Synthesis Model” by Lee, Smith, andValimaki
Likely additional papers on
• virtual plucking methods for waveguide string models,• piano modeling via physically directed modal synthesis, and• another approach to spring reverb modeling
Summary
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
Julius O. Smith III Askenfelt60, KTH, Sweden – 21 / 23
Summary
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
• Summary
• Conclusions
Julius O. Smith III Askenfelt60, KTH, Sweden – 22 / 23
A quick look at recent acoustic-modeling research at CCRMA:
• Coupled Strings Analysis and Synthesis — Nelson Lee (CS) —Fourth-order modes for low partials, waveguide model for upperpartials; new analysis techniques
• Haptic Virtual Instruments — Ed Berdahl (EE) — Real controllers(with force feedback) for virtual instruments
• Virtual Harpsichord — Jack Perng (Physics) — Monochord+jackmeasurements toward improved harpsichord synthesis models
• Microphone Array — Jonathan Abel et al. — Acousticallytransparent, configurable, software-calibrated microphone arrayfor sampling the 3D sound field
• Special Issue on Virtual Analog Audio Effects and MusicalInstruments — Vesa Valimaki et al., eds.
In Conclusion
Overview
Acoustic Guitar Models
Haptic Instruments
Harpsichord Models
Microphone Array
ASLP Special Issue
Summary
• Summary
• Conclusions
Julius O. Smith III Askenfelt60, KTH, Sweden – 23 / 23
Happy Birthday Anders!