Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16...
Transcript of Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16...
![Page 1: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/1.jpg)
Harry van Haaren, OpenAV, LAC 2017
![Page 2: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/2.jpg)
CTLRA
1) Modern Controllers
2) Why a library
3) Users and Mappings
4) Demo!
5) Bonus Stuff...
![Page 3: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/3.jpg)
Modern Controllers
![Page 4: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/4.jpg)
Modern Controllers
"Next Gen" Controllers!
● USB HID devices
● "HD" Screens
● Huge functionality!
● Application support…
![Page 5: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/5.jpg)
Modern Controllers
MIDI Mapping!
● APC40 released in 2014
○ USB MIDI device
○ Can't do full updates
![Page 6: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/6.jpg)
Modern Controllers
HD Screens?
● MIDI messages
○ SysEx?
○ No thanks :)
![Page 7: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/7.jpg)
Modern Controllers
Need for a better API
● Software ⇔ Hardware
○ Easier communication
○ Faster to support modern features
● Required for Tight Integration
○ Input Multiplexing
![Page 8: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/8.jpg)
Why A Library?
![Page 9: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/9.jpg)
![Page 10: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/10.jpg)
A Library for Controller Access
Developer implements library support
● Growing list of devices supported
● New devices added are "free"
● Don't re-implement device support
○ Testing is a time-sink!
![Page 11: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/11.jpg)
Generic Event API
Abstract away the Device
● Input from Events
○ Button, Slider, Encoder, Grid
● LED feedback function
● API to blit pixels to a screen
![Page 12: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/12.jpg)
Linux Support
Community Supported Devices
● Once-off implementations of little value
○ (Unless user has that exact controller)
● Centralize support in a library
○ Enabled by "Generic Events"
![Page 13: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/13.jpg)
Fancy Features
Hotplug
● Essential to save a Musician on stage
● Difficult to implement
○ Time consuming to test
![Page 14: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/14.jpg)
![Page 15: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/15.jpg)
![Page 16: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/16.jpg)
Users and Mappings
![Page 17: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/17.jpg)
Mappings and User-eXperience
Power User
● Can "script" controls as required
● Has huge flexibility and power
● Creates awesome mappings
Simple Workflow
● Can use software provided mapping
○ Just like MIDI mapping
● Not as powerful, but achieves goal
○ Mapping exists? Use it!
![Page 18: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/18.jpg)
Power User
Uniting Software and Hardware
● Match functionality of controller to software
○ Just as the user requires
○ Depends on Hardware / Software combo
○ Depends on Musicians Workflow
![Page 19: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/19.jpg)
Timeline (Just for fun :)
![Page 20: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/20.jpg)
Jan '16
Bought a USB HID drum-pad controller
Spring '16
Integrated drum-pad and Fabla 2.0
miniLAC '16
Demo Fabla 2.0 with a specific USB HID device
Summer '16
Understand requirements and design v1 "ctlr" API
Autumn '16
Implement v1 and discuss with other developers, design v2
![Page 21: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/21.jpg)
Dec '16
Development of Ctlra codebase
Jan '17
Finish initial implementation Write Ctlra paper for LAC
Feb '17
Mixxx integration and testing of v2 Ctlra API
(Specifically Hotplug)
Mar/Apr '17
Minor updates to API, Build out demo apps
LAC '17
Ctlra library release!
![Page 22: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/22.jpg)
Demos!
![Page 23: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/23.jpg)
Demo
1. Simple Events
2. Simple Feedback
3. Tight Integration
4. Scripting in C
5. DJ Hotplug
![Page 24: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/24.jpg)
Next Steps
1. Discuss Event Loops
2. Integrate with LV2 Atoms
3. Get Applications using it
4. Discuss "sharing" of devices
5. Support for BlueTooth, MIDI, OSC, Arduino, Serial etc..
![Page 25: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/25.jpg)
? Questions ?
![Page 26: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/26.jpg)
Bonus Demo AVTK + Ctlra
![Page 27: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/27.jpg)
Bonus Slides! Woop Woop :)
![Page 28: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/28.jpg)
Event Loops
How to manage threading / event handling
● Current API has idle() function
○ Must be called periodically
● Expose events via ringbuffer?
○ How to handle hotplug of new device.. New ring?
![Page 29: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/29.jpg)
LV2 Atom Integration
Ctlra Events map to LV2 Atoms quite nicely...
● Could the host pass these events "through" to the plugin?
○ Abstract the environment from the plugin?
○ Enable "sharing" of a device?
● Requires "Options" extension function to accept new device?
![Page 30: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/30.jpg)
USB Device Access
LibUSB
● Enables bulk, interrupt, isochronous, control endpoints
● API provides Sync and ASync modes
○ Async required for latency with multiple devices
HID Raw
● Won't support all devices (Screens are often USB "bulk" endpoints)
● Works better for certain devices..?
![Page 31: Harry van Haaren, OpenAV, LAC 2017 - Jean Monnet University · drum-pad controller Spring '16 Integrated drum-pad and Fabla 2.0 miniLAC '16 Demo Fabla 2.0 with a specific USB HID](https://reader030.fdocuments.net/reader030/viewer/2022040407/5eab3b07bfb39a40e00b5000/html5/thumbnails/31.jpg)
Timestamps
Arrival time of Event?
● Is it required at library level?
○ Time, Threading and ASync events… Fun!
● Expose an application defined callback to "set" the timestamp?