Paravirtualized USB Support for Xen · Modern webcams support •The UVC (USB Video Class) driver...
Transcript of Paravirtualized USB Support for Xen · Modern webcams support •The UVC (USB Video Class) driver...
Copyright 2008 FUJITSU LIMITED
Paravirtualized USB Support
for Xen
Noboru Iwamatsu
FUJITSU LABORATORIES LTD.
Background: Client-side virtualization and USB
Copyright 2008 FUJITSU LIMITED1
In client-side virtualization, special Service-VMs work in the background of User-VM, and provide various functions. To give some examples,
Authentication VM: Authenticate with USB-connected smartcard or biometrics.
Communication VM: VoIP/Messenger application using USB-connected webcam or audio.
In this situation, both User-VM and Service-VMs require using the USB devices at the same time by the same USB host controller.
Hypervisor
User VM Service VM Service VM
LaptopUSB Host Controller
User OS Service OS Service OS
AuthenticationVoIP/
Messenger
Office/Music
apps
USB devices
Usage example of
client-side virtualization
History and Motivation: Xen and USB support
Copyright 2008 FUJITSU LIMITED2
Xen 2.0.x(in PV-only age)
Paravirtualized USB support was temporarily into the tree.
But eventually removed and not released.
Xen 3.0 Qemu-dm supported UHCI emulation for HVM domain.
Xen 3.2PCI pass-through with IOMMU supported attaching USB
Host Controller to HVM domain.
Xen 3.4(current
unstable)
Not well supported
USB1.1 only
Works well. But the entire controller is assigned to
single domain and can’t be used from other domains.
Existing options are not suitable for the client-side
virtualization. So, we have started developing
paravirtualized USB driver, and proposed in XCI.
Now, “PV USB support” is on the roadmap.
Today, we would like to report our implementation details
and development status.
Overview: What’s paravirtualized USB driver?
Copyright 2008 FUJITSU LIMITED3
A split drivers for handling USB devices from guest domains.
frontend: Virtual USB 2.0 Host Controller driver acting as a proxy for backend.
backend: USB function driver handling the USB devices.
urb (USB request block):
In linux kernel, all USB drivers and subsystems
communicate with urb. Urb is a USB request block
structure (described in include/linux/usb.h).
It is much like “struct skbuff” in the networking code.
Frontend transfers urbs to backend through Xen, and backend transfers
the received urbs to the USB devices.
Xen Hypervisor
Dom 0
Dom0 OS
Guest VM 1 Guest VM 2
Guest OSGuest OS
PV USB
frontend driver
PV USB
frontend driver
PV USB
backend driver
USB native driver
USB Host Controller
USB devices
urb transferring
Implementation details(1/5):
USB basics: Driver architecture and urb lifecycle in kernel
Copyright 2008 FUJITSU LIMITED4
USB drivers in linux kernel is implemented as 3-layer module stacks.
USB Function Driver - Driver for each USB device (e.g. storage, printer, hid)
A urb is created by this driver, and is submitted to USB core.
If the urb is successfully transferred to the USB device, the completion handler of the urb is called.
USB Core - USB subsystem in linux kernel
The urb is enqueued to specific host controller driver for the specified device by this layer.
USB Host Controller Driver – Driver for USB host controller hardware
The enqueued urb is transferred to the hardware, and is backed to USB core after the driver get response.
USB Function Driver
USB Core
USB Host Controller Driver
usb_submit_urb()
usb_hcd_giveback_urb().urb_enqueue()
calling completion handler
usb_alloc_urb() usb_free_urb()
Hardware
(USB Host Controller)
User-space or other kernel subsystems
urb urb
Kernel-space
Implementation details(2/5):
Paravirtualized USB driver architecture and urb lifecycle
Frontend driver - Implemented as a USB host controller driver
The enqueued urb is mapped to RING request, and is sent to backend.
When the RING response is received, the urb is backed to USB core after substituting that status-value.
Backend driver - Implemented as a USB function driver
The urb is re-created from the RING request, and is submitted to the specified device.
When the urb is completed, the status of the urb is mapped to RING response and returned to frontend.
All existing code need not be modified, and maybe all USB drivers in kernel can work.
Copyright 2008 FUJITSU LIMITED5
PV USB backend
PV USB frontend
Xen frontend interface
RING xenbus
Virtual USB 2.0
Host Controller Driver
USB Function Driver
USB Core
Host Controller Driver
USB Core
USB Function Driver
Xen backend interface
RINGxenbus
urb urb
Cloned
urb
Dom0 kernel-space DomU kernel-space
Cloned
urb
Hardware
(USB Host Controller)
.urb_enqueue() usb_hcd_giveback_urb()
usb_submit_urb () completion handler
allocfree
Implementation details(3/5):
How to map urb to RING request/response?
The fields of the urb structure is mapped to the RING as follows.
The data buffers of urb are sharing between frontend and backend by using the grant table operations.
If the number of shared pages is 10, RING_SIZE is set to 32. Most of function drivers may be all right by this setting :-)
Copyright 2008 FUJITSU LIMITED6
struct urb
{
unsigned int pipe;
unsigned int transfer_flags;
void *transfer_buffer;
int transfer_buffer_length;
unsigned char *setup_packet;
int interval;
int start_frame;
int number_of_packets;
struct usb_iso_packet_descriptor iso_frame_desc[0];
int status;
int actual_length;
int error_count;
…
};
struct usbif_response
{
...
};
struct usbif_request
{
…
segs[];
};
Mapping to RING request
Mapping to RING response
Sharing pages with grant table
Implementation details(4/5):
Virtual USB host controller internals
Copyright 2008 FUJITSU LIMITED7
Virtual USB Host Controller
submit_waiting_queue
submit_in_progress_queue
giveback_waiting_queue
urb
.urb_enqueue() usb_hcd_giveback_urb()
RING request
RING response
urb
urb urb urb urb
urb
urb
after send, move to tail
move matched urb to tail
add to tail
delete from queue
urb urb
Virtual host controller has 3 queues for scheduling urbs.
submit_waiting_queue
The enqueued urb is added to the tail of this queue, and waits its turn to be sent to backend.
submit_in_progress_queue
While the RING request is sent and waits for the response, the urb is added to this queue.
giveback_waiting_queue
While the RING response is received and waits for backing to the USB core, the urb is added to this
queue.
RING responses are called
from interrupt context and
submit_wating_queue and
givback_wating_queue are
flushed by timer functions.
Implementation details(5/5):
How to plug the devices to guest domains?
Copyright 2008 FUJITSU LIMITED8
Guest VM 1
(domain ID:1)
Virtual
Host Controller
(vusb-0)Virtual
roothub
Guest VM 2
(domain ID:2)
Virtual
Host Controller
(vusb-0)Virtual
roothub
device
Dom 0
Host Controller
(usb1)
roothub
hub
device
device device device device
device
1-4
1-2.1 1-2.3
Hotplug-rule is set from sysfs interface in the backend driver.
The hotplug-rule format
• <usbbusname>:<domid>:<vusb number>:<virtual port number>
Example settings% echo 1-2.3:1:0:3 > /sys/bus/usb/drivers/usbback/new_vport
% echo 1-4:2:0:1 > /sys/bus/usb/drivers/usbback/new_vport
% echo 1-2.1:2:0:2 > /sys/bus/usb/drivers/usbback/new_vport
When new device is connected, the backend driver claims the device if its busnameis found on the settings. The frontend driver is notified of hotplug by xenbus.
Development Status
Paravirtualized USB driver has just started working.
Tested devices
The performance is about 6MB/s(Read/Write throughput of flash drive).
• 1/5 of native performance, but more than 4 times faster than USB1.1.
• There is still room for improvement.
Implementation Status
• Hotplug and disconnect functions are not completed yet.
Copyright 2008 FUJITSU LIMITED9
Type Name Manufacturer Driver Status
Keyboard FKB-108-EU FILCO usbhid
Mouse Cordless Notebook Mouse Logitech usbhid
Flash drive RUF2-R2GS Buffalo usb-storage
Flash drive RUF-C1G/U2 Buffalo usb-storage
HDD HDCN-U500 IO DATA usb-storage [1]
Webcam WebCam 3 USB Creative Labs ov511 [2]
Works Works with issues Not worked yet
1. SCSI command [READ_CAPCITY] fails.
2. Horizontal stripes are into video streaming image.
TODO & Future work
TODO
Many cleanups and bugfixes.
Performance upgrade.
Modern webcams support
• The UVC (USB Video Class) driver requires high bandwidth.
Suspend / Resume support.
Xend support.
Future work
Netchannel2 support.
Porting the frontend driver to stub domain.
Copyright 2008 FUJITSU LIMITED10
We will post the code by the end of this year!
Copyright 2008 FUJITSU LIMITED11