LightSyncWenjun Hu Hao Gu Qifan Pu
2D barcodes everywhere
“Transmitting” information (vs linking)
E.g., movie show times for nearby theaters
Receiver
Transmitter
4
Free, directional, lightweight links
5
Aesthetics…
6
Frame synchronization challenges
Variable
Received frame patterns
(& #components) varies by line
Due to Rolling Shutter
CMOS camera sensor scans one line (of pixels) at a time
12
Mixing pattern varies by line
13
Occasional missed frames
Say we have a phone camera whose current capture rate is C (if unsteady, basically the min C), then when the display rate is
Up to c/2, we get single frames interleaved with 2-frame mixes. The latter is due to phase offsets between the display and captured frames
c/2 to c, basically all 2-frame mixes
C to 2c, 2-frame interleaved with 3-frame mixes, with occasional missed frames. 3-frame mixes increasingly common as the display rate goes up.
>2c, basically always mixes of 3 frames or more, with occasional missed frames.
14
Higher camera capture rate, better performance
Supported frame capture rate (fps)
30
15
16
Requirements
In-frame color tracking per line
Need to guard against lost frames/lines
Inter-frame erasure coding
17
In-frame grayscale tracking per line
Need to guard against lost frames/lines
Inter-frame erasure coding
18
In-frame grayscale tracking per line
Need to guard against lost frames/lines
Inter-frame erasure coding
Bl or w per block → 8 mixing possibilities
?
?
?
Decoding is simply interference cancellation
21
RX
TX
27
Each received frame may be decode into several sent frames
29
LightSync
In-frame grayscale tracking per line
Need to guard against lost frames/lines
Inter-frame erasure coding
Coded frames
32
Performance
OK, so that was how LightSync works. Let’s now look at its performance.
33
Implementation
Metric: Effective frame capture rate
Actual throughput scales with per-frame capacity
35
36
Ideal
Ideal
37
For a 20fps phone camera
Conventional
Ideal
38
Slight drops at higher display rates
Ideal
39
Higher camera capture rate, better performance
42
Thank you!
Basically slows down display, only for TX up to 10fps
Spatial barcode design (per-frame layout)
COBRA: Optimized for phone-to-phone
Other work on visible light communications
45