Post on 24-Jan-2021
A Visible Light Vehicle-to-Vehicle Communication System Using Modulated Taillights
Michael PlattnerDepartment Mobility and Energy
University of Applied Sciences Upper Austria
Hagenberg im Mühlkreis, Austria
Email: Michael.Plattner@fh-hagenberg.at
Gerald OstermayerResearch Group Networks and Mobility
University of Applied Sciences Upper Austria
Hagenberg im Mühlkreis, Austria
Email: Gerald.Ostermayer@fh-hagenberg.at
Presenter – Michael Plattner
• Michael Plattner, BSc MSc
• Age: 27
• born in Innsbruck, Austria
• Assistant Professor for Mobile Systems at University of Applied Sciences Upper Austria
• PhD studentat Johannes Kepler University Linz
2
Research Topics
• Mobile Communication Systems
• Intelligent Transportation Systems
• Simulation and Modelling
• Connected Vehicles
3
Idea
Out-of-band channel using modulated taillights for V2V communication
4
Idea
5
V2V communication might be attacked by a man-in-the-middle
Idea
Establish encrypted connection
Verification key
6
Identity of sender can be veryfied using out-of-band channel.
Requirements
• Use state-of-the-art LED taillights
• Camera used as receiver
• Visible light spectrum
• Not perceivable for the human eye
7
UDPSOOK - Modulation
• Undersampled Differential Phase Shift On-Off Keying
• Modulation frequency multiple of cameras FPS
𝑒. 𝑔. 𝑓𝑆 = 30𝐻𝑧 𝑓𝑚𝑜𝑑 = 120𝐻𝑧
• Utilizes rolling shutter effect of cameras
• Information encoded in the phase shift between frames
8N. Liu, J. Cheng, and J. F. Holzman, “Undersampled differential phase shift on-off keying for optical camera communications,” Journal of Communications and Information Networks, vol. 2, no. 4, 2017, pp. 47–56.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/30 s
• Recorded with a high-speed camera
9The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/30 s
• Recorded with a high-speed camera
10The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Mirror at 45° blocks thesensor to see through the
view finder.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/30 s
• Recorded with a high-speed camera
11The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Mirror flips up and blocksthe view finder.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/30 s
• Recorded with a high-speed camera
12The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Shutter opens.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/30 s
• Recorded with a high-speed camera
13The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Sensor is exposed to light.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/30 s
• Recorded with a high-speed camera
14The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Shutter closes.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/30 s
• Recorded with a high-speed camera
15The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Shutter is closed and mirror flips back in place.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/30 s
• Recorded with a high-speed camera
16The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/8000 s
• Recorded with a high-speed camera
17The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/8000 s
• Recorded with a high-speed camera
18The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Shutter starts opening at the top.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/8000 s
• Recorded with a high-speed camera
19The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Shutter starts closing rightbehind.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/8000 s
• Recorded with a high-speed camera
20The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Only a narrow slot of thesensor is exposed to light.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/8000 s
• Recorded with a high-speed camera
21The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Bottom part of the imageis exposed later than the
top part.
UDPSOOK• Rolling shutter of DSLR camera
• Exposure time: 1/8000 s
• Recorded with a high-speed camera
22The Slow Mo Guys, Inside a Camera at 10,000fps, https://www.youtube.com/watch?v=CmjeCchGRQo
Shutter is now fully closed.
UDPSOOK• Rolling shutter effect
• Image captured line by line
• Fast moving objects get skewed
23Smarter Every Day, Rolling Shutter Explained (Why Do Cameras Do This?) - Smarter Every Day 172, https://www.youtube.com/watch?v=dNVtMmLlnoE
UDPSOOK• Rolling shutter effect
• Image captured line by line
• Fast moving objects get skewed
24Smarter Every Day, Rolling Shutter Explained (Why Do Cameras Do This?) - Smarter Every Day 172, https://www.youtube.com/watch?v=dNVtMmLlnoE
UDPSOOK• Rolling shutter effect
• Image captured line by line
• Fast moving objects get skewed
25Smarter Every Day, Rolling Shutter Explained (Why Do Cameras Do This?) - Smarter Every Day 172, https://www.youtube.com/watch?v=dNVtMmLlnoE
UDPSOOK• Rolling shutter effect
• Image captured line by line
• Fast moving objects get skewed
26Smarter Every Day, Rolling Shutter Explained (Why Do Cameras Do This?) - Smarter Every Day 172, https://www.youtube.com/watch?v=dNVtMmLlnoE
UDPSOOK• Rolling shutter effect
• Image captured line by line
• Fast moving objects get skewed
27Smarter Every Day, Rolling Shutter Explained (Why Do Cameras Do This?) - Smarter Every Day 172, https://www.youtube.com/watch?v=dNVtMmLlnoE
UDPSOOK• Rolling shutter effect
• Image captured line by line
• Fast moving objects get skewed
28Smarter Every Day, Rolling Shutter Explained (Why Do Cameras Do This?) - Smarter Every Day 172, https://www.youtube.com/watch?v=dNVtMmLlnoE
UDPSOOK• Rolling shutter effect
• Image captured line by line
• Fast moving objects get skewed
29Smarter Every Day, Rolling Shutter Explained (Why Do Cameras Do This?) - Smarter Every Day 172, https://www.youtube.com/watch?v=dNVtMmLlnoE
UDPSOOK
30
• Rolling shutter effect• Image captured line by line
• Fast flickering light source turns into stripe pattern
UDPSOOK
31
• Rolling shutter effect• Image captured line by line
• Fast flickering light source turns into stripe pattern
UDPSOOK
32
• Rolling shutter effect• Image captured line by line
• Fast flickering light source turns into stripe pattern
UDPSOOK
33
• Rolling shutter effect• Image captured line by line
• Fast flickering light source turns into stripe pattern
UDPSOOK
34
• Rolling shutter effect• Image captured line by line
• Fast flickering light source turns into stripe pattern
UDPSOOK
35
• Rolling shutter effect• Image captured line by line
• Fast flickering light source turns into stripe pattern
UDPSOOK
36
• Rolling shutter effect• Image captured line by line
• Fast flickering light source turns into stripe pattern
UDPSOOK
37
• Rolling shutter effect• Image captured line by line
• Fast flickering light source turns into stripe pattern
UDPSOOK
38
• Rolling shutter effect• Image captured line by line
• Fast flickering light source turns into stripe pattern
UDPSOOK
39
• Rolling shutter effect• Image captured line by line
• Fast flickering light source turns into stripe pattern
UDPSOOK• Only a small portion of the
stripe pattern is visible.
• The state depends on theposition of the light source.
40
UDPSOOK• Only a small portion of the
stripe pattern is visible.
• The state depends on theposition of the light source.
41
UDPSOOK• Only a small portion of the
stripe pattern is visible.
• The state depends on theposition of the light source.
42
UDPSOOK• Only a small portion of the
stripe pattern is visible.
• The state depends on theposition of the light source.
43
UDPSOOK
44
• Only a small portion of thestripe pattern is visible.
• The state depends on theposition of the light source.
UDPSOOK
45
• Only a small portion of thestripe pattern is visible.
• The state depends on theposition of the light source.
UDPSOOK
Without phase shifts:
With phase shifts:
46
UDPSOOK
Without phase shifts:
With phase shifts:
47
UDPSOOK
Without phase shifts:
With phase shifts:
48
UDPSOOK
Without phase shifts:
With phase shifts:
49
UDPSOOK – In practice
50
UDPSOOK – In practice
51
UDPSOOK – In practice
52
UDPSOOK – In practice
53
UDPSOOK – In practice
54
Phase shift applied,two consecutive ON-states
UDPSOOK – In practice
55
56
System Overview
Channel Coding
• Message contains 128-bit verification key
• Reed-Solomon channel coding• RS(24/16) with 8-bit symbols
• Additional start symbol
• Code word length: 200 bit
57
Receiver - Camera
• Frame Rate: 30 FPS
• Exposure Time: 1/2000 seconds
58
Receiver - Camera
• Vehicle Detection with YOLO framework• Every 20th frame for real-time performance
59J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” arXiv, 2015.
Receiver - Camera
• Static estimation of taillight ROI‘s
60
Receiver - Camera
• Crop ROI‘s and scale to 28x28 pixels
61
Receiver - Camera
• Classify states of taillights using neural network• Same state as before => Bit 0
• State changed => Bit 1
62
OFF ON
Receiver – Taillight State Recognition
• Convolutional Neural Network• Trained with >4000 images of taillights
• Various car models, environments, etc. to adapt to multiple scenarios.
63
Evaluation
• Transmitter• 1:24 car models
• Microcontroller ESP8266
• LED taillights with UDPSOOK modulation
• Receiver• Canon EOS 1100D DSLR camera
• Videos recorded with 30 FPS
• Exposure time set to 1/2000 s
64
Evaluation – Single Example Video
Bold rectangle…ON Thin rectangle…OFF
Result: Bit error rate = 2.6% 18 of 20 Messages (128 bit) received correctly
65
Evaluation – Single Example Video
Bold rectangle…ON Thin rectangle…OFF
Result: Bit error rate = 2.6% 18 of 20 Messages (128 bit) received correctly
66
Evaluation – Single Example Video
Bold rectangle…ON Thin rectangle…OFF
Result: Bit error rate = 2.6% 18 of 20 Messages (128 bit) received correctly
67
Evaluation – Single Example Video
Bold rectangle…ON Thin rectangle…OFF
Result: Bit error rate = 2.6% 18 of 20 Messages (128 bit) received correctly
68
Evaluation – Single Example Video
Bold rectangle…ON Thin rectangle…OFF
Result: Bit error rate = 2.6% 18 of 20 Messages (128 bit) received correctly
69
Evaluation – Single Example Video
Bold rectangle…ON Thin rectangle…OFF
Result: Bit error rate = 2.6% 18 of 20 Messages (128 bit) received correctly
70
Evaluation – Bit error rate
71
Evaluation – Bit error rate
72
Poor BER with moving transmitter is causedby vehicle detection every 20th frame. Can be improved by more frequent detection.
Evaluation – Bit error rate
73
Similar BER in dark and bright environments
Evaluation – Message error rate
74
Evaluation – Message error rate
75
Error bursts due toambiguous taillight statescause 10% message error
rate
Evaluation – Message error rate
76
Error bursts due toambiguous taillight statescause 10% message error
rate
OFF Ambiguous ON
Evaluation – Message error rate
77
Bright environment:Vehicle Detection: easyTaillight State Recognition: hard
Singular bit errors -> corrected by channel coding
Evaluation – Message error rate
78
Dark environment:Vehicle Detection: hardTaillight State Recognition: easy
Error bursts -> not corrected by channel coding
Evaluation – First reception time
79
Evaluation – First reception time
80
Approx. 5 seconds totransmit a 128-bit message
Conclusion
• Optical Out-of-Band Channel for Vehicle-to-Vehicle Communication
• Prototypes of car models in scale 1:24
• Camera with rolling shutter very short exposure time is needed.
• Results: • BER of 3.64% on average (1.94% standard deviation)
• Approx. 5 seconds to receive the first correct message
• Can be used for identity verification in platooning
81
Thanks for reading!
For questions, please contact Michael.Plattner@fh-hagenberg.at