ITS and Telematics in Japan - Computer Vision and Robotics...
Transcript of ITS and Telematics in Japan - Computer Vision and Robotics...
ITS and Telematics in Japan
Katsushi IkeuchiThe University of Tokyo
Outline
BackgroundIn-Vehicle terminals– Car navigation– ETC– AHS & ASV for safe driving
Road Traffic InfrastructureU Tokyo Research Program
Organization
Person
Vehicle
Advanced Information Society Dev. HQ(Director: Prime Minister) 8/1995
7/1995
4 Ministry Meeting(Police,Post&TelecomTransport,Commerce)
ITS Japan
1/1994
ITS World
Congress
Industries Universities
ISOCommit.
TC/204
9/1992
Navigation Systems
Emergency Vehicle Op.
PedestriansSafe Driving
Traffic Management
Road Management
Public Transport
Electronic Toll Collection
Commercial Vehicle Op.
ITS Key Development Fields in Japan
Characteristics of Japanese Market
Navigation
Driver Service (from 2003)
Safe Driving Assistance (from 2005)
Information for Drivers
Information for Drivers
StaticInformation
DynamicInformation
DynamicInformation
Package Media
Communication
- Road Map- City Map, etc.
- Traffic Congestion- Traffic Accident- Road Construction- E-mail, etc.
Information for Drivers
4th GenerationMobile Phone
ETC
10k 100k 1M 10M
Mobility
Data Transfer Rate (bps)
PDC
PHS
IMT-2000
Radio Beacon(VICS)
MMAC
Optical Beacon(VICS)
ETC
Digital Broadcasting
DSRC
Stationary
WalkingSpeed
High-SpeedDriving
Communication System in ITS
Communication Infrastructures
1980 1990 2000 2010
PSTN
High SpeedDriving
Movin
g Speed
1k 10k 100kData Transfer Rate (bps)
1st Generation 2nd Generation 3rd Generation 4th Generation
Analog MobilePhone
(High-CAP)
Digital MobilePhone(PDC)
IMT-2000(Japan W-CDMA)- Global Standard- Multi-media Capable(64K - 384k - 2Mbps)
IMT-2000(Japan W-CDMA)- Global Standard- Multi-media Capable(64K - 384k - 2Mbps)
FPLMTS ( )10Mbps
High Speed Move
CordlessPhone PHS
Wireless ATMMulti-media Capable(up to 12Mbps)
20-40GHz10 Mbps
ATM
IMT:International Mobile Telecommunications
WalkingSpeed
Stationary
i-Mode(PDC Packet)
Roadside-Vehicle Communication
Electro-magneticInduction
Milli-meterWave
1996 1997 1998 1999 2000 2001 2002 ---
Bus Locators
Radio Beacon(2.5 GHz:VICS) Electronic Toll Collection System
(5.8GHz:ETC)
Optical Beacon (800nm :VICS --> Nationwide expansion
MilliWave
RadioWave
Optical
DSRC
In-vehicle Terminals
Evolution of In-vehicle TerminalAdvances in Navigation System
DVD Car Navi
- Advances in Search Function
- Touch Panel
- Operation and Guide by Voice
- Handy Navigation, etc.
Mobile Communication- VICS- Interactive Communication via
Mobile Phone- Internet- Digital Satellite Broadcast- Information Contents Industry,
etc.
VICS Navi
Safety and Comfort
- Rear Monitor
- Blind Corner Monitor
- Zigzag Driving Detection Function- Shift Control Coordination
On-board Camera, HELP
A Multiplicity of Functions
AVN, ETC
- FM Multiplex with Subtitle- Coordination with TV and
Audio Systems- Display and Operation of Air
Conditioner- On-board Personal Computer
Car Navigation
In-vehicle Terminal“Car Navigation”
History of Car Navigation
A/M
OEM
Navigation Sold in Market Total 7.8Mil. VICS Sold in Market Total 2.8Mil.
Car Multimedia
Comm. NaviInternet
AVNDVD
D-GPS
VICSVoice Recognition
Detailed City MapVoice Guidance
Route Guidance
Road MapGPS
(Unit: 000)
2,500
2,000
1,500
1,000
500
091 92 93 94 95 96 97 98 99 2000 2001
1,300
861
627
337
1493012
1,010
43%
1,950
1,530
57%
DVD Qty(Est.)
(2,400)
1,400
35%
128314
577796
989
(1200)
VICS Qty(Est.)
GPS
GPS Antenna
1. Indication of Self-position2. Search of Destination (Destination Input)3. Calculation of Route4. Guidance to Destination
LCD Monitor
Navigation Computer
Speed Sensor
Map Disc
DVD-,CD-ROM
Architecture of Car Navigation System
3D Magnified Intersection Map 3D Satellite Map
•WIDE-VGA display•Gyro, D-GPS, Map-matching•Voice Control•Route Search (64 bit bus Risc CPU)•Destination Search (Address, Tel#)•Connectivity to VICS, i-mode, HELPNET, ETC
DVD Car Navigation Systems (e-navi:fine sped network)
VICS Center- Established in ’95 and Started in Apr., ’96.
- Road Traffic Info. In Real Time through Communication Media.
- Media: Radio Beacon, Optical Beacon,FM Multiplex Broadcast
Car Navigation- Road Traffic Info. Displayed by Maps,
Diagrams and Characters
- Less-congested Route Guidanceby a Triple Media Receiver in 1-DIN Body
Vehicle Information & Communication System (VICS)
Information Collection Information Providing
Ministry of ConstructionNational Police Agency
Japan Road TrafficInformation
Center
Japan Road TrafficInformation
Center
OtherInformation
Sources
OtherInformation
Sources
Radio BeaconRadio Beacon
Optical BeaconOptical Beacon
FM Multiplex Broadcast
Traffic Congestion Info.Map Display
Traffic Management
Center
VICS Center Communication Media
Car Navigation
Traffic Congestion Info.Diagram DisplayInformation Processing
!
!
First Phase Service Start Mar.1998
Tokyo, Chiba, Saitama, Kanagawa, Osaka , Aichi, Kyoto, Hyogo,Nagoya National Expressway
Second Phase Sep 2001
Fukuoka, Hiroshima, Miyagi, Hokkaido(around Sapporo)Shizuoka, Gunma, Okayama, Fukushima, Okinawa, Miyazaki, Gifu, Mie, Yamaguchi, Ibaragi, Hokkaido(around Asahikawa) Wakayama, Shiga, Nara, Tochigi,Yamanashi
Third Phase around 2005
Niigata, Ishikawa, Hokkaido( around Hakodate), Kumamoto, Oita, Kagawa, Ehime, Saga, Nagasaki, Kagoshima, Tokushima, Kochi, Fukui, Toyama, Yamagata, Akita Aomori, Iwate, Shimane, Tottori
9
20
VICS Service Coverage
Restaurant/Hotel/Gas Station, etc.
JapanRoad & TrafficInformation
Center
Kanagawa/Chiba/SaitamaTraffic Control
Centers
Japan HighwayPublic
Corporation
Police AgencyTraffic Control
Center
Tokyo MetropolitanExpresswayCorporation
In-vehicle Navigation System Personal Computer ATIS Terminal
Office/Center
Traffic Information Service ATIS/ITGS
MobileTelephone
TelephoneLine
Car
TelephoneLine
ATIS / ITGS Bi-directional Traffic Info. Service
Route guidance
I-mode functionsTransactionNews, Weather forecast, Town infoGame, KaraokeE-mail
i-mode Car Navigation
-Road traffic info. (Congestion, Travel time, etc.)-Weather Info. (Fog, Snow, Icy road, etc.)-Parking Info. Opening hours, Tariff, Availability-Reservation (Hotel, Restaurant, etc.)-Help (Accident, Illness, Theft, etc.-E-mail, Internet, Music download
IP (Information Provider)
Center
IMT-2000
- Voice- Data- Moving
Picture
64kbps
384kbps
Camerai-mode
HELP
H/F
GPS
I/F USB Bluetooth
ETC
DVD map
SD card
MPEG4
Voice Recognition
IMT-2000 Car Navigation
- IMT-2000 Terminal Equipped with GPS Unit.
- Download a Map and Indicate the Present Position.
- Search for POI.
- Route Guidance.
IMT-2000 (W-CDMA) Walking Navigation
Electronic Toll Collection
Electronic Toll Collection (ETC)
Vehicle Detector
CameraEquipment for Monitoring
Lane Control Equipment
Roadside WirelessEquipment
Toll Booth Roadside System ETC Vehicle Unit5.8GHz, 1Mbps
Bi-directional DataCommunication
- Reduction of Traffic Jams by Non-stop Automatic Toll Collection(35% of Total Jams are at Toll Booth)
- Started Around Chiba Area (Mar. 2000)- Service Upgrade by Cash-less System- Reduction of Management Cost of Toll Collection
Core Tech. :Toll Collection / Lane Management / Vehicle Detection /Licence Plate Recognition
Main Unit
Antenna
Core Tech.:Road to Vehicle CommunicationCoding / Vehicle Unit / Car Navi.
!
!
Operational Section(164 Tollgate)Planned Section Year 2001(600 Tollgate)
* 70% of Tollgate in Japan will be Coveredby the End of FT 2002 (900 Tollgate)
Development Plan
Optical Cable
Server
ETC Center
SA/PA, Michi-no-EkiManagement
ETC Terminal
ETC Terminal Connected with Navi.
Year 2000
ETC/DSRC Dual TerminalASK/QPSK)
ETC/DSRC Dual TerminalConnected with Car Navi.
Simplified Road-side Radio Equipment
ERP
Michi-no-Eki,SA/PA
Gas Station
Drive Through
Parking
Led by Government Agency
Led by Private Business
ERP Center
Charge Clearance, Provision of Information
Charge Clearance
Deployment of ETC and New ETC/DSRC System
Server
Server
Emergency Report System
Lung Resuscitation within Two Minutes of an Accident: 90% Chance of Saving the Life.
Accident Occurs Report Ambulance Arrivesat the Scene
Arrival at Hospital
Total:7.3 min.1
8.9
1
1.3
5.3
6.1Without HELP
With HELP
Total:16.3 min.
Emergency Reporting System (HELP)
HELP architecture
Rescue
Automatic Report
Manual ReportAccident Position
AccidentIllness
110
Police Dept
119
Fire Dept.
Emergency Report Center
Vehicle Unit
Road Service
Emergency Vehicle Support
Rescue
Cellular Telephone
GPS
Battery
OHM
NavigationSystem Air-
Bag Crash Information
Location Information
TEL AntennaManual Report
Accident
Power
PDC Module
Primary Battery
HELPTerminal
(Manual Button, Microphone, Indicator)
HELP Terminal
BrowserSoftware
(Smart Media)
EmergencyButton
LCD monitor
Speakerfor
Voice Guidance
Microphonefor
Voice Recognition
Hands-freeSet
MobilePhone
Modem
DedicatedMap DVD
Replace withStandard MapDVD
Car Navigation SystemCar Navigation System Mobile Communication UnitMobile Communication Unit
CD-ROMfor
HELP Function
CD-ROMfor MONET,
i-modeFunction
HELP Terminal (Option for Navigation System)
In-vehicle TerminalAssistance for Safe Driving
AHS: Advanced Cruise-Assist Highway System
2000 Proving Test Program “Smart Cruise 21”Oct.-Dec.: The Public Works Research Institute
of the Ministry of Construction
2003 Start Installation of “Smart Cruise 21”
2010 Establishment of 150,000 km long “Information Highway” Optical Fiber Network
2015 Realization of “Smart Way”
By AHS Research Association
Smart Cruise 21
Preventionof Lane Departure
Preventionof Overshooting on
Curve
Preventionof Collision with
Pedestrian
Maintaining Headway
Preventionof Right Turn Collisions
Preventionof Crossing Collisions
Preventionof Collisions with
Obstacles
- Blind Spot Assist (Parking Assist,Blind Corner Monitor Synthesize Several Images and Provide Wide Angle Image
- Visibility Enhancement (Night Vision) Obstacle Detection Using Infrared/Visible Camera and Warning System
- Reduction in Driving Operation(Look Off Detection, Stop & Go) Assist Using Camera Image, Radar, Lane-Marker
Image SynthesesImage Syntheses
Camera Image
Synthesized Image
Rear-viewCamera
Left Door Mirror Left Pillar
Image ClusteringImage Clustering
Real Image
Clustering
Infrared/VisibleCamera
Infrared/VisibleCamera
Detection of Face Direction
Lane-Marker
Detection of Face Direction
Lane-Marker
Night Vision
- Radar- Distance Measurement
by Stereo Image- Lane Marker
- Radar- Distance Measurement
by Stereo Image- Lane Marker
Sedan
Truck
Parking Assist
Look OffDrive
Detection
Blind Monitor
Stop & Go
ASV:Advanced Safety Vehicle
Right Pillar
Right Door Mirror
Rear view camera
108 wide angle back view 44 normal angle side view
Metal Housing CaseResin Housing Case
Front lateral view camera
From dealer’s option manualTaken on photo
On-board Monitoring Camera
Road Traffic Infrastructure
Mobile Communication
Diversity of Functions
Traffic Control SystemRoad Information System
- Beacon- FM Multiplex Broadcasting- Mobile Phone- Internet- Digital Broadcasting
- Collection of Traffic Info.- Collection of Weather Info.- Processing of Traffic Info.- Provision of Traffic Info.- Control of Traffic Signal
- ETC- Road Pricing- Dynamic Route Guidance- Priority Passing for Public Transport
- Support for EmergencyVehicle
- Decrease of Pollution
- Emergency Report- Anti-Theft, Trace- Traveling Support Road-Support for Pedestrian
Evolution of Road Traffic InfrastructureAdvances in Functions
Safety / Comfort
Information Collection Information Processing Traffic Control
Information Provision
AVI Sensor 2,200
Traffic Flow Measurement :2,050Traffic Control Center 170 Cities
Traffic Control Center(Processing System)
Traffic Signal
Multi-Pattern Display
Free-Pattern Display
Road-Side Communication
Ultrasonic Sensor:120,000Optical Beacon:36,000 165,000
2,700
810
Police Traffic Management System
90
(Inside Controlling Area:57,000)
Incident Detection System
Image Recording Device
Control Processing Device
Device for Detecting Unforeseen Occurrences
CCD Camera
CCD Camera
System is Controlled by Images
from Each Camera
CCD Camera
Display Board
Provide Information to the Following Vehicle
Travel Time Information Board
15
AVI(Automatic Vehicle Identifier) System
Measured Travel Time
License PlateLicense Plate
Image RecognitionImage Recognition
AVI: Automatic Vehicle Identifier
Recognition Unit
Sensor CameraCameraFlash
AVI Central Unit
AVI Terminal Unit(Point B)
AVI Terminal Unit(Point A)
- Prepare Travel Time Information
- Prepare Clustering Information
- Prepare OD Information
Bus Locating System
Electromagnetic InductionEffect Creates a SignalEach Time Buses Pass by.
○○
““
○○
Bus Approach Indicator Shows the Bus Location with a Flashing Light.
Data Collecting Station
Data Providing Station
Voice Command Station
Location Monitoring Servers
Operation Status Displays
Host Computers
Data AuthoringSystems
Center System
Monitor Bus Location Via Two-way Communication System Using Electromagnetic Induction Antennas at Bus Stops.Central Data Collecting Station Analyzes Bus Transit Information Supplied through MCA Control Station.Central Data Providing Station Delivers Bus Operation Status/Approaching Information to Each Bus Stop through MCA Control Station.
Bus ApproachIndicator
““
Social BenefitsSocial Benefits
Optimum DispatchingOptimum Dispatching
Character/VoiceMap display
Character/VoiceMap display
ManagementEfficiency
ManagementEfficiency
CustomerManagementCustomer
ManagementAutomatic
DispatchingAutomatic
Dispatching
Customer ServiceImprovement
Customer ServiceImprovement
Reduction of Driver’s StressReduction of
Driver’s Stress
OperationRecord
Management
OperationRecord
Management
Closest TaxiSearching
Closest TaxiSearching
IncidentInformationIncident
Information
Taxi Dispatching System
Taxi Radio
DestinationRoute Searching
DestinationRoute Searching
Fleet Operation Management System
GPS
Radio communication Controller
Control StationAC Power Supply
Controllerand
Microphone CPU
Office-side System
● System Specifications (Capability)Connectable Stations 2 Control Stations/systemMax. Vehicles 200 Vehicles/SystemNumber of Zone 16 Zones/SystemCustomer Listing 20,000 Customers/SystemNumber of Registered Conditions 99 Items /SystemPre-registered messages 15 Messages/System
10 Messages/Control StationRegistered Condition Selector
Option
GPS Receiver + GPS Antenna
Mobile Radio + Communication ControlUnit
Mobile System
- By Using MCA Radio Communication, an Office Display Indicates the Fleet Location and Status.- Computer Finds the Customer’s Location from Their Name or Telephone Number and Indicates on the Display.- Customer Information is Provided to a Delivery Truck Closest to the Customer.
(Note)
(Note) Car Navigation Unit can be used as the Mobile Radio Unit.
- Customer Location- Vehicle Location- Vehicle Direction/Speed- Trace of the Fleet
Digital MCA Control Station
Microphone
U Tokyo Research Program
Information gathering– Recognition of vehicle in infrared images– Land mark recognition– Event recognition at intersection
Information display– Modeling environmental map through vision– Visualization of arbitrary views for driver
Recognition of vehicles(in Infrared Images)
Eigen-window method•Recognize vehicle without reading number plates•Possible under occlusion•No need to segmentation
Previous methods•Detection; not recognition•Vehicle class by size
Eigen-window methodTraining images
Input images
votingEigenspace
Recognition results
Car-A
Voting space : Pos x Model#
Recognition results (outdoor)
daynight
daynight
Event recognition
TV camera
Surugadai Local analysis
Sakauchi LabNTT Optical fiber line
Event DBapplication
Event recognition
StoppingIllegal parking
Tracking
Delete
Extend
Move Blocks based on
Estimated motion vector
Adapting to
new formation
Next frame
Current frame Next frame(Middle situation)
Adaptive algorithm
Feature Extraction
1
2
Normalization Indexing
8
Motion vectors
HMM
State
0
State
1
State
n
a01
b01
(k)
a12
b12
(k)
an-1n
bn-1n
(k)
a00
b00
(k)
ann
bnn
(k)
a11
b11
(k)
Left - to - Right HMM
♦ BumpingClose in Impact Repulsion
Close in
Close in
Stop Restart
Overlap Pass
♦ Tandem
♦ Passing
Accident Models in HMM
0 1 2 3 0 8 7 0
0 1 2 0 8 7 0
0 1 2 3 2
Yokohama-harajuku AHS
Accident recognition
Accident recognition
Modeling Urban Scene
Omni camera
Moving Direction
1. Image base rendering2. Model base rendering3. Augmentation
Data Acquisition Vehicle
Outline
Acquiring omni-images along a path
Generate another images along a different path
Panoramic image
Perspective images
Image capturing
Basic idea
P
C
Synthesize image
C
B
A
B
A
2D image from light field
Synthesize Sequence(Changing lanes)
Synthesize Sequence(Rotation)
Merits
Image sequence different from acquired one
Acquired Synthesized
Constructing 3D Map
Omni image acquisition
3D digital map by Omni analysis
Texture mapping
Omni Spatio-temporal Image
Cross-section (an elliptic curve)D
epth Info
Digital Map
Depth info +
Correspondence between map and image
Cross-section (a radius line)
Panorama image
3D Map
Virtual Surugadai intersectionCurrent situation into virtual worldWarning to drivers
Outline
Tracking vehicles
Combine VR vehicles with the VR Surugadai
Generate arbitrary views with transparent building
Tracking
Making paths to run
Projecting detected coordinates onto the virtual world coordinateMaking paths to run by connecting them
Tracked images The virtual world coordinate system
Transparent buildings with vehicles
Modeling from Reality
Real Object
VR Image
Modeling from Reality
U Tokyo Program
Information gathering– Vehicle recognition based on Eigen window– Event recognition based HMM
Information display– Virtual Surugadai intersection – Modeling urban scene using Omni images
Summary
BackgroundIn-Vehicle terminals– Car navigation– ETC– AHS & ASV for safe driving
Road Traffic InfrastructureU Tokyo Program