Development Effort of Evaluation Technology in …...44 Development Effort of Evaluation Technology...
Transcript of Development Effort of Evaluation Technology in …...44 Development Effort of Evaluation Technology...
42
Development Effort of Evaluation Technology in Automatic Parking System
Masao SENDO Nobuhito YOKOTA
Susumu TAKAHASHI
Shinichi SUGIURA
Development Effort of Evaluation Technology in Automatic Parking System
Nowadays, every automotive manufacturer is working toward practical use of autonomous driving in 2020 as a molestone, and the range of automation is expanding year by year. Accordingly, functions of autonomous driving are becoming more complicated, and in the development of sensing products, it is necessary to perform an evaluation assuming a huge market environment scene including defect factors so as to ensure market quality. This paper introduces an ideal evaluation process and development efforts of an evaluation technology to realize it for evaluation of the target recoginition performance in the automatic parking system we are currently working on.
In Japan, “Public-Private ITS Initiative /Roadmaps 2017” was announced in May, 2017. (Fig. 1) Adopting the automated driving level defined by SAE (Society of Automotive Engineers), Japan shows the stance to follow the international movement. Various automotive manufacturers already announced that they will launch an autonomous driving vehicle (level 3) in 2020. As for full autonomous driving (level 4), the activities by the private and public sectors as one toward the establishment of a system has been vitalized. When a driving assistance system began appearing on the market, the products which partially support “acceleration, deceleration, and steering” was the mainstream. However, in accordance with the change from “assistance” to “automatic”, higher safety is required from the market while the number of sensors such as image camera, millimeter wave radar and others increases and their detection range expands. In order to secure this high safety, collection of vast amounts of field driving data, and advanced analysis are needed.
Introduction 1.
Abstract
自動運転システムに係るロードマップ①:自家用自動運転車(1)
・準自動 パイロット (SAEレベル2)
・SAEレベル2
・自動パイロット (SAEレベル3)
・完全自動 運転 (SAEレベル4)
【官民(S
IP1
含む)】
大規模社会実証に
向けた検討
【官民】
制度面等の
調査検討
【官民(S
IP1
含む)】
高速道路での
大規模実証実験
制度整備方針
(大綱)策定
交通事故死大幅削減、交通渋滞の緩和、物流交通の効率化、高齢者等の移動支援
世界一安全で円滑な道路交通社会
世界最先端のITS
高速道での 自動運転
一般道での 自動運転
高度安全運転支援システム(仮称)
安全運転サポート車
【民間】研究開発・ 実用化の推進
【関係各省】安全運転サポート車(サポカーS、サポカー)の普及啓発
【民間】研究開発・実用化の推進
【民間】研究開発・実用化の推進
【民間】研究開発・実用化の推進
【民間】研究開発・実用化の推進
【関係各省】制度の詳細検討、必要な制度改正等
準自動パイロット(L2)市場化
短期 中期 長期年度2017 2018 2019 2020 2021 2022 2023~25 2026~30
自動パイロット(L3)市場化※
高速道路完全自動運転(L4)市場化※
一般道路自動運転(L2)市場化
【民間】市場展開、 更なる高度化
【民間】市場展開
高度安全運転支援システム(仮称)市場化
【民間】市場展開
【民間】市場展開
【民間】市場展開
【自家用車】
Fig. 1 Roadmap of Autonomous Driving System
This paper introduces establishment of efficient and effective evaluation process and development effort for evaluation technology with the case study.
43
DENSO TEN Technical Review Vol.1
VICT Enginering Group has launched the products in a wide range of field such as sensing, HMI (Human Machine Interface), and telematics. In sensing field, we proceed with the development of a target recognition technology with camera in addition to millimater wave radar as a sensor which functions as “eye” for realizing autonomous driving. These system consists of sensor itself and recognition algorithm. Naturally, as for developing better products, it is important to understand the required performance of the target system, and to develop it on the assumption how it is used in the market. At present, we are developing the sensing technology of automatic parking system, in which the performance responding to various parking scene is required. In ensuring performance for the system with the prefix “automatic”, major manufacturers already take action such as driving over millions km in the market field and establishing evaluation envirounment by large-scale simulation. These approaches are far larger than conventional product evaluation scale, and they are steadily expanding in future.(Fig. 2) According also to global trend, drastic evolution is required for the way of evaluation.
Fig. 2 Evaluation Range in Vehicle Automation Level
Autonomousdriving
Full Autonomousdriving
Full Autonomous
driving
System
range
Driver support type autonomous driving
Reference) Ministry of Land, Infrastructure and Transport:The study on auto-pilot system interim summary
(Autom
ation
)
(Evaluation range
)
100%
Large
Independent systemACC
Collision damage reduction systemlane keeping system
System sophisticatedFurther integration ,high accuracy
System integrationACC+
lane keeping Assist
We defined ideal evaluation as follows.① Basic performance evaluation (bench)② Market simulation evaluation (field)③ Market environment evaluation (market)
Fig. 3 shows the flow of evaluation for the target recognition performance by camera.①Evaluate a camera alone in limited environment (movement and brightness etc.) in basic performance evaluation. ②Perform evaluation which simulates market environment in evaluation field with mounting the camera on vehicle in market simulation evaluation. ③Evaluate total performance under actual environment (market) in market environment evaluation. In recent years, as for the problem of sensing development, serious problems are found at the final stage in V shape developmental process. When problem caused by basic design is found in the final evaluation stage, the range of design change expands and huge rework and development delay occurs. Then we accelerated the evaluation contents assuming market environment in the pre-stage evaluation (basic performance evaluation and market simulation evaluation), as shown in Fig. 4, and incorporated it into the basic performance evaluation, and clearly defined the index for the completion level in each stage in order to prevent a serious problem in the final stage. Also, we defined it as the ideal target to realize evaluation which covers various market environments in market environment evaluation.
Fig. 3 Product Evaluation Process
Bench evaluation
Parking scene which includes various “trace”, “object”, and “environment” affects parking performance.Evaluate performance by simple evaluation condition and improve the quality.
A unit of camera Plural camerasActual vehicle
(Plural cameras)
Evaluation environment quantified error factor
③Market environment evaluation
Market adaptation (robustness)
①Basic performance evaluation②Market simulation
evaluation
Ideal evaluation process 3.
Autonomous driving technology and evaluation
2.
44
Development Effort of Evaluation Technology in Automatic Parking System
Fig. 5 3-axis Evaluation Equipment
Fig. 6 3-axis Evaluation Equipment Specification
・Synchronous control of X,Y and θ axis・Camera mount, 360-degree rotation・Adjustment of camera angle (Pan / Tilt / Roll)
■Camera moving part
■Data output
■User interface
・Setting of origin, moving trace, and moving velocity
・CAN output of moving position and velocity
・Trace drawing by dedicated software and operation
10m
variable
10m
θ axis
Plane view
Main function
Side view
Y axis fram
e
X axis frame
Y axis fram
e
② Market simulation evaluation : In this phase, we confirm the performance under actual environment by using a vehicle equipped with camera, and recognition target object after ensuring the safety. Therefore, safe and quantitative evaluation is needed under the environment assuming the usage of products in the market and driving pattern. DENSO TEN newly established a test field in 2017. (Fig. 7) A various type of parking frames which exist in the market are reproduced in this field. Also, evaluation vehicles are equipped with highly accurate GPS (Global Positioning System) and LiDAR (Laser Imaging Detection and Ranging), and evaluation can be performed by comparing the actual position of detecting object with detected result.
We explain target recognition performance evaluation on which we work right now to realize this evaluation process. As for the evaluation, safety is the first priority, and the method should be quantitative and efficient in order to judge the product function correctly. Therefore, we develop various evaluation technique, equipment, and tool to adjust each evaluation stage and we advance evaluation and analysis. ① Basic performance evaluation: We evaluate it by image by using a standard target whether a target recognition method itself operates as planned or not. In this phase, it is important to quantify the right recognition value and affecting error factor. If the position relation between sensor and recognition target object is clear, right extracting the problem is possible by comparing the recognition results. Then, we developed the equipment which reproduces the vehicle trace by moving a sensor dynamically. (Fig. 5) As shown in Fig. 6, the sensor is mounted on the intersection point of the frame which operates in X axis direction and Y axis direction. This equipment reproduces sensor motion mounted on vehicle by concurrent control of 3 axes with adding rotation function of θ direction. As the position of intersection is calculated in absolute coordinate, we can compare actual position relation with the recognition result by grasping position relation between the origin and the recognition target object. We can verify the design feasibility and the robustness with use of this equipment.
Fig. 4 Front-Loading of Evaluation
System design
Function design
Market environment evaluation
Market simulation evaluation
Basic performance evaluation
Algorithm design
Evaluation atdesign stage
Approach by evaluation technology 4.
45
DENSO TEN Technical Review Vol.1
For example , as shown in Fig. 8, Regarding the vehicle as the object, colar, shape, and size are described as property, and behaivour such as velocity, trace, and others are defined as method, which leads to systematization. For the object of each catalog, we can create various type of evaluation scene as shown in Fig. 9 by selecting and combining property, method, and paramater.
When we simply try different combinations with all objects in all catalog, a huge number of pattern are made. In the system which automatically judges the market environment, the quality must be secured in these evaluation scenes. However, performing all these patterns is unrealistic. Then, we challenged to narrow down the paramater based on the level of difficulty. For example, detecting parking frame is realized by detecting line segments which are edge extracted by the contrast (brightness difference) between line.
Fig. 8 Example of Object Systematizaton
Fig. 9 Evaluation Scene Created by Catalog
Vehicle catalog
Vehicle object
…
PropertyBody colorShapeSize
…
Method
Param
eter
VelocityTrace
…
Parking frame catalog
Parking frame object
Person/Roadside object/Weather etc.
……
PropertyShape of lineSize of lineColor of lineColor of road surface
…
Parameterロ type/U type…Thickness:cm/length:cm
R:G:BR:G:B
…
…
Weather
Vehicle
Trace
Arrangement
Road
surface
Person
Roadside
object
Parking
frame
Fig. 7 Evaluation Field of Automatic Parking System
③ Market environment evaluation : In this phase, suitability for actual market environment is performed. It is a key how we encompass the scenes including error factor and evaluate under the market environment where a huge number of scenes exist. We introduce the effort to improve the completeness of market environment scene and efficiently evaluate the target object. Firstly, we have comprehensively extracted recognition target object which may exist in the market , and the object which affects to recognition by using brainstorming method based on past image data etc. Then, the extracted objects are grouped by type. This group is called “catalog” for convenience sake. Recognition target object which is categorized to each catalog is called object. We define colar, shape, and size as property, and define things having behaivour such as velocity, trace, and others as method. Some of property and method are quantitative and converted into numerical values.
Kobe Test Field Simulation Evaluation of Market Environment (Automatic Parking)
Combination of road surface and line color of parking frame
Asphalt road surface Concrete road surface
Handicap mark GutterSpecial road surface
Brick / Lawn / Iron plateStraight 100m Grid line Slope
Stone pavement
132m
100m
107m
坂路
駐車区画
駐車路面バリエーション
区画線種
特殊路面駐車枠
グリッド
直線路
←至
本社D棟
46
Development Effort of Evaluation Technology in Automatic Parking System
As for the development of automatic parking system, we perform efficient and effective evaluation by using evaluation technology. In future development of automatic parking system, the expansion of evaluation range is expected in accordance with system advancing and complicating. We will work on not only evaluation of actual environment but also using virtual technology and AI (Artificial Intelligence) so as to ensure the quality.
・MySQL is a trademark or registered trademark in U.S.A and other countries of MySQL, Inc. and Sun Microsystems, Inc. in U.S.A.
ReferenceThe prime minister’s official residence : Public-Private ITS Initiative / Roadmaps,pp.67 [2017]Ministry of Land, Infrastructure and Transport :The study on auto-pilot system Interim Summary [2013]
Conclusion 5. As shown in Fig. 10, we investigated the past image data and paint color of road surface, and made color chart. Then, we measured the brightness with the type of lighting and illuminance which was expected in the market environment. It is found that it is difficult to extract the edge by the combination of small brightness difference for each color measured. When the white line is used as the standard in Fig. 10, the brightness difference was smallest for light-blue color of road surface, and then detecting it was difficult. If extracting edge in these combination is possible, combination with low level of difficulty is not necessary to be performed, which enables to reduce the number of pattern as for the property and paramater of color. We proceed with efficient and effective evaluatuon by performing this kind of verification and focusing about other paramaters. Control and operation are also important because we expect that measurement data size will be huge even after focusing. As for the control and the operation, system for data search from various aspect and processing it with high speed is necceary in order to proceed with the data analysis and simulation. Then, we developed an original data base. By adopting PHP (Hypertext Preprocessor)+ MySQL as data base application with WEB-based GUI (Graphical User Interface), manager and user can operate it without burden. As for NAS (Network Attached Storage), we must prevent the data access speed from decreasing in accordance with increase of data. We introduced scall-out NAS to solve this problem which easily improves the system performance when added. (Fig. 11)
Fig. 11 Scale-out NAS
Inside Appearance
Fig. 10 Combination Narrowed Down by Color Evaluation
線/路面色 想定輝度差
【Verification process】
Market environment
Easy
Difficult
Color of “line and road surface” Brightness of “color”
=?×?Blue×Yellow
Green×Red Light blue×Gray
White×Black
試験No.
輝度
D. Extract minimum combination
A:Create color chart of line and road surface
C: Analysis of brightness difference
蛍光灯 太陽灯水銀灯ナトリウム灯LED灯
太陽灯
B:Brightness evaluation by chart D: combination difficulty for color of line/road surface
Combination
White×
Black
White×Red
White×
Green
White×Gray
White×
Light b lue
Picture Difficulty
A. Extract color of line and road surface in market
B. Calculation of brightness in market
C. Verify combination of brightness difference
47
DENSO TEN Technical Review Vol.1
Profiles of Writers
Susumu TAKAHASHI
VICT Engineering GroupSystem Evaluation Dept
Nobuhito YOKOTA
VICT Engineering GroupSystem Evaluation Dept
Masao SENDO
VICT Engineering GroupSystem Evaluation Dept
Shinichi SUGIURA
VICT Engineering GroupSystem Evaluation Dept