Night Time Vehicle Detection for Real Time Traffic Monitoring ...

Night Time Vehicle Detection for Real Time Traffic Monitoring Systems: A

Review

Padmavathi.S1, Keerthana Gunasekaran2 1,2Department of Computer Science, Amrita School of Engineering, Coimbatore.

Abstract

Real time traffic surveillance using computer vision

system is an emerging research area. Many new

algorithms are being developed to perform the

surveillance in the most effective manner. The first and

critical step in these road traffic monitoring systems is

to detect and track the vehicles. In this paper, we

provide a brief review on the night time vehicle

detection techniques that have been used in the recent

years. The detection of vehicles in the night time can

prove to be challenging because the usual features of

the vehicles like the vehicle shadows, horizontal and

vertical edges that helps in the identification in day

time cannot be used during the night time. The only

salient features that are visible in the night time are

headlights, rear-lights and their beams, street-lamps,

horizontal signals such as zebra crossings and traffic

scenes with reflectors. Thus, in night time surveillance

the target objects are the vehicle headlights and rear

lights.

1. Introduction

The data for the real time traffic monitoring systems

can come various sources like the loop detectors,

ultrasonic detectors, microwave sensors, radar sensors

or video cameras. Due to the recent advancement in

computer vision and image processing techniques, the

video cameras have been found to be an efficient means

to collect and analyze the traffic data. Video based

camera systems are more sophisticated and robust

because the information that is associated with the

image sequences present in the video allows us to

identify and classify the vehicles in the most effective

manner. The temporal continuity of data in video

stream helps in improving the accuracy during vehicle

detection. A video based monitoring system must be

able to handle various weather and illumination

conditions.

The vehicle detection in day time can be done using

various methods which can be motion based,

knowledge based or appearance based. The various

techniques that are commonly employed in day time

vehicle detection is comprehensively reviewed in [1].

The methods mostly use the edges in the vehicle to

identify the moving objects in the screen through frame

differencing methods. Stauffer and Grimson [2] in 1999

published a novel method to detect the moving vehicles

using background subtraction. In this method each

pixel was modeled as mixture of Gaussians and based

on the variance and persistence, the Gaussians which

corresponds to the background colors were determined.

The pixel values that do not fit the background

distributions were considered to be foreground. This

method was able to handle very slow illumination

changes and the multimodal distributions caused by

swaying trees. This method was further improved in [2]

where an algorithm was proposed to learn the

descriptive mixture of the first few frames and the

result of this algorithm was used in the Gaussian

mixture model. In [3], a spatial temporal technique was

used to detect the vehicles. This method exploits the

information in the moving points, which can be

gathered by the difference between the successive

frames and the variations in the luminance in these

points. The SVM classifiers can also be used after

extracting the required features as suggested by [3].

The vehicle detection can also be obtained by using

Gabor filter and Kalman filter to predict the next

position [4]. Apart from these common methods, the

vehicles can also be detected using the changes in the

optical flow as reviewed in [5] and all these methods

try to accommodate the illumination changes and

weather conditions.

The methods used for the day time vehicle detection

cannot be used for the night time vehicle detection due

to various factors. In the night time, the bad

illumination causes strong noise which increases the

complexity of the detection task. The reflection of the

beams of headlights and rear lights can cause lots of

false alarms during the detection process. The moving

reflections of the headlights can introduce a lot of

foreground or background ambiguities. Moreover, at

night the camera images have very low contrast and a

weak light sensitivity thus making it difficult to use the

normal day time detection methods. An effective

method is to use infra-red thermal cameras as night

vision sensors to collect the traffic data during the night

Keerthana Gunasekaran et al, Int.J.Computer Technology & Applications,Vol 5 (2),451-456

IJCTA | March-April 2014 Available [email protected]

451

ISSN:2229-6093

time. This way the normal detection techniques can be

used. But unfortunately, this method is very expensive

and can be rarely seen in the real time scenarios and

moreover installing two different cameras for night and

day time is futile.

The night time environment can be classified into two

types. The unlit scene where there are no street lights

and the only features visible are the headlights and their

reflections. The headlights appear as a bright round

blob in contrast to the dark surroundings. The second

type is the lit scenes where the streets are illuminated

by the public light poles like most of the urban areas. In

this type of scenes, the background, pedestrians and

other objects are also visible. Here the headlights are

not clearly visible, especially when the vehicles are also

in white or some light colors. Hence the complexity of

extracting the headlights from the images is increased.

In this paper, Section II gives the general description of

the night time vehicle detection algorithms. Section III

sheds light on the various methods used in feature

extraction and Section IV provides an insight into

various methods used for headlight pairing and

verification. Section V elucidates the conclusion and

references.

2. Night Time Vehicle Detection Most of the night time detection algorithms use the

headlights and tail lights as the main features in the

detection of vehicles. The general template of the night

time vehicle detection algorithm as shown in Fig.1

consists of mainly four phases namely the

preprocessing phase, feature extraction and

classification to detect the vehicles.

Fig 1 – General Algorithm for night time vehicle

detection

The preprocessing phase is optional and is responsible

for the offline configuration of the traffic scene as

specified in [6]. This phase mainly consists of the

calibration of the camera and collecting the required

information about the background, lanes and camera

parameters which can be used in later stages to judge

the traffic flow and vehicle status. The preprocessing

stage also removes the noise using smoothing filters

and making the image easy to process.

3. Feature Extraction

The main features that can be used in the night time

vehicle detection are headlights, rear lights and

windshields. The various methods that can be used in

the extraction of these features are described in this

section.

3.1. Headlight Detection

Headlights are strong and consistent features which can

be used to reveal the presence of a vehicle at night. The

major difficulty lies in segmenting the headlights pixels

from the other bright pixels that are present in the

scene. Usually, the contrast between the headlights and

their surrounding is very high and the histogram of the

images of the night environment is bimodal. The

headlights can be identified based on the reflection

identity map and the reflection suppression map as used

in [7] but the method is complex and is not used

frequently. Most of the algorithms have two main steps

to detect the headlights namely bright pixel extraction

and headlight identification.

3.1.1. Bright Pixel Extraction

Third-order headings, as in this paragraph, are

discouraged. However, if you must use them, use 10-

point Times, boldface, initially capitalized, flush left,

preceded by one blank line, followed by a period and

your text on the same line.

3.1.1.1. Non-linear Luminance

The bright pixels can be extracted by extracting the

connected pixels that maximize the value of V/S ratio

(value on saturation) on the different color channels (R,

G, B). This can be done using the non-linear luminance

as the threshold value [8]. This can be followed by

white top hat transform to detect the presence of

vehicles.

) )) Eq. (1)

3.1.1.2. Canny Edge Detection

The first step is to smoothen the image using Gaussian

filter. The smoothened image is then differentiated

along the X and Y directions and non-maximum

suppression is performed as suggested in [8]. Finally

Input Video

Preprocessing

Feature Extraction

Vehicle Identification

(Analysis of Features)



452

ISSN:2229-6093

double thresholding is performed to get the edges of the

image. The binary image b(x, y) is created from an

intensity image I(x, y) according to the Eq. (2).

) { )

Eq. (2)

where T is the thresholding level. The thresholding is

done on the obtained region of interest. This method

used in [9] is very effective because the edges are not

missed n this detector and there will not be an image

without any edges present. However, the canny edge

detector can be improved by using the fuzzy

enhancement algorithm instead of smoothening the

image using simple Gaussian filter [10]. Before

performing the fuzzy enhanced canny edge detection,

the spatial homomorphism can be performed to

eliminate the illumination effects [10].

3.1.1.3 Multilevel Thresholding

The bright pixels can be extracted using the

thresholding. Ostu [11] has determined the optimal

threshold by maximizing the between class variance of

bright and dark regions. This method is improved in

[12]. The normalized probability is calculated as Pi.

The image I is split into k+1classes using k threshold

values. Here T is taken as the threshold set and Ci

represents the classes that the pixels are grouped into

using the thresholds. The between class variance and

within class variance can be calculated using the Eq.

(3) and (4) respectively

) ∑ )

Eq. (3)

) ∑

Eq. (4)

Where denotes the total variance and the overall mean is

denoted by .

∑

Eq. (5)

∑

Eq. (6)

The values , and represents the cumulative

probability, mean and standard deviation of class

Cn The separability factor can be calculated as follows

:

)

)

Eq. (7)

When the separability factor approaches to 1, the

classes of gray levels are completely separated. When

the following condition is satisfied, the largest

threshold value is used and the bright objects are

separated.

Eq. (8)

3.1.2. Headlight Identification

The bright pixel extraction extracts all the bright pixels

in the particular frame. But not all of these pixels

correspond to the headlights. They can be street lights,

reflections or other nuisance lights. To separate the

headlights from the other lights the following methods

are used:

3.1.2.1 Morphological Filtering

The morphological filtering can be used to remove

false positives. These functions can be used to remove

objects fewer than „p‟ pixels. The erosion and dilation

operations can be performed with the same structuring

element as mentioned in [9].The default connectivity is

8 for two dimensional image.

) Eq. (9)

3.1.2.2 Temporal Information

After the bright pixels are extracted, temporal

information can be used [13]. The past car light is used

to predict the next light position thus narrowing down

the area in which the headlight can be detected. This

method suggested in [14] helps in finding the vector of

light positions.

3.1.2.3 White Top Hat Transform

This method used in [15] is a powerful method for

detection of contrasted objects on non- uniform

background. The white hat transformation is defined as

the residue between the original image and its opening

[16]. The top hat transform is basically used to modify

the contrast of the image but the false regions can also

be removed using this principle.

) Eq. (10)

3.2. Rear Light Detection

A rear light can be detected effectively with the help of

red light filters. The threshold here is set using HSV

color space and the lights are detected [17] - [18].

Many different color spaces with widely varying

parameters have been used to segment red-color light

regions from images. The luminance(Y) and the red

component (Cr) can be combined to form a single



453

ISSN:2229-6093

component RGC. This RGC can be used as a threshold

to extract the lights [19].

Eq. (11)

3.3. Windshield Detection

Along with the headlights, windshield detection can

also aid in the detection of vehicles in the night time. In

the night time, the vehicle consists of a dark windshield

and pair of bright headlights. After determining the

position of windshield (between two headlights), the

median value of all the pixels in the windshield is

calculated. If the median is a dark value, then the

presence of vehicle can be confirmed. In [20] for each

pixel in the image, it is assumed that it is the center of a

front windshield and then a 3D windshield model is

projected to the position of that pixel. The probability

of the vehicle presence is calculated using the shape

and edge matching likelihoods.

4. Vehicle Identification After the identification of the features, the features

should be analyzed and the vehicles should be

identified as a whole. The methods that can be used to

analyze the features are as follows:

4.1. SVM Classification

Many of the vehicle detection algorithms use SVM

Classifier to confirm the presence of vehicle in the

scene [21]-[23]. The group of pixels can be combined

together and the eigenvalues are found. These values

are given as the input to the classifier [24]. The SVM

classifiers can be trained with many attributes of the

vehicles like the

Area in component in pixels

Coordinates (u,v) of the object‟s centroid

Estimated Distance of the Object‟s Centroid

Hat Value

Rectangularity

Aspect Ratio

Length of the object‟s contour

Circularity

Angle value of each light object‟s centroid

An input vector was defined for the classifier.

The output of the support vector machine is simply the

distance from the hyper plane. This distance will help

in the making the decision of whether the analyzed

object corresponds to a vehicle or not and this can also

be used as a threshold for separating nuisance light

sources and vehicles. For creating the training and test

sets, the ratio between positive (vehicles) and negative

(mainly reflections of traffic signs and headlights) must

be set to an appropriate value in order not to produce

wrong learning or a high percentage of false positive

detections (signs classified as vehicles) during the tests.

With the help of the training and testing data sets, the

vehicle presence can be estimated accurately using the

support vector machines.

4.2. Rule Based Component Analysis

A The identified vehicle headlights can be paired with

each other if certain rules are satisfied as mentioned in

[12],[26], [27]. These rules are based on the statistical

features of the vehicle headlights. The rules that can be

used in the estimation are

The components must be horizontally close to

each other and the vertical and horizontal

positions should be considered.

They have highly overlapped vertical projection

profiles.

The components are of similar size.

The width to height ratio of the bounding box

enclosing the two components must be greater.

Area of the pixels must be similar.

The symmetry condition must be satisfied.

The components should have a high horizontal

projection profile.

4.3. Symmetry Based Identification

Symmetry can be used to filter potential vehicle

candidates and make light pairs. The light pairs which

belong to the same vehicle are always symmetrical in

size, shape and intensity in normal conditions. This

property can be used to detect the vehicles in night time

environment. The correlation factor between two lights

can be calculated and if the result is within the allowed

threshold then it is concluded that both the lights

belong to the same vehicle as mentioned in [27]. The

correlation between two components can be calculated

using

∑ ) ̅) ) )̅

Eq. (12)

Where ̅and are the mean and standard deviation of

the component T and ̅and are the mean and

standard deviation of the component I. The correlation

factor is calculated for different color channels to

utilize the color value and to get a better accuracy in

grouping headlights together as a single vehicle. This

method is mainly used to identify the vehicle in back

view after the rear light identification. Apart from the

correlation factor, the intensity of the pixels on either

side of the symmetrical axis can also be compared to

observe the symmetry between two regions.



454

ISSN:2229-6093

The below table provides the accuracy of various

vehicle detection methods based on the previous

experiments conducted in [6]-[28].

Table -1

S.No Vehicle Detection

Methods

Accuracy False

Alarms

1 Rule Based

Identification

98% Low

2 Symmetry Based

Identification

92% Medium

3 SVM Classification 95% Low

4

Hypothesis

Generation and

Verification

94% Low

5. Conclusion This paper presented a critical review on the techniques

used in the night time vehicle detection based on the

data obtained from the stationary video surveillance

camera. These detection techniques coupled with the

tracking algorithms can increase the accuracy and

efficiency in great detail. Today many kinds of robust

tracking algorithms are available to aid vehicle

detection and traffic surveillance [29]. With the

reducing cost of the hardware and the increasing

demand for the intelligent transportation systems,

vehicle detection especially in the night when there is

heavy traffic flow will continue to be one of the hottest

research areas.

6. References [1] Wherever Z. Sun, G.Bebis, and R.Miller, “On-road

vehicle detection: A review,” IEEE Transactions

Pattern Analysis and Machine Intelligence, vol.

28, no. 5, pp. 694 –711, May 2006.

[2] C. Stauffer and W. Grimson, “Adaptive

background mixture models for real-time

tracking,” Proc. Conf. Computer Vision and

Pattern Recognition, vol. 2, pp. 246–252, June

1999.

[3] N.Greggio, A.Bernardino, C.Laschi, P.Dario and

J.S. Victor, “Self-Adaptive Gaussian Mixture

Models for Real-Time Video Segmentation and

Background Subtraction” in Proc. 10th

International IEEE conference on Intelligent

System Design and Application, November 2010.

[4] M.Celenk, J.Graham and S.Singh, "Traffic

Surveillance using Gabor Filter Bank and Kalman

Predictor", Proc. of 3rd

International Conference

on Computer Vision Theory and Applications,

January 2008.

[5] Z. Sun, G.Bebis, and R.Miller, “On-road vehicle

detection using optical sensors: A review,” Proc.

of 7th International IEEE Conference on

Intelligent Transportation Systems, October 2004.

[6] D.Neelima, G.Mamidisetti, “A Computer Vision

Model for Vehicle Detection in Traffic

Surveillance”, International Journal of

Engineering Science and Advanced Technology,

vol. 2, issue 5, pp 1203 – 1209, September 2012.

[7] N.Gajalakshmi, “Detection and pairing of Vehicle

Headlight in the Night”, International Journal of

Research in Engineering, Vol. 3, issue 9,

September 2013.

[8] K. Robert "Night-time traffic surveillance: A

robust framework for multi-vehicle detection,

classification and tracking", Proc. of 6th

International IEEE Conference on Advanced Video

Signal Based Surveillance, pp. 1 -6, September

2009.

[9] V.B. Sutar and L.S. Admuthe, “Night time Vehicle

Detection and Classification Using Support Vector

Machine”, IOSR Journal of VLSI and Signal

Processing,vol.1, issue 4, pp.01-09,December

2012.

[10] Wei-zhi Wang and Bing-han Liu, “The Vehicle

Edge Detection Based on Homomorphism

Filtering and Fuzzy Enhancement in Night-time

Environments”, Proc. of IEEE International

Conference on Intelligent Computing and

Intelligent Systems, pp. 714 – 718, October 2010.

[11] N. Otsu, "A Thresholding Selection Method from

Gray Level Histograms," IEEE Transactions on

Systems, Man and Cybernetics, vol.9, pp.62-66,

January 1979.

[12] Yen-Lin Chen, "Nighttime Vehicle Light

Detection on a Moving Vehicle using Image

Segmentation and Analysis Techniques", WSEAS

Transactions On Computers, vol. 8, issue 3, March

2009.

[13] S.Nandhini and P.G.S.Parthiban, “Automatic

Vehicle Detection during Nighttime Using Bright

Pixel Segment with Spatial Temporal Technique”,

International Journal of Electronics and Computer

Science Engineering, Vol. 1, No. 3, 2007.

[14] M.L. Eichner and T.P.Breckon, "Real-time video

analysis for vehicle lights detection using temporal

information", Proc. The Fourth European

Conference on Visual Media Production,

November 2007.

[15] K. Robert, "Video-based traffic monitoring at day

and night vehicle features detection tracking," in

Proc. of the Twelfth IEEE International

Conference on Intelligent Transportation Systems,

pp. 1-6, October 2009.

[16] P. Alcantarilla , L. M. Bergasa , P. Jimenez , M. A.

Sotelo , I. Parra , D. Fernandez and S. S.



455

ISSN:2229-6093

Mayoral "Night time vehicle detection for driving

assistance light beam controller", Proc. IEEE

Intelligent. Vehicles Symposium, pp.291 -296, June

2008.

[17] R. O'Malley , E. Jones and M. Glavin "Rear-lamp

vehicle detection and tracking in low exposure

color video for night conditions", IEEE

Transactions on Intelligent Transportation

Systems, vol. 11, no. 2, pp.453 -462, June 2010.

[18] M. Betke and H.Nguyen,” Highway Scene

Analysis from a moving Vehicle under Reduced

Visibility Conditions”, Proc. of IEEE International

Conference on Intelligent Vehicle, October 1998.

[19] H.K.Kim, K.Sagong, J.H. Park, F.Moutarde and

H.Y.Jung, “A Color Based Real-time Vehicle

Detection for Adaptive Headlamps Control”, 2011.

[20] J. Yang, Y. Wang, A. Sowmya and Z. Li, "Vehicle

detection and tracking with low-angle cameras," in

Proceedings of the 17th IEEE International

Conference on Image Processing (ICIP), pp. 685-

688, September 2010.

[21] J. Zhou , D. Gao and D. Zhang "Moving vehicle

detection for automatic traffic monitoring", IEEE

Transactions on Vehicular Technology, vol.

56, no. 1, pp.51 -59, January 2007.

[22] T.H.Thi, K.Robert, S.Lu and J.Zhang,” Vehicle

classification at nighttime using Eigenspaces and

Support Vector Machine”, National ICT of

Australia, 2007.

[23] H.-K Kim, S. Kuk, Mk.Kim and H-Y Jung, “An

Effective Method of Head lamp and Tail Lamp

Recognition for Night Time Vehicle Detection”,

World Academy of Science Engineering and

Technology, Vol. 44, August 2010.

[24] T.H.Thi, K.Robert, S.Lu and J.Zhang, “Vehicle

Classification at Nighttime using Eigenspaces and

Support Vector Machine”, National ICT of

Australia.

[25] J. Firl, M. Hoerter, M. Lauer, and C. Stiller,

“Vehicle detection, classification and position

estimation based on monocular video data during

night-time,” Proc. of the 8th Int. Symposium on

Automotive Lighting., 2009.

[26] S.Zhou, J.Li, Z.Shen and L.Ying, “A Night time

Application for a Real-Time Vehicle Detection

Algorithm Based on Computer Vision”, Research

Journal of Applied Sciences, Engineering and

Technology, March 2013.

[27] J.Wng, X.Sun and J.Guo, “A Region Tracking

Based Vehicle Detection Algorithm in Nighttime

Traffic Scenes”, Proc. of IEEE Sensors, pp. 16474-

16493, November 2013.

[28] R. Cucchiara and M. Piccardi, "Vehicle Detection

under Day and Night Illumination", in Proc. of

IIA-99 – 3rd

International ICSC Symposium on

Intelligent Industrial Automation, Special Session

on Vision Based Intelligent Systems for

Surveillance and Traffic Control, pp. 789-794,

June 1999.

[29] S.Sivaraman and M.M.Trivedi, ”Looking at

Vehicles on the Road: A Survey of Vision-Based

Vehicle Detection, Tracking and Behavior

Analysis”, IEEE Transactions on Intelligent

Transportation Systems, vol. 14, issue 4, pp. 1773-

1795, December 2013.



456

ISSN:2229-6093

Night Time Vehicle Detection for Real Time Traffic Monitoring ...

Documents

Transcript of Night Time Vehicle Detection for Real Time Traffic Monitoring ...