Autonomous Land Vehicle In a Neural Network
Seminar report submitted in partial fulfillment of the requirements for the award
of degree
of
BACHELOR OF TECHNOLOGY
IN
COMPUTER SCIENCE ENGINEERING
(2006-2010)
Submitted By
M.GIRIDHAR.GOWTHAM 06B11A0550
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
AL-AMEER COLLEGE OF ENGINEERING & INFORMATION TECHNOLOGY
(Affiliated to Jawaharlal Nehru Technological University, Kakinada, A.P)
Gudilova, Anandapuram, Visakhapatnam-531173
Abstract :
ALVINN (Autonomous Land Vehicle in a Neural Network) is a back propagation based neural
architecture Which has been used to successfully drive a car along a highway at highway speeds .
The input to ALVINN is a 30x32 images taken by a camera connected to a digitizer. The output is a
steering direction, which is accomplished By a motor connected to the steering column. One of the
main concerns with ALVINN has been the Speed at which it learns to drive in new situations.
While it is capable of on-line learning, the training time is Usually on the order of 2-3 minutes. The
speed is influenced by many factors: the training algorithm, the training set size, the dynamic
training set management, and the computational power available. In the 5 years since the original
ALVINN work was performed, not only has computational power increased, other learning
methods have also been developed which may allow a speedup in learning.
The motivation for looking at this is to see whether it is viable to
return to actively using ALVINN as a supplement to the RALPH (Rapid Adaptive Lateral Position
Handler) lane tracking system on Navlab . One of the main reasons ALVINN was replaced was the
long learning time when dealing with new road conditions,coupled with the requirement of driver
intervention during that period. It may be possible to remove both constraints by using faster
learning algorithms, and by using RALPH to train ALVINN when road conditions change. After
RALPH trained ALVINN, the neural net’s generalization capabilities would ensure that we don’t
have simply two copies of RALPH running.
To this end, we measure the performance of three learning algorithms on learning ALVINN.
The algorithms are quickprop, cascade correlation, and cascade2. The performance of these three
algorithms is compared against a backprop baseline. In addition, we also use hidden unit analysis to
determine how the network learns.
INDEX
Contents: Page No
1. Overview 1
2. On the Road with Alvinn 3
3. Technical Specification 6
Overview:
This sensor generates heading information required to steer a robotic vehicle by
"watching" the road. The processing performed on chip is ALVINN (Autonomous Land Vehicle In
a Neural Network), a neural network trained to drive without human intervention on public
highways. Circuitry for neural computations is integrated with a photo sensor array using VLSI in
order to directly sense road-image information.
Image-based control of a vehicle at high speeds is a demanding real-time task. While an image
sensor generates vast amounts of data, only a small fraction of the information is relevant. Human
drivers use their experience to extract needed information from what they see. The ALVINN neural
network provides a similar capability, extracting information required to stay on the road from
converted intensity images. Through a training process, the network learns to filter out image
details not relevant to driving. However, current implementations of ALVINN rely on conventional
sense-then-process vision methods that must needlessly digitize, transfer and process full video
frames.
VLSI technology provides the opportunity to integrate the imaging and computation required by the
ALVINN task. The resulting computational sensor intelligently extracts relevant information from
raw image input at the point of sensing. The bottleneck between image input and computer, present
in traditional system implementations, is eliminated. Local processing of image information reduces
system latency while increasing data throughput --- meeting the fundamental requirements of real-
time robotic-vision tasks. In addition, computational sensors are compact, rugged and cost-effective
because they are implemented on a monolithic silicon substrate.
1
Prior to ALVINN-on-a-chip, significant bandwidth and computation were wasted transferring and
processing image data from video cameras. As a result, system throughput was limited to only 10
frames / second. Much higher frame rates are required to obtain further gains in the speed and
performance of the driving task. Latency is another serious problem alleviated by a VLSI
implementation. Applications, like ALVINN, are sensitive to the real-time nature of the images, and
excessive latency limits system stability. When video cameras and frame stores are used, the image
data available to update vehicle heading is that taken by the camera several frames back. While
pipelining can improve system throughput, the latency in an imaging system built around a frame
store cannot be eliminated.
VLSI integration of the ALVINN system provides a practical, yet challenging, application which
combines and builds on our expertise in computational sensors, real-time connectionist image
processing and autonomous mobile systems. An intelligent, rapidly programmable sensor for
neural-network based imaging that is fast, cost-effective, and compact will be the result. Our
strategy is to simultaneously advance the technology of neural-network based imaging as we further
investigate the potential of VLSI-based computational sensors.
2
ON THE ROAD WITH ALVINN
The divided highway lances through the beauties of a Pennsylvania dawn, morning frost glinting
across the hills below. It's hard to enjoy the natural wonders, though, when ALVINN is behind the
wheel, doing 88 km/h in the fast lane. First he lurches right, crossing both lanes of the blacktop and
alarming bleary-eyed commuters trying to share the road. Then he careens to the left, skidding onto
the gravel shoulder and nearly into a ditch. Finally Todd Jochem, 27, a graduate student at
Pittsburgh's Carnegie Mellon University, wrests the wheel of the four-wheel-drive Humvee from
ALVINN while Dean Pomerleau, a C.M.U. robotics research scientist, makes excuses for their
friend's driving. ``I guess he's a little confused,'' says Pomerleau. ``We'll let him try again in a
minute.'' Still, ALVINN is a marvel--a road-hogging computer able to drive like a human. Well,
almost.
3
ALVINN, short for Autonomous Land Vehicle in the Neural Network, originated in 1985 as a
research project by C.M.U.'s Robotics Institute for the U.S. military. Early progress aroused the
U.S. Department of Transportation and U.S. automakers to the fascinating possibilities of fully
automated passenger vehicles. The lean-back comfort of a train meets the door-to-door convenience
of a car. Automobile travelers would be able to key in their destination, flip open a magazine and
leave the actual driving to ALVINN. Last October the DOT committed $160 million to an
Automated Highway System research consortium, which includes the C.M.U. team. A European
coalition is working on similar technology called Prometheus; Japanese automakers are also
tinkering with prototypes.
At the moment, ALVINN can go only where he has gone before. The technology-- cameras, laser
range finders, sonar and motors--works in three stages. First ALVINN ``sees'' the road by taping it
as well as sending out laser and sonar waves as Jochem drives. Then the computer goes into
training, replaying the tape thousands of times, studying the details--road signs, lane markers, speed
limits. Finally ALVINN drives a stretch of road, handling his own acceleration, braking and
steering.
Right now ALVINN is barely a student driver. Small but unexpected events cause him to swerve
and shake in gigabyte panic. His performance may improve this month when ``smart car'' map
scanning is integrated into the programming. ``There should be a substantial difference,'' promises
Pomerleau. The researchers say they are only a couple of years from achieving a preliminary goal,
4
perfecting ALVINN as an anti-collision device to jog sleepy drivers before they run off the road.
When a car drifts dangerously close to the border of its lane, a road-watching camera would do
something like ring an alarm bell or shake the driver's seat. ``Fully automated driving may be a
ways off,'' says Pomerleau, ``but in the near term we're doing work to save lives.'' For now,
however, if you see ALVINN coming at you on the road, you might want to keep a respectful
distance.
Training the vehicle on a fly over
5
Technical Specifications:
ALVINN (Autonomous Land Vehicle In a Neural Network)is a perception system which learns to
control the NAVLAB (National Autonomous Vehicles Lab) vehicles by watching a person drive.
ALVINN's architecture consists of a single hidden layer back-propagation network. The input layer
of the network is a 30x32 unit two dimensional ``retina'' which receives input from the vehicles
video camera. Each input unit is fully connected to a layer of five hidden units which are in turn
fully connected to a layer of 30 output units. The output layer is a linear representation of the
direction the vehicle should travel in order to keep the vehicle on the road.
6
To drive the vehicle, a video image from the onboard camera is injected into the input layer.
Activation is passed forward through the network and a steering command is read off the output
layer. The most active output unit determines the direction in which to steer.
To teach the network to steer, ALVINN is shown video images from the onboard camera as a
person drives, and told it should output the steering direction in which the person is currently
steering. The back-propagation algorithm alters the strengths of connections between the units so
that the network produces the appropriate steering response when presented with a video image of
the road ahead of the vehicle. After about 3 minutes of watching a person drive, ALVINN is able to
take over and continue driving on its own.
Because it is able to learn what image features are important for particular driving situations,
ALVINN has been successfully trained to drive in a wider variety of situations than other
autonomous navigation systems which require fixed, predefined features (like the road's center line)
for accurate driving. The situations ALVINN networks have been trained to handle include single
lane dirt roads, single lane paved bike paths, two lane suburban neighborhood streets, and lined
divided highways. In this last domain, ALVINN has successfully driven autonomously at speeds of
up to 70 mph, and for distances of over 90 miles on a public highway north of Pittsburgh.
Specialized networks are trained for each new road type. The networks are trained not only to
output the correct direction to steer, but an estimate of its reliability. ALVINN uses these reliability
estimates to select the most appropriate network for the current road type, and to switch networks as
the road type changes.
7
The current challenge for vision based on-road navigation researchers is to create systems that
maintain the performance of the existing lane keeping systems, while adding the ability to execute
tactical level driving tasks like lane transition and intersection detection and navigation.
There are many ways to add tactical functionality to a driving system. Solutions range from
developing task specific software modules to grafting additional functionality onto a basic lane
keeping system. Solutions like these are problematic because they either make reuse of acquired
knowledge difficult or impossible, or preclude the use of alternative lane keeping systems.
A more desirable solution is to develop a robust, lane keeper independent control scheme that
provides the functionality to execute tactical actions. Based on this hypothesis, techniques that are
used to execute tactical level driving tasks should:
Be based on a single framework that is applicable to a variety of tactical level actions,
Be extensible to other vision based lane keeping systems, and
Require little or no modification of the lane keeping system with which it is being used.
A framework, called Virtual Active Vision, which provides this functionality through intelligent
control of the visual information presented to the lane keeping system, has been developed. Novel
solutions based on this framework for two classes of tactical driving tasks, lane transition and
intersection detection and traversal, are presented in detail. Specifically, algorithms which allow the
ALVINN lane keeping system to robustly execute lane transition maneuvers like lane changing,
entrance and exit ramp detection and traversal, and obstacle avoidance have been tested.
8
Additionally, with the aid of active camera control, the ALVINN system enhanced with Virtual
Active Vision tools can successfully detect and navigate basic road intersections.
Stimulating the steering error
9
Top Related