2010 Simulated Car Racing Championship @ GECCO-2010

The 2010 Simulated Car Racing Championship @ GECCO-2010Daniele Loiacono, Luigi Cardamone, Martin V. Butz, and Pier Luca Lanzi

2010 Simulated Car Racing Championship @ GECCO-2010

2010 Simulated Car Racing Championship9 races during 3 conferences

ACM GECCO-2010, Portland, OR (USA), July 7-IEEE WCCI-2010, Barcelona (Spain), July 18-23

IEEE CIG-2010, Copenhagen (Denmark), August 18–21

Develop a driver for TORCS(hand-coded, learned, evolved, …)

Drivers will be awarded based on their score in each conference competition

At the end, the team with highest overall scorewins the championship

2


What is the structure of a race?

Three stages: warm up, qualifiers, actual race

During warm-up, each driver can explore the track and learn something useful

During qualifiers, each driver races alone against the clock (the best 8 drivers move to the race)

During the race all the drivers race together

3


Motivations

Proposing a relevant game-based competition

more representative of commercial games AI

more similar to a real-world problem

Proposing a funny and exciting competition

you can see and play with the entries of this competition

human players can interact with AI

a lot of programmed AI available for comparison

Proposing a challenging competition

not designed with Machine Learning in mind

computationally expensive

real-time

dealing with a lot of practical issues


What‘s new?

If everything seems under control, you're not going fast enough— Mario Andretti

Warm-up stage

Before qualifying stage, competitors have 100000 game ticks to race on the track

Allows track learning and optimization of parameters

Noisy sensors

Track sensors and opponent sensors are affected by a Gaussian noise (standard deviation equal to 10% of the readings)

Extended sensor model

Focus sensors

Z position and speed

Direction of track sensors fully customizable

Added clutch control and focus command

The Open Racing Car Simulator


The Open Racing Car Simulator

TORCS is a state of the art open source simulator written in C++

Main features

Sophisticated dynamics

Provided with several cars, tracks, and controllers

Active community of users and developers

Easy to develop your own controller

OS Support

Linux: binaries and building from sources

Windows: binaries and ―limited‖ bulding from sources support

OSX: legacy binaries and no building from sources support


The Open Racing Car Simulator & the Competition Software

TORCS

BOT BOT BOT

TORCS

PATCH

SBOT SBOT SBOT

BOT BOTBOT

UDP UDPUDP The competition server

Separates the bots from TORCS

Build a well-defined sensor model

Works in real-time


Sensors and actuators

Rangefinders for edges on the track and opponents

Speed, RPM, fuel, damage, angle with track, distance race, position on track, etc.

Four effectors: steering wheel [-1,+1], gas pedal [0, +1], Brake pedal [0,+1], Gearbox {-1,0,1,2,3,4,5,6}

Competitors


The competitors

Four entries submitted to this first leg

AUTOPIA, Madrid and Granada

J. Muñoz, Carlos III University of Madrid

S.Pohl, J. Quadflieg and T. Delbrügger, TU Dortmund

Joseph Alton, University of Birmingham

Two more entries from the 2009 championship

COBOSTAR (T. Lönneker & M.V. Butz, University of Würzburg)

POLIMI (Cardamone, Politecnico di Milano)


Industrial Computer Science Department.

Centro de Automática y Robótica

Consejo Superior de Investigaciones Científicas

Madrid, Spain

Contact:E. Onieva ([email protected])

AUTOPIA


Architecture Schema

Three basic modules for gear, steering and speed control

Steady stage genetic algorithm to compute the best weights to combine parameters for steering and target speed control

Opponents module

Acts on steering and brake signal to overtake opponents and avoid collisions

Learning Module in Warm-up Stage

Factors over the target speed in certain track segments


Opponents Management (Steering)

If O(-20,-10,10,20) detects an opponent with lateral displacement less than 20 meters, move the steering trackwidth/100 through the direction with more free distance.

If O(-40,-30,-20,-10) detects an opponent with lateral displacement less than 20 meters, move the steering trackwidth/50 through the right.

If O(-70,-60,-50)<15 meters, move the steering trackwidth/50 through the right.

If O(-140,-130,-120,-110,-100,-90,-80)<15meters , move the steering trackwidth/50 through the right.

Mirroring actions for positive sensors

O(X) Opponent Measure of Sensor Oriented at X

Steering Modification limited to 0.35


Learning Module (Warm-Up)

Running normally in warm-up stage.

Maintain a vector with as many real values as the track length in meters.

Vector initialized to 1.0

If the vehicle goes out of the track or suffers damage then multiply vector positions from 100 meters before the current position by 0.9.


Susanna Pohl, Jan Quadflieg and Tim DelbrüggerTU Dortmund

Mr Racer


Mr. Racer 2009-2010

Mr Racer 2009

Good classifier which identifies six situations

Acceleration/brake learned offline using an EA

Model of the track learned online

Simple heuristic to use the model: override the learned behaviour on straights and in full speed corners to drive flat out

Mr Racer 2010

Save the model after warmup, use it during qualifying and the race

Use the model to derive a plan consisting of target speeds and a racing line

Optimize the parameter set of the planning module, left to be done for the next round


Mr. Racer - Classification

Angle based measure mapped to six situations (straight, fast corner, slow corner, etc)


Mr. Racer 2010 – The catch

Noise completely breaks our classifier

Without a descent classifier we can‗t learn the track

Without a trackmodel we can‗t drive

Workaround for GECCO-2010: Classify the whole track as being straight

New classifier for the next leg at WCCI-2010


Department of Computer ScienceCarlos III University of Madrid

Jorge Muñoz Fuentes


Jorge Muñoz

Build a model of the track during the warm-up stage.

Two neural networks to predict the trajectory using the track model. Two neural networks to predict the target speed given the model of the track and the current car position

The four neural networks are trained with backpropagation using data retrieved from a human player.

The controller tries to imitate a human player.

A scripted policy is used to follow the trajectory (steering value), set the speed (accelerate and brake values), set the clutch and the gear


Jorge Muñoz

Other optimizations performed during the warm-up and used in the race:

The car remember where it goes out of the car or drives far form the trajectory and in the next laps goes slower in those points

The car remember where it follows the trajectory perfectly and tries to go faster in the next laps.

Overtaking is made by means of modifying the predicted trajectory, the modification is bigger in straighs than in turns

To avoid overtakin the car also modifies the trajectoy, trying to stay in front of the opponent car.


Joseph Alton

Joseph Alton


Joseph Alton

Bot made specially for the oval leg, and is only designed to be competent on this leg.

Plan to change and improve the bot as I learn and to meet the demands of the other tracks

Warm-up Learning

Car goes slowly around the track recording in each segment (e.g. 0 metre, 1 metre, 2 metre) whether it is turning or not

This information can then be loaded up in qualifying or finals to help find a more appropriate target speed.

Straight = fast, Turn = slower

During the race, the driver looks ahead to the next or the next two segments depending on the speed


COBOSTAR

Thies Lönneker and Martin V. Butz

University of Würzburg

http://www.coboslab.psychologie.uni-wuerzburg.de


CIG-2008 Champ

Luigi Cardamone

Politecnico di Milano

Qualifying


Scoring process: Warm-up Qualifying

Scoring process involves three motor high-speedway tracks:

Kiwi

Mango

Triangle

Kiwi is a (renamed and slightly modified) track of TORCS

Mango and Triangle has been developed by the organizers

Each controller raced for 100000 game ticks in the warm-up stage and then its performance is computed in the qualifying stage as the distance covered within 10000 game ticks


Qualifying: Kiwi

0 2000 4000 6000 8000 10000 12000 14000 16000 18000

Autopia

Cardamone

COBOSTAR

Jorge Muñoz

Joseph

MR.Racer


Qualifying: Mango

0 2000 4000 6000 8000 10000 12000 14000 16000

Autopia

Cardamone

COBOSTAR

Jorge Muñoz

Joseph

MR.Racer


Qualifying: Triangle

0 2000 4000 6000 8000 10000 12000 14000 16000

Autopia

Cardamone

COBOSTAR

Jorge Muñoz

Joseph

MR.Racer


Qualifyng summary

Competitor Kiwi Mango Triangle Total

COBOSTAR 10 10 8 28

Autopia 8 8 10 26

Mr. Racer 6 6 6 18

Jorge Muñoz 4 5 5 14

Cardamone 5 4 4 13

Joseph 3 3 3 9

What about qualifying?

COBOSTAR is the fastest driver (despite not optimized neither for high-speedways nor for

noisy sensors)

Experiments without noise shows that

Noise is not critical in high-speedways for driving alone

Some controllers (MR. Racer and Muñoz) perform even better with noise

The GECCO-2010 Leg


The Three GPs

For each track we run 5 races with random starting grids

Each race is scored using the F1 point system (10 to first, 8 to second, 6 to third, …)

Two points to the controller with lesser damage

Two points for the fastest lap of the race

35


Final Results

Competitor Kiwi Mango Triangle Total

Autopia 12 12 10 34

Jorge Muñoz 5.5 8 9 22.5

Mr. Racer 9 4 3 16

Cardamone 6 4.5 5.5 16

Joseph 4 5 6.5 15.5

COBOSTAR 4 4.5 5.5 14

What about the race?

Results do not confirm the qualifying performance

COBOSTAR is the worst performing controller!

The results suggest that noise is much more relevant for the opponent perception that for driving alone

Races without noise show that AUTOPIA and COBOSTAR being the top controllers (as expected from qualifyng)

Cardamone’s controller seems slighlty more reliable (as it is completely based on a Neural Network)

2010 Simulated Car Racing Championship @ GECCO-2010

Documents

Transcript of 2010 Simulated Car Racing Championship @ GECCO-2010