VEHICLE DETECTION AND COUNTING FROM UNMANNED AERIAL VEHICLE (UAV)

ANG KUAN KEE

UNIVERSITI TEKNIKAL MALAYSIA MELAKA

BORANG PENGESAHAN STATUS TESIS

JUDUL: VEHICLE DETECTION AND COUNTING FROM UNMANNED

AERIAL VEHICLE

SESI PENGAJIAN: 2014/2015

Saya ANG KUAN KEE

(HURUF BESAR)

mengaku membenarkan tesis (PSM/Sarjana/Doktor Falsafah) ini disimpan di Perpustakaan Fakulti Teknologi Maklumat dan Komunikasi dengan syarat-syarat kegunaan seperti berikut: 1. Tesis dan projek adalah hakmilik Universiti Teknikal Malaysia Melaka. 2. Perpustakaan Fakulti Teknologi Maklumat dan Komunikasi dibenarkan membuat salinan untuk tujuan pengajian sahaja. 3. Perpustakaan Fakulti Teknologi Maklumat dan Komunikasi dibenarkan membuat salinan tesis ini sebagai bahan pertukaran antara institusi pengajian tinggi. 4. ** Sila tandakan (/) ______ SULIT (Mengandungi maklumat yang berdarjah

keselamatan atau kepentingan Malaysia seperti yang termaktub di dalam AKTA RAHSIA RASMI 1972)

_______ TERHAD (Mengandungi maklumat TERHAD yang

telah ditentukan oleh organisasi/badan di

mana penyelidikan dijalankan)

______ TIDAK TERHAD

__________________________ ____________________________

(TANDATANGAN PENULIS) (TANDATANGAN PENYELIA)

Alamat tetap: 128, Jalan Batu Unjur6, DR. ASMALA BIN AHMAD Taman Bayu Perdana, 41200 Klang Nama Penyelia Selangor Tarikh: ______________ Tarikh: ____________

CATATAN: * Tesis dimaksudkan sebagai Laporan Akhir Projek Sarjana Muda (PSM) ** Jika tesis ini SULIT atau TERHAD, sila lampirkan surat daripada pihakberkuasa

VEHICLE DETECTION AND COUNTING FROM UNMANNED AERIAL VEHICLE (UAV)

ANG KUAN KEE

This report is submitted in partial fulfilment of the requirements for the Bachelor of Computer Science (Artificial Intelligence)

FACULTY OF INFORMATION AND COMMUNICATION TECHNOLOGY UNIVERSITI TEKNIKAL MALAYSIA MELAKA

2015

ii

DECLARATION

I hereby declare that this project report entitled

VEHICLE DETECTION AND COUNTING FROM UNMANNED AERIAL

VEHICLE (UAV)

is written by me and is my own effort and that no part has been plagiarized without

citations.

STUDENT :___________________________________ Date:_______

(ANG KUAN KEE)

SUPERVISOR :___________________________________ Date:_______

(DR. ASMALA BIN AHMAD)

iii

DEDICATION

I would like to dedicate my final year project report to my beloved family, friends

and my lecturers. For my supervisor, Dr. Asmala Ahmad and my evaluator, Dr

Sharifah Sakinah, always guiding, comment and helping me to complete up this

project. Besides that, family is the one always give me spirit support and life-long

love to me. They have been very supportive and encouraging in completion of my

thesis and throughout the years of my studies. Furthermore I also would like to

dedicate to the friends who have been supportive and helpings throughout the project

development.

iv

ACKNOWLEDGEMENT

I would like to take an opportunity to express my gratitude to those who contributed

and showed their kindness in helping and assisting me to complete this project. First

and foremost, I would like to thank my supervisor, Dr. Asmala Ahmad and my

evaluator, Dr. Sharifah Sakinah for all the guidance and encouragement. I am indebt

for the patient, support and motivation from the beginning to the end of my project.

Thank you also for instructing and teaching me in developing the understanding of

the project and eventually me enable to grasp the essence of this project. Besides that,

I also like to thank a PhD student, Mr. Nasruddin Abu Sari whom provided me with

the UAV data for this project. Most important, I am thankful to my family whom

continuously keeping up my spirit and supporting me in term of financial and moral.

They keep on encouraging me whenever I faced any difficulties throughout my study

period. Lastly, I would like to thank all my classmates and friends for their

cooperation and camaraderie and also those who had supported me in any aspects

throughout the project. Once again sincerely thank you all for supporting me to

complete this project successfully.

v

ABSTRACT

Efficiency traffic management is important in order to manage the traffic

efficiently especially during unexpected situations. To do so, continuous and reliable

information on traffic situation is important to users so that they can plan their route

by avoiding traffic congestion. UAV images are an important means by which

detection and classification of vehicle can be carried out to assist the traffic

management. This is done by capturing the latest information of the road situation

and reporting it to road users. In this study, I develop a procedure to detect and count

the number of vehicles from three different height categories of UAV images.

Initially, two unsupervised classification methods, ISODATA and K-Mean, are

considered to separate vehicle and non-vehicle pixels from the UAV images. The

study were conducted in two phase; phase one is classifying the objects within the

image and phase two, determining the total number of vehicles within the image.

Firstly, the unsupervised classifications were implemented and analysed to cluster

similar pixels within the selected region. Secondly, region resizing is carried out to

reduce the portion which is non-relevant from an image. In order to get the total

amount of vehicles in an image, a standard amount of pixels for a single vehicle is

computed. The accuracy assessment in this study is measured by means of accuracy

and error percentage. This is done by comparing between the actual numbers of

vehicle with the number of vehicles computed from UAV images. For both

classifiers, low category images are found to have the highest accuracies. Further

study needs to focus on an automated technique of resizing the region to reduce the

non-relevant portion from an image in order to get the exact regions of vehicles.

vi

ABSTRAK

Pengurusan trafik yang cekap adalah penting bagi menguruskan lalu lintas dengan

cekap terutama semasa dalam keadaan yang tidak dijangka. Untuk berbuat demikian,

maklumat keadaan trafik yang berterusan dan boleh dipercayai adalah penting

kepada pengguna supaya mereka dapat merancang laluan mereka dengan mengelak

kesesakan lalu lintas. Imej UAV adalah penting yang mana pengesanan dan

pengkelasan kenderaan boleh dilakukan untuk membantu pihak pengurusan lalu

lintas. Ini dilakukan dengan menangkap maklumat terkini keadaan jalan dan

melaporkan kepada pengguna jalan raya. Dalam kajian ini, saya membangunkan

prosedur untuk mengesan dan mengira bilangan kenderaan dari tiga kategori

ketinggian yang berbeza imej UAV. Pada mulanya, dua kaedah klasifikasi tanpa

pengawasan, ISODATA dan K-Min, digunakan untuk memisahkan piksel kenderaan

dan bukan kenderaan dari imej UAV. Kajian ini telah dijalankan dalam dua fasa; fasa

pertama mengklasifikasikan objek dalam imej dan fasa kedua menentukan jumlah

bilangan kenderaan dalam gambar. Dalam fasa pertama, klasifikasi tanpa

pengawasan telah dilaksanakan dan dianalisis untuk kelompok piksel yang sama di

dalam rantau yang dipilih. Dalam fasa kedua, pensaizan rantau dijalankan semula

untuk mengurangkan bahagian yang tidak berkaitan daripada imej. Dalam usaha

untuk mendapatkan jumlah kenderaan dalam imej, jumlah piksel piawan bagi

kenderaan tunggal dihitung. Penilaian ketepatan dalam kajian ini diukur berdasarkan

peratusan ketepatan dan kesilapan. Ini dilakukan dengan membandingkan antara

jumlah sebenar kenderaan dengan bilangan kenderaan yang dihitung dari imej UAV.

Bagi kedua-dua pengkelas, imej kategori rendah didapati mempunyai ketepatan

tinggi. Kajian lanjut perlu memberi tumpuan kepada teknik pensaizan rantau secara

automatik bagi mengurangkan bahagian yang tidak berkaitan daripada imej untuk

mendapatkan kawasan sebenar kenderaan.

vii

TABLE OF CONTENTS

CHAPTER SUBJECT PAGE

DECLARATION ii

DEDICATION iii

ACKNOWLEDGEMENTS iv

ABTRACT v

ABSTRAK vi

TABLE OF CONTENTS vii

LIST OF TABLES xi

LIST OF FIGURES xii

CHAPTER 1 INTRODUCTION

1.1 Introduction 1

1.2 Problem Statement 3

1.3 Objective 4

1.4 Scope 4

1.5 Project Significance 5

1.6 Expected Output 6

1.7 Conclusion 6

viii

CHAPTER 2 LITERATURE REVIEW

2.1 Introduction 7

2.2 Type of Vehicle 8

2.2.1 Statistic of Vehicle 10

2.3 Remote Sensing Interest 11

2.3.1 Review of Remote Sensing 12

2.3.2 Unmanned Aerial Vehicle Images (UAV images) 13

2.3.3 Detection and Classification for Efficient

Traffic Management 15

2.4 Unsupervised Learning 18

2.4.1 K-Means 19

2.4.2 ISODATA 21

2.5 Analysis 23

2.5.1 Problem Analysis 24

2.5.2 Data Requirement 24

2.5.3 Software Requirement 25

2.5.4 Ancillary Data Requirement 25

2.6 Conclusion 27

CHAPTER 3 METHODOLOGY AND DESIGN

3.1 Introduction 28

3.2 Phases 29

3.2.1 Phase One – Analysis and Preliminary Study 30

3.2.2 Phase Two – Design 30

3.2.3 Phase Three – Experimental and Results 31

3.2.4 Phase Four – Evaluation / Testing 31

3.2.5 Phase Five – Conclusion 31

ix

3.3 Project Schedule and Milestone 32

3.4 Design on Vehicle Detection and Classification 33

3.4.1 Input 35

3.4.2 Unsupervised Classification 35

3.4.3 Selecting ROI for Vehicle Detection 36

3.4.4 Non – Vehicle Detection 36

3.4.5 Subset / Resize 37

3.4.6 Selecting Region for Inter Class of 37

Vehicle Detection 37

3.4.7 Calculate the Number of Pixel for

Vehicle Detection 37

3.4.8 Output 38

3.4.9 Accuracy Assessment / Testing,

Compare and Analysis 38

3.5 Conclusion 38

CHAPTER 4 IMPLEMENTATION AND RESULTS

4.1 Introduction 39

4.2 Data Pre – Processing 40

4.3 Input 40

4.4 Unsupervised Classification 41

4.5 Selecting Area for Vehicle Detection 45

4.6 Non – Vehicle Detection 46

4.7 Region Resizing 49

4.8 Selecting Region for Inter Class of

Vehicle Detection 53

x

4.9 Calculate the Number of Pixel for

Vehicle Detection 55

4.10 Output 55

4.11 Accuracy Assessment / Testing,

Compare and Analysis 62

4.12 Conclusion 64

CHAPTER 5 ACCURACY ASSESMENT/ TESTING

5.1 Introduction 65

5.2 Samples for Testing Experiment 66

5.3 Comparison Analysis 67

5.4 Conclusion 69

CHAPTER 6 CONCLUSION

6.1 Introduction 70

6.2 Strengths 70

6.3 Weakness 72

6.4 Suggestion for Improvement 72

6.5 Conclusion 73

REFERENCES 74

xi

LIST OF TABLES

TABLE TITLE PAGE

2.1 UAV models 14

3.1 Schedule and Milestone of Final Year Project 33

3.2 Selection of ROI 36

4.5 Percentage of Number of Pixel for Every Category

and Classifiers 45

4.8.1 Number of Pixel for Single Vehicle of K-Mean 52

4.8.2 Number of Pixel for Single Vehicle of ISODATA 53

4.11 Accuracy Assessment of Output 61

5.3 Accuracy Assessment of 3 samples of data 68

6.1 Accuracy Assessment of Output 71

6.2 Accuracy Assessment of Low Category 71

xii

LIST OF FIGURES

FIGURE TITLE PAGE

2.1 Sales market of vehicle in 2012 to 2013 10

2.2 Sales and production report in December of 2013 and

December of 2014 11

2.3 Electromagnetic Spectrum (Department of Army, Feb 2014) 12

2.4 Flow Chart of simple iteration of K-means algorithm 20

2.5 Process of filter by K-Means 20

2.6 ISODATA algorithm by A.Dermanis, L.Biagi

According Richards et al. (2006) 22

2.7 Sample of Low Category of Traffic Road Map 25

2.8 Sample of Medium Category of Traffic Road Map 26

2.9 Sample of High Category of Traffic Road Map 26

3.1 Methodology design of the project 29

3.2 Flow Chart for Vehicle Detection and Classification 34

4.3.1 Low category 40

4.3.2 Medium category 40

4.3.3 High category 40

4.4.1 Classifier of K-Mean in Low category 41

4.4.2 Classifier of K-Mean in Medium category 42

4.4.3 Classifier of K-Mean in High category 42

4.4.4 Classifier of ISODATA in Low category 43

xiii

4.4.5 Classifier of ISODATA in Medium category 43

4.4.6 Classifier of ISODATA in High category 44

4.6.1 Non-vehicle for Low category of K-Mean 46

4.6.2 Non-vehicle for Medium category of K-Mean 47

4.6.3 Non-vehicle for High category of K-Mean 47

4.6.4 Non-vehicle for Low category of ISODATA 47

4.6.5 Non-vehicle for Medium category of ISODATA 48

4.6.6 Non-vehicle for High category of ISODATA 48

4.7.1 Low category of K-Mean Resize 49

4.7.2 Medium category of K-Mean Resize 49

4.7.3 High category of K-Mean Resize 50

4.7.4 Low category of ISODATA Resize 50

4.7.5 Medium category of ISODATA Resize 51

4.7.6 High category of ISODATA Resize 51

4.10.1 Output of low category for ISODATA 55

4.10.2 Output of medium category for ISODATA 56

4.10.3 Output of high category for ISODATA 57

4.10.4 Output of low category for K-Mean 58

4.10.5 Output of medium category for K-Mean 59

4.10.6 Output of high category for K-Mean 60

5.2.1 Sample 1 65

5.2.2 Sample 2 65

5.2.3 Sample 3 65

5.3.1 Output of Sample 1 66

5.3.2 Output of Sample 2 66

5.3.3 Output of Sample 3 66

CHAPTER I

INTRODUCTION

1.1. Introduction

Transportation has long become an essential means for people to travel from

one place to another. Among all modes of transportation, land transportations have

been known to have the most users due to their practicality and affordable cost. With

the increase in the world population, the number of land transportations particularly

car, bus and truck are also increasing drastically in all parts of the world. Although

giving good impact to economy, failing to manage the traffic of these vehicles

properly may cause a negative consequence, especially for developing countries like

Malaysia.

Nowadays, people start to have concern on traffic situation at their living and

working areas. Traffic situation already is as a part of daily life of people. This is

because no matter where they want either to go to school study, working, playing or

anywhere, they still need vehicle to send them going to the right destination. Heavy

2

traffic situation, undeniably, will bring the negative effects to them such as those in

terms of emotional, punctually attending, driving attitude and others.

Besides that, vehicle theft cases are rapidly increasing. Therefore, we need a

system which can help us to detect stolen vehicles so finding back stolen can be

carried out more effective. Therefore we need a more up-to-date approach to detect

vehicles on the earth surface so that searching tasks can be done immediately and

effectively.

This project aims to detect and count vehicles from unmanned aerial vehicle

(UAV) imagery. The result obtainedcan greatly help authorised agencies in traffic

management. By knowing the amount and location of vehicles, the relevant

authorised agencies can can update the latest and accurate information to motorists

so that they can plan their journey in a more efficient way. They will have more

alternative way to prevent situations such as traffic jam, accident and other. Besides

that, the traffic information enable motorists to plan their road trip and arrive the

destination on the right time and safety to prevent being trapped in heavy traffic.

In carrying this project, image processing techniques are used to detect and

count the number of vehicles, which include classification, enhancement and

thresholding.

UAV image is the source to be processing in this project. UAV also named as

unmanned aerial vehicle. It is a aircraft without human pilot. It is also act a remote

sensing to get more information or detail toward the object by detecting and classify

the objects on Earth. Therefore, it will through this aircraft to collect the source or

situation on the road. Therefore, i will use the source to done the processing of

detection and classification to identify the result.

3

Shi et al. (2012) reported that, UAVs are better than traditional manned

aircraft or other remote sensing operator. The reason they stated that is UAVs are no

require any high cost platform and it can reduce the error or effect come by human

instead of automate to remote it and sensing control itself. Besides that, UAVs also

enable produce high resolution images and given accurate data result by the

navigation sensor. Therefore, it can give an effective supplement work. So that, it

have the strength to detect alter between object of image over the time.

1.2. Problem statement

Traffic information is nowadays being more concerned by a motorists using the

road to travel to different places. However, the amount of vehicle is increasing day

by day. Road traffic needs to be efficiently managed to prevent road accident,

serious traffic jam and others. Such situations can cause negative effects not only to

motorists but also others. The convensional approaches of managing traffic is by

police patrol and CCTV. Nevertheless, such approaches are logistically expensive

and not practical for large areas because the need to place a large number of police

patrol and CCTV in order to manage traffic. Therefore an improved way for traffic

management is critically needed especially to provide motorists with the latest traffic

condition and to help detecting stolen vehicles.

4

1.3. Objective

This project embarks on the following objectives:

1. To identify methods for car detection and counting from UAV.

2. To propose suitable framework of image processing techniques for car

detection and counting from UAV.

3. To evaluate the performance of the techniques.

1.4. Scope

In the study, I divide the scopes into four parts which are software scope, data scope,

area of study scope and user scope.

i. Software Scope

In this project, it will mainly use two software tools to complete it. There

are ENVI 4.5 toolbox and Matlab R2010a. ENVI toolbox is for purpose

to process the image, classified the object and analysis the imagery.

However for the Matlab toolbox just as an assistance tool to complete the

task after generate the output from ENVI toolbox. Besides that, it also

will useful in analysis part the result.

ii. Data Scope

The data that will be used in this project is collect from Unmanned Aerial

Vehicle. It will collect the image from certain area. It also enables collect

movement image with support more clearly image to be done analysis.

Therefore, it can be done detection and segmentation directly and

effective. However, in this project, there are just focuses on certain type

vehicle such as car, lorry and bus. The reason there are just focuses on

this few type of vehicle is because this type of vehicle only always get the

more spacing on the road and affect the traffic situation. Besides that,

their size is bigger, this will cause the detection done in smooth and result

5

will be more precious. Furthermore, our data scope only had done the

detection on certain intensity value only. Due to UAVs image is RBG,

therefore in this project it will focusing on certain colour of vehicle.

iii. Area of Study Scope

Malacca was chosen as my focus area of study scope. Malacca is a

famous state which also named as historical state. Therefore every year,

season, they also successful attract thousands of visitor come from

everywhere. So that, traffic jam will be always happen especially on

certain peak hour or festival such as working hour or public holiday.

Therefore, the traffic situations on Malacca state are requires more

emphasize than other place.

iv. User Scope

The significance will be bring out from this project is useful for the

researcher. The researcher can through the output as a reference material

for them to further this project and enhance it to be better.

1.5. Project Significance

This project will be benefit toward government traffic management

particularly to determine the amount of vehicle using certain road. Therefore, it can

help toward reporting the traffic situation to the motorists. In this way the motorists

can have alternative travel planning in advance in order reach journey in a faster and

safer way. Besides that, government also can get to know the requiring of motorists

to make the adjustment to improve the situation such as enlarging the road size to

suit the capacity of vehicles using a particular road. In addition, this project can

decrease the number of stolen vehicles by providing information regarding the

location of the stolen vehicles in a shorter time. .

6

1.6. Expected Output

The expected result from this project is a processed image can provide the

information regarding vehicle location and amount. This will greatly assist

authorized parties in getting information regarding traffic situation on particular road

and convey it to traffic management to do some improvement work or update latest

news to the motorists. In addition, it also brings a lot of convenience for the

particular relevant unit to identify or track certain vehicle on the particular road. This

will be very useful especially for the police when they are requiring tracing

somebody or some cases.

1.7. Conclusion

This chapter are just brief on this project idea and direction. I have provided

the rationale for carrying this project and highlighted the problems long faced by

motorists. I believe this project will have significant impact to the societies upon its

completion.

CHAPTER II

LITERATURE REVIEW

2.1 Introduction

This chapter reviews studies and information related to vehicle detection and

counting. Literature review acts as a primary reference towards completion of this

project. Discussion on previous research which is related to the project topic is

discussed in detail. Literature review also is an important to prove that project is

possible to be carried out. In my project, literature review are as a reference for me to

ensure my project are work in the scope without go to digress. Besides that, through

the review, it brings out a background for my project to propose a suitable technique

in my project. Therefore, it assisted me in writing the analysis to get the right view

when developing the project.

Literature reviews from journal and research papers are more emphasize on

several part which are result and discussion. Through the result we can identify the

outcome from their research and for the discussion it help us to determine the

8

significance of project and problem for further future research. Therefore, from the

result and discussion of paper, it can carry out the abstract of that project. Trusted or

pure original sources are important. That is because it will be as a guidance of my

project, therefore in my project I just will reference the most recently paper to avoid

some conflict.

2.2 Type of vehicle

This project concerns on detection of few certain types of vehicle only. In

general, vehicles can be categorised as below.

Heavy vehicle are differentiate into many class, there are:

For a normal rigid truck size dimension is: