JavaOne 2016 - Faces Counter
-
Upload
coritel -
Category
Technology
-
view
116 -
download
0
Transcript of JavaOne 2016 - Faces Counter
![Page 1: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/1.jpg)
All Your Faces Belong To UsBuilding an Open Face Recognition Platform
![Page 2: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/2.jpg)
Copyright © 2016 Accenture All rights reserved. 2
About Us
Senior Technology ArchitectAccenture SpainGlobal Java Community LeadTwitter: @_deorsGitHub: deors
Senior Technology AnalystAccenture SpainGlobal Java SMETwitter: @locoporf1GitHub: locoporf1
![Page 3: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/3.jpg)
Copyright © 2016 Accenture All rights reserved. 3
Related Sessions in Java One 2016Monday, Sep 19 Tuesday, Sep 20 Wednesday, Sep 21Introduction to Java ME 8 [CON3189]
Java ME and Single-Board Computers for Creating Industrial Middleware [CON3187]
All Your Faces Belong to Us: Building an Open Face Recognition Platform [CON6217]
12:30 p.m. - 1:30 p.m 2:30 p.m. - 3:30 p.m 3:00 p.m. - 4:00 p.m
Hilton - Golden Gate 6/7/8 Hilton - Golden Gate 6/7/8 Hilton - Golden Gate 6/7/8
Kevin HookeJulio Palma
Jorge HidalgoJulio Palma
Jorge HidalgoMariano Rodriguez
![Page 4: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/4.jpg)
Copyright © 2016 Accenture All rights reserved. 4
• Internet of Things• Applications
• OpenCV library• Cascade Classifier for Object Detection
• Lbp vs Haar• Java API• Object Recognition Sequence Diagram
• MQTT protocol• Quality of Service
• Faces counter• Solution schema• Development environment
• Demo live!
Building an Open Face Recognition Platform
![Page 5: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/5.jpg)
Copyright © 2016 Accenture All rights reserved. 5
Internet Of Things
• Objects: Pressure sensor in tires Monitor implant in hearts GPS in cabs Thermometers in offices …
• With unique identifier• Connected to internet• Transfer data without requiring
human interaction
![Page 6: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/6.jpg)
Copyright © 2016 Accenture All rights reserved. 6
IoT Applications
• Smart Cities: smart street lights, traffic control, surveilance cameras, …
• Healthcare: remote monitoring, ambulance telemetry, drug tracking, …
• Smart Manufacturing: flow optimization, real time inventory, employee safety, …
• Wearable: entertaiment, fitness, smart watch, location & tracking, …
• Automotive: wire replacement, telemetry, predictive maintenance, …
• Building & Home automation: light & temperature control, access control, energy optimization, …
Smart homesAppliances
Wearables
Industrial internet
Healthcare Smart places
Service Industries
Smart citiesP
![Page 7: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/7.jpg)
Copyright © 2016 Accenture All rights reserved. 7
OpenCV Library
• Coded in C/C++• Free for academic or commercial use• Supports Windows, Linux, Mac OS, iOS and Android• Focused on real-time applications: optimized using hardware
accelaration and multi-core processing• C++, Phyton and Java interfaces• Current version: 2.4.13 (May, 2016)• Repository: https://github.com/opencv/opencv
![Page 8: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/8.jpg)
Copyright © 2016 Accenture All rights reserved. 8
Cascade Classifier
1. A XML file applied to every frame of the video stream to detect a particular object
2. OpenCV distribution contains some examples to detect eyes, cats, human faces, …
3. Can be generated to detect any object by training
![Page 9: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/9.jpg)
Copyright © 2016 Accenture All rights reserved. 9
Cascade Classifier Types
LBP Faster Calculations in integers Best option for
embedded/mobile
HAAR More accurate Calculations in floats
Falses Positives
Lost Frames / sec
LBP 0 2 5.72
HAAR 2 0 4.24
![Page 10: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/10.jpg)
Copyright © 2016 Accenture All rights reserved. 10
• org.opencv.core• Mat
• org.opencv.videoio• VideoCapture
• org.opencv.objdetect• CascadeClassifier
• org.opencv.imgproc• Imgproc
• org.opencv.imgcodecs• Imgcodecs
OpenCV Java API
![Page 11: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/11.jpg)
Copyright © 2016 Accenture All rights reserved. 11
Object Detection Sequence Diagram
read
VideoCapture
Mat
Imgproc
cvtColor
equalizeHist
CascadeClassfier
detectMultiScale
Imgcodecs
imwrite
new
new
rectangle
![Page 12: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/12.jpg)
Copyright © 2016 Accenture All rights reserved. 12
Low network bandwidth Low power usage Perfect for IoT Publish / Subscribe mechanism Can use WebSockets Downloads:
Client: http://www.eclipse.org/paho/ Broker:
http://projects.eclipse.org/projects/technology.mosquitto
MQTT Protocol
MQTT Broker
Topic
Device 1
publish
Device 2
Device N
subscribe
subscribe…
![Page 13: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/13.jpg)
Copyright © 2016 Accenture All rights reserved. 13
MQTT: Quality of Service
Publisher Broker Subscriber
publish
0: At most once Guarantees a best effort delivery It won’t be acknowledged by the
receiver or stored and redelivered by the sender
Fire and forgetpublish
![Page 14: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/14.jpg)
Copyright © 2016 Accenture All rights reserved. 14
MQTT: Quality of Service
Publisher Broker Subscriber
publish
1: At least once Guarantees that a message will be
delivered at least once to the receiver
But the message can also be delivered more than once
Application must be tolerating duplicates and process them accordingly
publish
puback
store message
delete message
![Page 15: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/15.jpg)
Copyright © 2016 Accenture All rights reserved. 15
MQTT: Quality of Service
Publisher Broker Subscriber
publish
1: Exactly once Guarantees that each message is
received only once by the counterpart
Safest and also the slowest quality of service level
publishpubrec
pubrel
pubcomp
store message
store message
![Page 16: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/16.jpg)
Copyright © 2016 Accenture All rights reserved. 16
Face Detector Device
• Raspberry PI (model 3)• Raspicam (V2) ~ 60 $}
![Page 17: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/17.jpg)
Copyright © 2016 Accenture All rights reserved. 17
Faces Dectector Solution Schema (I)
…
Face detector devices
publish
MQTT Broker Web Server
subscribe
MQTT MQTT
WS
![Page 18: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/18.jpg)
Copyright © 2016 Accenture All rights reserved. 18
Face Counter Solution Schema (II)
Java RuntimeFacesDetector App
OpenCV MQTT Paho
JNI
OpenCV
Raspicam drivers
publish subscribe
MQTT broker Java Runtime
Spring Boot
Monitor WebApp
Vaadin
![Page 19: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/19.jpg)
Copyright © 2016 Accenture All rights reserved. 19
Setting up a Raspberry PI: Install dependecies
sudo apt-get update
sudo apt-get install cmake ant build-essential pkg-config libpng12-0 libpng12-dev libpng++-dev libpng3 libpnglite-dev zlib1g-dbg zlib1g zlib1g-dev pngtools libtiff4 libtiffxx0c2 libtiff-tools libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-progs libavcodec-dev libavformat-dev libgstreamer0.10-0-dbg libgstreamer0.10-0libgstreamer0.10-dev libunicap2 libunicap2-dev libdc1394-22-dev libdc1394-22libdc1394-utils swig libv4l-0 libv4l-dev
![Page 20: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/20.jpg)
Copyright © 2016 Accenture All rights reserved. 20
Setting up a Raspberry PI: Enviroment variables
Edit ~/.bashrc file to add the following lines:export JAVA_HOME=/usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/export ANT_HOME=/usr/share/ant/
![Page 21: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/21.jpg)
Copyright © 2016 Accenture All rights reserved. 21
Setting up a Raspberry PI: Download OpenCV source code
wget https://codeload.github.com/Itseez/opencv/zip/3.1.0mv 3.1.0 opencv-3.1.0.zipunzip opencv-3.1.0.zip
![Page 22: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/22.jpg)
Copyright © 2016 Accenture All rights reserved. 22
Setting up a Raspberry PI: Create Makefile
cd opencv-3.1.0mkdir buildcd buildcmake -DCMAKE_BUILD_TYPE=RELEASE -DWITH_OPENCL=OFF -D BUILD_PERF_TESTS=OFF -DBUILD_SHARED_LIBS=OFF -DJAVA_INCLUDE_PATH=$JAVA_HOME/include -DJAVA_AWT_LIBRARY=$JAVA_HOME/jre/lib/amd64/libawt.so -DJAVA_JVM_LIBRARY=$JAVA_HOME/jre/lib/arm/server/libjvm.so -DCMAKE_INSTALL_PREFIX=/usr/local ..
![Page 23: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/23.jpg)
Copyright © 2016 Accenture All rights reserved. 23
Setting up a Raspberry PI: Compile, install and check• make install• ATTENTION!!! This process can take several hours!• Check if exist: build/libs/libopencv_java310.so and build/bin/opencv-
310.jar have been created!!!
![Page 24: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/24.jpg)
Copyright © 2016 Accenture All rights reserved. 24
Setting up a Raspberry PI: Install raspicam drivers• curl http://www.linux-projects.org/listing/uv4l_repo/lrkey.asc |
sudo agt-key add –• Edit file /etc/apt/source.list to add the following line: deb http://www.linux-projects.org/listing/uv4l_repo/raspbian/ jessie main• sudo apt-get update• sudo apt-get install uv4l uv4l-raspicam• sudo apt-get install uv4l-raspicam-extras• sudo raspi-config (Enable raspicam interface)• sudo rpi-update (Update firmware)
![Page 25: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/25.jpg)
Copyright © 2016 Accenture All rights reserved. 25
Setting up a Raspberry PI: Transfer files & run
• Using ftp client tool copy:• Faces detector properties and JAR files• MQTT Paho JAR file
• Edit properties file to customize• Run java -Djava.library.path=build/libs/libopencv_java310.so -jar
faces-counter.jar faces-counter.properties
![Page 26: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/26.jpg)
Copyright © 2016 Accenture All rights reserved. 26
Development Enviroment
Remote Mode (option I):• Oracle JDK • NetBeans• Putty• FileZilla (client)
Remote Mode (option II):• Oracle Java ME SDK• NetBeans
“Embedded” Mode• BlueJ or Geany• Oracle JDK
![Page 27: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/27.jpg)
Copyright © 2016 Accenture All rights reserved. 27
• Faces Detector: https://github.com/locoporf1/faces-detector
• WebApp Monitor: https://github.com/locoporf1/faces-detector-server
Face Detector App Source Code
![Page 28: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/28.jpg)
28
Question & Answers
![Page 29: JavaOne 2016 - Faces Counter](https://reader031.fdocuments.net/reader031/viewer/2022012923/5886537b1a28ab26598b492d/html5/thumbnails/29.jpg)
29
Live DEMO !!!