Wide IO Presentation PyData London
-
Upload
pydata -
Category
Technology
-
view
124 -
download
6
description
Transcript of Wide IO Presentation PyData London
Introduction to
Action Recognition
in Python@wideio
Bertrand NOUVEL, [email protected]
Jonathan KELSEY, [email protected]
Bernard HERNANDEZ, [email protected]
PYDATA LONDON 2014
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Outline
- (While you download the data) Forewords & Overview
- PART 1: Video-processing in python
- PART 2: The pipeline in details
- PART 3: Putting it together
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
FOREWORDS
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
GETTING READY
Python 2.x / 3.x
Numy/Scipy
PIL
OpenCV2
ANACONDA
WAKARI
GET THE ABSOLUTELY ESSENTIAL PACKAGES :
1)
2)GET THE SOURCE CODE
git clone https://bitbucket.org/wideio/pydata.git
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
WHO WE ARE
WIDE IO - Democratising the best algorithms
startup + consultancy
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Why do Computer Vision in Python?
Computer vision is difficult.
Multiparadigm
Best community
Lots of packages
Operator
overloading
Readable
Reflexive
Lightweight
Python is 1000 slower than C++
“””
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
ACTION RECOGNITION
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
What is action recognition ?
SEMANTIC GAPKTH, Human action dataset (Laptev)
Classify actions
Classification task
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Types of Systems
ACTION SPECIFICHOLISTIC
APPROACHES
FEATURE BASED
APPEARANCE
BASED
DEEP LEARNING
Many priors Less priors
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
The Very-Traditional Pipeline
RAW DATA FEATURE-EXTRACTION MACHINE LEARNING
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
The Very-Traditional Pipeline
VIDEO
FRAMESSPARSIFICATION
CLASSIFICATION
METHOD
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
The Very-Traditional Pipeline
VIDEO
FRAMESBAGS OF KEYPOINTS SVM
PART I
PROCESSING
VIDEOS
TOWARDS FRAMEWORKS
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
- PYFFMPEG (outdated)
- OPENCV
- MLT
- ON WAKARI
Reading Videos
anaconda$ vi io/player_mlt.py
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
- PYFFMPEG (outdated)
- OPENCV
- MLT
- ON WAKARI
Reading Videos
anaconda$ vi io/player_cv.py
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Displaying ImagesRecommended Alternatives
anaconda$ python io/display_pyglet.py
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Harris Corner Detector
Detecting interest points:
imsmooth=scipy.ndimage.gaussian_filter
def harris(I,alpha=0.04,si=1):
Ix,Iy = scipy.gradient(I)
H11 = imsmooth(Ix*Ix, si)
H12 = imsmooth(Ix*Iy, si)
H22 = imsmooth(Iy*Iy, si)
return ((H11*H22 - H12**2)
- alpha*(H11+H22)**2)
anaconda$ python keypoints/harris.py
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Optical flow - wrapper based code
anaconda$ python keyponats/custom_feature.py
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Use the two previous elements to compute keypoints that contain information about the movement
Ideas for extension: Make the same with a pyramidal approach.
Spatio-temporal keypoints
anaconda$ python keyponats/custom_feature.py
PART II
UNDERSTANDING THE KEY
ELEMENTS OF THE PIPELINE
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Feature extractions
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
SIFT (128)
SURF (64)Descriptor vector
Feature extractions
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Algorithms:
Type (density, connectivity, …)
Dimension (descriptor, position, …)
Recursive clustering
Connectivity (ward) Centroid (k-means) Density (optics)
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Support Vector Machines
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Support Vector Machines
Weighted-Support Vector Machines
Different support
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Support Vector Machines
Weighted-Support Vector Machines
Different support
Different relevance (outliers)
PART III
PUTTING EVERYTHING
TOGETHER
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Feature detection
Clustering
PCA Projection
BOW
Eigen Vectors & Mean
Graphic ModelModel.model
centers
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Distance of projection to BOW cluster centers with metric
Mean of stack to create histogram
Normalise (Area=1)
Invert with 'discriminator' to turn into weighting
PCA project with graphic model
Feature detection stack for image
WIDE IO - DEMOCRATISING THE BEST ALGORITHMS
INTRODUCTION TO ACTION RECOGNITION PYDATA LONDON 2014 - BERNARD HERNANDEZ - JONATHAN KESLEY - BERTRAND NOUVEL
Training data
Feature computer
Feature stack
Param grid
Training Labels
SVM predict all
Testing labelsTesting data
SVM auto train
Introduction to
Action Recognition
in Python@wideio
Bertrand NOUVEL, [email protected]
Jonathan KELSEY, [email protected]
Bernard HERNANDEZ, [email protected]
PYDATA LONDON 2014