Post on 10-Mar-2020
Introducing IBM PowerAI VisionAccelerate AI Vision deployments and increase user productivity
Srini Chitiveli (svchitiv@us.ibm.com)Product Manager
Teaching a computer to recognize a bicycle
ArtificialIntelligence
Mimic Humans
MachineLearning
Learn withExperience
Deep Learning(Neural Networks)
Self-Learn with More Data
Deep Learning Has Revolutionized Machine Learning
4
0
20
40
60
80
100
Source: Google Trends. Search term “Deep Learning”
# of Searches for Deep Learning from 2011 to 2017
TraditionalMachine Learning
Deep Learning
Accuracy
Data
5
Machine Learning
Deep Learning
Input
Deep Neural Network
OutputFeature Extraction & Classification
Input Feature Extraction
Features Classification Output
Machine Learning Algorithms
66
26% Errors
Machine Learning Based
3% Errors
Deep Learning Based
20162011
Humans
5% Error
DEPLOY & INFER
MAINTAIN
ACCURACY
Data Changes, Constant Iteration Required
Model Tuning/ Pruning, Scale &
Performance, Resiliency, Application
Access
DATA
PREPARATION
Complexity /
Technology Rapidly
Changing
Volume, Multi-
SourceLabeling &
Tagging,Ingestion
BUILD, TRAIN,
OPTIMIZE
Hyper-parameter Complexity,
Massive Compute Intensive
Iterations, Long Training Times,
Limited Resources
UP & RUNNING
Pain Points – Deep Learning Pipeline
Sharing Valuable Resources Across Multiple Users, Multiple Lines of Business, Multiple ApplicationsWith Security, Resiliency, and at Scale
IBM PowerAI Vision: ”Point-and-Click” AI for Images & Video
8
Label Image orVideo Data
Auto-Train AI Model
Package & Deploy AI Model
Core Capabilities
9
• RHEL and Ubuntu• Classification of Images• Object Detection• Auto labeling of images and Videos• Prebuilt models for classification• RESTFul interface to integrate into solutions • Bring Your Own Model - Import Custom Models to train and host for inference• Data Augmentation
• Advance metrics on trained models
• Multiple classification of results• Inference on remote x86 and Power remote servers• Compress and accelerate models for FPGA cards on Embedded devices• Optimize and deploy models on FPGAs in Data center (Alveo U200)
• IBM Intelligent Video Analytics for end user experience
Platform support for Training & Inference10
Platform Operating System CUDA
Power8 • RHEL 7.5• Ubuntu 18.04
CUDA 10
Power9 • RHEL 7.5• Ubuntu 18.04
CUDA 10
• Introduced support for Ubuntu 18.04
• Customers can now use this operating system to train and deploy models with PowerAI Vision
Image Classification
1. Get your images2. Create categories3. Assign images to
individual categories4. Train5. Deploy & Infer
LABEL
INFER
TRAINING DATASET
DEMO LINK
Object detection on static images
1. Get your images2. Draw boundaries
around objects of interest
3. Train4. Deploy & Infer
DEMO LINK
LABEL
INFER
TRAINING DATASET
Image Segmentation
• Ability to label objects with multiple point polygons.
• Segmentation helps compute relative size and position of objects compared to rest of the image
I need to count inventory on the shelves.
-- Retail supervisor
LABEL
INFER
Demo Link
Semi-Automatic Labeling using PowerAI Vision
14
Train DL Model
Define LabelsManually Label Some
Images / Video Frames
Manually LabelUse Trained DL
Model
Run Trained DL Model on Entire Input Data to Generate Labels
Correct Labels on Some Data
Manually Correct Labels on Some Data
Repeat Till Labels Achieve Desired Accuracy
Auto labeling on Videos & Images
1. Identify small video2. Manually label
dataset3. Train model4. Use model to auto
label video5. Audit dataset for
accuracy6. Re-train model for
higher accuracies
VIDEO
AUTO LABEL
APPLICATION COUNTING CARS
DEMO LINK
Installed base models▪ Prebuilt base-models for known objects around us
▪ Prebuilt base-models transfer learn faster on the defined topics
▪ Import custom base-models for transfer learning
Rich set of RESTFul APIs
APIs to programmatically:
• Create datasets for Classification and Object detection
• Export and Import datasets
• Trigger and monitor training process
• Deploy models for inference
• Build solutions by detecting objects in segmented areas
Augment limited datasets for higher accuracywork with limited data
• Generate variety of images for initial datasets
• Software can apply filters to augment data and increase images for training
• Augmented data reduces overfitting for small datasets and increases accuracy
Limited dataset Inbuilt augmentation algorithms Augmented datasets
Advanced metrics on model training19
• Introducing metrics to show accuracy of trained model
• Total Recall• Total Precision• PR Curve• Confusion
Matrix• Hyper Params• Loss Vs
Iteration• Total Accuracy
• Insights for Data scientists to identify issues with dataset
Multiple classification of Images
Group Name / DOC ID / Month XX, 2018 / © 2018 IBM Corporation
20
• Inference now result multiple categories for an image sorted by confidence
• Useful for users to analyze issues with datasets and balance out images per category to increase accuracies
• REST APIs for inference now accept a threshold for results
Bring your Own Model for Training and Inference21
• Data scientists can now out-source training and deployment jobs on their custom models
• Data scientists can focus on innovating models for futures
• Limited to TensorFlow only
22
Train on central server but deploy on several remote
servers
1 - N
Datacenter
PAI Vision for Training and
Inference
AC922
.
Trained models
1. Manually export trained models from Central server to remote locations
2. Import and deploy models with Inference-only license of PowerAI Vision
3. Once the models are deployed, each plant can work stand alone for inference.
4. Supports Power servers with minimum of one GPU
Provided in MVP
New feature (Inference-only)
Models trained on PAI Vision; Password protected
Remote Plant
PAI Vision for Inference
only
AC922/LC922/x86
Trained models
Remote Plant
Remote Plant
Remote Plant
PAI Vision for Inference
only
AC922/LC922/x86
Trained models
Inference on Remote servers
Group Name / DOC ID / Month XX, 2018 / © 2018 IBM Corporation
23
• Train on an AC944 with 4 GPUs
• Deploy models on remote servers with • no accelerator (aka CPU
only)• With GPU• With FPGA (Power only)
• Ability to use existing x86 servers for inference
• Run inference closer to the source of data (at the edge)
Version 1.1.1
Version 1.1.2
Deep Learning Models
COD:FRCNN
CIC: google-net
Base: CAFFE
COD: FRCNN, Tiny YOLOand Custom model
CIC: google-net and Custom model
Base: CAFFE and Tensorflow
Processors POWER POWER and x86
Acceleratoroptions
GPU GPU/CPU only/FPGA (Power Only)
OS RHEL 7.5 RHEL 7.5 and Ubuntu18.04
Train, Optimize and Infer on FPGA (Embedded and data center)
CPU +
AcceleratorNeural network processor
Embedded GPU
Embedded FPGA
FPGAs, CPUs, GPUs
Trained
DNN model
Model Parser & Compiler
Model Optimization(Layer Merge + Quantization)
Output model + weights
PowerAI VisionMap to Different
Platforms
Data Center: Train model & Compile to EdgeCloud or Edge
24
Xilink Alveo U200
Accelerateusingappropriatetechnology
• ML-Suite• TensorRT• PIE
Sim
ilar R
esu
lts
Why FPGA ?25
• Built only for inference, NOT for training
• Inexpensive• Embed into Servers• Embed into devices
like cameras• Use low power• Perform better or at
par with GPUs• Fast adoption in the
market place
* Source: Accelerating DNNs with Xilinx
Alveo Accelerator Cards White Paper
Introducing acceleration of models for FPGA
• Ability to compress optimized models for a reference FPGA card (Xilink ZC706 Evaluation kit)
• Provide support (L1, L2, L3) only for generating the model
• Instructions to compress models for other Xilink chips
• Building parts specialized in FPGA cards
• V3Red technologies
• KRTKL ( snickerdoodle board)
• PointRData Systems
Xilinx Zynq-7000 SoC ZC706 Evaluation Kit
Datacenter
PAI Vision for Training and
Inference
AC922
Trained models
Train to deploy on server with FPGA cards (Alveo U200)
Group Name / DOC ID / Month XX, 2018 / © 2018 IBM Corporation27
TRAIN
DEPLOY
• Model quantization now part of training
• Modeling optimization for FPGA is now a clickable job
• Support limited to Xilink Alveo U200 on Power system only
• Need server with FPGA in ICP cluster
Technology Preview
IBM Intelligent Video Analytics
Detect Changes to Patterns
Facial Recognition & People Search
Redaction of Faces
Video Analytics Software with Pre-Trained AI Models
Complex Event Monitoring with GUI-based Configuration
Targeted at Public Safety, Remote Monitoring, etc
28
demo
Customize intelligence video analytics
29
New AI Model
PowerAI Vision
Label
Train
Deploy
IBM IVA
GUI
AI Models
Event Detection
Live Video
New Video Training Data
GPU-Accelerated Power Servers for
Model Training
GPU-Accelerated Power Servers for
Inference
demo
30
Customizing Video Analytics with PowerAI Vision
DEMO PAIV
PowerAI VisionIntelligent Video
Analytics
Select library of training images/videos
Auto-label / classify objects of interest
Train Deploy & Infer
Run & Manage
Re-train
Power Play
DEMO INTEGRATION
Hardware Advantage: 5x Faster Data Communication with Unique
CPU-GPU NVLink High-Speed Connection
1 TB
Memory
Power 9
CPU
V100
GPU
V100
GPU
170GB/s
NVLink150 GB/s
1 TB
Memory
Power 9
CPU
V100
GPU
V100
GPU
170GB/s
NVLink150 GB/s
IBM AC922 Power SystemDeep Learning Server (4-GPU Config)
Store Large Models in System Memory
Operate on One Layer at a Time
Fast Transfer via NVLink
31
Inference API deployment for
Cloud
Video and Image Labeling
Data Preprocessing
Self-defined Training
with visualized monitoring
Custom Learning
Complete solution built from Ground Up
Data Lake & Data Stores
Distributed Computing
ML & DL Libraries & Frameworks
Accelerated Servers Storage
TrainingData
Inference accelerator
generation for edge
Inference
AI DEVELOPMENT WORKFLOW
Testing & Measurement
33
Track how customers navigate store, identify
fraudulent actions, detect low inventory for items
Generates track summary, flags missing objects, alerts on suspicious
behavior
Utilize surveillance cameras to ensure worker
safety compliance
Enable zone monitoring, heat maps, detection of
loitering
Identify faulty or worn out equipment in remote & hard to reach locations
Alert to schedule maintenance job, along
with critical infrastructure security
34
Identify, track, and apprehend suspects to improve public safety.
Facial recognition, Object missing / left behind,
Track Summary
Remotely assess potential threats and current status
of critical assets.
object recognition, asset condition analysis, theft
and vandalize, critical infrastructure security
Public and private monitoring of perimeters
and restricted areas.
Zone violation, Intrusion detection, Heat map,
Loitering
Possibilties
35
SMARTER CITIES
DRONE SURVEILLANCE SPORTS AND ENTERTAINMENT
Possibilties
36
RETAIL FRAUD EDGE INFERENCE
37
Value proposition
➢AI Made Simpleo Clicker tools to train models with no coding or expertise in technologies
➢End to End ecosystem o Disjoint activities streamlined into simple sequential taskso Life cycle management for models and datao Train on server but deploy on cloud or edge for inference
➢Enterprise grade offeringo Collaborative platform between several personaso Open architecture extensible with existing enterprise assetso Backed with support and services from IBM and business partners
Getting started with PowerAI Vision
Offering detailsIntroduction to IBM PowerAI Vision
IBM MarketPlace
IBM Developer Works
Access to SoftwareDownload Trial version of PowerAI Vision
Access Technology preview as a service
38
Developer JourneysTrain models for Advanced Driving Assistant Systems
Train models to detect flavors of Coke
Classify Photo resist wafers
Count cars and objects
VideosTrain models for Classification and Object detection
Train models for Advanced Driving Assistant Systems
Continuous learning for data labeling
Demos on YouTube
Srini Chitiveli (svchitiv@us.ibm.com)Product Manager
THANK YOU
Introducing IBM PowerAI VisionAccelerate AI Vision deployments and increase user productivity