CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC...
Transcript of CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC...
![Page 1: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/1.jpg)
© 2020 and Reg. TM – CMC Microsystems
CMC Infrastructure for Supporting Cloud and Edge
Computing Research
Lowering barriers to technology adoption
YASSINE HARIRI
CMC MICROSYSTEMS
![Page 2: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/2.jpg)
2
Shorten the development cycle
Software
Platforms
Development
Kits
Cloud-based
Accelerator
Cluster
High-
Performance
Platforms
Access platform-based microsystems design and prototyping environments
> Systems
> Equipment Rental, Test Fixtures
> Services for emerging processes and products
> Contract R&D
And more: training, webinars, events, CMC engineer support
![Page 3: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/3.jpg)
3
www.cmc.ca/Lab-Development-Systems
Cloud
> FPGA/GPU Cluster
Edge
> Xilinx ZCU102 Zynq Ultrascale+ MPSoC Evaluation Kit
RISC-V
> RISC-V Processor Design and Prototyping
Development Kit Rental
> Xilinx ZCU102 Zynq Ultrascale+ MPSoC Evaluation Kit
Machine Learning Platform An AI
solution for monitoring and
interpreting aerial surveillance video
and imaging… a project supported
by National Defence, Innovation for
Defence Excellence and Security
(IDEaS) program.
![Page 4: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/4.jpg)
End-to-end Deep Learning platform
© 2019 CMC Microsystems
Prototype Training
Cloud: FPGA/GPU cluster
Edge: MPSoC
Edge Inference
FPGA/GPU clusterDesktop
Cloud Inference
![Page 5: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/5.jpg)
CMC Cloud FPGA/GPU Cluster
© 2019 CMC Microsystems
CPUs, GPUs and FPGAs in pre-validated cluster to scale heterogenous computing workloads Machine learning training and inference (e.g. CNN for object detection, speech recognition)
Video Processing / Transcoding, Financial Computing, Database analytics, Networking
Quantum chemistry, molecular dynamics, climate and weather, Genomics
RISC-V Accelerators in Open Source Cloud Computing
FPGA/GPU cluster Specifications
![Page 6: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/6.jpg)
Software stack for the FPGA/GPU cluster
© 2019 CMC Microsystems
Applications
Middleware, Tools and Libraries
Hardware
ML Framework
![Page 7: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/7.jpg)
Edge Platform: Xilinx ZCU102
7
Courtesy: Xilinx Inc.
• Xilinx Zynq Ultrascale+ MPSoC (ZU9EG)
• Quad-core ARM A53
• Dual-core ARM R5
• ARM GPU
• 16nm FinFET+ programmable logic
• 4GB 64-bit DDR4 (processor)
• 512MB 16-bit DDR4 (FPGA)
• 2x FMC-HPC connectors
• HDMI video input and output
• DisplayPort video output
• SD Card
• Push buttons, DIP switches, LEDs
• USB UART
• Ethernet
![Page 8: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/8.jpg)
© 2019 and Reg. TM – CMC Microsystems
End-to-end Deep Learning platform
Use case
![Page 9: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/9.jpg)
Innovation for Defence Excellence and Security (IDEaS)
Object Detection, Classification and Tracking Using Heterogeneous Computing Architectures
© 2019 CMC Microsystems
Fixed-point Floating-point
Cloud (Training)Edge (Inference)
![Page 10: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/10.jpg)
© 2019 and Reg. TM – CMC Microsystems
Phase I: Training Flow
![Page 11: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/11.jpg)
DLRSD dataset2100 images 256x256 pixels, 21 class labels
© 2019 CMC Microsystems
![Page 12: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/12.jpg)
Step 1 - Data preparation
© 2019 CMC Microsystems
prepair_images.py
create_dataset_lmdb.sh
GLOG_logtostderr=1 $TOOLS/convert_imageset \
--resize_height=$RESIZE_HEIGHT \
--resize_width=$RESIZE_WIDTH \
--shuffle \
$TRAIN_DATA_ROOT \
$DATA/train.txt \
$EXAMPLE/train_lmdb
![Page 13: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/13.jpg)
Step 2 - Model definition
caffenet_train_val_1.prototxt
© 2019 CMC Microsystems
Change the path for input data and mean image Change the
number of
outputs from 1000 to 21
![Page 14: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/14.jpg)
Step 3 - Solver definition
• The solver provide parameters to perform model optimisation and guide the training and testing process.
• The content of solver_1.prototxt is as follow:
© 2019 CMC Microsystems
net: "/home/ideas/.local/install/caffe/cmcideas_dev0/caffenet_train_val_1.prototxt"
test_iter: 400
test_interval: 500
base_lr: 0.001
lr_policy: "step"
gamma: 0.1
stepsize: 5000
display: 20
max_iter: 10000
momentum: 0.9
weight_decay: 0.0005
snapshot: 2000
snapshot_prefix: "/home/ideas/.local/install/caffe/cmcideas_dev0/caffe_model_1"
solver_mode: GPU
![Page 15: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/15.jpg)
Step 4 - Model trainingAt this step, we are ready to train the model by executing the following CAFFE command from the terminal:
© 2019 CMC Microsystems
>caffe train -solver /home/ideas/.local/install/caffe/cmcideas_dev0/solver_1.prototxt 2>&1 | tee
/home/ideas/.local/install/caffe/cmcideas_dev0/train.log
>python /home/ideas/.local/install/caffe/cmcideas_dev0/plot_learning_curve.py
/home/ideas/.local/install/caffe/cmcideas_dev0/train.log
/home/ideas/.local/install/caffe/cmcideas_dev0/learning_curve.png
train.log
![Page 16: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/16.jpg)
Transfer Learning
© 2019 CMC Microsystems
• validation accuracy: ~98%, after 1000 iterations. • validation accuracy: ~85%, after 4000 iterations.
Concept: Instead of training the network from scratch, transfer learning trains an already trained model on a different dataset.
![Page 17: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/17.jpg)
© 2019 and Reg. TM – CMC Microsystems
Phase II: Inference Flow
![Page 18: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/18.jpg)
Xilinx DNNDK • Full-stack SDK for the Deep-learning Processor Unit
(DPU)
• Supports CNN quantization, compilation, optimization
and runtime support
• Network pruning supported by separate license
• Supports Caffe and TensorFlow
• Freely downloaded from Xilinx (registration required)
• Compatible with existing Xilinx tools/flows (Vivado,
SDSoC)
• Supported evaluation boards:
• ZCU102
• ZCU104
• Ultra9618
Accelerating Deep Learning for Embedded Vision at the Edge
![Page 19: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/19.jpg)
Build Hardware and Application Projects in the
SDSoC Development Environment
19
Accelerating Deep Learning for Embedded Vision at the Edge
dpucore_zcu102: ~3.5 hours
gstsdxtrafficdetect: ~10 seconds
![Page 20: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/20.jpg)
Implementation results: Xilinx Vivado
20
Accelerating Deep Learning for Embedded Vision at the Edge
![Page 21: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/21.jpg)
Run the application on ZCU102
21
Accelerating Deep Learning for Embedded Vision at the Edge
![Page 22: CMC Infrastructure for Supporting Cloud and Edge Computing ... · Microsoft PowerPoint - CMC Presentation Author: yassine Created Date: 3/26/2020 12:04:39 PM ...](https://reader035.fdocuments.net/reader035/viewer/2022062605/5fcb7b38e0d99a47af341446/html5/thumbnails/22.jpg)
ASIC Flow
(Synopsys, Cadence, Mentor…)
FPGA Flow
(Xilinx, Intel, Lattice…)IPs IPs
Algorithms
C/C++/SystemC Model
HLS
(Catapult…)
Optimized RTL
ASIC
HW Emulator
Edge FPGA
Cloud FPGA
Processor Models
ASIP, RISC-V…
Processor Design
(ASIP…)SDK
HW/SW Cosim./CoVer.
LIBs
28nm..
5G/Communication Image/Video Processing AI: Computer Vision, NLP..
A Unified Design Flow for Advanced Computing Platforms