Distributing Intelligenceproduction up-time Fraud detection mitigating losses & risk AI use cases We...

2019 Lenovo. All rights reserved.2019 Lenovo. All rights reserved.

Distributing Intelligence: Decentralize AI with Edge Computing

Valerio Rizzo | EMEA AI Solution Architect

2019 Lenovo Internal. All rights reserved.

DECENTRALIZING INTELLIGENCE6-Level rating of Edge Intelligence

Hypotetical Edge Computing architectureThe architecture modes of distributed training

Edge Inference modes

2019 Lenovo. All rights reserved.

OLD DEPLOYMENT OF AITraining + Inference


CURRENT DEPLOYMENT OF AILarger Training

Training + Inference


NEAR FUTURE DEPLOYMENT OF AI

Partial Model Aggregation

Larger Aggregation

Inference


DISTANT FUTURE DEPLOYMENT OF AI

Training + Inference at the edge


SE350

Tiny PC

SE350 Edge Server

1U Rack

Server


AI EDGE INFERENCING

CoreEdge

DataPreprocessing

Big dataInference Training

Data

Trained

Model

Edge/IoT and AI convergence.

Edge inferencing acceleration is now supported on SE350 via

an optional NVIDIA® Tesla T4 GPU

+


Smart Finance’s deep-learning algorithms don’t just look to

the obvious metrics […], it derives predictive power from

data points that would seem irrelevant to a human loan

officer. For instance, it considers the speed at which you

typed in your date of birth, how much battery power is left

on your phone, and thousands of other parameters.

What does an applicant’s phone battery have to do with creditworthiness? This is the kind of question

that can’t be answered in terms of simple cause and effect. But that’s not a sign of the limitations of AI.

It’s a sign of the limitations of our own minds at recognizing correlations hidden within massive streams

of data.

Kai-Fu Lee

MFG Quality control

Image diagnosis

Automated retail checkout

Training/document management

Call Center / Chat Bots

Machine maintenance, Commercial IoT & Fraud detection

Knowledge management, document search

Compliance checks

AI transforming enterprises

Computer

Vision

Natural Language Processing (NLP)

Predictive analytics

Knowledge graphs

AI Core

Capabilities

50%

65%

90%

Supply-chain forcastingimprovements

Lost sales reduced by

Defect detectionImprovement

Manufacturing

Error rate

reduction

Medical

Image

Diagnosis

85%

$60 Billion

25-50%

Global fraud savings

Time savings with credit decisions

Healthcare

Finance

Reduce costs, make better decisions, gain competitive advantage and impact human lives

Use cases

Current AI Adoption WW

Ready for the Industry 4.0

Primary Sector Secondary Sector Terziary Sector

• Largest national producer of corn and beef

meat

• Rice farms (Vercelli e Novara)

• Best and renowned wine producer (Asti,

Cuneo)

• Worldwide famous for the finest truffles (Alba,

Cuneo).

• Multiple energy power plants

• Very active in traditional segments (Steel Industry, mechanical,

textile, food)

• Important hub for cutting edge technologies (IT, Telcom, Space and

Aeronautic Engineer)

• Multiple activities related to car manifacturing directly or inderectly.

• Very dynamic industril areas like Alba (Agricultural and Food prod.),

Biella (textile-fashion), Valenza (oreficery), Casale Monferrato

(industrial freezing plants), area between Maggiore lakes and d’Orta

(houseware and tap and fittings).

• Captivating mountain turistic locations (Sestrière,

Sauze d’Oulx, Bardonecchia, Limone Piemonte)

• Point of interest closest to main lakes (Arona, Stresa,

Orta). Acqui Terme (in provincia di Alessandria)

• World Wide reowned spa resort and thermal stations.

• Exciting gastronomic food tourism in Monferrato and

in the Langhe (Very high quality of food and wine)


Are we late for the AI journey?

No Interest/No plans21%

Discovery/Evaluation31%

Piloting22%

Implementing in next year11%

Implementing in two years11%

Have already deployed4%

How would you describe your organizations adoption of Artificial Intelligence?

n = 400

Source: Cloud and Artificial Intelligence Perceptions Survey 2018, IDC, January, 2018


How Enterprises think of AIIt’s simple, right?

16

Let’s hire a

Data Scientist

Visual inspection for quality control

Image diagnosis Improving patient care

Virtual agents improving customer services

Predictive maintenance for production up-time

Fraud detection mitigating losses & risk

AI use cases

We need AI!

“I read an article in

Forbes and if we don’t

have an AI strategy,

we’re going to fall

behind the

competition!”

Data sources

Additional

sourcesSensorsDevices

Big Data

AI-enabled

applications

AI development

It’s not that easy!


Implementing AIRemoving the complexity of delivering a complete solution

AI Consultants SW Engineers &

AI Consultants Data Wranglers Data Scientist AI engineer

IoT/Edge

Devices

Big Data & Data

Mgmt.. SW

Big Data & Data

Mgmt. HW

Data Collection

SWModel Dev & Mgmt. SW DevOps SW

Model Dev & Mgmt. HW DevOps HW

Find

SolutionAwareness,

Demo, PoC

Data

Collection Extracting,

storing,

retrieving

Data

CleaningCleaning,

formatting,

labeling

Model

Development

&

Management

Model

Tuning &

Retraining

Application

Development

&

Deployment

• Customized solutions

• Reduced cost of deployment

• Access to innovative technology

2019 Lenovo. All rights reserved.2019 Lenovo Internal. All rights reserved.

NOV 2014

140#1

NOV 2015 NOV 2016 NOV 2017 JUN 2018

#1 PROVIDER OF SUPERCOMPUTERS IN THE WORLD

#1 in aggregate

performance

Most diverse

footprint

17 countries

NOV 2018

Position

MORE THAN 1 IN 4 RUN ON LENOVO

0

140+

systems

28%

share



Helping solve Humanity’s Greatest Challenges17 of the Top 25 Research Universities Run Lenovo HPC

Climate Change

Curing Cancer (AI)

Viral Outbreaks

Neuroscience Research

(AI)

Food Supply (AI)

Seismic Research

Particle Physics (AI)

Healthcare (AI)

Manufacturing (AI)

*https://www.timeshighereducation.com/world-university-rankings/2018

https://www.timeshighereducation.com/world-university-rankings/2018


Strategic Partnership between Lenovo and DFKI


MORRISVILLE

STUTTGART

BEIJING

TAIPEI

BC

MA

NU

FA

CT

UR

ING

FIN

AN

CE

RE

TA

IL

HE

ALT

H &

LIF

E S

CIE

NC

E

AC

AD

EM

IA

& R

ES

EA

RC

H

o Improving water conservation

o Better predict climate impact

o Secure global food supply for future generations

Mitigating climate impact on our global food supply with AI

Greatest Societal Challenges: Food, Energy, Water

Global food

production will

need to increase

50% by 2050

70%of fresh water is

used in agriculture

Lenovo AI for MFG

Crop MonitoringCoffe Bean Defects

Detection

Packaging Quality

Control

Compliance

Inspection

High-precision

detection of defects

Ideal for:

• Identifying cropping patterns

• Predicting crops under threat

from droughts and diseases

• Terrain Classification

• Credit scoring solutions using

agricultural data

• Perform all the tasks (data

augmentation, feature

extraction, classification, and

detection) using a single AI

model topology

• provides lightweight and fast

inference time (+5 items per

second)

• ensure quality of small items

• Defect Detection

• Lower Production Cost by

reducing lost sales

• Decrease or eliminate claims

• Increase accountability

• Reducing returns or claims

• Reduce cost and downtime

• Reduce Cost and Time of manual

inspection

• Assuring and Increasing High

Level of quality and long lasting

life of the products

Lenovo AI for Healthcare

Liver Tumor

Segmentation

Early Detection of

Prostate Cancer

Modelling Depth of

Anesthesia

Detection of retinal

PathologiesBCI

Ideal for:

• Analyzes CAT images of patients

to detect and classify tumors

• Enables easier diagnosis by

highlighting the locations and

status of tumors

• Enables faster collaboration

between physicians via 3D

modeling

• diagnosis and management of

cancer using multiple medical

imaging

• decrease cost by minimizing

unnecessary prostate biopsies

and surgeries

• Analysis using multiparametric

imaging

• Providing a safer intraoperative

environment

• Alternative approch to signal

analysis

• For commercial products, e.g.,

merged into patient monitor for AI

index for DOA

• Automatically detect several

pathologies with high accuracy

• Reusing trained models for

problems with larger datasets for

solving problems with little data

• High precision defect detection for

small electronic component

circuities

• Supporting children with ADHD or

patients with cognitive

impairments

• Monitoring mental focus and

concentration during tasks (eg.:

driving, working, learning, ect)

• Convolutional Neural Networks

(CNN) can be used to discover

electric signal features, classify

and predict them


LENOVO CONVERSATIONAL AGENT SYSTEM

MOLI

Multimode/Multimedia/Multilingual Cove rational Agent

Cooperate KGKG Management

Real Representatives

Dashboard

Crowdsourcing

FB

Wechat

APP

Web

H-M Interaction

IT Service Pool

User Profile

Self-learning

Users

Mitigating Deforestation with

AI.Computer Vision in the real world.15% of greenhouse gas emissions

are directly linked to deforestation.

15%

The Earth loses 18.7 million acres of forests per year,

which is equal to 27 soccer fields every minute.

a

By leveraging AI Computer Vision technology,

we can quickly analyze hours of drone footage

and monitor the survival rate of young trees.

Lenovo AI Innovation Center

collaboration with byteLAKE

Solutions to meet you where you are.

Starting your AI Journey

DeploySimplified Deployment

End-to-End Rapid

Deployment

DevelopTools for fast start

Plug-and-play

Simplicity

DiscoverExperience AI

Ecosystem of

Experts and Partners

Worldwide AI

Innovation Centers

HW & SW Platforms

for app development

Solution guidance &

Professional Services

Lenovo AI offerings

Discover

We are just getting started

on AI but don’t know how

to / didn’t identify a use

case

Discover

We already selected a

use case but want to test

technical feasibility and

prove business value

Develop

We are hiring some

talent (1-2 persons) and

looking for HW and SW

tools to get started

Deploy

We have multiple

projects in the pipeline

with teams and are in the

process of scaling AI for

better ROI

Launch:AI

workshop

AI innovation center

PoC

AI starter kits

+

LiCO

AI Ref architectures

+

LiCO

We are simply looking for infrastructure solutions

AI infrastructure (servers, storage, networking)

Reducing uncertainty and complexity

Lenovo AI Products & PortfolioStarter Kit and Scale-out solutions

The AI Data Scientist Workflow with Lenovo

Lenovo ThinkStation P520

Lenovo ThinkStation P920

Lenovo AI Ready

ThinkSystem Servers

Lenovo ThinkSystem SR670

Lenovo Scalable Infrastructure

AI Development AI InferenceAI Training

Powerful model development

• Bringing Deep Learning to your

Desktop

• Your own Sandbox environment for

prototyping & model development

Inference from Core to Edge

• Ruggedized IoT servers, taking

inferencing to the edge

• Dense GPU & CPU platforms for

inferencing in your data center

Scale out distributed Training

• Optimize training infrastructure use

with LiCO

• Expand as your models scale, and

your users grow

• Supported, integrated solutions for

maximized performance

https://www.google.de/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=2ahUKEwj1hLPayMfhAhWDLFAKHRROCcIQjRx6BAgBEAU&url=https://www.lenovo.com/ph/en/data-center/servers/racks/Thinksystem-SR670/p/77XX7SRSR67&psig=AOvVaw09ALovcs5mQ9LcyDMSN--r&ust=1555055657099456




Introducing the ThinkSystem SR670

• 2x top-bin 205W processors

• 2x 2U accelerator bays + interconnect bay

• Configurable by CPU

2U High-Performance system

with modular I/O flexibility to

optimize accelerated workloads

• Available GPU configurations• 4x NVIDIA V100 32GB (AI training)

• 8x NVIDIA T4 (AI Inferencing, Visualization,

Virtualization, Image/Video Processing)

Lenovo is ready for distributed training – but are users?

Source:

Towards Evaluation of Tensorflow Performance in a Distributed Compute Environment

Miro Hodak1, Ajay Dholakia1; 1Lenovo, Data Center Group, Morrisville, NC, USA

Accepted for Presentation/Publication at Tenth TPC

Technology Conference on Performance Evaluation & Benchmarking (TPCTC 2018)

Today’s need – solve the pain point

• What’s the most cost-effective way to deploy

GPUs for multiple users?

• How can I use them most efficiently?

Future need – future enabled

• What’s the most cost-effective way to deploy

multi-GPU systems for distributed training?

• How do I scale GPUs beyond 4? 8? 16? More?

Source:

https://towardsdatascience.com/measuring-actual-gpu-usage-for-deep-learning-training-e2bf3654bcfd

60% of AI workloads consume

less than half a GPU today

https://towardsdatascience.com/measuring-actual-gpu-usage-for-deep-learning-training-e2bf3654bcfd

User B User C User D

waiting in line…

SR670 with 4 GPU’s

User A

Approaches to deploying AI infrastructure

User A User B User C User D User E User n

……

User F

1:1

user to

workstation

or GPU server

• Bare metal speed (full performance) for AI training

• Each user has dedicated resources, whether they are in use or not

• No consistency, no workload scaling, can’t run jobs on other systems even if idle

Compute

(Servers)

Interconnect

(Networking)

Distributed Storage

Management

User Access

User A User B User C User D User E User n

…User F

Shared user access

to a cluster

• Bare metal speed (full performance) for AI training

• Run multiple jobs if resources are available

• Scale training beyond one system

• Consistent management with user flexibility

• Higher utilization, better TCO

AI step 1 is usually cloud – until they get the bill


LiCOLenovo Scalable, Open Sourced AI Solution

Hardware

Cluster mgmt

User interface

APIs

Model

Development

LiCO

• Simplifies AI Development

• End to End Solution

• Dynamic Scale-out Platform

• Real Time AI Solution Monitoring

Develop

LiCO 5.3

• AI Studio Module

• Lenovo Accelerated AI - no-code models

• Support NVIDIA NGC containers

• CLI access within LiCO for expert users

Introducing LiCO

LiCO = Lenovo Intelligent Computing Orchestration

• Lenovo-developed software

– that brings together a number of open-source tools into a single GUI

- to make the use and management of Lenovo computing resources easier

- for both HPC and AI workloads

• LiCO leverages an open-source base

– OpenHPC project software distro for cluster management tools

- augmented with Lenovo value-add open source components

- extended with open AI frameworks, workflows and job management

- to orchestrate and manage HPC and AI workloads


Artificial Intelligence of Things

IOT AIAIoT

AI and IOT overlap. IOT data provides the fuel for AI. Edge inferencing usually

happens on an edge server, IOT gateways, or embedded compute (cameras,

machines). AI training happens in Core data centers: cloud or private DC. Training

and Inferencing usually require accelerators (GPUs, ASICs, FPGAs)


Need of Edge computing

Data Explosion Resiliency Latency Regulation

Too much data;

Bandwidth is time

or cost limited

Data needs to

be acted on

at all times

Latency

requirements

outpace physics

Laws require

data to stay local

and encrypted


Edge Site Challenges

High Heat

Vibration

Dust

Wired

network

not always

available

No IT

staff on site

Attackers have

physical access

Must fit

anywhere

Need full

virtualized

stacks


SE350

Tiny PC

SE350 Edge Server

1U Rack

Server

AI EDGE INFERENCING

CoreEdge

DataPreprocessing

Big dataInference Training

Data

Trained

Model

Edge/IoT and AI convergence.

Edge inferencing acceleration is now supported on SE350 via

an optional NVIDIA® Tesla T4 GPU

+


Key Attributes

Rugged – Continuous

0-55°C, dust, high

shock and vibration

Connectivity – WiFi

and cellular LTE, plus

optical and copper

Management - xClarity

remote mgmt tools,

activation via app

Secure – physical and

cyber tamper detection

and encrypted storage.

Compact – 1U tall, short

depth, half width. Mounts

on rack, wall, shelf…

Performance – Server

class compute and

storage, AI acceleration,

hyper-converged stacks

Video Security & Surveillance SolutionsLenovo Pivot3® Video Smart City Infrastructure

20CAMERAS &

SENSORS

10,000 CAMERAS &

SENSORS

+

SOFTWARE DEFINED STORAGE

Resources can be aggregated

together into a single

shared storage pool, fully

accessible by all cameras

and IOT applications.

ADVANCED ANALYTICS

Facial recognition, vehicle

tracking and recognition,

threat and crime detection,

traffic monitoring,

infrastructure monitoring and

predictive maintenance.

ADVANCED FAULT

TOLERANCE

Patented erasure coding

protects data better than aging

RAID technology, withstanding

simultaneous failure of five

disks an appliance.

Virtual Infrastructure

Virtual SANVirtual Servers

SCALABLE

Easily scale storage, compute,

and bandwidth without

disruption as requirements

expand and change.

Kiana

Briefcase

Pro-Vigil

IBM

VMS

Access

control

Drone

control

Traffic

Mgmt

Milestone

OnISS

Genetec

Verint

Systems

Analytics Apps

BUILT-IN FAILOVER

Virtual servers restart with no

user intervention during

failures, and all previously

recorded video remains

accessible. No need for

redundant software, licensing

or hardware.

VIDEO MANAGEMENT

SYSTEM

Industry leading VMS software

aggregates , stores and

organizes video to both view

live and access recorded video.


Unified platform for simplifying interaction with all the underlying compute

resources

Prototipe and Experiment

Train on distributed clusters

Inference at the edge

Distributing Intelligenceproduction up-time Fraud detection mitigating losses & risk AI use cases We...

Documents

Transcript of Distributing Intelligenceproduction up-time Fraud detection mitigating losses & risk AI use cases We...