AWS | Data Analytics | AI | ML · AWS Marketplace for machine learning Natural Language Processing...

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark

AWS | Data Analytics | AI | ML Build Smarter Video Workflows | Transform the Viewing Experience

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark

Sample ML Use Cases in M&E

Automate the creation of rich metadata index (object, scene, activity, faces etc.), extracted from audio

visual content and integrated into digital and media asset management systems

AWS SERVICES TO BUILD USE CASE: REKOGNITION, SAGEMAKER, TRANSCRIBE, COMPREHEND

Media Metadata

Tagging

Automated captions, transcription and translation of audio content

AWS SERVICES TO BUILD USE CASE: TRANSCRIBE, TRANSLATE, COMPREHEND

Closed Captioning

Detect potentially inappropriate content to avoid compliance issues in global markets, and to increase

brand safety for advertisers

AWS SERVICES TO BUILD USE CASE: REKOGNITION, TRANSCRIBE, SAGEMAKER

Automated

Compliance Marking

Identify objects and emotion in content that enables users to integrate personalized ads into

subscriber video streams

AWS SERVICES TO BUILD USE CASE: SPARK/EMR, SAGEMAKER, MEDIATAILOR

Ad Personalization


Identify and path player activity in the field or identify the precise timecodes when actors enter and

leave a scene, both visually and in spoken dialog (off-screen voice)

AWS SERVICES TO BUILD USE CASE: REKOGNITION, TRANSCRIBE

Player/ Actor

identification &

activity pathing

Translate transcripts and metadata. Improve localization workflows and search experience

AWS SERVICES TO BUILD USE CASE: TRANSLATE

Language

Translation

Analysis disparate data and enable the consumer to make personalized content choices and the

business to predict behavior

AWS SERVICES TO BUILD USE CASE: SAGEMAKER

Content

Recommendations

Detect and pixelate faces captured incidentally to preserve the privacy of non-persons of interest in

news feeds and security footage

AWS SERVICES TO BUILD USE CASE: REKOGNITION, TRANSCRIBE, SAGEMAKER

Automated

Redaction

Sample ML Use Cases in M&E

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark

M L F R A M E W O R K S &

I N F R A S T R U C T U R E

A I S E R V I C E S

R E K O G N I T I O N

I M A G E

P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D

& C O M P R E H E N D

M E D I C A L

L E XR E K O G N I T I O N

V I D E O

Vis ion Speech Chatbots

A M A Z O N

S A G E M A K E R

B U I L D T R A I N

F O R E C A S TT E X T R A C T P E R S O N A L I Z E

D E P L O Y

Pre-bui l t a lgor i thms & notebooks

Data label ing (G R O U N D T R U T H )

One-c l ick model t ra in ing & tuning

Opt imization (N E O )

One-c l ick deployment & host ing

M L S E R V I C E S

F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e

E C 2 P 3

& P 3 d n

E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C

I N F E R E N C E

Reinforcement learningAlgor ithms & models ( A W S M A R K E T P L A C E

F O R M A C H I N E L E A R N I N G )

Language Forecast ing Recommendations


Amazon SageMaker Ground Truth

An accurate training

data set is ready for

use

Training data the model

understands is labeled automatically

Ambiguous data is sent to

human labelers for

annotation

Human labeled data is then sent

back to retrain and improve the

machine learning model

Raw data Active learning model

is trained from human

labeled data

ML Training data using your own game intelligence

AWS Marketplace for machine learning

Natural Language Processing

Computer Vision

Speech Recognition

Text Clustering

Text Generation

Text Classification

Grammar and Parsing

Named Entity Recognition

Text to Speech

Handwriting Recognition

Object Detection in Images

3D Images

Text OCR

Video Classification

Speaker Identification

Ranking

Regression

Anomaly Detection

Browse or searchAWS Marketplace

Subscribein a single click

Available through Amazon SageMaker

AWS Confidential - Do not Distribute

O C R + + s e r v i c e t o e a s i l y e x t r a c t t e x t a n d d a t a f r o m v i r t u a l l y a n y d o c u m e n t

A V A I L A B L E I N P R E V I E W T O D A Y

Amazon Textract

N o M L e x p e r i e n c e r e q u i r e d

NEW!

Customer DetailsCustomer Details

Orders

Totals

Orders

Totals


Traditional OCR only provides a “bag of letters”

MethOd Num' C'USte'S Rand mdex ated using two types of measures. The first is the average

TM~score 8 89.7% silhouette width itself, which is a measure of the clus-

ppm 9 39,396 ter compactness and separation. In general, clustering is

305C 9 895% based on the assumption that the underlying data form

compact clusters of similar characteristics. Larger aver-

R50 7 92.096

age Silhouette Width means that the result of a clustering


OCR++

AmazonTextract

Method Num. clusters Rand index

TM-score

FPFH

3DSC

RSD

VFH

Combined silhouette weights

Combined equal weights

8

9

9

7

8

7

7

89.7%

89.3%

89.5%

92.0%

85.3%

92.2%

90.2%

Aurora

Amazon Textract: an organized filing cabinet of document data


Amazon Textract: automatic document processing without data entry, or writing rules

Graceland, Memphis

Presley, Elvis Aaron

TCB Limited

12-12-1234

TN

01 08 1935 X

901 555-0187

3765 Elvis Presley Blvd.

38116

X

RCA Records

Rock n Roll Health

X

Presley, Elvis Aaron

Presley, Elvis AaronN A M E

Graceland, Memphis, TNA D D R E S S

12-12-1234I D

TCB LimitedC O M P A N Y

Government forms (e.g.

FDA new drug application,

financial disclosure form,

incident reporting)

Tax forms (US – e.g. W2,

1099-MISC, 990, 1040; UK –

e.g. P45; Canada – e.g. T4,

T5)

OCR++

Presley, Elvis AaronN A M E

Graceland, Memphis, TNA D D R E S S

12-12-1234I D

TCB LimitedC O M P A N Y

Graceland, Memphis

Presley

TCB Limited

12-12-1234

TN

901 555-0187

3765 Elvis Presley Blvd.

38116

Elvis

[email protected]

Amazon Textract: automatic document processing without data entry or writing rules

OCR++



Sentiment analysisDiscover insights and relationships in text, Social media, etc..

Entities

Key Phrases

Language

Sentiment

Sentiment Analysis

(Amazon Comprehend)

360º view of customers

(Data Lake)

Storage | Archival Storage |

Data Catalog


Amazon Athena

Amazon EMR

Amazon Redshift

Amazon Elasticsearch service

Amazon Kinesis

Amazon QuickSight

AnalyticsAmazon SageMaker

AWS Deep Learning AMIs

Amazon Rekognition

Amazon Lex

AWS DeepLens

Amazon Comprehend

Amazon Translate

Amazon Transcribe

Amazon Polly

Machine Learning

AWS IoT Core

Amazon Kinesis Data Firehose

Amazon Kinesis Data Streams

Amazon Kinesis Video Streams

Real-time Data Movement

AWS Direct Connect

AWS Snowball

AWS Snowmobile

AWS Database Migration Service

On-premises Data Movement

Data Lake on AWSStorage | Archival Storage | Data Catalog

Incoming Analytics Data Existing Asset Metadata

Data Lake + Media Assets = Content Lake


Media Enrichment and Analysis


Media enrichment: Automated Metadata Generation

Video

Rekognition

Video

Search Engine:

Amazon ElasticsearchAsset Management System

1. Video is uploaded

and stored to the

Data lake

2. Create metadata for

celebrities, emotions,

scene time, objects, voices

in video

3. The output is sent to the

digital/media asset

management system and

available in a search

engine

Dynamic search indexing

Transcribe

Data Lake on AWSStorage | Archival Storage | Data Catalog


Use-Case: From manual tagging to AI based tagging

Background

• Over 200,000 hours of content

• Only half of content is manually tagged

• How can we enrich our metadata in AWS?

• How can we unleash the value of content we already

own once in AWS?

Challenge

• Large scale video library

• High accuracy required

• Limited budget

• Ability to extract from video

• Keep up with daily increase in content

Results

• Solution developed within three weeks

• Live video frame based analysis

• Established, searchable baseline archive

• All content is now tagged and indexed

• Over 99,000 faces indexed and searchable

• Saved ~9,000 hours a year in manual curation costs


Live and Post-Production Subtitling and Translation


• Isolate and pixelate specified

faces

• Automate pipeline to

implement “privacy by design”

redaction

Automated redaction


Tag, filter, and redact inappropriate content

Person 99.2% Gun 84.6% Handgun 73.5%

Drink 96.4% Alcohol 80.1% Wine 69.9%


Personalization (Content Recommendation)

© 2019, Amazon Web Services, Inc. or its Affiliates.

Activity stream from app

Views, signups,

conversion, etc.

Inventory

Articles, products,

videos, etc.

Demographics (optional)

LOAD DATA(EMR Clus ter )

INSPECT

DATA

IDENTIFY

FEATURES

SELECT

ALGORITHMS

SELECT

HYPERPARAMETERS

TRAIN

MODELS

OPTIMIZE

MODELS

HOST

MODELS

BUILD FEATURE

STORE

CREATE

REAL-TIME

CACHES

Customizedpersonalization &recommendation

API

F u l l y m a n aged b y A m a zo n P e rs o na l i ze

Amazon Personalize

Machine learning personalization and recommendations

Age, location, etc.


Amazon Personalize


Discovery Workshops - ML Solutions Lab – ML ProServ

Brainstorming

Custom modeling

Training

Work side-by-side with Amazon experts

How we can help

Partner Ecosystem

Media Solutions


AWS Media Solutions

Accelerate Revenue

Get started with your project quickly and

start monetizing your content

One-Click Launch

Launch a fully-tested solution starter kit in

your AWS account with one click

Well-architected

Rely on the architecture as it reflects AWS

best practices

Agility

Save the resources of developing a

solution from the beginning

Expand and Add

Use the framework and available guides to expand the

solution starter kit based on your needs


AWS Media Solutions

Media Analysis Solution – Machine Learning in Media solution

Live Subtitling and Translation solution (in Beta)

Now

Available

Deploy – Launch – Analyze your Media !

aws.amazon.com/solutions > ‘Media & Entertainment’

Media2Cloud : End to End Solution


Thank You!

AWS | Data Analytics | AI | ML · AWS Marketplace for machine learning Natural Language Processing...

Documents

Transcript of AWS | Data Analytics | AI | ML · AWS Marketplace for machine learning Natural Language Processing...