© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

David Pearson, AWS AI Services

May 2017

What’s New in Amazon Rekognition

Extract Rich Image Metadata from Visual Content

Amazon AI

Intelligent Services Powered By Deep Learning

Rich Metadata from Visual Contentobjects, scenes, facial attributes, people

Amazon RekognitionDeep Learning-Based Image Recognition Service

Deer 98.8%

Wildlife 95.1%

Conifer 95.1%

Spruce 95.1%

Wood 78.3%

Tree 63.5%

Forest 63.5%

Vegetation 61.9%

Pine 60.6%

Outdoors 54.0%

Flower 53.9%

Plant 52.9%

Nature 50.7%

Field 50.7%

Grass 50.7%

DetectLabels

"Labels": [{

"Confidence": 98.9294204711914,"Name": "Moss"

},{

"Confidence": 98.9294204711914,"Name": "Plant"

},{

"Confidence": 97.35887908935547,"Name": "Creek"

},{

"Confidence": 97.35887908935547,"Name": "Outdoors"

},{

"Confidence": 97.35887908935547,"Name": "Stream"

},{

"Confidence": 97.35887908935547,"Name": "Water"

},

Beard: False 84.3%

Emotion: Happy 86.5%

Eyeglasses: False 99.6%

Eyes Open: True 99.9%

Gender: Male 99.9%

Mouth Open: False 86.2%

Mustache: False 98.4%

Smile: True 95.9%

Sunglasses: False 99.8%

Bounding Box

Height: 0.36716..

Left: 0.40222..

Top: 0.23582..

Width: 0.27222..

Landmarks

EyeLeftEyeRightNoseMouthLeftMouthRightLeftPupilRightPupilLeftEyeBrowLeftLeftEyeBrowRightLeftEyeBrowUp

:

Quality

Brightness 52.5%

Sharpness 99.9%

"BoundingBox": {

"Height": 0.3449999988079071,

"Left": 0.09666666388511658,

"Top": 0.27166667580604553,

"Width": 0.23000000417232513

},

"Confidence": 100,

"Emotions": [

{"Confidence": 99.1335220336914,

"Type": "HAPPY" },

{"Confidence": 3.3275485038757324,

"Type": "CALM"},

{"Confidence": 0.31517744064331055,

"Type": "SAD"}

],

"Eyeglasses": {"Confidence": 99.8050537109375,

"Value": false},

"EyesOpen": {Confidence": 99.99979400634766,

"Value": true},

"Gender": {"Confidence": 100,

"Value": "Female”}

DetectFaces

smart cropping

& ad overlays

sentiment

capture

demographic

analysis

face editing

& pixelation

Similarity 93%

Similarity 0%

"FaceMatches": [

{"Face": {"BoundingBox": {

"Height": 0.2683333456516266,

"Left": 0.5099999904632568,

"Top": 0.1783333271741867,

"Width": 0.17888888716697693},

"Confidence": 99.99845123291016},

"Similarity": 96

},

{"Face": {"BoundingBox": {

"Height": 0.2383333295583725,

"Left": 0.6233333349227905,

"Top": 0.3016666769981384,

"Width": 0.15888889133930206},

"Confidence": 99.71249389648438},

"Similarity": 0

}

],

"SourceImageFace": {"BoundingBox": {

"Height": 0.23983436822891235,

"Left": 0.28333333134651184,

"Top": 0.351423978805542,

"Width": 0.1599999964237213},

"Confidence": 99.99344635009766}

}

CompareFaces

Collection

IndexFaces

SearchFacesbyImage

Nearest neighbor

search

FaceID: 4c55926e-69b3-5c80-8c9b-78ea01d30690

Similarity: 97

FaceID: 02e56305-1579-5b39-ba57-9afb0fd8782d

Similarity: 92

FaceID: 02e56305-1579-5b39-ba57-9afb0fd8782d

Similarity: 85

New Features in Rekognition

Added to DetectFaces in

February 2017

Values returned as integers

for high and low estimates

Facilitates high scale,

demographic analysis (paired with gender attribute)

Estimated Age Range"AgeRange": {"High": 43,"Low": 26 }

"Gender": {"Confidence": 99.91,"Value": "Male” }

"AgeRange": {"High": 43,"Low": 26 }

"Gender": {"Confidence": 100,"Value": “Female” }

Demographic Analysis

• Touchless data gathering via in-store cameras

• Anonymous, high volume analysis of demographic (age range, gender)

• Extensible to sentiment analysis to measure service quality

Sentiment +

Demographic

Analysis

"AgeRange": {"High": 68,"Low": 48 }

"Gender": {"Confidence": 99.926…,"Value": "Male“ }

"Emotions": [{ "Confidence": 99.449…,

"Type": "HAPPY” } …

"Smile": {"Confidence": 73.576…,"Value": true }

"AgeRange": {"High": 55,"Low": 35 }

"Gender": {"Confidence": 100,"Value": “Female“ }

"Emotions": [{ "Confidence": 99.885…,

"Type": "HAPPY” } …

"Smile": {"Confidence": 99.075…,"Value": true }

Image Moderation

Detect images with explicit or suggestive adult content

Automate and optimize manual review processes

Hierarchical taxonomy provides greater control for geo-sensitive content

"ModerationLabels": [{

"Confidence": 83.55088806152344,"Name": "Suggestive","ParentName": ""

},{

"Confidence": 83.55088806152344,"Name": "Female Swimwear Or Underwear","ParentName": "Suggestive"

}]

}

DetectModerationLabels

Image Moderation

Detect images with explicit or suggestive adult content

Automate and optimize manual review processes

Hierarchical taxonomy provides greater control for geo-sensitive content

Top-Level Category Second-Level Category

Explicit Nudity

Nudity

Graphic Male Nudity

Graphic Female Nudity

Sexual Activity

Partial Nudity

Suggestive

Female Swimwear Or Underwear

Male Swimwear Or Underwear

Revealing Clothes

Optimizing manual review processes

• Automating the detection of inappropriate content with Rekognition

• Reducing volume of images for human curation increases review quality

Amazon Rekognition Consolehttps://console.aws.amazon.com/rekognition/home

Amazon Rekognition Customers• Law Enforcement and Public Safety

• Travel and Hospitality

• Digital Marketing and Advertising

• Media and Entertainment

• Internet of Things (IoT)

Amazon Rekognition Customers• Digital Asset Management

• Media and Entertainment

• Travel and Hospitality

• Influencer Marketing

• Systems Integration

• Digital Advertising

• Consumer Storage

• Law Enforcement

• Public Safety

• eCommerce

• Education

Law Enforcement and Public Safety

Washington County Sheriff (OR)

To follow leads from citizens & security cameras, a person

spends days manually searching thousands of images

The mobile and web app powered by Amazon Rekognition

compares new images with photos of previous offenders:

• Helps identify unknown theft suspects from security footage

• Provides leads by identifying possible witnesses & accomplices

• Identifies persons of interest who do not have identification

Travel and Hospitality

Anticipatory guest experiences for hotels using Amazon

Rekognition for facial recognition and sentiment capture

Kaliber is using Amazon Rekognition to help front desk agents

enhance relationships with guests:

• Recognize guests early for instant and personalized service

• Receive rich, contextualized guest information in real time

• Track guest sentiment throughout their stay

• Drive an 80% increase in guest satisfaction scores

Guest Workflow

Walk in Be recognized Be greeted

Capture sentiment to trigger actionsEnjoy personalized serviceLeave with a fond farewell

“Kaliber allows us to bond with our guests from the second they walk in my hotel.” – GM of a 5-star property

Influencer Marketing

Associate influencers with objects and scenes in social media

images in order to create high impact campaigns for clients

Using Amazon Rekognition for metadata extraction:

• Create rich media indexes of images from social media feeds, which

the application associates with influencers

• Enable analytics to profile environments where influence is strongest

• Connect client brands with the influencers most likely to have impact

Rekognition Demo with Video Frame Sampling

Media and Entertainment

Identify who is on camera for each of 8 networks so

that recorded video can be indexed and searched

Video frame-sampling facial recognition solution

using Amazon Rekognition:

• Indexed 97,000 people into a face collection in 1 day

• Sample frames every 6 secs and test for image variance

• Upload images to Amazon S3 and call Amazon Rekognition

to find best facial match

• Store time stamp and faceID metadata

C-SPAN Indexing Architecture

Video feeds encoded from 8 locations (3 networks and 5

federal courthouses)

Frames extracted into JPGs and hosted in

Amazon S3

Amazon SQS provides asynchronous decoupling

Search Amazon Rekognition collection for high similarity

matches

Results cache drives search and discovery

requests

R3 hashing detects if a scene significantly changes

IoT Use Casereal-time facial recognition at the edge

AWS Advanced Consulting Partner

• Migrations

• DevOps

• Managed Services

• Software & Hardware Engineering

• User Experience & Visual Design

• Rapid Prototyping

AWS Competencies: DevOps, IoT, Healthcare

NERF CS-18 N-Strike Elite Rapidstrike

Adafruit 2.8”

PiTFT displayRaspberry Pi 3

Amazon Rekognition

https://sturdy.cloud/sting/

Training

Image

Amazon Rekognition Availability and Pricing

Free Tier: 5000 images processed per month for first 12 months

General Availability in 3 regions:

US East (N. Virginia), US West (Oregon); EU (Ireland)

Image Analysis Tiers Price per 1000

images processed

First 1 million images processed* per month $1.00

Next 9 million images processed* per month $0.80

Next 90 million images processed* per month $0.60

Over 100 million images processed* per month $0.40

Developer Resources and more…

https://aws.amazon.com/blogs/ai/

https://aws.amazon.com/rekognition

Thank You!

pearsond@amazon.com