Disrupting The Creative Industry with AI - NVIDIA€¦ · Challenge: the client selected a sample...
Transcript of Disrupting The Creative Industry with AI - NVIDIA€¦ · Challenge: the client selected a sample...
® 2017 - Confidential
Disrupting The Creative Industry with AIMarco Marchesi - Head Of Tech
Daniel Cheetham - Global Chief Interactive Officer
® 2017 - CONFIDENTIAL
® 2017 - CONFIDENTIAL
® 2017 - CONFIDENTIAL
® 2017 - CONFIDENTIAL
® 2017 - CONFIDENTIAL® 2017 - CONFIDENTIAL
® 2017 - CONFIDENTIAL® 2017 - CONFIDENTIAL
® 2017 - CONFIDENTIAL
Transparently Immersive
Experiences
® 2017 - CONFIDENTIAL
® 2017 - CONFIDENTIAL
Greater Engagement
& Relevance
® 2017 - CONFIDENTIAL
Use Artificial Intelligence to create an image of a ‘perfect mum’, based on a data set of unrealistic depictions of motherhood in the media and on social networks.
Make sure the output resolution holds up on a massive digital out of home screen in London
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Goals
Dataset of faces (1796 images)
?
1024px
1024px
• Generate a face from a limited dataset of faces • Achieve photorealism, industry quality (and Megapixel size)
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Generative Models
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
• Variational Autoencoders (VAE)
• PixelRNNs
• Generative Adversarial Networks
VAE BEGAN
Oord, Aaron van den, Nal Kalchbrenner, and Koray Kavukcuoglu. "Pixel recurrent neural networks." arXiv preprint arXiv:1601.06759 (2016).
Berthelot, David, Tom Schumm, and Luke Metz. "Began: Boundary equilibrium generative adversarial networks." arXiv preprint arXiv:1703.10717 (2017).
Kingma, Diederik P., and Max Welling. "Auto-encoding Variational Bayes." arXiv preprint arXiv:1312.6114 (2013).
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Z Generator Fake Sample
Discriminator Real Sample DatasetScore
Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014
GAN
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
GAN
Probability that the image came from the dataset
Probability that the image came from the Generator
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Deep Convolutional GAN
Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).
“Traditional” GANs are difficult to train DCGAN
DCGAN characteristics:• Pooling Layers replaced with strided (D) and fractional-strided (G) convolutions
• Removed the FC Layers • Batch Normalization on D and G Layers• ReLU for G and LeakyReLU for D
Generator Architecture [Radford]
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Our Implementation
Characteristics:
• D and G trained alternatively twice (every 50 epochs)• Batch size from 128 (192px) down to 6 (1024px)• Epochs 100 to 500• LR 0.0002• GPUs 8GB & 12GB
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Training Process
A training sequence (from a batch size = 128)
video sequence here
Samples generated at 192x192px
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
First Results
A special case in testing
z = np.random.uniform(-1, 1, size=(sample_num, z_dim))
z is usually a vector in a random uniform (or gaussian) distribution
z = np.zeros((sample_num, z_dim))
z = [0,0,0,…,0]
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Optimizing the Latent Space
Idea: z as a hyperparameterto minimize the testing loss
Our best choice at 1024px
z = np.random.uniform(-0.5, 0.5, size=(sample_num, z_dim))
Recent ideas:Generative Latent Optimization
(GLO)
Bojanowski, Piotr, et al. "Optimizing the Latent Space of Generative Networks." arXiv preprint arXiv:1707.05776 (2017).
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Optimizing the Latent Space
Samples generated at 192x192px with reduced latent space random distribution
22
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
New Results
82px
189px 2810px
326px
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
New Challenge: upscaling
Challenge: the client selected a sample of 256px
Idea: upscale to 1024px and find z from G(z) BiGAN
Sample selected at 256px
Donahue, Jeff, Philipp Krähenbühl, and Trevor Darrell. "Adversarial feature learning." arXiv preprint arXiv:1605.09782 (2016).
Super-resolution?
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
New Challenge: upscaling
Problem: P(G(z)) changes when we scale up (new
training and different model)
Samples generated at 1024px
BiGAN needs further investigation
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
New Challenge: Z mapping
Solution: Semi-automatic z mapping
G(z)
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
New Challenge: Z mapping
The “Perfect Mum” image The final ad campaign
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Final Result
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Some Stats & Campaign Results
● Sales increase 160% above projected launch target.
● 4.2 million social impressions achieved double Dove’s usual engagement levels.
We generated the first Megapixel image using GANs with some relevant limitations:
Time Dataset Hardware
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
Conclusions
The Turing Test of Creative Retouching
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
What’s next
The Turing Test of Creative Retouching
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
What’s next
The Turing Test of Creative Retouching
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
What’s next
• Marchesi, Marco. "Megapixel Size Image Creation using Generative Adversarial Networks." arXiv preprint arXiv:1706.00082 (2017).
• Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014• Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).
• Kingma, Diederik P., and Max Welling. "Auto-encoding Variational Bayes." arXiv preprint arXiv:1312.6114 (2013).• Oord, Aaron van den, Nal Kalchbrenner, and Koray Kavukcuoglu. "Pixel recurrent neural networks." arXiv preprint arXiv:1601.06759 (2016).
• Berthelot, David, Tom Schumm, and Luke Metz. "Began: Boundary equilibrium generative adversarial networks." arXiv preprint arXiv:1703.10717 (2017).
• Bojanowski, Piotr, et al. "Optimizing the Latent Space of Generative Networks." arXiv preprint arXiv:1707.05776 (2017).• Donahue, Jeff, Philipp Krähenbühl, and Trevor Darrell. "Adversarial feature learning." arXiv preprint arXiv:1605.09782 (2016).
DISRUPTING THE CREATIVE INDUSTRY WITH AI Happy Finish.
References
www.happyfinish.com Everything is possible.