Chinese Painting Generation Using Generative Adversarial...

1
Chinese Painting Generation Using Generative Adversarial Networks Yuan Chen ? , Ying Chen † , Guanyang Wang ‡ ? Stanford Applied Physics, † Yahoo!, ‡ Stanford Math Email: {ychen12, ychen107, guanyang}@stanford.edu Introduction: Which Picture is Fake? Example (Reconstructing Problem): reconstruct a Chinese painting from its edge map 1 Left: Our input (edge of the target painting). 2 Middle and Right: One original painting, the other ’drawn’ by computer. 3 Guess which is the real one? The ’fake’ image is generated by Generative adversarial networks (GANs). The purpose of our project is to generate realistic-looking Chinese paintings using different types of GANs. Datasets and Preprocessing Scratching: All the Chinese paintings are scratched from Google and Baidu. The dataset is cleaned to prevent repeated and unrelated images. Reshaping: All images are shaped to be 256 × 256. Edge Extracting: The edges are extracted by Canny edge detection. Implemented Methods cGANs: Used for image-to-image translation (edge→ painting, black white painting→ colored painting) 1 Condition on an input image and generate a corresponding output image. DCGANs: Used for creating new paintings from noise. 1 Both the generator and discriminator use the deep convolutional neural network architecture. WGANs: Used for creating new paintings from noise. 1 Remove log in the loss formula in the original GANs paper. 2 Do not apply sigmoid at the output of D . 3 Clip the weight of D to [-0.01, 0.01]. cGANs performance Edges → Paintings: Left three images: Input Edge, Fake Painting, True Painting Right two images: Loss of Discriminator, Loss of Generator 1 Tones and colorization methods of Chinese paintings are learned. 2 Sometimes cGANs have an effect on sharpening in the spectral dimension, i.e. making images more colorful. Performance of DCGANs and WGANs Generating Paintings: Generate paintings using our full dataset of Chinese Paintings. Left: Paintings generated by WGAN Middle: Paintings generated by DCGAN Top Right: Loss of Discriminator of WGAN Bottom Right: Loss of Generator of WGAN 1 DCGAN and WGAN give results with similar qualities. 2 WGAN is more stable while training. 3 WGAN is slightly slower in training than DCGAN.

Transcript of Chinese Painting Generation Using Generative Adversarial...

Page 1: Chinese Painting Generation Using Generative Adversarial ...cs231n.stanford.edu/reports/2017/posters/311.pdf · Chinese Painting Generation Using Generative Adversarial Networks Yuan

Chinese Painting Generation Using Generative Adversarial NetworksYuan Chen?, Ying Chen†, Guanyang Wang‡

? Stanford Applied Physics, † Yahoo!, ‡ Stanford MathEmail: {ychen12, ychen107, guanyang}@stanford.edu

Introduction: Which Picture is Fake?

Example (Reconstructing Problem): reconstruct a Chinese painting from its edge map

1 Left: Our input (edge of the target painting).2 Middle and Right: One original painting, the other ’drawn’ by computer.3 Guess which is the real one?

The ’fake’ image is generated by Generative adversarial networks (GANs). The purpose of our project isto generate realistic-looking Chinese paintings using different types of GANs.

Datasets and Preprocessing

Scratching: All the Chinese paintings are scratched from Google and Baidu. The dataset is cleaned toprevent repeated and unrelated images.Reshaping: All images are shaped to be 256× 256.Edge Extracting: The edges are extracted by Canny edge detection.

Implemented Methods

cGANs: Used for image-to-image translation (edge→ painting, black white painting→ colored painting)1 Condition on an input image and generate a corresponding output image.

DCGANs: Used for creating new paintings from noise.1 Both the generator and discriminator use the deep convolutional neural network architecture.

WGANs: Used for creating new paintings from noise.1 Remove log in the loss formula in the original GANs paper.2 Do not apply sigmoid at the output of D.3 Clip the weight of D to [−0.01, 0.01].

cGANs performance

Edges → Paintings:

Left three images: Input Edge, Fake Painting, True PaintingRight two images: Loss of Discriminator, Loss of Generator

1 Tones and colorization methods of Chinese paintings are learned.2 Sometimes cGANs have an effect on sharpening in the spectral dimension, i.e. making images more

colorful.

Performance of DCGANs and WGANs

Generating Paintings: Generate paintings using our full dataset of Chinese Paintings.

Left: Paintings generated by WGAN Middle: Paintings generated by DCGANTop Right: Loss of Discriminator of WGAN Bottom Right: Loss of Generator of WGAN

1 DCGAN and WGAN give results with similar qualities.2 WGAN is more stable while training.3 WGAN is slightly slower in training than DCGAN.

Generative Adversarial Networks - HAW Hamburgubicomp/projekte/master... · Generative Adversarial Networks ... Outline 1. Basic idea 2. Generative Adversarial ... com/2017/01/generative-adversarial-networks-hot-topic-machine-learning

Generative Adversarial Networks - HAW Hamburgubicomp/projekte/master... · Generative Adversarial Networks ... Outline 1. Basic idea 2. Generative Adversarial ... com/2017/01/generative-adversarial-networks-hot-topic-machine-learning

Conditioning of Generative Adversarial Networks · 2019-01-31 · Generative Adversarial Networks are: •Generative Models •Parametric –Latent Vector •Differentiable –Allow

Conditioning of Generative Adversarial Networks · 2019-01-31 · Generative Adversarial Networks are: •Generative Models •Parametric –Latent Vector •Differentiable –Allow

Eye In-Painting With Exemplar Generative …...Eye In-Painting with Exemplar Generative Adversarial Networks Brian Dolhansky, Cristian Canton Ferrer Facebook Inc. 1 Hacker Way, Menlo

Eye In-Painting With Exemplar Generative …...Eye In-Painting with Exemplar Generative Adversarial Networks Brian Dolhansky, Cristian Canton Ferrer Facebook Inc. 1 Hacker Way, Menlo

Generative Adversarial Transformers

Generative Adversarial Transformers

Wasserstein Generative Adversarial Networksproceedings.mlr.press/v70/arjovsky17a/arjovsky17a.pdf · Wasserstein Generative Adversarial Networks Martin Arjovsky1 Soumith Chintala2

Wasserstein Generative Adversarial Networksproceedings.mlr.press/v70/arjovsky17a/arjovsky17a.pdf · Wasserstein Generative Adversarial Networks Martin Arjovsky1 Soumith Chintala2

Collaborative Sampling in Generative Adversarial … › pdf › 1902.00813.pdfGenerative Adversarial Networks Generative Adversarial Networks (GANs) (Goodfellow et al. 2014) consist

Collaborative Sampling in Generative Adversarial … › pdf › 1902.00813.pdfGenerative Adversarial Networks Generative Adversarial Networks (GANs) (Goodfellow et al. 2014) consist

ChipGAN: A Generative Adversarial Network for Chinese Ink ...alumni.media.mit.edu/~shiboxin/files/He_MM18.pdfChipGAN: A Generative Adversarial Network for Chinese Ink Wash Painting

ChipGAN: A Generative Adversarial Network for Chinese Ink ...alumni.media.mit.edu/~shiboxin/files/He_MM18.pdfChipGAN: A Generative Adversarial Network for Chinese Ink Wash Painting

Generative Adversarial Networks (GANs)

Generative Adversarial Networks (GANs)

Generative Adversarial Nets: Applications and Extensionsice.dlut.edu.cn/valse2018/ppt/Generative... · Generative Adversarial Nets: Applications and Extensions Wangmeng Zuo Vision

Generative Adversarial Nets: Applications and Extensionsice.dlut.edu.cn/valse2018/ppt/Generative... · Generative Adversarial Nets: Applications and Extensions Wangmeng Zuo Vision

ADVERSARIAL EXAMPLES FOR GENERATIVE MODELS

ADVERSARIAL EXAMPLES FOR GENERATIVE MODELS

Adversarial Distillation of Bayesian Neural Network Posteriorsproceedings.mlr.press/v80/wang18i/wang18i.pdf · 2.3. Generative Adversarial Networks Generative Adversarial Networks

Adversarial Distillation of Bayesian Neural Network Posteriorsproceedings.mlr.press/v80/wang18i/wang18i.pdf · 2.3. Generative Adversarial Networks Generative Adversarial Networks

Lecture 9: Unsupervised, Generative & Adversarial Networks · Lecture 9: Unsupervised, Generative & Adversarial Networks ... GENERATIVE & ADVERSARIAL NETWORKS - 20 o Autoencoders

Lecture 9: Unsupervised, Generative & Adversarial Networks · Lecture 9: Unsupervised, Generative & Adversarial Networks ... GENERATIVE & ADVERSARIAL NETWORKS - 20 o Autoencoders

Generative Adversarial Networks (part 2)deeplearning.cs.cmu.edu/document/slides/bstriner_gans_part_2.pdf · Unrolled Generative Adversarial Networks Unrolled Generative Adversarial

Generative Adversarial Networks (part 2)deeplearning.cs.cmu.edu/document/slides/bstriner_gans_part_2.pdf · Unrolled Generative Adversarial Networks Unrolled Generative Adversarial

Unsupervised Learning – Generative Models (PixelRNNs, PixelCNNs) – Generative ... · 2019. 3. 28. · Generative Adversarial Networks Ian Goodfellow et al., “Generative Adversarial

Unsupervised Learning – Generative Models (PixelRNNs, PixelCNNs) – Generative ... · 2019. 3. 28. · Generative Adversarial Networks Ian Goodfellow et al., “Generative Adversarial

Generative adversarial nets

Generative adversarial nets

GENERATIVE ADVERSARIAL NETWORKS - Deep Learningdeeplearning.cs.cmu.edu/F20/document/recitation/GANS-ppt.pdf · 2021. 3. 22. · Generative Adversarial Networks Generative Models We

GENERATIVE ADVERSARIAL NETWORKS - Deep Learningdeeplearning.cs.cmu.edu/F20/document/recitation/GANS-ppt.pdf · 2021. 3. 22. · Generative Adversarial Networks Generative Models We

Hypothetical Exploration on Generative Adversarial Networks · Convolutional Neural Networks, Generative Adversarial Net- works, Laplacian Pyramid of Adversarial Networks. I. INTRODUCTION

Hypothetical Exploration on Generative Adversarial Networks · Convolutional Neural Networks, Generative Adversarial Net- works, Laplacian Pyramid of Adversarial Networks. I. INTRODUCTION

AnomiGAN: Generative Adversarial Networks forpsb.stanford.edu/psb-online/proceedings/psb20/Bae.pdf2.4. Generative Adversarial Networks GANs25 are designed to complement other generative

AnomiGAN: Generative Adversarial Networks forpsb.stanford.edu/psb-online/proceedings/psb20/Bae.pdf2.4. Generative Adversarial Networks GANs25 are designed to complement other generative

Discriminator-Free Generative Adversarial Attack

Discriminator-Free Generative Adversarial Attack

EmotiGAN: Emoji Art using Generative Adversarial Networkscs229.stanford.edu/proj2017/final-reports/5244346.pdfA. Generative Adversarial Networks A Generative Adversarial Network[4]

EmotiGAN: Emoji Art using Generative Adversarial Networkscs229.stanford.edu/proj2017/final-reports/5244346.pdfA. Generative Adversarial Networks A Generative Adversarial Network[4]