Generative Adversarial Network (GAN) and its Applications...

19
Generative Adversarial Network (GAN) and its Applications to Human Language Processing Hung-yi Lee Full version of tutorial: https://www.slideshare.net/ssuserf10be3/icassp-2018- tutorial-generative-adversarial-network-and-its-applications-to-signal-processing- and-natural-language-processing

Transcript of Generative Adversarial Network (GAN) and its Applications...

Page 1: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

GenerativeAdversarialNetwork(GAN)anditsApplicationsto

HumanLanguageProcessing

Hung-yiLee

Fullversionoftutorial:https://www.slideshare.net/ssuserf10be3/icassp-2018-tutorial-generative-adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

Page 2: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

SupervisedLearning

NeuralNetwork

Manykindsofnetworks:

Howtofindthefunction?

Giventheexamplesofinputs/outputsas(trainingdata):{(x1,y1),(x2,y2),……,(x1000,y1000)}

Ø FullyconnectedfeedforwardnetworkØ Convolutionalneuralnetwork(CNN)Ø Recurrentneuralnetwork(RNN)

𝑥 𝑦“Howareyou”

“Cat”

redhair,green eyes

Page 3: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

ThankstoGAN

Transformanobjectfromonedomaintoanotherwithoutpaireddata (e.g.styletransfer)

DomainX DomainYNeuralNetwork

DomainX DomainY

Nopaireddata!

Howtoachievethat?https://youtu.be/-3LgL3NXLtI

Page 4: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

UnsupervisedLearningbyGAN

Itisgood.It’sagoodday.

Iloveyou.

Itisbad.It’sabadday.

Idon’t loveyou.

PositiveSentences NegativeSentences

Page 5: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

• Negative sentence to positive sentence:it's a crappy day → it's a great dayi wish you could be here → you could be hereit's not a good idea → it's good ideai miss you → i love youi don't love you → i love youi can't do that → i can do thati feel so sad → i happyit's a bad day → it's a good dayit's a dummy day → it's a great daysorry for doing such a horrible thing → thanks for doing a great thingmy doggy is sick → my doggy is my doggymy little doggy is sick → my little doggy is my little doggy

Learnmorefromourposter

Page 6: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

summarydocument

UnsupervisedLearningbyGAN

Itisgood.It’sagoodday.

Iloveyou.

Itisbad.It’sabadday.

Idon’t loveyou.

UnsupervisedSummarization

Page 7: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

AbstractiveSummarization

• Nowmachinecandoabstractivesummarybyseq2seq(writesummariesinitsownwords)

summary 1

summary 2

summary 3

LabelledTrainingData

summary

seq2seq(initsownwords)

Supervised:Weneedlotsoflabelledtrainingdata.

Page 8: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

UnsupervisedAbstractiveSummarization• Nowmachinecandoabstractivesummarybyseq2seq(writesummariesinitsownwords)

summary 1

summary 2

summary 3

seq2seq

Page 9: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

UnsupervisedAbstractiveSummarization• Document:澳大利亞今天與13個國家簽署了反興奮劑雙邊協議,旨在加強體育競賽之外的藥品檢查並共享研究成果 ……

• Summary:• Human:澳大利亞與13國簽署反興奮劑協議• Unsupervised:澳大利亞加強體育競賽之外的藥品檢查

• Document:中華民國奧林匹克委員會今天接到一九九二年冬季奧運會邀請函,由於主席張豐緒目前正在中南美洲進行友好訪問,因此尚未決定是否派隊赴賽 ……

• Summary:• Human:一九九二年冬季奧運會函邀我參加• Unsupervised:奧委會接獲冬季奧運會邀請函

感謝王耀賢同學提供實驗結果

Page 10: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

UnsupervisedAbstractiveSummarization• Document:據此間媒體27日報道,印度尼西亞蘇門答臘島的兩個省近日來連降暴雨,洪水泛濫導致塌方,到26日為止至少已有60人喪生,100多人失蹤 ……

• Summary:• Human:印尼水災造成60人死亡• Unsupervised:印尼門洪水泛濫導致塌雨

• Document:安徽省合肥市最近為領導幹部下基層做了新規定:一律輕車簡從,不準搞迎來送往、不準搞層層陪同 ……

• Summary:• Human:合肥規定領導幹部下基層活動從簡• Unsupervised:合肥領導幹部下基層做搞迎來送往規定:一律簡

感謝王耀賢同學提供實驗結果

Page 11: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

Semi-supervisedLearning

24

26

28

30

32

34

0 10k 500k

ROUG

E-1

Numberofdocument-summarypairsused

WGAN Reinforce Supervised

Usingmatcheddata

(3.8Mpairsareused)

(unpublished result)

Page 12: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

summarydocument

SpeakerA SpeakerB

UnsupervisedLearningbyGAN

Itisgood.It’sagoodday.

Iloveyou.

Itisbad.It’sabadday.

Idon’t loveyou.

UnsupervisedVoiceConversion

Page 13: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

VoiceConversion

Page 14: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

Inthepast

Today

SpeakerA SpeakerB

Howareyou? Howareyou?

Goodmorning Goodmorning

SpeakerA SpeakerB

天氣真好 Howareyou?

再見囉 Goodmorning

SpeakersAandBaretalkingaboutcompletelydifferentthings.

Page 15: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

SpeakerA SpeakerB

我

感謝周儒杰同學提供實驗結果

Page 16: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

summarydocument

SpeakerA SpeakerB

UnsupervisedLearningbyGAN

Itisgood.It’sagoodday.

Iloveyou.

Itisbad.It’sabadday.

Idon’t loveyou.

Audio Text

Page 17: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

UnsupervisedSpeechRecognition

Thedog is……

Thecatsare……

Thewomanis……

Themanis……

Thecatis……CycleGAN

“The”=

AcousticPatternDiscovery

Canweachieveunsupervisedspeechrecognition?

p1

p1 p3 p2

p1 p4 p3 p5 p5

p1 p5 p4 p3

p1 p2 p3 p4

[Liu, et al., arXiv, 2018] [Chen, et al., arXiv, 2018]

Page 18: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

UnsupervisedSpeechRecognition

• PhonemerecognitionAudio:TIMITText:WMT

supervised

WGAN-GP

Gumbel-softmax

Page 19: Generative Adversarial Network (GAN) and its Applications ...aliensunmin.github.io/aii_workshop/2nd/slides/2.pdftutorial-generative -adversarial-network-and-its-applications-to-signal-processing-and-natural-language-processing

summarydocument

SpeakerA SpeakerB

UnsupervisedLearningbyGAN

Itisgood.It’sagoodday.

Iloveyou.

Itisbad.It’sabadday.

Idon’t loveyou.

Audio Text


Lec: Generative Adversarial Networks

Lec: Generative Adversarial Networks

Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

Generative Adversarial Nets - Semantic Scholargenerative adversarial text to image synthesis. Scott Reed, ZeynepAkata. ... Conditional Sequence Generative Adversarial Nets[J]. arXiv

Generative adversarial nets

Generative adversarial nets

Generative Adversarial Networks (GANs) - Sciencesconf.org · Generative Adversarial Networks (GANs) Anirban Mukhopadhyay TU Darmstadt, Germany. Introduction Generative vs. Discriminative

Generative Adversarial Networks (GANs) - Sciencesconf.org · Generative Adversarial Networks (GANs) Anirban Mukhopadhyay TU Darmstadt, Germany. Introduction Generative vs. Discriminative

3D Generative Adversarial Networks inference ...

3D Generative Adversarial Networks inference ...

Generative Adversarial Networks (GANs)

Generative Adversarial Networks (GANs)

GENERATIVE ADVERSARIAL NETWORKS - Deep Learningdeeplearning.cs.cmu.edu/F20/document/recitation/GANS-ppt.pdf · 2021. 3. 22. · Generative Adversarial Networks Generative Models We

GENERATIVE ADVERSARIAL NETWORKS - Deep Learningdeeplearning.cs.cmu.edu/F20/document/recitation/GANS-ppt.pdf · 2021. 3. 22. · Generative Adversarial Networks Generative Models We

Generative Adversarial Networksbhiksha/courses/deeplearning/Spring.2018/... · 2018. 3. 29. · Generative Adversarial Networks Generative adversarial networks (GANs) are relatively

Generative Adversarial Networksbhiksha/courses/deeplearning/Spring.2018/... · 2018. 3. 29. · Generative Adversarial Networks Generative adversarial networks (GANs) are relatively

ESRGAN: Enhanced Super-Resolution Generative Adversarial ...openaccess.thecvf.com/content_ECCVW_2018/papers/11133/Wang_ESRGAN... · ESRGAN: EnhancedSuper-Resolution Generative Adversarial

ESRGAN: Enhanced Super-Resolution Generative Adversarial ...openaccess.thecvf.com/content_ECCVW_2018/papers/11133/Wang_ESRGAN... · ESRGAN: EnhancedSuper-Resolution Generative Adversarial

Semi-Supervised Generative Adversarial Hashing for Image Retrievalopenaccess.thecvf.com › ...Wang_Semi-Supervised_Generative_Adver… · Semi-Supervised Generative Adversarial Hashing

Semi-Supervised Generative Adversarial Hashing for Image Retrievalopenaccess.thecvf.com › ...Wang_Semi-Supervised_Generative_Adver… · Semi-Supervised Generative Adversarial Hashing

Lecture 19: Generative Adversarial Networksrgrosse/courses/csc321_2018/readings/L19 GANs.pdfLecture 19: Generative Adversarial Networks Roger Grosse 1 Introduction Generative modeling

Lecture 19: Generative Adversarial Networksrgrosse/courses/csc321_2018/readings/L19 GANs.pdfLecture 19: Generative Adversarial Networks Roger Grosse 1 Introduction Generative modeling

Adversarial Distillation of Bayesian Neural Network Posteriorsproceedings.mlr.press/v80/wang18i/wang18i.pdf · 2.3. Generative Adversarial Networks Generative Adversarial Networks

Adversarial Distillation of Bayesian Neural Network Posteriorsproceedings.mlr.press/v80/wang18i/wang18i.pdf · 2.3. Generative Adversarial Networks Generative Adversarial Networks

Generative Adversarial Transformers

Generative Adversarial Transformers

Generative Adversarial Networks - HAW Hamburgubicomp/projekte/master... · Generative Adversarial Networks ... Outline 1. Basic idea 2. Generative Adversarial ... com/2017/01/generative-adversarial-networks-hot-topic-machine-learning

Generative Adversarial Networks - HAW Hamburgubicomp/projekte/master... · Generative Adversarial Networks ... Outline 1. Basic idea 2. Generative Adversarial ... com/2017/01/generative-adversarial-networks-hot-topic-machine-learning

Generative Adversarial Nets - courses.cs.washington.edu

Generative Adversarial Nets - courses.cs.washington.edu

Generative Adversarial Perturbations - arXiv

Generative Adversarial Perturbations - arXiv

Image Representation Learning with Generative Adversarial ... · Image Representation Learning with Generative Adversarial Networks Keywords machine learning, generative models, image

Image Representation Learning with Generative Adversarial ... · Image Representation Learning with Generative Adversarial Networks Keywords machine learning, generative models, image

Generative Adversarial Imitation Learning

Generative Adversarial Imitation Learning

Generative Adversarial Networks (GANs) - Emanuele Sansone · List of recent papers on Generative Adversarial Networks: PAPERs about GANs 20142016 Generative Adversarial Networks ICLR

Generative Adversarial Networks (GANs) - Emanuele Sansone · List of recent papers on Generative Adversarial Networks: PAPERs about GANs 20142016 Generative Adversarial Networks ICLR

Geometry-Consistent Generative Adversarial Networks for ...openaccess.thecvf.com/content_CVPR_2019/papers/Fu_Geometry-Consistent... · Geometry-Consistent Generative Adversarial Networks

Geometry-Consistent Generative Adversarial Networks for ...openaccess.thecvf.com/content_CVPR_2019/papers/Fu_Geometry-Consistent... · Geometry-Consistent Generative Adversarial Networks