Deep Learning for Natural Language Processing Introduction ...
23
Deep Learning for Natural Language Processing Introduction to Generation Tasks Richard Johansson [email protected]
Transcript of Deep Learning for Natural Language Processing Introduction ...
Deep Learning for Natural LanguageProcessing
Introduction to Generation Tasks
Richard Johansson
-20pt
this module: generation tasks
I our remaining lectures will be focused ongeneration tasks
I translation, summarization, imagecaptioning, . . .
I we’ll start with an introduction to thistype of task
-20pt
definition
I conditional generation: given an input, generate a text
I the input can be of different types
-20pt
machine translation
-20pt
machine translation
-20pt
machine translation
-20pt
summarization, headline generation
example by Narayan et al. (2019)
-20pt
normalization and error correctionI historical → modernI unstandardized → standardizedI ungrammatical → grammatical
example by Bollmann et al. (2018)
-20pt
email auto-responses
example by Kannan et al. (2016)
-20pt
generation from a structure (“data-to-text”)
-20pt
generation from a structure: examples
(Konstas et al., 2017) (Gehrmann et al., 2018)
-20pt
image captioning
example by Vinyals et al. (2015)
-20pt
neural generation models: quick intro
-20pt
neural generation models: quick intro
-20pt
neural generation models: quick intro
-20pt
neural generation models: quick intro
-20pt
neural generation models: quick intro
-20pt
neural generation models: quick intro
-20pt
neural generation models: quick intro
-20pt
neural generation models: quick intro
-20pt
neural generation models: quick intro
-20pt
this lecture block
I evaluation of generation systemsI introduction to machine translationI neural models for machine translation
-20pt
references
M. Bollmann, A. Søgaard, and J. Bingel. 2018. Multi-task learning forhistorical text normalization: Size matters. In Proceedings of the Workshopon Deep Learning Approaches for Low-Resource NLP.
S. Gehrmann, F. Dai, H. Elder, and A. Rush. 2018. End-to-end content andplan selection for data-to-text generation. In INLG .
A. Kannan, K. Kurach, and S. Ravi et al. 2016. Smart reply: Automatedresponse suggestion for email. In KDD.
I. Konstas, S. Iyer, M. Yatskar, Y. Choi, and L. Zettlemoyer. 2017. NeuralAMR: Sequence-to-sequence models for parsing and generation. In ACL.
S. Narayan, S. Cohen, and M. Lapata. 2019. What is this article about?Extreme summarization with topic-aware convolutional neural networks.JAIR 66.
O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. 2015. Show and tell: A neuralimage caption generator. In CVPR.
