CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning...

Post on 25-Aug-2020

6 views 0 download

Transcript of CS11-747 Neural Networks for NLP Unsupervised and Semi ... · Learning Features vs. Learning...

CS11-747 Neural Networks for NLP Unsupervised and Semi-supervised

Learning of StructureGraham Neubig

Sitehttps://phontron.com/class/nn4nlp2019/

Supervised, Unsupervised, Semi-supervised

• Most models handled here are supervised learning

• Model P(Y|X), at training time given both

• Sometimes we are interested in unsupervised learning

• Model P(Y|X), at training time given only X

• Or semi-supervised learning

• Model P(Y|X), at training time given both or only X

Learning Features vs. Learning Structure

Learning Features vs. Learning Discrete Structure

• Learning features, e.g. word/sentence embeddings:

this is an example

• Learning discrete structure:

• Why discrete structure? • We may want to model information flow differently • More interpretable than features?

this is an example this is an examplethis is an example this is an example

Unsupervised Feature Learning (Review)

• When learning embeddings, we have an objective and use the intermediate states of this objective

• CBOW

• Skip-gram

• Sentence-level auto-encoder

• Skip-thought vectors

• Variational auto-encoder

How do we Use Learned Features?

• To solve tasks directly (Mikolov et al. 2013)

• And by proxy, knowledge base completion, etc., to be covered in a few classes

• To initialize downstream models

What About Discrete Structure?

• We can cluster words

• We can cluster words in context (POS/NER)

• We can learn structure

What is our Objective?• Basically, a generative model of the data X

• Sometimes factorized P(X|Y)P(Y), a traditional generative model

• Sometimes factorized P(X|Y)P(Y|X), an auto-encoder

• This can be made mathematically correct through variational autoencoder P(X|Y)Q(Y|X)

Clustering Words in Context

A Simple First Attempt• Train word embeddings

• Perform k-means clustering on them

• Implemented in word2vec (-classes option)

• But what if we want single words to appear in different classes (same surface form, different values)

Hidden Markov Models• Factored model of P(X|Y)P(Y) • State→state transition probabilities • State→word emission probabilities

<s> JJ NN NN LRB NN RRB … </s>

Natural Language Processing ( NLP ) …

PE(Natural|JJ) * PE(Language|JJ) * PE(Processing|JJ) * …

PT(JJ|<s>) * PT(NN|JJ) * PT(NN|NN) * PT(NN|LRB) * …

Unsupervised Hidden Markov Models

• Change label states to unlabeled numbers

0 13 17 17 6 12 6 … 0

Natural Language Processing ( NLP ) …

PE(Natural|13) * PE(Language|17) * PE(Processing|17) * …

PT(13|0) * PT(17|13) * PT(17|17) * PT(6|17) * …

• Can be trained with forward-backward algorithm

Hidden Markov Models w/ Gaussian Emissions

• Instead of parameterizing each state with a categorical distribution, we can use a Gaussian (or Gaussian mixture)!

• Long the defacto-standard for speech • Applied to POS tagging by training to emit word embeddings by

Lin et al. (2015)

0 13 17 17 6 12 6 … 0

A Simple Approximation: State Clustering (Giles et al. 1992)

• Simply train an RNN according to a standard loss function (e.g. language model)

• Then cluster the hidden states according to k-means, etc.

Featurized Hidden Markov Models (Tran et al. 2016)

• Calculate the transition/emission probabilities with neural networks! • Emission: Calculate representation of each word in vocabulary w/

CNN, dot product with tag representation and softmax to calculate emission prob

• Transition Matrix: Calculate w/ LSTMs (breaks Markov assumption)

Problem: Embeddings May Not be Indicative of Syntax

(He et al. 2018)

noun proper

verb base

cardinal number

verb gerund

adjective

noun singular

noun plural

adverb

verb past tenseverb past participleverb 3rd singular

Learning POS Taggersw/ Latent Embeddings

(He et al. 2018)

z1<latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit><latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit><latexit sha1_base64="jbtr+W9vpRfg2saI7YImOblkNik=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcqMCOqu6MZlRccW2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7GN9m8/USVZjJ6gElMfYGHEQsZwWCo++e+26/WnIaTl70M3ALUUFGtfvW3N5AkETQCwrHWXdeJwU+xAkY4nVZ6iaYxJmM8pF0DIyyo9tPc6tQ+MczADqUyJwI7Z/9vpFhogWFklFnTc7OMASm5rhsVjETWsmfyu56IoB6IeiZSOtQLRiC89FMWxQnQiMx8hAm3QdpZLPaAKUqATwzARDHzFZuMsMIETHgVk5G7mMgy8M4aVw3n7rzWvC7CKqMjdIxOkYsuUBPdohbyEEFD9Ire0Lv1Yn1Yn9bXTFqyip1DNFfW9x/uTZqw</latexit>

z2<latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit><latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit><latexit sha1_base64="ZNKuRe23lzCJMNesh6cRgweKXJI=">AAACCHicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3Lio4W2qFk0kwbmkyG5I5Qh36BuNXvcCVu/Qs/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DBy0TRahHJJeqE2BNOYuoBww47cSKYhFw+hiMr7P54xNVmsnoHiYx9QUeRixkBIOh7p77zX615jScvOxl4Baghopq96u/vYEkiaAREI617rpODH6KFTDC6bTSSzSNMRnjIe0aGGFBtZ/mVqf2iWEGdiiVORHYOft/I8VCCwwjo8yanptlDEjJdd2oYCSylj2T3/VEBPVA1DOR0qFeMALhhZ+yKE6ARmTmI0y4DdLOYrEHTFECfGIAJoqZr9hkhBUmYMKrmIzcxUSWgddsXDac27Na66oIq4yO0DE6RS46Ry10g9rIQwQN0St6Q+/Wi/VhfVpfM2nJKnYO0VxZ33/v75qx</latexit>

z3<latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit><latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit><latexit sha1_base64="VwYPdoeVjXkUU912kwmjdymaS+E=">AAACCHicZVBLTsMwFHT4lvIrsGQTUSGxqKoEkIBdBRuWRRBaqY0qx3Vaq3Yc2S9IJeoJEFs4ByvElltwDG6Ak2ZB2ydZbzxvnjWeIOZMg+P8WEvLK6tr66WN8ubW9s5uZW//UctEEeoRyaVqB1hTziLqAQNO27GiWASctoLRTTZvPVGlmYweYBxTX+BBxEJGMBjq/rl31qtUnbqTl70I3AJUUVHNXuW325ckETQCwrHWHdeJwU+xAkY4nZS7iaYxJiM8oB0DIyyo9tPc6sQ+NkzfDqUyJwI7Z/9vpFhogWFolFnTM7OMASm5rhkVDEXWsmfyux6LoBaIWiZSOtRzRiC89FMWxQnQiEx9hAm3QdpZLHafKUqAjw3ARDHzFZsMscIETHhlk5E7n8gi8E7rV3Xn7rzauC7CKqFDdIROkIsuUAPdoibyEEED9Ire0Lv1Yn1Yn9bXVLpkFTsHaKas7z/xkZqy</latexit>

x3<latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit>

x2<latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit>

x1<latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit>

e1<latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit><latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit><latexit sha1_base64="WgitCLnES1r4TeJXeVDWd7cTvpM=">AAACDXicZVDLSgMxFM3UV62vqks3wSK4KGVGBHVXdOOygmMLbSmZNNOGJpMhuSOUof8gbvU7XIlbv8HP8A/MTGdh2wvhnpx7bjg5QSy4Adf9cUpr6xubW+Xtys7u3v5B9fDoyahEU+ZTJZTuBMQwwSPmAwfBOrFmRAaCtYPJXTZvPzNtuIoeYRqzviSjiIecErBUpxfIlM0G3qBacxtuXngVeAWooaJag+pvb6hoIlkEVBBjup4bQz8lGjgVbFbpJYbFhE7IiHUtjIhkpp/mfmf4zDJDHCptTwQ4Z/9vpEQaSWBslVkzC7OMAaWEqVsVjGXWsmfyu5nKoB7IeibSJjRLRiC87qc8ihNgEZ37CBOBQeEsGzzkmlEQUwsI1dx+BdMx0YSCTbBiM/KWE1kF/kXjpuE+XNaat0VYZXSCTtE58tAVaqJ71EI+okigV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAMuJzw</latexit>

e2<latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit>

e3<latexit sha1_base64="LgiizHRmhHOKEEbwqVR4tWBS3lw=">AAACDXicZVDLTgIxFL2DL8QXauLGzURi4oKQQRfqjujGJSQiJECwUzrQ0E4n7R0TMuEfjFv9DlfGrd/gxn/wD+wAC4GbNPf03HOb0+NHghv0vG8ns7K6tr6R3cxtbe/s7uX3Dx6MijVldaqE0k2fGCZ4yOrIUbBmpBmRvmANf3ibzhtPTBuuwnscRawjST/kAacELdVs+zJh4+5FN1/wSt6k3GVQnoFC5aj28wgA1W7+t91TNJYsRCqIMa2yF2EnIRo5FWyca8eGRYQOSZ+1LAyJZKaTTPyO3VPL9NxAaXtCdCfs/42ESCMJDqwybWZuljKolDBFq8KBTFv6zORuRtIv+rKYirQJzIIRDK46CQ+jGFlIpz6CWLio3DQbt8c1oyhGFhCquf2KSwdEE4o2wZzNqLyYyDKon5euS17NZnUD08rCMZzAGZThEipwB1WoAwUBL/AKb86z8+58OJ9TacaZ7RzCXDlff9YqntM=</latexit><latexit sha1_base64="RMNkU799kfoHwXHS6vi9XiRQ2tw=">AAACDXicZVDLSgMxFM3UV62vquDGzWARXJQy1YW6K3XjsgXHFtqhZNJMG5pMhuSOUIb+g7jVX3DrSlwJfoMb/8E/MDPtwrYXwj0599xwcvyIMw2O823lVlbX1jfym4Wt7Z3dveL+wb2WsSLUJZJL1faxppyF1AUGnLYjRbHwOW35o5t03nqgSjMZ3sE4op7Ag5AFjGAwVLvri4ROehe9YsmpOFnZy6A6A6XaUfOHvdY/G73ib7cvSSxoCIRjrTtVJwIvwQoY4XRS6MaaRpiM8IB2DAyxoNpLMr8T+9QwfTuQypwQ7Iz9v5FgoQWGoVGmTc/NUgak5LpsVDAUaUufye56LPyyL8qpSOlALxiB4MpLWBjFQEMy9RHE3AZpp9nYfaYoAT42ABPFzFdsMsQKEzAJFkxG1cVEloF7XrmuOE2TVR1NK4+O0Qk6Q1V0iWroFjWQiwji6Ak9oxfr0Xqz3q2PqTRnzXYO0VxZX39bgqCP</latexit><latexit sha1_base64="RMNkU799kfoHwXHS6vi9XiRQ2tw=">AAACDXicZVDLSgMxFM3UV62vquDGzWARXJQy1YW6K3XjsgXHFtqhZNJMG5pMhuSOUIb+g7jVX3DrSlwJfoMb/8E/MDPtwrYXwj0599xwcvyIMw2O823lVlbX1jfym4Wt7Z3dveL+wb2WsSLUJZJL1faxppyF1AUGnLYjRbHwOW35o5t03nqgSjMZ3sE4op7Ag5AFjGAwVLvri4ROehe9YsmpOFnZy6A6A6XaUfOHvdY/G73ib7cvSSxoCIRjrTtVJwIvwQoY4XRS6MaaRpiM8IB2DAyxoNpLMr8T+9QwfTuQypwQ7Iz9v5FgoQWGoVGmTc/NUgak5LpsVDAUaUufye56LPyyL8qpSOlALxiB4MpLWBjFQEMy9RHE3AZpp9nYfaYoAT42ABPFzFdsMsQKEzAJFkxG1cVEloF7XrmuOE2TVR1NK4+O0Qk6Q1V0iWroFjWQiwji6Ak9oxfr0Xqz3q2PqTRnzXYO0VxZX39bgqCP</latexit>

f�(e)<latexit sha1_base64="8oxgnabmEamXCM6ELWdhWTrrt8c=">AAACHHicZVA9SwNBEJ3z2/gVFSubwyhECOFio3ZBG0sFzwgmhL3NnlncvT1254Rw3G8RW+2tbazEVvBnWNi7d0lh4sAyb9+8Wd6+IBbcoOd9OVPTM7Nz8wuLpaXlldW18vrGlVGJpsynSih9HRDDBI+YjxwFu441IzIQrBXcnebz1j3ThqvoEgcx60hyG/GQU4KW6pa3wm7aDmTajvs8y6o5ZNl+t1zx6l5R7n/QGIFKc/fn5RUAzrvl73ZP0USyCKkgxtw0vBg7KdHIqWBZqZ0YFhN6R27ZjYURkcx00sJ+5u5ZpueGStsToVuwfzdSIo0k2LfKvJmxWc6gUsLUrAr7Mm/5M8XdDGRQC2QtF2kTmgkjGB51Uh7FCbKIDn2EiXBRuXlUbo9rRlEMLCBUc/sVl/aJJhRtoCWbUWMykf/AP6gf170Lm9UJDGsBtmEHqtCAQ2jCGZyDDxRSeIQneHYenDfn3fkYSqec0c4mjJXz+QtiiKWs</latexit><latexit sha1_base64="JbCPJ2fHAGZa8Ebw6DTj/vvFjWE=">AAACHHicZVDLSgMxFM34rK2PqrhyM1iFCqXMuFF3RTcuK1hbaEvJpJk2NJkMyZ1CGeZbxK3u/QFxJW4FP8OFrs20Xdj2Qrgn554bTo4XcqbBcb6speWV1bX1zEY2t7m1vZPf3bvXMlKE1ojkUjU8rClnAa0BA04boaJYeJzWvcF1Oq8PqdJMBncwCmlb4F7AfEYwGKqTP/A7ccsTcSvssyQpppAmp518wSk747IXgTsFhcrxz8vrMPdb7eS/W11JIkEDIBxr3XSdENoxVsAIp0m2FWkaYjLAPdo0MMCC6nY8tp/YJ4bp2r5U5gRgj9n/GzEWWmDoG2Xa9MwsZUBKrktGBX2RtvSZ8V2PhFfyRCkVKe3rOSPgX7RjFoQR0IBMfPgRt0HaaVR2lylKgI8MwEQx8xWb9LHCBEygWZORO5/IIqidlS/Lzq3J6gpNKoMO0REqIhedowq6QVVUQwTF6BE9oWfrwXqz3q2PiXTJmu7so5myPv8Ah3SnJg==</latexit><latexit sha1_base64="JbCPJ2fHAGZa8Ebw6DTj/vvFjWE=">AAACHHicZVDLSgMxFM34rK2PqrhyM1iFCqXMuFF3RTcuK1hbaEvJpJk2NJkMyZ1CGeZbxK3u/QFxJW4FP8OFrs20Xdj2Qrgn554bTo4XcqbBcb6speWV1bX1zEY2t7m1vZPf3bvXMlKE1ojkUjU8rClnAa0BA04boaJYeJzWvcF1Oq8PqdJMBncwCmlb4F7AfEYwGKqTP/A7ccsTcSvssyQpppAmp518wSk747IXgTsFhcrxz8vrMPdb7eS/W11JIkEDIBxr3XSdENoxVsAIp0m2FWkaYjLAPdo0MMCC6nY8tp/YJ4bp2r5U5gRgj9n/GzEWWmDoG2Xa9MwsZUBKrktGBX2RtvSZ8V2PhFfyRCkVKe3rOSPgX7RjFoQR0IBMfPgRt0HaaVR2lylKgI8MwEQx8xWb9LHCBEygWZORO5/IIqidlS/Lzq3J6gpNKoMO0REqIhedowq6QVVUQwTF6BE9oWfrwXqz3q2PiXTJmu7so5myPv8Ah3SnJg==</latexit>

ei ⇠ N (µzi ,⌃zi)<latexit sha1_base64="8CLMr+o0dZvNiJKf88rFwAP//jQ=">AAACQ3icZVBNa9tAEB0laZO4H3GaYy9LQiGlxki5JIUcQnvJKaS0bgyWEKv1yl6yqxW7o4Ir9F9Kfkzptb31P+TS0Guhu7IPiT2wzJs3b5aZl5VSWAzD38Ha+sajx5tb250nT5893+nuvvhsdWUYHzAttRlm1HIpCj5AgZIPS8OpyiS/yq7f+/7VF26s0MUnnJU8UXRSiFwwio5Ku6dxpmrepILEVigSK4pTRmV90Rz6TqyqJq2/pqLpkbb+KCaKLqjXafcg7IdtkFUQLcDB2X785hsAXKbdP/FYs0rxApmk1o6isMSkpgYFk7zpxJXlJWXXdMJHDhZUcZvU7ZUNeeWYMcm1ca9A0rL3J2qqrF/fKX2yD3qeQa2l7TkVTpVP/pu2tjOV9TLV8yJjc7u0COYnSS2KskJesPkeeSUJauIdJWNhOEM5c4AyI9wphE2poQyd7x3nUbTsyCoYHPXf9sMPzqt3MI8teAn7cAgRHMMZnMMlDIDBDfyAn/Ar+B7cBnfB37l0LVjM7MGDCP79BwcMtOc=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit>

zi ⇠<latexit sha1_base64="A0vAn9BPw9kDXHS9rkuB77rpTAE=">AAACDnicZVDLTgIxFO3gC/GFunTTSExcEDJjTNQd0Y1LTEQwMCGd0oGGdjpp75jghI8wbvU7XBm3/oKf4R/YgVkI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKWtSJZRuB8QwwSPWBA6CtWPNiAwEawWjm2zeemLacBXdwzhmviSDiIecErDU43OP467hEvfKFbfmTgsvAy8HFZRXo1f+7fYVTSSLgApiTMdzY/BTooFTwSalbmJYTOiIDFjHwohIZvx0aniCTyzTx6HS9kSAp+z/jZRIIwkMrTJrZm6WMaCUMFWrgqHMWvbM9G7GMqgGspqJtAnNghEIL/2UR3ECLKIzH2EiMCichYP7XDMKYmwBoZrbr2A6JJpQsBGWbEbeYiLLoHlWu6q5d+eV+nUeVhEdoWN0ijx0geroFjVQE1Ek0St6Q+/Oi/PhfDpfM2nByXcO0Vw5339MaJ0J</latexit><latexit sha1_base64="A0vAn9BPw9kDXHS9rkuB77rpTAE=">AAACDnicZVDLTgIxFO3gC/GFunTTSExcEDJjTNQd0Y1LTEQwMCGd0oGGdjpp75jghI8wbvU7XBm3/oKf4R/YgVkI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKWtSJZRuB8QwwSPWBA6CtWPNiAwEawWjm2zeemLacBXdwzhmviSDiIecErDU43OP467hEvfKFbfmTgsvAy8HFZRXo1f+7fYVTSSLgApiTMdzY/BTooFTwSalbmJYTOiIDFjHwohIZvx0aniCTyzTx6HS9kSAp+z/jZRIIwkMrTJrZm6WMaCUMFWrgqHMWvbM9G7GMqgGspqJtAnNghEIL/2UR3ECLKIzH2EiMCichYP7XDMKYmwBoZrbr2A6JJpQsBGWbEbeYiLLoHlWu6q5d+eV+nUeVhEdoWN0ijx0geroFjVQE1Ek0St6Q+/Oi/PhfDpfM2nByXcO0Vw5339MaJ0J</latexit><latexit sha1_base64="A0vAn9BPw9kDXHS9rkuB77rpTAE=">AAACDnicZVDLTgIxFO3gC/GFunTTSExcEDJjTNQd0Y1LTEQwMCGd0oGGdjpp75jghI8wbvU7XBm3/oKf4R/YgVkI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKWtSJZRuB8QwwSPWBA6CtWPNiAwEawWjm2zeemLacBXdwzhmviSDiIecErDU43OP467hEvfKFbfmTgsvAy8HFZRXo1f+7fYVTSSLgApiTMdzY/BTooFTwSalbmJYTOiIDFjHwohIZvx0aniCTyzTx6HS9kSAp+z/jZRIIwkMrTJrZm6WMaCUMFWrgqHMWvbM9G7GMqgGspqJtAnNghEIL/2UR3ECLKIzH2EiMCichYP7XDMKYmwBoZrbr2A6JJpQsBGWbEbeYiLLoHlWu6q5d+eV+nUeVhEdoWN0ijx0geroFjVQE1Ek0St6Q+/Oi/PhfDpfM2nByXcO0Vw5339MaJ0J</latexit>

Markov Structure

xi ⇠<latexit sha1_base64="4XmkaafD4EWeU6wrjePkY8qE6No=">AAACE3icZVDLTgIxFO3gC/GFunTTSExcEDIYE3VHdOMSE0dIGCSd0oGGdjpp7xjJhN8wbvU7XBm3foCf4R/YGVgI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKfOoEkq3A2KY4BHzgINg7VgzIgPBWsHoJpu3npg2XEX3MI5ZV5JBxENOCVjq0Q9k+jzpcewbLnGvXHFrbl54GdRnoIJm1eyVf/2+oolkEVBBjOnU3Ri6KdHAqWCTkp8YFhM6IgPWsTAikplumrue4BPL9HGotD0R4Jz9v5ESaSSBoVVmzczNMgaUEqZqVTCUWcueye9mLINqIKuZSJvQLBiB8LKb8ihOgEV06iNMBAaFs4Rwn2tGQYwtIFRz+xVMh0QTCjbHks2ovpjIMvDOalc19+680riehVVER+gYnaI6ukANdIuayEMUafSK3tC78+J8OJ/O11RacGY7h2iunO8/lwWfXA==</latexit><latexit sha1_base64="4XmkaafD4EWeU6wrjePkY8qE6No=">AAACE3icZVDLTgIxFO3gC/GFunTTSExcEDIYE3VHdOMSE0dIGCSd0oGGdjpp7xjJhN8wbvU7XBm3foCf4R/YGVgI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKfOoEkq3A2KY4BHzgINg7VgzIgPBWsHoJpu3npg2XEX3MI5ZV5JBxENOCVjq0Q9k+jzpcewbLnGvXHFrbl54GdRnoIJm1eyVf/2+oolkEVBBjOnU3Ri6KdHAqWCTkp8YFhM6IgPWsTAikplumrue4BPL9HGotD0R4Jz9v5ESaSSBoVVmzczNMgaUEqZqVTCUWcueye9mLINqIKuZSJvQLBiB8LKb8ihOgEV06iNMBAaFs4Rwn2tGQYwtIFRz+xVMh0QTCjbHks2ovpjIMvDOalc19+680riehVVER+gYnaI6ukANdIuayEMUafSK3tC78+J8OJ/O11RacGY7h2iunO8/lwWfXA==</latexit><latexit sha1_base64="4XmkaafD4EWeU6wrjePkY8qE6No=">AAACE3icZVDLTgIxFO3gC/GFunTTSExcEDIYE3VHdOMSE0dIGCSd0oGGdjpp7xjJhN8wbvU7XBm3foCf4R/YGVgI3KS5p+ee25yeIBbcgOv+OIWV1bX1jeJmaWt7Z3evvH/wYFSiKfOoEkq3A2KY4BHzgINg7VgzIgPBWsHoJpu3npg2XEX3MI5ZV5JBxENOCVjq0Q9k+jzpcewbLnGvXHFrbl54GdRnoIJm1eyVf/2+oolkEVBBjOnU3Ri6KdHAqWCTkp8YFhM6IgPWsTAikplumrue4BPL9HGotD0R4Jz9v5ESaSSBoVVmzczNMgaUEqZqVTCUWcueye9mLINqIKuZSJvQLBiB8LKb8ihOgEV06iNMBAaFs4Rwn2tGQYwtIFRz+xVMh0QTCjbHks2ovpjIMvDOalc19+680riehVVER+gYnaI6ukANdIuayEMUafSK3tC78+J8OJ/O11RacGY7h2iunO8/lwWfXA==</latexit>

f�(ei)<latexit sha1_base64="ISFMuiBWVv5rfvJnqq9uM8Osfvo=">AAACHnicZVBNS8NAEN34WetX1YMHL8EiVCglEUG9Fb14rGBsoS1hs920S3ezYXcilJAfI171d3gSr/oz/Adu0h5sO7DM2zdvhpkXxJxpcJwfa2V1bX1js7RV3t7Z3duvHBw+aZkoQj0iuVSdAGvKWUQ9YMBpJ1YUi4DTdjC+y+vtZ6o0k9EjTGLaF3gYsZARDIbyK8ehn/YCkfbiEcuyWg5p5rNzv1J1Gk4R9jJwZ6CKZtHyK7+9gSSJoBEQjrXuuk4M/RQrYITTrNxLNI0xGeMh7RoYYUF1Py0OyOwzwwzsUCrzIrAL9n9HioUWGEZGmSc9V8sZkJLrulHBSOQpH1P89UQE9UDUc5HSoV5YBMLrfsqiOAEakekeYcJtkHZulj1gihLgEwMwUcycYpMRVpiAsbRsPHIXHVkG3kXjpuE8XFabtzOzSugEnaIactEVaqJ71EIeIihDr+gNvVsv1of1aX1NpSvWrOcIzYX1/QfnyqO9</latexit><latexit sha1_base64="ISFMuiBWVv5rfvJnqq9uM8Osfvo=">AAACHnicZVBNS8NAEN34WetX1YMHL8EiVCglEUG9Fb14rGBsoS1hs920S3ezYXcilJAfI171d3gSr/oz/Adu0h5sO7DM2zdvhpkXxJxpcJwfa2V1bX1js7RV3t7Z3duvHBw+aZkoQj0iuVSdAGvKWUQ9YMBpJ1YUi4DTdjC+y+vtZ6o0k9EjTGLaF3gYsZARDIbyK8ehn/YCkfbiEcuyWg5p5rNzv1J1Gk4R9jJwZ6CKZtHyK7+9gSSJoBEQjrXuuk4M/RQrYITTrNxLNI0xGeMh7RoYYUF1Py0OyOwzwwzsUCrzIrAL9n9HioUWGEZGmSc9V8sZkJLrulHBSOQpH1P89UQE9UDUc5HSoV5YBMLrfsqiOAEakekeYcJtkHZulj1gihLgEwMwUcycYpMRVpiAsbRsPHIXHVkG3kXjpuE8XFabtzOzSugEnaIactEVaqJ71EIeIihDr+gNvVsv1of1aX1NpSvWrOcIzYX1/QfnyqO9</latexit><latexit sha1_base64="ISFMuiBWVv5rfvJnqq9uM8Osfvo=">AAACHnicZVBNS8NAEN34WetX1YMHL8EiVCglEUG9Fb14rGBsoS1hs920S3ezYXcilJAfI171d3gSr/oz/Adu0h5sO7DM2zdvhpkXxJxpcJwfa2V1bX1js7RV3t7Z3duvHBw+aZkoQj0iuVSdAGvKWUQ9YMBpJ1YUi4DTdjC+y+vtZ6o0k9EjTGLaF3gYsZARDIbyK8ehn/YCkfbiEcuyWg5p5rNzv1J1Gk4R9jJwZ6CKZtHyK7+9gSSJoBEQjrXuuk4M/RQrYITTrNxLNI0xGeMh7RoYYUF1Py0OyOwzwwzsUCrzIrAL9n9HioUWGEZGmSc9V8sZkJLrulHBSOQpH1P89UQE9UDUc5HSoV5YBMLrfsqiOAEakekeYcJtkHZulj1gihLgEwMwUcycYpMRVpiAsbRsPHIXHVkG3kXjpuE8XFabtzOzSugEnaIactEVaqJ71EIeIihDr+gNvVsv1of1aX1NpSvWrOcIzYX1/QfnyqO9</latexit>

Point mass at

xi = f�(ei)<latexit sha1_base64="270VJLD7gt0vH7AtNf4cVJpk6VA=">AAACKXicZVC7TisxEJ3lTXgFKGksHhJIUbShgVtcKYKGEiQCSCRaeR0vsbDXK3v26kar7fkYRAmUfAMV0PIFFPTYCQWPkaw5c+aMNXPiTAqLYfgcjIyOjU9MTk1XZmbn5heqi0snVueG8RbTUpuzmFouRcpbKFDys8xwqmLJT+PLfd8//ceNFTo9xn7GO4pepCIRjKKjoupqO1bF/zIS5C9JosJX7awnynLTQ+4aW1F1LayHgyC/QeMTrDXX328fAOAwqr61u5rliqfIJLX2vBFm2CmoQcEkLyvt3PKMskt6wc8dTKnitlMMbinJhmO6JNHGvRTJgP06UVBlFcWeU/pkv/U8g1pLW3Mq7Cmf/DeD2vZVXItVzYuMTeyPRTDZ7RQizXLkKRvukeSSoCbeN9IVhjOUfQcoM8KdQliPGsrQuVtxHjV+OvIbtLbrf+rhkfNqD4YxBSuwCpvQgB1owgEcQgsYXMEN3MF9cB08Bk/By1A6EnzOLMO3CF4/AGOkqtY=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit>

NeuralProjector

A Simpler Method: Map Directly to Space of Prior

Example of Markov prior

xf(e;�)

<latexit sha1_base64="FsElmiZYiYnGj5o7UHjArdk6Bk8=">AAACUXicZVBNT9tAEB2blo9AIWmPvViNkIIURU6E1KJeUHvpkUoEEDiK1psxWbEf1u4YiCz/i17pr+LCX+HEOsmBwEirefvmzertS3MpHMXxUxCuffi4vrG51dje+bS712x9PnOmsByH3EhjL1LmUAqNQxIk8SK3yFQq8Ty9+V3Pz2/ROmH0Kc1yHCl2rUUmOCNPXWadBH8m+VQcjJvtuBfPK3oP+kvQhmWdjFvBIJkYXijUxCVz7qof5zQqmSXBJVaNpHCYM37DrvHKQ80UulE5t1xF+56ZRJmx/miK5uzrjZIppxhNvbJubmVWM2SMdF2voqmqW/3M/O5mKu2mqluLrMvcGyOU/RiVQucFoeYLH1khIzJRHU80ERY5yZkHjFvhvxLxKbOMkw9xxUSqqkYj0XjHjVJMT8rkviqTVJX3VbXK44JHz/uU+28zfQ+Gg95RL/572D7+tYx7E77CN+hAH77DMfyBExgCBw3/4AH+B4/BcwhhuJCGwXLnC6xUuP0C/9e00g==</latexit><latexit sha1_base64="FsElmiZYiYnGj5o7UHjArdk6Bk8=">AAACUXicZVBNT9tAEB2blo9AIWmPvViNkIIURU6E1KJeUHvpkUoEEDiK1psxWbEf1u4YiCz/i17pr+LCX+HEOsmBwEirefvmzertS3MpHMXxUxCuffi4vrG51dje+bS712x9PnOmsByH3EhjL1LmUAqNQxIk8SK3yFQq8Ty9+V3Pz2/ROmH0Kc1yHCl2rUUmOCNPXWadBH8m+VQcjJvtuBfPK3oP+kvQhmWdjFvBIJkYXijUxCVz7qof5zQqmSXBJVaNpHCYM37DrvHKQ80UulE5t1xF+56ZRJmx/miK5uzrjZIppxhNvbJubmVWM2SMdF2voqmqW/3M/O5mKu2mqluLrMvcGyOU/RiVQucFoeYLH1khIzJRHU80ERY5yZkHjFvhvxLxKbOMkw9xxUSqqkYj0XjHjVJMT8rkviqTVJX3VbXK44JHz/uU+28zfQ+Gg95RL/572D7+tYx7E77CN+hAH77DMfyBExgCBw3/4AH+B4/BcwhhuJCGwXLnC6xUuP0C/9e00g==</latexit><latexit sha1_base64="FsElmiZYiYnGj5o7UHjArdk6Bk8=">AAACUXicZVBNT9tAEB2blo9AIWmPvViNkIIURU6E1KJeUHvpkUoEEDiK1psxWbEf1u4YiCz/i17pr+LCX+HEOsmBwEirefvmzertS3MpHMXxUxCuffi4vrG51dje+bS712x9PnOmsByH3EhjL1LmUAqNQxIk8SK3yFQq8Ty9+V3Pz2/ROmH0Kc1yHCl2rUUmOCNPXWadBH8m+VQcjJvtuBfPK3oP+kvQhmWdjFvBIJkYXijUxCVz7qof5zQqmSXBJVaNpHCYM37DrvHKQ80UulE5t1xF+56ZRJmx/miK5uzrjZIppxhNvbJubmVWM2SMdF2voqmqW/3M/O5mKu2mqluLrMvcGyOU/RiVQucFoeYLH1khIzJRHU80ERY5yZkHjFvhvxLxKbOMkw9xxUSqqkYj0XjHjVJMT8rkviqTVJX3VbXK44JHz/uU+28zfQ+Gg95RL/572D7+tYx7E77CN+hAH77DMfyBExgCBw3/4AH+B4/BcwhhuJCGwXLnC6xUuP0C/9e00g==</latexit>

?

max log pGHMM(f�(x))<latexit sha1_base64="QrUGXd8IoThzpkp7A2ek+zngVOU=">AAACcnicZVFLaxsxEJa3r9R9OekxF7WmYIMx61BoegvpobkEUqibQGSMVp61RfRYpNl0jdjf0F/Ta/s78kNyr9bZQ50MiPnmmwczn7JCSY9petNJHj1+8vTZzvPui5evXr/p7e798LZ0AqbCKusuMu5BSQNTlKjgonDAdabgPLv60uTPr8F5ac13XBcw03xpZC4Fx0jNe0OmeUUZZcouaTEPDKHC8PXk9LSuB3mMi5WsB6waDue9fjpON0YfgkkL+qS1s/lu54AtrCg1GBSKe385SQucBe5QCgV1l5UeCi6u+BIuIzRcg5+FzU01/RCZBc2ti88g3bD/dwSuvea4ipWN81u5hkFrlR/FKlzpxjVjNrFf62yU6VFT5Hzu7y2C+eEsSFOUCEbc7ZGXiqKljX50IR0IVOsIuHAynkLFijsuMKq8tUSm626XGfgprNbcLAKr6sAyHaq63ubhjofIR5Un9zV9CKYH48/j9NvH/tFxK/cO2SfvyYBMyCdyRE7IGZkSQX6R3+QP+du5TfaTd0n7N0mn7XlLtiwZ/QPP2MC/</latexit><latexit sha1_base64="QrUGXd8IoThzpkp7A2ek+zngVOU=">AAACcnicZVFLaxsxEJa3r9R9OekxF7WmYIMx61BoegvpobkEUqibQGSMVp61RfRYpNl0jdjf0F/Ta/s78kNyr9bZQ50MiPnmmwczn7JCSY9petNJHj1+8vTZzvPui5evXr/p7e798LZ0AqbCKusuMu5BSQNTlKjgonDAdabgPLv60uTPr8F5ac13XBcw03xpZC4Fx0jNe0OmeUUZZcouaTEPDKHC8PXk9LSuB3mMi5WsB6waDue9fjpON0YfgkkL+qS1s/lu54AtrCg1GBSKe385SQucBe5QCgV1l5UeCi6u+BIuIzRcg5+FzU01/RCZBc2ti88g3bD/dwSuvea4ipWN81u5hkFrlR/FKlzpxjVjNrFf62yU6VFT5Hzu7y2C+eEsSFOUCEbc7ZGXiqKljX50IR0IVOsIuHAynkLFijsuMKq8tUSm626XGfgprNbcLAKr6sAyHaq63ubhjofIR5Un9zV9CKYH48/j9NvH/tFxK/cO2SfvyYBMyCdyRE7IGZkSQX6R3+QP+du5TfaTd0n7N0mn7XlLtiwZ/QPP2MC/</latexit><latexit sha1_base64="QrUGXd8IoThzpkp7A2ek+zngVOU=">AAACcnicZVFLaxsxEJa3r9R9OekxF7WmYIMx61BoegvpobkEUqibQGSMVp61RfRYpNl0jdjf0F/Ta/s78kNyr9bZQ50MiPnmmwczn7JCSY9petNJHj1+8vTZzvPui5evXr/p7e798LZ0AqbCKusuMu5BSQNTlKjgonDAdabgPLv60uTPr8F5ac13XBcw03xpZC4Fx0jNe0OmeUUZZcouaTEPDKHC8PXk9LSuB3mMi5WsB6waDue9fjpON0YfgkkL+qS1s/lu54AtrCg1GBSKe385SQucBe5QCgV1l5UeCi6u+BIuIzRcg5+FzU01/RCZBc2ti88g3bD/dwSuvea4ipWN81u5hkFrlR/FKlzpxjVjNrFf62yU6VFT5Hzu7y2C+eEsSFOUCEbc7ZGXiqKljX50IR0IVOsIuHAynkLFijsuMKq8tUSm626XGfgprNbcLAKr6sAyHaq63ubhjofIR5Un9zV9CKYH48/j9NvH/tFxK/cO2SfvyYBMyCdyRE7IGZkSQX6R3+QP+du5TfaTd0n7N0mn7XlLtiwZ/QPP2MC/</latexit>

Information Loss

Normalizing Flow (Rezende and Mohamed 2015)

• Basic idea: a way to map from one probability distribution to another in an invertible manner

ei ⇠ N (µzi ,⌃zi)<latexit sha1_base64="8CLMr+o0dZvNiJKf88rFwAP//jQ=">AAACQ3icZVBNa9tAEB0laZO4H3GaYy9LQiGlxki5JIUcQnvJKaS0bgyWEKv1yl6yqxW7o4Ir9F9Kfkzptb31P+TS0Guhu7IPiT2wzJs3b5aZl5VSWAzD38Ha+sajx5tb250nT5893+nuvvhsdWUYHzAttRlm1HIpCj5AgZIPS8OpyiS/yq7f+/7VF26s0MUnnJU8UXRSiFwwio5Ku6dxpmrepILEVigSK4pTRmV90Rz6TqyqJq2/pqLpkbb+KCaKLqjXafcg7IdtkFUQLcDB2X785hsAXKbdP/FYs0rxApmk1o6isMSkpgYFk7zpxJXlJWXXdMJHDhZUcZvU7ZUNeeWYMcm1ca9A0rL3J2qqrF/fKX2yD3qeQa2l7TkVTpVP/pu2tjOV9TLV8yJjc7u0COYnSS2KskJesPkeeSUJauIdJWNhOEM5c4AyI9wphE2poQyd7x3nUbTsyCoYHPXf9sMPzqt3MI8teAn7cAgRHMMZnMMlDIDBDfyAn/Ar+B7cBnfB37l0LVjM7MGDCP79BwcMtOc=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit><latexit sha1_base64="N3CijJIqFBowULdbrh+WNrhPgX8=">AAACQ3icZVBNSwMxEM36bf2qevQSFEGxlK0XFTwUvXgSRauCW5Zsmm2DyWZJZoW67L/wF/hjxKve/A9eFK+CybYHPwbCvHnzJsy8KBXcgO+/eiOjY+MTk1PTlZnZufmF6uLShVGZpqxFlVD6KiKGCZ6wFnAQ7CrVjMhIsMvo5tD1L2+ZNlwl59BPWVuSbsJjTglYKqzuB5HMWRFyHBgucSAJ9CgR+XGx4TqBzIowvwt5UcNlfca7kgypzbC65tf9MvB/0BiCteZqsHX/2uyfhNX3oKNoJlkCVBBjrht+Cu2caOBUsKISZIalhN6QLru2MCGSmXZeXlngdct0cKy0fQngkv05kRNp3PpW6ZL51XMMKCVMzaqgJ11y35S16cuoFsmaE2kTmz+LQLzbznmSZsASOtgjzgQGhZ2juMM1oyD6FhCquT0F0x7RhIL1vWI9avx15D9obdf36v6p9eoADWIKraBVtIEaaAc10RE6QS1E0QN6Qs/oxXv03rwP73MgHfGGM8voV3hf3z4qtm0=</latexit>

xi = f�(ei)<latexit sha1_base64="270VJLD7gt0vH7AtNf4cVJpk6VA=">AAACKXicZVC7TisxEJ3lTXgFKGksHhJIUbShgVtcKYKGEiQCSCRaeR0vsbDXK3v26kar7fkYRAmUfAMV0PIFFPTYCQWPkaw5c+aMNXPiTAqLYfgcjIyOjU9MTk1XZmbn5heqi0snVueG8RbTUpuzmFouRcpbKFDys8xwqmLJT+PLfd8//ceNFTo9xn7GO4pepCIRjKKjoupqO1bF/zIS5C9JosJX7awnynLTQ+4aW1F1LayHgyC/QeMTrDXX328fAOAwqr61u5rliqfIJLX2vBFm2CmoQcEkLyvt3PKMskt6wc8dTKnitlMMbinJhmO6JNHGvRTJgP06UVBlFcWeU/pkv/U8g1pLW3Mq7Cmf/DeD2vZVXItVzYuMTeyPRTDZ7RQizXLkKRvukeSSoCbeN9IVhjOUfQcoM8KdQliPGsrQuVtxHjV+OvIbtLbrf+rhkfNqD4YxBSuwCpvQgB1owgEcQgsYXMEN3MF9cB08Bk/By1A6EnzOLMO3CF4/AGOkqtY=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit><latexit sha1_base64="PuNsspGjpcc2SeVmkDWfr5ab2W0=">AAACKXicZVBNSwMxEM36bf2qevQS/IAKpWy9qAeh6MWjgrUFW5ZsmrXBZLMks8Wy7N0fIx7Vu//Ak3r1F3jQs0nrQetAmDdv3oSZFyaCG/D9V29sfGJyanpmtjA3v7C4VFxeOTcq1ZTVqRJKN0NimOAxqwMHwZqJZkSGgjXCqyPXb/SYNlzFZ9BPWFuSy5hHnBKwVFBcb4Uyu84Djg9wFGSuaiVdnuclB5ltbAfFDb/iDwL/B9UfsFHb/Lx/6s19nQTFj1ZH0VSyGKggxlxU/QTaGdHAqWB5oZUalhB6RS7ZhYUxkcy0s8EtOd6yTAdHStsXAx6wvycyIo0k0LVKl8yfnmNAKWHKVgVd6ZL7ZlCbvgzLoSw7kTaRGVkEor12xuMkBRbT4R5RKjAo7HzDHa4ZBdG3gFDN7SmYdokmFKy7BetRddSR/6C+U9mv+KfWq0M0jBm0htZRCVXRLqqhY3SC6oiiG3SHHtCjd+s9ey/e21A65v3MrKI/4b1/A4iQrFA=</latexit>

NeuralProjector

xf�1(e;�)

<latexit sha1_base64="0TvUX8MzRC5izm1cmvW9PX4J8iQ=">AAACWXicZVBNSyNBEO2M7hrHdY169NIYBBdimJGFVbyIXjwqGBWcbOjp1JjG/hi6a9QwzD/xqv9J8MfYk+RgtKCp169eNa9fmkvhMIreGsHC4o+fS83lcOXX6u+11vrGlTOF5dDjRhp7kzIHUmjooUAJN7kFplIJ1+n9aT2/fgDrhNGXOM6hr9idFpngDD01aLWy/+VeXO0mcESTfCT+DFrtqBtNin4H8Qy0yazOB+uN/WRoeKFAI5fMuds4yrFfMouCS6jCpHCQM37P7uDWQ80UuH45sV7RHc8MaWasPxrphP28UTLlFMORV9bNzc1qBo2RruNVOFJ1q5+Z3N1YpZ1UdWqRdZn7YgSzg34pdF4gaD71kRWSoqF1THQoLHCUYw8Yt8J/hfIRs4yjD3PORKqqMEw0PHKjFNPDMnmqyiRV5VNVzfMw5cHzPuX4a6bfQW+/e9iNLv62j09mcTfJFtkmuyQm/8gxOSPnpEc4eSDP5IW8Nt6DIGgG4VQaNGY7m2Sugs0PzFK2FA==</latexit><latexit sha1_base64="0TvUX8MzRC5izm1cmvW9PX4J8iQ=">AAACWXicZVBNSyNBEO2M7hrHdY169NIYBBdimJGFVbyIXjwqGBWcbOjp1JjG/hi6a9QwzD/xqv9J8MfYk+RgtKCp169eNa9fmkvhMIreGsHC4o+fS83lcOXX6u+11vrGlTOF5dDjRhp7kzIHUmjooUAJN7kFplIJ1+n9aT2/fgDrhNGXOM6hr9idFpngDD01aLWy/+VeXO0mcESTfCT+DFrtqBtNin4H8Qy0yazOB+uN/WRoeKFAI5fMuds4yrFfMouCS6jCpHCQM37P7uDWQ80UuH45sV7RHc8MaWasPxrphP28UTLlFMORV9bNzc1qBo2RruNVOFJ1q5+Z3N1YpZ1UdWqRdZn7YgSzg34pdF4gaD71kRWSoqF1THQoLHCUYw8Yt8J/hfIRs4yjD3PORKqqMEw0PHKjFNPDMnmqyiRV5VNVzfMw5cHzPuX4a6bfQW+/e9iNLv62j09mcTfJFtkmuyQm/8gxOSPnpEc4eSDP5IW8Nt6DIGgG4VQaNGY7m2Sugs0PzFK2FA==</latexit><latexit sha1_base64="0TvUX8MzRC5izm1cmvW9PX4J8iQ=">AAACWXicZVBNSyNBEO2M7hrHdY169NIYBBdimJGFVbyIXjwqGBWcbOjp1JjG/hi6a9QwzD/xqv9J8MfYk+RgtKCp169eNa9fmkvhMIreGsHC4o+fS83lcOXX6u+11vrGlTOF5dDjRhp7kzIHUmjooUAJN7kFplIJ1+n9aT2/fgDrhNGXOM6hr9idFpngDD01aLWy/+VeXO0mcESTfCT+DFrtqBtNin4H8Qy0yazOB+uN/WRoeKFAI5fMuds4yrFfMouCS6jCpHCQM37P7uDWQ80UuH45sV7RHc8MaWasPxrphP28UTLlFMORV9bNzc1qBo2RruNVOFJ1q5+Z3N1YpZ1UdWqRdZn7YgSzg34pdF4gaD71kRWSoqF1THQoLHCUYw8Yt8J/hfIRs4yjD3PORKqqMEw0PHKjFNPDMnmqyiRV5VNVzfMw5cHzPuX4a6bfQW+/e9iNLv62j09mcTfJFtkmuyQm/8gxOSPnpEc4eSDP5IW8Nt6DIGgG4VQaNGY7m2Sugs0PzFK2FA==</latexit>

• Advantage: optimizing probability can be shown to be possible by following equation

�1<latexit sha1_base64="whzOwvpgac/k7KeUY3/DRCrw/7o=">AAACTnicdVBNbxMxEPUGSkMotIUjF4sIiUOIdrdR096qcuHYSg2p1I0qrzPbWPHHyp4tjVb7I7jCr+qRX8IJtXYSJILESNY8v3ljPb+8lMJhHP+MWk+ebj3bbj/vvNh5+Wp3b//1F2cqy2HEjTT2MmcOpNAwQoESLksLTOUSxvn8U5iPb8E6YfQFLkqYKHajRSE4Q0+NP2ZCF7i43uvG/eOjw3RwSON+HA+TNAkgHQ4OBjTxTKguWdfZ9X6UZlPDKwUauWTOXSVxiZOaWRRcQtPJKgcl43N2A1ceaqbATeql34a+98yUFsb6o5Eu2b83aqacYjjzytDcxiwwaIx0Pa/CmQotPLO8u4XKe7nqBZF1hfvHCBZHk1roskLQfOWjqCRFQ0M2dCoscJQLDxi3wn+F8hmzjKNPcMNErppOJ9PwlRulmJ7W2V1TZ7mq75pmk4cVD573Kf+Jkv4fjNL+cT8+H3RPTtdxt8lb8o58IAkZkhPymZyREeFkTr6R7+RHdB/9in5HDytpK1rvvCEb1Wo/AhvGtlc=</latexit><latexit sha1_base64="whzOwvpgac/k7KeUY3/DRCrw/7o=">AAACTnicdVBNbxMxEPUGSkMotIUjF4sIiUOIdrdR096qcuHYSg2p1I0qrzPbWPHHyp4tjVb7I7jCr+qRX8IJtXYSJILESNY8v3ljPb+8lMJhHP+MWk+ebj3bbj/vvNh5+Wp3b//1F2cqy2HEjTT2MmcOpNAwQoESLksLTOUSxvn8U5iPb8E6YfQFLkqYKHajRSE4Q0+NP2ZCF7i43uvG/eOjw3RwSON+HA+TNAkgHQ4OBjTxTKguWdfZ9X6UZlPDKwUauWTOXSVxiZOaWRRcQtPJKgcl43N2A1ceaqbATeql34a+98yUFsb6o5Eu2b83aqacYjjzytDcxiwwaIx0Pa/CmQotPLO8u4XKe7nqBZF1hfvHCBZHk1roskLQfOWjqCRFQ0M2dCoscJQLDxi3wn+F8hmzjKNPcMNErppOJ9PwlRulmJ7W2V1TZ7mq75pmk4cVD573Kf+Jkv4fjNL+cT8+H3RPTtdxt8lb8o58IAkZkhPymZyREeFkTr6R7+RHdB/9in5HDytpK1rvvCEb1Wo/AhvGtlc=</latexit><latexit sha1_base64="whzOwvpgac/k7KeUY3/DRCrw/7o=">AAACTnicdVBNbxMxEPUGSkMotIUjF4sIiUOIdrdR096qcuHYSg2p1I0qrzPbWPHHyp4tjVb7I7jCr+qRX8IJtXYSJILESNY8v3ljPb+8lMJhHP+MWk+ebj3bbj/vvNh5+Wp3b//1F2cqy2HEjTT2MmcOpNAwQoESLksLTOUSxvn8U5iPb8E6YfQFLkqYKHajRSE4Q0+NP2ZCF7i43uvG/eOjw3RwSON+HA+TNAkgHQ4OBjTxTKguWdfZ9X6UZlPDKwUauWTOXSVxiZOaWRRcQtPJKgcl43N2A1ceaqbATeql34a+98yUFsb6o5Eu2b83aqacYjjzytDcxiwwaIx0Pa/CmQotPLO8u4XKe7nqBZF1hfvHCBZHk1roskLQfOWjqCRFQ0M2dCoscJQLDxi3wn+F8hmzjKNPcMNErppOJ9PwlRulmJ7W2V1TZ7mq75pmk4cVD573Kf+Jkv4fjNL+cT8+H3RPTtdxt8lb8o58IAkZkhPymZyREeFkTr6R7+RHdB/9in5HDytpK1rvvCEb1Wo/AhvGtlc=</latexit>

+X

log���det

@f�1�

@xi

���<latexit sha1_base64="SkZYIwUJm36ifl+bt7GkjNGFpcg=">AAAClHicdZFbixMxFMfTWS9rvXX1wQdfgkUQrGVmLNvdB2HZRRAEWcG6C5s6ZDKZNmwuQ3JmbRnzofw04pt+EzNtRSt4IOTkdy6c809eSeEgjr93op1r12/c3L3VvX3n7r37vb0HH52pLeMTZqSx5zl1XArNJyBA8vPKcqpyyc/yy5M2fnbFrRNGf4BlxaeKzrQoBaMQUNZ7i59j4mpFpJmRYzH7QoAvoCk4eFJayhpSUQuCSlx+al4kPgtgLrz/w8kiE35VmvX68fDwYD8d7eN4GMfjJE1aJx2PXo5wEkhrfbSx02yvk5LCsFpxDUxS5y6SuIJp03ZmkvsuqR2vKLukM34RXE0Vd9NmtbXHTwMpcGlsOBrwiv5d0VDlFIV5yGwvtxVrCRgj3SBkwVy1V9tm9XZLlQ9yNWiTrCvdP4NAeTBthK5q4Jqt5yhricHgVmFcCMsZyGVwKLMirILZnAYtIfzD1hC58t0u0fwzM0pRXTRkEYTNVbPwfpvzNeeBB5V/S4n/70zS4eEwfj/qHx1v5N5Fj9ET9AwlaIyO0Bt0iiaIoa/oG/qBfkaPolfRSfR6nRp1NjUP0ZZF734BeF7PyQ==</latexit><latexit sha1_base64="SkZYIwUJm36ifl+bt7GkjNGFpcg=">AAAClHicdZFbixMxFMfTWS9rvXX1wQdfgkUQrGVmLNvdB2HZRRAEWcG6C5s6ZDKZNmwuQ3JmbRnzofw04pt+EzNtRSt4IOTkdy6c809eSeEgjr93op1r12/c3L3VvX3n7r37vb0HH52pLeMTZqSx5zl1XArNJyBA8vPKcqpyyc/yy5M2fnbFrRNGf4BlxaeKzrQoBaMQUNZ7i59j4mpFpJmRYzH7QoAvoCk4eFJayhpSUQuCSlx+al4kPgtgLrz/w8kiE35VmvX68fDwYD8d7eN4GMfjJE1aJx2PXo5wEkhrfbSx02yvk5LCsFpxDUxS5y6SuIJp03ZmkvsuqR2vKLukM34RXE0Vd9NmtbXHTwMpcGlsOBrwiv5d0VDlFIV5yGwvtxVrCRgj3SBkwVy1V9tm9XZLlQ9yNWiTrCvdP4NAeTBthK5q4Jqt5yhricHgVmFcCMsZyGVwKLMirILZnAYtIfzD1hC58t0u0fwzM0pRXTRkEYTNVbPwfpvzNeeBB5V/S4n/70zS4eEwfj/qHx1v5N5Fj9ET9AwlaIyO0Bt0iiaIoa/oG/qBfkaPolfRSfR6nRp1NjUP0ZZF734BeF7PyQ==</latexit><latexit sha1_base64="SkZYIwUJm36ifl+bt7GkjNGFpcg=">AAAClHicdZFbixMxFMfTWS9rvXX1wQdfgkUQrGVmLNvdB2HZRRAEWcG6C5s6ZDKZNmwuQ3JmbRnzofw04pt+EzNtRSt4IOTkdy6c809eSeEgjr93op1r12/c3L3VvX3n7r37vb0HH52pLeMTZqSx5zl1XArNJyBA8vPKcqpyyc/yy5M2fnbFrRNGf4BlxaeKzrQoBaMQUNZ7i59j4mpFpJmRYzH7QoAvoCk4eFJayhpSUQuCSlx+al4kPgtgLrz/w8kiE35VmvX68fDwYD8d7eN4GMfjJE1aJx2PXo5wEkhrfbSx02yvk5LCsFpxDUxS5y6SuIJp03ZmkvsuqR2vKLukM34RXE0Vd9NmtbXHTwMpcGlsOBrwiv5d0VDlFIV5yGwvtxVrCRgj3SBkwVy1V9tm9XZLlQ9yNWiTrCvdP4NAeTBthK5q4Jqt5yhricHgVmFcCMsZyGVwKLMirILZnAYtIfzD1hC58t0u0fwzM0pRXTRkEYTNVbPwfpvzNeeBB5V/S4n/70zS4eEwfj/qHx1v5N5Fj9ET9AwlaIyO0Bt0iiaIoa/oG/qBfkaPolfRSfR6nRp1NjUP0ZZF734BeF7PyQ==</latexit>

when is not invertiblef<latexit sha1_base64="mEEGp2ktLOyduAzcmi2vdTnECNI=">AAACSHicdVBNbxMxEPUGWkqgpaVHLhYREoco2l2iprlVcOGYSIRE6kaV15ltrPpjZc9CotX+Aq7wq/gF/AxOFTfsJEgEiZGseX7zxnp+eSmFwzj+EbUePDw4fHT0uP3k6fHJs9Oz5x+dqSyHCTfS2FnOHEihYYICJcxKC0zlEqb53bswn34C64TRH3BdwlyxWy0KwRl6alzcnHbi3vDyIu1f0LgXx4MkTQJIB/03fZp4JlSH7Gp0cxal2cLwSoFGLplz10lc4rxmFgWX0LSzykHJ+B27hWsPNVPg5vXGaUNfeWZBC2P90Ug37N8bNVNOMVx6ZWhubxYYNEa6rlfhUoUWntnc3Vrl3Vx1g8i6wv1jBIvLeS10WSFovvVRVJKioSEVuhAWOMq1B4xb4b9C+ZJZxtFnt2ciV027nWn4zI1STC/qbNXUWa7qVdPs87DlwfM+5T9R0v+DSdob9uJxv3P1dhf3EXlBXpLXJCEDckXekxGZEE6AfCFfybfoe/Qzuo9+baWtaLdzTvaq1foN2xqzzg==</latexit><latexit sha1_base64="mEEGp2ktLOyduAzcmi2vdTnECNI=">AAACSHicdVBNbxMxEPUGWkqgpaVHLhYREoco2l2iprlVcOGYSIRE6kaV15ltrPpjZc9CotX+Aq7wq/gF/AxOFTfsJEgEiZGseX7zxnp+eSmFwzj+EbUePDw4fHT0uP3k6fHJs9Oz5x+dqSyHCTfS2FnOHEihYYICJcxKC0zlEqb53bswn34C64TRH3BdwlyxWy0KwRl6alzcnHbi3vDyIu1f0LgXx4MkTQJIB/03fZp4JlSH7Gp0cxal2cLwSoFGLplz10lc4rxmFgWX0LSzykHJ+B27hWsPNVPg5vXGaUNfeWZBC2P90Ug37N8bNVNOMVx6ZWhubxYYNEa6rlfhUoUWntnc3Vrl3Vx1g8i6wv1jBIvLeS10WSFovvVRVJKioSEVuhAWOMq1B4xb4b9C+ZJZxtFnt2ciV027nWn4zI1STC/qbNXUWa7qVdPs87DlwfM+5T9R0v+DSdob9uJxv3P1dhf3EXlBXpLXJCEDckXekxGZEE6AfCFfybfoe/Qzuo9+baWtaLdzTvaq1foN2xqzzg==</latexit><latexit sha1_base64="mEEGp2ktLOyduAzcmi2vdTnECNI=">AAACSHicdVBNbxMxEPUGWkqgpaVHLhYREoco2l2iprlVcOGYSIRE6kaV15ltrPpjZc9CotX+Aq7wq/gF/AxOFTfsJEgEiZGseX7zxnp+eSmFwzj+EbUePDw4fHT0uP3k6fHJs9Oz5x+dqSyHCTfS2FnOHEihYYICJcxKC0zlEqb53bswn34C64TRH3BdwlyxWy0KwRl6alzcnHbi3vDyIu1f0LgXx4MkTQJIB/03fZp4JlSH7Gp0cxal2cLwSoFGLplz10lc4rxmFgWX0LSzykHJ+B27hWsPNVPg5vXGaUNfeWZBC2P90Ug37N8bNVNOMVx6ZWhubxYYNEa6rlfhUoUWntnc3Vrl3Vx1g8i6wv1jBIvLeS10WSFovvVRVJKioSEVuhAWOMq1B4xb4b9C+ZJZxtFnt2ciV027nWn4zI1STC/qbNXUWa7qVdPs87DlwfM+5T9R0v+DSdob9uJxv3P1dhf3EXlBXpLXJCEDckXekxGZEE6AfCFfybfoe/Qzuo9+baWtaLdzTvaq1foN2xqzzg==</latexit>

log p(x) = log pGHMM(f�1

� (x))<latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="jBAzhNAJ26J1i9dTzZ4SO1PLZqY=">AAACOXicZVHLTsMwEHTKq4Ty6JlLBELiUFVpL8ANiQtHkAhUIhVynA216kdkb6BVlB/gylfxCXwFJ8QVp+2BlpWsHc/uWuPZJBfcYhh+eo219Y3Nrea2v9Pyd/f2D1r3VheGQcS00GaQUAuCK4iQo4BBboDKRMBDMr6q6w8vYCzX6g6nOQwlfVY844yio26eDo7DbjiL4D/oLcAxWcRT2+vHqWaFBIVMUGsfe2GOw5Ia5ExA5ceFhZyyMX2GRwcVlWCH5UxnFZw4Jg0ybdxRGMzYvxMllVZSHLnOOtmlWs2g1sJ2XBeOZJ3qZ2Z3O5VJJ5GdusnYzK4Iwex8WHKVFwiKzXVkhQhQB7UnQcoNMBRTBygz3H0lYCNqKEPn3JKIRFa+Hyt4ZVpKqtIynlRlnMhyUlXLPMx5cLwzubdq6X8Q9bsX3fA2JE1ySI7IKemRM3JJrskNiQgjKXkj796H9+V9z3fR8BZLaZOl8H5+AY4YsXk=</latexit><latexit sha1_base64="Ku8E5xMhx2yjUIS3NBNvbWk1Bus=">AAACcnicZVFNbxMxEJ0sXyUUmnLlYlGBUilEmwoJOCAhcaCXSkUitFI3RF5nNrHqj5U9C4ks/xF+DVf4B/wbvEkOpB3J8ps3z9bMm7JW0lOe/+1kd+7eu/9g72H30f7jJwe9w/2v3jZO4FhYZd1lyT0qaXBMkhRe1g65LhVelNcf2/rFd3ReWvOFVjVONJ8bWUnBKVHT3utC2Tmr+8XymL1nm2QaCsIlhU+nZ2cx9quU1wsZv4VXo9gKj6e9o3yYr4PdBqMtOIJtnE8POyfFzIpGoyGhuPdXo7ymSeCOpFAYu0Xjsebims/xKkHDNfpJWI8X2YvEzFhlXTqG2Jr9/0Xg2mtOi6RsL79TaxmyVvlBUtFCt1f7zTr3K10OSj1oRc5X/kYjVL2dBGnqhtCITR9VoxhZ1lrJZtKhILVKgAsn0yhMLLjjgpLhO02UOna7hcEfwmrNzSwUyxiKUodljLs8bnhMfHJ5dNPT22B8Mnw3zD/nsAfP4Dn0YQRv4AOcwjmMQcBP+AW/4U8G2ctssFlH1tnu5SnsRDb8B6n/wos=</latexit><latexit sha1_base64="Ku8E5xMhx2yjUIS3NBNvbWk1Bus=">AAACcnicZVFNbxMxEJ0sXyUUmnLlYlGBUilEmwoJOCAhcaCXSkUitFI3RF5nNrHqj5U9C4ks/xF+DVf4B/wbvEkOpB3J8ps3z9bMm7JW0lOe/+1kd+7eu/9g72H30f7jJwe9w/2v3jZO4FhYZd1lyT0qaXBMkhRe1g65LhVelNcf2/rFd3ReWvOFVjVONJ8bWUnBKVHT3utC2Tmr+8XymL1nm2QaCsIlhU+nZ2cx9quU1wsZv4VXo9gKj6e9o3yYr4PdBqMtOIJtnE8POyfFzIpGoyGhuPdXo7ymSeCOpFAYu0Xjsebims/xKkHDNfpJWI8X2YvEzFhlXTqG2Jr9/0Xg2mtOi6RsL79TaxmyVvlBUtFCt1f7zTr3K10OSj1oRc5X/kYjVL2dBGnqhtCITR9VoxhZ1lrJZtKhILVKgAsn0yhMLLjjgpLhO02UOna7hcEfwmrNzSwUyxiKUodljLs8bnhMfHJ5dNPT22B8Mnw3zD/nsAfP4Dn0YQRv4AOcwjmMQcBP+AW/4U8G2ctssFlH1tnu5SnsRDb8B6n/wos=</latexit><latexit sha1_base64="xChZ9DUVMtsn1KFXp2HSXCyWRic=">AAACfXicZVDbahsxEJW3t9S9Oc1jX0RNiwOu2Q2FtA+B0Dw0L4EU4iSQdY1WnrVFdFmk2cZG6Ef6NXlN/6B/U63thzoZEHPmzBkxc4pKCodp+reVPHr85OmzreftFy9fvX7T2X577kxtOQy5kcZeFsyBFBqGKFDCZWWBqULCRXF91PQvfoF1wugzXFQwUmyqRSk4w0iNO59zaaa06uXzXXpAV8XY5whz9N+PT05C6JWxrmYi/PSfstAId8edbjpIl0EfgmwNumQdp+Pt1l4+MbxWoJFL5txVllY48syi4BJCO68dVIxfsylcRaiZAjfyy/MC/RCZCS2NjU8jXbL/T3imnGI4i8omuY1ew6Ax0vWjCmeqSc03y9otVNEvVL8RWVe6e4tg+WXkha5qBM1Xe5S1pGhoYyWdCAsc5SICxq2Ip1A+Y5ZxjIZvLFGo0G7nGm64UYrpic/nweeF8vMQNnlY8RD56HJ239OHYLg3+DpIf6Tdw29ru7fIO/Ke9EhG9skhOSanZEg4+U1uyR35k5DkY9JPBitp0lrP7JCNSPb/ARdTw3A=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit><latexit sha1_base64="oq/g0iP5izTeEta3oHw43r5mrTc=">AAACfXicZVDbahsxEJW3t9S9xGkf+yJqWhxwzW4IpHkIhOaheQmkUDeBrGu08qwtossizSY2Qj/Sr+lr+gf9m2ptP9TJgJgzZ86ImVNUUjhM07+t5NHjJ0+fbT1vv3j56vV2Z+fND2dqy2HIjTT2smAOpNAwRIESLisLTBUSLorrk6Z/cQPWCaO/46KCkWJTLUrBGUZq3NnPpZnSqpfPd+kRXRVjnyPM0X89PTsLoVfGupqJ8NN/ykIj3B13uukgXQZ9CLI16JJ1nI93Wnv5xPBagUYumXNXWVrhyDOLgksI7bx2UDF+zaZwFaFmCtzIL88L9ENkJrQ0Nj6NdMn+P+GZcorhLCqb5DZ6DYPGSNePKpypJjXfLGu3UEW/UP1GZF3p7i2C5eeRF7qqETRf7VHWkqKhjZV0IixwlIsIGLcinkL5jFnGMRq+sUShQruda7jlRimmJz6fB58Xys9D2ORhxUPko8vZfU8fguHe4HCQftvvHn9Z271F3pH3pEcyckCOySk5J0PCyS/ym9yRPwlJPib9ZLCSJq31zFuyEcnBPxiTw3Q=</latexit>

Results of Learning w/ HMM (He et al. 2018)

noun proper

verb base

cardinal number

verb gerund

adjective

noun singular

noun plural

adverb

verb past tenseverb past participleverb 3rd singular

Unsupervised Phrase-structured Composition Functions

Soft vs. Hard Tree Structure• Soft tree structure: use a differentiable gating function

• Hard tree structure: non-differentiable, but allows for more complicated composition methods

x1 x2

x1,2

x3

x2,3

x1,30.2 0.8

Soft

x1 x2 x3

x2,3

x1,3Hard

One Other Paradigm: Weak Supervision

• Supervised: given X,Y to model P(Y|X)

• Unsupervised: given X to model P(Y|X)

• Weakly Supervised: given X and V to model P(Y|X), under assumption that Y and V are correlated

• Note: different from multi-task or transfer learning because we are given no Y

• Note: different from supervised learning with latent variables, because we care about Y, not V

• Can choose whether to use left node, right node, or combination of both

Gated Convolution (Cho et al. 2014)

• Trained using MT loss

Learning with RL (Yogatama et al. 2016)

• Intermediate tree-structured representation for language modeling

• Predict that tree using shift-reduce parsing, sentence representation composed in tree-structured manner

• Reinforcement learning with supervised loss, prediction loss

Learning w/ Layer-wise Reductions (Choi et al. 2017)

• Choose one parent at each layer, reducing size by one

• Train using Gumbel-straighthrough reparameterization trick

• Faster and more effective than RL?

• Williams et al. (2017) find that this gives less trivial trees as well

Difficulties in Learning Latent Structure

(Williams et al. 2018)• Unfortunately, many models learn trivial structure...

• e.g. balanced binary, left/right branching

• Why? One explanation: tension between (untrained) parser, and (untrained) downstream task model

Learning Dependencies

Phrase Structure vs. Dependency Structure

• Previous methods attempt to learn representations of phrases in tree-structured manner

• We might also want to learn dependencies, that tell which words depend on others

Dependency Model w/ Valence (Klein and Manning 2004)

• Basic idea: top-down dependency based language model that generates left and right sides, then stops

I saw a girl with a telescopeROOT• For both the right and left side, calculate whether to

continue generating words, and if yes generatee.g., a slightly simplified view for word “saw”Pd(<cont> | saw, ←, false) * Pw(I | saw, ←, false) *Pd(<stop> | saw, ←, true) * Pd(<cont> | saw, →, false) * Pw(girl | saw, ←, false) * Pd(<cont> | saw, →, true) * Pw(with | saw, ←, true) *Pd(<stop> | saw, ←, true)

Unsupervised Dependency Induction w/ Neural Nets (Jiang et al. 2016)

• Simple: parameterize the decision with neural nets instead of with count-based distributions

• Like DMV, train with EM algorithm

Invertible Projections for DMV (He et al. 2018)

z1<latexit sha1_base64="kbJpa6Sam6jLYh1k9CLbrRqm2Rk=">AAACMnicZVDLSsNAFJ34Nr5a3ekmWAQXpSQiqLuiG5eK1gqmlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOcpF1YvTDcM+eeO5w5UcKZAd9/d2Zm5+YXFpeW3ZXVtfWNSnXzxqhUE9oiiit9G2FDOZO0BQw4vU00xSLitB3dnxXz9gPVhil5DaOEdgTuSxYzgsFSV0/doFup+Q2/LO8/CCaghiZ10a0622FPkVRQCYRjY+4CP4FOhjUwwmnuhqmhCSb3uE/vLJRYUNPJSq+5t2eZnhcrbY8Er2R/b2RYGIFhYJVFM1OzggGluKlbFQxE0YpnyrsZiageiXoh0iY2f4xAfNzJmExSoJKMfcQp90B5RS5ej2lKgI8swEQz+xWPDLDGBGx6UyYikbtuKOkjUUJg2cvCYZ6FkciGee7aNIO/2f0HrYPGScO/PKw1TyexLqEdtIv2UYCOUBOdowvUQgT10TN6Qa/Om/PhfDpfY+mMM9nZQlPlfP8AG16q6w==</latexit><latexit sha1_base64="kbJpa6Sam6jLYh1k9CLbrRqm2Rk=">AAACMnicZVDLSsNAFJ34Nr5a3ekmWAQXpSQiqLuiG5eK1gqmlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOcpF1YvTDcM+eeO5w5UcKZAd9/d2Zm5+YXFpeW3ZXVtfWNSnXzxqhUE9oiiit9G2FDOZO0BQw4vU00xSLitB3dnxXz9gPVhil5DaOEdgTuSxYzgsFSV0/doFup+Q2/LO8/CCaghiZ10a0622FPkVRQCYRjY+4CP4FOhjUwwmnuhqmhCSb3uE/vLJRYUNPJSq+5t2eZnhcrbY8Er2R/b2RYGIFhYJVFM1OzggGluKlbFQxE0YpnyrsZiageiXoh0iY2f4xAfNzJmExSoJKMfcQp90B5RS5ej2lKgI8swEQz+xWPDLDGBGx6UyYikbtuKOkjUUJg2cvCYZ6FkciGee7aNIO/2f0HrYPGScO/PKw1TyexLqEdtIv2UYCOUBOdowvUQgT10TN6Qa/Om/PhfDpfY+mMM9nZQlPlfP8AG16q6w==</latexit><latexit sha1_base64="kbJpa6Sam6jLYh1k9CLbrRqm2Rk=">AAACMnicZVDLSsNAFJ34Nr5a3ekmWAQXpSQiqLuiG5eK1gqmlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOcpF1YvTDcM+eeO5w5UcKZAd9/d2Zm5+YXFpeW3ZXVtfWNSnXzxqhUE9oiiit9G2FDOZO0BQw4vU00xSLitB3dnxXz9gPVhil5DaOEdgTuSxYzgsFSV0/doFup+Q2/LO8/CCaghiZ10a0622FPkVRQCYRjY+4CP4FOhjUwwmnuhqmhCSb3uE/vLJRYUNPJSq+5t2eZnhcrbY8Er2R/b2RYGIFhYJVFM1OzggGluKlbFQxE0YpnyrsZiageiXoh0iY2f4xAfNzJmExSoJKMfcQp90B5RS5ej2lKgI8swEQz+xWPDLDGBGx6UyYikbtuKOkjUUJg2cvCYZ6FkciGee7aNIO/2f0HrYPGScO/PKw1TyexLqEdtIv2UYCOUBOdowvUQgT10TN6Qa/Om/PhfDpfY+mMM9nZQlPlfP8AG16q6w==</latexit>

z2<latexit sha1_base64="k9/8EMjjLfM8r2mBl7kTM6haUuo=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmRVB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzTr1bOqf31cbpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+AR0qquw=</latexit><latexit sha1_base64="k9/8EMjjLfM8r2mBl7kTM6haUuo=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmRVB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzTr1bOqf31cbpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+AR0qquw=</latexit><latexit sha1_base64="k9/8EMjjLfM8r2mBl7kTM6haUuo=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmRVB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzTr1bOqf31cbpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+AR0qquw=</latexit>

?

z3<latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="bAUTGLr9EgIAz6GcyetjDEUesRE=">AAACIXicZVDLTsJAFJ3iCysqbHXTSExcEFLcqDsTNy4xESGhhEynU5gwj2bmViFNf8CtX+LXGBfGX3FaWAjcZHLPnHvu5MwJE84M+P63U9nZ3ds/qB66RzX3+OS0XnsxKtWE9ojiSg9CbChnkvaAAaeDRFMsQk774eyhmPdfqTZMyWdYJHQk8ESymBEMluqO602/7ZflbYPOCjTRqsYN5yyIFEkFlUA4NmbY8RMYZVgDI5zmbpAammAywxM6tFBiQc0oK33m3qVlIi9W2h4JXsn+38iwMALD1CqLZtZmBQNKcdOyKpiKohXPlHezEGErFK1CpE1sNoxAfDvKmExSoJIsfcQp90B5RSZexDQlwBcWYKKZ/YpHplhjAja5NROhyF03kPSNKCGwjLJgnmdBKLJ5nrs2zM5mdNugd92+a/tPPqqic3SBrlAH3aB79Ii6qIcIitA7+nA+nS/nZ5l5xVmF30Br5fz+AWoppzk=</latexit><latexit sha1_base64="cHNML6NFzNaVJV0shKLwGy2tbDA=">AAACJ3icZVDLTsMwENzwJpTnES4RFRKHqkrhANyQuHAEQWmlpqoc12kt/IjsDVCifAJX+BI+BnFC/AhO6YHSlawdz85a44lTwS2G4Yc3N7+wuLS8suqvVdY3Nre2K3dWZ4ayJtVCm3ZMLBNcsSZyFKydGkZkLFgrvr8o560HZizX6hZHKetKMlA84ZSgo26ee8e9rWpYD8cVzILGBFRhUle9bW836muaSaaQCmJtpxGm2M2JQU4FK/wosywl9J4MWMdBRSSz3XzstQgOHNMPEm3cURiM2b8bOZFWEhw6Zdns1KxkUGtha06FQ1m28pnx3Y5kXItlrRQZm9h/RjA57eZcpRkyRX99JJkIUAdlLkGfG0ZRjBwg1HD3lYAOiSEUXXpTJmJZ+H6k2CPVUhLVz6OnIo9imT8Vhe/SbPzPbhY0j+pn9fA6hBXYg304hAacwDlcwhU0gcIAXuAV3rx379P7+o19zpvkvwNT5X3/AMQVqVA=</latexit><latexit sha1_base64="cHNML6NFzNaVJV0shKLwGy2tbDA=">AAACJ3icZVDLTsMwENzwJpTnES4RFRKHqkrhANyQuHAEQWmlpqoc12kt/IjsDVCifAJX+BI+BnFC/AhO6YHSlawdz85a44lTwS2G4Yc3N7+wuLS8suqvVdY3Nre2K3dWZ4ayJtVCm3ZMLBNcsSZyFKydGkZkLFgrvr8o560HZizX6hZHKetKMlA84ZSgo26ee8e9rWpYD8cVzILGBFRhUle9bW836muaSaaQCmJtpxGm2M2JQU4FK/wosywl9J4MWMdBRSSz3XzstQgOHNMPEm3cURiM2b8bOZFWEhw6Zdns1KxkUGtha06FQ1m28pnx3Y5kXItlrRQZm9h/RjA57eZcpRkyRX99JJkIUAdlLkGfG0ZRjBwg1HD3lYAOiSEUXXpTJmJZ+H6k2CPVUhLVz6OnIo9imT8Vhe/SbPzPbhY0j+pn9fA6hBXYg304hAacwDlcwhU0gcIAXuAV3rx379P7+o19zpvkvwNT5X3/AMQVqVA=</latexit><latexit sha1_base64="RLvPvxg5oSd4xWwIdnMbsHCGlB0=">AAACMnicZVDLTgIxFO3gC8e37nQzkZi4IGTQhbojunGJUYSEIaRTOtDYx6S9o+BkPsGtfokfY1wZt36EHWAhcpPmnp57bnN6wpgzA77/4RQWFpeWV4qr7tr6xubW9s7uvVGJJrRBFFe6FWJDOZO0AQw4bcWaYhFy2gwfrvJ585Fqw5S8g1FMOwL3JYsYwWCp2+fuaXe75Ff8cXnzoDoFJTStenfH2Q96iiSCSiAcG9Ou+jF0UqyBEU4zN0gMjTF5wH3atlBiQU0nHXvNvCPL9LxIaXskeGP270aKhREYBlaZNzMzyxlQipuyVcFA5C1/Znw3IxGWQ1HORdpE5p8RiM47KZNxAlSSiY8o4R4oL8/F6zFNCfCRBZhoZr/ikQHWmIBNb8ZEKDLXDSR9IkoILHtpMMzSIBTpMMtcm2b1f3bzoHFSuaj4N36pdjmNtYgO0CE6RlV0hmroGtVRAxHURy/oFb05786n8+V8T6QFZ7qzh2bK+fkFHbaq6Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit><latexit sha1_base64="GoWTV2IhQuk8x82feQPyqOSmaP4=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSQAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx197vVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwzw9qrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fHvaq7Q==</latexit>

z4<latexit sha1_base64="qRJO9a4HbiuS4+vcPiAoCJbeVKg=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmUlB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzSPq2dV/7pebpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+ASDCqu4=</latexit><latexit sha1_base64="qRJO9a4HbiuS4+vcPiAoCJbeVKg=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmUlB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzSPq2dV/7pebpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+ASDCqu4=</latexit><latexit sha1_base64="qRJO9a4HbiuS4+vcPiAoCJbeVKg=">AAACMnicZVDLSgMxFM3UVx1fVXe6GSyCi1KmUlB3RTcuFa0tdErJpJk2NI8huaPWYT7BrX6JHyOuxK0fYaZ2YeuFcE/OPTecnDDmzIDvvzuFhcWl5ZXiqru2vrG5VdreuTMq0YQ2ieJKt0NsKGeSNoEBp+1YUyxCTlvh6CKft+6pNkzJWxjHtCvwQLKIEQyWunnq1Xulsl/1J+X9B7UpKKNpXfW2nb2gr0giqATCsTGdmh9DN8UaGOE0c4PE0BiTER7QjoUSC2q66cRr5h1apu9FStsjwZuwfzdSLIzAMLTKvJmZWc6AUtxUrAqGIm/5M5O7GYuwEopKLtImMnNGIDrtpkzGCVBJfn1ECfdAeXkuXp9pSoCPLcBEM/sVjwyxxgRsejMmQpG5biDpA1FCYNlPg8csDUKRPmaZa9OszWf3HzSPq2dV/7pebpxPYy2ifXSAjlANnaAGukRXqIkIGqBn9IJenTfnw/l0vn6lBWe6s4tmyvn+ASDCqu4=</latexit>

z5<latexit sha1_base64="7v8NYCmbh9REcawh1qKc3rzsbb8=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSBAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx1D7rVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwz/drrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fIo6q7w==</latexit><latexit sha1_base64="7v8NYCmbh9REcawh1qKc3rzsbb8=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSBAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx1D7rVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwz/drrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fIo6q7w==</latexit><latexit sha1_base64="7v8NYCmbh9REcawh1qKc3rzsbb8=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapSBAJuFVw4FkEBiUSV4zithR+RvQFKlE/gCl/CxyBOiCsfgdP2QGEla8ezs9Z4opQzA77/7kxNz8zOzVcW3MWl5ZXV6tr6lVGZJrRDFFf6JsKGciZpBxhwepNqikXE6XV0d1rOr++pNkzJSxikNBS4J1nCCAZLXTx1D7rVmt/wh+X9B80xqKFxtbtrzmYQK5IJKoFwbMxt008hzLEGRjgt3CAzNMXkDvforYUSC2rCfOi18HYsE3uJ0vZI8Ibs740cCyMw9K2ybGZiVjKgFDd1q4K+KFv5zPBuBiKqR6JeirRJzB8jkByFOZNpBlSSkY8k4x4or8zFi5mmBPjAAkw0s1/xSB9rTMCmN2EiEoXrBpI+ECUElnEePBZ5EIn8sShcm2bzb3b/QWevcdzwz/drrZNxrBW0hbbRLmqiQ9RCZ6iNOoigHnpGL+jVeXM+nE/naySdcsY7G2iinO8fIo6q7w==</latexit>

z6<latexit sha1_base64="pGOJAC+5XIxMYgPoGWdA3fbkKOc=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapShHjcKrhwLIICEokqx3FaCz8iewOUKJ/AFb6Ej0GcEFc+AqftgcJK1o5nZ63xRClnBnz/3Zmanpmdm68suItLyyur1bX1K6MyTWiHKK70TYQN5UzSDjDg9CbVFIuI0+vo7rScX99TbZiSlzBIaShwT7KEEQyWunjqHnSrNb/hD8v7D5pjUEPjanfXnM0gViQTVALh2Jjbpp9CmGMNjHBauEFmaIrJHe7RWwslFtSE+dBr4e1YJvYSpe2R4A3Z3xs5FkZg6Ftl2czErGRAKW7qVgV9UbbymeHdDERUj0S9FGmTmD9GIDkKcybTDKgkIx9Jxj1QXpmLFzNNCfCBBZhoZr/ikT7WmIBNb8JEJArXDSR9IEoILOM8eCzyIBL5Y1G4Ns3m3+z+g85e47jhn+/XWifjWCtoC22jXdREh6iFzlAbdRBBPfSMXtCr8+Z8OJ/O10g65Yx3NtBEOd8/JFqq8A==</latexit><latexit sha1_base64="pGOJAC+5XIxMYgPoGWdA3fbkKOc=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapShHjcKrhwLIICEokqx3FaCz8iewOUKJ/AFb6Ej0GcEFc+AqftgcJK1o5nZ63xRClnBnz/3Zmanpmdm68suItLyyur1bX1K6MyTWiHKK70TYQN5UzSDjDg9CbVFIuI0+vo7rScX99TbZiSlzBIaShwT7KEEQyWunjqHnSrNb/hD8v7D5pjUEPjanfXnM0gViQTVALh2Jjbpp9CmGMNjHBauEFmaIrJHe7RWwslFtSE+dBr4e1YJvYSpe2R4A3Z3xs5FkZg6Ftl2czErGRAKW7qVgV9UbbymeHdDERUj0S9FGmTmD9GIDkKcybTDKgkIx9Jxj1QXpmLFzNNCfCBBZhoZr/ikT7WmIBNb8JEJArXDSR9IEoILOM8eCzyIBL5Y1G4Ns3m3+z+g85e47jhn+/XWifjWCtoC22jXdREh6iFzlAbdRBBPfSMXtCr8+Z8OJ/O10g65Yx3NtBEOd8/JFqq8A==</latexit><latexit sha1_base64="pGOJAC+5XIxMYgPoGWdA3fbkKOc=">AAACMnicZVDLTsMwEHR4lvAqcINLRIXEoapShHjcKrhwLIICEokqx3FaCz8iewOUKJ/AFb6Ej0GcEFc+AqftgcJK1o5nZ63xRClnBnz/3Zmanpmdm68suItLyyur1bX1K6MyTWiHKK70TYQN5UzSDjDg9CbVFIuI0+vo7rScX99TbZiSlzBIaShwT7KEEQyWunjqHnSrNb/hD8v7D5pjUEPjanfXnM0gViQTVALh2Jjbpp9CmGMNjHBauEFmaIrJHe7RWwslFtSE+dBr4e1YJvYSpe2R4A3Z3xs5FkZg6Ftl2czErGRAKW7qVgV9UbbymeHdDERUj0S9FGmTmD9GIDkKcybTDKgkIx9Jxj1QXpmLFzNNCfCBBZhoZr/ikT7WmIBNb8JEJArXDSR9IEoILOM8eCzyIBL5Y1G4Ns3m3+z+g85e47jhn+/XWifjWCtoC22jXdREh6iFzlAbdRBBPfSMXtCr8+Z8OJ/O10g65Yx3NtBEOd8/JFqq8A==</latexit>

z7<latexit sha1_base64="UaMLlKfO/9t66m7p0UvzLTfZft8=">AAACMnicZVDLSsNAFJ34rPFZ3ekmWAQXpaQiqLuiG5cVjQomlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOctF1YvTDcM+eeO5w5ccqZAd9/d2Zm5+YXFitL7vLK6tr6RnXz2qhMExoQxZW+jbGhnEkaAANOb1NNsYg5vYnvz8r5zQPVhil5BcOURgL3JEsYwWCpy6fOUWej5jf8UXn/QXMCamhS7U7V2Q67imSCSiAcG3PX9FOIcqyBEU4LN8wMTTG5xz16Z6HEgpooH3ktvD3LdL1EaXskeCP290aOhREY+lZZNjM1KxlQipu6VUFflK18ZnQ3QxHXY1EvRdok5o8RSI6jnMk0AyrJ2EeScQ+UV+bidZmmBPjQAkw0s1/xSB9rTMCmN2UiFoXrhpI+EiUElt08HBR5GIt8UBSuTbP5N7v/IDhonDT8i8Na63QSawXtoF20j5roCLXQOWqjABHUQ8/oBb06b86H8+l8jaUzzmRnC02V8/0DJiaq8Q==</latexit><latexit sha1_base64="UaMLlKfO/9t66m7p0UvzLTfZft8=">AAACMnicZVDLSsNAFJ34rPFZ3ekmWAQXpaQiqLuiG5cVjQomlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOctF1YvTDcM+eeO5w5ccqZAd9/d2Zm5+YXFitL7vLK6tr6RnXz2qhMExoQxZW+jbGhnEkaAANOb1NNsYg5vYnvz8r5zQPVhil5BcOURgL3JEsYwWCpy6fOUWej5jf8UXn/QXMCamhS7U7V2Q67imSCSiAcG3PX9FOIcqyBEU4LN8wMTTG5xz16Z6HEgpooH3ktvD3LdL1EaXskeCP290aOhREY+lZZNjM1KxlQipu6VUFflK18ZnQ3QxHXY1EvRdok5o8RSI6jnMk0AyrJ2EeScQ+UV+bidZmmBPjQAkw0s1/xSB9rTMCmN2UiFoXrhpI+EiUElt08HBR5GIt8UBSuTbP5N7v/IDhonDT8i8Na63QSawXtoF20j5roCLXQOWqjABHUQ8/oBb06b86H8+l8jaUzzmRnC02V8/0DJiaq8Q==</latexit><latexit sha1_base64="UaMLlKfO/9t66m7p0UvzLTfZft8=">AAACMnicZVDLSsNAFJ34rPFZ3ekmWAQXpaQiqLuiG5cVjQomlMl00g7OI8zcaGvIJ7jVL/FjxJW49SOctF1YvTDcM+eeO5w5ccqZAd9/d2Zm5+YXFitL7vLK6tr6RnXz2qhMExoQxZW+jbGhnEkaAANOb1NNsYg5vYnvz8r5zQPVhil5BcOURgL3JEsYwWCpy6fOUWej5jf8UXn/QXMCamhS7U7V2Q67imSCSiAcG3PX9FOIcqyBEU4LN8wMTTG5xz16Z6HEgpooH3ktvD3LdL1EaXskeCP290aOhREY+lZZNjM1KxlQipu6VUFflK18ZnQ3QxHXY1EvRdok5o8RSI6jnMk0AyrJ2EeScQ+UV+bidZmmBPjQAkw0s1/xSB9rTMCmN2UiFoXrhpI+EiUElt08HBR5GIt8UBSuTbP5N7v/IDhonDT8i8Na63QSawXtoF20j5roCLXQOWqjABHUQ8/oBb06b86H8+l8jaUzzmRnC02V8/0DJiaq8Q==</latexit>

x1<latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit><latexit sha1_base64="wPlw8y/0IqfHJrazcSQaHuwQ07k=">AAACDXicZVDLSgMxFM34rPVVdekmWAQXpcyIoO6KblxWcGyhLSWTZtrQZDIkd8Qy9B/ErX6HK3HrN/gZ/oGZ6SxseyHck3PPDScniAU34Lo/zsrq2vrGZmmrvL2zu7dfOTh8NCrRlPlUCaXbATFM8Ij5wEGwdqwZkYFgrWB8m81bT0wbrqIHmMSsJ8kw4iGnBCzV7gYyfZ72vX6l6tbdvPAy8ApQRUU1+5Xf7kDRRLIIqCDGdDw3hl5KNHAq2LTcTQyLCR2TIetYGBHJTC/N/U7xqWUGOFTanghwzv7fSIk0ksDIKrNm5mYZA0oJU7MqGMmsZc/kdzORQS2QtUykTWgWjEB41Ut5FCfAIjrzESYCg8JZNnjANaMgJhYQqrn9CqYjogkFm2DZZuQtJrIM/PP6dd29v6g2boqwSugYnaAz5KFL1EB3qIl8RJFAr+gNvTsvzofz6XzNpCtOsXOE5sr5/gMr950D</latexit>

e2<latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit><latexit sha1_base64="wBcNk6euWGMrg+Da+mnoLSs6/Fg=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcq0COqu6MZlBUcL7VAyaaYNzWNI7ghl6D+IW/0OV+LWb/Az/AMz01nY9kK4J+eeG05OGHNmwPN+nNLa+sbmVnm7srO7t39QPTx6NCrRhPpEcaW7ITaUM0l9YMBpN9YUi5DTp3Bym82fnqk2TMkHmMY0EHgkWcQIBkt1+6FI6WzQGlRrXsPLy10FzQLUUFGdQfW3P1QkEVQC4diYXtOLIUixBkY4nVX6iaExJhM8oj0LJRbUBGnud+aeWWboRkrbI8HN2f8bKRZGYBhbZdbMwixjQClu6lYFY5G17Jn8bqYirIeinom0icySEYiugpTJOAEqydxHlHAXlJtl4w6ZpgT41AJMNLNfcckYa0zAJlixGTWXE1kFfqtx3fDuL2rtmyKsMjpBp+gcNdElaqM71EE+IoijV/SG3p0X58P5dL7m0pJT7ByjhXK+/wAOWpzx</latexit>

e1<latexit sha1_base64="5hfya15Z2veNhmcIusOWF706ODA=">AAACS3icZVDLSgMxFM3Ud31VXboZLIKLUmZEUHdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6cCm4xCD68ysLi0vLK6lp1fWNza7u2s3tndWYYtJkW2nRiakFwBW3kKKCTGqAyFnAfP16W8/snMJZrdYvjFHqSPiiecEaxpCLoh/1aPWgGk/L/g3AG6mRW1/0d7zgaaJZJUMgEtbYbBin2cmqQMwFFNcospJQ90gfoOqioBNvLJ2YL/9AxAz/Rxh2F/oT9vZFTaSXFoVOWzc7NSga1FrbhVDiUZSufmdztWMaNWDZKkbGJ/WMEk7NezlWaISg29ZFkwkftl8H4A26AoRg7QJnh7is+G1JDGbr45kzEsqhWIwXPTEtJ1SCPRkUexTIfFcU8D1MeHO9SDv9m+h+0j5vnzeDmpN66mMW9SvbJATkiITklLXJFrkmbMDIkL+SVvHnv3qf35X1PpRVvtrNH5qqy9AOQxLSV</latexit><latexit sha1_base64="5hfya15Z2veNhmcIusOWF706ODA=">AAACS3icZVDLSgMxFM3Ud31VXboZLIKLUmZEUHdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6cCm4xCD68ysLi0vLK6lp1fWNza7u2s3tndWYYtJkW2nRiakFwBW3kKKCTGqAyFnAfP16W8/snMJZrdYvjFHqSPiiecEaxpCLoh/1aPWgGk/L/g3AG6mRW1/0d7zgaaJZJUMgEtbYbBin2cmqQMwFFNcospJQ90gfoOqioBNvLJ2YL/9AxAz/Rxh2F/oT9vZFTaSXFoVOWzc7NSga1FrbhVDiUZSufmdztWMaNWDZKkbGJ/WMEk7NezlWaISg29ZFkwkftl8H4A26AoRg7QJnh7is+G1JDGbr45kzEsqhWIwXPTEtJ1SCPRkUexTIfFcU8D1MeHO9SDv9m+h+0j5vnzeDmpN66mMW9SvbJATkiITklLXJFrkmbMDIkL+SVvHnv3qf35X1PpRVvtrNH5qqy9AOQxLSV</latexit><latexit sha1_base64="5hfya15Z2veNhmcIusOWF706ODA=">AAACS3icZVDLSgMxFM3Ud31VXboZLIKLUmZEUHdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6cCm4xCD68ysLi0vLK6lp1fWNza7u2s3tndWYYtJkW2nRiakFwBW3kKKCTGqAyFnAfP16W8/snMJZrdYvjFHqSPiiecEaxpCLoh/1aPWgGk/L/g3AG6mRW1/0d7zgaaJZJUMgEtbYbBin2cmqQMwFFNcospJQ90gfoOqioBNvLJ2YL/9AxAz/Rxh2F/oT9vZFTaSXFoVOWzc7NSga1FrbhVDiUZSufmdztWMaNWDZKkbGJ/WMEk7NezlWaISg29ZFkwkftl8H4A26AoRg7QJnh7is+G1JDGbr45kzEsqhWIwXPTEtJ1SCPRkUexTIfFcU8D1MeHO9SDv9m+h+0j5vnzeDmpN66mMW9SvbJATkiITklLXJFrkmbMDIkL+SVvHnv3qf35X1PpRVvtrNH5qqy9AOQxLSV</latexit>

e3<latexit sha1_base64="Yo9t2xRNhbcucp40+9En1vW30ug=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYqqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfS2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFabl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+UjLSX</latexit><latexit sha1_base64="Yo9t2xRNhbcucp40+9En1vW30ug=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYqqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfS2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFabl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+UjLSX</latexit><latexit sha1_base64="Yo9t2xRNhbcucp40+9En1vW30ug=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYqqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfS2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFabl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+UjLSX</latexit>

e4<latexit sha1_base64="pOqccpiEKJr77/8/ytM6r0hBvEQ=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUyloO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3rVr7Yhb3Ojkgh+SYNMkpaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+WcLSY</latexit><latexit sha1_base64="pOqccpiEKJr77/8/ytM6r0hBvEQ=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUyloO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3rVr7Yhb3Ojkgh+SYNMkpaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+WcLSY</latexit><latexit sha1_base64="pOqccpiEKJr77/8/ytM6r0hBvEQ=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUyloO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3rVr7Yhb3Ojkgh+SYNMkpaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+WcLSY</latexit>

e5<latexit sha1_base64="MjmPb/ACrHEtZbpogr0zpFexMKk=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqaiqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfa2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFSbl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+YVLSZ</latexit><latexit sha1_base64="MjmPb/ACrHEtZbpogr0zpFexMKk=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqaiqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfa2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFSbl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+YVLSZ</latexit><latexit sha1_base64="MjmPb/ACrHEtZbpogr0zpFexMKk=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqaiqLuiG5cVrBWcUjLpHRvMY0juaMswv+BWv8q9/+FKXJhpu7D2Qrgn554bTk6UCG4xCD690sLi0vJKebWytr6xubW9s3tndWoYtJkW2txH1ILgCtrIUcB9YoDKSEAneroq5p1nMJZrdYujBLqSPioec0axoELonfa2q0E9GJc/DxpTUCXTavV2vOOwr1kqQSET1NqHRpBgN6MGOROQV8LUQkLZE32EBwcVlWC72dhs7h86pu/H2rij0B+zfzcyKq2kOHDKotmZWcGg1sLWnAoHsmjFM+O7HcmoFslaITI2tv+MYHzezbhKUgTFJj7iVPio/SIYv88NMBQjBygz3H3FZwNqKEMX34yJSOaVSqjghWkpqepn4TDPwkhmwzyf5WHCg+Ndyo3/mc6D9nH9oh7cnFSbl9O4y2SfHJAj0iBnpEmuSYu0CSMD8kreyLv34X15397PRFrypjt7ZKZKS7+YVLSZ</latexit>

e6<latexit sha1_base64="MmCjOdrAP44kfWvHfghiTcRDIeQ=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYiPnZFNy4rWCs4pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHthXBPzj03nJwoEdxiEHx6pYXFpeWV8mplbX1jc2t7Z/fO6tQwaDMttLmPqAXBFbSRo4D7xACVkYBO9HRVzDvPYCzX6hZHCXQlfVQ85oxiQYXQO+1tV4N6MC5/HjSmoEqm1erteMdhX7NUgkImqLUPjSDBbkYNciYgr4SphYSyJ/oIDw4qKsF2s7HZ3D90TN+PtXFHoT9m/25kVFpJceCURbMzs4JBrYWtORUOZNGKZ8Z3O5JRLZK1QmRsbP8Zwfi8m3GVpAiKTXzEqfBR+0Uwfp8bYChGDlBmuPuKzwbUUIYuvhkTkcwrlVDBC9NSUtXPwmGehZHMhnk+y8OEB8e7lBv/M50H7eP6RT24Oak2L6dxl8k+OSBHpEHOSJNckxZpE0YG5JW8kXfvw/vyvr2fibTkTXf2yEyVln4Bmji0mg==</latexit><latexit sha1_base64="MmCjOdrAP44kfWvHfghiTcRDIeQ=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYiPnZFNy4rWCs4pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHthXBPzj03nJwoEdxiEHx6pYXFpeWV8mplbX1jc2t7Z/fO6tQwaDMttLmPqAXBFbSRo4D7xACVkYBO9HRVzDvPYCzX6hZHCXQlfVQ85oxiQYXQO+1tV4N6MC5/HjSmoEqm1erteMdhX7NUgkImqLUPjSDBbkYNciYgr4SphYSyJ/oIDw4qKsF2s7HZ3D90TN+PtXFHoT9m/25kVFpJceCURbMzs4JBrYWtORUOZNGKZ8Z3O5JRLZK1QmRsbP8Zwfi8m3GVpAiKTXzEqfBR+0Uwfp8bYChGDlBmuPuKzwbUUIYuvhkTkcwrlVDBC9NSUtXPwmGehZHMhnk+y8OEB8e7lBv/M50H7eP6RT24Oak2L6dxl8k+OSBHpEHOSJNckxZpE0YG5JW8kXfvw/vyvr2fibTkTXf2yEyVln4Bmji0mg==</latexit><latexit sha1_base64="MmCjOdrAP44kfWvHfghiTcRDIeQ=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqYiPnZFNy4rWCs4pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHthXBPzj03nJwoEdxiEHx6pYXFpeWV8mplbX1jc2t7Z/fO6tQwaDMttLmPqAXBFbSRo4D7xACVkYBO9HRVzDvPYCzX6hZHCXQlfVQ85oxiQYXQO+1tV4N6MC5/HjSmoEqm1erteMdhX7NUgkImqLUPjSDBbkYNciYgr4SphYSyJ/oIDw4qKsF2s7HZ3D90TN+PtXFHoT9m/25kVFpJceCURbMzs4JBrYWtORUOZNGKZ8Z3O5JRLZK1QmRsbP8Zwfi8m3GVpAiKTXzEqfBR+0Uwfp8bYChGDlBmuPuKzwbUUIYuvhkTkcwrlVDBC9NSUtXPwmGehZHMhnk+y8OEB8e7lBv/M50H7eP6RT24Oak2L6dxl8k+OSBHpEHOSJNckxZpE0YG5JW8kXfvw/vyvr2fibTkTXf2yEyVln4Bmji0mg==</latexit>

e7<latexit sha1_base64="+2fC0KPrqycQirOuKDePs7MeCTY=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFqO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3p7X2xSzudXJADskxaZIWaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+cHLSb</latexit><latexit sha1_base64="+2fC0KPrqycQirOuKDePs7MeCTY=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFqO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3p7X2xSzudXJADskxaZIWaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+cHLSb</latexit><latexit sha1_base64="+2fC0KPrqycQirOuKDePs7MeCTY=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFqO6KblwqWBU6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW19Y3y5tb2zm6lundndWoYdJgW2jxE1ILgCjrIUcBDYoDKSMB99HRZzO+fwViu1S2OE+hJ+qh4zBnFggqh3+pXakEjmJT/HzRnoEZmdd2veifhQLNUgkImqLXdZpBgL6MGOROQl8PUQkLZE32EroOKSrC9bGI2948cM/BjbdxR6E/YvxsZlVZSHDpl0ezcrGBQa2HrToVDWbTimcndjmVUj2S9EBkb2wUjGJ/1Mq6SFEGxqY84FT5qvwjGH3ADDMXYAcoMd1/x2ZAaytDFN2ciknm5HCp4YVpKqgZZOMqzMJLZKM/neZjy4HiXcnMx0/+gc9I4bwQ3p7X2xSzudXJADskxaZIWaZMrck06hJEheSVv5N378L68b+9nKi15s519MlellV+cHLSb</latexit>

x4<latexit sha1_base64="9frhfHTAh/tVZBbTFF8T6hotkFU=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvdPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCuoK0qw==</latexit><latexit sha1_base64="9frhfHTAh/tVZBbTFF8T6hotkFU=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvdPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCuoK0qw==</latexit><latexit sha1_base64="9frhfHTAh/tVZBbTFF8T6hotkFU=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvdPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCuoK0qw==</latexit>

x5<latexit sha1_base64="Iqq29VCa1Eput5eq9gbubvV+118=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFUXdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6UCG4xCD680sLi0vLK6lp5fWNza7tS3bmzOjUM2kwLbToRtSC4gjZyFNBJDFAZCbiPHi+L+f0TGMu1usVxAj1JHxSPOaNYUOGof9Kv1IJGMCn/P2jOQI3M6rpf9Y7CgWapBIVMUGu7zSDBXkYNciYgL4ephYSyR/oAXQcVlWB72cRs7h84ZuDH2rij0J+wvzcyKq2kOHTKotm5WcGg1sLWnQqHsmjFM5O7HcuoHsl6ITI2tn+MYHzWy7hKUgTFpj7iVPio/SIYf8ANMBRjBygz3H3FZ0NqKEMX35yJSOblcqjgmWkpqRpk4SjPwkhmozyf52HKg+Ndys2/mf4H7aPGeSO4Oa61LmZxr5I9sk8OSZOckha5ItekTRgZkhfySt68d+/T+/K+p9KSN9vZJXNVWvoBvGa0rA==</latexit><latexit sha1_base64="Iqq29VCa1Eput5eq9gbubvV+118=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFUXdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6UCG4xCD680sLi0vLK6lp5fWNza7tS3bmzOjUM2kwLbToRtSC4gjZyFNBJDFAZCbiPHi+L+f0TGMu1usVxAj1JHxSPOaNYUOGof9Kv1IJGMCn/P2jOQI3M6rpf9Y7CgWapBIVMUGu7zSDBXkYNciYgL4ephYSyR/oAXQcVlWB72cRs7h84ZuDH2rij0J+wvzcyKq2kOHTKotm5WcGg1sLWnQqHsmjFM5O7HcuoHsl6ITI2tn+MYHzWy7hKUgTFpj7iVPio/SIYf8ANMBRjBygz3H3FZ0NqKEMX35yJSOblcqjgmWkpqRpk4SjPwkhmozyf52HKg+Ndys2/mf4H7aPGeSO4Oa61LmZxr5I9sk8OSZOckha5ItekTRgZkhfySt68d+/T+/K+p9KSN9vZJXNVWvoBvGa0rA==</latexit><latexit sha1_base64="Iqq29VCa1Eput5eq9gbubvV+118=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFUXdFNy4VrBY6pWTSOzaYx5Dc0ZZhfsGtfpV7/8OVuDDTdmH1Qrgn554bTk6UCG4xCD680sLi0vLK6lp5fWNza7tS3bmzOjUM2kwLbToRtSC4gjZyFNBJDFAZCbiPHi+L+f0TGMu1usVxAj1JHxSPOaNYUOGof9Kv1IJGMCn/P2jOQI3M6rpf9Y7CgWapBIVMUGu7zSDBXkYNciYgL4ephYSyR/oAXQcVlWB72cRs7h84ZuDH2rij0J+wvzcyKq2kOHTKotm5WcGg1sLWnQqHsmjFM5O7HcuoHsl6ITI2tn+MYHzWy7hKUgTFpj7iVPio/SIYf8ANMBRjBygz3H3FZ0NqKEMX35yJSOblcqjgmWkpqRpk4SjPwkhmozyf52HKg+Ndys2/mf4H7aPGeSO4Oa61LmZxr5I9sk8OSZOckha5ItekTRgZkhfySt68d+/T+/K+p9KSN9vZJXNVWvoBvGa0rA==</latexit>

x6<latexit sha1_base64="KY5/j4VzeKfzF+AkT011ZxYsr7Q=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFfOyKblwqWC10Ssmkd2wwjyG5oy3D/IJb/Sr3/ocrcWGm7cLqhXBPzj03nJwoEdxiEHx4pYXFpeWV1bXy+sbm1nalunNndWoYtJkW2nQiakFwBW3kKKCTGKAyEnAfPV4W8/snMJZrdYvjBHqSPigec0axoMJR/6RfqQWNYFL+f9CcgRqZ1XW/6h2FA81SCQqZoNZ2m0GCvYwa5ExAXg5TCwllj/QBug4qKsH2sonZ3D9wzMCPtXFHoT9hf29kVFpJceiURbNzs4JBrYWtOxUOZdGKZyZ3O5ZRPZL1QmRsbP8Ywfisl3GVpAiKTX3EqfBR+0Uw/oAbYCjGDlBmuPuKz4bUUIYuvjkTkczL5VDBM9NSUjXIwlGehZHMRnk+z8OUB8e7lJt/M/0P2keN80Zwc1xrXcziXiV7ZJ8ckiY5JS1yRa5JmzAyJC/klbx5796n9+V9T6Ulb7azS+aqtPQDvkq0rQ==</latexit><latexit sha1_base64="KY5/j4VzeKfzF+AkT011ZxYsr7Q=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFfOyKblwqWC10Ssmkd2wwjyG5oy3D/IJb/Sr3/ocrcWGm7cLqhXBPzj03nJwoEdxiEHx4pYXFpeWV1bXy+sbm1nalunNndWoYtJkW2nQiakFwBW3kKKCTGKAyEnAfPV4W8/snMJZrdYvjBHqSPigec0axoMJR/6RfqQWNYFL+f9CcgRqZ1XW/6h2FA81SCQqZoNZ2m0GCvYwa5ExAXg5TCwllj/QBug4qKsH2sonZ3D9wzMCPtXFHoT9hf29kVFpJceiURbNzs4JBrYWtOxUOZdGKZyZ3O5ZRPZL1QmRsbP8Ywfisl3GVpAiKTX3EqfBR+0Uw/oAbYCjGDlBmuPuKz4bUUIYuvjkTkczL5VDBM9NSUjXIwlGehZHMRnk+z8OUB8e7lJt/M/0P2keN80Zwc1xrXcziXiV7ZJ8ckiY5JS1yRa5JmzAyJC/klbx5796n9+V9T6Ulb7azS+aqtPQDvkq0rQ==</latexit><latexit sha1_base64="KY5/j4VzeKfzF+AkT011ZxYsr7Q=">AAACS3icZVDLSgMxFM3Ud321unQzWAQXpUxFfOyKblwqWC10Ssmkd2wwjyG5oy3D/IJb/Sr3/ocrcWGm7cLqhXBPzj03nJwoEdxiEHx4pYXFpeWV1bXy+sbm1nalunNndWoYtJkW2nQiakFwBW3kKKCTGKAyEnAfPV4W8/snMJZrdYvjBHqSPigec0axoMJR/6RfqQWNYFL+f9CcgRqZ1XW/6h2FA81SCQqZoNZ2m0GCvYwa5ExAXg5TCwllj/QBug4qKsH2sonZ3D9wzMCPtXFHoT9hf29kVFpJceiURbNzs4JBrYWtOxUOZdGKZyZ3O5ZRPZL1QmRsbP8Ywfisl3GVpAiKTX3EqfBR+0Uw/oAbYCjGDlBmuPuKz4bUUIYuvjkTkczL5VDBM9NSUjXIwlGehZHMRnk+z8OUB8e7lJt/M/0P2keN80Zwc1xrXcziXiV7ZJ8ckiY5JS1yRa5JmzAyJC/klbx5796n9+V9T6Ulb7azS+aqtPQDvkq0rQ==</latexit>

x7<latexit sha1_base64="QuysJyVG4LzFzaADM3QOo12uCc8=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvbPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCwC60rg==</latexit><latexit sha1_base64="QuysJyVG4LzFzaADM3QOo12uCc8=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvbPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCwC60rg==</latexit><latexit sha1_base64="QuysJyVG4LzFzaADM3QOo12uCc8=">AAACS3icZVDLSgMxFM3UV61vXboZLIKLUqZFUHeiG5cVrAqdUjLpHRuax5Dc0ZZhfsGtfpV7/8OVuDDTdmHrhXBPzj03nJwoEdxiEHx6paXlldW18nplY3Nre2d3b//e6tQwaDMttHmMqAXBFbSRo4DHxACVkYCHaHhdzB+ewViu1R2OE+hK+qR4zBnFggpHvbPebjWoB5Py/4PGDFTJrFq9Pa8Z9jVLJShkglrbaQQJdjNqkDMBeSVMLSSUDekTdBxUVILtZhOzuX/smL4fa+OOQn/C/t3IqLSS4sApi2bnZgWDWgtbcyocyKIVz0zudiyjWiRrhcjY2C4Ywfi8m3GVpAiKTX3EqfBR+0Uwfp8bYCjGDlBmuPuKzwbUUIYuvjkTkcwrlVDBC9NSUtXPwlGehZHMRnk+z8OUB8e7lBuLmf4H7Wb9oh7cnlYvr2Zxl8khOSInpEHOyCW5IS3SJowMyCt5I+/eh/flfXs/U2nJm+0ckLkqrfwCwC60rg==</latexit>

cat stopped dog inThe a Paris

x3<latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit><latexit sha1_base64="r7RTsVKDbYXIRCyPj9szuInKMVA=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcqMCuqu6MZlBccW2qFk0kwbmkyG5I5Yhv6DuNXvcCVu/QY/wz8wM52FbS+Ee3LuueHkBDFnGhznxyqtrK6tb5Q3K1vbO7t71f2DRy0TRahHJJeqE2BNOYuoBww47cSKYhFw2g7Gt9m8/USVZjJ6gElMfYGHEQsZwWCoTi8Q6fO0f96v1pyGk5e9DNwC1FBRrX71tzeQJBE0AsKx1l3XicFPsQJGOJ1WeommMSZjPKRdAyMsqPbT3O/UPjHMwA6lMicCO2f/b6RYaIFhZJRZ03OzjAEpua4bFYxE1rJn8rueiKAeiHomUjrUC0YgvPJTFsUJ0IjMfIQJt0HaWTb2gClKgE8MwEQx8xWbjLDCBEyCFZORu5jIMvDOGtcN5/6i1rwpwiqjI3SMTpGLLlET3aEW8hBBHL2iN/RuvVgf1qf1NZOWrGLnEM2V9f0HLzudBQ==</latexit>

x2<latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit><latexit sha1_base64="OUe9NQ3ny9IilmHPU3ONQ4TtfRo=">AAACDXicZVDLSgMxFM3UV62vqks3g0VwUcpMEdRd0Y3LCo4ttEPJpJk2NJkMyR2xDP0Hcavf4Urc+g1+hn9gZjoL214I9+Tcc8PJCWLONDjOj1VaW9/Y3CpvV3Z29/YPqodHj1omilCPSC5VN8CachZRDxhw2o0VxSLgtBNMbrN554kqzWT0ANOY+gKPIhYygsFQ3X4g0ufZoDmo1pyGk5e9CtwC1FBR7UH1tz+UJBE0AsKx1j3XicFPsQJGOJ1V+ommMSYTPKI9AyMsqPbT3O/MPjPM0A6lMicCO2f/b6RYaIFhbJRZ0wuzjAEpua4bFYxF1rJn8rueiqAeiHomUjrUS0YgvPJTFsUJ0IjMfYQJt0HaWTb2kClKgE8NwEQx8xWbjLHCBEyCFZORu5zIKvCajeuGc39Ra90UYZXRCTpF58hFl6iF7lAbeYggjl7RG3q3XqwP69P6mktLVrFzjBbK+v4DLZmdBA==</latexit>

Learning Dependency Heads w/ Attention (Kuncoro et al. 2017)

• Given a phrase structure tree, what child is the head word, the most important word in the phrase?

• Idea: create a phrase composition function that uses attention: examine if attention weights follow heads defined by linguistics

Other Examples

Learning about Word Segmentation from Attention (Boito et al. 2017)

• We want to learn word segmentation in an unsegmented language

• Simple idea: we can inspect the attention matrices from a neural MT system to extract words

Learning Segmentations w/ Reconstruction Loss (Elsner and Shain 2017)• Learn segmentations of speech/text that allow for easy re-

construction of the original • Idea: consistent segmentation should result in easier-to-

reconstruct segments • Train segmentation using policy gradient

Learning Language-level Features (Malaviya et al. 2017)

• All previous work learned features of a single sentence

• Can we learn features of the whole language? e.g. Typology: what is the canonical word order, etc.

• A simple method: train a neural MT system on 1017 languages, and extract its representations

Questions?