ACTA UNIVERSITATIS UPSALIENSIS Studia Linguistica Upsaliensia
ACTA UNIVERSITATIS UPSALIENSIS Studia...
Transcript of ACTA UNIVERSITATIS UPSALIENSIS Studia...
![Page 1: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/1.jpg)
ACTA UNIVERSITATIS UPSALIENSIS Studia Linguistica Upsaliensia
24
![Page 2: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/2.jpg)
![Page 3: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/3.jpg)
Linguistically Informed Neural Dependency Parsing for
Typologically Diverse Languages
Miryam de Lhoneux
![Page 4: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/4.jpg)
![Page 5: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/5.jpg)
![Page 6: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/6.jpg)
![Page 7: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/7.jpg)
si bi itht w
lc1(s1) rc1(s1)s1 ◦
/
δ
bi bw f w +lc> 0.5 +lc
bi bw f w+lc
∗= p < .05 ∗∗= p < .01
![Page 8: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/8.jpg)
bas rc
δ
p < .05p < .01
δ
p < .05 p < .01
+c
δ
p < .05 ∗∗= p < .01
| < μ−σ < | < μ− < μ < μ+ < | < μ+σ <
|
δ
![Page 9: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/9.jpg)
∗= p < .05 ∗∗= p < .01
δ
p < .05 p < .01
![Page 10: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/10.jpg)
Σσ
B β
htx1:t
Σσ
B β
![Page 11: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/11.jpg)
[[Φ]]Φ s0 s1
b ΣΣ−s0 s0 Σ
B B−b b
bi+rc +lc
bi bwf w
bi bwf w
+rc +lc
![Page 12: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/12.jpg)
![Page 13: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/13.jpg)
![Page 14: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/14.jpg)
![Page 15: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/15.jpg)
![Page 16: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/16.jpg)
![Page 17: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/17.jpg)
![Page 18: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/18.jpg)
![Page 19: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/19.jpg)
![Page 20: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/20.jpg)
![Page 21: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/21.jpg)
![Page 22: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/22.jpg)
![Page 23: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/23.jpg)
![Page 24: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/24.jpg)
![Page 25: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/25.jpg)
![Page 26: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/26.jpg)
![Page 27: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/27.jpg)
![Page 28: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/28.jpg)
![Page 29: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/29.jpg)
![Page 30: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/30.jpg)
G (V,A)V A (h,r,d)
h r dn w1:n
A AV,
V {I, love,syntax} A{(love,nsub j, I),(love,ob j,syntax)}
x1:n x1, ...,xn
![Page 31: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/31.jpg)
(wi,r,w j) ∈ A G = (V,A)wi →∗ wk i < k < j i < j j < k < i j < i
→∗A
{(love,nsub j, I),(love,nmod,syntax)}
![Page 32: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/32.jpg)
c0(x = (w1, . . . ,wn)) = ([ ], [1, . . . ,n,0], /0)
Ct = {c ∈C |c = ([ ], [0],A)}
d (σ |i, j|β ,A)⇒ (σ , j|β ,A∪{( j,r, i)})d (σ |i| j,β ,A)⇒ (σ |i,β ,A∪{(i,r, j)})
(σ , i|β ,A)⇒ (σ |i,β ,A) i �= 0
Σσ B β
LEFT−ARC
SHIFT
RIGHT−ARCthe brown fox jumped root
Transitions: Configuration:
STACK BUFFER
BΣ
![Page 33: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/33.jpg)
si bi itht w lc1(s1) rc1(s1)
s1 ◦
s1.w;s1.t;s1.wt;s2.w;s2.t;s2.wt;b1.w;b1.t;b1.wt;
s1.wt ◦ s2.wt;s1.wt ◦ s2.w;s1.wt ◦ s2.ts1.w◦ s2.wt;s1.t ◦ s2.wt;s1.w◦ s2.ws1.t ◦ s2.t;s1.t ◦b1.t
s2.t ◦ s1.t ◦b1.t;s2.t ◦ s1.t ◦ lc1(s1).t;s2.t ◦ s1.t ◦ rc1(s1).t;s2.t ◦ s1.t ◦ lc1(s2).t;s2.t ◦ s1.t ◦ rc1(s2).t;s2.t ◦ s1.w◦ rc1(s2).t;s2.t ◦ s1.w◦ lc1(s1).t;s2.t ◦ s1.w◦b1.t
d d
di
j r ( j,r, i)d j
i (i,r, j)i
c φ(·)
![Page 34: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/34.jpg)
i ←s T
c ← cs(s)c
tp ← argmaxt ·φ(c, t)to ← o(c,T )
tp �= toφ(c, to),φ(c, tp)
c ← to(c)
cs(s) stp c
φ(c, t)to tp
,φ(c, to),φ(c, tp)o(c,T )
c T
![Page 35: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/35.jpg)
x yhi
Wx+bx f
y
h = f (W1x+b)y = so f tmax(W2h)
![Page 36: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/36.jpg)
����
�
�h
p
xt xlxw
xw
xt
xl
xw xt xl h
hp
h = (W w1 xw +Wt
1xt +W l1xl +b1)
3
p = so f tmax(W2h)
![Page 37: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/37.jpg)
0x x x x
hh h h0 1hh1
1
2
2
t
t...
ht x1:t
x1:n ht
ht xtht−1 ht
h = RNN(x1:n)
ht = f (ht−1,xt)
ht
![Page 38: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/38.jpg)
ft = σ(Wf xt +Uf ht−1 +b f )
it = σ(Wixt +Uiht−1 +bi)
ot = σ(Woxt +Uoht−1 +bo)
ct = ft ct−1 + it tanh(Wcxt +Ucht−1 +bc)
ht = ot tanh(ct)
tσ ft it ot
ft ct−1(it tanh(Wcxt +Ucht−1 +bc)ct
tanh
x1:n
f
b xn:1i xi
[v1;v2]
(x1:n,xi) = [ f (x1:i); b(xn:i)]
cφ(·)
![Page 39: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/39.jpg)
φ(·)
n w1:n x1:nxi wi
ti wi xi
xi = [e(wi);e(ti)]
vic
s2 s1 s0 b0
d d
d d
vi = (x1:n, i)φ(c) = [vs2 ;vs1 ;vs0 ;vb0 ]
(φ(c)) =W 2tanh(W 1φ(c)+b1)+b2
![Page 40: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/40.jpg)
LSTM f
LSTM b
LSTM f
LSTM b
LSTM f
LSTM b
concat concat concat concat
LSTM f
LSTM b
MLP
concat
LSTM f
LSTM b
XX
Vthe Vfox Vjumped Vroot
X
Vbrown
X the X brown fox jumped root
score(LEFT−ARC),score(RIGHT−ARC),score(SHIFT)
![Page 41: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/41.jpg)
multiple models
multiple languages
one model
multiple languageslanguages have equal status
source and target languages
Multilingual parsing
multi−source single−source
polymonolingual
polyglot
cross−lingual
![Page 42: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/42.jpg)
![Page 43: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/43.jpg)
![Page 44: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/44.jpg)
![Page 45: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/45.jpg)
![Page 46: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/46.jpg)
Task A Task B Task C Task specific
layers
Shared layers
![Page 47: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/47.jpg)
![Page 48: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/48.jpg)
![Page 49: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/49.jpg)
![Page 50: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/50.jpg)
![Page 51: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/51.jpg)
![Page 52: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/52.jpg)
![Page 53: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/53.jpg)
![Page 54: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/54.jpg)
concat
Cf Cf Cf
Cb Cb Cb
h et
ce(wi)wi ch j 1 ≤ j ≤ m wi
ce(wi) = (ch1:m)
i xie(wi), ce(wi)
e(ti) xi
xi = [e(wi);ce(wi);e(ti)]
pe(wi)
![Page 55: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/55.jpg)
xi = [e(wi);ce(wi)]
![Page 56: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/56.jpg)
![Page 57: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/57.jpg)
![Page 58: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/58.jpg)
c0(x = (w1, . . . ,wn)) = ([ ], [1, . . . ,n,0], /0)
Ct = {c ∈C |c = ([ ], [0],A)}
d (σ |i, j|β ,A)⇒ (σ , j|β ,A∪{( j,r, i)}) j �= 0 ∨ σ = [ ]
d (σ |i| j,β ,A)⇒ (σ |i,β ,A∪{(i,r, j)})(σ , i|β ,A)⇒ (σ |i,β ,A) i �= 0
(σ |s0, b|β , A)⇒ (σ , b|s0|β , A) β �= 0∧ s0 < b
Σσ B β
everbest ever
examplebest a ever
c c|a
![Page 59: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/59.jpg)
SWAP
LEFT−ARC
SHIFT
RIGHT−ARCthe brown fox jumped root
Transitions: Configuration:
STACK BUFFER
s0s0 b
s0 bb
s0
![Page 60: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/60.jpg)
[ ]Σ [ 1 2 4 3 ]B[ 1 ]Σ [ 2 4 3 ]B[ 1 2 ]Σ [ 4 3 ]B[ 1 2 4 ]Σ [ 3 ]B[ 1 2 ]Σ [ 3 4 ]B[ 1 2 3 ]Σ [ 4 ]B[ 1 2 ]Σ [ 4 ]B
[ 1 ]Σ [ 4 ]B
[ 1 4 ]Σ [ ]B[ 1 ]Σ [ ]B
O(n)O(n2) n
s0 b
o(c,T )
![Page 61: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/61.jpg)
i ←s T
c ← cs(s)c
tp ← argmaxt ·φ(c, t)to ← o(c,T )
tp �= toφ(c, to),φ(c, tp)
c ← to(c)
c Tto
p(c)
o(t;c,T ) ct
T
![Page 62: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/62.jpg)
[ ]Σ [ ]B[ ]Σ [ ]B[ ]Σ [ ]B[ ]Σ [ ]B
[ ]Σ [ ]B[ ]Σ [ ]B
[ ]Σ [ ]B[ ]Σ [ ]B
[ ]Σ [ ]B[ ]Σ [ ]B[ ]Σ [ ]B[ ]Σ [ ]B[ ]Σ [ ]B
[ ]Σ [ ]B
[ ]Σ [ ]B[ ]Σ [ ]B
![Page 63: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/63.jpg)
i ←s T
c ← cs(s)c
←{t | o(t;c,T ) = }tp ← argmaxt∈ (c) ·φ(c, t)to ← argmaxt∈ (c) ·φ(c, t)
tp /∈φ(c, to),φ(c, tp)
c ← k,p c, to, tp, i
c ← tp(c)
k,p c, to, tp, ii > k ()< p
tp(c)
c, to
[ ]Σ [ ]B ⇒ [ ]Σ [ ]B
s0 s1
s0s0 s1
![Page 64: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/64.jpg)
C( ;c,T ) (b,s0) s0s0 H = {s1}∪β
D = {b}∪βT (s0,d) (h,s0) h ∈ H d ∈ D
C( ;c,T ) (s1,s0) s0s0 B =
{b}∪β T (s0,d)(h,s0) h,d ∈ B
C( ;c,T ) b bH = {s1}∪σ
D = {s0,s1}∪σ T(b,d) (h,b) h ∈ H d ∈ D
tt
![Page 65: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/65.jpg)
s1 s0 b
[ ]Σ [ ]B
O(n2)
![Page 66: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/66.jpg)
i
(i) i(i) i i
(s0)> (b)
(s0)> (b)
(b,s0) s0 s0b
s1|βb|β s0
(s0)(h(s0))
(s1,s0) s0s0 s1
b|β s0
![Page 67: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/67.jpg)
C ( ) = | (s0)| + [[h(s0) �= b s0 ∈ (h(s0))]]
(s0) = [ ] s0 (h(s0))
C ( ) = | (s0)| + [[h(s0) �= s1 s0 ∈ (h(s0))]]
(s0) = [ ] s0 (h(s0))
i ∈ B−b b < i (b)> (i)
C ( ) = 0
C ( ) = |{d ∈ (b) |d ∈ Σ}| + [[h(b) ∈ Σ−s0 b ∈ (h(b))]]
b (h(b)) h(b) ∈ Σ−s0 d ∈ Σ(b)
[[Φ]]Φ s0 s1
b ΣΣ−s0 s0 Σ B
B−b b
(s0)(h(s0))
b bΣ s0
Σb
![Page 68: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/68.jpg)
/
/
/
/
/
/
p < 0.01p < 0.05
![Page 69: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/69.jpg)
di d
![Page 70: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/70.jpg)
α
pagg
xie(wi) pe(wi)
ce(wi)
xi = [e(wi); pe(wi);ce(wi)]
![Page 71: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/71.jpg)
δ
δ δ δ
![Page 72: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/72.jpg)
![Page 73: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/73.jpg)
![Page 74: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/74.jpg)
=++−−−+−=+−++−=++=−+==+=
![Page 75: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/75.jpg)
� �
� �
� �
� �
� �
� �
�
![Page 76: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/76.jpg)
![Page 77: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/77.jpg)
![Page 78: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/78.jpg)
![Page 79: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/79.jpg)
![Page 80: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/80.jpg)
![Page 81: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/81.jpg)
![Page 82: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/82.jpg)
The largest city in Minnesota
Recursive
The largest city in Minnesota
Recurrent
![Page 83: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/83.jpg)
![Page 84: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/84.jpg)
ptpt
xi wie(wi) pe(wi)
e(ti)
xi = max{0,W ([e(wi); pe(wi);e(ti)])+b}xi
![Page 85: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/85.jpg)
TOP
TOP
tp
an decision was made ROOT
REDUCE−LEFT(amod)
SHIFT
...
TOP
Stack Buffer
softmax
overhasty
nmod
Actions
ch d
r
tanh hc
xi
xi
c = tanh(W [h;d;r]+b)
![Page 86: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/86.jpg)
CfoxCfox = tanh(W[Cfox,Cthe,left−det]+b)
Cfox
Cthe Cbrown Cfox Cjumped Croot
LSTM f
LSTM b
LSTM f
LSTM b
LSTM f
LSTM b
concat concat concat concat
LSTM f
LSTM b
concat
LSTM f
LSTM b
XX
Vthe Vfox Vjumped Vroot
X
Vbrown
X the X brown fox jumped root
det
vi
ivi
t cti
tc0
i
vi = [ (x1:n, i);cti]
c0i = (x1:n, i)
d h ir ct
i
![Page 87: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/87.jpg)
detc
dh
h d rh h ct
h
+rc +lc
cti = tanh(W [ct−1
i ;d;r]+b)
cti = i([ct−1
i ;d;r])
e(wi)wi e(ti) ce(wi)
xi = [e(wi);e(ti);ce(wi)])
f (x, i)xi x
vi wi bw
![Page 88: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/88.jpg)
f wbi
vi = f (x, i)bw(x, i) = (xn:1, i)f w(x, i) = (x1:n, i)bi(x, i) = [bw(x, i); f w(x, i)]
![Page 89: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/89.jpg)
bi+rc +lc
+lc+rc
![Page 90: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/90.jpg)
bi bwf w
R =−0.838 p < .01
p > .05
![Page 91: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/91.jpg)
bwbi
f w
bw bi
![Page 92: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/92.jpg)
bi bwf w +rc +lc
![Page 93: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/93.jpg)
+ +
− −
bi bi+ lc+ + − −
![Page 94: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/94.jpg)
− +bw f w
− −
bw f w
f wbw
bi bw f w
bw f w bibw f w
![Page 95: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/95.jpg)
bi bw f w +lc> 0.5 +lc
+ + + −
− + − −
bw f w
f w bw
![Page 96: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/96.jpg)
bi bw f w+lc
+ ++ −− +− −
bi bw f w +lc
![Page 97: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/97.jpg)
![Page 98: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/98.jpg)
![Page 99: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/99.jpg)
![Page 100: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/100.jpg)
![Page 101: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/101.jpg)
![Page 102: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/102.jpg)
![Page 103: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/103.jpg)
V vi mviAUXi
wauxaux←−−wmv
![Page 104: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/104.jpg)
viV wmv mvi waux
vi AUXi v vi+1V
vi V AUXimvi
mvimvi
mvi
V
wdaux←−−wh wd wh
wh auxi V,
![Page 105: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/105.jpg)
Vi wh auxi
![Page 106: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/106.jpg)
![Page 107: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/107.jpg)
![Page 108: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/108.jpg)
∗= p < .05 ∗∗= p < .01δ δ
![Page 109: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/109.jpg)
![Page 110: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/110.jpg)
![Page 111: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/111.jpg)
I did thisVerbForm=Fin
FMV
nsubj obj
Figure 5.5. Finite main verb in a UD tree.
I could easily have done thisNFMV
nsubjaux
advmod
aux
root
dobj
I could easily have done thisMAUX
nsubj advmod
aux
aux
root
dobj
Figure 5.6. Example sentence with an AVC annotated in UD (top), and in MS (bottom).AVC subtree in thick blue.
therefore represent the word in the context of the sentence) to the correspond-ing vectors for verb types (the vectors at the input of the BiLSTM) in orderto better understand what part of the representation is context-dependent. Wefinally compare the verb type vectors learned by the parser to verb type vectorslearned with a language modeling objective. We expect the vectors learned bythe parser to encode information about agreement and transitivity to a greaterextent than the vectors learned using a language modelling objective. I explainthis in more detail in Section 5.3.3.
Collecting FMVs such as in Figure 5.5 in UD treebanks is straightforward:verbs are annotated with a feature called VerbForm which has the value Finif the verb is finite. We find candidates using the feature VerbForm=Fin andonly keep those that are not dependent of a copula or an auxiliary dependencyrelation to make sure they are not part of a larger verbal construction.
We can collect AVCs in the UD and in the MS training data using the firstpart of the transformation and backtransformation algorithms respectively whichI defined in Section 5.2.1. We scan the sentence left-to-right, looking for aux-iliary dependency relations and collecting information about what is the outer-most auxiliary and what is the main verb. When we have our set of FMVs andAVCs, we can create our task data sets.
111
![Page 112: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/112.jpg)
vi
xie(wi)
ce(wi)
vi = BiLST M(x1:n, i)xi = [e(wi);ce(wi)]
![Page 113: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/113.jpg)
It seems interesting to find out whether or not a composition function can beuseful to model the specific case of transfer relations such as is found in AVCs.For this reason, we also investigate the composed representation of AVCs.
We are therefore interested in finding out whether or not an LSTM trainedwith a parsing objective can learn the notion of dissociated nucleus as well aswhether or not a recursive composition function can help to learn this notion.As just mentioned, the head of an AVC in UD is a non-finite main verb, whichwe will refer to as NFMV, as depicted in Figure 5.6. The head of an AVC in MSis the outermost auxiliary, which we will refer to as the main auxiliary MAUX,as also depicted in that figure. We therefore look at NFMV and MAUX tokenvectors for the respective representation schemes and consider two definitionsof these: One where we use the BiLSTM encoding of the main verb token vi,and the other where we construct a subtree vector ct
i by recursively composingthe representation of AVCs as auxiliaries get attached to their main verb, asin Equation 4.5, repeated in Equation 5.3. When training the parser, we con-catenate this composed vector to a vector of the head of the subtree to form vi.In Chapter 4, we used two different composition functions, one using a sim-ple recurrent cell and one using an LSTM cell. We saw that the one using anLSTM cell performed better. However, in this set of experiments, we only dorecursive compositions over a limited part of the subtree: only between auxil-iaries and NFMVs. This means that the LSTM would only pass through twostates in most cases, and never more than 4.8 This does not allow us to learnproper weights for the input, output and forget gates. An RNN cell seems moreappropriate here and we only use that.
cti = tanh(W [ct−1
i ;d;r]+b) (5.3)
LSTM f
LSTM b
concat concat concat
LSTM f
LSTM b
concat
LSTM f
LSTM b
LSTM f
LSTM b
XX
V I Vdone VthatVhave
X I X have done thatI h a v e d o n e t h a t
e(done) e(that)e(I) e(have)
NFMVMAUX
tok
char
type
tok
chartype
Figure 5.7. Example AVC with vectors of interest.
8The maximum number of auxiliaries in one AVC in our dataset is 3.
113
![Page 114: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/114.jpg)
LSTM f
LSTM b
LSTM f
LSTM b
LSTM f
LSTM b
concat concat concatconcat
LSTM f
LSTM b
I
e(I)
XX
Vdid Vthat V !
X
V I
X did that !d i d t h a t !
e(did) e(that) e(!)
FMV
toktok
char
type
punct
I
![Page 115: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/115.jpg)
![Page 116: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/116.jpg)
![Page 117: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/117.jpg)
bas rc
![Page 118: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/118.jpg)
![Page 119: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/119.jpg)
δ
p < .05 p < .01δ δ
![Page 120: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/120.jpg)
![Page 121: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/121.jpg)
δp < .05
p < .01δ δ
p < .01
![Page 122: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/122.jpg)
have donedid !
FMV NFMVMAUXpunct
did !
FMV NFMVMAUX
have done
punct
![Page 123: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/123.jpg)
![Page 124: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/124.jpg)
δ
+cδ
p < .05∗∗= p < .01
![Page 125: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/125.jpg)
![Page 126: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/126.jpg)
le lel tanh
l
le = tanh(Wl +b)
![Page 127: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/127.jpg)
�
�
�
�
![Page 128: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/128.jpg)
te(wi) wi
xi = [pe(wi);ce(wi);e(ti); te(wi)]
![Page 129: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/129.jpg)
![Page 130: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/130.jpg)
| < μ−σ < | < μ− < μ < μ+ < | < μ+σ < |
![Page 131: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/131.jpg)
μ ± , = σ/√
N
μ ±σ
![Page 132: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/132.jpg)
![Page 133: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/133.jpg)
concat concat concat concatconcat
concat
concat
MLP
LSTM f
LSTM b
LSTM f
LSTM b
LSTM f
LSTM b
LSTM f
LSTM b
LSTM f
LSTM b
XX
Vthe Vfox Vjumped Vroot
X
Vbrown
X the X brown fox jumped root
(score(LEFT−ARC),score(RIGHT−ARC),score(SHIFT), score(SWAP))
the
h et
CfCf Cf
CbCbCb
State
Word
Character
�
�
te(wi) wiwi
xi = [e(wi);ce(wi); te(wi)]
�
�
ch j = [e(ch j); te(wi)]
![Page 134: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/134.jpg)
� �
φ(c) = [vs1 ;vs0 ;vb0 ; te(s1,s0,b0)]
![Page 135: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/135.jpg)
![Page 136: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/136.jpg)
![Page 137: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/137.jpg)
� �
� � �
�
� �
� �
� �
� �
� � �
�
� �
�
� � �
� � �
� �
� �
�
� � �
� � �
� �
� �
� � �
�
� � �
� �
� �
�
![Page 138: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/138.jpg)
δ
δ
� �
� �
�� �
� �
� �
�
�
�
�
p < 0.01
![Page 139: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/139.jpg)
� �
![Page 140: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/140.jpg)
� � �
� � �
�
� �
�
� �
� �
� �
� �
� �
� � �
� � �
� � �
� �
� � �
�
� �
� � �
�
� �
�
� �
� � �
� �
� �
�
![Page 141: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/141.jpg)
![Page 142: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/142.jpg)
![Page 143: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/143.jpg)
![Page 144: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/144.jpg)
![Page 145: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/145.jpg)
![Page 146: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/146.jpg)
![Page 147: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/147.jpg)
![Page 148: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/148.jpg)
![Page 149: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/149.jpg)
![Page 150: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/150.jpg)
![Page 151: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/151.jpg)
![Page 152: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/152.jpg)
![Page 153: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/153.jpg)
![Page 154: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/154.jpg)
![Page 155: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/155.jpg)
![Page 156: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/156.jpg)
![Page 157: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/157.jpg)
![Page 158: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/158.jpg)
![Page 159: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/159.jpg)
![Page 160: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/160.jpg)
![Page 161: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/161.jpg)
![Page 162: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/162.jpg)
![Page 163: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/163.jpg)
![Page 164: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/164.jpg)
![Page 165: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/165.jpg)
![Page 166: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/166.jpg)
![Page 167: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/167.jpg)
![Page 168: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/168.jpg)
![Page 169: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/169.jpg)
![Page 170: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/170.jpg)
![Page 171: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/171.jpg)
∗ = p < .05∗∗= p < .01
δ δ
![Page 172: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/172.jpg)
δp < .05
p < .01δ δ
![Page 173: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/173.jpg)
xi = [pe(wi);ce(wi);e(ti)]xi = [pe(wi);ce(wi);e(ti); tbe(wi)]
![Page 174: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/174.jpg)
![Page 175: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/175.jpg)
![Page 176: ACTA UNIVERSITATIS UPSALIENSIS Studia …uu.diva-portal.org/smash/get/diva2:1357373/FULLTEXT01.pdf+RNj3Njc S BNjaR0n,jCRNYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYSe](https://reader030.fdocuments.net/reader030/viewer/2022040405/5e9ba871605872013a102d1d/html5/thumbnails/176.jpg)
ACTA UNIVERSITATIS UPSALIENSIS Studia Linguistica Upsaliensia
Editors: Michael Dunn and Joakim Nivre 1. Jörg Tiedemann, Recycling translations. Extraction of lexical data from parallel
corpora and their application in natural language processing. 2003. 2. Agnes Edling, Abstraction and authority in textbooks. The textual paths towards
specialized language. 2006. 3. Åsa af Geijerstam, Att skriva i naturorienterande ämnen i skolan. 2006. 4. Gustav Öquist, Evaluating Readability on Mobile Devices. 2006. 5. Jenny Wiksten Folkeryd, Writing with an Attitude. Appraisal and student texts
in the school subject of Swedish. 2006. 6. Ingrid Björk, Relativizing linguistic relativity. Investigating underlying assump-
tions about language in the neo-Whorfian literature. 2008. 7. Joakim Nivre, Mats Dahllöf and Beáta Megyesi, Resourceful Language Tech-
nology. Festschrift in Honor of Anna Sågvall Hein. 2008. 8. Anju Saxena & Åke Viberg, Multilingualism. Proceedings of the 23rd Scandinavi-
an Conference of Linguistics. 2009. 9. Markus Saers, Translation as Linear Transduction. Models and Algorithms for
Efficient Learning in Statistical Machine Translation. 2011. 10. Ulrika Serrander, Bilingual lexical processing in single word production. Swedish
learners of Spanish and the effects of L2 immersion. 2011. 11. Mattias Nilsson, Computational Models of Eye Movements in Reading : A Data-
Driven Approach to the Eye-Mind Link. 2012. 12. Luying Wang, Second Language Acquisition of Mandarin Aspect Markers by
Native Swedish Adults. 2012. 13. Farideh Okati, The Vowel Systems of Five Iranian Balochi Dialects. 2012. 14. Oscar Täckström, Predicting Linguistic Structure with Incomplete and Cross-
Lingual Supervision. 2013. 15. Christian Hardmeier, Discourse in Statistical Machine Translation. 2014. 16. Mojgan Seraji, Morphosyntactic Corpora and Tools for Persian. 2015. 17. Eva Pettersson, Spelling Normalisation and Linguistic Analysis of Historical Text
for Information Extraction. 2016. 18. Marie Dubremetz, Detecting Rhetorical Figures Based on Repetition of Words:
Chiasmus, Epanaphora, Epiphora. 2017. 19. Josefin Lindgren, Developing narrative competence: Swedish, Swedish-German
and Swedish-Turkish children aged 4–6. 2018. 20. Vera Wilhelmsen, A Linguistic Description of Mbugwe with Focus on Tone and
Verbal Morphology. 2018. 21. Yan Shao, Segmenting and Tagging Text with Neural Networks. 2018. 22. Ali Basirat, Principal Word Vectors. 2018. 23. Marc Tang, A typology of classifiers and gender. 2018. 24. Miryam de Lhoneux, Linguistically Informed Neural Dependency Parsing for
Typologically Diverse Languages. 2019.