Motoki Satosato-motoki.com/pdf/nlp2018_sato-suzuki.pdf · 24 %) !("$: Positive!("$: Negative There...

��

"��+1, -�02, ��.*1,3, � .�1,3

��

1 : )!�,(��#�� (NAIST) � %&'2 : NTT��(��/%&�3 : ��%&�$�� %&��AIP

��

1. ��– '��(��'�� – �%� ��'��(�

2. ��"$3. ��4. �#5. ��&6. ��!�

2

��l ��(��)��

[Szegedy et al., 2014, Goodfellow et al .,2015]

3l [Szegedy et al., 2014] : “Intriguing properties of neural networks.”, ICLR 2014.l [Goodfellow et al.,2015]:“Explaining and Harnessing Adversarial Examples”, ICLR 2015.

[Goodfellow et al., 2015]��

�� (Adversarial Perturbation)

��(Adversarial Example)

��l ��(��)��

[Szegedy et al., 2014, Goodfellow et al .,2015]

4[Goodfellow et al., 2015]��

�� (Adversarial Perturbation)

��(Adversarial Example)

��

�#�� l �#��– B �� ( 2 – �# 5 %�� # ��$��

5

l ��– B "� – "��!�� 0 ) +– !�� %��

�$��

G

�34�;'

6

95�$ ,*�$

��

��

��

��

�� )��

�� )��

,*�$��

1. <&�/1%��"�+:/-��( ��72�.�6��)

2. 8!#��=0�� (��

��

1. $��– &��'��&�� – �#� ��&��'�

2. ��3. ��4. �"5. ��%6. ��!�

7

LM

l S– E A B P S

2 . 2 0 ,

– J2 7 . 28 ,

– T L S ][. 1 ,

l N BP M

8

) ( ( (

]

l M AV f T Goodfellow et al .,2015]

9

M , 1,2 ) ,1 1 0

M M A Ml A , ( ) ,

[Miyato et al., 2016]

l a[ A T V M (1 ,7 ��– [A TA eT ;

, 1,2 ) ,1 1 0 ) 7

d (1 ,7 �� A ;

2 , ��

l Takeru Miyato, Andrew M Dai, and Ian Goodfellow, ICLR 2017

“Adversarial training methods for semi-supervised text classification.”

l :LSTM + Pre-Training (Language Model) + Adversarial Training

1070 10 1

�� (�: 1.0)

, ��

11

[ :] :

]

�[ : � ��

� 0 7

7

� 7, L2M

�

�M1 2

2 , ��

l Takeru Miyato, Andrew M Dai, and Ian Goodfellow, ICLR 2017

“Adversarial training methods for semi-supervised text classification.”

l :LSTM + Pre-Training (Language Model) + Adversarial Training

1270 10 1

��

1. (!��– *��+��*�� – �'�" ��*��+�

2. ��$&3. ��4. �%5. ��)6. ��#�

13

14

��

��

��

��

��

��

��, ��

" ��

15

)��# (&� ��

)��!�

�(&��

��α��)��

�(&� ��)�� '%�� a�!

�*$�α��

��α�(&�� d�)��

:

16

: :

��

��

��

��

• a 0

• .

��

1. & ��– (��)��(�� – �%�!��(��)�

2. ��#$3. ��4. ��5. ��'6. ��"�

17

l ( 1 2 01)– : , �� – 7 )(

(Train : 21,246 Dev: 3,754 Test: 25,000 Unlabeled: 50,000)

18((

19• .• .

(Semi-supervised : )

��

1. ' ��– (��)��(�� – �&�!��(��)�

2. ��#%3. ��4. �$5. ��6. ��"�

20

21

��( ) ��

��(2

��

�� (��( L

.

��

�� .

��

22

( )

(2

(( L

.

.

%,�*!�� )�'$

l."�� , .

23

This movie turned out to be better than I had expected it to be Some parts were pretty funny It was nice to have a movie with a new plot <eos>

��) %,&(: Positive

This movie turned out to be worse than I had expected it to be Some parts were pretty funny It was nice to have a movie with a new plot <eos>

-�� %,&(: Negative

�� )�#+�*!� �

!(�&��% #

24

��%

!("$: Positive)��

!("$: Negative

There is really but one thing to say about this sorry movie It should never have been made The first one one of my favourites An American Werewolf in London is a great movie with a good plot good actors and good FX But this one It stinks to heaven with a cry of helplessness <eos>

There is really but one thing to say about that sorry movie It should never have been made The first one one of my favourites An American Werewolf in London is a great movie with a good plot good actors and good FX But this one It stinks to heaven with a cry of helplessness <eos>

�� % �'�&��

� ��

25

�� ,��

�� ,��

��

1. !��– #��$��#��– � ��#��$�

2. ��3. �� 4. ��5. �� "6. ��

26

��

l ��

l . .l �� .l � ��

27

Motoki Satosato-motoki.com/pdf/nlp2018_sato-suzuki.pdf · 24 %) !("$: Positive!("$: Negative There...

Documents

Transcript of Motoki Satosato-motoki.com/pdf/nlp2018_sato-suzuki.pdf · 24 %) !("$: Positive!("$: Negative There...