Chengyu Cloze Test - University of Rochestertetreaul/Presentations-and-Posters/0516.pdfChengyu Cloze...
1
Chengyu Cloze Test 1. INTRODUCTION • Chengyu (౮) is a special type of Chinese idiom. • 96% Chengyus consist of four characters each. Chengyus were mainly created from ancient stories, literature and sayings. • A typical way to measure a Chinese learner's Chengyu knowledge is Cloze Test. 2. METHOD 3. EXPERIMENT 4. ANALYSIS Example Zhiying Jiang, Boliang Zhang, Lifu Huang, Heng Ji Cloze Test Coherence Checking in College Entrance Exam Human 70% 42.3% System 89.5% 35.7% 我们喜欢⽤经济去控制⼀个国家的命脉，⽤信 仰去控制⼀个种族，⽤利益让别⼈为我们 _____。 We like to use economy to control a country’s faith, use belief to control a race and use profit to control others so that they can ____ for us. credit to Birmingham Museum and Art Gallery’s photostream Answer: ⽕中取栗 (pull chestnuts from the embers) This is a Chengyu whose origin is from foreign literature. From the 17 century French fabulist Jean de la Fontaine's "The Monkey and the Cat". Bertrand the monkey persuades Raton the cat to pull chestnuts from the embers amongst which they are roasting, promising nothing. As the cat scoops them from the fire one by one, burning his paw in the process, the monkey gobbles them up. It's used to describe a person used unwittingly or unwillingly by another to accomplish the other's own purpose with his own risk but gets nothing. • We propose a neural approach to incorporate the definition of each Chengyu as background knowledge. • We apply a Bi-LSTM network to encode each word with a contextual embedding. • We further compare the representations of the Chengyu definition and the contextual embedding of each word in the query, and take the weighted sum of the query word contextual embeddings to determine the probability score of the candidate Chengyu Figure1: Architecture Overview p i = sof tmax(W T β R) ↵ i = exp(e i ) P n i=1 exp(e i ) e i = d T · W ↵ · h i R = n X i=1 ↵ i · h i Attention Score Weighted Sum Predicted Probability We perform two tests: (1) cloze test: for each sentence, we take out the ground-truth Chengyu, and let the system select a Chengyu from four candidates. (2) coherence checking in college entrance exam: we collected 14 problem sets, where each problem set consists of four sentences including Chengyus. The system select the sentence that contains the most appropriate Chengyu. Table1: System and Human Accuracy Comparison Table 1 shows our approach achieves comparable performance as human experts. For 18% of our system recommended Chengyus which don't exactly match the ground truth, they are also acceptable choices for the given query contexts. Positive Example 这事已势不可遏，任何想阻挡他的⼈都如 ____，简直是不⾃量⼒。 This event is unstoppable, anyone who tries to stop it will be like ____, almost not recognizing his/her own limited power. System: 蚍蜉撼树 (an ant shaking a tree, to describe one fails to recognize one’s limited power) Analysis: The definition significantly enriches the semantic meanings of Chengyu itself. 蚍蜉撼树 (an ant shaking a tree) is a metaphor to describe ⾃不量⼒(fail to recognize one's own limited power). Negative Example 村上春树____，29岁才写他的第⼀部作品。 Haruki Murakami ____, he was already at age 29 when he wrote his first works. Ground Truth:⼤器晚成 (takes a long time to make a great instrument) Analysis: We need to know “age 29” is relatively late to produce the first works for a writer.
Transcript of Chengyu Cloze Test - University of Rochestertetreaul/Presentations-and-Posters/0516.pdfChengyu Cloze...
Chengyu Cloze Test
1. INTRODUCTION• Chengyu ( ) is a special type of Chinese
idiom. • 96% Chengyus consist of four characters
each. Chengyus were mainly created from ancient stories, literature and sayings.
• A typical way to measure a Chinese learner's Chengyu knowledge is Cloze Test.
2. METHOD
3. EXPERIMENT
4. ANALYSIS
Example
Zhiying Jiang, Boliang Zhang, Lifu Huang, Heng Ji
Cloze Test Coherence Checking in College Entrance Exam
Human 70% 42.3%System 89.5% 35.7%
我们喜欢⽤经济去控制⼀个国家的命脉，⽤信仰去控制⼀个种族，⽤利益让别⼈为我们_____。 We like to use economy to control a country’s faith, use belief to control a race and use profit to control others so that they can ____ for us.
credit to Birmingham Museum and Art Gallery’s photostream
Answer: ⽕中取栗 (pull chestnuts from the embers)This is a Chengyu whose origin is from foreign literature. From the 17 century French fabulist Jean de la Fontaine's "The Monkey and the Cat". Bertrand the monkey persuades Raton the cat to pull chestnuts from the embers amongst which they are roasting, promising nothing. As the cat scoops them from the fire one by one, burning his paw in the process, the
monkey gobbles them up. It's used to describe a person used unwittingly or unwillingly by another to accomplish the other's own purpose with his own risk but gets nothing.
• We propose a neural approach to incorporate the definition of each Chengyu as background knowledge.
• We apply a Bi-LSTM network to encode each word with a contextual embedding.
• We further compare the representations of the Chengyu definition and the contextual embedding of each word in the query, and take the weighted sum of the query word contextual embeddings to determine the probability score of the candidate Chengyu
Figure1: Architecture Overview
pi = softmax(WT�R)
<latexit sha1_base64="NkZx5sr718SuMocg30kp5GQNZww=">AAACNnicbVBNSwMxFMz6bf2qevQSLIJeyq4IehFEL16EKq0tdGvJpm9taLK7JG/Fsuyv8uLv8ObFgyJe/QmmtYK2DgSGmfd4mQkSKQy67rMzNT0zOze/sFhYWl5ZXSuub1ybONUcajyWsW4EzIAUEdRQoIRGooGpQEI96J0N/PodaCPiqIr9BFqK3UYiFJyhldrFC18x7GqV+Qj3GIRZkudtQY+piUNU7H533K9bP/MDQJbTG5pVc/rjXOV77WLJLbtD0EnijUiJjFBpF5/8TsxTBRFyyYxpem6CrYxpFFxCXvBTAwnjPXYLTUsjpsC0smHsnO5YpUPDWNsXIR2qvzcypozpq8BODkKYcW8g/uc1UwyPWpmIkhQh4t+HwlRSjOmgQ9oRGjjKviWMa2H/SnmXacbRNl2wJXjjkSfJ9X7Zc8ve5UHp5HRUxwLZIttkl3jkkJyQc1IhNcLJA3kmr+TNeXRenHfn43t0yhntbJI/cD6/AGeKrhQ=</latexit><latexit sha1_base64="NkZx5sr718SuMocg30kp5GQNZww=">AAACNnicbVBNSwMxFMz6bf2qevQSLIJeyq4IehFEL16EKq0tdGvJpm9taLK7JG/Fsuyv8uLv8ObFgyJe/QmmtYK2DgSGmfd4mQkSKQy67rMzNT0zOze/sFhYWl5ZXSuub1ybONUcajyWsW4EzIAUEdRQoIRGooGpQEI96J0N/PodaCPiqIr9BFqK3UYiFJyhldrFC18x7GqV+Qj3GIRZkudtQY+piUNU7H533K9bP/MDQJbTG5pVc/rjXOV77WLJLbtD0EnijUiJjFBpF5/8TsxTBRFyyYxpem6CrYxpFFxCXvBTAwnjPXYLTUsjpsC0smHsnO5YpUPDWNsXIR2qvzcypozpq8BODkKYcW8g/uc1UwyPWpmIkhQh4t+HwlRSjOmgQ9oRGjjKviWMa2H/SnmXacbRNl2wJXjjkSfJ9X7Zc8ve5UHp5HRUxwLZIttkl3jkkJyQc1IhNcLJA3kmr+TNeXRenHfn43t0yhntbJI/cD6/AGeKrhQ=</latexit><latexit sha1_base64="NkZx5sr718SuMocg30kp5GQNZww=">AAACNnicbVBNSwMxFMz6bf2qevQSLIJeyq4IehFEL16EKq0tdGvJpm9taLK7JG/Fsuyv8uLv8ObFgyJe/QmmtYK2DgSGmfd4mQkSKQy67rMzNT0zOze/sFhYWl5ZXSuub1ybONUcajyWsW4EzIAUEdRQoIRGooGpQEI96J0N/PodaCPiqIr9BFqK3UYiFJyhldrFC18x7GqV+Qj3GIRZkudtQY+piUNU7H533K9bP/MDQJbTG5pVc/rjXOV77WLJLbtD0EnijUiJjFBpF5/8TsxTBRFyyYxpem6CrYxpFFxCXvBTAwnjPXYLTUsjpsC0smHsnO5YpUPDWNsXIR2qvzcypozpq8BODkKYcW8g/uc1UwyPWpmIkhQh4t+HwlRSjOmgQ9oRGjjKviWMa2H/SnmXacbRNl2wJXjjkSfJ9X7Zc8ve5UHp5HRUxwLZIttkl3jkkJyQc1IhNcLJA3kmr+TNeXRenHfn43t0yhntbJI/cD6/AGeKrhQ=</latexit><latexit sha1_base64="NkZx5sr718SuMocg30kp5GQNZww=">AAACNnicbVBNSwMxFMz6bf2qevQSLIJeyq4IehFEL16EKq0tdGvJpm9taLK7JG/Fsuyv8uLv8ObFgyJe/QmmtYK2DgSGmfd4mQkSKQy67rMzNT0zOze/sFhYWl5ZXSuub1ybONUcajyWsW4EzIAUEdRQoIRGooGpQEI96J0N/PodaCPiqIr9BFqK3UYiFJyhldrFC18x7GqV+Qj3GIRZkudtQY+piUNU7H533K9bP/MDQJbTG5pVc/rjXOV77WLJLbtD0EnijUiJjFBpF5/8TsxTBRFyyYxpem6CrYxpFFxCXvBTAwnjPXYLTUsjpsC0smHsnO5YpUPDWNsXIR2qvzcypozpq8BODkKYcW8g/uc1UwyPWpmIkhQh4t+HwlRSjOmgQ9oRGjjKviWMa2H/SnmXacbRNl2wJXjjkSfJ9X7Zc8ve5UHp5HRUxwLZIttkl3jkkJyQc1IhNcLJA3kmr+TNeXRenHfn43t0yhntbJI/cD6/AGeKrhQ=</latexit>
↵i =exp(ei)Pni=1 exp(ei)
<latexit sha1_base64="o0TmM3qxep7gZ7SLzxBSw8tf/uA=">AAACHHicbVDLSsNAFJ34rPVVdelmsAh1UxIVdFMounFZwT6gqWEyvWmHTiZhZiKWkA9x46+4caGIGxeCf+P0AWrrgQuHc+7l3nv8mDOlbfvLWlhcWl5Zza3l1zc2t7YLO7sNFSWSQp1GPJItnyjgTEBdM82hFUsgoc+h6Q8uR37zDqRikbjRwxg6IekJFjBKtJG8wolLeNwnHqu4gSQ0deE+LoHHjrLUVUnopaziZLepyPCP4xWKdtkeA88TZ0qKaIqaV/hwuxFNQhCacqJU27Fj3UmJ1IxyyPJuoiAmdEB60DZUkBBUJx0/l+FDo3RxEElTQuOx+nsiJaFSw9A3nSHRfTXrjcT/vHaig/NOykScaBB0sihIONYRHiWFu0wC1XxoCKGSmVsx7RMTkjZ55k0IzuzL86RxXHbssnN9WqxeTOPIoX10gErIQWeoiq5QDdURRQ/oCb2gV+vRerberPdJ64I1ndlDf2B9fgM1D6IJ</latexit><latexit sha1_base64="o0TmM3qxep7gZ7SLzxBSw8tf/uA=">AAACHHicbVDLSsNAFJ34rPVVdelmsAh1UxIVdFMounFZwT6gqWEyvWmHTiZhZiKWkA9x46+4caGIGxeCf+P0AWrrgQuHc+7l3nv8mDOlbfvLWlhcWl5Zza3l1zc2t7YLO7sNFSWSQp1GPJItnyjgTEBdM82hFUsgoc+h6Q8uR37zDqRikbjRwxg6IekJFjBKtJG8wolLeNwnHqu4gSQ0deE+LoHHjrLUVUnopaziZLepyPCP4xWKdtkeA88TZ0qKaIqaV/hwuxFNQhCacqJU27Fj3UmJ1IxyyPJuoiAmdEB60DZUkBBUJx0/l+FDo3RxEElTQuOx+nsiJaFSw9A3nSHRfTXrjcT/vHaig/NOykScaBB0sihIONYRHiWFu0wC1XxoCKGSmVsx7RMTkjZ55k0IzuzL86RxXHbssnN9WqxeTOPIoX10gErIQWeoiq5QDdURRQ/oCb2gV+vRerberPdJ64I1ndlDf2B9fgM1D6IJ</latexit><latexit sha1_base64="o0TmM3qxep7gZ7SLzxBSw8tf/uA=">AAACHHicbVDLSsNAFJ34rPVVdelmsAh1UxIVdFMounFZwT6gqWEyvWmHTiZhZiKWkA9x46+4caGIGxeCf+P0AWrrgQuHc+7l3nv8mDOlbfvLWlhcWl5Zza3l1zc2t7YLO7sNFSWSQp1GPJItnyjgTEBdM82hFUsgoc+h6Q8uR37zDqRikbjRwxg6IekJFjBKtJG8wolLeNwnHqu4gSQ0deE+LoHHjrLUVUnopaziZLepyPCP4xWKdtkeA88TZ0qKaIqaV/hwuxFNQhCacqJU27Fj3UmJ1IxyyPJuoiAmdEB60DZUkBBUJx0/l+FDo3RxEElTQuOx+nsiJaFSw9A3nSHRfTXrjcT/vHaig/NOykScaBB0sihIONYRHiWFu0wC1XxoCKGSmVsx7RMTkjZ55k0IzuzL86RxXHbssnN9WqxeTOPIoX10gErIQWeoiq5QDdURRQ/oCb2gV+vRerberPdJ64I1ndlDf2B9fgM1D6IJ</latexit><latexit sha1_base64="o0TmM3qxep7gZ7SLzxBSw8tf/uA=">AAACHHicbVDLSsNAFJ34rPVVdelmsAh1UxIVdFMounFZwT6gqWEyvWmHTiZhZiKWkA9x46+4caGIGxeCf+P0AWrrgQuHc+7l3nv8mDOlbfvLWlhcWl5Zza3l1zc2t7YLO7sNFSWSQp1GPJItnyjgTEBdM82hFUsgoc+h6Q8uR37zDqRikbjRwxg6IekJFjBKtJG8wolLeNwnHqu4gSQ0deE+LoHHjrLUVUnopaziZLepyPCP4xWKdtkeA88TZ0qKaIqaV/hwuxFNQhCacqJU27Fj3UmJ1IxyyPJuoiAmdEB60DZUkBBUJx0/l+FDo3RxEElTQuOx+nsiJaFSw9A3nSHRfTXrjcT/vHaig/NOykScaBB0sihIONYRHiWFu0wC1XxoCKGSmVsx7RMTkjZ55k0IzuzL86RxXHbssnN9WqxeTOPIoX10gErIQWeoiq5QDdURRQ/oCb2gV+vRerberPdJ64I1ndlDf2B9fgM1D6IJ</latexit>
ei = dT ·W↵ · hi<latexit sha1_base64="rrQ4mAY2khR6RJEdtFS7XK1oZZo=">AAACHXicbVDLSsNAFJ34tr6qLt0MFsFVSUTQjVB041KhL2hqmExu7ODkwcyNWEJ+xI2/4saFIi7ciH/jtM1CWw8MHM45l7n3+KkUGm3725qbX1hcWl5Zraytb2xuVbd32jrJFIcWT2Siuj7TIEUMLRQooZsqYJEvoePfXYz8zj0oLZK4icMU+hG7jUUoOEMjedVj8MRZcJM3C5cHCVI3YjhQUe4iPKAf5p2i8HKXyXTAysTAE161ZtftMegscUpSIyWuvOqnGyQ8iyBGLpnWPcdOsZ8zhYJLKCpupiFl/I7dQs/QmEWg+/n4uoIeGCWgYaLMi5GO1d8TOYu0Hka+SY6W19PeSPzP62UYnvZzEacZQswnH4WZpJjQUVU0EAo4yqEhjCthdqV8wBTjaAqtmBKc6ZNnSfuo7th15/q41jgv61ghe2SfHBKHnJAGuSRXpEU4eSTP5JW8WU/Wi/VufUyic1Y5s0v+wPr6AZaZo3U=</latexit><latexit sha1_base64="rrQ4mAY2khR6RJEdtFS7XK1oZZo=">AAACHXicbVDLSsNAFJ34tr6qLt0MFsFVSUTQjVB041KhL2hqmExu7ODkwcyNWEJ+xI2/4saFIi7ciH/jtM1CWw8MHM45l7n3+KkUGm3725qbX1hcWl5Zraytb2xuVbd32jrJFIcWT2Siuj7TIEUMLRQooZsqYJEvoePfXYz8zj0oLZK4icMU+hG7jUUoOEMjedVj8MRZcJM3C5cHCVI3YjhQUe4iPKAf5p2i8HKXyXTAysTAE161ZtftMegscUpSIyWuvOqnGyQ8iyBGLpnWPcdOsZ8zhYJLKCpupiFl/I7dQs/QmEWg+/n4uoIeGCWgYaLMi5GO1d8TOYu0Hka+SY6W19PeSPzP62UYnvZzEacZQswnH4WZpJjQUVU0EAo4yqEhjCthdqV8wBTjaAqtmBKc6ZNnSfuo7th15/q41jgv61ghe2SfHBKHnJAGuSRXpEU4eSTP5JW8WU/Wi/VufUyic1Y5s0v+wPr6AZaZo3U=</latexit><latexit sha1_base64="rrQ4mAY2khR6RJEdtFS7XK1oZZo=">AAACHXicbVDLSsNAFJ34tr6qLt0MFsFVSUTQjVB041KhL2hqmExu7ODkwcyNWEJ+xI2/4saFIi7ciH/jtM1CWw8MHM45l7n3+KkUGm3725qbX1hcWl5Zraytb2xuVbd32jrJFIcWT2Siuj7TIEUMLRQooZsqYJEvoePfXYz8zj0oLZK4icMU+hG7jUUoOEMjedVj8MRZcJM3C5cHCVI3YjhQUe4iPKAf5p2i8HKXyXTAysTAE161ZtftMegscUpSIyWuvOqnGyQ8iyBGLpnWPcdOsZ8zhYJLKCpupiFl/I7dQs/QmEWg+/n4uoIeGCWgYaLMi5GO1d8TOYu0Hka+SY6W19PeSPzP62UYnvZzEacZQswnH4WZpJjQUVU0EAo4yqEhjCthdqV8wBTjaAqtmBKc6ZNnSfuo7th15/q41jgv61ghe2SfHBKHnJAGuSRXpEU4eSTP5JW8WU/Wi/VufUyic1Y5s0v+wPr6AZaZo3U=</latexit><latexit sha1_base64="rrQ4mAY2khR6RJEdtFS7XK1oZZo=">AAACHXicbVDLSsNAFJ34tr6qLt0MFsFVSUTQjVB041KhL2hqmExu7ODkwcyNWEJ+xI2/4saFIi7ciH/jtM1CWw8MHM45l7n3+KkUGm3725qbX1hcWl5Zraytb2xuVbd32jrJFIcWT2Siuj7TIEUMLRQooZsqYJEvoePfXYz8zj0oLZK4icMU+hG7jUUoOEMjedVj8MRZcJM3C5cHCVI3YjhQUe4iPKAf5p2i8HKXyXTAysTAE161ZtftMegscUpSIyWuvOqnGyQ8iyBGLpnWPcdOsZ8zhYJLKCpupiFl/I7dQs/QmEWg+/n4uoIeGCWgYaLMi5GO1d8TOYu0Hka+SY6W19PeSPzP62UYnvZzEacZQswnH4WZpJjQUVU0EAo4yqEhjCthdqV8wBTjaAqtmBKc6ZNnSfuo7th15/q41jgv61ghe2SfHBKHnJAGuSRXpEU4eSTP5JW8WU/Wi/VufUyic1Y5s0v+wPr6AZaZo3U=</latexit>
R =nX
i=1
↵i · hi
<latexit sha1_base64="qBxIvjvoRT7yjuxKgtdOX85iYXA=">AAACHXicbVBNSwMxFMz6bf2qevQSLIKnsiuCXgqiF49VbCt065JNs20wyS7JW7Es+0e8+Fe8eFDEgxfx35ht96DWgcAw8x4vM2EiuAHX/XJmZufmFxaXlisrq2vrG9XNrbaJU01Zi8Yi1tchMUxwxVrAQbDrRDMiQ8E64e1Z4XfumDY8VlcwSlhPkoHiEacErBRUD31JYKhl5gO7hzDKLvO84ZtUBhlvePlNpnLsE5EMScB92o8BDwMeVGtu3R0DTxOvJDVUohlUP/x+TFPJFFBBjOl6bgK9jGjgVLC84qeGJYTekgHrWqqIZKaXjdPleM8qfRzF2j4FeKz+3MiINGYkQztZZDF/vUL8z+umEB33Mq6SFJiik0NRKjDEuKgK97lmFMTIEkI1t3/FdEg0oWALrdgSvL+Rp0n7oO65de/isHZyWtaxhHbQLtpHHjpCJ+gcNVELUfSAntALenUenWfnzXmfjM445c42+gXn8xtWXKND</latexit><latexit sha1_base64="qBxIvjvoRT7yjuxKgtdOX85iYXA=">AAACHXicbVBNSwMxFMz6bf2qevQSLIKnsiuCXgqiF49VbCt065JNs20wyS7JW7Es+0e8+Fe8eFDEgxfx35ht96DWgcAw8x4vM2EiuAHX/XJmZufmFxaXlisrq2vrG9XNrbaJU01Zi8Yi1tchMUxwxVrAQbDrRDMiQ8E64e1Z4XfumDY8VlcwSlhPkoHiEacErBRUD31JYKhl5gO7hzDKLvO84ZtUBhlvePlNpnLsE5EMScB92o8BDwMeVGtu3R0DTxOvJDVUohlUP/x+TFPJFFBBjOl6bgK9jGjgVLC84qeGJYTekgHrWqqIZKaXjdPleM8qfRzF2j4FeKz+3MiINGYkQztZZDF/vUL8z+umEB33Mq6SFJiik0NRKjDEuKgK97lmFMTIEkI1t3/FdEg0oWALrdgSvL+Rp0n7oO65de/isHZyWtaxhHbQLtpHHjpCJ+gcNVELUfSAntALenUenWfnzXmfjM445c42+gXn8xtWXKND</latexit><latexit sha1_base64="qBxIvjvoRT7yjuxKgtdOX85iYXA=">AAACHXicbVBNSwMxFMz6bf2qevQSLIKnsiuCXgqiF49VbCt065JNs20wyS7JW7Es+0e8+Fe8eFDEgxfx35ht96DWgcAw8x4vM2EiuAHX/XJmZufmFxaXlisrq2vrG9XNrbaJU01Zi8Yi1tchMUxwxVrAQbDrRDMiQ8E64e1Z4XfumDY8VlcwSlhPkoHiEacErBRUD31JYKhl5gO7hzDKLvO84ZtUBhlvePlNpnLsE5EMScB92o8BDwMeVGtu3R0DTxOvJDVUohlUP/x+TFPJFFBBjOl6bgK9jGjgVLC84qeGJYTekgHrWqqIZKaXjdPleM8qfRzF2j4FeKz+3MiINGYkQztZZDF/vUL8z+umEB33Mq6SFJiik0NRKjDEuKgK97lmFMTIEkI1t3/FdEg0oWALrdgSvL+Rp0n7oO65de/isHZyWtaxhHbQLtpHHjpCJ+gcNVELUfSAntALenUenWfnzXmfjM445c42+gXn8xtWXKND</latexit><latexit sha1_base64="qBxIvjvoRT7yjuxKgtdOX85iYXA=">AAACHXicbVBNSwMxFMz6bf2qevQSLIKnsiuCXgqiF49VbCt065JNs20wyS7JW7Es+0e8+Fe8eFDEgxfx35ht96DWgcAw8x4vM2EiuAHX/XJmZufmFxaXlisrq2vrG9XNrbaJU01Zi8Yi1tchMUxwxVrAQbDrRDMiQ8E64e1Z4XfumDY8VlcwSlhPkoHiEacErBRUD31JYKhl5gO7hzDKLvO84ZtUBhlvePlNpnLsE5EMScB92o8BDwMeVGtu3R0DTxOvJDVUohlUP/x+TFPJFFBBjOl6bgK9jGjgVLC84qeGJYTekgHrWqqIZKaXjdPleM8qfRzF2j4FeKz+3MiINGYkQztZZDF/vUL8z+umEB33Mq6SFJiik0NRKjDEuKgK97lmFMTIEkI1t3/FdEg0oWALrdgSvL+Rp0n7oO65de/isHZyWtaxhHbQLtpHHjpCJ+gcNVELUfSAntALenUenWfnzXmfjM445c42+gXn8xtWXKND</latexit>
Attention Score
Weighted Sum
Predicted Probability
We perform two tests: (1) cloze test: for each sentence, we take out the ground-truth Chengyu, and
let the system select a Chengyu from four candidates. (2) coherence checking in college entrance exam: we collected 14 problem
sets, where each problem set consists of four sentences including Chengyus. The system select the sentence that contains the most appropriate Chengyu.
Table1: System and Human Accuracy Comparison
Table 1 shows our approach achieves comparable performance as human experts. For 18% of our system recommended Chengyus which don't exactly match the ground truth, they are also acceptable choices for the given query contexts.Positive Example这事已势不可遏，任何想阻挡他的⼈都如____，简直是不⾃量⼒。 This event is unstoppable, anyone who tries to stop it will be like ____, almost not recognizing his/her own limited power.System: 蚍蜉撼树 (an ant shaking a tree, to describe one fails to recognize one’s limited power)Analysis: The definition significantly enriches the semantic meanings of Chengyu itself. 蚍蜉撼树(an ant shaking a tree) is a metaphor to describe ⾃不量⼒(fail to recognize one's own limited power).
Negative Example村上春树____，29岁才写他的第⼀部作品。 Haruki Murakami ____, he was already at age 29 when he wrote his first works.Ground Truth:⼤器晚成 (takes a long time to make a great instrument)Analysis: We need to know “age 29” is relatively late to produce the first works for a writer.
![CLOZE TEST Go with Frequently Used Words …prepadda.com/.../English-Cloze-Test[].pdf · MOCKTIME.COM ONLINE TEST SERIES CORRESPONDENCE COURSE CLOZE TEST INTRODUCTION A Cloze reading](https://static.fdocuments.net/doc/165x107/5aa8d9367f8b9a90188c1254/cloze-test-go-with-frequently-used-words-pdfmocktimecom-online-test-series.jpg)
