) with lexicographers - Collaborating with DBP on Wordnet...
Transcript of ) with lexicographers - Collaborating with DBP on Wordnet...
![Page 1: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/1.jpg)
a day (or two) with lexicographersCollaborating with DBP on Wordnet Checking
Dr Lim Lian Tze ([email protected])The Second Wordnet Bahasa Workshop
Malaysia
![Page 2: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/2.jpg)
dewan bahasa & pustaka (dbp)
• Institute of Language and Literature• Established as Balai Pustaka in 1956• Autonomous powers to:
• formulate specific policies (regarding use of B. Malaysia)• organise language and literature construction and developmentprogrammes
• undertake the publishing and sale of books
2
![Page 3: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/3.jpg)
wordnet workshop at dewan bahasa & pustaka (dbp)
3
![Page 4: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/4.jpg)
workshop agenda
• 2-hour seminar: Introduction to Wordnets• 2-hour hands-on: Editing/verifying Wordnet Bahasa contents
4
![Page 5: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/5.jpg)
we wanted the process to be
• Easy to set up• Easy to collect• Easy for validators• (Hopefully) not too confusing
5
![Page 6: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/6.jpg)
google forms!
• Easy to design and set up• Responses saved to a spreadsheet on Google Docs• Can pre-fill fields:https://docs.google.com/forms/<formID>/viewform?entry.
<entry1ID>=Answer_1&entry.<entry2ID>=Answer_2&entry.
<entry3ID>=Answer_3
6
![Page 7: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/7.jpg)
pre-populating from wordnet bahasa
Thanks to Francis, Luis and David7
![Page 8: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/8.jpg)
the google form design
8
![Page 9: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/9.jpg)
b. malaysia members
9
![Page 10: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/10.jpg)
optional: definition and comments
10
![Page 11: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/11.jpg)
which synsets to check?
• Most frequent concepts/synsets with no B. Malaysia equivs(based on counts from ntu-mc and semcor combined)
• Most frequent concepts/synsets (almost certainly containingbad B. Malaysia entries)
• Adding entries from DBP’s own resources to synsets• Concentrating on “groups of words” e.g. colours; dishes; artisticforms…
• Checking on “false friends” e.g. http://en.wikipedia.org/wiki/Comparison_of_Malaysian_and_Indonesian#False_friends
11
![Page 12: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/12.jpg)
which synsets to check?
• Most frequent concepts/synsets with no B. Malaysia equivs(based on counts from ntu-mc and semcor combined)
• Most frequent concepts/synsets (almost certainly containingbad B. Malaysia entries)
• Adding entries from DBP’s own resources to synsets• Concentrating on “groups of words” e.g. colours; dishes; artisticforms…
• Checking on “false friends” e.g. http://en.wikipedia.org/wiki/Comparison_of_Malaysian_and_Indonesian#False_friends
11
![Page 13: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/13.jpg)
some results from the hands-on editing
• ≃ 20 staff from Bahagian Perancangan Bahasa & Perkamusan• 138 English–B. Malaysia synsets checked and edited in 2 hours• Great job!!
12
![Page 14: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/14.jpg)
collected responses
13
![Page 15: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/15.jpg)
causes of error/confusion
• Getting used to wordnet
• Getting used to the interface• Distracted by the English or (existing) B. Malaysia members• (English) synset definitions is important• Lack of B. Malaysia definition• Frequently used words→ highly polysemous, ambiguous• English Wordnet sense granularity is very (too?) fine(re Christiane’s discussion on alternatives)
• Underspecified
• Different ways to ‘cut’, ‘carry’
• Better guidelines needed
14
![Page 16: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/16.jpg)
causes of error/confusion
• Getting used to wordnet• Getting used to the interface
• Distracted by the English or (existing) B. Malaysia members• (English) synset definitions is important• Lack of B. Malaysia definition• Frequently used words→ highly polysemous, ambiguous• English Wordnet sense granularity is very (too?) fine(re Christiane’s discussion on alternatives)
• Underspecified
• Different ways to ‘cut’, ‘carry’
• Better guidelines needed
14
![Page 17: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/17.jpg)
causes of error/confusion
• Getting used to wordnet• Getting used to the interface• Distracted by the English or (existing) B. Malaysia members
• (English) synset definitions is important• Lack of B. Malaysia definition• Frequently used words→ highly polysemous, ambiguous• English Wordnet sense granularity is very (too?) fine(re Christiane’s discussion on alternatives)
• Underspecified
• Different ways to ‘cut’, ‘carry’
• Better guidelines needed
14
![Page 18: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/18.jpg)
causes of error/confusion
• Getting used to wordnet• Getting used to the interface• Distracted by the English or (existing) B. Malaysia members• (English) synset definitions is important
• Lack of B. Malaysia definition• Frequently used words→ highly polysemous, ambiguous• English Wordnet sense granularity is very (too?) fine(re Christiane’s discussion on alternatives)
• Underspecified
• Different ways to ‘cut’, ‘carry’
• Better guidelines needed
14
![Page 19: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/19.jpg)
causes of error/confusion
• Getting used to wordnet• Getting used to the interface• Distracted by the English or (existing) B. Malaysia members• (English) synset definitions is important• Lack of B. Malaysia definition
• Frequently used words→ highly polysemous, ambiguous• English Wordnet sense granularity is very (too?) fine(re Christiane’s discussion on alternatives)
• Underspecified
• Different ways to ‘cut’, ‘carry’
• Better guidelines needed
14
![Page 20: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/20.jpg)
causes of error/confusion
• Getting used to wordnet• Getting used to the interface• Distracted by the English or (existing) B. Malaysia members• (English) synset definitions is important• Lack of B. Malaysia definition• Frequently used words→ highly polysemous, ambiguous
• English Wordnet sense granularity is very (too?) fine(re Christiane’s discussion on alternatives)
• Underspecified
• Different ways to ‘cut’, ‘carry’
• Better guidelines needed
14
![Page 21: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/21.jpg)
causes of error/confusion
• Getting used to wordnet• Getting used to the interface• Distracted by the English or (existing) B. Malaysia members• (English) synset definitions is important• Lack of B. Malaysia definition• Frequently used words→ highly polysemous, ambiguous• English Wordnet sense granularity is very (too?) fine(re Christiane’s discussion on alternatives)
• Underspecified
• Different ways to ‘cut’, ‘carry’
• Better guidelines needed
14
![Page 22: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/22.jpg)
causes of error/confusion
• Getting used to wordnet• Getting used to the interface• Distracted by the English or (existing) B. Malaysia members• (English) synset definitions is important• Lack of B. Malaysia definition• Frequently used words→ highly polysemous, ambiguous• English Wordnet sense granularity is very (too?) fine(re Christiane’s discussion on alternatives)
• Underspecified
• Different ways to ‘cut’, ‘carry’
• Better guidelines needed
14
![Page 23: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/23.jpg)
causes of error/confusion
• Getting used to wordnet• Getting used to the interface• Distracted by the English or (existing) B. Malaysia members• (English) synset definitions is important• Lack of B. Malaysia definition• Frequently used words→ highly polysemous, ambiguous• English Wordnet sense granularity is very (too?) fine(re Christiane’s discussion on alternatives)
• Underspecified• Different ways to ‘cut’, ‘carry’
• Better guidelines needed
14
![Page 24: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/24.jpg)
causes of error/confusion
• Getting used to wordnet• Getting used to the interface• Distracted by the English or (existing) B. Malaysia members• (English) synset definitions is important• Lack of B. Malaysia definition• Frequently used words→ highly polysemous, ambiguous• English Wordnet sense granularity is very (too?) fine(re Christiane’s discussion on alternatives)
• Underspecified• Different ways to ‘cut’, ‘carry’
• Better guidelines needed
14
![Page 25: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/25.jpg)
other feedback
• Wordnet Bahasa marks lemma–synset mappings which are onlyused in
• Malaysia (zsm)• Indonesia (ind)• both (msa)
• Also useful to check for ‘false friends’• Brunei (kxd)??e.g. ‘tambak’ means ‘kitchen’ only in Bahasa Melayu Brunei(Nasruddin Abdullah, 1988)
• From Ethnologue:• zsm, ind: > 80% lexical similarity• zsm, kxd: 80–82% lexical similarity
15
![Page 26: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/26.jpg)
possible collaboration on wordnet bahasa?
• Data• Checking entries (focus on needs/areas)• Writing definitions• B. Malaysia/B. Indonesian/B. Brunei distinctions
• Checking/Verification (semi-automatic)
16
![Page 27: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/27.jpg)
other possible collaborations
• Resources or expertise for verifying/adding synset members• Most common words/concepts• ‘Groups of friends’• Domain-specific terminologies (train experts?)
• Definitions: data/resources, guidelines• for human consumption (readers’ understanding)?• for machine consumption (easier NLP parsing/analysis)?
• Culture-specific synsets (‘expand’ approach) e.g. clothingapparel items, food, etc
• Tie-in with other projects• May be easier to attract interest
• Others!
17
![Page 28: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/28.jpg)
qatlho’
Danke谢谢Grazie
Спасибо
ขอบคุณ
9C4#5$Ìشكرا Merçi
Gracias
நன்றி
Obrigado
Ευχαριστώ
감사합니다ध यवाद
Terima kasih
Thank you
ありがとう
Tapadh leibhཐུགས་རྗེ་ཆེ་།
Go raibh maith agaibh
Xin cảm ơn
18
![Page 29: ) with lexicographers - Collaborating with DBP on Wordnet ...155.69.254.223/events/2016-ws-wn-bahasa/pdfx/lim2.pdf · a day (or two) with lexicographers CollaboratingwithDBPonWordnetChecking](https://reader030.fdocuments.net/reader030/viewer/2022040912/5e86acde885c35433743e5ba/html5/thumbnails/29.jpg)
references
Nasruddin Abdullah. (1988). Kamus kata dan ungkapan am : BahasaIndonesia, Bahasa Malaysia, Bahasa Melayu Brunei. Kuala Lumpur:Dewan Bahasa dan Pustaka.
19