هم‌مرجع‌یابی مبتنی بر پیکره در متون فارسی

مرجع‌یابی یا مرجع‌گزینی یا پیدا‌کردن واژگان هم‌مرجع در متن، یکی از وظایف مهم در پردازش زبان طبیعی است که یک بخش عملیاتی مهم در مسائلی مانند خلاصه‌سازی خودکار، پرسش و پاسخ خودکار و استخراج اطلاعات به‌شمار می‌رود. طبق تعاریف زمانی، دو واژه زمانی هم‌مرجع هستند که هر دو به موجودیت واحدی در متن یا جهان حقیقی ارجاع بدهند. تاکنون برای حل این مساله تلاش‌های متعددی صورت گرفته است که بنابر نتایج این مطالعات، عملیات مرجع‌گزینی را می‌توان با روش‌های متفاوتی مانند روش‌های قاعده‌مند، مبتنی بر قوانین مکاشفه‌ای و روش‌های یادگیری ماشین (بانظارت یا بی‌ناظر) انجام داد. نکته قابل توجه این است که در سال‌های اخیر استفاده از پیکره‌های برچسب‌گذاری‌شده در این زمینه رواج زیادی داشته و منجر به تولید نتایج مناسبی هم شده است. با تکیه بر این موضوع، در پژوهش حاضر، یک پیکره از واژگان هم‌مرجع تولید شده که حدود یک‌میلیون واژه به‌همراه برچسب موجودیت نامدار دارد. در بخش مرجع‌گزینی تمام گروه‌های اسمی، ضمایر و موجودیت‌های نامدار برچسب‌گذاری شده‌اند و برچسب‌های موجودیت نامدار پیکره شامل هفت برچسب است. در پژوهش حاضر با استفاده از این پیکره، یک ابزار مرجع‌گزینی خودکار با استفاده از ماشین بردار پشتیبان تولید شده که دقت آن بر روی داده‌های آزمایش طلایی در حدود شصت درصد است.

Corpus based coreference resolution for Farsi text

Coreference resolution or finding all expressions that refer to the same entity in a text, is one of the important requirements in natural language processing. Two words are coreference when both refer to a single entity in the text or the real world. So the main task of coreference resolution systems is to identify terms that refer to a unique entity. A coreference resolution tool could be used in many natural language processing tasks such as machine translation, automatic text summarization, question answering, and information extraction systems. Adding coreference information can increase the power of natural language processing systems.The coreference resolution can be done through different ways. These methods include heuristic rulebased methods and supervised/unsupervised machine learning methods. Corpus based and machine learning based methods are widely used in coreference resolution task in recent years and has led to a good performance. For using such these methods, there is a need for manually labeled corpus with sufficient size. For Persian language, before this research, there exists no such corpus. One of the important targets here, was producing a through corpus that can be used in coreference resolution task and other associated fields in linguistics and computational linguistics. In this coreference resolution research, a corpus of coreference tagged phrases has been generated (manually annotated) that has about one million words. It also has named entity recognition (NER) tags. Named entity labels in this corpus include 7 labels and in coreference task, all noun phrases, pronouns and named entities have been tagged. Using this corpus, a coreference tool was created using a vector space machine, with precision of about 60% on golden test data.As mentioned before, this article presents the procedure for producing a coreference resolution tool. This tool is produced by machine learning method and is based on the tagged corpus of 900 thousand tokens. In the production of the system, several different features and tools have been used, each of which has an effect on the accuracy of the whole tool. Increasing the number of features, especially semantic features, can be effective in improving results. Currently, according to the sources available in the Persian language, there are no suitable syntactic and semantic tools, and this research suffers from this perspective.The coreference tagged corpus produced in this study is more than 500 times bigger than the previous Persian language corpora and at the same time it is quite comparable to the prominent ACE and Ontonotes corpora.The system produced has an fmeasure of nearly 60 according to the CoNLL standard criterion. However, other limited studies conducted in Farsi have provided different accuracy from 40 to 90%, which is not comparable to the present study, because the accuracy of these studies has not been measured with standard criterion in the coreference resolution field.