یک مدل موضوعی احتمالاتی مبتنی بر روابط محلّی واژگان در پنجره‌های هم‌پوشان

بسیاری از مدل های موضوعی مانند lda که مبتنی بر هم رخدادی واژگان در سطح یک سند هستند قادر به بهره گیری از روابط محلی واژگان نیستند. برخی از مدل های موضوعی مانند btm سعی کرده اند با ترکیب موضوعات و مدل های زبانی n-gram، این مشکل را حل کنند. امّا btm مبتنی بر ترتیب دقیق واژگان است؛ بنابراین با مشکل تُنُکی روبه روست. در این مقاله یک مدل موضوعی احتمالاتی جدید معرفی شده که قادر به مدل کردن روابط محلی واژگان با استفاده از پنجره های هم پوشان است. بر اساس فرضیه هم رخدادی، رخداد هم زمان واژگان در پنجره های کوتاه تر، گواه محکم تری بر ارتباط معنایی آنهاست. در مدل پیشنهادی، هر سند، مجموعه ای از پنجره های هم پوشان فرض می شود، که هریک متناظر با یکی از واژگان متن است. موضوعات بر مبنای هم رخدادی واژگان در این پنجره های هم پوشان استخراج می شوند. به عبارت دیگر، مدل پیشنهادی، روابط محلی واژگان را بدون وابستگی به ترتیب دقیق آنها مدل می کند. آزمایش های ما نشان می دهد که روش پیشنهادی، موضوعات منسجم تری را تولید و در کاربرد خوشه بندی اسناد، دقیق تر از دو مدل lda و btm عمل می کند.

A Probabilistic Topic Model based on Local Word Relationships in Overlapped Windows

A probabilistic topic model assumes that documents are generated through a process involving topics and then tries to reverse this process, given the documents and extract topics. A topic is usually assumed to be a distribution over words. LDA is one of the first and most popular topic models introduced so far. In the document generation process assumed by LDA, each document is a distribution over topics and each word in the document is sampled from a chosen topic of that distribution. It assumes that a document is a bag of words and ignores the order of the words. Probabilistic topic models such as LDA which extract the topics based on documentslevel word cooccurrences are not equipped to benefit from local word relationships. This problem is addressed by combining topics and ngrams, in models like Bigram Topic Model (BTM). BTM modifies the document generation process slightly by assuming that there are several different distributions of words for each topic, each of which correspond to a vocabulary word. Each word in a document is sampled from one of the distributions of its selected topic. The distribution is determined by its previous word. So BTM relies on exact word orders to extract local word relationships and thus is challenged by sparseness. Another way to solve the problem is to break each document into smaller parts for example paragraphs and use LDA on these parts to extract more local word relationships in these small parts. Again, we will be faced with sparseness and it is wellknown that LDA does not work well on small documents. In this paper, a new probabilistic topic model is introduced which assumes a document is a set of overlapping windows but does not break the document into those parts and assumes the whole document as a single distribution over topics. Each window corresponds to a fixed number of words in the document. In the assumed generation process, we walk through windows and decide on the topic of their corresponding words. Topics are extracted based on words cooccurrences in the overlapping windows and the overlapping windows affect the process of document generation because; the topic of a word is considered in all the other windows overlapping on the word. On the other words, the proposed model encodes local word relationships without relying on exact word order or breaking the document into smaller parts. The model, however, takes the word order into account implicitly by assuming the windows are overlapped. The topics are still considered as distributions over words. The proposed model is evaluated based on its ability to extract coherent topics and its clustering performance on the 20 newsgroups dataset. The results show that the proposed model extracts more coherent topics and outperforms LDA and BTM in the application of document clustering.