A Malay text corpus analysis for sentence compression using pattern-growth method

A text summary extracts serves as a condensed representation of a written input source where important and salient information is kept. however,the condensed representation itself suffer in lack of semantic and coherence if the summary was produced in verbatim using the input itself. sentence compression is a technique where unimportant details from a sentence are eliminated by preserving the sentence’s grammar pattern. in this study,we conducted an analysis on our developed malay text corpus to discover the rules and pattern on how human summarizer compresses and eliminates unimportant constituent to construct a summary. a pattern-growth based model named frequent eliminated pattern (faspe) is introduced to represent the text using a set of sequence adjacent words that is frequently being eliminated across the document collection. from the rules obtained,some heuristic knowledge in sentence compression is presented with confidence value as high as 85% - that can be used for further reference in the area of text summarization for malay language. © 2016 penerbit utm press. all rights reserved.