opening the black box of comprehensibility criterion used for machine translation evaluation

Translation quality assessment has been around for several decades and along with the emergence of machine translation and its increasing popularity, machine translated output quality assessment has developed into a hot area of research within which various criteria were introduced for assessing mt output such as expert judgment, computerized assessment, and end-user comprehension as well. the present study was an attempt to focus on the latter investing on comprehensibility as the most canonical feature of evaluation. in this study, two translations of the same source text, one human translated and the other machine translated were given to two similar groups of student-teachers of farhangian teacher education university of arak. using path analyses, the significance of the factors contributing to the comprehension of each type of translation output was investigated. the results showed that although both types of output are superficially comparable in terms of comprehensibility, machine translated texts are not comparable human translation output in terms of satisfying all aspects of comprehension. that is to say, comprehensibility has to be seen as a user dependent criterion which is seriously affected by the internal variables such as reader's topic familiarity and reader's purpose among others. moreover, it has to be noted that comprehensibility, as a translation quality assessment has to be considered as a hopeless criteria in terms of generalizability; that is, a machine translated output being comprehensible to a group of readers is not necessarily useful for other readers or the same readers deciphering the text for a different purpose.