>
Fa   |   Ar   |   En
   The Investigation of Rater Expertise in Oral Language Proficiency Assessment: A Multifaceted Rasch Analysis  
   
نویسنده Bijani Houman
منبع Journal Of Language Horizons, Alzahra University - 2018 - دوره : 2 - شماره : 2 - صفحه:103 -124
چکیده    Since scoring oral language proficiency is performed by raters, they are an essential part of performance assessment. one important feature of raters is their teaching and rating experience which has attracted considerable attention. in a majority of previous studies on rater training, extremely severe or lenient raters, benefited more from training programs and thus results of this training showed significant severity/leniency reduction in their rating behavior. however, they mostly investigated the application of facets on only one or two facets and few have used a pre, posttraining design. besides, empirical studies have reported contrasting outcomes, not showing clearly which group of raters does rating more reliably than the other. in this study, 20 experienced and inexperienced raters rated the oral performances produced by 200 testtakers before and after a training program. the results indicated that training leads to higher measures of interrater consistency and reduces measures of biases towards using rating scale categories. moreover, since it is almost impossible to completely eradicate rater variability even if training is applied, rater training procedure had better had better be regarded as a procedure to make raters more selfconsistent (intrarater reliability) rather than consistent with each other (interrater reliability). the findings of this study indicated that inexperienced and experienced raters’ rating quality improved after training; however, inexperienced raters underwent higher consistency and less bias. hence, there is no evidence that inexperienced raters should be excluded from rating solely because of their lack of adequate experience. moreover, inexperienced raters, being more economical than the experienced ones, cost less for decisionmakers for rating. therefore, instead of charging a bulky budget on experienced raters, decision-makers had better use the budget for establishing better training programs.
کلیدواژه Bias ,Interrater Consistency ,Intrarater Consistency ,Multifaceted Rasch Measurement (Mfrm) ,Rater Expertise
آدرس Islamic Azad University, Zanjan Branch, Department Of English Language Teaching, Iran
پست الکترونیکی houman.bijani@gmail.com, houman.bijani@iauz.ac.ir
 
   بررسی مهارت مصححین در سنجش مهارت گفتاری: بهره گیری از مدل چند وجهی راش  
   
Authors بیژنی هومن
  
 
 

Copyright 2023
Islamic World Science Citation Center
All Rights Reserved