همبستگی بین رتبه‌بندی دانشگاه‌ها به دو روش کتاب‌سنجی محض و داوری غنی‌شده با کتاب‌سنجی: نمونه مورد مطالعه نظام تعالی پژوهش انگلیس و نظام‌های رتبه‌بندی معتبر جهانی

هدف: برخی رتبه‌بندی‌های دانشگاهی مانند چارچوب تعالی پژوهش (رف)، برای دستیابی به کیفیت بالاتر، از روش داوری استفاده می‌کنند. بااین‌حال، کارایی این روش پایین است. پژوهش حاضر به‌منظور بررسی میزان هم‌گرایی نتایج رتبه‌بندی‌های دارای رویکرد متفاوت به داوری، همبستگی نمره‌های دانشگاه‌های انگلیس را در «رف» و سامانه‌های رتبه‌بندی معتبر لایدن، شانگهای، کیو-اس و تایمز بررسی می‌کند. روش‌شناسی: پژوهش حاضر، به روش اسنادی با رویکرد تحلیل محتوای کمی انجام شده است. داده‌ها به روش سرشماری گردآوری و با تحلیل همبستگی و رگرسیون تحلیل شده‌اند. یافته‌ها: قوت همبستگی نمره دانشگاه‌ها در «رف» با «تایمز» و «کیو-اس» قوی، با «شانگهای» متوسط و با ابعاد «لایدن» از ضعیف تا قوی است. نمره کل در «کیو-اس» متاثر از گرایش پزشکی است. تاثیر گرایش موضوعی بر نمره در برخی ابعاد سامانه لایدن مشاهده می‌شود. نتیجه‌گیری: رتبه‌بندی مبتنی بر داوری غنی‌شده با کتاب‌سنجی، هم‌گرایی قوی و درنتیجه مشابهت بالایی را با روش مبتنی بر تلفیق کتاب‌سنجی با آمار عملکردی و نظرسنجی نشان می‌دهد. اما تلفیق کتاب‌سنجی با آمار عملکردی صرف مشابهت متوسطی با آن دارد و نمی‌تواند جایگزین آن شود. تاثیر گرایش موضوعی بر نتایج رتبه‌بندی، کاربرد این نتایج برای مقایسه دانشگاه‌های دارای پوشش موضوعی متفاوت را به چالش می‌کشد.

a correlation study of bibliometric-based and informed-peer-review university rankings: the case of uk research excellence framework (ref) and the world’s prestigious university ranking systems

purpose: informed peer-review university ranking systems like ref involve peer review to achieve high-quality university performance evaluations while considering bibliometric facts. despite its advantages and methodological success, it is not possible to implement the review-based exercise in all countries due to unaffordable costs, vulnerability to bias, and differences in cultural, economic, managerial, and infrastructural conditions. as an alternative, some science systems prefer to consider using the results of international university rankings to gauge the academic performance of their higher education institutions. using a different methodology, the international ranking systems tend to maintain their efficiency by heavily relying on the bibliometric information extracted from databases, the performance data gathered from official authorities, and academic and employer surveys. this gives rise to the question of to what extent these two types of systems are convergent in their results. if their different methodologies lead to similar results, they could be interpreted to have similar effectiveness in their evaluation of academic performance. therefore, by relying on the results of the international ranking systems, one can avoid the shortcomings of the review-based method, and maintain both the efficiency and effectiveness of the evaluation systems. to reveal the convergence of the results obtained from the methods, the present study explores a sample of british universities evaluated by ref (2014) to investigate the correlation between their scores in ref and the world’s prestigious university rankings.methodology: using a quantitative content analysis method, the present study concentrates on a collection of 150 british universities evaluated simultaneously by ref (2014) and at least one of the world’s prestigious ranking systems including qs, the, leiden and arwu. due to the small size of the population, all ref members are examined without sampling. the evaluation results of these universities are extracted from these systems and entered into a checklist. also, the subject fields and disciplines covered by the universities are collected to investigate their probable effects on the results. the universities’ subject coverage similarity is calculated using the cosine similarity measure and k nearest neighbor technique in the knime data mining platform. finally, correlation and regression analyses are applied to analyze the data.findings: uk universities’ scores in ref are found to be significantly correlated to theirs in international ranking systems. they are highly correlated to qs and the’s, while being moderately associated with arwu’s, and weakly-to-strongly correlated to leiden’s. the regression analyses show no significant effects of subject coverage on the overall scores, except for medical tendency’s effect on qs. however, the subject coverage affects some dimension scores. while it does not significantly predict any of the dimensions of arwu, it partially predicts the citations and industry income dimensions in the, with medical and technical, and engineering subjects respectively having the highest positive predicting power. in qs, the subject coverage partially predicts the dimensions of academic reputation, faculty/student ratio, international faculty ratio, international student ratio, and citation per faculty. medical science has the highest positive effect on the dimensions of academic reputation, faculty/student ratio, and citations per faculty. moreover, basic sciences have the highest negative effect on the international student ratio and international faculty ratio.