University League Tables: RAE 2008 and Staff Exclusions
The Complete University Guide recognises the importance of publishing league tables that reflect the numbers of academic staff engaged in research as well as its quality.
When we first published University Rankings 2010 in April, we were the first to use data from the 2008 Research Assessment Exercise as one of its nine criteria. But unlike previous years, when we drew on the 2001 RAE, we were unable to use the "intensity" staff data after this component was dropped at a late stage because of confusion over definitions of eligibility. As far as we are aware, there has been no suggestion that this significant omission will be rectified at a later date and it is clearly in the interest of some universities that the status quo should remain.
However use of the RAE 2008 outcome in our Research Assessment measure without an estimate of the proportion of eligible staff who took part in the exercise tells only half the story.
Accordingly, we have collaborated with Professor Peter Diggle and his colleague, Professor Amanda Chetwynd, at the University of Lancaster, to produce estimated figures for the proportion of staff submitted by universities using a simple statistical technique based on a significant sample of cases where this information was known. For a full explanation please see the paper published by Professor Diggle "RAE 2008, League Tables and Staff Exclusions".
We have produced a revised version of the main University League Table published in The Independent on April 30 incorporating the outcome of this work, to see the results - The University League Table with revised Research Assessment, revised May 28th.
This new Table will form the basis for our main Ranking Tables in the future.
The effect on the rankings has been minimal for most universities. Of the 113 universities listed, the position of 41 remains unchanged. A further 30 moved up or down by one place, 27 by two to four places and five by five places or more.
The original league table remains as first published and can be found at University League Table 2010.
Bernard Kingston
Notes
For some university applicants, the full picture of the research performance of their potential university will be of importance. By engaging with the university community to deliver the fullest possible information we are building a comprehensive body of data in this vital area. At this stage, however, the revised tables are provided as a basis for discussion with the university sector and are not aimed directly at potential applicants. We welcome input. Comments should be addressed to
admin@thecompleteuniversityguide.co.uk.
The proportion of staff entered into the RAE was either obtained from the university or estimated using a standard statistical method applied to 2007-08 HESA staff data. The quality of research measure (the grade point average) and the intensity of research measure (the proportion of staff entered) were then combined with a relative weight of 2:1 to create an overall research assessment measure.
The paper by Professor Peter Diggle is published on the HE Staff section "RAE 2008: University League Tables and Staff Exclusions"
The methodology behind the table remains as for the original University League Table 2010 (see The Background) the Revised Definition of "Research Assessment" Measure is as detailed below.
Each university department entered in the assessment exercise achieved a quality profile which gave the proportion of research in each of four categories from 4* to 1* (with any remaining activity being unclassified). The categories 4* to 1* were given a numerical value of 4 to 1 which allowed a grade point average to be calculated. An overall average was then calculated weighted according to the number of staff in each department. The proportion of staff entered into the RAE was either obtained from the university or estimated using a standard statistical method applied to 2007-08 HESA staff data. The quality of research measure (the grade point average) and the intensity of research measure (the proportion of staff entered) were then combined with a relative weight of 2:1 to create an overall research assessment measure. Only the grade point average appears listed in the ranking on this website but the overall measure was used in the calculations.
The main table is based on nine measures: Student Satisfaction, Research Assessment, Entry Standards, Student:Staff Ratio; Spending on Academic Services; Spending on Facilities; Good Honours degrees achieved; Graduate Prospects and Completion.
To view the official University League Tables 2010 as first published in April 2009
