The 2014 Research Excellence Framework was a grand and impressive undertaking.
First, take every single higher education institution (HEI) in the UK, and every department, undertaking research with public funds. Second, assess the extent to which the researchers within them are producing excellent research (and all research assessed in the REF is ostensibly to be considered excellent) and the extent to which the research has an impact on wider society, the economy, public understanding, health, and so on.
The results are used to determine the allocation of quality-related (QR) research funding. They also help to demonstrate the strength of UK research and the UK higher education sector, and are used by departments and other organisations to plan, to adapt and to demonstrate the UK’s research strengths. But not everyone is a fan.
It certainly doesn’t come cheap. A 2015 analysis put the cost of operating the 2014 exercise at £246 m across the sector, with around 94% of that cost falling on HEIs. As well as the financial burden, the work takes time away from the business of doing research. Many also suggest that the existing process is open to game-playing, such as departments strategically choosing which researchers to enter in the last REF.
And there are many who are opposed to the development of the impact agenda, in which researchers have to demonstrate the value of their research outside of academia. But others would argue that many of the functions and outputs from the REF are things that departments would have to do anyway, and that, with so much public money (but, of course, not enough) going to HEIs to do research, there is a value and a necessity in periodically demonstrating what has happened to this money.
The return of the Stern (review)
With the next REF planned for some time after 2021, there was time to look once again at the process.
So, taking account of the mixed views on the REF, as well as priorities from the current government, Lord Stern was commissioned to complete a review of the existing process to find out ways to improve the process. His proposals, reporting in July 2016, included requiring institutions to submit all research active staff, for outputs not to be portable between institutions, and for new levels of assessment at the institutional level to be introduced.
Following the Stern review, many of these proposals were then incorporated into a consultation run by the four funding agencies in the UK – HEFCE, HEFCW, the SFC and the Department for the Economy, Northern Ireland. These included Stern’s suggestions above, that HEIs would not be able to choose which units of assessment (UoA) staff are allocated to, that staff would be decoupled from outputs, that the definition of impact should be broadening, and that further measures should be introduced to support interdisciplinary research, such as interdisciplinary champions.
Umming and ahhing
Views on these proposals were mixed. We took part in a number of workshops and had conversations with partners across the sector to understand priorities.
There was universal opposition to some proposals – such as the suggestion that HEIs could not choose the UoAs that staff would be submitted to. But there were a range of views on others, such as proposals for minimum and maximum numbers of outputs per researcher.
For example, some preferred a minimum of one output, others zero. Each creates different incentives and effects: one output would ensure that all staff are not just eligible but are included within the REF, but could create incentives to change contracts. Zero outputs would avoid this issue, but would still allow for game-playing by allowing departments to cluster outputs around fewer researchers. Both have implications for how departments allocate resources and support all their researchers.
There is some acceptance in the sector that many of these changes are inevitable – such as submitting all research active staff – but there is room to move on the detail. Even so, some organisations proposed even more radical changes to the REF, such as the Royal Society, who proposed a new process that would focus entirely on the institution.
Building on our existing policy positions, and as well as engaging with sector partners, we held a workshop with our members to understand the views and concerns across the physics community.
The main points of the IOP response to the consultation were:
- Equality and diversity provisions, such as only chairs of panels receiving unconscious bias training, should be broadened and strengthened
- Proposals for supporting the assessment of interdisciplinary research should be strengthened by introducing an interdisciplinary main panel to work with champions on sub-panels
- HESA cost centres should not be used to map research-active staff to UoAs
- There should be an aim to keep the average number of outputs per staff member of two per FTE, with a minimum of one output per FTE and a maximum of six
- Ways to properly assess of outputs tha are non-traditional in format and those with large numbers of authors should be explored
- Losing and gaining departments should both be allowed to submit the same outputs, where justification can be given
Inevitably, just as not everyone within the community is an enthusiastic supporter of the REF, it’s unlikely that everyone will agree with everything in our response to the consultation.
It’s important to recognise that, just as there are different interests at play between disciplines – for example, engineers are much more comfortable with an increased emphasis on impact than other disciplines are – and the same is also true within a discipline. Physics is a broad church.
The road to REF 2021
This is far from the end of the story. The road to REF 2021 is barely getting started. HEFCE recently announced the appointment of the chair REF equality panel, the chair and members of the REF Interdisciplinary Advisory Panel and have opened up applications for the chairs of the Main Panels.
It is likely that the funding bodies will reflect on the feedback from their recent consultation to adapt their proposals further, and will then come back with another consultation on the REF later in 2017 or in 2018.
Following this, they will then have to start nailing down aspects of the 2021 exercise to allow for researchers, departments and institutions to plan, to adapt, and to prepare. The REF affects the higher education sector in myriad ways, and departments often have to plan processes for collecting information and prioritising work many years in advance.
During this time, we’ll be keeping close tabs on the development of the next REF. We’ll be speaking to those with expertise and be looking for opportunities to increase our understanding on the effects of the proposed changes to the REF, including by commissioning research and further analysis.
At the same time we’ll continue to engage with the sector and with our members to gather views on the current proposed changes, and as information comes out on the direction that they funding councils are moving in.
As such, we continue to be very interested to hear views from the physics community on how the REF affects you, your views on the proposed changes, and how the REF can be further improved for physics. Do please comment below, or send any thoughts to firstname.lastname@example.org.