Is the REF worth it?
University initiatives to generate income and "play the REF game" demoralise and discourage researchers and risk overlooking opportunities to build a research culture that improves understanding and generates useful, new knowledge. Our research calls into question the value of REF participation.
Our research sought to understand what makes a research culture enabling for researchers, and whether we should focus efforts on supporting individuals or teams. Through application of a survey and semi- structured interviews, this research gathered data which highlights that university policies formed in response to the UK's REF strongly affect researcher identities and sense of worth. Two themes emerged from the data.
First, the way we value research is perplexing: rather than valuing most highly the substantive shifts in knowledge that typically emerge from long- term projects striving for excellence, we rely on a small group of REF assessors to understand the importance of research as illustrated in a strictly limited set of documents. These can be contradictory because REF Units of Assessment can have a set of assessors from a narrow ontology or epistemology or methodology, and they will rate research higher that fits with their axiology. If your work does not align with these facets because you have different values or because you strongly question the accuracy or validity of an often-implemented methodological approach, then your research will not score highly. When impact is very difficult to measure and when the perception of quality is rated by a select few, there are incentives both to align one's research with particular bodies that can provide high profile impact references (such as government departments) and to align one's research with the REF assessors' preferences.
Arguably, the best research changes our future for the better, so we need to enable and support researchers who challenge as well as extend today's knowledge. However, not all university researchers feel supported or enabled because their challenges to established knowledge may not fit well with the entrenched perception of assessors, especially if that challenge undermines the assessors' contributions and status. Every researcher we spoke to very strongly believed in the importance of their research, but some felt that others did not see, rate, or value their research in the way that they think is correct. The REF process gives them a lower sense of worth, reduces their confidence, enthusiasm, and hope for change. They could either change their values or accept that something they do not value highly may well gain a higher rating in the REF.
Second, the REF is a system that is designed to assist government in allocating resources for research to universities as a reward for the production of impactful research output adjusted for (perceived) quality. Universities align their REF policies so that they encourage the production of research that meets with those assessors' metrics. Researchers who do not have research outputs that are going to be aligned with the highest ratings by REF assessors have concerns that resource allocations and support within their university will be lower, and they will have fewer opportunities for promotion.
In many disciplines, job promotion is heavily influenced by publication in academic journals, but the highest rated journals tend to be consistent with a discipline's mainstream views, with journal editors operating as gatekeepers and only letting through those papers that conform with their worldviews regarding preferences for particular methodologies, axiologies, epistemologies, and ontologies. In other disciplines, academic promotion is based on income generation, which doesn't necessarily align with contributions to knowledge. Frequently these metrics are at odds with a researcher's identity and intrinsic motivations, and this can demoralise researchers, slow the rate of progress in their research, and reduce their rate of generating new knowledge, which could be to the detriment to society. These metrics can perpetuate the production of knowledge that is consistent with a constrained worldview and encourage preference falsification. There is a lot to fix in our world at the moment, and some researchers are concerned about power relations preserving the present level of knowledge rather than opening the gates for a more informed future.
All researchers we spoke to had a strong desire to contribute to their own university's REF entry, which illustrates their need to give back to their community, but the comparatively low expected ratings of some of their research can make them feel peripheral to their group. This is where a nurturing research culture comes in: all positively-engaged researchers contribute to strengthening the research atmosphere within a department and this enables a group to achieve. Remember, REF resources are allocated to a university to reflect their collective performance to a REF Unit of Assessment, and not an individual researcher's performance. Nurturing of an open and respectful constructively critical research culture is necessary for all to achieve, irrespective of the REF assessors' metrics. Thus, a REF 'grade' is not simply a reflection of some select 'superstar' researchers within a department but instead is a reflection the extent that the research culture enables research activities. Of issue here is whether there is a better way for government to allocate resources for research, whether universities respond to this measurement exercise in a way that
enables or constrains all of their researchers, and whether university policies discriminate against particular types of research output.
Our research revealed evidence consistent with the view that researchers operate in an ecosystem in which co-operation and competition both enables and constrains individual researchers and different aspects of research. The research environment, with characteristics of both co-operation and competition, needs to be healthy and constructively critical in ways that stimulate and value further thought, extensively defined. A positive research culture is one that is open to alternative viewpoints.
Our evidence suggests that the governments' approach to allocate research funds to universities through the REF system demoralises some researchers, stunts their rate of producing research output, and could perpetuate existing views of knowledge and understanding, which can prevent us from moving forward. This is contrary to the need to encourage an open and free academic system that advances knowledge to create a better and more informed future.
