Assessing Excellence

After years of trying to reform how it evaluates U.K. research departments for basic university funding, the Higher Education Funding Council for England (HEFCE) announced this week its new evaluation strategy. The new strategy is based on a mix of peer review and an evaluation of research impact, such as societal or economic benefits.

HEFCE is a U.K. government agency that distributes nearly $3 billion dollars annually for university research. Between 30 to 50% of a chemistry department's operating budget can come from that pot. The remaining funding comes from grants and tuition fees, which form the basic operating budgets of most U.S. universities.

The new HEFCE evaluation strategy replaces the so-called Research Assessment Exercise (RAE), which involves experts in a field assessing university departments across the U.K. The year-long process that took place approximately every six years, many say, is arduous and expensive. The RAE rankings dictated funding to universities and helped university administrators make long-term decisions. In 2001, the RAE rankings led to the closure for several chemistry departments.

In 2006, to reform the RAE, HEFCE proposed that the assessment of research excellence would switch from a peer review process to a statistical approach that uses metrics such as citations. This proposal caused an onslaught of criticism from scientists who worried that exchanging human reviewers for equations would not result in fair evaluations (C&EN, April 27, page 26).

The new Research Excellence Framework (REF) will be implemented in 2013. It "builds on the success of the RAE," noted David Sweeny, HEFCE's director of research, innovation, and skills, in a statement. But "it incorporates important new features particularly in recognizing the impact of research."

Under the REF, 60% of the evaluation will come from a primarily peer-reviewed evaluation of academic excellence in a research unit and 15% will come from an assessment of the quality of the research environment in that unit. The remaining 25% will assess the "impact" of the research unit's individuals, based on things such as case studies about social and economic benefits. The details of the "impact" component of the new REF will be ironed out in a 2010 pilot exercise.

Some observers worry that the inclusion of economic impact in the criteria will hinder basic "blue skies" research in the U.K. "History has taught us that some of the biggest breakthroughs have come from speculative research, and it is wrong to try and measure projects purely on their economic potential," Sally Hunt, the general secretary for the U.K.'s University & College Union, noted in a statement.