The Australian Government has released a definitive, if incomplete,
description of Australia’s Research Quality Framework (RQF) which
is our equivalent of the UK’s RAE. If familiar with the RAE, you
will recognize the family resemblance. I extract the essentials of the
RQF for an international readership, and analyze some of the consequences
likely to flow from it. To see the documentation, see
http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issu
s/research_quality_framework/rqf_development_2006.htm.
ESSENTIAL POINTS
1. The first RQF assessment will be based on submissions by the 38
Australian universities by 30 April 2008. Funding based on the assessment
will flow in calendar year 2009. Six years will elapse before the next
assessment (ie 2014), but there is provision to shorten this.
2. The Unit of Assessment is the Research Group. Research Groups
will be defined by up to three RFCD four-digit codes (to allow for
multi-disciplinary groups). The RFCD classification is uniquely
Australian, and for example there are six four-digit codes in the field
of ICT. Engineering has more but for example Civil Engineering is one. If
you are interested in the codes see
http://www.research.utas.edu.au/publications/docs/14_rfcd.doc, the four
digit codes are the sub-headings.
3. Each Research Group will be allocated to and assessed by one of
13 Panels. The Panel is determined by the primary RFCD code. Thus
Mathematics, Computing and Information Technology is Panel 4.
4. Each University will submit an Evidence Portfolio (EP) for each
identified Research Group. There is provision for cross-university
Research Groups.
5. The ratings will be based on Quality and Impact separately. These
words have peculiar (ie not common-usage) meanings. Approximately,
Quality is a bag of quantifiable metrics, and Impact is all the soft
things like Fellowships of Academies, Honors, journal associate
editorships, etc. The relative importance of Quality and Impact will vary
by Panel and is similarly not yet resolved. Quality is based on the best
four publications (Research Output) of each researcher in the group over
the six years 2002-2007, on a full list of all Research Output from the
group including honorary and emeritus professors, and on competitive
grants received over the period. Impact is covered in the Context
Statement of the EP
6. Impact for each Research Group will be assessed on a scale of 1
(not important) to 5 (prestigious)..
7. Impact is rated A (outstanding) to E (poor).
8. Research Groups which rate below 2 for Quality, or below D for
Impact, will attract no funding to their university, though the two
factors are separately aggregated for the University. The weighting of
funding is stated to be linear with rating, but the gradient will be
determined during 2007.
9. The Panels require access to the electronic versions of any of
the Research Output within four working days. The Panels will (a) rank
the outputs by things like journal impact factors, journal standing, etc,
(b) assess citation counts, both in aggregate and by the percentage that
fall in the top decile for the discipline, and (c) competitive grant
income.
10. The RQF is based on a semi-centralized IT model (or
semi-decentralized). In other words, the full-texts of the research
outputs (publications) will be held in IRs in each university, while the
RQF secretariat will run a repository with all the EPs and develop the
citation counts independent of the universities (in conjunction with
Thomson Scientific and possibly EndNote Web). The Australian Government
will be approached for funds to universities to establish these IRs.
ANALYSIS FOR OPEN ACCESS
· The RQF will actually use citation metrics in the assessment,
not just test them as a “shadow exercise” as in the next RAE.
This will mean that the OA citation advantage will suddenly look very
attractive to Australian universities, though it is a bit late to do
anything about it five years into a six-year window. However, with 2014
in mind, there will be pressure to increase citations.
· Every university will have to have an IR to hold the full-text
of Research Outputs. About half already do, with EPrints and DSpace being
the most popular software with a few Fedora-based repositories and
outsourced ProQuest hosts. There will be funding to establish
repositories.
· I expect a mad scramble in the smaller universities, with
outsourcing and hosting solutions being very attractive. Money fixes
everything. The ones that have been dithering will regret it.
· All Research Output generated by all Research Groups will have
to be in the IRs for the RQF. This may amount to 50% of the university
research production over six years, or more or less depending on how
research intensive it is. There are two corollaries: (a) this is Mandate
by Money, and (b) there will be frantic activity over 2007 to put in the
backlog of 2002-2006 publications.
· Since one does not know what Research Output will be needed in
2014, and only a general clue in 2007, 100% institutional mandates are
likely to spring up all over the place, in the form of Mandate by
Administration. What I mean by this is that the deposition of the paper
will be integrated with the already present administrative annual
requirement to report the publication to the Australian Government.
· Although it is nowhere stated explicitly that I can see, I read
between the lines that the RQF may be expecting to get access to the
publisher’s pdf. This means that it will have to be in the
repository as “restricted access” in most cases or as a link
to an OA source. There is no reason why the OA postprint cannot be there
as “open access” as well, of course, and if a citation
advantage is to be got, it will need to be.
Please feel free to blog this or forward this to anyone you think may be
interested. My apologies for cross-posting.
Arthur Sale
Professor of Computing (Research)
University of Tasmania
Received on Fri Nov 17 2006 - 04:07:26 GMT