Update on the Australian ERA (research evaluation)
Below is the Minister's recent release on ERA. To help members of
this list to interpret it,
· Consultation on the submission drafts sent out has meant
that the time window for the submission for the PCE and HCA clusters
has been pushed out from 20 April - 8 May 2009 to June (PCE) and
August 2009 (HCA).
· The PCE and HCA cluster evaluations have morphed into a
"trial".
· Esteem indicators will not be included in this trial, and
the ARC will do more work on what esteem metrics it can create/find
in the full ERA.
Personal analysis
My guess is that universities (mainly) complained about preparing
submissions in the original timeframe, resulting in the push-out. It
was certainly tight. Conveniently for the Government in these
financial times it is now in the next financial year. Nevertheless
universities will still be in full preparation mode as 4 or 5 months
still does not give them much time.
The omission of esteem also effectively causes the morphing into a
trial, probably better called a dummy run. The real ERA has thus been
delayed a year, and it is unclear whether the PCE and HCA clusters
will still be the first to be done properly in 2010. That would seem
sensible to capitalize on the work already done by both universities
and the ARC's committees. We await the finalized Submission and
Technical guidelines due in March.
Note that we also still await the selected supplier of the citation
data for the HCA cluster. The PCE cluster data will be supplied by
Scopus as I previously posted.
It is not known if the PCE evaluation will use Scopus' SJR metrics
(the PageRank algorithm applied to cites instead of hyperlinks). It
would be nice if it did, to make the metric mix richer and because
the SJR seems to be a better measure of the scientific value of a
paper than raw cite counts.
Arthur Sale
Emeritus Professor of Computer Science
University of Tasmania
PS Roughly PCE = Physical, Chemical & Earth Sciences Cluster; HCA =
Humanities and Creative Arts Cluster (including Fine Art, Music,
Drama and Architecture), but they are both defined by ANZSRC codes.
http://minister.innovation.gov.au/Carr/Pages/ERATIMEFRAME.aspx
ERA TIMEFRAME
The Minister for Innovation, Industry, Science and Research, Senator
Kim Carr, today announced the timeframe for the Excellence in
Research
for Australia (ERA) trial, which will evaluate the Physical, Chemical
and Earth Sciences (PCE) and Humanities and Creative Arts (HCA)
clusters.
"The Australian Government is committed to the development of a
world-
class research quality and evaluation system that has the confidence
of the sector," Senator Carr said.
"Recent feedback on the ERA Submission Guidelines and ERA-SEER
Technical Specifications raised concerns about the proposed timing of
the ERA trial.
"To ensure universities are adequately prepared for the trial,
submissions for the PCE cluster will now commence in June 2009, and
submissions for the HCA cluster will commence in August 2009.
"The outcomes of these trials will inform the full ERA process in
2010.
"This timeframe will ensure that universities are able to collect and
submit quality and accurate data for ERA.
"Recent feedback also raised concerns about the scope of some of the
data to be collected.
"Following this feedback, esteem indicators will not be included in
the ERA trial. I have asked the ARC to further investigate the
collection of esteem indicators, which will be included in future
evaluations.
"The expert review and feedback that the ARC has received so far has
been crucial in shaping ERA.
"I trust universities will be pleased with the timeframe I have
announced today and I ask that they continue to provide constructive
feedback on ERA to the ARC."
The ARC will release the final ERA Submission Guidelines and ERA-SEER
Technical Specifications for the trial, in early March. You can find
out more about ERA at www.arc.gov.au
Media contacts:
Patrick Pantano, Minister's office, 0417 181 936
Sheena Ireland, ARC, 0412 623 056
Received on Wed Feb 25 2009 - 02:41:47 GMT