Tuesday, May 31, 2011

ERA loses its ABC


Several years ago the Australian academic community was told that it needed metrics and direction to become the lean tool of academic excellence expected (presumably bitter competition within the community is insufficient) by whoever cares about these things. This was going to be provided under the banner of Excellence in Australian Reseach (ERA), by the Australian Research Council (ARC). A major innovation was to be the replacement of the generally universal system for ranking and estimating the relative profile of journals, with our own home-grown scale. In all it's wisdom, the Australian government (how many PhDs between them??) was going to boldy break out and devise it's own system. Needless to say, the rest of the world didn't give a hoot, resulting in journals or papers being more or less important depending upon which side of Australia's border you happend to be on.

After no doubt not inconsiderable sums of money, much huffing and puffing and a revision when serious flaws arose (journals ranked way out of their league, or missing entirely) a list of journals and ranks was released. Of particular note is that rankings were partly determined by lobying from the industry, and decided on by a 'panel of (not impartial) experts'. Journals were nominated a value within an apparantly arbitrary tier system from A* (highest) through A and B to C (lowest). The Sesame Street System. After several years, things had settled down, people for the most part ignored the new system, unless scraping for a few boosted publication rankings in local ARC grant proposals.
Fine.
Until today.
It has been anounced that the rankings are already to be discontinued. The media statement is available here:

http://minister.innovation.gov.au/Carr/MediaReleases/Pages/IMPROVEMENTSTOEXCELLENCEINRESEARCHFORAUSTRALIA.aspx

Blimey, what a lot of waffle!!!

So, in summary:

"These improvements are:
- The refinement of the journal quality indicator to remove the prescriptive A*, A, B and C ranks"


Refinement?? So they are getting rid of the Sesame Street System (A,B,C), but are still providing some 'quality indicator'?? How is this going to be any different? And what was the original intention of 'perscriptive rank' if not to provide a guide for researchers as to the perceived 'quality' of a journal??

The one problem with the system stated is "...the setting of targets for publication in A and A* journals by institutional research managers."

Well... ummm... dah! Who would be naive enough to not see the blindingly obvious extension that higher rated journals would be targeted!?!? As is already the case with Impact Factors?!? Or can I read between the lines/guess that because personal submissions/cases were made for journal ranks, they discovered people were inflating their own journals/journals-they-already-published-in rank and skewing the field...?

- The introduction of a journal quality profile, showing the most frequently published journals for each unit of evaluation;

What?!?! Who cares? Unless I'm odd, the rate of issue is not high on my priority list, and quantity certainly isn't any measure of quality...

"As with some other aspects of ERA, the rankings themselves were inherited from the discontinued Research Quality Framework (RQF) process of the previous government..." under the advice of the ARC, while he later goes on to say: "I have made the decision to remove the rankings, based on the ARC’s expert advice."

So the same body that was inept under the last government are now excellent...?? Someone hand me the sick bag...

1 comment:

  1. The world is moving. You want to see real impact? Don't look to your paper. Look to your data. Publish your data. On the Internet. The world will thank you.

    ReplyDelete