Re: AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM Digest - 8 Aug 2008 to 9 Aug 2008 (#2008-153)

From: Yves Gingras <gingras.yves_at_UQAM.CA>
Date: Sun, 10 Aug 2008 12:41:28 -0700

It could be expected that my colleague Jean-Claude Guedon would offer
himself to teach us in his usual cynical, smiling, tongue-in-cheek
manner the simplistic and obvious constructivist explanation about
what "sticks" in society; as if we, poor naïve, did not know this
basic fact that, in society, all representations are born and die in
struggles...

But has he says: this is  "somewhat irrelevant" to my real point
which is: by fighting these absurd measures which can in fact
generate stupid university policies we can contribute to better
ranking and policies. For following his logic  if people would choose
their universities on the basis of Feng Shui or even Chineese
horsocope (as some may do already) then we would have to live with it
if for some contingent reason it happened to "stick" and be used by
students and politicians... This remembers me - believe me: it is
true!- a deputy minister of SCIENCE who choose his collaborators
based on their astral signs!...

But one can see things differently and recall that it took years for
Canadian universites to pull out of Macelan's rankings, but they did.
That was a minimum logical step to do even though to make money
Maclean will of course continue to publish it on its own; but at
least universities will not use their own money to create that
misconceived ranking and in this way give it credibility. So, the
same with Shnaghai and Web-rankings: a systematic destruction by
people who work in that field (and I may recall that as Scientific
Director of the Observatoire des sciences et des technologies (OST),
I have been working on research evaluation for more than 10 years
with many Ministries and Universities and Research centers and know
how to construct a well defined indicator of research impact based on
publications and citations. I also know how to recognize false
indicators by analyzing their properties. Of course they can be
debated and interpreted - as my colleague love to do constatnly- but
they are like inflation index or unemployment index: based on
controlled and coherent data so that even interpretations are
constrained.

So, beyond (and after) the basic constructivist soiology of my
colleague one can go a step further, which I took as implicit in my
text: use every opportunity to recall to managers and politicians
that 1) Shanghai indicators and the bizarre one on Web-visibility are
ill-conceived and 2) we can create much better indicators 3) using
bad indicators can lead to dangerous policies like bad medical
diagnostic may lead to giving the wrong pill...

As someone who has a certain expertise in indicators, I prefer to try
to convince people to use good instead of bad indicatgors and I keep
my sociology 101 for my classes. And of course "good" and "bad" are
also socially constructed as my colleague will urge to add...  Bu as
a social agent, I fight (naively?) using my expertise (as
intellectuals should do) to make sure the conceptual houses that are
built in our society are not based on ill-conceived plans. For in
analalogy with real houses, badly constructed ones generate
strcutural problems and eventually fall; sometimes on real people...
 

But I stop here and I will NOT do like my good friend Stevan: loose
much more time in dilettante and unending exchanges with our
colleague about obvious facts that lead to more talk and less action.
For one can be content with observing the world from above, with the
smile of those in the know, or one can try to make it less absurd,
even if that means going against the dominant wind.


Yves Gingras


De : Guédon Jean-Claude <jean.claude.guedon_at_UMONTREAL.CA>
Date : Sat, 9 Aug 2008 08:03:20 -0400
Objet : Re: University ranking

The criticism of the university rankings in terms of measuring "what"
is quite correct. However, it is also somewhat irrelevant. What is
important in the end, whether we like it or not (and I certainly do
not like it any more than the previous commentators) is that it
creates a benchmark that sticks, so to speak, and is used. If there
ever was a good example of social construction of "reality", this is
it. What is at stake here is not quality measurement; rather, it is
"logo" building for a globalized knowledge economy. If
administrators, the press and governmental bureaucracies pay
attention - and they obviously have begun to do so - then the
strategy works.

The solution? In the absence of an institutionally effective
critique, all we can hope for is the existence of competing and
somewhat divergent indices... But I fear that, in the end, they will
converge in one way or another and will create all the distortions
that such artificial and, ultimately, manipulative metrics produce.

Jean-Claude Guédon
Received on Mon Aug 11 2008 - 03:25:10 BST