The message below is from Isidro Aguillo, the Scientific
Director of the Laboratory of Quantitative Studies of the
Internet
http://internetlab.cindoc.csic.es/miembros.asp?id=1
of the Centre for Scientific Information and Documentation
http://www.cindoc.csic.es/eng/info/infobjetivos.html
Spanish National Research Council
http://www.csic.es/index.do and editor
of Cybermetrics, the International Journal of Scientometrics,
Informetrics and Bibliometrics
http://www.cindoc.csic.es/cybermetrics/
Dr. Aguillo makes the very valid point (in response to Declan Butler's
Nature news article about the use of unvalidated university rankings)
http://www.nature.com/nature/journal/v447/n7144/full/447514b.html
that web metrics provide new and potentially useful information
not available elsewhere. This is certainly true, and web metrics
should certainly be among the metrics that are included in the multiple
regression equation that should be tested and validated in order
to weight each of the candidate component metrics and to develop norms
and benchmarks for reliable widespread use in ranking and evaluation.
Among other potential useful sources of candidate metrics are:
University Metrics:
http://www.universitymetrics.com/tiki-index.php
Harzing's Google-Scholar-based metrics:
http://www.harzing.com/pop.htm
Citebase:
http://citebase.eprints.org/
Citeseer:
http://citeseer.ist.psu.edu/
and of course Google Scholar itself:
http://scholar.google.com/advanced_scholar_search?hl=en&lr=
Stevan Harnad
On Tue, 5 Jun 2007, Isidro F. Aguillo wrote:
> Dear all:
>
> As our ranking (Webometrics) is listed in the Nature paper, we wish to
> add some points to the debate:
>
> - Rankings has been very successful in certain areas. Original purpose
> of Chinese Ranking was to help students for choosing foreign
> universities and from our own data this is still the major use, so there
> is clearly a gap to be filled. In other area, Shanghai data is
> probably behind major reorganization of French university system in
> 2006. As the Web data includes larger number of institutions in
> developing countries we have noticed similar debates in the Middle East
> and South East Asia.
>
> - Our main aim for preparing the Web ranking was not to classify
> institutions but to encourage Web publication, even farther than the
> current OA initiatives as they are focused on the formal scholar
> publication and we call for open archives of raw data, teaching
> material, multimedia resources, software and other academic and
> para-academic material. It was a great surprise to discover that there
> is already a big academic digital divide in web contents that affects
> not only to developing regions but to many European countries
>
> - Taking into account the naming system of the Web domains, the
> institution is a "natural" unit in the webometric analysis. An
> additional advantage is that webpages reflect a lot of more activities
> that only scientific publications but unfortunately in a way that is
> difficult to discriminate specific contributions. As the evaluation of
> universities should consider other aspects than research output, Web
> indicators could be combined with other indicators as we intend to do
> in the near future.
>
> A new edition of our ranking (http://www.webometrics.info) covering over
> 4000 universities worldwide is scheduled for the July. Any comments and
> suggestions are welcomed.
>
> Best regards,
> ***************************************
> Isidro F. Aguillo
> isidro -- cindoc.csic.es
> Ph:(+34) 91-5635482 ext. 313
>
> Cybermetrics Lab
> CINDOC-CSIC
> Joaquin Costa, 22
> 28002 Madrid. SPAIN
>
> www.webometrics.info
> www.cindoc.csic.es/cybermetrics
> internetlab.cindoc.csic.es
> ****************************************
>
> From: Stevan Harnad <harnad -- ecs.soton.ac.uk>
> To: AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM_at_LISTSERVER.SIGMAXI.ORG
> Subject: "Academics strike back at spurious rankings" (Nature, 31 May)
http://openaccess.eprints.org/index.php?/archives/251-guid.html
> Academics strike back at spurious rankings
> D Butler, Nature 447, 514-515 (31 May 2007) doi:10.1038/447514b
> http://www.nature.com/nature/journal/v447/n7144/full/447514b.html
>
> This news item in Nature lists some of the (very valid) objections to the
> many unvalidated university rankings -- both subjective and objective --
> that are in wide use today.
>
> These problems are all the more reason for extending Open Access (OA)
> and developing OA scientometrics, which will provide open, validatable
> and calibratable metrics for research, researchers, and institutions in
> each field -- a far richer, more sensitive, and more equitable spectrum
> of metrics than the few, weak and unvalidated measures available today.
>
> Some research groups that are doing relevant work on this are, in the UK:
> (1) our own OA scientometrics group at Southampton (and UQaM, Canada),
> and our collaborators Charles Oppenheim (Loughborough) and Arthur Sale
> (Tasmania); (2) Mike Thelwall (Wolverhampton); in the US: (3) Johan
> Bollen & Herbert van de Sompel at LANL; and in the Netherlands: (5)
> Henk Moed & Anton van Raan (Leiden; cited in the Nature news item).
>
> Below are excerpts from the Nature article, followed by some references.
>
> Universities seek reform of ratings.
> http://www.nature.com/nature/journal/v447/n7144/full/447514b.html
>
> [A] group of US colleges [called for a] boycott [of] the most
> influential university ranking in the United States... Experts argue
> that these are based on dubious methodology and spurious data,
> yet they have huge influence...
>
> "All current university rankings are flawed to some extent; most,
> fundamentally,"
>
> The rankings in the U.S. News & World Report and those published by
> the British Times Higher Education Supplement (THES) depend heavily
> on surveys of thousands of experts - a system that some contest. A
> third popular ranking, by Jiao Tong University in Shanghai, China,
> is based on more quantitative measures, such as citations, numbers
> of Nobel prizewinners and publications in Nature and Science. But
> even these measures are not straightforward.
>
> Thomson Scientific's ISI citation data are notoriously poor for
> use in rankings; names of institutions are spelled differently from
> one article to the next, and university affiliations are sometimes
> omitted altogether. After cleaning up ISI data on all UK papers for
> such effects... the true number of papers from the University of
> Oxford, for example, [were] 40% higher than listed by ISI...
>
> Researchers at Leiden University in the Netherlands have similarly
> recompiled the ISI database for 400 universities: half a million
> papers per year. Their system produces various rankings based on
> different indicators. One, for example, weights citations on the
> basis of their scientific field, so that a university that does well
> in a heavily cited field doesn't get an artificial extra boost.
>
> The German Center for Higher Education Development (CHE) also offers
> rankings... for almost 300 German, Austrian and Swiss universities...
> the CHE is expanding the system to cover all Europe.
>
> The US Commission on the Future of Higher Education is considering
> creating a similar public database, which would offer competition
> to the U.S. News & World Report.
>
> ---------------------------------------------------------------------------
>
> Bollen, Johan and Herbert Van de Sompel. Mapping the structure of
> science through usage. Scientometrics, 69(2), 2006
> http://dx.doi.org/10.1007/s11192-006-0151-8
>
> Hardy, R., Oppenheim, C., Brody, T. and Hitchcock, S. (2005) Open
> Access Citation Information.
> http://eprints.ecs.soton.ac.uk/11536/
>
> Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003)
> Mandated online RAE CVs Linked to University Eprint
> Archives: Improving the UK Research Assessment Exercise
> whilst making it cheaper and easier. Ariadne 35.
> http://www.ecs.soton.ac.uk/~harnad/Temp/Ariadne-RAE.htm
>
> Shadbolt, N., Brody, T., Carr, L. and Harnad, S. (2006) The Open
> Research Web: A Preview of the Optimal and the Inevitable, in Jacobs,
> N., Eds. Open Access: Key Strategic, Technical and Economic Aspects,
> chapter 21. Chandos. http://eprints.ecs.soton.ac.uk/12453/
>
> Harnad, S. (2007) Open Access Scientometrics and the UK Research
> Assessment Exercise. Invited Keynote, 11th Annual Meeting of the
> International Society for Scientometrics and Informetrics. Madrid,
> Spain, 25 June 2007 http://arxiv.org/abs/cs.IR/0703131
>
> Kousha, Kayvan and Thelwall, Mike (2006) Google Scholar Citations and
> Google Web/URL Citations: A Multi-Discipline Exploratory Analysis.
> In Proceedings International Workshop on Webometrics, Informetrics
> and Scientometrics & Seventh COLLNET Meeting, Nancy (France).
> http://eprints.rclis.org/archive/00006416/
>
> Moed, H.F. (2005). Citation Analysis in Research Evaluation.
> Dordrecht (Netherlands): Springer.
>
> van Raan, A. (2007) Bibliometric statistical properties of the 100
> largest European universities: prevalent scaling rules in the science
> system. Journal of the American Society for Information Science and
> Technology http://www.cwts.nl/Cwts/Stat4AX-JASIST.pdf
>
> Stevan Harnad
> AMERICAN SCIENTIST OPEN ACCESS FORUM:
> http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/
> To join or leave the Forum or change your subscription address:
> http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
>
> UNIVERSITIES: If you have adopted or plan to adopt an institutional
> policy of providing Open Access to your own research article output,
> please describe your policy at:
> http://www.eprints.org/signup/sign.php
>
> UNIFIED DUAL OPEN-ACCESS-PROVISION POLICY:
> BOAI-1 ("green"): Publish your article in a suitable toll-access journal
> http://romeo.eprints.org/
> OR
> BOAI-2 ("gold"): Publish your article in a open-access journal if/when
> a suitable one exists.
> http://www.doaj.org/
> AND
> in BOTH cases self-archive a supplementary version of your article
> in your institutional repository.
> http://www.eprints.org/self-faq/
> http://archives.eprints.org/
> http://openaccess.eprints.org/
>
Received on Tue Jun 05 2007 - 16:31:51 BST