On Tue, 25 Mar 2003, Bob Parks wrote:
> You (Stevan) ought to be able to get to the JCR product on
> http://wos.mimas.ac.uk/jcrweb/
The UK national site license allows human access but not software-agent
access, which is what we need (not just for this analysis of putative
same-journal citation bias, but for a variety of other impact-related
studies we want to do). We have been discussing the possibility of
a collaborative research project with ISI, however, and if this goes
forward then we could have a look at same-journal citation patterns too.
> In www.iaes.org/journal/aej/dec_01/liner_pdf.pdf,
> they correct for self-citations
I believe that's only for author self-citations, not for same-journal
citations.
>sh>As a measure of "degree of egocentricity" relative to overall
>sh>in/out/self citation patterns I (who am not a statistician!)
>sh>would at first be inclined to look at the mean and standard error
>sh>for the ratio: S = self/(in + out)
>sh> T = in/(in + out)
>sh>and then for: 1/(T-S) as a rough measure of egocentricity.
>
> Well that is (in +out)/(in - self) and I am not sure what
> that really means.
>
> I would think that self/(in + out + self) would be a better
> first try. Near 1 is very egocentric and near 0 is not very
> egocentric... [but] I don't think ONE summary measure can capture
> it very well, given the three in/out/self types.
I agree that one measure will not be sensitive enough. We will
experiment with this. (I think that "self" is part of "in" by the way,
in the ISI figures, though I may be wrong.)
>sh>It goes without saying that once the journal literature is open-access,
>sh>potential journal-based biases like this will be far less consequential
>
> HUH? Why? If we have OA (complete, universal, all refereed articles),
> AND we have journals, then why would journals (in that utopian future)
> change their current biases?
I said it would be far less consequential. It is consequential in the
toll-access era, because journal impact factors partly determine which
journals are subscribed to (licensed) by institutions, and therefore
they partly determine what we do and do not have (toll)-access to.
When all the annual 2,000,000 papers in all 20,000 refereed journals
are self-archived and openly accessible to all potential users web-wide,
whether or not their institutions can afford a subscription (license)
to the toll-access version, then it *matters* far less what the journal
impact factor happens to be, whether or not it has been inflated, and
whether or not a given institution, as a result, subscribes to
(licenses) the toll-access version. The self-archived open-access
version of everything is available to all would-be users in any case.
Moreover, the scientometric correction for any inflated same-journal
citations could even be corrected in authors' *personal* citation
counts, if desired. The open-access full-text database could be used
in a much more powerful and flexible way by all users and evaluators of
research productivity. (In other words, I may lose a few citations because
they are detectably just part of the inflated same-journal citations of
some journal they have appeared in. (But I hope you agree that this bit
of fine-tuning is not likely to be very consequential either.)
> So, is the 'consequential' bias "there is every temptation to get those
> journal impact factors as high as possible" and that would go away in OA?
> If journals serve the same purpose in the UOA (Utopian OA) as they do now,
> won't that temptation be the same?
The core purpose served by journals in the UOA (universal open access)
era will be exactly the same as it is now: to provide peer-review
and to certify publication standards as having been successfully
met (at that journal's established quality level). This is the core
"publish-or-perish" function, and it remains unchanged. How long the
additional journal publication functions (paper version, publisher's
online PDF, dissemination, storage, access-provision) will
continue to be needed (and hence paid for), instead of being off-loaded
entirely on the interoperable instiututional self-archiving network,
is not something that I or anyone can or need guess.
http://www.nature.com/nature/debates/e-access/Articles/harnad.html#B1
All that needs to be understood is that once there is open access,
no more potential usage or impact will be lost because of inability to pay
access tolls: and that is the *only* thing open-access is about.
http://www.ecs.soton.ac.uk/~harnad/Temp/unto-others.html
(And, as I said, if there is still any residual temptation to inflate
journal impact through same-journal citation, it will matter a good deal
less, and will be a lot more detectable and correctable.)
>sh>because there will be many direct measures of a paper's or author's
>sh>research impact, among which the citation impact factor of the journal
>sh>in which the paper appeared will be a relatively minor one.
>
> Well, I can only think within my own profession. If journals are
> around in the UOA, I think that their 'rankings' will be about the
> same and for the same reasons. Some will get higher, some lower,
> but for the most part they will remain the same.
But who will care, since it is the impact of the research and the
researcher that matters, not the impact of the journal (which is merely
the average impact of the papers it publishes)?
> I don't see why what journal the article appeared in as being
> a minor measure. The current situation is based on the referee
> system and self selection. Top journals have top referees and
> get top articles. UOA will not lessen that, and I doubt that
> dept chairs, or deans will think that an article in a third
> tier journal is worth much even if all of the other 'direct
> measures' available in UOA are high.
But I agree completely! The top journals (i.e., the ones exercising
the most rigorous peer review and selectivity, hence maintaining the
highest quality standards) will continue to be given due weight for
that -- along with the weight coming from the article's and the author's
various measures of research impact (usage ["hits"], citations, "authority
co-citations," etc.). Research impact will not be estimated by just the
one-dimensional measure consisting of the journal's average citation
count, but by a rich and diverse regression equation, with multiple
weighted predictors.
http://www.ecs.soton.ac.uk/~harnad/Temp/self-archiving.htm
Stevan Harnad
Received on Wed Mar 26 2003 - 00:41:52 GMT