I think the following comments on the Craig et al. review are meant to
be friendly to OA, but they do seem to involve some rather profound
misunderstandings:
On Mon, 21 May 2007, arif wrote:
> I think any OA citation study ignores a significant artifact.
> That is, most authors citing other authors will already have institutional
> subscriptions that allow them broad access.
But that is precisely what the studies comparing OA and non-OA
articles' citation counts are trying to test: The only difference
between OA and non-OA articles is that the OA ones are accessible to
everyone and the non-OA ones are only accessible to those at subscribing
institutions. So the comparisons between their citation counts (especially
those between OA and non-OA articles in the same journal and same year)
are testing how much citation impact the non-OA articles are losing, in
not being OA.
> These are publicly subsidized
> through library purchases of journal subscriptions (where costs have
> increased 300% above inflation in recent years.) Their experience of
> doing literature review makes non-OA articles economically the same,
> on an individual level, as OA. The author will not be cited more if
> they publish in OA, but by different types of people, and they will be
> read by different people including more people who aren't authors.
I could not follow this. The "types of people" by which a non-OA articles
will be cited are the types that are at institutions that can afford
subscription access to the journal in which that article was published.
The additional "types of people" that provide the additional citations
for the articles that have been made OA are those who could not afford
access to the subscription version.
(I am afraid that you may implicitly be assuming that these OA/non-OA
comparisons as being made between different *journals*: OA (Gold) ones
and non-OA ones. There are a few such studies too, but they suffer from
comparing apples and oranges, as journals differ in subject matter,
quality and citations independently of whether they are OA or non-OA. The
right comparison is between articles in the same journal and year that
have and have not been made (Green) OA by author self-archiving. And
that is what most of the citation studies are based on.)
> For authors, it's those without privileges who are largely without access.
> Thus, they will have a hard time authoring in journals they don't get
> to read.
This also seems to address a completely different issue, since the
question is not about authoring, but about accessing and citing. The
extra citations of OA articles come from the fact that that the article
-- regardless of which journal it appeared in -- is freely accessible to
all would-be users online. The users can then cite it in their own
articles, regardless of which journal they publish their articles in.
> The citation study cannot capture the effect of all the people
> who haven't got into publishing in journals because of lack of access.
> Apart from being methodologically flawed, it's logically inept.
I am lost. And it is not clear here whether the "citation study" refers
to the Craig et al. critical review, or the many empirical studies it is
reviewing. In any case, this has nothing to do with success or failure at
getting articles published; it has to do with success or failure in
getting articles accessed (and hence potentially applied, used, and
cited).
> The question for authors shouldn't be "will I get a citation advantage",
> but will my article be readable by all who have a desire and capacity to
> read and benefit from it? Getting cited is not the spirit of scholarship,
> merely a by-product.
Correct. And citations are only one among many potential measures of
usage and impact. Downloads are another. But we have citation counts for
OA and non-OA articles to compare systematically, whereas we do not have
download counts for systematic comparison. (But we do know that early
download counts are correlated with later citation counts.)
Brody, T., Harnad, S. and Carr, L. (2006) Earlier Web Usage Statistics
as Predictors of Later Citation Impact. Journal of the American
Association for Information Science and Technology (JASIST) 57(8)
pp. 1060-1072.
http://eprints.ecs.soton.ac.uk/10713/
Getting cited is not the sole or primarily goal of scholarship, but
getting one's findings read, used, and built upon, is.
> Thus, you can detect the bias of the article
> already, it's assumption that the only important question for authors is
> "will I get cited?", as if we're children and not thinkers. It plays
> into the deplorable elitism in scholarship that passes for meritocracy.
I think you may be missing the point. The Craig et al. article was
commissioned by a publishing consortium to find flaws in the studies
that have reported increased citation impact for OA articles. The
purpose of the Craig et al. critique was to try to delay or deter the
provision of OA through author self-archiving and Green OA mandates.
The purpose of the OA/citation impact studies themselves was to show the
advantages of OA. Authors and their institutions do care about
citations; so if OA increases citations, that increases their motivation
to provide OA (as 85% of them are still not doing) and even to mandate
it.
> I read an abstract a couple years ago for a wonderful study that was
> doing participatory action research with people with mental illness, and
> in the PAR paradigm, we say that the participants are the researchers,
> sometimes attributing co-authorship to them. I was disappointed not
> to be able to access the article, which would have cost me $40 or so.
> I e-mailed the author to tell him that those with mental illness, his
> population, are quite often not wealthy enough to read this work, and
> that is a shame, and that unfortunately my university didn't subscribe
> to the journal so I wouldn't be able to use it in my review.
So, did he send you a reprint? OA would provide limitless eprints, to all
would-be users. And the prospect of increased citations increases the
incentive to provide OA.
> Those who wrote the article reinforce what Harold Innis called a
> 'monopoly of knowledge', and Innis was a Canadian who studied mediums of
> communication along with Marshall McLuhan. Those who already have get
> more, those who don't are shut out. Innis was not a mechanistic thinker,
> this was a pattern he had observed. He didn't negate the possibility of
> a communication medium that had potential to de-monopolize.
This is all theory and ideology. But inducing authors to self-archive
is a practical matter, and the citation studies are trying to show them
the benefits -- while the publishing lobby is trying to show them that
those benefits are not real.
> In my view, the citation study article is rubbish. It tries to take
> a positivist approach without context or rationality. It is entirely
> self-serving and a shameful case of science finding what it's looking for.
> And it's boring to read.
No comment.
> Most authors who publish probably have institutional access that makes
> non-OA journals functionally the same for them as OA journals, ie. they
> don't make purchases from their wallet per article and they don't
> personally buy subscriptions. It's fine to talk of evidence-based,
> but where's the logic?!
No, most authors who publish do not have access to all or most of the
potentially pertinent journals. That's the point of OA.
> So the research question is ultimately flawed, and it ignores the fact
> that both OA and non-OA articles likely suffer since they exist in
> different places in web-space, in different types of searching, and in
> different cultures of research.
No, OA articles are freely available, webwide (e.g., via Google Scholar)
whereas non-OA articles are only available to licensed subscribers.
> A more interesting research question would measure the impact of open
> access in regions where subscription to an on-line journal is limited
> or non-existent.
That *is* what the citation studies are measuring, indirectly.
> Or take articles that have the least and most OA
> citations and determine the characteristic of the author by a) their
> awareness of OA and b) the level of access they already have subsidized
> through their institutions.
OA citations? Citations are citations, and an article is either OA or
not OA. One could of course try to study more closely who is citing
what, and even to look at what they have paid access to. But that is a
rather more detailed study than necessary. The handwriting is already
on the wall: More access, more impact. And one could almost have deduced
that from logic alone (though one could not have induced how much
impact), by simply noting that access is a necessary (though not a
sufficient) condition for impact -- and that most institutions cannot
afford access to most of the world's 24,000 journals.
Stevan Harnad
> \----- Original Message -----
> From: "Stevan Harnad" <harnad_at_ecs.soton.ac.uk>
> To: <AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM_at_LISTSERVER.SIGMAXI.ORG>
> Sent: Sunday, May 20, 2007 11:57 AM
> Subject: [BOAI] Craig et al.'s review of the OA citation advantage
>
>
> ** Cross-Posted **
>
> Craig, Ian; Andrew Plume, Marie McVeigh, James Pringle & Mayur
> Amin (2007) Do Open Access Articles Have Greater Citation Impact?
> A critical review of the literature. Journal of Informetrics.
> http://www.publishingresearch.net/Citations-SummaryPaper3_000.pdf.pdf
>
> I've read Craig et al.'s critical review ("proposed by the Publishing
> Research Consortium") concerning the OA citation Impact effect and
> will shortly write a short, mild review. But first here is a commentary
> from Bruce Royan, followed by Sally Morris's posting, followed by a few
> remarks from me. -- Stevan Harnad
>
> g ---------- Forwarded message ----------
> g Date: Sun, 20 May 2007 08:00:02 +0100
> g From: Bruce Royan <bruce.royan-- concurrentcomputing.co.uk>
> g To: diglib--infoserv.inist.fr
> g Subject: RE: [DIGLIB] Recent research tempers citation advantage of open
> g access
> g
> g Hmm.
> g
> g Sally claims that according to this article "the relationship between
> g open access and citation, once thought to be almost self-evident,
> g has almost disappeared."
>
> g Now I'm no Informetrician, but my reading of the article is that
> g the authors reluctantly acknowledge that Open Access articles do
> g have greater citation impact, but claim that this is less because
> g they are Open Access per se, and more because:
>
> g -they are available sooner than more conventionally
> g published articles, or
>
> g -they tend to be better articles, by more prestigious
> g authors
>
> g Sally's point of view is understandable, since she is employed by a
> g consortium of conventional publishers. It's interesting to note that
> g the employers of the authors of this article are Wiley-Blackwell,
> g Thomson Scientific, and Elsevier.
>
> g Even more interesting is that, though this article has been accepted
> g for publication in the conventional "Journal of Informetrics", a pdf
> g of it (described as a summary, but there are 20 pages in JOI format,
> g complete with diagrams, references etc) has already been mounted on
> g the web for free download, in what might be mistaken for an example
> g of green route open access.
>
> g Could this possibly be in order to improve the article's impact?
> g
> g Professor Bruce Royan http://www.linkedin.com/in/bruceroyan
> g Concurrent Computing Limited. Registered Office:
> g Wellington House, Aylesbury Rd, Princes Risborough, Bucks HP27 0JP
> >>
> >>From: Sally Morris info--publishingresearch.net
> >>Sent: 17 May 2007 18:00
> >>Subject: [DIGLIB] Recent research tempers citation advantage of open access
> g
> g 'Do Open Access Articles Have Greater Citation Impact?
> g A critical review of the literature'
> g Ian Craig, Andrew Plume, Marie McVeigh, James Pringle and Mayur Amin.
> g
> >>A new, comprehensive review of recent bibliometric literature finds
> >>decreasing evidence for an effect of 'Open Access' on article citation
> >>rates. The review, now accepted for publication in the Journal of
> >>Informetrics, was proposed by the Publishing Research Consortium (PRC) and
> >>is available at its web site at www.publishingresearch.net. It traces the
> >>development of this issue from Steve Lawrence's original study in Nature in
> >>2001 to the most recent work of Henk Moed and others.
> g
> >>Researchers have delved more deeply into such factors as 'selection bias'
> >>and 'early view' effects, and began to control more carefully for the
> >>effects of disciplinary differences and publication dates. As they have
> >>applied these more sophisticated techniques, the relationship between open
> >>access and citation, once thought to be almost self-evident, has almost
> >>disappeared.
> g
> >>Commenting on the paper, Lord May of Oxford, FRS, past president of the
> >>Royal Society, said 'In December 2005, the Royal Society called for an
> >>evidence-based approach to the scholarly communications debate. This
> >>excellent paper demonstrates that there is actually little evidence of a
> >>citation advantage for open access articles.'
> g
> >>The debate will certainly continue, and further studies will continue to
> >>refine current work. The PRC welcomes this discussion, and hopes that this
> >>latest paper may be a catalyst for a new round of informed scholarly
> >>exchange.
> g
> >>Sally Morris
> >>on behalf of the Publishing Research Consortium
> >>Email: info--publishingresearch.net
> >>Website: www.publishingresearch.net
>
> It is notoriously tricky (at least since David Hume) to "prove" causality
> empirically. The thrust of the Craig et al. critique is that despite the
> fact that virtually all studies comparing the citation counts for OA and
> non-OA articles keep finding the OA citation counts to be higher, it has
> not been proven beyond a reasonable doubt that the relationship is causal.
>
> I agree: It is merely highly probable, not proven beyond a reasonable
> doubt, that articles are more cited because they are OA, rather than
> OA merely because they are more cited (or both OA and more cited merely
> because of a third factor).
>
> And I also agree that not one of the studies done so far is without some
> methodological flaw that could be corrected.
>
> But it is also highly probable that the results of the methodologically
> flawless versions of all those studies will be much the same as the
> results of the current studies. That's what happens when you have a
> robust major effect, detected by virtually every study, and only ad hoc
> methodological cavils and special pleading to rebut each of them with.
>
> But I am sure those methodological flaws will not be corrected by these
> authors, because -- OJ Simpson's "Dream Team" of Defense Attorneys comes
> to mind -- Craig et al's only interest is evidently in finding flaws and
> alternative explanations, not in finding out the truth -- if it goes
> against their client's interests...
>
> Iain D.Craig: Wiley-Blackwell
> Andrew M.Plume, Mayur Amin: Elsevier
> Marie E.McVeigh, James Pringle: Thomson Scientific
>
> Here is a preview of my rebuttal. It is mostly just common sense,
> if one has no conflict of interest, hence no reason for special
> pleading and strained interpretations:
>
> (1) Research quality is a necessary, but not a sufficient condition for
> citation impact: The research must also be accessible to be cited.
>
> (2) Research accessibility is a necessary but not a sufficient condition
> for citation impact: The research must also be of sufficient quality to
> be cited.
>
> (3) The OA impact effect is the finding that an article's citation counts
> are positively correlated with the probability that that article has
> been made OA: The more an article's citations, the more likely that that
> article has been made OA.
>
> (4) This correlation has at least three causal interpretations:
>
> (4a) OA articles are more likely to be cited.
>
> (4b) More-cited articles are more likely to be made OA.
>
> (4c) A third factor makes it more likely that some articles will be
> both made OA and more cited.
>
> (5) Each of these causal interpretations is correct, and hence a
> contributor to the OA impact effect:
>
> (5a) The better the article, the more likely it is to be cited,
> hence the more citations it gains if it is made more accessible
> (3a). (OA Article Quality Advantage, QA)
>
> (5b) The better the article, the more likely it is to be made OA
> (3b). (OA Article Quality Bias, QB)
>
> (5c) 10% of articles (and authors) receive 90% of citations. The
> authors of the better articles know they are better, and hence are
> more likely both to be cited and to make their articles OA, so as
> to maximize their visibility, accessibility and citations (3c). (OA
> Author QB and QA)
>
> (6) In addition to QB and QA, there is an OA Early Access effect (EA):
> providing access earlier increases citations.
>
> (7) The OA citation studies have not yet isolated and estimated the
> relative sizes of each of these (and other) contributing components
> (OA also increases downloads, and downloads are correlated with later
> citations).
>
> (8) But the handwriting is on the wall as to the benefits of making
> articles OA, for those with eyes to see, and no conflicting interests
> to blind them.
>
> I do agree completely, however, with erstwhile Princetonian Bob May's call
> for "an evidence-based approach to the scholarly communications debate."
>
> Stevan Harnad
> http://www.princeton.edu/~harnad/
> http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
Received on Mon May 21 2007 - 16:33:33 BST