Rethinking Metrics of International Growth and Impact of Open Access

AAAS 2012, “Flattening the World: Building a Global Knowledge Society”Vancouver, Canada

Symposium on Innovations in Reducing International Knowledge Isolation

Leslie ChanBioline InternationalUniversity of Toronto Scarborough

Key points• Open Access as an enabler • “Journal” no longer serves the needs of networked

scholarship• From Wealth of Nations to Wealth of Networks• Need to rethink measurements of “impact” and

values, especially for development• Innovations are happening in the “peripheries” but

there are gatekeepers • Aligning funding and reward policies with new metrics

The problem

• Access to literature from the North• Dissemination of research from the South• The barriers are structural and institutional

http://www.medknow.com/

The World of Journal Publishing According to Thomson’s ISI Science Citation Index

Data from 2002http://www.worldmapper.org/display.php?selected=205

“… at a recent editorial team meeting, we discussed a research paper from a LMIC author. The science was well done and with a little editing for English, the paper was potentially publishable. But should we send it out for review? The question we were wrestling with was whether its findings were sufficiently new to make it worthy of page space in the journal. This is always a consideration for all manuscripts, since competition for space is intense and a priority is to publish interesting research that adds something new to the field, rather than too many replications of studies already done. So the initial response when deciding whether to send the paper out for peer review was: Reject. We already know this, don't we?”

“No journal can afford to devote all or even most of its precious page space to studies essentially finding again what others already found, with only the places changing. And this may be a good place to remind authors that we almost never publish prevalence studies, unless they are truly the first ever done (and sometimes not even then), since they tend to be of interest primarily in the countries within which they were conducted.”

So who decide on what is “new” and legitimate knowledge?And

Who have access to that knowledge?

“We editors seek a global status for our journals, but we shut out the experiences and practices of those living in poverty by our (unconscious) neglect. One group is advantaged, while the other is marginalised.”Richard Horton, THE

LANCET • Vol 361 • March 1, 2003

“Research or reviews that cover diseases unlikely to be encountered in the western world will not gather the citations that some editors seek.But if this commercial environment does seriously skew content away from what matters to those people the journal claims to serve, as it surely does at some journals, the culture of medicine is distorted, even harmed.”Richard Horton (2003)

The Political Economy of Knowledge Production

“Is the scientific paper a fraud?”“I mean the scientific paper may be a fraud because it misrepresents the processes of thought that accompanied or give rise to the work that is described in the paper. That is the question and I will say right away that my answer to it is ‘yes’. The scientific paper in its orthodox form does embody a totally mistaken conception, even a travesty, of the nature of scientific though”.

Sir Peter Medawar(From a BBC talk, 1964)

http://contanatura-hemeroteca.weblog.com.pt/arquivo/medawar_paper_fraud.pdf

“Weeds” or Vegetables?

http://www.bioline.org.br

The Centro de Referência em Informação Ambiental, CRIA (Reference Center on Environmental Information) http://www.cria.org.br/

26

Retrieved April 26, 2010 from: http://maps.google.com/maps/ms?ie=UTF8&hl=en&msa=0&msid=117328677437228595850.00044ec490b69019e64a3&z=2

Active Bioline Journals graphed with Google Maps.

10th Anniversary!

29

Retrieved April 26, 2010 from: http://analytics.google.com

Bioline users March 27 – April 26, 2010

But

The World of Journal Publishing According to Thomson’s ISI Science Citation Index

Data from 2002http://www.worldmapper.org/display.php?selected=205

http://thomsonreuters.com/

http://ke.thomsonreuters.com/#/index.html

$$$

From “Big” science to Networked science

Knowledge for local problem solving

OPEN ACCESS ?

The IF is negotiable and doesn’t reflect actual citation counts

http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0030291

The IF cannot be reproduced, even if it reflected actual citations

http://jcb.rupress.org/content/179/6/1091.full

The IF is not statistically sound, even if it were reproducible and reflected actual citations

http://www.mathunion.org/fileadmin/IMU/Report/CitationStatistics.pdf

The IF are more eff

http://iai.asm.org/content/early/2011/08/08/IAI.05661-11.full.pdf+html?view=long&pmid=21825063

http://beyond-impact.org/

http://altmetrics.org/manifesto/

The SURFfoundation has released Users, Narcissism and Control—Tracking the Impact of Scholarly Publications in the 21st Century.

Governance of Knowledge Commons

Need for policy alignment and institutional redesign

Rethink the values and reward system

Social Accounting and Expanded Values

Broadening the definition of “success”, “impact”, “value” and “capital”

Business value monetary return, financial capital, efficiency, competiveness

Scholarly value Reputation and citation; trust; symbolic capital

Institutional value Public mission, community outreach, intellectual capital

Social value Equity, participation, diversity, social capital

Political value Evidence based policy, transparency, accountability, civic capital

Institutional Design

Sustainability as a set of institutional structures and processes that build and protect the knowledge commons (after Sumner 2005, Mook and Sumner 2010)

Conclusions

• Open Access is just the substrate, but an essential one

• Metrics drive behaviour, but we have been using the wrong metrics

Thank You! chan@utsc.utoronto.ca

http://www.openoasis.org

http://www.bioline.org.br

http://www.openaccessmap.org