Center for Information and Communication Studies Trust in Reading, Citing, and Publishing Carol...

Center for Information and Communication Studies

Trust in Reading, Citing, and Publishing

Carol TenopirUniversity of Tennessee, Knoxville, USA

[email protected]

NFAIS 2014

mailto:[email protected]


Trust and authority in scholarly communication project:

• Funding by the Alfred P. Sloan Foundation

• Research led by David Nicholas at CIBER Research Ltd. in the UK and Carol Tenopir at the Center for Information and Communication Studies (CICS), University of Tennessee

• September 2012-November 2013

•Collaborators include Taylor & Francis, SAGE, PLoS, Biomedcentral, Wiley, and Elsevier

CIBER Research Ltd.


Aims of project:

•Study how academics assign authority and trustworthiness to sources they read, cite, and publish in

•Examine behaviors and attitudes of academics in changing digital times


TRUST:Quality, Reliability, Trustworthiness

All in all then, perceived quality/reliability/trustworthiness is the prime criterion

scholars use in the discovery process (finding information), in the information management process (separating dispensable from indispensable relevant

material), in the citation process (formally using information) and in the dissemination process

(where and how researchers choose to have their work published).


Multiple Methods to Study Trust

• 14 focus groups of 66 academics in the UK & US

• In-depth interviews of 87• International survey of over

3800 researchers• Surveys by Tenopir & King

1977-2013 provide some context


Mean amount of reading by academics in Australia, U.S. and U.K.

n=2117, 6 UK institutions, June 2011n=837, 5 US institutions, January 2013n=133, 2 AU institutions, 2012

0 5 10 15 20 25 30

8.19

10.03

9.58

6.82

6.8

6.95

24.55

20.81

22.32

Oth

er P

ub

lica

tio

ns

B

ooks

Art

icle

s

AU US UK


This represents a big investment in time. Academics report spending…

•49 minutes per article•33 minutes per article•29 minutes per article

•106 minutes per book•101 minutes per book•~100 minutes per book*

•42 minutes per other•29 minutes per other•23 minutes per other

*estimated


Traditional Indicators of TRUST in Journal Articles

• Journal Name

• Journal Reputation

• Author Expertise


The reality of trust (from focus groups):

• They read many things they “trust” but would never cite (e.g. Wikipedia)

• Politics influence citing and publishing

• Cite to protect yourself and add “trustworthiness”

• Publish to help your career

• Use different criteria for reading, citing, and publishing


Which is most important in your publications to help readers trust?

1. Metrics like journal impact factor

2. Our brand name is trusted

3. Article level metrics (Altmetrics)

4. Abstracts

5. Well respected authors


How trustworthiness is determined for reading

• Read abstract and methodology

• Check for credible data and

sound logic

• Look at source’s references

• Colleague recommendations

• Familiarity with author or journal

• Peer-review linked to quality

• Impact factor a factor...


How trustworthiness is determined for citing

1.The author is known and trusted

2.The journal or conference is known and trusted

3.Seminal work in the field4.Supports methodology5.The research

group/institution is known and trusted


How trustworthiness is determined when deciding where to publish

• Traditional metrics (e.g., impact factor) still important

• Influenced by tenure

• Institutional research policies

• Audience of a journal

• Likelihood of getting published


How trustworthiness is determined by different disciplines

Life Scientists• More likely to use article content to

determine trustworthiness for reading

• Rely more heavily on peer review for reading

• More willing to cite & publish in peer-reviewed open access journals

Humanities Researchers• Read the entire information source

• More likely to use authority to determine trustworthiness for reading

• Feel most pressure to publish in traditional scholarly sources

Photos from www.freedigitalphotos.net

Alfred P. Sloan Foundation International Survey May – July 2013 (n=3,650) Significant differences based on mean score


How trustworthiness is determined by different age groups for reading

• Read abstract • Rely on colleagues

opinion• Check if index by

authoritative indexing body

• Look at number of downloads

• Check for sound logic

• Is it peer-reviewed?

• Check the name of the author

Younger Researchers are more likely to…

Older Researchers are more likely to…

Photos from www.freedigitalphotos.net

Alfred P. Sloan Foundation International Survey May – July 2013 (n=3,650)Significant differences based on mean score


How trustworthiness is determined by different age groups for publishing and

citingYounger researchers more likely to…

• trust nontraditional methods of dissemination

• feel pressure to publish in highly ranked journals to obtain research grants

• cite people they know because they trust them

• cite open access journals if properly peer-reviewed

Alfred P. Sloan Foundation International Survey May – July 2013 (n=3,650)Significant differences based on mean score


Peer Review

Across the board, researchers use the peer review process to determine trustworthiness in what they read, what they cite and where they publish.

But…


Comments on Peer Review

“Whilst peer review is essential it sometimes worries me that reviewers' opinions can be skewed by their own personal and professional prejudices…”

“I think it is difficult to publish data which goes against current trends in high impact peer review journals.”

“I am sure the fame of the author has an influence on accepting the paper for publishing.”


Trust and Impact Factor• More important for deciding

where to publish than what to read or cite

• Recognize that low-quality articles could be published in high IF journals

• High IF journals may lack innovative and fresh papers

• On the whole, younger academics trust impact factor for reading and citing more than older faculty do


Comments on Impact Factor

“It’s good because you want people to pay attention to your work BUT high impact factors do not always have the right audience for those who would use your work.”

“It’s an imperfect method but it’s the only one we have.”

“My tenure committee cares [about impact factor] but no one else does.”

“…I have found that impact factor and reputation are reasonably good indicators of overall empirical quality, they are insufficient by themselves to judge the quality of the findings of any given study…”


Trust and Altmetrics• Most participants in were unfamiliar with concept.• Others were skeptical of what the various altmetrics

actually meant.• Participants do like metrics that can be quickly

understood.• Authors like being able to see the number of people

who have viewed or downloaded their article.• Although they didn’t use the term, some alternative

metrics were mentioned...


I think it is “connectiveness” through name of friend who sent the link. You need to connect to a source to have trust. If information is isolated, just floating out there, I don’t trust it, but if it is connected to others then I trust it.

What is Trust in online environment?


Have digital communications changed how trust is determined?

“There is no one way to share or spread information, but do I want to trust all these new things?”

“Can no longer just say “only peer reviewed”. I’m unlearning what I used to do. I have to re-assess reliability when everything already digital. I used to be comfortable with print.”

“I’ve broadened what I consider a reliable source. It is easier to verify a source and I am less tentative.”

“We are better researchers in the digital era because we can look at research in more modalities.”


Open Access

• A lot of confusion• PLoS One considered

trustworthy• Common thoughts:

– Too expensive– Lower quality– Quick publication time– No review system– Suspicious of journal’s

motives


Use of Social Media

Image from: shopforfollowers.com


Which type of social media is used for “work related purposes” by most US faculty?

1. Blogs

2. YouTube videos

3. RSS feeds

4. Twitter

5. Comments on articles

6. Social Networking


Academics use a variety of social media for work related purposes

US n=579

Per

cent

Blogs

Video

s/You

tube

RSS Fee

ds

Twitter

User C

omm

ents

Podca

sts

Socia

l Net

workin

g0

10

20

30

40

50

60

70

80

90

100

5970

3323

5546

57


Top 3 social media used occasionally (U.S. 2012)

Blogs Social Networking Youtube/video

45 44 48

5 1317

2418

26

26 259

Daily

Weekly

Monthly

Occasionally

N=579

Per

cent

100%


Feelings about Social Media

2.20 2.22

3.54

Strongly Agree

Agree

Neither Agree nor Disagree

Disagree

Strongly Disagree

Average Ranking


Comments on social media“Same issues with social media as with art. We may pretend to know what is “good art” but often we don’t really know, so how can you judge quality with no basis?”

“I use Wikipedia to remind myself what I once knew.”

“There are different levels of quality of tweets. I don’t cite Twitter but I may use report linked via Twitter.”

“Social media can make top-tiered research more accessible.”


Influence of trust on use of social media

• Most are engaged at least occasionally.

• More critical and hesitant about trusting.

• Use many of the same standards to judge quality of social media as they use for traditional sources.

• Less trusting because no standard to judge quality.

• Less likely to create because not rewarded by university or tenure committee.


Overall findings:

?

TENURE


Trust and Authority in

digital environment

Phase 1: Scholarly communic

ation

Phase 2:Communic

ation outside

academia

Phase 3: Communic

ation in other

countries

Phase 4: Mobile

communication

•Academic researchers UK & US

•Government researchers•Lab researchers•Corporate researchers


Thanks!

For more information:http://cics.cci.utk.edu/cicsprojects/Sloan

Carol [email protected]

http://cics.cci.utk.edu/cicsprojects/Sloan

Center for Information and Communication Studies Trust in Reading, Citing, and Publishing Carol...

Documents

Transcript of Center for Information and Communication Studies Trust in Reading, Citing, and Publishing Carol...