ED 251 062

AUTHORTITLE

SPANS AGENCY

PUB DATECONTRACTNOTE

PUB TYPE

DOCUMENT RESUME

IR 050 856

Watt, James H., Jr.; Stefanov, RebeccaAn Exploratory Study of Factors Affecting Usage of anOn-Line Computer-Based Bibliographic RetrievalService.National Inst. of General Medical Sciences (NIH),Bets- -4da, Md.25 84NO1-ft._-0-234523p.; Paper presented at the Annual Conference of theIntern tional Communication Association (SanFrancisco, CA, May 24-28, 1984).Reports - Research/Technical (143)Speeches/Conference Papers (150)

EDRS PRICE MF01/PC01 Plus Postage.DESCRIPTORS Databases; Health Materials; *Health Personnel;

Information Needs; *Information Retrieval;Information Seeking; *Information Sources; OnlineSystems; Questionnaires; *Training Methods; *UserSatisfaction (Information); Use Studies

IDENTIFIERS *End Users; *MEDLINI

ABSTRACTThe situational and personal factors affecting the

usage of a computer based information service by health careprofessionals are described. Of 166 health care professionals whowere iientified as potential users of MEDLIN' at the University ofConnecticut, 126 carried out one or more online searches. Pre-use andpost-use questionnaires were administered to solicit information inthe following areas: job profile, information needs, current use ofinformation sources, satisfaction with information sources,experience with automated information systems, beliefs aboutcomputers and automated information systems, physical difficulty inusing the system, and type of =LINE training received. It wasconcluded that the relevance of the information available to theimmediate job-related problems of the user is the major determinantof the first use of the service. Total information needs,dissatisfaction with other sources of information, and beliefs aboutthe satisfying nature of computer interaction and the reliability ofcomputer machinery are predictive of more frequent use. This reportis part of a National Institutes of Health (NIH) project carried outat five locations in the United States. (Author/THC)

***********************************************************************Reproductions supplied by EDRS are the best that can be made

from the original document.***********************************************************************

ti_14

4.,

I, U.S . DEPARTMENT OF EDUCATIONNATIONAL INSTITUTE OF EDUCATION

EDUCATIONAL RESOURCES INFORMATIONCINTIR ERSCr

I* The. document has NMI , recendutect cosmteered tnon the proon u uowezatchn(Inclement,' rt

Mem. chi t. 'we. been n.srir ft. .T1Slf414IsMith t.on m..stme

Puens ..t se.* u. oiu nwt, §.1.1ted re Irv% u

IDIrult an r% me+ rstuuliy IellfeV.111 at/C..1i Nitpusmon eel.",

CV AN EXPLORATORY STUDY OF FACTORS AFFECTING USAGE OF AN ON-LINE.1) COMPUTER-BASED BIBLIOGRAPHIC RETRIEVAL SERVICEC:2

C:3

LU

James H. Watt, Jr.

Rebecca Stefancv

Department of Communication SciencesThe University of Connecticut

Box U-85Storrs, CT 06268

(203) 486-2628

Presented to the International Communication Associationconference.

Human Communication Technology Special Interest Group

San Francisco. May 25, 1934

This research was partially supported by NIH Award No.NO1-NS-0-2345

2

annual

'PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY

James Watt

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER tERIC)."

ABSTRACT

An exploratory study of the situational and personal factors

affecting the usage of a computer-based information service by

health care professionals is described. The user population was

free to use, or not to use, the service, and to use it as fre-

quently as they chose. It is concluded that the relevance of the

information to the immediate job-related problems of the user

population is the major determinant of the first use of the

service. Total information needs, dissatisfaction with other

sources of information. and beliefs about the satisfying nature

of computer interaction and the reliability of computing machi-

nery are predictive of more frequent use.

AN EXPLORATORY STUDY OF FACTORS AFFECTING USAGE OF AN ON-LINECOMPUTER-BASED BIBLIOGRAPHIC RETRIEVAL SERVICE

INTRODUCTION

his paper describes an exploratory study. The objective of

study was to explore factors which might predict the usage

a d non-usage of a computer-based bibliographic information re-

trieral system, and for those who chose to use it, the amount of

use they made of the system. It involves a population of users

who are not normally oriented toward using computer systems.

The potential users of the service were not forced to use it

to carry out their jobs. This situation is somewhat different

from other studies of user acceptance of data retrieval systems.

The information provided by the system described has varying

value for different members of the potential user population.

Some persons might find the ability to scan over a million jour-

nal entries quite central to their jobs. Other persons might

need this service only rarely, if at all.

The service provided by the information retrieval system can

also be obtained without the use of computers, at a good univer-

sity library, for example. However, it requires much more time

and effort to retrieve literature citations and abstracts manu-

ally, but this must be balanced against the time and effort

required to learn to use the computer-based information service.

There are enough unique aspects to the setting and user

population in this study to make confident predictions about user

behavior impossible. For that reason, an exploratory approach

was taken.

To study the possible determinants of use of this system, a

set of general factors which logically might affect usage pat-

terns were developed. Specific questions about these factors

were administered to the potential user population via a pre-use

and a later post-use questionnaire.

The data analysis of these rather extensive measurements

proceeded more from general speculations about what ought to

affect usage of the service than from formal hypotheses. The

data analyses presented in the latter part of this paper are thus

more descriptive than inferential, but they may represent some-

thing of value to persons interested in providing information

services for a population of users similar to those described

below.

Some General Factors Which May Affect Use of InformationRetrieval Systems

Physical Factors. The first requirement for use of a commu-

nication srstem based on computer hardware physical access to

the hardware. This very basic requirement is often assumed by

t;lose who study man-machine interaction. In a realistic situa-

tion, a user may have the choice of using some alternative to the

computer information system. If the computer terminal is at a

remote location, or if it is heavily used and the potential user

must wait for access, the individual will probably be less likely

to make use of the information system. The amount of use made of

2

the computer system is probably dependent on a simple time and

information cost/benefit function. In the case of bibliographic

citations, if a library with printed indices is closer, and if

the user feels that enough information can be obtained from this

source in a reasonably short time period, the person will probab-

ly not use the computer-based system.

Eason and Damodaran (1981) discuss this time minimization in

the context of determining interaction system requirements for

task-oriented users, but its effect extends to initial access, as

well. Sackman (1970) also relates the time to learn to access

information to the liklihood that use of the information will be

made.

Benefit Factors. Another quite basic requirement of a com-

puter-based communication system is that the potential user must

have some need for the service provided. These systems are often

developed with little knowledge of the real information needs of

the user population, and then presented to the users after deve-

lopment is completed. If the service is subsequently not used,

its failure may *'den be attributed to lack of "user-friendliness"

or some other such structural factor, when in reality the service

simply provides nothing that a potential user can use or could

not obtain through other methods. Eason (1976) provides a very

simple statement of this fact:

"The task the person is performing defines his interests in

information at that time."

A new information system, and especially one which requires

training to use, must solve some problem, or make some task

3

easier for potential users, for it to be accepted and used.

Satisfaction with Information Systems. This category is

really made up of satisfaction with the information systems cul--

rently being used, and satisfaction with the information system

being introduced.

The individual user's satisfaction with his current informa-

tion collecting systems should predict usage. If the person is

happy with the current system, there is no reason to expect that

he/she will expend the energy and time to learn a new system. If

he/she is unhappy with the status quo, we can expect more usage

of the new system.

The other type of satisfaction has more to do with the

user's personality. Malde, quoted in Eason and Damordaran

(1981), made a distinction between "thing-oriented" and "people-

oriented" individuals. "Thing-oriented" persons were more effi-

ciency centered, and felt that working with computer systems was

more personally satisfying that did "people-oriented" indivi-

duals. There is apparently some intrinsic satisfaction gained in

using computers, beyond that of obtaining information from them,

possibly because computers are seen as efficiency-enhancers.

Satisfaction of users with experience may also change.

Gaines and Shaw (1983) state experienced users often report

frustration with a seemingly rigid computer system. Hiltz and

Turoff (1978) describe a similar phase in experienced users,

which they called the "saturation phase". They describe it as

being produced by the evolution of new information needs within

4

the user (partly because of experience with the current informa-

tion system) which the system cannot meet. Satisfaction may thus

he related to the general category discussed next, the user's

prior experience with information retrieval systems.

Prior Experience. Coombs and Alty (1981) point out that

most computer-based data systems function primarily as one-way

information sources, and do not adapt to the needs of individual

users. The effectiveness of this kind of system is dependent

upon the level of the user's knowledge. Persons with prior

experience with computers, and especially with similar informa-

tion systems, might reasonably be expected to make more use of a

computer-based system than would a novice computer user.

Beliefs about Computers. This area has received some atten-

tion. Sackman (1970) found that non-users felt that computer

hardware was "unreliable and untrustworthy; ... software is unna-

turally difficult and requires too much time and painstaking

attention". Hiltz and Turoff (1978) document an apparently com-

mon belief that the computer will break if they do something

wrong. During some pilot interviewing for this study, some

individuals expressed beliefs that computers stifled creativity.

Others showed a general distrust of machinery, including compu-

ters. This was often expressed as a "machines are out to get mc"

sentiment (the proposed name for this is the "Woody Allen Syn-

drome").

5

THE EXPLORATORY STUDY

These general factors may affect any user population's

acceptance of a computer-based information service. This study

was carried out to determine which of them operate in the

specific setting of bibliographic information retrieval service

available to a professional population, the members of which are

naive or casual users of computer services.

The On -Line Service.

MEDLINE is a computer-based data retrieval system which

contains information about professional literature of interest to

health professionals. Typical information is a complete liters-

i.ure citation with author, journal, etc., and a short abstract of

the article. The data base is maintained by the National Library

of Medicine, under the administrative umbrella of the National

Institutes of Health. Over 3000 journals are indexed under

25,000+ key terms. The data base contains over 1,000,000 en-

tries, and is growing rapidly.

MEDLINE was designed to permit searches by persons familiar

with medical terminology, but not expert in computer operations.

However, each user must be able to operate a computer terminal,

dial into a timesharing service, enter account numbers and pass-

words, etc. MEDLINE dialog is not a natural language system, so

the user must also enter search requests in a fixed grammar, and

must interpret sometimes cryptic responses from the system. MED-

LINE is fairly typical of large data base information retrieval

6

systems in this respect. While the MEDLINE dialog is much easier

to use than a formal data base language, it still would not be

correct to call it "user-fri ndly".

To help new users, there is a rather extensive tutorial

program, MEDLEARN, which gives new users some experience opera-

ting typical computer terminals (how to backspace, cancel com-

mands. interrupt the MEDLINE mainframe computer's searches,

etc.). Two large users' manuals are also provided. These give

information about operational procedures, and also list the index

terms.

Many medical and research libraries also provide a MEDLINE-

trained person, called the Technical Information Specialist

(TIS), to help with searches. The TIS may sit with the user

while the search is carried out, giving advice. Or the liser

might submit a search topic to the TIS, and have the TIS carry

out all the computer operations.

A user thus has the choice of three ways of using MEDLINE:

1) an unassisted search; 2) an assisted search, where a TIS is

present for advice; and 3) a submitted search where the TIS does

all the computer interaction.

The Study Objectives

In 1980, NIH instituted an evaluation study of MEDLINE,

focussing on its usefulness to professionals in communication

disorders. Of particular interest were the determinants of usage

of the service. Factors such as job category, information needs,

experience vith computers, etc. were to be examined to determine

why some professionals used the MEDLINE service heavily, while

others did not. Other evaluations of MEDLINE were also to be

made, but those will not be discussed here.

The evaluation project began in September, 1980 and con-

tinued unt4.1 March, 1983. The initial six months were devoted to

developing a questionnaire to gather information about potential

users, and to administering this questionnaire and training the

potential users in MEDLINE procedures. The next 18 months were

spent monitoring MEDLINE usage, and the final six months were

used to process the data gained during the monitored usage phase.

The evaluation project was carried out at five locations in

the United States. A MEDLINE center was established at each

location, and a full-time TIS was provided for each center.

However, the data reported here were collected at the University

of Connecticut only. The full final report, with data from all

five centers, will be published shortly by the National Insti-

tutes of Health.

Study Procedures

The Sample. The potential user population at the University

of Connecticut MEDLINE center consisted of166 professionals in

Language Pathology (36% of the total group), Speech Pathology

(24%), Audiology (8%), with the remaining 32% spread among spe-

cialties such as Aural Rehabilitation, Psychoacoustics, Otolaryn-

gology, etc. The group included practicing professionals as well

as students pursuing advanced degrees (a little less than 22Z of

the total potential user group were students). This population

was recruited primarily during the initial six month start-up

period, and during the first 12 months of the monitored usage

period.

Pre-Use questionnaire. Before any searching was done, the

user was asked to fill out an extensive pre-use questionnaire.

This 13-page se1f-administered questionnaire contained questions

in the following areis:

1. Job Profile. Questions in this section included primary

and secondary job interests, highest degree granted, percentage

of time spent in different job settings, such as clinics, hospi-

tals, universities, etc.

2. Information Needs. Potential users were asked how fre-

quently they needed updates from various sources of professional

information such as journals, books, continuing education, etc.

3. Current Use of Information Sources. Respondents were

asked how frequently they used each source of information to keep

current and for comprehensive reviews of their area of spec-

iality.

4. Satisfaction with Information Sources. The respondent's

satisfaction with each source of information was recorded on an

8-point Likert-type scale.

$

912

5. Experience with Autometed Information Systems. Each

potential user was asked about prior experience with other bib-

liographic data base systems such as ERIC (Educational Resources

Information Center), BRS (Bibliographic Retrieval Services),

Lockheed Info Systems, etc. Both frequency of prior use and

satisfaction with the system were recorded.

6. Belies about Computers and Automated Information Sys-

tems. Fourteen items, listed in Appendix A, were developed from

the literature and from pilot interviews with potential users

during the initial six-month period of the study. Each item

represents a different belief, with no theoretical rational

items, so each question was initially treated as a single

indicator item in the data analysis.

7. Physical Difficulty in Using the Service. Potential

users were asked to estimate the travel time in minutes to the

MEDLINE center.

Post-Search Questionnaire. Each potential user who actually

carried out a search was asked to fill out a short questionnaire

immediately after completing the search. This questionnaire

recorded the search mode (Unassisted, Assisted, or Submitted),

the number of references generated, the user's satisfaction with

these references, and the user's satisfaction with the TIS's

help, if appropriate.

Post-Use Questionnaire. Each potential user who actually

carried out a search filled out a post-use questionnaire which

1310

contained items in the above 6 categories identical (or nearly

so--some minor changes were made in the wording of sore items by

request of the project director) to those in the pre-use

questionnaire. This permitted some before-after comparisons.

Potential users who did not carry out a search were asked to fill

out an abbreviated questionnaire, contai-ing some of the pre-use

questions which allowed them to serve as a pseudo-control group

for some analyses.

An additional question was asked of users in the post-use

questionnaire:

8. Type of Training Received. Three types of training were

available: Workshops, given at periodic intervals, TIS indivi-

dual training sessions, available by appointment, and training by

other MEDLINE users. Each user was asked to indicate the primary

type of training received.

RESULTS

Users vs. Non-Users.

Of the 166 potential users, 126 actually carried out one or

more on-line searches. Thirty-eight did no searching, and two

did not complete the questionnaire sequence sufficiently to per-

mit their inclusion in the date anillysis.

Chi-square statistics based on cross-tabulations (for non-

quantitative indicators) or t-tests (for onentitaPive indicators)

were used to determine which indicators in the eight question-

1114

naire areas were significantly related to the decision to

become a user or non-user of MEDLINE's services.

The data are most striking for their lack of predictive

power. Only one interest area, Speech Pathology, had a slightly

higher percentage of non-users than the other areas. Those with

degrees granted before 1966 or after 1976 were slightly more

likely to use the service, but these groups are also the ones in

which academic faculty and students are most likely to fall.

The kinds of job tasks carried out by the respondents, the

type of MEDLINE training received, and the travel time to the

MEDLINE center showed no ability to discriminate between users

and non-users. The appeal (and lack of appeal) of MEDLINE showed

no evidence of being produced by differences in job requirements,

training, or the difficulty in reaching the service. Neither did

any of the opinion items about computer systems predict use/non-

use.

In fact, the only item to clearly discriminate users from

non-users was the frequency with which the respondent needed

updates from journals, with those with higher frequency needs

being much more likely to be users (t=2.40, p<.02).

The dominant predictor was thus none of the items relating

to ease of use, familiarity with computer systems, beliefs about

computer systems, or training, but it is a rather important one:

whether the respondent had a need for the primary service pro-

vided by the computer-based communication system.

1512,

Amount of Searching Done ki Users

The object of this analysis was to identify factors which

discriminated users who carried out a number of searches from

those who carrieC out few or no searches. Linear regression was

used to predict the number of searches from a set of indices

which were constructed to summarize the individual indicators

into wore abstract variables. The constructed indices were:

Total Frequency of Update (TOTUP). This index was construc-

ted by summing the responses to the 8 categories of information

needs in the pre-use questionnaire. It represents the respon-

dent's total need for current information from all sources.

Total Information Usage (TOTINT). This index was construc-

ted by summing the frequency of usage responses in the 12 infor-

mation source categories of the preuse questionnaire. These

responses were weighted as follows, before summing:

"Every couple of years" = 1

"Annually" . 2

"Every 3-6 months" = 6

"Monthly" = 24"More than once a month" = 48

Total Use of Information Bases (TOT,SE). This index was

constructed by summing the number of times per year the respon-

dent used automated information retrieval services as reported

in the pre-use questionnaire. It thus represents prior experi-

ence with information retrieval systems.

1613

Total Satisfaction with Sources of Information (TOTSAT).

This index was constructed by summing the level of satisfaction

reported by respondents for all 13 possible sources listed in the

pre-use questionnaire.

In addition to these constructed indices, a number of indi-

vidual variables were included as potential predictors of the

level of searching carried out. All were taken from the pre-use

questionnaire. They were:

- - The 14 computer and information system belief items (seeAppendix A).

-- The number of minutes travel time to the MEDLINE center.

- - The year the highest degree was granted.

-- Type of training (from post-use questionnaire)a) workshopb) instruction from a friendc) by TIS

-- Professional interest in Speech Pathology (primary,secondary, or not at all)

Although some of these items did not discriminate between users

and non-users, an examination of zero-order product-moment corre-

lations indicated reasonably high covariance between the indivi-

dual items selected and the number of searches carried out by an

individual.

Stepwise regression was used in an exploratory analysis to

identify the strongest predictors of the amount of searching

done. Table 1 shows the results cf this regression.

Four statistically significant predictors were found. In

concert, they explain 12% of the variance in the number of sear-

chess This tigure is significant (p < .001). Each of the par-

tial statistics for the individual predictors are also signifi-

cant at p < .05. None of the additional predictors reached

significance. In addition, a simultaneous regression which in-

troduced all predictors in the same equation isolated the same

four predictor variables as the only significant ones. Taken

together, these results indicate that the four variables are the

optimal set for predicting the number of searches carried out.

The variable with the strongest predictive power was the

total information usage index (TOTINT). High information usage

of other types of information sources was predictive of more

frequent MEDLINE searches. Apparently there exists a group whose

members are high consumers of information. This group is more

inclined to use MEDLINE.

The second most predictive variable was one of the belief

items:

"Playing with computers can be fun and personallysatisfying."

Agreement with this item was predictive of a higher number of

searches. It may reflect a positive orientation toward automated

information systems, )r perhaps it is reflective of the possible

higher acquaintence of the high information consumers with compu-

ter equipment.

The third item was also a belief item:

"Computers, like most machines, always break down when youneed them most."

15 18

Agreement with this item predicted more searches. This may be an

unfortunate side-effect of reliance on a time-sharing computer

system for critical information. Persons likely to do more

searching are also those most likely to have been disappointed by

equipment and system failures.

The last predictive item was the total satisfaction with

sources of information (TOTSAT) index. The lower this index, the

higher the number of searches the respondent was likely to carry

out. Apparently those dissatisfied with more traditional sources

of information will use newer services such as MEDLINE to seek

satisfaction.

CONCLUSIONS

In this exploratory study, we reach the conclusion that the

best predictor of initial use of a computer-based information

service is a felt need for the information supplied. We find no

evidence of the predictive power of prior experience, personality

or belief variables, or even the extensiveness of training

received.

A composite picture of the more frequent user of MEDLINE

emerges from the analysis of those who actually made the decision

to use the service. He/She is a high consumer of information,

but one who is somewhat dissatisfied with current sources of

information. He/She regards computer equipment positively, but

is frustrated by unreliablity in the system'.

16

19

To the degree that the user population described in this

paper is representative of other task-oriented professionals who

are not primarily intereoted in computers, but in the information

which they can provide, we may draw some tentative conclusions:

1) Any computer-based information service must provide

information directly related to the jobs carried out by the

professionals. If they do not recognize the need for the infor-

mation service, they will not even sample the service. If the

potential user population is merely unaware of the value of the

system, an educational program might be called for. But if the

service cannot easily be shown to be valuable to the current

tasks being carried out by the professionals, it is unlikely to

be accepted.

2) Those who really see the need for the service will use

it despite inconveniences, and they will use it even if they have

had little or no prior experience with computer systems.

3) Increasing the reliability and availability of the in-

formation service will increase use of the system.

4) Increasing computer literacy and decreasing fear of

computing machinery by educational programs should increase use

of the information system.

17 20

S

TABLE I

NUMBER OF USES OF MEDLINE PREDICTED BY USER CHARACTERISTICS

PredictorVariable

BetaWeight F for Beta

======.2===Sig.112====

Total Information Usage .232 7.57 < .01

(TOTINT)

Total Satisfaction with - .183 4.67 < .05

Sources of Information(TOTSAT)

"Playing with computerscan be fun..." (Agree) .215 6.98 < .01

"Computers ... alwaysbreak down just when youneed them" (Agree)

.186 5.37 < .025

d.f. for Betas . 1:149

fig====t=============a=============Maa===

Multiple R . .343

F for Regression = 4.87 (4;146 d.f.) p < .001

APPENDIX A

BELIEFS ABOUT COMPUTERS AND AUTOMATED INFORMATION SYSTEMS

1. Computers are expensive pieces of sophisticated equipmentthat should be utilized only by trained experts.

2. Recent technology has made computer operation so simple evena layman can put them to good use.

3. Computers may be good for other people, but I personally haveno need to use one.

4. In today's society, everyone should know how to operate a

computer.

If I use a computer, I am afraid I might damagt.

6. Computers, like other machines, always break LJwn just whenyou need them most.

7. It takes too long to learn to use a computer for it to beuseful to me.

8. Computers are detrimental because they reduce creativethinking and restrict human development.

9. Playing with computers can be fun and personally satisfying.

10. } 1ving a computer-based information system available providesCommunication Disorder Specialists with an additional, usefultool.

11. Having a computer-based information Bytes would provide mewith en additional useful tool in my profession.

12. Computers are beneficial because they nave time* And Elveprofessionals greater freedom to investigate new ideas.

13. Computer systems and languages are just too complicated to

use for relatively simple jobs like reviewing professionalliterature.

14. Computer information eystems reduce the danger that I will beunaware of an important advance in my speciality.

==========n============

Answered on an 8-point scale, ranging from 1-DISAGREE to 8=AGREE

19 22

REFERENCES

Alty, J.L. and Coombs, M.J. Communicating with universitycomputer users: a case study. In M.J. Coombs and J.L. Alty,cosnuall skills and the user interface. New York: AcademicPress, 1981.

Coombs, M.J. and Alty, J.L. Computing skills and the user

interface. New York: Academic Press, 1981.

Eason, K.D. Understanding the naive computer user. ComputerJournal, 19, 3-7.

Eason, I.D. and Damodarian, L. The needs of the commercial user.In M.J. Coombs and J.L. Alty, Computing skills and the user

interface. New York: Academic Press, 1981.

Gaines, B.Z. and Shaw, M.L.G. Dialog engineering. In M.E. Simeand M.J. Coombs, itsitiim for Human-Computer Communication.New York: Academic Press, 1983.

Hiltz, S.R. and Turoff, M. The network nation: Humancommunication via computer. Reading, Mass.: Addison-Wesley,

Sackman, H. Man-Computer problem solving. Princeton, N.J.:

Auerbach, 1970.