The Counseling Psychologist
-
Upload
dante-sparda -
Category
Documents
-
view
223 -
download
0
Transcript of The Counseling Psychologist
-
7/28/2019 The Counseling Psychologist
1/29
http://tcp.sagepub.com
The Counseling Psychologist
DOI: 10.1177/00110000042722572005; 33; 299The Counseling Psychologist
Carolyn A. Karr and Lisa M. LarsonFrom 1990 to 1999
Use of Theory-Driven Research in Counseling: Investigating Three Counseling Psychology Journ
http://tcp.sagepub.com/cgi/content/abstract/33/3/299The online version of this article can be found at:
Published by:
http://www.sagepublications.com
On behalf of:
Division of Counseling Psychology of the American Psychological Association
can be found at:The Counseling PsychologistAdditional services and information for
http://tcp.sagepub.com/cgi/alertsEmail Alerts:
http://tcp.sagepub.com/subscriptionsSubscriptions:
http://www.sagepub.com/journalsReprints.navReprints:
http://www.sagepub.com/journalsPermissions.navPermissions:
http://tcp.sagepub.com/cgi/content/refs/33/3/299Citations
at KANSAS STATE UNIV LIBRARIES on May 14, 2009http://tcp.sagepub.comDownloaded from
http://www.div17.org/http://tcp.sagepub.com/cgi/alertshttp://tcp.sagepub.com/cgi/alertshttp://tcp.sagepub.com/subscriptionshttp://tcp.sagepub.com/subscriptionshttp://www.sagepub.com/journalsReprints.navhttp://www.sagepub.com/journalsReprints.navhttp://www.sagepub.com/journalsPermissions.navhttp://www.sagepub.com/journalsPermissions.navhttp://tcp.sagepub.com/cgi/content/refs/33/3/299http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/cgi/content/refs/33/3/299http://www.sagepub.com/journalsPermissions.navhttp://www.sagepub.com/journalsReprints.navhttp://tcp.sagepub.com/subscriptionshttp://tcp.sagepub.com/cgi/alertshttp://www.div17.org/ -
7/28/2019 The Counseling Psychologist
2/29
10.1177/0011000004272257THE COUNSELING PSYCHOLOGIST / May 2005Karr, Larson / USE OF THEORY-DRIVEN RESEARCHUse of Theory-Driven Research in Counseling:
Investigating Three Counseling Psychology Journals
From 1990 to 1999
Carolyn A. KarrUniversity of MissouriKansas City
CCS Assessment Center
Lisa M. LarsonIowa State University
Three major journals in counseling psychology were sampled from 1990 to 1999 to
assess the percentage of quantitative, empirical articles that were theory driven. Only
43%of thestudies utilized a theoryor model,and57% predicted therelationbetweenthevariables, withfew studies specifyingthe strength of the relation. Studies sampledin the
Journal of Counseling Psychology (63%) andthe Journal of Vocational Behavior (65%)
reported a significantly higher percentage of theory-driven research than the Journal of
Counselingand Development(43%). A higher proportion ofJournal of Counseling Psy-
chology studies compared with Journal of Counseling and Development studies
anchored findingsto theory.Fewstudies replicatedpreviousstudies.This paucity implies
that replication is underutilized by the field as a potentially fruitful way to substantiate
empirical knowledge. A binary logistic regressionwas conductedto examine trends over
time. No trends emerged. Suggestions for future implementation of theory and
implications for counseling psychology as a field are offered.
Counseling psychologists have alerted the field that too little theory-
driven research is being generated (Harmon, 1982; Heppner, Kivlighan, &
Wampold, 1999; Strong, 1984, 1991a, 1991b; Tracey, 1991; Tracey &
Glidden-Tracey, 1999). An excellent example of theory-driven research has
been thetestof John Hollands hexagonof sixinterests (e.g.,Fouad,Harmon,
& Borgen, 1997). Recent studies in multicultural (e.g., Prosser, 2003) and
vocational psychology(e.g., Blanchard& Lichtenberg, 2003) indirectlysug-
gest a greater need for theory-driven research in these areas as well.
A recent review of the process and outcome literature for test interpreta-
tion found that only 9% of studies were based on theory (Hanson & Good-
year, 2000). This observation is not limited to counseling psychology, how-
ever. Meehl (1978, 1990, 1993) voiced similar criticisms about psychology
in general, and social psychology has been concerned with the related areas
of meaningful hypothesis construction (e.g., Wallach & Wallach, 1994) and
299
Theauthorswould like to thankDougBonett forhis suggestions andguidancewith thestatisticalanalyses and Jennifer Swaim for her aid in data compilation.
THE COUNSELING PSYCHOLOGIST, Vol. 33 No. 3, May 2005 299-326
DOI: 10.1177/0011000004272257
2005 by the Society of Counseling Psychology
at KANSAS STATE UNIV LIBRARIES on May 14, 2009http://tcp.sagepub.comDownloaded from
http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/ -
7/28/2019 The Counseling Psychologist
3/29
replication of studies (e.g., Neuliep & Crandall, 1991). Their concerns are
noteworthy giventheusefulness of theory-driven research and thepaucity of
empirical studies that test established theories.
Theories andmodelsare crucial forseveral reasons. Theories aremeant to
be organized explications of the dynamics that underlie a given psychologi-
cal phenomenon (Strong, 1991a). They are fundamental tools to understand
the construct being researched. Meehl (1990) articulated a more specific
function of theories in empirical studies by advocating that theories should
be used to help determine whether the substantive theory, the statistical
hypotheses, and the empirical observations are related in terms of logical
inference and theoretical causality.
The definition of theory used for this study was a general principle for-
mulated to explain a group of related phenomena (Chaplin, 1985, p. 467).For the purposes of this study, a model was construed as a description of the
assumed structure of a set of observations (Everitt & Wykes, 1999, p. 119).
Although similar, the former utilizes a general tenet to explain related inter-
actions, while the latter describes the expected observable interactions in
more detail. By definition, theories and models are similar in function and
scope. Forster (2000) stated that the best way to distinguish theories and
models is to discuss each in conjunction with predictive hypotheses. In his
conceptualization, the threeare hierarchically arranged, with theories at the
most general level, models applied to concrete systems in the middle, and
predictive hypotheses at the lowest level, which result from fitting models to
data (Forster, 2000, p. 233). He emphasized that the essential point of this
tripartite distinction is that predictive accuracy is a property of predictive
hypotheses at the very bottom of the hierarchy, and is traded-off against thetruth at thenext levelupthe levelof models (Forster, 2000, p. 233). In this
way, both theories1 and models are tested by the utilization of tailored
predictive hypotheses.
Theory-driven research also provides a conceptual framework that mini-
mizes the chances that researchers efforts will lead to the generation of a
multitude of unconnected facts (Strong, 1991a). Strong (1991b) explained
the furthering of scientific thought necessarily involves a circular pro-
cess that begins with theory, carries this theory to the context of testing, is
held to specific observations, and is interpreted through the context of dis-
covery. Discovery then follows into further redefinition and redescription of
theory. This grounded, circular process ensures that utility is blended with
conceptualization.
Moreover, use of theory easily guarantees that a meaningful question is
asked because of the network of supplementary and predictive informa-
tion that is part of the theory (Meehl, 1978). Kazdin (1992) noted that much
theory-free research is considered meaningful by using the rationale that a
300 THE COUNSELING PSYCHOLOGIST / May 2005
at KANSAS STATE UNIV LIBRARIES on May 14, 2009http://tcp.sagepub.comDownloaded from
http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/ -
7/28/2019 The Counseling Psychologist
4/29
particular study has not yet been conducted. He emphasized, however, that
literally thousands of variables could be researched together. Conglomera-
tions of variables without links to theoretical networks suggesting the spe-
cific andsignificant reasons why theyare chosen areessentially meaningless.
The specificity of the hypotheses is also important in conducting theory-
driven research. In particular, research questions should be asked in terms of
whether the data will fall within a specific meaningful range, based on theo-
ries that include components that are likewise relatively narrow and focused
(Meehl, 1993). Instead, many hypotheses based on specific theories do not
identify the strength of the relations among the theoretical constructs.
Meehl (1978,1990, 1993) haschampioned thebenefitsof hypotheses that
predict whether experimental observations will fall within a certain, rela-
tively narrownumeric range andwhether theobservationswill be specifiedapriori. He explains the advantages of numeric hypothesizing over traditional
significance testing; numeric hypothesizing allows researchers to ask, does
the theory have sufficient verisimilitude to warrant our continuing to test it
and amend it rather than, is the theory literally true? (Meehl, 1990). He
defines verisimilitude as the relationship between the theory and the real
world which the theory speaks about (Meehl, 1990, p. 113).
Wampold, Davis, and Good (1990) have further specified the need for
hypothesis validity, which is the extent to which research results reflect
theoretically derivedpredictions aboutthe relations betweenor amongcon-
structs (p. 360). In this manner, hypothesis validity ensures that the rela-
tion between constructs is informed by theory. Therefore, by utilizing
hypotheses that include precise numeric ranges (e.g., effect sizes, variability,
and confidence intervals), we can better assess how expectations and result-ing observations are linked, while simultaneously evaluating thecurrent the-
ory (Greenwald, Gonzalez, Harris, & Guthrie, 1996; Howard, Curtin, &
Johnson, 1991; Levin & Robinson, 1999; Wampold et al., 1990).
The magnitude and direction of the relation should be stated a priori
because theory identifies more than the important constructs. Theory also
should identify the posited strength of the relation among the constructs
not the exact statistical value (e.g., .09) but rather small, medium, and large
effects and positive and negative relations. This specification provides the
template for hypotheses to be formulated. Examining the expected magni-
tude of the relation a priori, in comparison with the actual magnitude of the
relation, contributes important information in determining how variables
are related. For example, based on social cognitive theory, if we expect self-
efficacy andoutcome expectations to be moderately related (rvalues ranging
from .3 to .5) and find instead that the relation across five studies is weaker
(i.e., r= .15), then the theory must be adjusted.
Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 301
at KANSAS STATE UNIV LIBRARIES on May 14, 2009http://tcp.sagepub.comDownloaded from
http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/ -
7/28/2019 The Counseling Psychologist
5/29
Identifying discrepancies can further hone theoretical relations, contrib-
uting to advances in theory. By examining expectations of effect magnitude
in theory and prior research, and then incorporating these expectancies into
the studys hypotheses, potential extra variables or relations are more easily
identified, which could lead to new discoveries. In practice, identifying the
magnitude of relations a priori is much like the use of effect sizes (Cohen,
1969) in meta-analyses, in that effect size provides a quantified measure of
the relation for comparison purposes. Allen (1995) stated that effect size
indicates how strongly the independent variable influences the dependent
variable, or the strength of the relationship (p. 82). Effect sizes, used in this
way, provide a measure of themagnitude of a givenrelationand aregenerally
more informative than hypotheses and analyses that simply aim to reject the
null hypothesis (Cohen, 1994).Effect sizes are also used in replication of research, because strict null
hypothesis testing does not include the probability that a result can be repli-
cated(Gigerenzer & Murray, 1987;Lykken,1968). Replication of researchis
important in the assessment and reformation of theory, using effect sizes as
necessary vehicles to assess relations and further knowledge (Cohen, 1994).
Prior research can be useful in determining numeric range estimates for
hypothesized relations, as wellas informingthe directionof these relations.
Some researchers question the viability of theory-driven research and
even suggest that theory can thwart research progress under certain condi-
tions. Greenwald, Pratkanis, Leippe, and Baumgardner (1986) argued that
over-reliance on theory can cause confirmation bias when researchers per-
severe by revising procedures until obtaining a theory-predicted result.
Researchers discovered this bias when they noticed poor preliminary datatendedto cloud theinterpretationof laterdata of betterquality(Bruner & Pot-
ter, 1964; Wyatt & Campbell, 1951).
Others have agreed that research is frequently developed by way of
confirmation-biased practices (e.g., Popper, 1959). More specifically, chang-
ing conditions and procedures until the desired result is obtained is consid-
ered theoryconfirming (Greenwald et al., 1986) or prejudiceagainst thenull
hypothesis (Greenwald, 1975), which strays from the ideal empirical meth-
ods of hypothesis testing and resulting theory disconfirmation. Remedies are
offered in the form of utilizing condition-seeking and design approaches,
which are used, respectively, to discover conditions on which an existing
finding depends and to specify conditions that can produce an unobtainable
result. Greenwald et al. (1986; refer for lengthier discussion) stated that the
need for theorytoguide empiricalprogress, along with thestimulus to theory
that is provided by new findings, rescues thecondition-seeking methodfrom
being just a means of cultivating empiricaltrivialities(p. 224). Thefinal step
302 THE COUNSELING PSYCHOLOGIST / May 2005
at KANSAS STATE UNIV LIBRARIES on May 14, 2009http://tcp.sagepub.comDownloaded from
http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/ -
7/28/2019 The Counseling Psychologist
6/29
is then to revise, expand, and/or rearticulate theories to incorporate the new
conditions.
A relatedobjectionto theory-drivenresearchis thecontention that theory-
driven researchnecessitates abandoningstudies derived fromempiricaldata.
In actuality, empirical data can inform future studies and inform/revise the-
ory. The authors are not suggesting that all research should be based on the-
ory. Rather, most studies should be theory-driven, with initially atheoretical
data used to specify and refine existing theories. Or new theories can be con-
structed to explain existing empirical and currently atheoretical data. By uti-
lizing remedies for theory-driven research problems, we ensure the produc-
tion of meaningful, theory-grounded data. The current study attempts to
identify the current proportion of theoretical to atheoretical research. By
identifying thetheoriescurrently utilizedin thesamplewewill also launchaneffort to determine the extent to which theories have been substantiated or
further developed by new data.
Still, despite these objections, theory-driven research may unify con-
ducted research, especially among findings linked to similar, but different,
fields within psychology (Staats, 1983). For example, many objectives of
counseling psychology are similar to those of clinical psychology, social
work, and school psychology.Yet findings and viewpoints have been largely
linked within one field without integrating findings into a larger, unifying
framework. Shared theoretical frameworks would serve to unify both psy-
chological research and disciplines (Forsyth & Strong, 1992), benefiting not
only the pursuit of science but also professional relationships and activities
that can then contribute to a more common scientific ground. In short, the
generation of theory-driven research is vital.The purpose of this study was to empirically examine during 10 years
(1990 to 1999) the extent to which major counseling psychology journals
havepublished theory-driven research. Specifically, the authors will address
two questions. For the first question, the overall proportion of theory-driven
research articles to quantitative research articles will be determined in a
representative sample of counseling psychology journals. Although the
profession has called for more theory-driven research, no studies have been
conducted to examine the extent to which major journals in counseling psy-
chology publish theory-driven research. Omer and Dar (1992) examined the
research purposes posited in studies appearing in the Journal of Consulting
and Clinical Psychologyacrossthe 1960s,1970s,and1980s. They found that
the trend in clinical research has moved away from theoretical bases for
research questions and has moved toward pragmatism.
Two factors led to the selection of this particular time frame: (a) The aim
of the study was to measure trends in utilization of theory for the 10 years
Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 303
at KANSAS STATE UNIV LIBRARIES on May 14, 2009http://tcp.sagepub.comDownloaded from
http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/ -
7/28/2019 The Counseling Psychologist
7/29
before the new century (2000) to assess how the field should conduct
research in the current century, and (b) an active discussion regarding the
optimal degree of theory utilization in counseling psychology surfaced just
before the1990s (e.g., Heppner & Claiborn,1989; Serlin, 1987) andagain in
the early 1990s (Gelso, 1991; Patton & Jackson, 1991; Strong, 1991a,
1991b), with similar concerns voiced elsewhere in psychology during this
time (e.g., Gergen, 1991). Potential trends and/or changes in theory utiliza-
tion in counseling psychology during the 1990s could then at least be par-
tially attributed to discussions held around this time. The field would benefit
if future research addressed these concerns for the first 10 years of the cur-
rent century (2000 to2009) forthe purpose of comparison. Giventhe valueof
theory-driven research, data must be collected to ascertain the extent to
which researchers are embedding research questions in theory.For the second question, for those studies that appear to be theory driven,
the authors will determine the extent to which the studies report a priori the
directionand strengthof therelations of certain variables in thetheory. Meehl
(1978) argued that if research is to inform the broader discipline of psychol-
ogy, the parameters of the strength of the hypothesized relation must be
specified a priori.
METHOD
Sample: The Studies
The three most prominent and primarily quantitative journals in counsel-ing psychology were sampled: Journal of Counseling Psychology (JCP),
Journal of Vocational Behavior(JVB),andJournal of Counseling and Devel-
opment(JCD). The Counseling Psychologistwas excluded because its arti-
cles are more conceptual. These three journals were selected because they
have traditionally been associated with empirical research. For example,
when studies have examined the prominence of counseling psychology pro-
grams across the United States, the extent to which the faculty publish in
these journals (including The Counseling Psychologist) is used to measure
the productivity of the counseling psychology faculty.
To obtain a representative sample with sufficient power, every seventh
article was sampled. A power analysis was conducted to determine adequate
sample size. The power analysis was conducted before data collection, and
indicated sampling every seventh article would generate a sample size withsufficient statistical power. By utilizing Power and Precision for Windows
(Biostat,2001), an estimated150 total studies analyzed viac2 with sevencri-
teria acrossthree journals (effect size = .50;a= .05) yielded a powerestimate
304 THE COUNSELING PSYCHOLOGIST / May 2005
at KANSAS STATE UNIV LIBRARIES on May 14, 2009http://tcp.sagepub.comDownloaded from
http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/ -
7/28/2019 The Counseling Psychologist
8/29
of .99. Actual sample size of the studies2 consisted of the following:JCP, n =
70;JVB, n = 63; andJCD, n = 60;N= 193. Because the total number of stud-
iessampled wasmore than 150, power is considered to be of a sufficient level
for this study.
Procedures
Many decision rules were employed during the selection and analysis of
the sampled studies. The sampling began with the first volume in each jour-
nalthatwaspublished in1990. If anarticle containedmore than onestudy, all
the studies in that article were used, resulting in a total of 193 studies in JCP,
JVB, and JCD. If the seventh article was not data-based, the next article
would be sampled. Thus, reviews and conceptual articles were skipped ifthey were the seventh article. Given that qualitative articles are generally
used to build, rather than disconfirm, theory, they also were not sampled.
Thus, if the seventh article was qualitative, the next article was sampled.
Qualitative studies were excluded for three primary reasons. First, this
studywasa foray into measuringresearchtrends in this mannerin counseling
psychology; qualitative studies were not included to keep the decision crite-
riauniform for theentiresample. Second, qualitative studies arebased on the
inductive, rather thandeductive, approach.Thisstudyexaminedthe quantita-
tive research tradition in the field rather than the qualitative, which contains
different assumptions. Third, research would benefit the assessment of the-
ory utilization for qualitative studies but was beyond the scope of this study,
partly because of the aforementioned factors.
Development of the Content of the Questions
The seven questions presented in Table 1 were derived from the literature
calling for more theory-driven research. Questions 1 through 4 and Question
7 were derived from Tracey andothers (Meehl, 1978; Tracey, 1991; Tracey &
Glidden-Tracey, 1999), who argued that theory should be used at all stages,
from conceptualization of questions and hypotheses to interpretation of
results and the discussion of the findings within the field. Question 3 was
based on Wallach and Wallach (1998), who argued that useful hypotheses
shouldbespecific.For example, Wampoldet al.(1990) noted that thefollow-
ingstatedpurpose of a study is an ambiguous research hypothesis (one iden-
tified threat to hypothesis validity):
Thepresentstudy attempted todetermine a) therelationof parentaladjustmentmeasuresof suchvariablesasdepression,marital satisfaction, parenting stress,and other negative life stressors to mothers and fathers perceptions of their
Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 305
at KANSAS STATE UNIV LIBRARIES on May 14, 2009http://tcp.sagepub.comDownloaded from
http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/http://tcp.sagepub.com/ -
7/28/2019 The Counseling Psychologist
9/29
TABLE1:P
ercentagesofSampledStudiesCodedYesor
NoontheQuestionsand
2byJournalandO
verall
JCP
JVB
JCD
Overall
JCP/
JVB/
JCP/
n=70
n=63
n=60
N=193
JVB
JCD
JCD
Question
Yes
No
Yes
No
Yes
No
Yes
No
c2
c2
c2
1.Isthestudy
embeddedinatheory/model?
44.3
5
5.7
47.6
52.4
36.7
63.3
43.0
57.0
.15a
1.51
.77
2.Ifso,istheintroductionorganizedaround
thistheory/m
odel?b
90.3
9.7
80.0
20.0
81.8
18.2
84.3
15.7
1.29
.03
.81
3.Ofthestudiesthatutilizeatheory/model,are
thehypothesesdrawndirectlyfromt
heory?
87.1
1
2.9
73.3
26.7
77.3
22.7
79.5
20.5
1.83
.11
.88
4.Ofthestudiesthatutilizeamodel/theory,are
thefindings
anchoredtothatmodel/theory?
86.7
1
3.3
73.3
26.7
59.1
40.9
74.4
25.6
1.67
1.17
5.15*
5.Istherelatio
nbetweenthevariablespredicted?
62.9
3
7.1
65.1
34.9
43.3
56.7
57.5
42.5
.07
5.86*
4.96*
6.Isthepredic
teddegreeoftherelationbetween
thevariablessignificant?c
17.1
8
2.9
7.9
92.1
6.7
93.3
10.9
89.1
2.52
.07
3.29
7.Isthestudy
adirectextensionofoneormore
previousstu
dies?
17.1
8
2.9
11.1
88.9
8.3
91.7
12.4
87.6
.99
.27
2.21
NOTE:JCP=
JournalofCounselingPsychology;JCD=JournalofCounselingandDevelopment;JVB=Jour
nalofVocationalBehavior.
a.Eachc
2equationincludedonedegreeoffreedom.
b.ForQuestions2through4,asubsetofthesamplewasused(JCP,n=31;JVB,n=30;JCD,n=22).
c.Nostudiesi
npilotdatapredictedthestrengthoftherelatio
nsbetweenthevariables.
*p