3408479
Transcript of 3408479
IMPROVING STUDENT CRITICAL THINKING AND PERCEPTIONS OF
CRITICAL THINKING THROUGH DIRECT INSTRUCTION IN RHETORICAL
ANALYSIS
by
Lauren A. McGuire
BONITA WILCOX, Ph.D., Faculty Mentor and Chair
MAUREEN MCGLYNN, Ph.D., Committee Member
HOWARD JACOBS, Ph.D., Committee Member
Barbara Butts Williams, Ph.D., Dean, School of Education
A Dissertation Presented in Partial Fulfillment
Of the Requirements for the Degree
Doctor of Philosophy
Capella University
May 2010
UMI Number: 3408479
All rights reserved
INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion.
UMI 3408479
Copyright 2010 by ProQuest LLC. All rights reserved. This edition of the work is protected against
unauthorized copying under Title 17, United States Code.
ProQuest LLC 789 East Eisenhower Parkway
P.O. Box 1346 Ann Arbor, MI 48106-1346
© Lauren A. McGuire, 2010
Abstract
This study investigated the effect of direct instruction in rhetorical analysis on students’
critical thinking abilities, including knowledge, skills, and dispositions. The researcher
investigated student perceptions of the effectiveness of argument mapping; Thinker’s
Guides, based on Paul’s model of critical thinking; and Socratic questioning. The explicit
goals of the study were to promote critical thinking knowledge, skills, and dispositions
and to determine improvement in student perceptions of critical thinking knowledge,
skills, and dispositions. A mixed methodology teacher action approach was used in this
one-group pretest-posttest design research study which consisted of a small sample size
(N = 15) and pertained to a 1-semester critical thinking college course that was taught by
the researcher in a naturalistic setting. Changes in critical thinking were evaluated using
the pretest and posttest scores on the California Critical Thinking Skills Test 2000 and the
Pre- and Post- Critical Thinking Surveys. Data from the research findings indicated there
were no significant differences between the pretest and posttest scores for induction,
analysis, and evaluation; however, there was a slight difference between the pretest and
posttest scores for inference and deduction with a negligible significance level of (p =
0.2). Findings from the Pre- and Post- Critical Thinking Surveys indicated a 2.2 mean
gain in critical thinking knowledge, a 1.8 mean gain in critical thinking skills, and a 1.8
mean gain in critical thinking dispositions. These data suggest that purposeful
implementation of the above named interventions could strengthen students’ perceptions
of critical thinking and of their own critical thinking abilities.
iv
Acknowledgments
I wish to express my deepest gratitude to Dr. Bonita Wilcox, who served as my
mentor during the dissertation process and who has been truly inspirational. I will be
forever grateful for her knowledge, expertise, insightful guidance, and patience. She has
been instrumental in my achievements, and I consider myself incredibly lucky to have
had her as my mentor. I also wish to thank the other members of my committee, Dr.
Maureen McGlynn and Dr. Howard Jacobs, for their positive feedback, assistance, and
counsel. Finally, I wish to thank my husband, Terry, for his love and support and for his
unremitting encouragement and tolerance throughout this long, arduous, yet gratifying
educational journey.
v
Table of Contents
Acknowledgments iv
List of Tables viii
List of Figures ix
CHAPTER 1. INTRODUCTION 1
Background of the Study 4
Statement of Problem 7
Purpose of the Study 7
Research Questions 8
Significance of the Study 8
Definition of Terms 9
Assumptions and Limitations 12
Theoretical Framework 14
Nature of the Study 18
Organization of the Remainder of the Study 19
CHAPTER 2. LITERATURE REVIEW 20
Introduction 20
The Critical Thinking Movement 21
Defining Critical Thinking 24
Importance of Teaching Critical Thinking 30
Teaching Critical Thinking 34
Specific Strategies 38
Teaching Critical Thinking: Related Studies 42
vi
Conclusion 49
CHAPTER 3. METHODOLOGY 52
Introduction 52
Statement of Problem 52
Research Questions 53
Research Design and Methodology 54
Population and Sample 55
Instrumentation 57
Field Test 60
Data Collection 60
Data Analysis 63
Ethical Considerations 64
Limitations of the Study 65
Summary 66
CHAPTER 4. DATA COLLECTION AND ANALYSIS 68
Introduction 68
Statement of the Problem 69
Research Questions 69
Overview of Data Collection 70
Data Collection and Instruments 70
Demographic Characteristics 72
Results 75
Summary 96
vii
CHAPTER 5. RESULTS, CONCLUSIONS, AND RECOMMENDATIONS 98
Introduction 98
Overview of Methodology 99
Summary of Findings 99
Open-Ended Questions 109
Summary of Pre- and Post- Critical Thinking Surveys: Knowledge, Skills, and Dispositions 111
Limitations 119
Recommendations 120
Conclusions 123
REFERENCES 125
APPENDIX A. PRE- CRITICAL THINKING SURVEY 132
APPENDIX B. POST- CRITICAL THINKING SURVEY 136
APPENDIX C. REQUEST FOR PERMISSION TO DR. KIMBERLY METCALF 142
APPENDIX D. LETTER OF PERMISSION FROM DR. KIMBERLY METCALF 143
viii
List of Tables
Table 1. Sample Demographic Characteristics 73
Table 2. Sample Demographics: Skills and Abilities 74
Table 3. Student Perceptions of Critical Thinking: Argument Mapping 77
Table 4. Student Perceptions of Critical Thinking: Paul’s Thinker’s Guides 79
Table 5. Student Perceptions of Critical Thinking: Socratic Questioning 80
Table 6. Responses to Critical Thinking Survey: Knowledge 83
Table 7. Responses to Critical Thinking Survey: Skills 84
Table 8. Responses to Critical Thinking Survey: Dispositions 86
Table 9. Results of Pre- Critical Thinking Survey 88
Table 10. Results of Post- Critical Thinking Survey 89
Table 11. Pre- and Post- Critical Thinking Surveys: Gains 90
Table 12. CCTST–2000 Pretest Scores 92
Table 13. CCTST–2000 Posttest Scores 92
Table 14. t-Test Analyses: CCTST Pretest and Posttest Totals 93
Table 15. t-Test Analyses: Analysis, Inference, and Evaluation 94
Table 16. T-Test Analysis: Induction and Deduction 95
ix
List of Figures
Figure 1. Comparison of mean scores by question type 95
1
CHAPTER 1. INTRODUCTION
Educators agree that the development of higher order or cognitive intellectual
abilities is of utmost importance and that critical thinking “is central to both personal
success and national needs” (Paul, 2004, p. 2). In his article, Paul (1995f) cited
Kennedy’s research findings; the first finding indicated
National assessments in virtually every subject indicate that, although our students can perform basic skills pretty well, they are not doing well on thinking and reasoning. American students can compute, but they cannot reason. . . . They can write complete and correct sentences, but they cannot prepare arguments. . . . Moreover, in international comparisons, American students are falling behind . . . particularly in those areas that require higher order thinking. . . . Our students are not doing well at thinking, reasoning, analyzing, predicting, estimating, or problem solving. (p. 19)
Educators are beginning to explore those pedagogical practices that could
effectively develop student critical thinking knowledge, skills, and dispositions across the
academic disciplines. Further, instructional strategies that advance critical thinking
pedagogy on a consistent basis could enable instructors with the ability to encourage in
their students the transfer of those critical thinking skills learned in the academic
environment to their professional and personal lives.
In order to advance critical thinking pedagogy and encourage students’ critical
thinking abilities, however, educators must possess a clear definition of what critical
thinking is. As the concept of critical thinking is highly complex, a variety of definitions
exist, so it is difficult to pinpoint the exact meaning of the skills involved in this intricate
2
process. P. A. Facione (2000), using the two-sentence definition of critical thinking
generated from the Delphi Report, identified critical thinking as “the process of
purposeful, self-regulatory judgment. This process gives reasoned consideration to
evidence, context, conceptualizations, methods, and criteria” (p. 5). P. A. Facione argued
that a true definition of critical thinking involves both skills and habits of mind or
dispositions. Similar to P. A. Facione and his concern with the necessity for an
inclination toward critical thinking, Paul (1993) asserted that critical thinking is an
intellectually disciplined process “of actively and skillfully conceptualizing, applying,
analyzing, synthesizing or evaluating information” (p. 3). Halpern (1999) believed that
critical thinking is “purposeful, reasoned, and goal-directed” (p. 70), while maintaining
that “it is the kind of thinking involved in solving problems, formulating inferences,
calculating likelihoods, and making decisions” (p. 70). In a statement in 1997 for the
National Council for Excellence in Critical Thinking Instruction, Scriven and Paul (1987)
defined critical thinking as “self-guided, self-disciplined thinking which attempts to
reason at the highest level of quality in a fair-minded way” (p. 1). As with P. A. Facione,
Scriven and Paul contended that “critical thinking varies according to the motivation
underlying it” (p. 1), it is “never universal in any individual” (p. 1), and “the development
of critical thinking skills and dispositions is a life-long endeavor” (p. 1). Critical thinking
is not something that can be learned immediately; however, educators can encourage
students to be aware of their thinking so that it becomes intentional rather than accidental.
Critical thinking is a complex skill that takes a lifetime to master and that involves
deliberate practice and engagement (van Gelder, 2005).
3
The greatest contribution of the community college, according to Elder (2000),
has been its emphasis on teaching students those abilities needed for competency in the
workplace, yet Elder claimed that “training students for job performance in narrowly
defined skill areas no longer serves students well” (p. 1). Rather, current instructional
practices need to begin to place more emphasis on those critical thinking skills necessary
for survival in the new economy. What follows is the necessity for educators to begin to
teach those intellectual skills that encourage mental flexibility and intellectual discipline
so they can deal appropriately with complex intellectual tasks.
A primary goal for educators, then, is to provide students with opportunities to
struggle with concepts, find meaning, distinguish bias, and use logic in arguments so they
may gain a deeper understanding of the world in which they live. The question is what
teaching methodologies can be used by educators to promote the development of
students’ critical thinking skills?
Educators are beginning to place more emphasis on increasing students’ critical
thinking abilities while attempting to include critical thinking curriculum into a variety of
academic disciplines. Although colleges and universities offer critical thinking courses,
critical thinking can be embedded in the instruction of a variety of academic disciplines,
and faculty can engineer their course focus so that it is more thinking-skills-based
(Halpern, 1999). Mendelman (2007) believed critical thinking should be taught in every
course in the humanities, and stated
In a day and age in which more and more children grow up engaged with primarily passive activities like television, video games, and the Internet, teaching critical reading is one of the most important, and most difficult, burdens of the classroom. (p. 300)
4
Critical thinking abilities are necessary for success in college and for responsible
citizenry in a highly competitive society. As Elder suggested, true critical thinkers
Recognize the complexities in developing as thinkers, and commit themselves to life-long practice toward self-improvement. They embody the Socratic principle: The unexamined life is not worth living, because they realize that many unexamined lives together result in an uncritical, unjust, dangerous world. (as cited in Scriven & Paul, 1987, p. 2)
Background of the Study
Students entering postsecondary institutions often lack the necessary level of
literacy and cognitive skills required for academic success or for successful living in a
global economy. National assessments indicate that “although students can perform basic
skills pretty well, in international comparisons, American students are not doing well at
thinking, reasoning, analyzing, predicting, estimating, or problem solving” (Paul, 1995g,
p. 3). Van Gelder (2005) maintained that “a majority of people cannot, even when
prompted, reliably exhibit basic skills of general reasoning and argumentation” (p. 2).
Students are, or seem to be, unable to provide the essential evidence needed to support
their reasoning. Jackson (2008) discussed the need for educators to provide opportunities
for students to grapple with complexities so they may cultivate their problem-solving
abilities. Jackson interviewed Norbert Elliot, a teacher at New Jersey Institute of
Technology (NJIT). Elliot had launched a vigorous campaign at NJIT for information
literacy. His chief concern with students was their seeming inability to “sift, use, and
make meaning out of the information around them, in other words to read their world” (as
cited in Jackson, 2008, p. 163). In an internal assessment of information literacy, Elliot
observed that
5
The writing portfolios of NJIT’s seniors scored an average of 6.14 on a scale of 12, or just below “satisfactory.” Students could find and cite sources better than they were able to judge their relevance and authority, and were even less able to use information to support their argument. (as cited in Jackson, 2008, p. 165)
College students, when confronted with controversial issues, are all too often prone to
shallow, narrow-minded assumptions. The challenge for educators, then, is to encourage
disciplined thought and to teach students that “uncertainty is an ally of good thinking and
knowledge is an evidence-based construction” (Jackson, 2008, p. 229).
A strong critical thinking pedagogy that encourages students’ critical knowledge,
skills, and dispositions may improve students’ academic success while encouraging those
abilities needed for transfer and for competency in the workplace. Elder (2000)
contended, however, that traditional education is not nurturing the intellectual capabilities
needed for personal and academic success. Often, students are merely asked to write
down facts rather than to question or reflect on their reading, and, as a result, they are
incapable of drawing inferences and of engaging in complex conversations about the
literature (both fiction and nonfiction prose) they read. Elder suggested further that “as
the economic structure of the world becomes more complex” (p. 1) and “as we become
increasingly more interdependent both at home and abroad, ‘training’ students for job
performance in narrowly defined skill areas no longer serves students well” (p. 1).
Willsen (1997) discussed the increasing complexities of the world and suggested
these new global realities “are rapidly working their way into the deepest structures of
our lives: economic, social, and environmental realities—realities with profound
implications for teaching and learning” (p. 1). The question is whether educational
institutions are preparing students to adapt to and accommodate for these complexities.
6
Cultivating students’ abilities to develop higher order thinking skills and problem-solving
strategies could better prepare students for survival in this complex world. As Willsen
(1997) reasoned
We must sooner or later abandon the traditional attempt to teach our fellow citizens what to think. Such efforts cannot prepare us for the real world we must, in fact, face. We must concentrate instead on teaching ourselves how to think, thus freeing us to think for ourselves, critically, fairmindedly, and deeply. We have no choice, not in the long haul, not in the face of the irrepressible logic of accelerating change and increasing complexity. (p. 16)
It is the responsibility of educational institutions to promote and develop students’
critical thinking abilities. Sternberg (2003), in his discussion of the future of education in
the United States, argued that educational institutions far too often emphasize rote
memorization; while “rote memorization requires recital and repetition” (p. 1), critical
thinking “requires skillful analysis, evaluation, and interpretation” (p. 1). Although all
individuals need a “knowledge base” (Sternberg, 2003, p. 1). or store of information, this
knowledge base must prove useful for living, and instead of emphasizing the
memorization of trivial facts, Sternberg encouraged educators to teach usable information
that students can transfer into their lives. Rather than being taught how to think critically,
students are far too often being taught to do little more than recall and recognize; they
“are becoming highly susceptible to the commission of cognitive fallacies” (Sternberg,
2003, p. 1) and, as a result, they may tend to “act on their prejudices and their fears”
(Sternberg, 2003, p. 1) rather than on reasoned judgment.
7
Statement of Problem
It is not known whether direct instruction in rhetorical analysis will improve
learners’ critical thinking knowledge, skills, and dispositions. This study explored
instructional pedagogy to promote students’ critical thinking abilities. The researcher
investigated whether direct instruction in rhetorical analysis in a college critical thinking
course could make a difference in students’ critical thinking skills. The interventions
included argument mapping; Thinker’s Guides, based on Paul’s (1995f) model of critical
thinking; and Socratic questioning. These interventions were supplemented with course
work that involved reader/writer workshops and peer review.
Purpose of the Study
The purpose of this study was to determine whether there would be improvement
in students’ critical thinking abilities, including knowledge, skills, and dispositions, after
the intervention of direct instruction in rhetorical analysis. The researcher investigated
student perceptions of the effectiveness of the following instructional strategies:
argument mapping; Thinker’s Guides, based on Paul’s (1995f) model of critical thinking;
and Socratic questioning. The goal of the study was, on the one hand, to promote critical
thinking knowledge, skills, and dispositions, and, on the other hand, to determine
improvement in student perceptions of critical thinking knowledge, skills, and
dispositions so that they may transfer those skills across the academic disciplines and into
their personal and professional lives.
8
Research Questions
The following research questions guided this study:
1. What were the students’ perceptions of critical thinking before and after direct instruction in argument mapping?
2. What were the students’ perceptions of critical thinking before and after direct instruction in Paul’s Thinker’s Guides?
3. What were the students’ perceptions of critical thinking before and after direct instruction in Socratic questioning?
4. What did the students report about the changes in their knowledge, skills, and dispositions relating to critical thinking after the direct instruction in rhetorical analysis?
Significance of the Study
A primary goal for educators is to instill in their students a questioning, reflective,
critical mind. The American educational system has been a target for reform to teach
students to think critically, and the educational arena is beginning to take the issue of
critical thinking seriously. The California State University General Education
Requirements include a graduation requirement in critical thinking. In light of the
increasing demands and accelerating changes occurring in the 21st century, educators are
experiencing profound challenges, and the question continues as to which pedagogical
practices would be most effective for the instruction and assessment of critical thinking.
Paul (1995a) believed that “only through an explicit shift to a critical conception
of education, with an explicit critique and rejection of the assumptions of didactic
education, can we achieve significant reform” (p. 278). As national assessment of student
achievement far too often focuses on lower order thinking and learning, Paul (1995a)
9
argued that assessment must focus on higher order thinking, reasoning, and authentic
performance. Paul (1995c) further contended that “critical thinking is the heart of well-
conceived educational reform and restructuring because it is at the heart of the changes of
the 21st Century” (pp. 97–98). At the request of the U.S. Department of Education, Paul
(1995f) developed A Model for the National Assessment of Higher Order Thinking. This
paper outlined the problem of lower order learning; explained 21 criteria for higher order
thinking assessment; defended the view that a clear, substantial concept of critical
thinking meets those criteria; articulated four domains of critical thinking; offered
recommendations as to how to assess the domains of critical thinking; and discussed the
value of the proposed recommendations for educational reform (Paul, 1995f).
The development and implementation of pedagogy that promotes student
engagement in the learning process could encourage student critical thinking abilities and
the transfer of those abilities necessary for academic achievement, personal success, and
success in the work force. Although critical thinking abilities can be encouraged in all
academic disciplines, many community colleges offer specific courses in critical
thinking. Using direct instruction in rhetorical analysis, this study adds to the knowledge
of instructional practices in critical thinking courses that encourage student engagement
in critical thinking and that may alter student perception of critical thinking and promote
awareness for making critical thinking a lifelong endeavor.
Definition of Terms
Argument. A connected series of statements intended to establish a definite
proposition. Arguments “constitute a body of evidence in relation to some proposition (an
10
idea that is true or false). The proposition is expressed in some claim, and the evidence is
expressed in other claims” (van Gelder, 2005, p. 4). Paul (1995d) asserted that
In emphasizing critical thinking, we continually try to get our students to move from the first sense of the word to the second; that is, we try to get them to see the importance of giving reasons to support their views without getting their egos involved in what they are saying. (p. 522)
Argument mapping. A visual representation of the structure of an argument in
informal logic. It is a map that “makes the logical structure of the argument completely
explicit” (van Gelder, 2005, p. 4). These maps are often used in the teaching of reasoning
and critical thinking, and can support the analysis of pros and cons when deliberating
over problems.
Critical thinking. In 1990, a panel of experts, who included 46 men and women
from throughout the United States and Canada and who represented a variety of scholarly
disciplines, participated in a research project conducted on behalf of the American
Philosophical Association. This 2-year Delphi project was conducted by P. A. Facione
(2006), and the final Delphi Report generated a consensus definition of critical thinking
(P. A. Facione, 2006). The final Delphi Report defined critical thinking as “purposeful,
self-regulatory judgment that results in interpretation, analysis, evaluation, and inference,
as well as explanation of the evidential, conceptual, methodological, criteriological, or
contextual considerations upon which that judgment is based” (P. A. Facione, 1990, p. 2).
Critical Thinking Course. The critical thinking course is designed explicitly to
develop the student’s critical thinking, reading, and writing skills beyond the level
achieved in a basic English course. It focuses primarily on the analysis and evaluation of
11
expository and argumentative discourse and on writing analytical and argumentative
essays.
Direct instruction. Created by Engelmann and Becker, direct instruction is a
model for teaching that emphasizes well-developed, highly scripted, and carefully
planned lessons designed around small learning increments and clearly defined and
prescribed teaching tasks. Through the use of carefully prescribed instructional practices,
direct instruction can improve student academic performance (Engelmann, n.d.).
Paul’s Thinker’s Guides. Paul’s model of critical thinking is comprised of three
basic concepts: elements of reasoning, intellectual standards, and intellectual traits.
Thinking can be divided into seven components, including purpose; point of view;
assumptions, implications, and consequences; data and information; inferences and
interpretations; concepts; and the question at issue. Moreover, thinking can be measured
against standards, such as clarity, accuracy, precision, relevance, depth, breadth, logic,
significance, and fairness (Paul, 1995g). Paul (1995g) maintained that all thinkers should
cultivate positive intellectual traits such as intellectual humility, intellectual perseverance,
intellectual integrity, intellectual courage, confidence in reason, and intellectual empathy.
Rhetorical analysis. Aristotle defined rhetoric as “the faculty of observing in any
given case the available means of persuasion” (as cited in Shea, Scanlon, & Aufses, 2008,
p. 1). Rhetoric is skillfully constructed written, spoken, or visual argument; it is the
means by which individuals appeal to an audience; and it includes a coherent, balanced
exchange of opposing views. Effective rhetoric has a context and a purpose, and it offers
“tools to resolve conflicts without confrontation, to persuade readers or listeners to
support [a] position, or to move others to take action” (Shea et al., 2008, p. 1). Rhetorical
12
analysis is the act of carefully scrutinizing the tools and techniques the author, speaker, or
visual artist has used to appeal to the audience. It is, according to Shea et al. (2008),
nurturing the student’s ability to discover the “nutritional value in the books, stories,
essays, and poems [they] study in school” (p. 35).
Scaffolding. Structure and guidance provided by educators to support students
with challenging tasks. Many theorists believe that scaffolding may improve problem-
solving performance. Ways educators can assist students with difficult problems include
simplifying or dividing a task into smaller units, modeling problem solutions, asking
probing questions, pointing out errors, keeping students focused and motivated, and
giving frequent feedback (Ormrod, 2004).
Socratic questioning. “A mode of questioning that deeply probes the meaning,
justification, or logical strength of a claim, position, or line of reasoning” (Paul, 1995d, p.
18). Socratic questioning encourages rigorous, thoughtful dialogue, and involves the
practice of thoughtful, probing questioning, which enables students to examine the logic
and validity of their ideas.
Assumptions and Limitations
Students enrolled in community college courses in California constituted the
research sample. Although the findings should prove useful, it is doubtful they can be
generalized to a target population in other locations unless the sample sizes share similar
characteristics. Students enrolled in California community colleges are similar to students
enrolled in courses elsewhere; however, as Gall, Gall, and Borg (2003) maintained,
“Generalizations to other cases can be done, but it must be done on a case-by-case basis”
13
(p. 10). Therefore, generalization may not be possible, nor is it necessary as it is truly up
to the reader to find similarities in an action research study to a different site or location.
Possible limitations exist due to the modest sample size of 15 students and the
short term of instruction necessitated by an 18-week semester course. Research suggests
that critical thinking is a lifelong process and that successful interventions may need to
occur over an academic year or over several years. Ideally, a longitudinal study would be
most beneficial in determining growth in critical thinking knowledge, skills, and
dispositions as a result of the interventions. The researcher was the teacher of an intact
class, and no attempt at randomization occurred; therefore, the study was limited to a
description of an existent situation rather than creating an experimental situation. In
teacher action research studies, the data are collected and analyzed to assess and improve
educational practice. Since the researcher was the instructor in the critical thinking
course, there was potential for researcher bias as the researcher may have authority over
the participants. The researcher understood this and attempted to approach the study with
caution and without preconceived assumptions of possible outcomes or results. Care was
taken to maintain clarity and objectivity through sharing with research participants and
through personal reflection, assessment, and modification of instructional practices. For
further consideration is the Hawthorne effect which refers research situations where
“experimental conditions are such that the mere fact that individuals are aware of
participating in an experiment, are aware of the hypothesis, or are receiving special
attention improves their performance” (Gall et al., 2003, p. 376). The researcher
accommodated for this concern by providing equal guidance and attention to all students
in the course, and research participants were not given special attention. Finally, this
14
study was limited to examination of the interventions employed in a critical thinking
course and did not attempt to measure the influence of out-of-classroom experiences in
growth in critical thinking.
Theoretical Framework
P. A. Facione’s (2006) definition of and approach to critical thinking offers a core
of critical thinking skills that includes analysis, interpretation, inference, explanation,
evaluation, and self-regulation. Interpretation occurs when the individual comprehends
the meaning of a variety of experiences, and includes categorizing, decoding, clarifying,
recognizing a problem without bias, and distinguishing the main idea from subordinate
ideas. Analysis is when the learner identifies the intended and inferential relationships
among statements, and includes the examination of ideas, the analysis of arguments and
the ability to examine alternative approaches to a problem and identify any unstated
assumptions. Evaluation includes the learner’s ability to consider the credibility of
assumptions and to compare strengths and weaknesses of alternative views or beliefs.
Inference refers to a person’s ability to construct meaning and to identify the implications
of a particular position (P. A. Facione, 2006).
P. A. Facione (2006) suggested that “we should be cautious about proposals
suggesting oversimplified ways of understanding how humans think” (p. 12). P. A.
Facione’s method of critical thinking involves two systems that are considered essential
decision-making tools. System 1, the intuitive, is reactive, deliberate, and holistic,
whereas System 2 is reflective, analytical, and evaluative. System 1 thinking relies on
what P. A. Facione referred to as “cognitive shortcuts [which include] key situational
15
characteristics, readily associated ideas, and vivid memories to arrive quickly and
confidently at a judgment” (p. 12). This system is most helpful when decisions must be
made quickly and when immediate action is required. System 2 thinking is more
reflective and is useful when individuals are in “unfamiliar situations and have more time
to figure things out” (P. A. Facione, 2006, p. 13). It allows deliberation, planning ahead,
and considering options; “it is reasoning based on what we have learned through careful
analysis, evaluation, explanation, and self-correction; and it values intellectual honesty
and fair-mindedness” (P. A. Facione, 2006, p. 13). P. A. Facione asserted that individuals
who have a disposition toward critical thinking are inquisitive, systematic, judicious,
truth seeking, analytical, open-minded, and confident in their reasoning.
The ideal critical thinkers, according to P. A. Facione (2006), are characterized by
their cognitive skills and by their approach to life. They illustrate
Clarity in stating a question, orderliness in working with complexity, diligence in seeking relevant information, reasonableness in selecting and applying criteria, care in focusing attention on the concern at hand, persistence through difficulties encountered, and precision to the degree permitted by the subject and the circumstances. (P. A. Facione, 2006, p. 9)
Finally, P. A. Facione (2006) asserted that good critical thinkers can misuse those
superior skills in immoral and unethical ways through manipulation and exploitation.
Ultimately, being a good critical thinker is dependent on the character and integrity of the
individual. For this reason, “knowledge and skills are not sufficient. We must look to a
broader set of outcomes including habits of mind and dispositions, such as civic
engagement, concern for the common good, and social responsibility” (P. A. Facione,
2006, p. 11).
16
P. A. Facione’s (2006) definition of critical thinking, generated by the scholars
involved in the Delphi project, was the working definition for the study. The Delphi
consensus definition offers a means by which to address the U.S. Department of
Education’s Education Goals: 2000 mandate (P. A. Facione, 2000). The California
Critical Thinking Skills Test–Form 2000 (CCTST–2000), which was used in this study,
derived its construct validity from this critical thinking conceptualization. However,
although the CCTST–2000 targets the core critical thinking skills of analysis,
interpretation, inference, evaluation, and explanation, it does not measure critical thinking
dispositions. P. A. Facione’s reasoning motivation tool, the California Critical Thinking
Dispositions Inventory (CCTDI), was designed to measure the dispositional dimension of
critical thinking; however, the CCTST–2000 proved a valid and reliable instrument for
the research purposes as “high scores on the CCTST are often correlated with a strong
disposition toward critical thinking (high scores on the CCTDI). Individuals with high
scores on the CCTST–2000 and low scores on the CCTDI are relatively rare” (P. A.
Facione, 2000, p. 11).
Paul and Elder (2006a), leaders in the critical thinking movement, developed a
model for critical thinking that provides a practical approach for developing students’
critical thinking abilities. Paul and Elder applied the core elements of critical thinking and
the characteristics of good critical thinkers from the Delphi Report, generated by scholars
in critical thinking, to address the needs of educators and students (P. A. Facione, 2006).
Paul’s (2004) model insists there are universal standards for critical thinking, which
include clarity, accuracy, precision, relevance, consistency, depth, and breadth. Paul
maintained that critical thinkers analyze, assess, and upgrade their thinking, while
17
advocating that instructors design curriculum so that students understand the sense and
logic of what they learn, and that this learning will increase comprehension and insight.
Teaching critical thinking appeals to reason and evidence, encourages students to
discover and process information, and provides occasions for students to think through to
conclusions, defend positions, consider alternative viewpoints, analyze concepts, clarify
issues, solve problems, and transfer ideas to new concepts.
Van Gelder (2005) drew on cognitive science for teaching critical thinking, and
reasoned that because critical thinking is a difficult, lifelong process, it requires repeated
engagement, practice, and some theoretical knowledge for transfer. As arguments are
presented in “streams of words, whether written or spoken” (van Gelder, 2005, p. 4), van
Gelder advocated handling arguments through argument mapping. The logic of
arguments is expressed in sequences, and those sequences can be diagrammed or mapped
out for better, more explicit understanding. Argument maps follow a particular set of
conventions in which the main point is put at the top of the argument tree. Arrows then
indicate that a claim is evidence. The use of the color green and the word reason indicate
they are supporting evidence (van Gelder, 2005). Van Gelder claimed argument maps
clearly and visibly make the logic of arguments more straightforward; as a result,
students’ critical thinking skills and abilities improve when they practice this skill.
In his article, Twardy (2003) claimed that “argument mapping greatly enhances
student critical thinking, more than tripling absolute gains made by other methods” (p. 1).
The results of his study, conducted at the School of Computer Science and Software
Engineering at Monash University in Australia, indicated that students who used van
Gelder’s argument mapping to analyze arguments showed significant gains on the
18
posttest scores using the CCTST. The CCTST is the most widely used measurement of
critical thinking skills as it targets “those core critical thinking skills regarded to be
essential elements in college education” (P. A. Facione, 1992, p. 2). Argument maps
could prove to be useful tools for the analysis of formal arguments as this practice has the
potential to improve students’ ability to closely diagnose and map out rationale while
constructing their own arguments.
Nature of the Study
The mixed-methodology approach was used in this teacher action research study.
The study pertained to one semester of a college critical thinking course that examined
the effect of direct instruction in rhetorical analysis in students’ critical thinking
knowledge, skills, and dispositions. The study occurred in a naturalistic setting, and the
sample was comprised of students enrolled in the critical thinking course. A variety of
works were used to encourage analysis of authors’ rhetorical strategies and stylistic
choices. To reflect the increasing importance of graphics and visual images in texts
published in print and electronic media, students were asked to analyze how such images
both relate to written texts and served as alternative forms of texts themselves. The
interventions were supplemented with course work that involved reader/writer workshops
and peer review.
A Pre- Critical Thinking Survey was given at the beginning of the course and a
Post- Critical Thinking Survey was administered at the end of the course. The qualitative
survey was implemented for analysis of patterns of students’ responses; the observations
made by the survey results offered insights into student perceptions of the most effective
19
instructional interventions. The CCTST–2000 was administered as a pretest at the
beginning of the course and measured the students’ critical thinking skills. The same test
was given as a posttest and measured growth in critical thinking abilities at the end of the
course. Statistical analysis was done on the CCTST–2000 pretest and posttest scored by
Insight Assessment.
Null Hypothesis: There will be no difference in scores on the pretest and posttest
on the California Critical Thinking Skills Test–Form 2000 indicating changes in student
critical thinking skills before and after direct instruction in rhetorical analysis.
Organization of the Remainder of the Study
The remainder of the study is organized into four chapters, in the following
manner: Chapter 2 presents a discussion, evaluation and critique of prior research dealing
with the trends in critical thinking, direct instruction in rhetorical analysis, argument
mapping, Paul’s Thinker’s Guides, and Socratic questioning classroom methodologies;
chapter 3 explains how the study was conducted; chapter 4 includes data analysis and
results and presents a report of the data supported by tables and figures and an
explanation of the statistical analysis; and chapter 5 offers conclusions and
recommendations, evaluates the work, and provides insight into and interpretation of the
study’s results.
20
CHAPTER 2. LITERATURE REVIEW
Introduction
Ideally, individuals possess the ability to fairly view issues and their world with
an open mind and from a variety of perspectives. It is unknown whether direct instruction
in critical thinking can meet the challenges educators encounter in their attempts to
encourage and develop those essential abilities for reflective, informed thinking. This
need for promoting students’ critical thinking knowledge, skills, and dispositions, and the
responsibility of educational institutions for developing these abilities is addressed by a
variety of scholars and philosophers (P. A. Facione, 2006; Halpern, 2006; Paul, 1995c;
Perkins, 1993; van Gelder, 2005). This chapter explores critical thinking and is divided
into the following sections: an overview of the critical thinking movement, definitions for
critical thinking, the importance of critical thinking, teaching critical thinking, specific
strategies for teaching critical thinking, and related studies of critical thinking. This is
followed by a brief summary that emphasizes the need for teaching or embedding critical
thinking in instructional methodologies. The purpose of this study was to determine
whether there would be improvement in student critical thinking knowledge, skills, and
dispositions on pretest and posttest scores before and after direct instruction in argument
mapping, Paul’s Thinker’s Guides, and Socratic questioning.
21
The Critical Thinking Movement
A fundamental understanding of the critical thinking movement begins with
contributions from Socrates, who developed a method of asking meaningful questions,
where “confused meanings, inadequate evidence, or self-contradictory beliefs often
lurked beneath smooth but largely empty rhetoric” (Paul, Elder, & Bartell, 1997a, p. 1).
Socrates questioned the assumptions and beliefs of those in authority and established
Socratic questioning, which is a methodology that advocates the importance of asking
probing questions and seeking evidence to examine rhetoric. Socrates’s search for the
essence of reason and truth encouraged a thorough examination of statements, and an
understanding of their evidence, assumptions, theories, reasoning, and implications.
Socrates’s practice and method of reflective, well-reasoned, systematic thinking
influenced the writings of Plato, Aristotle, Thomas Aquinas, Erasmus, Francis Bacon,
Descartes, Sir Thomas Moore, Hobbes and Lock, Robert Boyle, and Sir Isaac Newton
(Paul et al., 1997a).
Twentieth-century theorists whose writings have contributed significantly to
critical thinking theory and education are William Graham Sumner, John Dewey, Edward
Glaser, Jean Piaget, and Lev Vygotsky. Dewey’s pragmatic approach to critical thinking
advocates student-centered rather than subject-centered education. He believed that
genuine education comes about through experience; however, “the quality of any
experience has two aspects. There is an immediate aspect of agreeableness or
disagreeableness, and there is its influence upon later experiences” (Brookfield, Tennant,
& Pogson, 2005, p. 326). Education and life are interrelated, and educators must design
and carefully monitor positive educational experiences. For Dewey, “knowledge is the
22
product of an interaction between the experiencing subject and the external objective
world” (Brookfield et al., 2005, p. 333). Students learn best by doing, and continuity of
experience is essential to growth.
Glaser contributed significantly to critical thinking research. He encouraged
teaching critical thinking and introduced the Watson–Glaser Critical Thinking Appraisal.
He defined the ability to think critically as
[a] an attitude of being disposed to consider in a thoughtful way the problems and subjects that come within the range of one’s experiences, [b] knowledge of the methods of logical inquiry and reasoning, and [c] some skill in applying those methods. (as cited in Scriven & Paul, 1987, p. 2)
Piaget was a biologist who contributed extensively to educational theory and to
the study of child maturation and development (Atherton, 2005). Piaget’s theory is
considered “the single most global theory of intellectual development; it incorporates
such diverse topics as language; logical reasoning; moral judgments; and conceptions of
time” (Ormrod, 2004, p. 144). Piaget proposed the idea of scheme and believed that
knowledge can be understood in terms of structures or operations that, through
assimilation and accommodation, alter during a child’s cognitive development. The
Piagetian view of cognitive development espouses that there are four cognitive structures
that correspond to the stages of child development, and that “children progress through
different types of thinking as they develop toward mature adult thought” (Brookfield et
al., 2005, p. 27). Piaget’s view emphasized individual thought and autonomy. He
believed “the individual will interpret and act accordingly to conceptual categories or
schemas that are developed in interaction with the environment” (Oxford, 1997, p. 39),
that people are intrinsically motivated and actively involved in the learning process, and
23
that cognitive development results from the interactions that individuals have with their
physical environments.
In contrast to Piaget’s view of learning as an individual endeavor, Vygotsky
stressed the importance of past experiences, prior knowledge, society, and culture on
promoting cognitive growth (Dahms et al., 2008). Whereas Piaget was concerned with
the characteristics exhibited by children of a particular age, Vygotsky focused on the
process of child development. Vygotsky believed knowledge is developed through social
interaction, that learning occurs through language and shared experiences, and adults
“foster children’s learning and development in an intentional and somewhat systematic
manner” (Ormrod, 2004, p. 150). Individuals react to, alter, and adapt to their
environment. Vygotsky’s sociocultural perspective includes the concept of the Zone of
Proximal Development (ZPD), which assumes that learning is social and human potential
is limitless, “but the practical limits of human potential depend upon quality social
interactions” (Dahms et al., 2008, p. 3). In Vygotsky’s view, cognitive processes begin to
develop as a result of social interaction within specific cultures, cognitive development
occurs as children discuss and internalize these processes, and “through both informal
conversations and formal schooling, adults convey to children the ways in which their
culture interprets and responds to the world” (Ormrod, 2004, p. 151). Although children
are unable to perform certain tasks independently, their ZPD changes through the
assistance and support of others who are more knowledgeable, and children begin to
reach optimum performance. As students’ abilities develop and certain tasks are
mastered, they begin to acquire the readiness for other more complex skills and to solve
problems through a process referred to by theorists as scaffolding. The assistance of their
24
peers, the teachers who model the processes in a collaborative classroom environment,
and “through both informal conversations and formal schooling, adults convey to
children the ways in which their culture interprets and responds to the world” (Ormrod,
2004, p. 151).
Defining Critical Thinking
Cognitive psychologists and philosophers have offered a variety of definitions for
critical thinking (Chance, 1986; Ennis, 1992; P. A. Facione, 2000; McPeck, 1981; Paul,
1995f; Paul et al., 1997a; Scriven & Paul, 1987; van Gelder, 2005). Chance focused on
argument and defined critical thinking as “the ability to analyze facts, generate and
organize ideas, defend opinions, make comparisons, draw inferences, evaluate arguments,
and solve problems” (p. 6). Chance reasoned that critical thinking involves the “ability to
analyze facts, generate and organize ideas, defend opinions, make comparisons, draw
inferences, evaluate arguments, and solve problems” (p. 6).
Ennis (2002) defined critical thinking as “reasonable reflective thinking focused
on deciding what to believe or do” (p. 1). Ennis viewed an ideal critical thinker as one
who is open-minded; mindful of alternatives; well-informed; judges the credibility of
sources; identifies conclusions, reasons, and assumptions; judges the quality of an
argument, including the acceptability of its reasons, assumptions, and evidence; can
develop and defend a reasonable position; asks appropriate clarifying questions;
formulates plausible hypotheses; plans experiments well; defines terms in a way
appropriate for the context; draws conclusions when warranted, but with caution; and
integrates all items in this list when deciding what to believe or do.
25
McPeck (as cited in Hatcher, 2000) defined critical thinking as “the skill and
propensity to engage in an activity with reflective skepticism” (p. 2). Hatcher, on the
other hand, believed reflective skepticism is a “vague notion [that has] negative
connotation[s]” (p. 2) and that critical thinking is constructive and its purpose “is not
always negative, but is often to arrive at a positive judgment on an issue” (p. 2). Hatcher
viewed critical thinking as open-ended and that focuses on areas where “evidence and
arguments are appropriate” (p. 3). He believed critical thinking is “thinking that attempts
to arrive at a judgment only after honestly evaluating alternatives with respect to
available evidence and arguments” (p. 3).
As with McPeck, P. A. Facione (2000) was concerned with the dispositions or
habits of mind the individual has toward critical thinking. He maintained the cognitive
skills of “analysis, interpretation, inference, explanation, evaluation, and of monitoring
and correcting one’s own reasoning are at the heart of critical thinking” (p. 2). Individuals
must possess skill in critical thinking, but they must also have the internal motivation to
think. Critical thinking skills can be developed through practice and guidance as good
critical thinking is a complex, purposeful process. Although humans may possess critical
thinking abilities, they may not always choose to act on reasoned judgment for a variety
of personal or professional reasons. N. C. Facione and Facione (2001) suggested
argument mapping and analysis can provide useful ways to understand and interpret the
rationality of a set of arguments involving significant human problems, while maintaining
that “once the argument has been mapped and analyzed, the reasoning it manifests can
then be evaluated for its logical merit” (p. 268).
26
P. A. Facione served as the lead investigator to coordinate an international effort
to determine the extent to which experts agreed on the definition of critical thinking for
the purposes of college-level teaching and assessment. The result was the 1990 APA
Delphi Report. An international group of experts, who included 46 men and women
throughout the United States and represented a variety of scholarly disciplines, was asked
to generate a consensus definition of critical thinking, including its core cognitive skills.
The report included a description of the Delphi research methodology, addressed the skill
dimension of critical thinking, focused on the dispositional dimension of critical thinking,
and concluded with 15 recommendations pertaining to critical thinking instruction and
assessment. The research project lasted 2 years, was conducted on behalf of the American
Philosophical Association, and continues to influence critical thinking theory, teaching,
and assessment (P. A. Facione, 2006). The panel’s consensus statement regarding critical
thinking and the ideal critical thinker is as follows: “We understand critical thinking to be
purposeful, self-regulatory judgment that results in interpretation, analysis, evaluation,
and inference, as well as explanation of the evidential, conceptual, methodological,
criteriological, or contextual considerations upon which what judgment is based” (P. A.
Facione, 1990, p. 2). The Delphi panel further concluded that the ideal critical thinker is
Habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fair-minded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in the selection of criteria, focused in inquiry, and persistent in seeking results which are as precise as the subject and the circumstances of inquiry. (P. A. Facione, 1990, p. 2)
Critical thinking cognitive skills include interpretation, analysis, evaluation, inference,
explanation, and self-regulation.
27
Scriven and Paul (1987), like P. A. Facione, asserted that individuals may be
subject to undisciplined thought and that critical thinking varies upon the motivation
underlying it. Paul (1995c) was concerned with the metacognitive aspects of critical
thinking and characterized critical thinking as “thinking about your thinking while you’re
thinking in order to make your thinking better” (p. 91). In line with the metacognitive
aspects of critical thinking, Paul et al. (1997a) maintained that critical thinking requires
“the systematic monitoring of thought; that thinking, to be critical, must not be accepted
at face value but must be analyzed and assessed for its clarity, accuracy, relevance, depth,
breadth, and logicalness” (p. 5). Finally, Paul (1995f) defined critical thinking as “the
intellectually disciplined process of actively and skillfully conceptualizing, applying,
analyzing, synthesizing, and/or evaluating information gathered from, or generated by,
observation, experience, reflection, reasoning, or communication, as a guide to belief and
action” (p. 8).
Paul and Elder (2004) contributed significantly to educational practices for
promoting critical reflection, metacognition, and literary analysis. Their contention was
that few readers have the abilities to skillfully read and translate the author’s intended
meaning in the text, that how students read should be determined by what they read, and
that “skilled readers do not read blindly; they read purposely. They have an agenda, goal,
or objective” (p. 36). Paul and Elder emphasized the importance of students
understanding the purpose of their reading and the author’s purpose in writing, perceiving
ideas in a text as being interconnected, connecting with a text while reading, and
formulating questions and seeking answers to those questions while reading. Further,
28
they encouraged explicating the thesis of a paragraph, analyzing the logic of what is
being read, and evaluating and assessing the logic of a reading.
Critical thinking is the process of “actively and skillfully conceptualizing,
applying, analyzing, synthesizing, and/or evaluating information” (Scriven & Paul, 1987,
p. 1). People who think critically can analyze their own thinking and realize they can
improve their own reasoning; a highly cultivated critical thinker raises vital questions and
problems, gathers and assesses relevant information, thinks open-mindedly, and
communicates effectively (Elder, 2007; Scriven & Paul, 1987). Carroll (2007) suggested
that “one of the key elements of critical thinking is the recognition that one’s worldview
can be a major hindrance to being fair-minded” (p. 4); hence, students must be willing to
negotiate previously held positions and beliefs while considering opposing viewpoints.
Teaching critical thinking through direct instruction in rhetorical analysis could improve
students’ critical thinking ability, for in order to teach students to think critically, “we
must teach them to try to understand how one’s worldview is likely to be embedded with
prejudices, biases, and false notions” (Carroll, 2007, p. 4).
Van Gelder (2005) contended that critical thinking is a complex skill that involves
deliberate practice for transfer. As P. A. Facione suggested that argument mapping is a
useful approach to understanding and interpreting the rationality of a set of arguments,
van Gelder proposed that the effective core to critical thinking is in learning how to
handle arguments. Argument maps make reasoning more easily understandable, students
can see the reasoning and can identify important issues, students can follow extended
critical thinking procedures because the arguments are presented in diagrammatic form,
and instructors can see the student’s line of reasoning when it is laid out in diagrams. Van
29
Gelder’s contention was that student critical thinking abilities improve when instruction
is based on argument mapping, and that “one semester of instruction based on argument
mapping can yield reasoning skill gains of the same magnitude as would normally be
expected to occur over an entire undergraduate education” (p. 5).
Socratic teaching is a powerful instructional strategy that encourages students to
discover their own reasoning through a series of questioning. The Socratic questioner
“acts as the logical equivalent of the inner critical voice which the mind develops when it
develops critical thinking abilities” (Paul & Elder, 1997, p. 1). Through a series of
questions, students are forced to think in a reasonable fashion. The Socratic questioner, or
the instructor, attempts to stimulate and encourage discussion while exploring students’
reasoning with probing questions. At the same time, the instructor/questioner should
ensure the discussion remains focused and intellectually responsible and summarize what
has or has not been accomplished (Paul & Elder, 1997). Socratic questioning has the
potential to develop students’ critical thinking abilities because it helps students to
monitor their thinking, to become more rational, and to pay more attention to that “inner
voice of reason” (Paul & Elder, 1997, p. 1).
Socratic questioning can be used for a variety of purposes, which include
exploring complex ideas, getting to the truth of some concept, uncovering assumptions,
analyzing concepts, and distinguishing “what we know from what we don’t know”
(Elder, 2006, p. 5). Elder encouraged educators to use Socratic questioning so they can
use it in their everyday lives when confronted with complex issues, “understand the
thinking of others [and] trace the implications of what they and others think and do” (p.
5).
30
Importance of Teaching Critical Thinking
Studies indicate that the United States has not been performing as well as other
industrialized nations, in terms of critical thinking skills (National Center for Education
Statistics, 2007). Students must learn to think skillfully and independently so they may
cultivate their own problem-solving abilities and come to reliable conclusions. The goals
of this study were to provide students with the analytical, problem-solving skills needed
in a variety of academic settings and in their everyday lives. Specifically, the researcher
investigated whether the interventions implemented in the study are effective in
promoting student critical thinking knowledge, skills, and dispositions, and whether there
is growth in student perceptions of critical thinking knowledge, skills, and dispositions.
Sternberg (2003) reasoned that “a future of successfully intelligent thinkers is
important to personal satisfaction and national achievement” (p. 434). Teaching students
to think reflectively and critically should be a primary goal of educational institutions, for
although students may be knowledgeable, they may not have been taught how to think
analytically; hence, they could become highly vulnerable to the fallacious reasoning
exhibited by political leaders and within the media in its various forms (Sternberg, 2003).
The ultimate goal for educators is to promote lifelong learning by enhancing students’
problem-solving abilities so they may “apply these steps not just in school problems, but
in problems [they encounter] in everyday life” (Sternberg, 2003, p. 3).
Brookfield (2003) referred to the transfer of those skills learned in the classroom
and lifelong learning as “the organizing concept for adult education” (p. 2). Brookfield
claimed “there are forms of learning we engage in that are visible in a much more
heightened form in adulthood as compared to childhood and adolescence” (p. 2). Adult
31
learning is distinct to childhood and adolescent learning in that it includes the capacity to
think dialectically, to employ practical logic, to know how one knows what one knows,
and to think reflectively. Brookfield defined critical reflection as “the process by which
adults become critically reflective regarding the assumptions, beliefs and values which
they have assimilated during childhood and adolescence” (p. 10). This critical reflection
occurs over a period of time as a result of interpersonal, work-related, and political
experiences.
Society is experiencing tremendous economic, political, technological, and social
changes. These complexities demand a healthy, democratic populace who exercise
critical thought. When discussing the importance of critical thinking, Brookfield et al.
(2005) deduced that
Without the capacity to think and act critically, we would never move beyond those assumptions we assimilated uncritically in childhood. We would believe totally in the myths, folk wisdom, and values we encountered in authority figures in our early lives. We would make no attempt to change social structures or to press for the collective social action. (p. 46)
Under these circumstances, individuals might believe they are victims of circumstances
that seem beyond their control, unaware that they have the capacity to prompt action
through social change.
Brookfield (1995) encouraged educators to take a critical reflective stance toward
teaching and to help students confront their world or environment with compassion,
understanding, and fairness; nevertheless, the sincerity of intentions “does not guarantee
the purity of practice” (p. 1). When teachers reflect critically on their practice and model
the critical thinking process, students begin to understand what is involved in a critical
analysis of assumptions. For educators, the reflective process involves questioning their
32
practice and discovering assumptions about their practice and their students. This
reflective habit will lead to informed action, add meaning to instruction, and encourage
the creation of democratic classrooms.
Paul (1995a) believed critical thinking is the “essential foundation for education
because it is the essential foundation for adaptation to the everyday personal, social, and
professional demands of the 21st Century and thereafter” (p. xi). In view of the rapidly
changing world and the new global realities, there is a vital need for individuals to
develop those skills and abilities that enable them to respond and adapt to these changes.
Research findings from “Policy Issues in Teaching Education” indicated that
Although students can perform basic skills pretty well, they are not doing well on thinking and reasoning. American students can compute, but they cannot reason. . . . They can write complete and correct sentences, but they cannot prepare arguments. . . . Moreover, in international comparisons, American students are falling behind . . . particularly in those areas that require higher order thinking. (as cited in Paul, 1995g, p. 19)
In order to adapt to the complexities and demands of the 21st century, Paul
(1995b) recommended that educators “cultivate minds that habitually probe the logic of
the systems of the status quo as well as the logic of the possible variations and alternative
systems” (p. xii), and that rather than memorizing the conclusions of others, “students
should reason to those conclusions on the basis of their own disciplined thought” (p. xii).
However, not only are many of the educational institutions ill-prepared for the rapid
changes occurring in society, but educators and students, seemingly, fail to have a clear
idea of what critical thinking is. Students have an obligation to be responsible for their
thoughts, their conduct, and their lives, and educators can attempt to encourage this
33
accountability and improve students’ approach to problem solving through direct
instruction in argument and critical thinking.
Elder (2000) discussed the emphasis of teaching students the skills needed to
become competent employees at the community college level. As society becomes more
complex, and as a rapid change in technology occurs, “training students for job
performance in narrowly defined skill areas no longer serves students well” (Elder, 2000,
p. 1). Elder’s contention was that students are not prepared for the challenges of the
current job market. Therefore, educators should encourage in their students the
intellectual tools that “will render them mentally flexible and intellectually disciplined”
(Elder, 2000, p. 2). Successful employees must be able to utilize disciplined reasoning
and the metacognitive process so they can direct and redirect their thinking. Rather than
emphasizing the transfer of information, educators should encourage students to rethink
their thinking and to reason, analyze, judge, and interpret that information.
P. A. Facione (2006) discussed the value of critical thinking and the need for an
informed citizenry who can make good judgments while offering practices for nurturing
the habits of mind or dispositions for critical thinking. He suggested the ideal critical
thinker “can be characterized not merely by her or his cognitive skills but also by how
she or he approaches life and living in general” (p. 9). P. A. Facione related a study of
over 1,100 college students that indicated “scores on a college level critical thinking
skills test significantly correlated with college GPA” (pp. 17–18). His contention was that
there is a “significant correlation between critical thinking and reading comprehension”
(p. 18); however, this goal may be somewhat limited. Apart from the college experience,
students must learn to stand on their own, think for themselves, and make their own
34
contributions to society; hence, “learning critical thinking, cultivating the critical spirit, is
not just a means to this end, it is part of the goal itself” (P. A. Facione, 2006, p. 18).
To date, there are a number of researchers who have examined the effect of direct
instruction in critical thinking. This study, however, examined this issue by determining
whether there was improvement in students’ critical thinking knowledge, skills, and
dispositions on pretest and posttest scores after direct instruction in argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning. Of added importance was the
researcher’s investigation of the students’ perceptions of the effectiveness of the
instructional interventions and of their own growth in critical thinking abilities at the end
of the course.
Teaching Critical Thinking
An assumption among many educators is that students who attend college will
develop the necessary critical thinking skills simply by attendance in class and
participation in class discussion. Conversely, with the current emphasis on improving
critical thinking and student achievement, educational institutions, colleges, and
universities have been offering courses designed to improve students’ critical thinking
skills, either as complete, isolated courses or within academic disciplines. Although
improving students’ critical thinking abilities has become a primary goal for education,
there is continual debate over the most appropriate or effective pedagogical strategies
educators can use to improve those abilities.
Gardner (1993) advocated that people have a variety of different intelligences. He
identified seven components of intelligences that he believed are distinct from each other.
35
Although Gardner valued individuals who can who can think critically about literature
and world events, he cautioned that “one must be careful not to assume that it is a
particular, dissociable variety of human cognition” (p. 44), while suggesting that
“particular domains of human competence seem to require their own brand of critical
thinking” (p. 44). The kind of critical thinking required for a musician might be quite
different from the critical thinking required for a historian or biologist because “each
domain exhibits its own particular logic of implications” (Gardner, 1993, p. 14); hence,
training for one domain does not necessarily provide transfer to other domains. Further,
instructors should not assume that critical thinking skills taught in a standalone critical
thinking course will transfer to those skills needed for a history course; rather, Gardner
believed that
Only if the lessons of critical thinking are deliberately revisited in each of the relevant classes or exercises is there any possibility that a more general virtue like reflectiveness or taking the perspective of the other has any chance of emerging. (p. 44)
As with Gardner, Sternberg (2008) believed that conventional notions of
intelligence and intelligence tests do not necessarily reflect talent or wisdom. Similar to
Gardner’s theory of multiple intelligences, Sternberg’s theory of successful human
intelligences, often referred to as the triarchic theory, consists of three components:
practical intelligence, experiential intelligence, and componential intelligence. Practical
intelligence refers to “internal abilities [or] mental mechanisms” (Sternberg, 2008, p. 24)
and can lead to more-or-less intelligent behavior; experiential intelligence examines
individuals’ “experience in handling a task” (Sternberg, 2008, p. 24); and componential
intelligence refers to the external world and includes “environmental adaptation,
36
environmental selection, and environmental shaping” (Sternberg, 2008, p. 24). His theory
takes into account the strengths and differences of individuals while considering the
sociocultural context in which they live. Sternberg believed intelligence can be increased
through study and practice, and encouraged educators to assist students so they can use
and develop all of their skills and perform well in all areas.
Sternberg (2003) defined the ideal critical thinker as a good problem solver;
however, students must be taught to transfer the problem-solving skills they learn in
school to their everyday lives. Successful intelligent thinkers have “the creative skills to
generate new ideas, analytical abilities to know whether they are good ideas, and the
practical abilities to know how to implement the ideas and to persuade others of the value
of their ideas” (Sternberg, 2003, p. 4). Yet, although Sternberg believed these to be
valuable skills, he advocated that schools should include wisdom-related skills in the
school curriculum.
If appropriately taught, intelligence and critical thinking can lead to wisdom.
Sternberg (2003) defined wisdom as “the application of intelligence and experience
toward the attainment of a common good” (p. 7). The dilemma exists when students use
their intelligence in foolish ways; hence, it is the responsibility of educators to encourage
students to be responsible and to balance their intrapersonal, interpersonal, and
extrapersonal interests so their knowledge is used for good purposes. Sternberg’s research
involved teaching wisdom as part of a course in American history. Students in the
experimental group learn to understand things from different points of view, whereas
students in the control group learn the historical material in a standard way. The
definitive goal of the study was to teach students that “resolution of difficult life
37
problems requires people to want to understand each other and to reach a resolution,
whenever possible, with which all of those people can live” (Sternberg, 2003, p. 7).
Halpern (1999) believed critical thinking can be taught as rhetorical analysis,
problem solving, decision making, or cognitive process, and suggested that
Regardless of the academic background of the instructor or the language used to describe critical thinking, all of these approaches share a set of common assumptions: there are identifiable critical thinking skills that can be taught and learned, and when students learn these skills and apply them appropriately, they become better thinkers. (p. 70)
As with Sternberg, Halpern (1999) maintained critical thinking skills should be
taught within a variety of contexts and for transfer so that students can apply the
knowledge and skills into other domains. In agreement with Paul (2004) and P. A.
Facione (2006), Halpern affirmed that critical thinking instruction must address student
dispositions, while acknowledging “it is not enough to teach college students the skills of
critical thinking if they are not inclined to use them” (p. 72). Further, students should not
only have a disposition toward critical thinking, they should also possess the willingness
to apply it. It is, therefore, crucial that students be taught the value of critical, reflective
thinking, and the essential effort needed to achieve it.
As indicated by Halpern (1999), it is vital that teachers encourage students to
apply the knowledge and skills learned in one context to other situations. Students should
be encouraged to transfer critical thinking knowledge, skills, and dispositions learned in
the educational environment to their personal and professional lives. Perkins (1993)
offered a variety of strategies to accomplish this complex task. He viewed the teacher as a
guide and mentor and suggested the three basic tools that can elicit thoughtful learning
are Socratic method-discussion, didactic instruction, and coaching for understanding
38
performances through practice, self-assessment, and informative feedback. Perkins
asserted that “knowledge and skill in themselves do not guarantee understanding” (p. 2),
and encouraged educators to promote transfer by helping students discover the
connections between their lives and the subject matter they are being taught. Perkins
noted understanding a topic of study is “a matter of being able to perform in a variety of
thought-demanding ways with the topic, for instance to: explain, muster evidence, find
examples, generalize, apply concepts, analogize, represent in a new way” (p. 4). Perkins
outlined six priorities for educators who teach for understanding: making learning a long-
term, thinking-centered process; providing for rich, ongoing assessment; supporting
learning with powerful representations; paying heed to developmental factors; inducting
students into the discipline; and teaching for transfer.
Specific Strategies
Argument Mapping
Van Gelder (2005) drew on lessons from cognitive science while discussing the
importance of developing students’ critical thinking abilities, and, as with Paul (1995c),
emphasized the need to improve teaching and educational institutions. He indicated that
individuals, as a rule, are not naturally critical and that critical thinking is a difficult, yet
not impossible, skill that takes time and deliberate practice to master. Students do not
become better thinkers merely through exposure to good critical thinking; they must
engage in critical thinking itself. He further contended that knowledge of the critical
thinking theory “is the basis for self-monitoring and correction” (p. 4) and that this
knowledge of theory encourages and improves teacher guidance and feedback.
39
Van Gelder (2005) believed a central part of improving this skill is in handling
arguments, that “arguments are presented or expressed in streams of words” (p. 4), and
that the structure of the arguments is expressed in “sequences of words or sentences” (p.
4). Arguments are, generally, linear and can be mapped out so that students can view the
logical structure of the rhetoric. Van Gelder claimed that “critical thinking skills improve
faster when instruction is based on argument mapping” (p. 5) and that “students in classes
based heavily on argument mapping consistently improve their skills much faster than
students in conventional classes” (p. 5). The advantages of argument maps are that they
make reasoning more easily understandable; students can see the reasoning, so they can
more easily identify important issues; arguments presented in diagrammatic form allow
students to follow the extended critical thinking procedures; and arguments laid out in
diagrams allow the teacher to see what the student is thinking (van Gelder, 2005).
Argument maps offer a straightforward, effective tool for improving students’ critical
thinking abilities.
Paul’s Model of Critical Thinking
Paul (2004) stated studies of higher education demonstrate that most college
faculty lack a substantive concept of critical thinking, most college faculty do not realize
they lack this substantive concept, and that lecture and memorization are still the norm in
college instruction. Paul’s (2004) model of critical thinking was created in an attempt to
provide a concept of critical thinking and a common model for instructional design. His
purpose for designing this model was to encourage educators to place emphasis on higher
order learning as “it is precisely these higher order thinking skills that are routinely
40
sacrificed when coverage and lower order recall dominate the classroom” (Paul, 1995f, p.
105).
Paul (1995f) discussed the dangers of a nonsubstantive concept of critical
thinking, while emphasizing that students’ critical thinking abilities cannot be refined
without reference to the “values, traits of mind, and dispositions that underlie those
skills” (p. 8). Assessment must be specific and consistent, an accurate concept of critical
thinking must be fostered, and appropriate assessment strategies must align with a true
concept of critical thinking so it is incorporated into testing and teaching. A real danger
exists when “departments of education incorporate a non-substantive concept of critical
thinking into statewide curriculum” (Paul, 1995f, p. 119).
Paul’s model of critical thinking is a metacognitive approach that consists of four
domains of critical thinking: elements of thought, abilities, affective traits, and
intellectual standards. Elements of thought refer to the kind of thinking that is conceptual
and inferential, and they are the basic building blocks of thinking; abilities include
thinking about complex issues; affective dimensions are the essential attitudes,
dispositions, and traits of mind needed for critical thinking; assessment of affective
dimensions is an important part of the assessment of higher order thinking and refers to
the validity and reliability in terms of the elements of thought; and intellectual standards
are the standards implicit in the assessment.
Paul and Elder took the essential elements of good critical thinkers and developed
The Thinker’s Guide to the Art of Socratic Questioning: Based on Critical Thinking
Concepts & Tools (Paul & Elder, 2006b) and The Thinker’s Guide to Analytic Thinking:
How to Take Thinking Apart and What to Look for When You Do (Elder & Paul, 2007).
41
The researcher used both of these Thinker’s Guides in the study. The Thinker’s Guides
address the following: critical thinking concepts and tools, how to study and learn a
discipline, the art of asking essential questions, active and cooperative learning, how to
improve student learning, how to write a paragraph, fallacies: the art of mental trickery
and manipulation, the human mind, critical and creative thinking, analytic thinking,
scientific thinking, how to read a paragraph, and ethical reasoning (Bessick, 2008).
Socratic Questioning
Paul and Elder (2000) asserted that educators concentrate far too much on
“coverage” (p. 1) of course material over “engaged thinking” (p. 1) partly because they
believe “answers can be taught separate from questions” (p. 1). Conversely, Paul and
Elder (2000) argued that questioning encourages thinking and learning because all
assertions and statements are, ultimately, answers to questions. They reasoned that
thinking is driven by questions as questions stimulate thinking, “define tasks, express
problems and delineate issues” (p. 1). Paul and Elder (1997) referred to Socratic
questioning as a highly disciplined, systematic approach to inquiry and “the most
powerful, teaching tactic for fostering critical thinking” (p. 1). Along with allowing
educators a more focused approach to their students and to their instruction, Paul (1995e)
acknowledged that “Socratic questioning is at the heart of critical teaching” (p. 335). as it
presents opportunities for students to develop and evaluate their ideas. Socratic
questioning
Raises basic issues; probes beneath the surface of things; pursues problematic areas of thought; helps students discover the structure of their own thought; helps students develop sensitivity to clarity, accuracy, and relevance; helps students arrive at judgment through their own reasoning; and helps students note claims, evidence, conclusions, questions-at-issue, assumptions, implications,
42
consequences, concepts, interpretations, points of view—the elements of thought. (Paul, 1995e, p. 336)
Similar to Paul and Elder (2000), Newton believed “intelligence is a reaction to
the social and physical world” (p. 289) and that educators, through higher cognitive
questioning, encourage students to adapt and use their knowledge in a problem-solving
manner. While suggesting educators are in need of concrete strategies for improving
critical thinking, Newton (1978) contended that educators should stimulate rather than
indoctrinate learners. In line with Piaget and Dewey, she perceived the learner as an
active participant in the learning process and presented a theoretical basis for higher
cognitive questioning, which includes analysis, synthesis, and evaluation, as an avenue to
critical thinking. Newton referred to Dewey’s definition of reflective thinking as the
“careful and persistent examination of an action, proposal, belief, analysis or use of
knowledge in the light of the grounds to justify it” (p. 287), while encouraging educators
to teach “that knowledge and those skills through discussion of controversial social
issues” (p. 287), literary criticism, and rhetorical analysis.
Teaching Critical Thinking: Related Studies
Paul et al. (1997b) conducted a study of teacher preparation programs to assess
the extent to which they prepare candidates to teach critical thinking and problem-solving
skills. Using both quantitative and qualitative data, the researchers’ objectives were to
assess current teaching practices and knowledge of critical thinking among faculty,
identify representative teaching practices that improve critical thinking, and develop
policy recommendations based on the results of the study. The results of their research
43
were based on a study of randomly selected California professors from 38 public and 28
private colleges and universities. In-depth interviews were utilized to determine the
extent to which students in teacher preparation programs were being taught to encourage
and teach critical thinking instruction. The researchers also sought to determine the
faculty’s conception of critical thinking. Findings of the research study indicated that
although the majority of the faculty claimed critical thinking is a primary objective of
instruction, only a small minority could clearly explain what critical thinking is. Paul et
al. maintained that although teacher educators claim to be committed to teaching critical
thinking, “few have had any in-depth exposure to research on the concept and most have
only a vague understanding of what it is and what is involved in bringing it successfully
into instruction” (p. 10). Paul et al. claimed educators must have a clear concept of what
critical thinking is and must be taught “ways that facilitate skill in critical thinking and
the ability to teach it to others” (p. 2) so that students will graduate and enter the work
force with the problem-solving abilities needed for clarity of thought and intellectual
discipline.
Meinecke (1997), in an attempt to develop an operational definition of critical
thinking for New Mexico State University (NMSU) College of Business Administration
and Economics, designed a naturalistic study to examine existing definitions of critical
thinking and approaches to teaching critical thinking. Meinecke used surveys, interviews,
and classroom observations to collect data to determine existing instructional practices.
The study revealed that “business college faculty did not have a conceptual definition of
critical thinking and often failed to teach critical thinking in their classrooms” (Meinecke,
1997, Abstract). Meinecke’s conceptual definition of critical thinking was “an ability to
44
engage in a process of objectively analyzing a problem with an open mind and
intellectual honesty” (Abstract). She believed critical thinking is a recursive process that
includes reason assessment, a critical spirit, and metacognition, and maintained that
critical thinking can be developed through a reflective pedagogical paradigm and that
students should be “placed at the center of the classroom environment with autonomous
control over their own learning process” (p. 208).
Meinecke (1997) contended that educators at NMSU need to create a community
of inquiry by focusing on combined classes “with an emphasis on hands-on experience
taught with a common language that includes their unified conceptual definition of
critical thinking” (p. 212). Further, she encouraged educators at NMSU to create
classroom environments that promote critical and creative thinking through the
application of the six dimensions of thinking, as developed by Tishman, Perkins, and Jay,
which include a language of thinking, thinking dispositions, mental management,
strategic spirit, higher order knowledge, and transfer.
There is a growing number of educators who are attempting to design
instructional pedagogy that will promote students’ critical thinking skills. Several studies
have focused on direct instruction of critical thinking through the use of argument
mapping, Paul’s Thinker’s Guides, and Socratic questioning (Bessick, 2008; Reed, 1998;
Scanlan, 2006; Tsui, 1998; Van Erp, 2008; Yang, 2008), and their findings suggested that
although these instructional strategies prove useful for promoting critical thinking, further
research is needed in this area.
Bessick (2008) discussed precollege preparation and the responsibility of
educational institutions in improving higher order thinking skills. Her contention was that
45
students entering colleges and universities are deficient in reading, writing, mathematics,
and critical thinking skills; however, “because of the lack of preparedness in the basics,
schools have become so focused on general academics that efforts to enhance the
development of critical thinking skills in their students is minimal” (p. 33). Bessick
believed direct instruction in critical thinking and reasoning can improve student learning
outcomes; however, the ultimate educational goal should be to enable students to transfer
those skills learned in the classroom to their everyday lives. In her study, Bessick
examined the importance of teaching critical thinking skills and their relationship to
academic achievement. Bessick conducted an experimental quantitative study of
freshmen students being tutored in repeat courses in which she examined the effect of
direct instruction in critical thinking on the students’ critical thinking ability and
academic achievement using van Gelder’s rational argument mapping and Paul’s
Thinker’s Guides. Data analysis revealed the intervention programs using the Thinker’s
Guides based on Paul’s model of critical thinking and van Gelder’s rational argument
mapping program revealed no significant improvement in students’ critical thinking
skills. However, the Thinker’s Guides group and the control group demonstrated
improvement in students’ academic achievement. Bessick maintained there is a need to
improve the critical thinking skills and that “instruction in critical thinking, whether
through direct instruction or independent study in addition to tutoring may contribute to
the improvement of students’ academic achievement” (p. 153). Her findings suggested
that further research is needed using a larger sample size to determine the extent to which
direct instruction using the Thinker’s Guides and the rational argument mapping program
can improve students’ critical thinking ability and academic achievement.
46
Scanlan’s (2006) study was conducted in order to explore strategies to improve
students’ critical thinking skills by incorporating Paul’s elements and standards of
reasoning into all standards-based curriculum. The project researched the effect of
focused critical thinking training on the composition skills of 12th graders, specifically
through educational techniques developed by Paul. Scanlan included persuasive essays,
and assessment was measured through a series of rubrics. The results of his research
indicated the value of incorporating Paul’s elements and standards of reasoning is in the
gains students made in terms of clarity of writing, level of analysis, use of supporting
information, and organization of ideas.
Reed (1998) conducted a study at a community college in central Florida in which
she investigated the effect of integrating Paul’s model of critical thinking into a U.S.
history course on community college students’ abilities to think critically about U.S.
history and about everyday issues, dispositions toward thinking critically, and knowledge
of history content. Reed found that students’ critical thinking abilities improved in a
single course, the students’ end-of-term knowledge of history content did not suffer when
training in critical thinking abilities was integrated into course material, and age and
gender did not play significant roles in developing college students’ critical thinking
abilities. Reed’s study revealed that students’ abilities to think critically can improve in a
single course when provided with deliberate instruction and that Paul’s model “can be
successfully integrated into an introductory history course with statistically significant
benefits to students’ abilities to think critically within a domain and to their general
critical thinking abilities” (p. 160).
47
Yang (2008) conducted a study designed to examine the effectiveness of teaching
critical thinking skills “through asynchronous discussion forums (ADFs) with the
facilitation of teaching assistants, to investigate student interaction patterns and the depth
of their critical thinking demonstrated via the asynchronous discussion forum” (p. 241).
The major goal was to investigate whether students’ critical thinking skills would
improve after they participated in Socratic dialogues as taught by the instructor and then
as modeled and facilitated by the teaching assistants. The main task of the teaching
assistants was to focus on the quality of the student interaction, including the
assumptions, reasoning, and evidence provided by the students. The qualitative analysis
provided a detailed description of how students’ discussions moved from the lower to the
higher phases of critical thinking. Results from the CCTST indicated that instructors who
use Socratic dialogues during small-group online discussions can successfully improve
students’ critical thinking skills in a large university class. This improvement in learning
is accomplished as “students interact with their peers or instructor and benefit from
combining their levels of expertise, offering support, distributing the thinking load, and
confronting alternative points of view” (Yang, 2008, p. 261). The results also indicated
that “learning is also accomplished as students elaborate, clarify, and justify their
personal responses/solutions” (Yang, 2008, p. 261).
Van Erp’s (2008) study consisted of an investigation of the lived experiences of
the instructor and a group of learners in three online graduate courses in which critical
thinking skills were emphasized alongside course content. Students completed a pre- and
postcourse self-assessment reflection and participated in postcourse interviews. The
courses included strategies that promote critical thinking skills, including asynchronous
48
discussion, instructor modeling and coaching, explicit instruction about critical thinking,
self-assessment writing, Socratic questioning, and scoring rubrics that included critical
thinking criteria. Van Erp used P. A. Facione’s definition in the study while investigating
the correlation between the definitions of critical thinking found in literature and the
themes identified in learners’ definitions; the terms “‘reflective’ and ‘reflection’ were
themes in both the pre and the post course self-assessment reflections” (p. 97). Van Erp
believed the strongest themes that emerged in the precourse self-assessment were open-
mindedness and the nature of asynchronous discussion; moreover, throughout the
interview process, the strongest theme to emerge from the student interviews was
instructor feedback. Learners’ critical thinking ability was enhanced when they were
required to examine assumptions and to be open-minded while considering possibilities.
Further, the conversations and the systematic, carefully crafted questions designed by the
instructor challenged the learners’ assumptions and improved student learning and the
quality discussion. Van Erp recommended embedding critical thinking throughout an
entire program while supporting van Gelder’s (2005) premise that critical thinking is a
difficult skill to master and that instructors should scaffold learning and be systematic
and intentional in their approach through carefully designed discussion questions. Quality
questions that scaffold learning can impact learning because they challenge learners to
think reflectively and critically.
Tsui (1998) suggested that although developing student critical thinking skills is a
primary goal of education, educators lack sufficient knowledge as to how this educational
aim can be achieved. Tsui combined quantitative and qualitative data to investigate the
impact of college on the development of student critical thinking skills and to examine
49
factors that lie inside as well as outside of the classroom. The sample included 13,216
students from more than 200 four-year colleges and universities. Quantitative data were
drawn from longitudinal questionnaire surveys of a national sample of college students.
The qualitative data were gathered by interviewing 55 participants and observing 28
classes at four institutional case study sites.
Tsui (1998) found that critical thinking abilities can be developed positively in
association with substantive writing, critical discussion, class presentations, student-led
inquiry, emphasis on analysis over recall, and the constructivist philosophy to learning;
negative factors for promoting critical thinking include lectures and multiple-choice
examinations. Further, Tsui discovered that infusion of critical thinking into the
curriculum is more successful when “faculty have enough confidence in their students to
challenge them [and are] enthused enough to partake in collegial exchanges about
effective pedagogy” (Abstract). Additionally, Tsui’s study revealed that an important
factor in enhancing critical thinking abilities is engagement in critical dialogue with
peers.
Conclusion
Promoting students’ critical thinking abilities should be of primary concern to all
educators. There are a variety of definitions for critical thinking; however, the researcher
found P. A. Facione’s (2006) metacognitive approach with its emphasis on analysis,
interpretation, inference, explanation, evaluation, and self-regulation to be most suitable
for this study.
50
Paul (1995b) viewed critical thinking as the essential foundation for education as
individuals must have the ability to adapt to the demands of the 21st century. Frequently,
instructional practices emphasize the content rather than the quality of instruction.
Although students can perform basic skills, pedagogical practices that encourage higher
order thinking skills could enhance student critical thinking. Another key obstacle to
promoting critical thinking is that teachers and students may not have a clear idea of what
critical thinking involves.
As Paul (2004), P. A. Facione (2006), and Perkins (1993) contended, critical
thinking should be taught within a variety of contexts, and teachers should encourage
students to transfer the knowledge and skills learned in the classroom into their everyday
lives. What is essential is that educators develop the appropriate skills and strategies
needed to enhance students’ higher cognitive skills.
While there is continual debate regarding the best methods for improving
students’ critical thinking abilities, critical thinking, although a complex skill, may well
be successfully taught through direct instruction. Bessick (2008) confirmed the
importance of direct instruction in critical thinking skills and academic achievement
using a combination of van Gelder’s argument mapping and Paul’s Thinker’s Guides.
Scanlan (2006) and Reed (1998) discovered that student learning and critical thinking can
be enhanced by using Paul’s model of critical thinking. Van Erp (2008) and Yang (2008)
concluded that Socratic dialogues can improve students’ critical thinking abilities;
however, Van Erp and Yang stressed the importance of timely feedback, coaching, and
frequent modeling. Finally, Tsui (1998) found that critical thinking abilities can be
51
developed positively in association with substantive writing, critical discussion, class
presentations, and student-led inquiry.
Sternberg (2003) argued that teaching students to think critically should be a
primary goal of educational institutions so students acquire the abilities to transfer those
skills into their everyday lives. Paul (1995b) avowed that critical thinking should be the
essential foundation for education, while arguing that educational institutions are ill-
prepared for the rapid changes occurring in society, but educators and students do not
have a clear idea of what critical thinking is. Elder (2000) commented on students being
ill-prepared for the current job market, while suggesting that educators encourage
intellectual discipline. Similarly, P. A. Facione (2006) addressed the value of critical
thinking and the need for an informed citizenry. Direct instruction in critical thinking
using a variety of instructional strategies could promote those critical thinking skills and
traits of mind necessary for productive, successful living.
52
CHAPTER 3. METHODOLOGY
Introduction
Cultivating critical thinking, intellectual growth, and lifelong learning
opportunities that afford students the knowledge and skills necessary for success in life is
a fundamental goal of all educational institutions. Educators are beginning to explore
those pedagogical practices that could effectively develop higher order thinking skills.
Instructional strategies that advance critical thinking pedagogy on a consistent basis
could positively impact the range and quality of student critical thinking skills’
performance. This study investigated whether direct instruction in rhetorical analysis in a
critical thinking course would improve students’ critical thinking abilities. This chapter
provides a description of the methodology used in the study and includes a statement of
the problem, the research questions, the research design and methodology, a discussion of
the population and sample, and a detailed description of the instrumentation, data
collection, and data analysis. This is followed by a discussion of ethical considerations,
limitations of the study, and a summary.
Statement of Problem
It is not known whether an intervention in rhetorical analysis will improve student
critical thinking knowledge, skills, and dispositions. This study was an exploration and
53
evaluation of instructional pedagogy designed to promote students’ critical thinking
abilities. The researcher investigated whether direct instruction in rhetorical analysis in a
college critical thinking course could make a difference in students’ critical thinking
knowledge, skills, and dispositions. The interventions included argument mapping;
Thinker’s Guides, based on Paul’s model of critical thinking; and Socratic questioning.
These interventions were supplemented with course work that involved reader/writer
workshops and peer review.
Research Questions
The following research questions guided this study:
1. What were the students’ perceptions of critical thinking before and after direct instruction in argument mapping?
2. What were students’ perceptions of critical thinking before and after direct instruction in Paul’s Thinker’s Guides?
3. What were the students’ perceptions of critical thinking before and after direct instruction in Socratic questioning?
4. What did the students report about the changes in their knowledge, skills, and dispositions relating to critical thinking after the direct instruction in rhetorical analysis?
Null Hypothesis: There will be no difference in scores on the pretest and posttest
on the California Critical Thinking Skills Test–Form 2000 indicating changes in student
critical thinking skills before and after direct instruction in rhetorical analysis.
54
Research Design and Methodology
The triangulation methodology involves using quantitative and qualitative
methods and “gives the researcher greater understanding of the issues under
investigation” (Cowden, Chivore, Maravanyika, Nyagura, & Sibanda, 1999, p. 11). This
mixed methodology approach was used in this one-group pretest–posttest design to
determine whether direct instruction in critical thinking, using the aforementioned
interventions, would improve students’ critical thinking knowledge, skills, and
dispositions. The independent variable for this study was the critical thinking
interventions. The dependent variables were the posttest scores on the CCTST–2000 and
on the Pre- and Post- Critical Thinking Surveys. The researcher examined the
relationship between the variables involving teacher instruction of the interventions, the
students’ mastery of those skills, and the students’ perceptions of the effectiveness of the
interventions. This blend of methodologies enhanced the relevance and strength of the
study as a combination of qualitative and quantitative methods were used to cross-check
the validity of data. The quantitative approach is formal and conclusive and involves
“collecting numerical data on observable behaviors of samples and then subjecting these
data to numerical analysis” (Gall et al., 2003, p. 23).
The researcher was the teacher of an intact class, and no attempt at randomization
occurred; therefore, the study was limited to a description of an existent situation rather
than creating an experimental situation. In teacher action research, the data are collected
and analyzed to assess and improve educational practice. Since the researcher was the
instructor of the critical thinking course, there was potential for researcher bias as the
researcher may have had authority over the participants. The researcher understood this
55
and attempted to approach the study with caution and without preconceived assumptions
of possible outcomes or results. Care was taken to maintain clarity and objectivity
through sharing with research participants and through personal reflection, assessment,
and modification of instructional practices.
The researcher administered a Pre- Critical Thinking Survey at the beginning of
the course to measure the dependent variable; implemented the experimental treatment,
or the independent variable; and administered a Post- Critical Thinking Survey at the end
of the course. The qualitative survey was used to support description and analysis by
providing contextual information, and by exploring areas that are impossible to
conceptualize (Cowden et al., 1999). Further, the qualitative data aided in looking for
patterns, and the observations made by the survey results offered insights into student
perceptions of the most effective instructional interventions. The CCTST–2000 was
administered as a pretest at the beginning of the course and measured the students’
critical thinking skills. The same test was given as a posttest and measured growth in
critical thinking abilities at the end of the course. Statistical analysis was administered on
the CCTST–2000 pretest and posttest scored by Insight Assessment.
Population and Sample
The population of this study was a convenience sample of students who attended
a mid-sized culturally diverse community college in southern California. The district
serves a geographic area that is semirural in nature. The college offers two-year academic
transfer programs and corresponding vocational education programs. Students at this
community college reflect the diversity of the surrounding community. The demographic
56
makeup of the college reveals that 66.5% of the student population is White/non-
Hispanic, 8.7% Black, 20% Hispanic, 3.7% Asian, and 1.1% Native American. The total
number of students enrolled in composition classes is approximately 2,000 students;
however, it was not convenient for the researcher to collect data from this entire
population. Hence, the population was limited to those students enrolled in a critical
thinking class taught by the researcher.
Convenience sampling suited the purposes of this study as it was the most
practical approach for the researcher. The sample consisted of students enrolled in an on-
campus critical thinking course taught by the researcher. Total enrollment in the critical
thinking course was 29 students at the beginning of the semester; 19 of those students
agreed to participate in the study; however, three students dropped from the course and
one student did not show up the final day for the posttest; thus, the final research sample
consisted of 15 students.
The critical thinking course was designed to develop the students’ critical
thinking, reading, and writing skills. Basic English is a prerequisite to this course. The
critical thinking course is not a required course; however, it is required for students who
are English majors. The critical thinking course focused primarily on the analysis and
evaluation of expository and argumentative discourse, and on writing analytical and
argumentative essays. The objectives of the course and the purpose of studying
composition was to help students improve their writing, reading, and critical thinking
skills to a level that will enable them to transfer those skills to their professional and
personal lives. All of the students in the sample received the interventions that were
57
provided throughout the semester. Gains in student critical thinking abilities were
expected due to the ongoing instruction of the interventions.
Instrumentation
The CCTST–2000 was ordered from Insight Assessment and was administered by
the researcher as a pretest and posttest. There are three versions of the CCTST: Form A,
Form B, and Form 2000. The CCTST–2000 is based on the APA Delphi consensus
conceptualization of critical thinking. This conceptualization of critical thinking is an
expression of expert consensus based on the participation of 46 leading theorists,
teachers, and critical thinking assessment specialists, and was reaffirmed in the national
survey conducted by the National Center for Higher Education Teaching, Learning and
Assessment at Pennsylvania State University. The Delphi study consensus definition of
critical thinking has been endorsed by educators and scholars around the world (N. C.
Facione, Facione, Blohm, & Gittens, 2008). The CCTST–2000 is specifically designed to
measure the skills dimension of critical thinking. The items on the CCTST–2000
emphasize a variety of critical thinking skills, which include analysis of the meaning of a
given sentence, drawing a correct inference from a set of assumptions, and evaluating or
justifying the inference provided.
According to N. C. Facione et al. (2008), “the primary use of the CCTST is to
gather valid and reliable data about the baseline, entrance-level, or exit-level critical
thinking skills of various groups of people, commonly college level students and working
adults” (p. 11). The items are given in standard English and are set in contexts and
address topics that are familiar to college-age students. The CCTST–2000 is appropriate
58
for college undergraduate and graduate students and for the researcher’s study (N. C.
Facione et al., 2008).
The CCTST–2000 targets the core critical thinking skills of analysis,
interpretation, inference, evaluation, and explanation. The CCTST–2000 takes 45
minutes to complete under timed conditions and can also be administered untimed. The
CCTST–2000 can be taken in the booklet using a paper-and-pencil format with a
Scantron® sheet or through an online formal set up by Insight Assessment.
Reliability
Items in the CCTST–2000 include the critical thinking skills identified by the
Delphi experts: interpretation, analysis, evaluation, explanation, and inference. They are
discipline-neutral; sex-role and social class stereotypic context have been avoided; and
“equal numbers of male and female referents are used to decrease possible gender and
cultural bias” (N. C. Facione et al., 2008, p. 27). Validation studies “have produced
consistency estimates . . . ranging from .68 to .80” (N. C. Facione et al., 2008, p. 27).
According to N. C. Facione et al.
Reliability ratings of .65 to .75 have been suggested to be considered sufficient for placing confidence in instruments of this type. The reliability coefficients demonstrated by the CCTST is more than sufficient to inspire confidence in the internal consistency of its items to measure the overall construct. (p. 27)
The CCTST has been tested by its developers as well as by independent
researchers (Bondy, Koenigseder, Ishee, & Williams, 2001; N. C. Facione & Facione,
1997; P. A. Facione, Facione, Blohm, Howard, & Giancarlo, 1998; Jacobs, 1995, 1999).
According to Pike (as cited in Laird, 2005), the CCTST–2000 is considered to be more
reliable than Forms A and B. Laird suggested the instrument is suitable for educational
59
assessment; however, “test administrators should pay particular attention to the reliability
and inter-correlation of the total score and subscales within the group under study” (p. 4).
Validity
The validity of the CCTST–2000 can be examined in terms of content validity,
construct validity, and criterion validity. Content validity refers to how well the test items
represent the critical thinking domain. CCTST–2000 test items were chosen for their
theoretical relationship to the Delphi panel’s conceptualization of critical thinking (N. C.
Facione et al., 2008). Construct validity refers to the extent the CCTST–2000 measures
what it claims to measure, in other words, the Delphi experts’ definition or
conceptualization. As more students enroll in critical thinking courses, and as their test
results improve as a result of this enrollment, well-crafted multiple-choice items, as those
in the CCTST–2000, can validly and reliably measure this improvement in higher order
thinking skills. Further, face validity is supported by the types of questions asked on the
CCTST–2000 that encourage students to make judgments, identify correct analyses, draw
inferences, evaluate reasoning, and justify inferences and evaluations (N. C. Facione et
al., 2008). Criterion validity refers to a test’s ability to predict external criteria, such as a
higher level of college success. The CCTST–2000 is reported to have a content validity
of .74 and moderate criterion validity with grade point average and SAT math and verbal
scores (N. C. Facione et al., 2008).
The second and third instruments that were used in the study, the Pre- and Post-
Critical Thinking Surveys, were qualitative in nature. The second instrument was
comprised of two sections: a student demographic survey and the Pre- Critical Thinking
Survey. The Post- Critical Thinking Survey included three sections: the student
60
demographic survey, the Critical Thinking Survey, and a Student Perceptions of Critical
Thinking Survey. The student demographic portion of the survey included questions
concerning gender, race, age, current year in college, academic major, student status,
students’ perceptions of their writing and comprehension skills, and previous academic
experience in critical thinking. The Pre- and Post- Critical Thinking portion of the survey
offered useful information regarding the students’ critical thinking knowledge, skills, and
dispositions. Finally, the Student Perceptions of Critical Thinking Survey provided
valuable qualitative feedback concerning the effectiveness of the instructional
interventions included in the study.
Field Test
To determine whether the surveys had merit and to correct any flaws, the Pre- and
Post- Critical Thinking Surveys were administered to four teachers at the community
college level for review before conducting the research study. The participants of this
field test were encouraged to make criticisms and recommendations for improving the
surveys. Following the review, respondents indicated the surveys were appropriate,
understandable, and relevant to the research study.
Data Collection
On July 13, 2009, a research permission letter was sent to the dean of humanities
at the study college requesting permission to conduct research in a critical thinking
course during the Fall semester, along with a detailed explanation of the purpose of the
research. During the first week of the course, students and the researcher read and
61
discussed the critical thinking course syllabus, a detailed description of the research study
was given, and a request was made for those students who wished to participate in the
study. Participation in the research by students was strictly voluntary. Data collection
began and an informed consent letter was given to those who were interested in
participating in the study. Students were assured that participation in the study was
strictly voluntary, that participation or nonparticipation would in no way affect their
grade, and that their personal identity would not be released in the dissertation. At the end
of the first week of the semester, the informed consent letters were collected from those
students who wished to participate in the study.
A pre-investigation of student dispositions toward critical thinking was utilized. A
collection of data to determine student views of their own critical thinking dispositions
and abilities was gathered during the first week of the course. Students completed the
Pre- Critical Thinking Survey (Appendix A) self-assessment, which included a Student
Demographic Survey and questions concerning critical thinking dispositions, knowledge,
and skills. During the second week of the course, the CCTST–2000 was administered as a
pretest.
Embedded within critical thinking course curriculum, each of the independent
variables (interventions) was systematically included in the researcher’s instructional
strategies at 4-week intervals throughout the semester. Utilization of argument mapping
was introduced during the first 4 weeks; this was followed by instruction in Paul’s
Thinker’s Guides; during the final 4 weeks, instructional content included Socratic
questioning.
62
During the final week of the semester, the CCTST–2000 was again administered
as a posttest. The paper-and-pencil version consisted of the test booklets (10-page
documents that contain directions and the 34 multiple-choice questions) and scannable
answer forms. Before beginning the test, the students were given an identification
number, and a master list was kept of the test takers’ names matched with their personal
identification numbers (PINs). This list was kept in a secure place to protect the privacy
of the test takers.
Upon completion of the CCTST–2000 pretest and posttest, exams were shipped
back to Insight Assessment, where they were scanned and scored using the CapScore
system. Insight Assessment then sent the research data file and a report summarizing the
survey results. Reports were generated that included descriptive statistics for the group as
a whole as well as subscale statistics. The CCTST–2000 is normed for 4-year college
students, 2-year college students, graduate students, and workplace manager-level
employees. These norm groups were used in the analysis of the test-taker data, and the
comparison with norm sample information was included in the Basic CapScore (N. C.
Facione et al., 2008).
Students also completed a Post- Critical Thinking Survey (Appendix B) during
the final week of the semester that consisted of the following sections: the Student
Demographic Survey; self assessment questions concerning critical thinking knowledge,
skills, and dispositions; and questions concerning the extent to which instruction in
argument mapping, Paul’s Thinker’s Guides, and Socratic questioning increased their
understanding of and ability to apply critical thinking concepts to practical situations in
63
life. The data from the Pre- and Post- Critical Thinking Surveys (Appendixes A & B) was
analyzed using tables in a Microsoft Word document to identify significant themes.
Data Analysis
Research Questions 1–3 focused on student perceptions regarding a positive
relationship between argument mapping, Paul’s Thinker’s Guides, and Socratic
questioning, and the acquisition of critical thinking skills. Research Question 4 focused
on the students’ perceptions of the changes in their knowledge, skill, dispositions, and
attitudes after direct instruction in critical thinking. These questions were addressed in the
Likert-scaled Student Perceptions of Critical Thinking and were analyzed using tables in
a Microsoft Word document and the highlighting tool to identify significant themes and
instructional methodologies most beneficial for developing students’ critical thinking
abilities. Additional charts and graphs were used to investigate patterns and insights. This
tool was particularly effective in recognizing which critical thinking instructional
strategies are most effective.
Research Question 5 focused on the extent to which the pretest and posttest scores
indicate changes in the students’ knowledge, skills, and dispositions relating to critical
thinking after direct instruction in argument mapping, Paul’s Thinker’s Guides, and
Socratic questioning. For both the pretest and posttest, the CCTST–2000 was the sole
quantitative instrument used. The CCTST–2000 involves an overall score on one’s
critical thinking skills and five subscales. The five subscales are Analysis, Evaluation,
Inference, Deductive Reasoning, and Inductive Reasoning. Analysis, Inference, and
Evaluation “draw together the major core skills identified in the theory of critical
64
thinking advanced in The APA Delphi Report. Inductive Reasoning vs. Deductive
Reasoning is represented in the fourth and fifth CCTST–2000 sub-scale” (N. C. Facione
et al., 2008, p. 12). The total score on the CCTST–2000 provides the overall measure of
students’ critical thinking skills.
Students were assigned a PIN for reasons of confidentiality; additionally, the
group indicator on the CapScore response form allowed for institution-specific
information to be coded onto the response form. For the pretest, each student indicated
001 on his or her response form. For the posttest, each student indicated 002 on his or her
response form. CapScore provided a total and scale CCTST–2000 scores for each test
taker by PIN and descriptive statistics for both Groups 1 and 2 separately. The CapScore
answer forms were returned along with a PC-formatted disk with an Excel and a tab-
delimited text file, including total and subscale CCTST–2000 scores for each test taker by
PIN (N. C. Facione et al., 2008). Changes in students’ critical thinking scores were
analyzed comparing the mean pre-CCTST–2000 score to the mean post-CCTST–2000
score using a t test.
Ethical Considerations
Certain procedures had to be followed in order to obtain permission for this
research study. A written description of the research design was submitted to the
administration. In a research study in which human participants are involved, students
must be informed about what will occur during the research study. Consent was obtained
from the students and the administrative faculty. Participants received an explanation of
the test and course room procedures to be used and a letter describing the research and
65
conditions of their participation (Gall et al., 2003). Meetings with the administration
occurred in order to obtain their interest and cooperation.
The research study was examined by Capella University’s Institutional Review
Board to ensure that it complied with “institutional regulations, professional standards of
conduct and practice and the participants provisions of the Code of Federal Regulations
for the Protection of Human Subjects” (Gall et al., 2003, p. 66). Student confidentiality
was respected, and participants were told who would have access to data and that their
privacy would be protected. Random codes were given to participants in the study to
ensure participant anonymity.
Limitations of the Study
A limitation to this study was the small sample size, which limits the conclusions
that can be generalized to the target population. A random sample from the accessible
population in the critical thinking course was used in the study. The researcher gathered
data from the Pre- Critical Thinking Survey (Appendix A) to determine the degree of
similarity between the accessible sample population with regard to age, gender
distribution, and ethnic distribution in the critical thinking course to the target population.
However, generalization may not be possible, nor is it necessary. It is truly up to the
reader to find similarities in an action research study to a different site or location.
In view of the fact that the researcher adopted a teacher action research approach
in which the researcher was the instructor, there was the potential for researcher bias as
the researcher could have real or perceived authority over the participants; however, the
researcher administered the CCTST–2000 and the Student Perceptions of Critical
66
Thinking Surveys as pre- and posttests. Since the Student Perceptions of Critical
Thinking Survey is qualitative in nature, the researcher provided a thorough description
of the findings. These findings were checked by the results generated by the CCTST–
2000, which is a standardized test and in which there is no room for bias. This
triangulation method contributed to the validity of the study as it extracted multiple
perspectives and data sources and helped to “eliminate biases that might result from
relying exclusively on any one data-collection method” (Gall et al., 2003, p. 464).
Suitable controls over extraneous variables that could have threatened the validity
of the experiment needed to be established in order to allow a precise interpretation of
data. Experimental designs often provide adequate controls for sources of internal
validity, but the results have restricted external validity. Another possible limitation of
the study was that the researcher had a certain amount of control over what occurred in
the classroom and very little control over what occurred in the students’ lives outside of
the classroom.
Other possible limitations to the research study included (a) the length of the
study as the development of critical thinking skills requires practice over an extended
period of time and (b) the response rate of the participants involved in the final data
analysis.
Summary
This study investigated whether direct instruction in rhetorical analysis in a
critical thinking course can improve students’ critical thinking skills. Argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning were used as treatment interventions in
67
increments for 12 weeks during the semester. Data were collected from the pretest at the
beginning of the course and from the posttest during the final week of the course using
the results from the CCTST–2000. Students were also required to complete Pre- and
Post- Student Surveys as a means to better conceptualize students’ perceptions of what
they believed to be the most beneficial and applicable method for enhancing their critical
thinking skills. Final scores on the CCTST–2000 posttest and the Student Perceptions of
Critical Thinking Surveys were used to determine student achievement in critical
thinking.
68
CHAPTER 4. DATA COLLECTION AND ANALYSIS
Introduction
The purpose of this study was to determine whether there would be improvement
in students’ critical thinking abilities, including knowledge, skills, and dispositions, after
the intervention of direct instruction in rhetorical analysis. The researcher investigated
student perceptions of the effectiveness of the following instructional strategies:
argument mapping; Thinker’s Guides, based on Paul’s model of critical thinking; and
Socratic questioning. The goal of the study was, on the one hand, to promote critical
thinking knowledge, skills, and dispositions and, on the other hand, to determine
improvement in student perceptions of critical thinking knowledge, skills, and
dispositions so they may transfer those skills across the academic disciplines and into
their personal and professional lives.
This chapter provides a statement of the problem, a review of the research
questions, an overview of data collection and instruments, and a discussion of student
demographics. This is followed by an overview of the data and the results as they relate
to the four research questions and the null hypothesis.
69
Statement of the Problem
It is not known whether direct instruction in rhetorical analysis will improve
learners’ critical thinking knowledge, skills, and dispositions. This study explored
instructional pedagogy to promote students’ critical thinking abilities. The researcher
investigated whether direct instruction in rhetorical analysis in a college critical thinking
course could make a difference in students’ critical thinking skills. The interventions
included argument mapping; Thinker’s Guides, based on Paul’s model of critical
thinking; and Socratic questioning. These interventions were supplemented with course
work that involved reader/writer workshops and peer review.
Research Questions
The following questions directed the study:
1. What were the students’ perceptions of critical thinking before and after direct instruction in argument mapping?
2. What were the students’ perceptions of critical thinking before and after direct instruction in Paul’s Thinker’s Guides?
3. What were the students’ perceptions of critical thinking before and after direct instruction in Socratic questioning?
4. What did the students report about the changes in their knowledge, skills, and dispositions relating to critical thinking after the direct instruction in rhetorical analysis?
Null Hypothesis: There will be no difference in scores on the pretest and posttest
on the California Critical Thinking Skills Test–Form 2000 indicating changes in student
critical thinking skills before and after direct instruction in rhetorical analysis.
70
Overview of Data Collection
The mixed-methodology approach was used in this action research study. The
study pertained to one semester of a college critical thinking course that was taught in a
naturalistic setting by the researcher. The overall goal of the study was to examine the
effect of direct instruction in rhetorical analysis in students’ critical thinking knowledge,
skills, and dispositions. Twenty-nine students were initially enrolled in the critical
thinking course. Of the 29 students, 19 agreed to participate in the study. Three of those
students dropped the course and one student was not present the final day of the course
and did not take the posttest; thus, the final research sample was comprised of 15 (N =
15) of the 26 students enrolled in the critical thinking course.
Data Collection and Instruments
The triangulation or mixed-methodology approach was used in this one-group
pretest–posttest design to determine whether direct instruction in critical thinking, using
the aforementioned interventions, would improve students’ critical thinking knowledge,
skills, and dispositions. The independent variable for this study was the critical thinking
interventions. The dependent variables were the posttest scores on the CCTST–2000 and
on the Pre- and Post- Critical Thinking Surveys. The researcher examined the
relationship between the variables involving teacher instruction of the interventions, the
students’ mastery of those skills, and the students’ perceptions of the effectiveness of the
interventions.
Research Questions 1–3 focused on student perceptions regarding a positive
relationship between argument mapping, Paul’s Thinker’s Guides, and Socratic
71
questioning, and the acquisition of critical thinking skills. Research Question 4 focused
on the students’ perceptions of the changes in their knowledge, skill, dispositions, and
attitudes after direct instruction in critical thinking. These questions were addressed in the
Likert-scaled Pre- and Post- Critical Thinking Surveys and were analyzed using tables in
a Microsoft Word document to identify significant themes and instructional
methodologies most beneficial for developing students’ critical thinking abilities.
Research Question 5, the null hypothesis, focused on the extent to which the
pretest and posttest scores indicate changes in the students’ knowledge, skills, and
dispositions relating to critical thinking after direct instruction in argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning. For both the pretest and posttest, the
CCTST–2000 was the sole quantitative instrument used. The CCTST–2000 test involves
an overall score on one’s critical thinking skills and five subscales. The five subscales are
Analysis, Evaluation, Inference, Deductive Reasoning, and Inductive Reasoning.
Analysis, Inference, and Evaluation “draw together the major core skills identified in the
theory of critical thinking advanced in The APA Delphi Report. Inductive Reasoning vs.
Deductive Reasoning is represented in the fourth and fifth CCTST–2000 sub-scale” (N.
C. Facione et al., 2008, p. 12). The total score on the CCTST–2000 provided the overall
measure of students’ critical thinking skills.
Students were assigned a PIN for reasons of confidentiality; additionally, the
group indicator on the CapScore response form allowed for institution-specific
information to be coded onto the response form. For the pretest, each student indicated
001 on his or her response form. For the posttest, each student indicated 002 on his or her
response form. CapScore provided a total and scale CCTST–2000 scores for each test
72
taker by PIN and descriptive statistics for both Groups 1 and 2 separately. The CapScore
answer forms were returned along with a PC-formatted disk with an Excel and a tab-
delimited text file, including total and subscale CCTST–2000 scores for each test taker by
PIN (N. C. Facione et al., 2008). Changes in students’ critical thinking scores were
analyzed comparing the mean pre-CCTST–2000 score to the mean post-CCTST–2000
score using a t test.
The second and third instruments used in the study, the Pre- and Post- Critical
Thinking Surveys (Appendixes A & B) were qualitative in nature. The Post- Critical
Thinking Survey included three sections: the student demographic survey, the Critical
Thinking Survey, and a Student Perceptions of Critical Thinking Survey. The Student
Perceptions of Critical Thinking portion of the Post- Critical Thinking Survey provided
valuable qualitative feedback concerning the effectiveness of the instructional
interventions included in the study.
Demographic Characteristics
The characteristics of the sample were based on the data obtained from the
student demographic portion of the Pre- Critical Thinking Survey. This portion of the
survey included questions concerning gender, race, age, current year in college, academic
major, student status, students’ perceptions of their writing and comprehension skills, and
previous academic experience in critical thinking. Tables 1 and 2 provide information
concerning student demographic characteristics.
73
Table 1. Sample Demographic Characteristics
Category n
Gender Male 8 Female 7
Race Black 1 Indian/Alaskan 1 Asian/Pacific Islander 0 Hispanic 2 White/non-Hispanic 11
Age 17–19 10 20–22 3 23–25 1 26–30 1 Over 30 0
Year in college Freshman 2 Sophomore 13 Junior 0 Senior 0
Discipline major Humanities 3 Sciences 4 Mathematics 2 Education 3 Social Sciences 3
Student status Full-time, degree-seeking 13 Part-time, degree-seeking 2 Full-time, other credit 0 Part-time, other credit 0
74
Table 2. Sample Demographic: Skills and Abilities
Category N
English skills Excellent 2 Very good 4 Good 6 Fair 2 In need of improvement 1
Writing skills Excellent 0 Very good 6 Good 7 Fair 1 In need of improvement 1
Reading comprehension Excellent 0 Very good 6 Good 6 Fair 3 In need of improvement 0
Taken Critical Thinking before Yes 2 No 13
Taken Critical Thinking in Grades 9–12 that taught critical thinking explicitly
Yes 6 No 9
Taken Critical Thinking course in college that taught critical thinking explicitly
Yes 2 No 13
75
Of the 15 research participants, 8 were male and 7 were female, 11 were
White/non-Hispanic, 2 were Hispanic, 1 was Indian/Alaskan, and 1 was Black. Students’
academic majors were comprised of a variety of disciplines, including humanities,
sciences, mathematics, education, and social sciences. The majority of the students (13)
had never taken a college critical thinking course, were full-time degree-seeking
sophomores between the ages of 17 and 22, and believed their English and writing skills
and reading comprehension were good to excellent.
Results
This section has been organized around the study research questions. The
following format is used for each of the research questions: restatement of the research
question, a paragraph provided for clarification, a visual representation of the data, and an
explanation of the information provided by the visual.
Research Questions 1–3 focused on student perceptions regarding a positive
relationship between argument mapping, Paul’s Thinker’s Guides, and Socratic
questioning, and the acquisition of critical thinking skills. Research Question 4 focused
on the students’ perceptions of the changes in their knowledge, skill, dispositions, and
attitudes after direct instruction in critical thinking. These questions were addressed in the
Likert-scaled Student Perceptions of Critical Thinking portion of the Post- Critical
Thinking Survey. The null hypothesis focused on finding no difference in student critical
thinking skills before and after direct instruction in rhetorical analysis. This hypothesis
was addressed in the CCTST–2000 pretest and posttest.
76
Research Question 1: What were the students’ perceptions of critical thinking
before and after direct instruction in argument mapping?
Argument mapping is a way of laying out visually reasoning and evidence for and
against a statement or claim. Good argument maps clarify and organize thinking by
showing the logical relationships between thoughts that are expressed simply and
precisely. They help students organize and navigate around complex information, clarify
reasoning, and communicate reasoning effectively while supporting critical thinking. The
independent variable used to address this question was the instruction and
implementation of argument mapping. Student responses on the Pre- and Post- Critical
Thinking Surveys (Appendixes A & B) served as dependent variables. Students were
asked to circle the appropriate number for each item in response to the overall
effectiveness of argument mapping with 1 being the low score and 5 being the high score.
A score of 1 or 2 indicated the intervention was ineffective; 3 indicated effective, and 4
or 5 indicated highly effective. Table 3 provides student responses concerning students’
perceptions of critical thinking after direct instruction in argument mapping.
There were 10 questions in this portion of the survey. Analysis using this Likert-
type scale revealed that 9 (60%) of the 15 students in the course designated argument
mapping as an effective to highly effective strategy for identifying simple and complex
arguments; the author’s central thesis; the rhetorical function of each paragraph; claims,
objections, counterarguments, and inferences made by the author; the structure of the
reasoning process; and the missing layers in an argument. Six (40%) of the students had
mixed feelings, 3 students indicated the intervention was ineffective in helping them
identify a simple argument, 2 a complex argument, 3 the author’s central thesis, 3 the
77
rhetorical function of each paragraph, 1 the claims made by the author, 2 the objections
made by the author, 2 the counterarguments, 3 the inferences made by the author, 2 the
structure of the reasoning process, and 4 the missing layers in an argument.
Table 3. Student Perceptions of Critical Thinking: Argument Mapping
To what extent does argument mapping help you identify . . .
Student responses
Low score
High score
1 2 3 4 5
A simple argument 1 2 5 5 2
A complex argument 2 4 7 2
The author’s central thesis 3 3 6 3
The rhetorical/argumentative function of each paragraph
3 7 5
The claims made by the author 1 5 6 3
The objections made by the author 2 4 7 2
The counter arguments 2 5 7 1
The inferences made by the author 1 2 3 5 4
The structure of the reasoning process 2 2 10 1
The missing layers in an argument 1 3 5 4 2
78
Research Question 2: What were the students’ perceptions of critical thinking
before and after direct instruction in Paul’s Thinker’s Guides?
Paul’s model of critical thinking is a metacognitive approach that consists of four
domains of critical thinking: elements of thought, abilities, affective traits, and
intellectual standards. Paul and Elder took the essential elements of good critical thinkers
and developed The Thinker’s Guide to the Art of Socratic Questioning: Based on Critical
Thinking Concepts & Tools (Paul & Elder, 2006) and The Thinker’s Guide to Analytic
Thinking: How to Take Thinking Apart and What to Look for When You Do (Elder &
Paul, 2007). The researcher used both of these Thinker’s Guides in the study. The
independent variable used to address this questions was the instruction and
implementation of Paul’s Thinker’s Guides, which was the second intervention used in
this study. Student responses on the Post- Critical Thinking Survey (Appendix B) served
as the dependent variable. Table 4 provides students’ responses concerning their
perceptions of critical thinking after direct instruction in Paul’s Thinker’s Guides.
Analysis using this Likert-type scale indicated that 6 (40%) of the students
believed Paul’s Thinker’s Guides had been effective to highly effective in helping them
identify the author’s purpose, questions or problems the author had, point of view,
assumptions, inferences, bias or narrowness, contradictions in point of view, author’s
evidence and conclusions, and the elements of thought and how they assist in critical
thinking skills. Nine (60%) of the students offered mixed responses, indicating the
Thinker’s Guides had been ineffective in helping them identify the point of view,
unstated assumptions, inferences, bias or narrowness, and contradictions in point of view.
79
Table 4. Student Perceptions of Critical Thinking: Paul’s Thinker’s Guides
To what extent does Paul’s Thinker’s Guides help you identify . . .
Student responses
Low score
High score
1 2 3 4 5
Author’s purpose 2 5 8
Author’s questions or problem 2 6 7
Point of view 1 2 4 8
Unstated assumptions 1 6 3 5
Inferences 2 4 6 3
Bias or narrowness 2 2 9 2
Contradictions in point of view 3 9 3
Author’s evidence 5 5 5
Author’s conclusions 2 6 7
Elements of thought and how they assist in critical thinking abilities
11 4
Research Question 3: What were the students’ perceptions of critical thinking
before and after direct instruction in Socratic questioning?
Socratic questioning is a highly disciplined, systematic approach of inquiry that
aids students in developing and evaluating their ideas. The Socratic method provides
opportunities for instructors to explore student thinking and model the necessary moves
80
for Socratic questioning as they question, analyze, and speculate out loud. The instruction
and implementation of Socratic questioning served as the independent variable. Student
responses on the Post- Critical Thinking Survey (Appendix B) served as the dependent
variable. Table 5 provides student responses to this question.
Table 5. Student Perceptions of Critical Thinking: Socratic Questioning
To what extent does Socratic questioning help you identify . . .
Student responses
Low score
High score
1 2 3 4 5
Goals and purposes of the writer 3 9 3
Nature of the question 3 9 3
Problem being addressed 1 11 3
Data and information to support 3 11 1
Alternative interpretations of data and information
1 3 8 3
Key concepts and ideas 5 6 4
Assumptions being made 1 2 7 5
Implications and consequences of what is being said
1 11 3
Alternative points of view 2 6 7
Effectiveness of Socratic questioning in preparation for discussion, writing, and for developing your arguments
3 7 5
81
Analysis using this Likert-type scale indicated that 12 (80%) of the students
indicated Socratic questioning was effective to highly effective in helping them identify
the goals and purposes of the writer, the problem being addressed, relevant data and
information to support, key concepts and ideas, alternative points of view, and preparing
them for discussion, writing, and developing arguments. Three (20%) of the students
offered mixed responses, suggesting that Socratic questioning was ineffective in helping
them identify alternative interpretations of data and information, assumptions being
made, and implications and consequences of what is being said.
Students were also asked to rate which of the instructional interventions extended
their knowledge of critical thinking. The rating criteria consisted of 1 = true, 2 = false,
and 3 = I don’t know. Eight (53%) of the students believed argument mapping had
extended their knowledge of critical thinking, 13 (86%) indicated the Thinker’s Guides
had extended their knowledge of critical thinking, and all 15 (100%) of the students
believed Socratic questioning had extended their knowledge of critical thinking.
The final question on the Post- Critical Thinking Survey (Appendix B) offered
additional insights into students’ perceptions of the instructional interventions presented
in the course. Students were asked to identify which of the three strategies taught in the
critical thinking course had been most effective in helping them become better thinkers.
Responses to this question revealed the students preferred Paul’s Thinker’s Guides or
Socratic questioning and that argument mapping had not been effective in helping them
become better thinkers. Nine (60%) of the students designated the Thinker’s Guides as
being most effective in helping them become better thinkers and 6 (40%) reported
Socratic questioning had been most effective in helping them become better thinkers.
82
Research Question 4: What did the students report about the changes in their
knowledge, skills, and dispositions relating to critical thinking after the direct instruction
in rhetorical analysis?
Student responses to the Knowledge, Skills, and Dispositions portions of the Pre-
and Post- Critical Thinking Surveys (Appendixes A & B) served as dependent variables.
Students were asked to rate each of the questions for how it described them using a 3-
point scale in which 1 = true, 2 = false, 3 = I don’t know.
Table 6 presents student responses to the Knowledge portion of the Pre- and Post-
Critical Thinking Surveys.
Data from the Knowledge portion of the Pre- and Post- Critical Thinking Surveys
revealed an increase in 11 of the critical thinking elements addressed in the critical
thinking skills questions. There were 6-point gains in students’ understanding of
synthesis as a higher level thinking skill and of fallacious reasoning; 4-point gains were
achieved in students’ ability to clearly define critical thinking and evaluation and in their
perceptions of strong critical thinkers; students demonstrated 3-point gains in their
definitions of comprehension and analysis as they relate to Bloom’s taxonomy; and there
were 1-point gains in students’ understanding of evaluation, bias, inference, and premise.
83
Table 6. Responses to Critical Thinking Survey: Knowledge
Question
Correct responses
Gain Pretest Posttest
Comprehend synthesis as it relates to Bloom’s taxonomy 5 11 6
Recognize the elements of critical thinking 10 14 4
Understand comprehension as it relates to Bloom’s taxonomy
11 14 3
Understand the connection between evaluation and making judgments based on analysis
12 13 1
Understand the relationship between synthesis and creative thinking
10 10 0
Comprehend the qualities of strong critical thinkers 4 8 4
Distinguish analysis as it relates to Bloom’s taxonomy 2 5 3
Identify the essential elements of an argument 14 14 0
Define bias 12 13 1
Define conclusion 14 13 –1
Define evaluation 9 13 4
Define evidence 14 14 0
Define fallacious reasoning 6 12 6
Define inference 8 9 1
Define premise 10 11 1
84
Table 7 presents student responses to the Skills portion of the Pre- and Post-
Critical Thinking Surveys.
Table 7. Responses to Critical Thinking Survey: Skills
Question
Student responses
Gain
Pretest Posttest
True
False
Don’t know
True
False
Don’t know
Recognize connotation & denotation
9 4 2 14 1 5
Recognize author’s purpose 10 3 2 15 5
Consider points of view 14 1 15 1
Evaluate information for relevance
8 4 3 14 1 5
Evaluate source credibility 9 4 2 9 4 2 1
Comprehend point of view 8 4 3 13 2 5
Difficulty recognizing difference between assumptions & facts
2 12 1 3 12 0
Avoid generalizations 10 4 1 11 2 2 1
Analyze arguments, interpretations, & beliefs
12 2 1 14 1 2
Difficulty comparing perspectives & considering opposing points of view
2 12 1 2 11 2 –1
85
Table 7. Responses to Critical Thinking Survey: Skills (continued)
Question
Student responses
Gain
Pretest
Posttest
True
False
Don’t know
True
False
Don’t know
Offer relevant evidence 8 5 2 11 3 1 3
Transfer insights learned in classroom to real life
13 1 1 12 2 1 1
Difficulty recognizing contradictions
6 8 1 8 7 –1
Fair-minded 12 1 2 12 1 2 0
As indicated in Table 7, for the Skills portion of both Pre- and Post- Critical
Thinking Surveys, students illustrated an understanding of the importance of considering
opposing points of view, with scores of 14 and 15, respectively, out of 15 possible points.
The highest gains, 5 points, occurred in recognizing the connotative and denotative
meaning of words, recognizing the author’s purpose, evaluating information for its
relevance, and accurately comprehending one’s point of view. A 3-point gain occurred in
students’ acknowledgment of the necessity to offer relevant, constructive evidence; and
there was a 2-point increase in students grasping the importance of analyzing and
evaluating arguments, interpretations, and beliefs. Further, there were 1-point gains in
students understanding the importance of considering alternative points of view,
86
acknowledging the merit of evaluating the credibility of sources of information, and
avoiding generalizations and oversimplifications.
Table 8 presents student responses to the Dispositions portion of the Pre-and Post-
Critical Thinking Surveys.
Table 8. Responses to Critical Thinking Survey: Dispositions
Question
Student responses
Pretest Posttest
True
False
Don’t know
True
False
Don’t know
Consider myself to be thoughtful 15 15
Think of ways to improve thinking 9 5 1 13 2
Prefer complex to simple problems 7 7 1 10 4 1
Enjoy situations that requires a lot of thinking
6 7 2 10 5
Prefer tasks which challenge my thinking abilities
8 5 1 11 3 1
Avoid situations which require in-depth thinking
4 11 4 8 3
Enjoy long deliberation 1 12 2 4 5 6
Prefer thinking about small, daily projects
4 9 2 4 10 1
Prefer projects which require very little thought
4 10 1 2 13
87
Table 8. Responses to Critical Thinking Survey: Dispositions (continued)
Question
Student responses
Pretest Posttest
True
False
Don’t know
True
False
Don’t know
Relying on thought to make my way to the top is appealing
11 3 1 12 2
Enjoy discovering solutions to problems
12 1 2 13 1 1
Do not enjoy learning new ways to think
1 14 14 1
The notion of thinking abstractly is frightening
2 11 2 1 14
Prefer tasks that are intellectual 11 3 1 13 1 1
Frequently deliberate about issues 9 4 2 12 1 2
On the pretest and posttest surveys, all of the research participants (N = 15)
indicated they considered themselves to be thoughtful, and 14 of the 15 students on both
surveys reported they enjoyed learning new ways to think. There were 4-point gains in
students’ willingness to discover ways to improve their thinking and to handle situations
that require a lot of thinking. Data from the survey reflected 3-point gains in students’
preference for tasks that challenge their thinking abilities, for attempting projects that
require considerable thinking, for abstract thinking, and for deliberating about issues even
when those issues do not affect them personally. Further, there were 2-point gains in
88
students’ preference for tasks that are intellectually difficult, for solving complex
problems, and for tasks that challenge their thinking abilities. Finally, the data revealed 1-
point gains in student preference for thinking about long-term projects, relying on
thought to make their way to the top, and discovering new solutions to problems.
Table 9 provides the total scores for student responses to the Knowledge, Skills,
and Dispositions portions of the Pre- Critical Thinking Survey.
Table 9. Results of Pre- Critical Thinking Survey
Survey Total score M
Knowledge 141 9.4
Skills 156 10.4
Dispositions 145 9.6
The total sum of all scores for the Knowledge portion of the Pre -Critical
Thinking Survey for the 15 students was 141 points (62%) out of a possible 225 points.
The measure of central tendency, or mean score of 9.4 was calculated by dividing the
sum of all scores by the number of participants (N = 15). The total sum of all scores for
the Skills portion of the Pre- Critical Thinking Survey was 156, with an average or mean
score of 10.4. Finally, the total sum of all scores for the Dispositions portion of the Pre-
Critical Thinking Survey was 145, with a mean score of 9.6.
Table 10 presents the total scores of student responses to the Knowledge, Skills,
and Dispositions portions of the Post- Critical Thinking Survey.
89
Table 10. Results of Post- Critical Thinking Survey
Survey Total score M
Knowledge 174 11.6
Skills 184 12.2
Dispositions 171 11.4
The total sum of all scores for the 15 research participants on the Knowledge
portion of the Post- Critical Thinking Survey was 174 (77%). The average or mean score
was 11.6. The total sum of all scores for the Skills portion of the Post- Critical Thinking
Survey was 182, with an average or mean score of 12.3. The total sum of all scores for
the Dispositions portion of the Post- Critical Thinking survey was 171, yielding a mean
score of 11.4.
Table 11 provides a display of the total raw scores and gains for the Pre- and Post-
Critical Thinking Surveys.
Table 11 illustrates a 33-point (2.2 mean) gain in student scores in the Knowledge
portion of the Post- Critical Thinking Survey. Data obtained from the Skills portion of the
Post- Critical Thinking Survey conveyed a 26-point (1.8 mean) increase in student scores,
and data acquired from the Dispositions portion of the Post- Critical Thinking Survey
denoted a 26-point (1.8 mean) increase in student scores.
90
Table 11. Pre- and Post- Critical Thinking Surveys: Gains
Survey
Pretest Posttest
Total score gain
Mean gain
Total score
M
Total score
M
Knowledge 141 9.4 174 11.6 33 2.2
Skills 156 10.4 184 12.2 26 1.8
Dispositions 145 9.6 171 11.4 26 1.8
On the Post- Critical Thinking Survey, students were asked to rate the
interventions used in the research project using a 3-point scale. Fourteen of the 15
students (93%) indicated their knowledge, skills, and dispositions had improved through
direct instruction in critical thinking.
Null Hypothesis: There will be no difference in scores on the pretest and posttest
on the California Critical Thinking Skills Test–Form 2000 indicating changes in student
critical thinking skills before and after direct instruction in rhetorical analysis.
The independent variable used to address this question was direct instruction in
rhetorical analysis using the treatment interventions, which included argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning. The dependent variable was the
CCTST–2000 posttest total ranked score.
The CCTST–2000 was ordered from Insight Assessment and was administered by
the researcher as a pretest and posttest. The CCTST–2000 was based on the APA Delphi
91
consensus conceptualization of critical thinking. The CCTST–2000 targets the core
critical thinking skills of analysis, interpretation, inference, evaluation, and explanation.
At the end of the semester, the batch of CapScore CCTST answer forms was
returned to Insight Assessment. Insight Assessment then scanned and scored those
answer forms and returned a digital file reporting total and scale CCTST scores for each
test taker by PIN and descriptive statistics for the group of test takers as a whole.
The five subscales reported by Insight Assessment for the CCTST are analysis,
evaluation, inference, deductive reasoning, and inductive reasoning. A total score for
overall critical thinking was provided by Insight Assessment.
The CCTST–2000 consists of 34 multiple-choice questions. Each question is
worth 1 point, for a total raw score of 34. The test takes 45 minutes to complete under
timed conditions; however, for the purposes of this study, the test was administered
untimed using a paper-and-pencil format with a Scantron sheet.
Insight Assessment provided an Excel file and a Word file. The Excel file
included the scored pretest and posttest data for each test taker by PIN. The Word file
included descriptive statistics as well as additional t-test analyses. The basic descriptive
statistics provided as part of the CapScore scoring service were mean, median, trimmed
mean (the mean of the middle 90% of the data, used to remove outlier effect), standard
deviation, standard error of the mean, minimum, maximum, and first and third quartiles.
The median is the second quartile. This split the data into four groups, with 25% of the
data in each group.
The pretest scores for the CCTST–2000 are provided in Table 12. Note the total
mean score is 14.9, with a deviation of responses of 4.55.
92
Table 12. CCTST–2000 Pretest Scores
Variable N M Mdn Trimmed mean SD
Induction 15 9.000 10.000 9.154 2.699
Deduction 15 5.867 6.000 5.769 2.642
Analysis 15 3.800 4.000 3.846 1.373
Inference 15 6.600 6.000 6.615 2.720
Evaluation 15 4.467 4.000 4.462 1.885
Total1 15 14.870 14.000 14.920 4.550
Table 13 displays the statistical analysis of the posttest scores. Note the total mean
score is 16.1, with a standard deviation of 6.35. As the total mean score for the CCTST–
2000 pretest was 14.9, these data suggested a 2.2 mean gain in student scores.
Table 13. CCTST–2000 Posttest Scores
Variable N M Mdn Trimmed mean SD
Induction 15 9.000 9.000 8.923 2.928
Deduction 15 7.067 7.000 6.923 3.674
Analysis 15 3.800 4.000 3.769 1.699
Inference 15 7.667 7.000 7.538 3.109
Evaluation 15 4.600 4.000 4.385 2.230
Total2 15 16.070 15.000 15.850 6.350
93
A paired samples t test and confidence interval (CI) were calculated and analyzed
by Insight Assessment for the CCTST–2000 pretest and posttest composite and subscale
scores to determine whether the difference in mean scores was statistically significant.
The CI was calculated to determine if all of the values within the range were defined by
the confidence limits of a sample statistic. The confidence limits “define the upper and
lower value of a range of values for a sample statistic that is likely to contain a population
parameter” (Gall et al., 2003, p. 638). The results of the t-test analyses are presented in
Tables 14, 15, and 16.
Table 14. T-Test Analyses: CCTST–2000 Pretest and Posttest Totals
Variable
N
M
SD
SE mean
95% CI for mean difference
t value
p value
Upper Lower
Total2 15 16.07 6.35 1.640 –0.788 3.188 1.29 0.216
Total1 15 14.87 4.55 1.170
Difference 15 1.20 3.59 0.927
There did not appear to be a statistical significance between the pretest and
posttest in any of the five categories at the 5% significance level; however, a paired t test
revealed a p value of 0.123 for the difference in values from the pre- to the posttest for
the variable deduction and a p value of 0.112 for the variable inference.
94
Table 15. T-Test Analyses: Analysis, Inference, and Evaluation
Variable
N
M
SD
SE mean
95% CI for mean difference
t value
p value
Upper Lower
Analysis
Analysis2 15 3.800 1.699 0.439 –0.936 0.936 0.00 1.000 Analysis1 15 3.800 1.373 0.355 Difference 15 0.000 1.690 0.436
Inference
Inference2 15 7.667 3.109 0.803 –0.281 2.415 1.70 0.112 Inference1 15 6.600 2.720 0.702 Difference 15 1.067 2.434 0.628
Evaluation
Evaluation2 15 4.600 2.230 0.576 –0.868 1.134 0.29 0.779 Evaluation1 15 4.467 1.885 0.487 Difference 15 0.133 1.807 0.467
This indicated the researcher can be approximately 88% confident there is a
difference in deduction scores from the pretest to the posttest scores and approximately
89% confident there is a difference in inference scores from the pretest to posttest scores.
There were no significant differences between the pretest and posttest scores for
induction, analysis, and evaluation. Although Figure 1 illustrates a slight difference
between the pretest and posttest scores for inference and deduction, the significance level
was negligible (p > 0.2).
95
Table 16. T-Test Analysis: Induction and Deduction
Variable
N
M
SD
SE mean
95% CI for mean difference
t value
p value
Upper Lower
Induction
Induction2 15 9.000 2.928 0.756 –1.256 1.256 0.00 1.000 Induction1 15 9.000 2.699 0.697 Difference 15 0.000 2.268 0.586
Deduction
Deduction2 15 7.067 3.674 0.949 –0.369 2.769 1.64 0.123 Deduction1 15 5.067 2.642 0.682 Difference 15 1.200 2.833 0.732
Figure 1. Comparison of mean scores by question type.
96
In this study, the null hypothesis was accepted. No statistical evidence was found
to suggest changes in student critical thinking skills; however, there was a difference
between the pretest and posttest scores for the variables deduction and inference.
Summary
This chapter provided a description of the research sample along with a discussion
of student demographics. This was followed by a summary of data analysis and
procedures and an overview of the data and results as they relate to the four research
questions. Research Questions 1–3 were concerned with students’ perceptions of critical
thinking before and after direct instruction in argument mapping, Paul’s Thinker’s
Guides, and Socratic questioning. Student responses to the survey questions indicated
that although students found all three of the instructional strategies to be effective, the
majority of the students preferred Socratic questioning and Paul’s Thinker’s Guides as
highly effective interventions for developing their knowledge of critical thinking and for
helping them become better thinkers.
The fourth question was concerned with students’ perceptions of the changes in
critical thinking knowledge, skills, and dispositions after direct instruction in rhetorical
analysis. For this question, the researcher examined the Knowledge, Skills, and
Dispositions portions of the Pre- and Post- Critical Thinking Surveys. Student responses
on the Knowledge portion of the Post- Critical Thinking Survey revealed a 2.2 increase in
student scores, the Skills section suggested a 1.8 increase in student scores, and the
average increase in student scores on the Dispositions portion of the survey was 1.8.
97
Data from the CCTST was used to address the null hypothesis: There will be no
difference in scores on the pretest and posttest on the California Critical Thinking Skills
Test–Form 2000 indicating changes in student critical thinking skills before and after
direct instruction in rhetorical analysis. Results from the CCTST–2000 revealed that none
of the paired t tests was significant at the 0.05 level of significance; however, a paired t
test revealed a p value of 0.123 for the difference in values from the pre- to the posttest
for the variable deduction and a p value of 0.112 for the variable inference. There were
no significant differences between the pretest and posttest scores for induction, analysis,
and evaluation. Although there was a slight difference between the pretest and posttest
scores for inference and deduction, the significance level was negligible (p > 0.2).
98
CHAPTER 5. RESULTS, CONCLUSIONS, AND RECOMMENDATIONS
Introduction
The purpose of this study was to determine whether there would be improvement
in students’ critical thinking abilities, including knowledge, skills, and dispositions, after
the intervention of direct instruction in rhetorical analysis. The researcher investigated
student perceptions of the effectiveness of the following instructional strategies:
argument mapping; Thinker’s Guides, based on Paul’s model of critical thinking; and
Socratic questioning. The goal of the study was, on the one hand to promote, critical
thinking knowledge, skills, and dispositions and, on the other hand, to determine
improvement in student perceptions of critical thinking knowledge, skills, and
dispositions so they may transfer those skills across the academic disciplines and into
their personal and professional lives.
This chapter presents a summary of the findings, discussion, and important
conclusions that were drawn from the data presented in chapter 4. Further, it provides
suggestions for further research, recommendations for practice, and connections to what
is known from the review of literature.
99
Overview of Methodology
The mixed methodology approach was used in this action research study. The
study pertained to one semester of a college critical thinking course that was taught in a
naturalistic setting by the researcher. The overall goal of the study was to examine the
effect of direct instruction in rhetorical analysis in students’ critical thinking knowledge,
skills, and dispositions. Twenty-nine students were initially enrolled in the Critical
Thinking 104 course. Of the 29 students, 19 agreed to participate in the study. Three of
those students dropped the course and one student was not present the final day of the
course and did not take the posttest; thus, the final research sample was comprised of 15
(N = 15) of the 29 students enrolled in the critical thinking course.
The triangulation or mixed-methodology approach was used in this one-group
pretest–posttest design to determine whether direct instruction in critical thinking, using
the aforementioned interventions, would improve students’ critical thinking knowledge,
skills, and dispositions. The independent variable for this study was the critical thinking
interventions. The dependent variables were the pre- and posttest scores on the CCTST–
2000 and Pre- and Post- Critical Thinking Surveys. The researcher examined the
relationship between the variables involving teacher instruction of the interventions, the
students’ mastery of those skills, and the students’ perceptions of the effectiveness of the
interventions.
Summary of Findings
Research Question 1 addressed students’ perceptions of critical thinking before
and after direct instruction in argument mapping. There were 10 questions in this portion
100
of the Post- Critical Thinking Survey (Appendix B). Analysis using a Likert-type scale
revealed that 9 (60%) of the 15 students in the course designated argument mapping as an
effective to highly effective strategy for identifying simple and complex arguments; the
author’s central thesis; the rhetorical function of each paragraph; claims, objections,
counterarguments, and inferences made by the author; the structure of the reasoning
process; and the missing layers in an argument. Six (40%) of the students had mixed
responses; students indicated that the intervention was ineffective in helping them
identify (a) simple and complex arguments; (b) the author’s central thesis and the
rhetorical function of each paragraph; (c) the claims, objections, and inferences made by
the author; (d) the counterarguments; (e) the structure of the reasoning process, and (f)
the missing layers in an argument.
In an effort to improve argument analysis skills, students were explicitly taught
how to construct argument maps. Repeated individual and collaborative practice
encouraged students to evaluate and modify the premise, structure, support, and logic of
their own written arguments; as well, it offered them practical tools for evaluating the
validity of arguments presented in the text and in a variety of editorials and media
images. Once students became familiar with and grew accustomed to practicing this
instructional intervention, their responses to the assigned readings and analysis of
required essays revealed some improvement in argument analysis skills; however,
argument mapping was the first intervention presented by the teacher/researcher during
the first 5 weeks of an 18-week college course. At this point in time, students were
gradually becoming accustomed to close prose rhetorical analysis. Van Gelder (2005)
acknowledged “representing arguments in diagrams tends to be slow and cumbersome”
101
(p. 5), and student responses to argument mapping indicated they found this process to
be, at times, somewhat tiresome. Although the majority of the students believed argument
mapping contributed to their perceptions of critical thinking, students may not have been
given a sufficient amount of time to practice and develop the skills necessary for
identifying simple and complex arguments, the author’s thesis, the rhetorical functions of
each paragraph, objections, counterarguments, inferences made by the author, and the
structure of the reasoning process.
As with this current study, Bessick’s (2008) study examined the effect of direct
instruction in critical thinking using van Gelder’s rational argument mapping and Paul’s
Thinker’s Guides. Her study consisted of freshman students who took repeat courses and
who received individual tutoring throughout the semester. Bessick’s study revealed no
significant effect of the intervention group on subjects’ critical thinking skills as
measured by the CCTST–2000, yet students demonstrated improvement in academic
achievement, which may have been a result of the individual tutoring students received
throughout the semester and the fact they were taking repeated courses. Although Bessick
believed direct instruction in critical thinking and reasoning can improve student learning
outcomes, she also acknowledged the ultimate educational goal should be to enable
students to transfer those skills learned in the classroom to their everyday lives.
Similarly, van Gelder (2005) acknowledged individuals are not naturally inclined
toward critical thinking; nonetheless, students’ critical thinking can improve through
guided instruction, which exposes students to good critical thinking and engages them in
the critical thinking process. Argument mapping offers students the essential tools for
recognizing the logical structure of rhetoric; however, the students had a limited amount
102
of time to practice and develop this skill as each intervention was facilitated at 5-week
intervals. Student improvement in and appreciation for argument mapping occurs as a
result of effective instruction, repeated modeling of the procedures involved, and practice
over an extended period of time.
Research Question 2 targeted students’ perceptions of critical thinking before and
after direct instruction in Paul’s Thinker’s Guides. The independent variable used to
address this question was the instruction and implementation of Paul’s Thinker’s Guides,
which was the second intervention used in this study. Student responses on the Post-
Critical Thinking Survey served as the dependent variable. Analysis using a Likert-type
scale indicated that 6 (40%) of the students believed Paul’s Thinker’s Guides had been
effective to highly effective in helping them identify the author’s purpose, questions or
problems the author had, point of view, assumptions, inferences, bias or narrowness,
contradictions in point of view, the author’s evidence and conclusions, and the elements
of thought and how they assist in critical thinking skills. Nine (60%) of the students
offered mixed responses, indicating the Thinker’s Guides had been ineffective in helping
them identify the point of view, unstated assumptions and inferences, bias or narrowness,
and contradictions in point of view.
Students appeared to enjoy the application of Paul’s Thinker’s Guides as the
guides offer a comprehensible set of guidelines for analysis of textual as well as visual
arguments. The teacher/researcher taught Paul’s model explicitly, provided handouts, and
guided the students through frequent modeling of the process. Student written responses
and in-class discussion of the required readings suggested that the use of Paul’s elements
of reasoning provided a more thorough understanding of the rhetorical devices used by
103
authors; nevertheless, students appeared to have the most difficulty with identifying
unstated assumptions and inferences, bias, and contradictions in point of view and,
initially, found Paul’s model to be difficult and even frustrating. For all of the required
essays for the course, students were guided through a variety of activities to assist them in
creating final drafts. The teacher/researcher engaged the students in prewriting activities;
students then used their prewriting as a guide as they developed a rough draft of their
essays. This was followed by peer-editing sessions in which students examined the drafts
of their peers using Paul’s elements of reasoning. Teacher/researcher observations of the
peer-editing sessions and of student final drafts suggested this intervention assisted
students in offering a more systematic analysis and constructive criticism of their peers’
essays while increasing the thoroughness of assessment, evaluation, and modification of
their own written essays.
Correspondingly, Reed’s (1998) one-semester study investigated the effect of
integrating Paul’s model of critical thinking into a U.S. history course on community
college students’ ability to think critically about U.S. history and everyday issues,
dispositions toward thinking critically, and knowledge of history content. Unlike the
current study, Reed’s study consisted of four sections: two sections were experimental
groups and two sections were control groups. Students participating in the study took
three pretests and four posttests to determine the effectiveness of Paul’s model. Students
in both the control and experimental groups took the Ennis–Weir Critical Thinking Essay
Test as a pretest and posttest. Unlike the current study, results from Reed’s study revealed
that “posttest means increased by 3.28 points in the experimental group” (p. 143). One
possible reason could be this researcher’s small sample size of (N = 15) compared to
104
Reed’s size of (N = 52). Another significant contributing factor for Reed’s success could
have been that the instructor for Reed’s study participated in intensive training in Paul’s
model, while this researcher had limited preparation, which consisted of 1 week of
training, for implementing this intervention. Regardless, Reed’s study revealed that
students’ ability to think critically can improve in a single course when provided with
deliberate instruction and that Paul’s model “can be successfully integrated into an
introductory history course with statistically significant benefits to students’ abilities to
think critically within a domain and to their general critical thinking abilities” (p. 160).
Paul’s model of critical thinking and the Thinker’s Guides are invaluable tools for
facilitating and encouraging substantive critical thinking skills and for providing a
common model for instructional design, which can contribute to higher level learning.
Nevertheless, it is vital that students possess the “values, traits of mind, and dispositions
that underlie those skills” (Paul, 1995f, p. 119). As assessment should be specific,
consistent, and appropriate, it is necessary for students to possess the intrinsic motivation
and dispositions that contribute to the changes in perspective needed for substantial
learning. As the semester progressed, students’ performance on essays and homework
assignments reflected an increase in their ability to think critically and analytically and to
analyze arguments effectively. As with argument mapping, Paul’s Thinker’s Guides
demand continued practice over a period of time.
Student essays and responses to homework assignments revealed a marked
improvement in evaluation, analysis, and synthesis. Teacher/researcher observations of
class discussion and of students’ written homework assignments and essays revealed that
students’ ability to analyze the logic of articles, essays, and their own written work
105
improved. Concurrently, the students who illustrated this growth in analytical skills were
also those who appeared to be intrinsically motivated and who approached the course
work and the in-class discussions and collaboration with vigor and enthusiasm.
Research Question 3 was concerned with students’ perceptions of critical thinking
before and after direct instruction in Socratic questioning. The instruction and
implementation of Socratic questioning served as the independent variable. Student
responses on the Post- Critical Thinking Survey served as the dependent variable.
Analysis using a Likert-type scale indicated that 12 (80%) of the students revealed
Socratic questioning was effective to highly effective in helping them identify the goals
and purposes of the writer, the problem being addressed, relevant data and information to
support, key concepts and ideas, alternative points of view, and preparing them for
discussion, writing, and developing arguments. Three (20%) of the students offered
mixed responses, suggesting that Socratic questioning was ineffective in helping them
identify alternative interpretations of data and information, assumptions being made, and
implications and consequences of what is being said.
There are three kinds of Socratic questioning that can be utilized by educators to
probe student thinking: spontaneous, exploratory, and focused. Spontaneous questioning
provides opportunities for teachers to explore with students “how [they] might find out if
something is true, logical, or reasonable” (Paul & Elder, 2006b, p. 48). This type of
questioning provides opportunities for educators to “listen critically” (Paul & Elder,
2006b, p. 48) and for students to “become self-correcting” (Paul & Elder, 2006b, p. 48).
Spontaneous Socratic questioning moves include asking for examples, evidence, or
reasons; asking a group whether they agree, suggesting parallel examples; and providing
106
an analogy, asking for a paraphrase, or rephrasing a student response (Paul & Elder,
2006b).
Exploratory Socratic questioning is used by teachers to explore student thinking
on diverse issues. It can be used to explore bias or student values, to identify clarity of
thought, or to probe controversial issues (Paul & Elder, 2006b). This type of questioning
can be used for introducing a subject to the class and for analyzing a topic. An issue can
be raised for discussion and students could be required to write about the issue or to form
groups to discuss the issue. As with spontaneous Socratic questioning, exploratory
questioning requires minimal preplanning, although teachers might consider preparing
appropriate questions that could be used when discussing a topic or an issue.
Focused Socratic questioning occurs when instructors wish to explore and discuss
specific topics or issues. It includes having students engage in a discussion that
encourages them to “clarify, sort, analyze and evaluate thoughts and perspectives,
distinguish the known from the unknown [and] synthesize relevant factors and
knowledge” (Paul & Elder, 2006b, p. 50). Focused questioning requires preplanning,
which could include reflecting on a variety of perspectives, conclusions, concepts,
implications, and consequences, and constructing a set of questions for the Socratic
dialogue to allow students to think through a specific concept.
There are also a variety of patterns in which Socratic dialogue can be facilitated in
the classroom. To encourage reasoning and discussion, Socratic questioning can begin
with whole-class discussion, which then leads to small-group speaking and listening,
whole-class Socratic speaking and listening, individual writing, and small-group speaking
and listening. An alternative to this would be a schema that begins with whole-class
107
Socratic discussion to clarify students’ understanding of the question; this would be
followed by small-group research, individual writing, and then large-group Socratic
evaluation of the research. A third pattern would begin with whole-class Socratic
discussion with an introduction of the lesson and concepts involved, whole-class listening
and observing as the teacher models the task, followed by individual or group practice
(Paul & Elder, 2006b).
Socratic questioning in the critical thinking course included a combination of
Spontaneous questioning and exploratory questioning. Spontaneous questioning occurred
as the teacher/researcher discussed the concepts and ideas taken from the required
reading assignments, which generally consisted of a variety of arguments found in the
text, on the Internet, or in the news media. The researcher would introduce the key
concepts or questions from the readings to stimulate student thinking. In order to allow
students to reflect on the concepts or issues, students were given 5–10 minutes to free-
write a response. This was followed by class discussion of the issue as the
teacher/researcher asked for clarification, examples, and evidence, while offering
examples, providing analogies, asking for a paraphrase, or rephrasing student responses.
Exploratory Socratic questioning was used when whole group discussion was
used to introduce key questions or concepts to the class and to stimulate thinking.
Students were often given a prompt that dealt with a controversial issue discussed in a
previous reading assignment. They were then allowed 5–10 minutes to free-write their
response. This was often followed by small-group speaking and listening in order to
encourage students to think and reason cooperatively and to assess their ideas. For the
purpose of disciplining the questioning, students were given a script that required them to
108
focus on Paul’s elements of reasoning. The teacher/researcher assigned a questioner and a
recorder for each group. In this way, students worked with their peers in an informal
setting rather than having the teacher/researcher address focused formal Socratic
questioning techniques. Thus, students reasoned individually and cooperatively, applying
and assessing their ideas using preassigned questions to assist them in developing clarity,
accuracy, precision, relevance, depth, breadth, logic, significance, and fairness. This was
followed by class discussion for review and assessment of the small-group work and for
introducing new questions for further thought, discussion, and writing.
The collaborative process was used most frequently for the Socratic questioning
in the critical thinking course and, at times, it was highly productive; however, using the
single Socratic questioner to the whole group might have been more powerful and
engaging for the students. Further, researcher observations revealed that in order for the
collaborative process to be more effective, Socratic questioning should be consistently
modeled and monitored, and students must clearly comprehend their assigned roles in the
Socratic process; otherwise, the likelihood exists that students will either enter into a
debate or that certain students will monopolize the dialogue. Everyone should be allowed
to participate and to go beneath the surface of what is being discussed, constantly probing
the complexities of their ideas and the questions. Students must also be assured they do
not need to come to a group consensus, only that open discussion is allowed and that
students stay on topic.
In support of the merits of implementing Socratic questioning into instructional
pedagogy, Yang’s (2008) study was designed to investigate the effects of teaching critical
thinking skills, with the major goal of investigating whether students’ critical thinking
109
skills would improve after they participated in Socratic dialogues. Yang measured the
critical thinking scores using the CCTST; however, Yang achieved positive results,
indicating that instructors who use Socratic dialogues during small-group online
discussions can successfully improve students’ critical thinking skills. This improvement
in learning was accomplished “as students interact with their peers or instructor and
benefit from combining their levels of expertise, offering support, distributing the
thinking load, and confronting alternative points of view” (Yang, 2008, p. 261).
Paul (1995e) believed Socratic questioning embodies the spirit of critical thinking
as it presents opportunities for students to develop and evaluate their ideas; nevertheless,
the vital component for success with this instructional strategy is appropriate modeling of
the necessary moves for this kind of questioning. Moreover, rather than indoctrinating
learners, Newton (1978) contended that educators should stimulate learners while
encouraging them to become active participants in the learning process. Although higher
cognitive questioning is a viable strategy for teaching critical thinking, and “intelligence
is a reaction to the social and physical world” (Newton, 1978, p. 289), learners must
possess the willingness to acknowledge alternative perspectives while they adapt, refine,
and apply their newly acquired knowledge in a problem-solving manner (Newton, 1978).
Open-Ended Questions
Students were also asked to rate which of the instructional interventions extended
their knowledge of critical thinking. Eight (53%) of the students designated argument
mapping as a valuable instructional approach for contributing to their knowledge of
critical thinking, 13 (86%) of the students indicated the Thinker’s Guides had extended
110
their knowledge of critical thinking, and all 15 (100%) of the students suggested Socratic
questioning had played a significant role in furthering their knowledge of critical
thinking.
Concurrently, students were asked to identify which of the three strategies that
had been taught in the critical thinking course had been most effective in helping them
become better thinkers. Responses to this question revealed argument mapping had not
been effective in helping them become better thinkers. Nine (60%) of the students stated
Paul’s Thinker’s Guides had been most effective in helping them become better thinkers,
and 6 (40%) of the students designated Socratic questioning as being effective in helping
them become better thinkers.
As mentioned, argument mapping was the first intervention presented by the
teacher/researcher, and students were gradually becoming accustomed to the close
scrutiny required for rhetorical analysis. Once students had been exposed to Paul’s
Thinker’s Guides and Socratic questioning, they might have been more accustomed to
this process. Further, students may not have been given a sufficient amount of time to
practice and develop the skills necessary for identifying the structure and logic of an
argument and for effective argument mapping. As well, students may have preferred
Paul’s Thinker’s Guides because they appear to encourage more reflection on the
elements of thinking, on key concepts, and on the assumptions and inferences being
made, whereas argument mapping involves a more visual examination of rhetoric.
Argument maps are spatial layouts for presenting information, they rely on encouraging
the reader or writer to view the structure and the overall essence of the argument, and
they assist in conveying the essential logic of the rhetoric. Students may not have found
111
this process to be as interesting as Paul’s model, which appears to be more analytical and
to allow for the development of higher level thinking skills.
Research Question 4 was concerned with students’ perceptions of the changes in
their knowledge, skills, and dispositions relating to critical thinking after the direct
instruction in rhetorical analysis. Student responses to the Knowledge, Skills, and
Dispositions portions of the Pre- and Post- Critical Thinking Surveys (Appendixes A &
B) served as dependent variables.
Halpern (1999) contended that critical thinking skills can be taught as rhetorical
analysis, problem solving, and decision making, yet her contention is that these skills
should be taught within a variety of contexts and for transfer. However, students must not
only be taught the necessary skills needed for critical thinking, they must also have a
disposition toward and a willingness to apply them.
Summary of Pre- and Post- Critical Thinking Surveys: Knowledge, Skills, and Dispositions
Data from the Knowledge portion of the Pre- and Post- Critical Thinking Surveys
revealed an increase in 11 of the critical thinking elements addressed in the critical
thinking skills questions. Students experienced gains in the understanding of synthesis as
a higher level thinking skill and of fallacious reasoning. Gains were also realized in
students’ ability to clearly define critical thinking, to evaluate, to distinguish strong
critical thinkers, and to define comprehension and analysis as they relate to Bloom’s
taxonomy. Further, there was an increase in students’ comprehension of evaluation, bias,
inference, and premise.
112
For the Skills portion of both the Pre- and Post- Critical Thinking Surveys,
students illustrated an understanding of the importance of considering opposing points of
view, with the highest gains in recognizing the connotative and denotative meaning of
words, recognizing the author’s purpose, evaluating information for its relevance, and
accurately comprehending one’s point of view. There were also gains in students’
acknowledgment of the necessity to offer relevant, constructive evidence; grasping the
importance of analyzing and evaluating arguments and interpretations; understanding the
importance of considering alternative points of view; acknowledging the merit of
evaluating the credibility of sources of information; and avoiding generalizations and
oversimplifications.
On the pretest and posttest Dispositions portions of the surveys, all of the research
participants (N = 15) indicated they considered themselves to be thoughtful, and 14 of the
15 students on both surveys reported they enjoyed learning new ways to think. There
were also gains in students’ willingness to discover ways to improve their thinking, for
attempting tasks that challenge their thinking abilities, for abstract thinking, and for
deliberating about issues even when those issues do not affect them personally.
Additionally, there was an increase in students’ willingness to solve complex problems,
think about long-term projects, rely on thought to make their way to the top, and discover
new solutions to problems.
The total sum of all scores for the Knowledge portion of the Pre- Critical
Thinking Survey for the 15 students was 141 points (62%) out of a possible 225 points.
The measure of central tendency, or mean score of 9.4 was calculated by dividing the
sum of all scores by the number of participants (N = 15). The total sum of all scores for
113
the Skills portion of the Pre- Critical Thinking Survey was 156, with an average or mean
score of 10.4. Finally, the total sum of all scores for the Dispositions portion of the Pre-
Critical Thinking Survey was 145, with a mean score of 9.6.
The total sum of all scores for the 15 research participants on the Knowledge
portion of the Post- Critical Thinking Survey was 174 (77%). The average or mean score
was 11.6. The total sum of all scores for the Skills portion of the Post- Critical Thinking
Survey was 182, with an average or mean score of 12.3. The total sum of all scores for
the Dispositions portion of the Post- Critical Thinking survey was 171, yielding a mean
score of 11.4. These data revealed a 33-point (2.2 mean) gain in Knowledge, a 26-point
(1.8 mean) increase in Skills, and a 26-point (1.8 mean) increase in Dispositions on the
Post- Critical Thinking Survey.
Data from the Post- Critical Thinking Survey suggested instructors can make a
positive impact on students’ perceptions of their own critical thinking abilities through
direct instruction in rhetorical analysis. The highest gains were achieved in the
Knowledge portion of the Post- Critical Thinking Survey. This 2.2 mean gain in student
scores indicated an increase in students’ comprehension of synthesis and fallacious
reasoning, ability to clearly define critical thinking and evaluation, and discernment of
strong critical thinkers. Students also experienced gains in their ability to characterize
evaluation, comprehension, and analysis as they relate to Bloom’s taxonomy, and in their
understanding of evaluation, bias, inference, and premise.
The 1.8 mean gain in the Skills portion of the Post- Critical Thinking Survey
implied an increase in understanding of the importance of considering opposing
viewpoints. The highest gains, 5 points, occurred in students’ ability to distinguish the
114
connotative and denotative meaning of words, recognize the author’s purpose, evaluate
information for relevance, and accurately comprehend point of view. Further, gains were
realized in students’ ability to offer constructive evidence, recognize the importance of
analyzing and evaluating arguments, consider alternative points of view, acknowledge the
importance of evaluating the reliability of sources, and avoid generalizations.
The 1.8 mean gain in the Dispositions portion of the Post- Critical Thinking
Survey revealed an increase in students’ desire to enjoy learning new ways to think. The
highest gains were achieved in students’ willingness to discover ways to expand their
thinking and to handle situations that require extensive thinking. Additional gains were
made in students’ preference for tasks that challenge their thinking abilities for
attempting projects that require considerable thinking, for abstract thinking, and for
deliberating about issues. Finally, gains were realized in students’ preference for
intellectually difficult tasks, for solving complex problems, for deliberating about long-
term projects, for relying on thought for successful living, and for discovering new
solutions to problems.
The final question on the Post- Critical Thinking Survey asked students if they
believed their knowledge, skills, and dispositions had improved through direct instruction
in critical thinking. Fourteen of the 15 students (93%) indicated direct instruction had
been instrumental in developing their critical thinking.
Although the students’ overall scores on the CCTST–2000 were not at the level of
significance, the implementation of the research interventions—argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning—did enhance students’ knowledge of
critical thinking and their perceptions of their own critical thinking abilities. Explicit
115
teaching of the components of critical thinking provided the students with the necessary
vocabulary, tools, and awareness of their own critical thinking dispositions essential for
effective argument analysis, and it could provide them with the knowledge of how to
apply that vocabulary and knowledge to the analysis of their own writing, to the rhetoric
in written texts, and to the rhetoric with which they are confronted in the media.
The null hypothesis focused on the extent to which the pretest and posttest scores
indicate changes in the students’ knowledge, skills, and dispositions relating to critical
thinking after direct instruction in argument mapping, Paul’s Thinker’s Guides, and
Socratic questioning. For both the pretest and posttest, the CCTST–2000 was the sole
quantitative instrument used. The total score on the CCTST–2000 provided the overall
measure of students’ critical thinking skills. Changes in students’ critical thinking scores
were analyzed comparing the mean pre-CCTST–2000 score to the mean post-CCTST–
2000 score using a t test. The independent variable used to address the null hypothesis
was direct instruction in rhetorical analysis using the treatment interventions, which
included argument mapping, Paul’s Thinker’s Guides, and Socratic questioning. The
dependent variable was the CCTST–2000 posttest total ranked score.
Data from the pretest scores for the CCTST–2000 indicated the total mean score
was 14.9, with a deviation of responses of 4.55. Statistics from the CCTST–2000 posttest
scores revealed a total mean score of 16.1, with a standard deviation of 6.35. As the total
mean score for the CCTST–2000 pretest was 14.9, these data suggested a 2.2 mean gain
in student scores.
A paired samples t test and CI were calculated and analyzed by Insight
Assessment for the CCTST–2000 pretest and posttest composite and subscales scores to
116
determine whether the difference in mean scores was statistically significant. The CI was
calculated to determine if all of the values within the range were defined by the
confidence limits of a sample statistic. The resulting data suggested there does not appear
to be a statistical significance between the pretest and posttest in any of the five
categories at the 5% significance level; however, a paired t test revealed a p value of
0.123 for the difference in values from the pre- to the posttest for the variable deduction
and a p value of 0.112 for the variable inference. This indicated the researcher can be
approximately 88% confident there is a difference in deduction scores from the pretest to
the posttest scores and approximately 89% confident there is a difference in inference
scores from the pretest to posttest scores. There were no significant differences between
the pretest and posttest scores for induction, analysis, and evaluation. Although there was
a slight difference between the pretest and posttest scores for inference and deduction, the
significance level was negligible (p > 0.2).
In this study, the null hypothesis was accepted. No statistical evidence was found
to suggest changes in student critical thinking skills; however, there was a slight
difference between the pretest and posttest scores for the variables deduction and
inference. Although N. C. Facione et al. (2008) contended that “measures of CT skills
have been demonstrated to capture gains in CT over time periods of one quarter or one
semester” (p. 28), there were no significant differences in students’ pretest and posttest
scores on the CCTST–2000. Nonetheless, this research study still could provide insight
for educators who are interested in instructional pedagogy that may well promote
students’ critical thinking skills.
117
The critical thinking course could be revised to include more emphasis on
instructional strategies that target the core critical thinking skills of analysis,
interpretation, inference, evaluation, and explanation. Specifically, more time could be
spent on the analysis of the logic of a variety of arguments, on recognizing bias, and on
evaluating rhetoric for its credibility. Further, students often have a tendency to base their
decisions on preconceived notions, assumptions, and inferences, and, often, they appear
to have a difficult time accommodating for opposing points of view. Encouraging
students to be fair-minded through discussion and modeling and helping them to
recognize the logic or the fallacious reasoning in texts, in the media, and in their own
thinking and writing could promote the quality of fair-mindedness and the critical
thinking skills needed for intelligent, productive decision making. As P. A. Facione,
Facione, and Giancarlo (2000) affirmed, “To the open-minded, imaginative, and
intellectually adventurous, reflection on human experience can reveal a rich array of
possibilities” (p. 7).
Encouraging students’ critical thinking abilities and designing pedagogy that can
promote those skills is a noteworthy cause; however, teacher/researcher observations in
the critical thinking course suggested that student motivation was a contributing factor to
student success and achievement. P. A. Facione et al. (2000) argued it is not sufficient to
focus on critical thinking skill development without nurturing the “internal motivation to
use those skills in the appropriate circumstances” (p. 34). An effective tool for nurturing
students’ critical thinking dispositions, such as modeling good critical thinking behaviors,
could have been used more frequently and deliberately by the researcher in the critical
thinking course.
118
Tucker’s (2008) study at Kent School of Social Work examined the development
of graduate-level social work students’ critical thinking abilities via a critical-thinking-
infused curriculum. Tucker utilized the CCTST–2000 to measure the students’ critical
thinking performance on pre- and posttest scores. Similar to this researcher’s study,
Tucker’s pretest and posttest scores revealed no significant differences except for the
inference subscale. Students improved in their ability to “develop hypotheses, deliberate
and question relevant information from a variety of sources, consider alternatives and
potential consequences, and draw conclusions” (Tucker, 2008, p. 225). Although
Tucker’s study did not yield an overall improvement in critical thinking, as with the
current study, data analysis revealed a slight increase in deductive scores. Tucker
maintained the reason for the lack of improvement in scores was, in part, due to an
ineffective curriculum and lack of motivation on the part of teachers and faculty.
Regardless of this, she believed students’ critical thinking skills can be improved by
providing a course specifically designed for critical thinking. She further claimed that
“critical thinking skills are not the result of merely rewording goals and objectives, but of
explicit attention to skill development in each individual course and session through the
creation of purposeful learning activities” (p. 234). Student motivation to be open-minded
is essential; however, the need exists for the faculty to be motivated to create critical
thinking pedagogy that repeatedly encourages thoughtful analysis and purposeful
reflection.
119
Limitations
Students enrolled in community college courses in California constituted the
research sample. Although the findings should prove useful, it is doubtful they can be
generalized to a target population in other locations unless the sample sizes share similar
characteristics. Students enrolled in California community colleges are similar to students
enrolled in courses elsewhere; however, as Gall et al. (2003) maintained, “generalizations
to other cases can be done, but it must be done on a case-by-case basis” (p. 10).
Therefore, generalization may not be possible, nor is it necessary, as it is truly up to the
reader to find similarities in an action research study to a different site or location.
Possible limitations exist due to the modest sample size of 15 (N = 15) students
and the short term of instruction necessitated by an 18-week semester course. Research
suggests that critical thinking is a lifelong process and that successful interventions may
need to occur over an academic year or over several years. Ideally, a longitudinal study
would be most beneficial in determining growth in critical thinking knowledge, skills,
and dispositions as a result of the proposed interventions. The researcher was the teacher
of an intact class, and no attempt at randomization occurred; therefore, the study was
limited to a description of an existent situation rather than creating an experimental
situation. In teacher action research, data are collected and analyzed to assess and
improve educational practice. Since the researcher was the instructor in the critical
thinking course, there was potential for researcher bias as the researcher may have
authority over the participants. The researcher understood this and attempted to approach
the study with caution and without preconceived assumptions of possible outcomes or
results. Care was taken to maintain clarity and objectivity through sharing with research
120
participants and through personal reflection, assessment, and modification of
instructional practices. This study was limited to examination of the interventions
employed in a critical thinking course and did not attempt to measure the influence of
out-of-classroom experiences in growth in critical thinking. For further consideration is
the Hawthorne effect, research participants were aware of the fact they were participating
in a research study, and this may have affected their performance (Gall et al., 1996).
Recommendations
Recommendations for Future Research
This study examined whether there would be improvement in students’ critical
thinking abilities, including knowledge, skills, and dispositions, after the intervention of
direct instruction in rhetorical analysis using argument mapping; Thinker’s Guides, based
on Paul’s model of critical thinking; and Socratic questioning. Further research should be
conducted to investigate instructional pedagogies that could promote critical thinking
knowledge, skills, and dispositions so students may transfer those skills across the
academic disciplines and into their personal and professional lives. Additional
consideration should be given to the examination of the relationship between critical
thinking skills and critical thinking dispositions and designing research studies that
investigate instructional pedagogy that explicitly addresses both of these variables.
This research study consisted of a modest sample size of 15 students and a short
term of instruction necessitated by an 18-week semester course. Further research is
needed using a larger sample size and, possibly, a control group and an experimental
group to determine the extent to which direct instruction using argument mapping, Paul’s
121
Thinker’s Guides, and Socratic questioning can improve students’ critical thinking
abilities.
To increase student learning and achievement in critical thinking and the overall
effectiveness of the course, the length of the study could be extended to an entire year, or
a longitudinal study could be conducted across several critical thinking courses. This
could provide instructors with more time to implement the instructional interventions,
enhance student comprehension and application of those interventions, and provide
deeper insight as to the effectiveness of the research study.
Moreover, a clear definition of critical thinking should be agreed upon by faculty
members who wish to teach critical thinking. The Delphi Report’s definition of critical
thinking could prove useful for those who are attempting to conceptualize and implement
direct instruction in critical thinking in a variety of academic disciplines.
Recommendations for Practice
Although the students’ overall scores on the CCTST–2000 were not at the level of
significance, the implementation of the research interventions—argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning—did improve students’ knowledge of
critical thinking and their perceptions of their own critical thinking abilities. Educators
need to consider explicit teaching of the components of critical thinking as it could
provide the students with the necessary vocabulary, tools, and awareness of their own
critical thinking dispositions essential for effective argument analysis, and with the
knowledge of how to apply that vocabulary and knowledge to the analysis of their own
writing, to the rhetoric in written texts, and to the rhetoric with which they are confronted
in the media.
122
To improve students’ critical thinking knowledge, skills, and dispositions,
educators could develop instructional pedagogy with purposeful learning activities that
encourage critical thinking abilities. Students must be taught how to think critically, and
frequent and explicit teacher modeling of argument mapping, Paul’s Thinker’s Guides,
and Socratic questioning is essential for student comprehension and understanding.
Furthermore, students need to be given opportunities for consistent, repeated practice of
these skills over an extended period of time.
As well, support from the administrative staff along with the implementation of
teacher training in critical thinking instructional strategies, such as argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning, could improve the quality of
instruction and student learning. Ideally, staff members could be encouraged to observe
and collaborate with instructors from other colleges and universities who have
successfully implemented critical thinking into their course curriculum. In California, the
budget deficit is hindering optimum teacher performance. Appropriate resources,
including serviceable and functional technology, are not necessarily imperative but
certainly beneficial for teacher/learner performance and achievement.
What is of utmost importance is creating a classroom that encourages
collaboration, open dialogue, and an acceptance of diverse values, beliefs, and
perspectives. Students should be allowed to openly express their opinions without fear of
judgment, censure, or reproach, and educators can encourage optimal critical thinking
behaviors and attitudes through effective modeling of those behaviors.
123
Conclusions
There is a growing need for educators to attempt to design instructional pedagogy
that will promote students’ critical thinking skills. The researcher employed a variety of
instructional strategies to support and promote the implementation of argument mapping,
Paul’s Thinker’s Guides, and Socratic questioning.
Data from the Post- Critical Thinking Survey suggested instructors can make a
positive impact on students’ perceptions of their own critical thinking abilities through
direct instruction in rhetorical analysis. The highest gains were achieved in the
Knowledge portion of the Post- Critical Thinking Survey. This 2.2 mean gain in student
scores indicated an increase in students’ comprehension of synthesis and fallacious
reasoning, in their ability to clearly define critical thinking and evaluation, and in their
discernment of strong critical thinkers. There was also a 1.8 mean gain in the Skills
portion of the survey. This gain in student scores occurred in students’ ability to
distinguish the connotative and denotative meaning of words, recognize the author’s
purpose, evaluate information for relevance, and accurately comprehend point of view.
As well, there was a 1.8 mean gain in the Dispositions portion of the Post- Critical
Thinking Survey. The highest gains in this category were achieved in students’
willingness to discover ways to expand their thinking and to handle situations that require
extensive thinking. Finally, when asked if they believed their knowledge, skills, and
dispositions had improved through direct instruction in critical thinking, 14 of the 15
students (93%) indicated direct instruction had been instrumental in developing their
critical thinking.
124
Although data from the research findings indicated there were no significant
differences between the pretest and the posttest scores for induction, analysis, and
evaluation, there was a slight difference between the pretest and posttest scores for
inference and deduction; however, the significance level was negligible (p > 0.2).
Nonetheless, findings from the study suggested that repeated, purposeful implementation
of instructional pedagogy, such as argument mapping, Paul’s Thinker’s Guides, and
Socratic questioning, could strengthen students’ perceptions of critical thinking and of
their own critical thinking abilities. Further, the results and the procedures used in this
study could encourage teachers to develop a critically reflective classroom environment
by participating in teacher training courses that support and facilitate the development of
effective pedagogies and assessment tools for the encouragement of student critical
thinking abilities.
125
REFERENCES
Atherton, J. S. (2005). Learning and teaching: Piaget’s developmental theory. Retrieved March 1, 2009, from http://www.learningandteaching.info/learning/piaget.htm
Bessick, S. C. (2008). Improved critical thinking skills as a result of direct instruction
and their relationship to academic achievement. Retrieved from Dissertations & Theses: Full Text database. (AAT 3303591)
Bondy, K. N., Koenigseder, L. A., Ishee, J. H., & Williams, B. G. (2001). Psychometric
properties of the California Critical Thinking Tests. Journal of Nursing Measurement, 9, 309–328.
Brookfield, S. (1995). Becoming a critically reflective teacher. The getting of wisdom:
What critically reflective teaching is and why it’s important. Retrieved May 11, 2009, from National-Louis University Web site: http://www.nl.edu/academics/ cas/ace/facultypapers/StephenBrookfield_Wisdom.cfm?RenderForPrint=1
Brookfield, S. (2003). Adult cognition as a dimension of lifelong learning. In J. Field &
M. Leicester (Eds.), Lifelong learning: Education across the lifespan (pp. 89–101). New York: RoutledgeFalmer.
Brookfield, S. D., Tennant, M., & Pogson, P. (2005). Theory and methods of educating
adults. New York: Wiley. Carroll, R. T. (2007). Teaching critical thinking. Retrieved November 11, 2008, from
http://www.skepdic.com/essays1 Chance, P. (1986). Thinking in the classroom: A survey of programs. New York:
Teachers College Press. Cowden, E. M., Chivore, B. R. S., Maravanyika, O. E., Nyagura, L. M., & Sibanda, I. M.
(1999). Overview of educational research methodologies and assessment criteria of the quality of educational research. In Zimbabwe Education Sector Analysis Review Group (Ed.), Review of education sector analysis in Zimbabwe 1990–1996 (pp. 9–14). Retrieved March 26, 2010, from http://unesdoc.unesco.org/ images/0011/001177/117705eo.pdf
126
Dahms, M., Geonnotti, K., Passalacqua, D., Schilk, J. N., Wetzel, A., & Zulkowsky, M. (2008). The educational theory of Lev Vygotsky: An analysis. Retrieved March 1, 2009, from http://www.newfoundations.com/GALLERY/Vygotsky.html
Elder, L. (2000). Why critical thinking is essential to the community college mission and
why it will be difficult to achieve. Retrieved December 7, 2008, from http://www .criticalthinking.org/page.cfm?PageID=529&CategoryID=73
Elder, L. (2006, April). Socratic dialogue in instruction. Paper presented at the
Cambridge Academy on Critical Thinking, England. Retrieved March 27, 2010, from http://www.criticalthinking.org/print-page.cfm?pageID=571
Elder, L. (2007). Why critical thinking? Retrieved March 26, 2010, from http://www
.criticalthinking.org/page.cfm?PageID=796&CategoryID=103 Elder, L., & Paul, R. (2007). The thinker’s guide to analytic thinking: How to take
thinking apart and what to look for when you do. Dillon Beach, CA: Foundation for Critical Thinking.
Engelmann, K. (n.d.). What is direct instruction (DI)? Retrieved December 2, 2008, from
http://www.nifdi.org/ Ennis, R. (1992, March). Critical thinking: What is it? Proceedings of the 48th annual
meeting of the Philosophy of Education Society, Denver, Colorado. Retrieved May 26, 2006, from http://www.ed.uiuc.edu/PES/92_docs/Ennis.HTM
Ennis, R. (2002). An outline of goals for a critical thinking curriculum and its
assessment. Retrieved October 12, 2009, from http://faculty.ed.uiuc.edu/rhennis/ outlinegoalsctcurasses3.html
Facione, N. C., & Facione, P. A. (1997). Critical thinking assessment in nursing
education programs: An aggregate data analysis. Millbrae, CA: The California Academic Press.
Facione, N. C., Facione, P. A., Blohm, S. W., & Gittens, C. A. (2008). California Critical
Thinking Skills Test: CCTST test manual 2008 edition. Millbrae, CA: California Academic Press.
Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of
educational assessment and instruction: Executive summary: The Delphi report. Retrieved March 1, 2009, from http://www.insightassessment.com/9dex.html
Facione, P. A. (1992). Manual, the California Critical Thinking Skills Test, Form A and
Form B. Millbrae, CA: California Academic Press.
127
Facione, P. A. (2000). The California Critical Thinking Skills Test: CCTST—Form 2000. Millbrae, CA: Insight Assessment/The California Academic Press
Facione, P. A. (2006). Critical thinking: What it is and why it counts. Retrieved October
20, 2006, from www.calpress.com/pdf_files/what&why.pdf Facione, P. A., Facione, N. C., Blohm, S. W., Howard, K., & Giancarlo, C. A. (1998).
The California Critical Thinking Skills Test manual (revised). Millbrae, CA: The California Academic Press.
Facione, P. A., Facione, N. C., & Giancarlo, C. A. (2000). The disposition toward critical
thinking: Its character, measurement, and relationship to critical thinking skill. Informal Logic, 20(1), 61–84. Retrieved June 12, 2009, from http://www.insight assessment.com/9articles2.html
Gall, M. D., Gall, J. P., & Borg, W. R. (1996). Educational research: An introduction
(6th ed.). New York: Allyn & Bacon. Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An introduction
(7th ed.). New York: Allyn & Bacon. Gardner, H. (1993). Multiple intelligences: The theory in practice. New York: Basic
Books. Halpern, D. F. (1999, December). Teaching for critical thinking: Helping college students
develop skills and dispositions of a critical thinker. New Directions for Teaching and Learning, 80, 69–74. Retrieved November 18, 2008, from http://www3 .interscience.wiley.com/journal/101523939/abstract?CRETRY=1&SRETRY=0
Halpern, D. F. (2006). Halpern critical thinking assessment using everyday situations:
Background and scoring standards. Mahwah, NJ: Erlbaum. Hatcher, D. L. (2000). Critical thinking: A new definition and defense. Retrieved October
12, 2009, from The Center of Critical Thinking Web site: http://www.bakeru.edu/ crit/literature/dlh_ct_defense.htm
Jackson, M. (2008). The erosion of attention and the coming dark age: Distracted. New
York: Prometheus Books. Jacobs, S. S. (1995). Technical characteristics and some correlates of the California
Critical Thinking Skills Test, Forms A and B. Research in Higher Education, 36, 89–108.
128
Jacobs, S. S. (1999). The equivalence of Forms A and B of the California Critical Thinking Skills Test. Measurement and Evaluation in Counseling and Development, 31, 211–222.
Laird, T. F. N. (2005). The California Critical Thinking Skills Test (CCTST). Retrieved
February 1, 2009, from Wabash College, Center of Inquiry in the Liberal Arts Web site: http://liberalarts.wabash.edu/cila/displayStory_print.cfm?news_ED= 5121
McPeck, J. (1981). Critical thinking and education. New York: St. Martin’s. Meinecke, J. S. (1997). Toward an operational definition of critical thinking for the
NMSU college of Business Administration and Economics. Retrieved from Dissertations & Theses: Full Text database. (AAT 9818805)
Mendelman, L. (2007, December). Critical thinking and reading. Journal of Adolescent &
Adult Literacy, 51(4), 300–302. Retrieved October 1, 2008, from Academic Search Premier database.
Metcalfe, K. C. (2007). Assessing a feasible effective critical thinking protocol for
community colleges. Retrieved from Dissertations & Theses @ Capella University database. (AAT 3291948)
National Center for Education Statistics. (2007). Digest of education statistics: 2007.
Retrieved March 2, 2009, from http:/nces.ed.gov/programs/digest/d07/ Newton, B. T. (1978). Theoretical bases for higher cognitive questioning—An avenue to
critical thinking. Education, 9(3), 286–291. Retrieved September 20, 2008, from Academic Search Premier database. (AAT 4721735)
Ormrod, J. E. (2004). Learning theory and the educational process. Boston: Pearson. Oxford, R. (1997). Constructivism: Shape-shifting, substance, and teacher education
applications. Peabody Journal of Education, 72, 35–36. Retrieved March, 1, 2009, from http://www.newfoundations.com/GALLERY/Piaget.html
Paul, R. (1993). Critical thinking, moral integrity, and citizenship: Teaching for the
intellectual virtues. Retrieved October 25, 2006, from http://www.criticalthinking .org/resources/articles/ct-moral-integrity.shtml
Paul, R. (1995a). The critical connection: Higher order thinking that unifies curriculum,
instruction, and learning. In J. W. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to prepare students for a rapidly changing world (pp. 103–151). Santa Rosa, CA: Foundation for Critical Thinking.
129
Paul, R. (1995b). Introduction. In J. W. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to prepare students for a rapidly changing world (pp. xi–xv). Santa Rosa, CA: Foundation for Critical Thinking.
Paul, R. (with Binker, A. J. A.). (1995c). Critical thinking: Basic questions and answers.
In J. W. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to prepare students for a rapidly changing world (pp. 91–100). Santa Rosa, CA: Foundation for Critical Thinking.
Paul, R. (with Binker, A. J. A.). (1995d). Glossary: A guide to critical thinking terms and
concepts. In J. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to prepare students for a rapidly changing world (pp. 521–552). Santa Rosa, CA: Foundation for Critical Thinking.
Paul, R. (with Binker, A. J. A.). (1995e). Socratic questioning. In J. Willsen & A. J. A.
Binker (Eds.), Critical thinking: How to prepare students for a rapidly changing world (pp. 335–365). Santa Rosa, CA: Foundation for Critical Thinking.
Paul, R. (with Nosich, G.). (1995f). A model for the national assessment of higher order
thinking. In J. W. Willsen & A. J. A. Binker (Eds.), Critical thinking: How to prepare students for a rapidly changing world (pp. 103–151). Santa Rosa, CA: Foundation for Critical Thinking.
Paul, R. (with Willsen, J.). (1995g). Critical thinking: Identifying the targets. In J.
Willsen & A. J. A. Binker (Eds.), Critical thinking: How to prepare students for a rapidly changing world (pp. 17–36). Santa Rosa, CA: Foundation for Critical Thinking.
Paul, R. (2004). The state of critical thinking today: As the organizer in developing
blueprints for institutional change. Retrieved December 11, 2008, from http:// www.criticalthinking.org/professionalDev/the-state-ct-today.cfm
Paul, R. (2005). Choosing a model of critical thinking. Retrieved November 7, 2008,
from Surry Community College Web site: http://www.surry.edu/about/ct/ choosingn/choosing.html
Paul, R., & Elder, L. (1997, April). Socratic questioning. Retrieved December 7, 2008,
from http://www.criticalthinking.org/page.cfm?PagelID=606&CategoryID=64 Paul, R., & Elder, L. (2000). The role of questions in teaching, thinking and learning.
Retrieved December 6, 2008, from http://www.criticalthinking.org/articles/ thinking-some-purpose.cfm
130
Paul, R., & Elder, L. (2003, Winter). Critical thinking . . . and the art of close reading, part I. Journal of Development Education, 27(2), 36–39. Retrieved October 1, 2008, from Academic Search Premier database. (AAT 11542636)
Paul, R., & Elder, L. (2004, Fall). Critical thinking . . . and the art of close reading, part
III. Journal of Developmental Education, 28(1), 36–37. Retrieved October 1, 2008, from Academic Search Premier database. (AAT 14576885)
Paul, R., & Elder, L. (2006a). First International Academy on Critical Thinking.
Retrieved February 8, 2007, from http://www.criticalthinking.org/conference/ cambridge.shtml
Paul, R., & Elder, L. (2006b). The thinker’s guide to the art of Socratic questioning:
Based on critical thinking concepts & tools. Dillon Beach, CA: Foundation for Critical Thinking.
Paul, R., Elder, L., & Bartell, T. (1997a). A brief history of the idea of critical thinking.
Retrieved November 8, 2008, from http://www.criticalthinking.org/aboutCT/brief HistoryCT.cfm
Paul, R., Elder, L., & Bartell, T. (1997b). Research findings and policy
recommendations: Study of 38 public universities and 28 private universities to determine faculty emphasis. Retrieved December 7, 2008, from http://www .criticalthinking.org/page.cfm?PageID=506&CategoryID=69
Perkins, D. (1993, Fall). Teaching for understanding. American Educator: The
Professional Journal of the American Federation of Teachers, 17(3), 8, 28, 35. Retrieved November 25, 2008, from http://www.exploratorium.edu/IFI/resources/ workshops/teachingforunderstanding.html
Reed, J. H. (1998). Effect of a model for critical thinking on student achievement in
primary source document analysis and interpretation, argumentative reasoning, critical thinking dispositions, and history content in a community college history course. Retrieved from Dissertations & Theses: Full Text database. (AAT 9911510)
Scanlan, J. S. (2006). The effect of Richard Paul’s universal elements and standards of
reasoning on twelfth grade composition. Master’s thesis, Alliant International University, San Diego, CA. Retrieved March 2, 2009, from http://www.critical thinking.org/research/Abstract-JSCANLAN.cfm
Scriven, M., & Paul, R. (1987). Defining critical thinking. Retrieved November 9, 2008,
from http://www.criticalthinking.org/aboutCT/define_critical_thinkng.cfm
131
Shea, R. H., Scanlon, L., & Aufses, R. D. (2008). The language of composition: Reading, writing, rhetoric. Boston: Bedford/St. Martin’s.
Sternberg, R. J. (2003). Four alternative futures for education in the United States: It’s
our choice. School Psychology Quarterly, 18(4), 431–445. Retrieved October 11, 2008, from ProQuest Education Journal database. (AAT 611229291)
Sternberg, R. J. (2008, January). Interdisciplinary problem-based learning. Liberal
Education, 94(1), 12–17. Retrieved October 11, 2008, from ProQuest Education Journals database. (AAT 1429139651)
Texas State Technical College. (2004). Critical Thinking Surveys. Retrieved January 4,
2009, from Center for Excellence in Teaching Web site: http://www.harlingen.tstc .edu/cetl/about.aspx
Tsui, L. (1998). Fostering critical thinking in college students: A mixed-methods study of
influences inside and outside of the classroom. Retrieved from Dissertations & Theses: Full Text database. (AAT 9917229)
Tucker, T. (2008). Predictors of critical thinking as a component of an outcomes
assessment in a graduate level school of social work. Retrieved from Dissertations & Theses: Full Text database. (AAT 3333829)
Twardy, C. (2003). Argument maps improve thinking. Teaching Philosophy, 27(2), 1–19. Van Erp, N. L. (2008). Critical thinking in online graduate courses: A phenomenological
study. Retrieved from Dissertations & Theses @ Capella University database. (AAT 3316338)
van Gelder, T. (2005). Teaching critical thinking. College Teaching, 53(1), 41–46.
Retrieved September 2, 2005, from Academic Search Premier database. (AAT 15348172)
Willsen, J. (1997, March). Chapter 1: Accelerating change, the complexity of problems,
and the quality of our thinking. Community College Times, 9(5). Retrieved October 25, 2006, from http://www.criticalthinking.org/resources/articles/accelerating-change.shtml
Yang, Y. (2008, June 1). A catalyst for teaching critical thinking in a large university
class in Taiwan: Asynchronous online discussion with the facilitation of teaching assistants. Educational Technology Research and Development, 56(3), 241–264. Retrieved January 28, 2009, from ERIC database. (ERIC Document Reproduction Service No. EJ791102)
132
APPENDIX A. PRE- CRITICAL THINKING SURVEY
(Modified from Metcalfe, 2007)
Part 1: Student Demographic Survey The following information is being gathered for statistical purposes only. Please answer each question on the form provided. 1. Gender a. Male b. Female 2. Race a. African-American b. American Indian/Alaskan Native c. Asian or Pacific Islander d. Hispanic e. White – Non-Hispanic 3. Current Age a. 17-19 b. 20-22 c. 23-25 d. 26-30 e. Over 30 4. Current year in college a. Freshman b. Sophomore c. Junior d. Senior 5. In what discipline is your major? a. Humanities b. Sciences (including nursing, computer science). c. Social Sciences (including psychology) d. Mathematics e. Education
133
6. Student status: a. Full-time degree seeking student b. Part-time degree seeking student c. Full-time other credit student d. Part-time other credit student 7. I would consider my English skills to be a. Excellent b. Very good c. Good d. Fair e. In need of improvement 8. I would consider my writing skills to be a. Excellent b. Very good c. Good d. Fair e. In need of improvement 9. I would consider my reading comprehension skills to be a. Excellent b. Very good c. Good d. Fair e. In need of improvement 10. Have you ever taken a Critical Thinking course (or a course similarly labeled) that
was devoted to teaching critical thinking skills or abilities? a. Yes b. No 11. Did you take one or more courses in grades 9-12 that explicitly taught critical
thinking skills while incorporating them into regular course work? a. Yes b. No 12. Have you taken one or more courses at a two or four year college or university
that explicitly taught critical thinking skills while incorporating them into regular course work?
a. Yes b. No Part II: Pre- and Post- Critical Thinking Survey
134
Please rate each of the following statements for how it describes you. Use a 3-point scale in which: 1 = True. 2 - False. 3 = Don’t Know. Choose a number for each statement. 1 2 3 3: Knowledge 13) _____Synthesis is a higher-level thinking skill than summary. 14) _____Critical thinking is using disciplined, self-directed thinking to achieve a desired
outcome. 15) _____Comprehension is a lower-level thinking skill than knowledge. 16) _____Evaluation focuses on making a judgment based on analysis of a statement or
proposition. 17) _____Synthesis can be equated with creative thinking. 18) _____Strong critical thinkers are those who use the intellectual skills of critical
thinking selectively to promote and serve their own personal interests. 19) _____Analysis is a higher level skill than synthesis. 20) _____An argument consists of reasons for or against something 21) _____To be biased toward a particular idea or issue is to be fair-minded toward
opposing points of view. 22) _____A conclusion is the last step in the reasoning process. 23) _____To evaluate is to judge or determine the worth or quality of something. 24) _____Evidence is the data on which a judgment or conclusion might be based or by
which proof or probability might be established. 25) _____Fallacies or fallacious reasoning conforms to the rules of good reasoning. 26) _____An inference is an intellectual act by which one concludes that something is
true in light of something else being true 27) _____A premise is the last step in the reasoning process. Critical Thinking: Skills 28) _____I can recognize the connotative and denotative meaning of words when I read. 29) _____I can recognize the author’s purpose for writing a specific piece of literature. 30) _____Considering alternative points of view is helpful in problem solving. 31) _____Evaluating information for its relevance is a valuable critical thinking ability. 32) _____When exploring an issue, I frequently evaluate the credibility of the sources of
information. 33) _____I can accurately comprehend one’s point of view or frame of reference. 34) _____I have difficulty recognizing the difference between assumptions, beliefs, and
facts. 35) _____I avoid generalizations and oversimplifications. 36) _____I analyze and evaluate arguments, interpretations, beliefs, or theories. 37) _____I have difficulty comparing perspectives, interpretations, or theories.
135
38) _____I have difficulty taking into account opposing points of view. 39) _____I find it easy to offer relevant, constructive evidence to support my points of
view. 40) _____I frequently compare or transfer insights learned in the classroom to real life
situations. 41) _____When exploring an issue, I sometimes have difficulty recognizing
contradictions.. 42) _____I consider myself to be fair-minded. Critical Thinking: Dispositions 43) _____I consider myself a thoughtful person. 44) _____I frequently think about ways to improve my thinking. 45) _____I would prefer complex to simple problems. 46) _____I like to have the responsibility of handling a situation that requires a lot of
thinking. 47) _____I would rather do something which challenges my thinking abilities. 48) _____I try to anticipate and avoid situations where there is a likely chance I will have
to think in-depth about something. 49) _____I find satisfaction in deliberating hard and for long hours. 50) _____I prefer to think about small, daily projects than long-term ones. 51) _____I prefer projects which require very little thought. 52) _____The idea of relying on thought to make my way to the top appeals to me. 53) _____I really enjoy a task that involves coming up with new solutions to problems. 54) _____Learning new ways to think doesn’t excite me very much. 55) _____The notion of thinking abstractly is frightening to me. 56) _____I would prefer a task that is intellectual, difficult, and important to one that is
somewhat important but does not require much thought. 57) _____I usually end up deliberating about issues even when they do not affect me
personally. Note. The questions in the “Dispositions” portion of the Pre- Critical Thinking Survey are from Assessing a Feasible Effective Critical Thinking Protocol for Community Colleges (p. 227), by K.C. Metcalfe, 2007, Ph.D. Dissertation, Capella University, United States – Minnesota. Adapted for use with permission.
136
APPENDIX B. POST- CRITICAL THINKING SURVEY
(Modified from Metcalfe, 2007) Part 1: Student Demographic Survey
The following information is being gathered for statistical purposes only. Please answer each question on the form provided. 1. Gender a. Male b. Female 2. Race a. African-American b. American Indian/Alaskan Native c. Asian or Pacific Islander d. Hispanic e. White – Non-Hispanic 3. Current Age a. 17-19 b. 20-22 c. 23-25 d. 26-30 e. Over 30 4. Current year in college a. Freshman b. Sophomore c. Junior d. Senior 5. In what discipline is your major? a. Humanities b. Sciences (including nursing, computer science). c. Social Sciences (including psychology) d. Mathematics e. Education 6. Student status:
137
a. Full-time degree seeking student b. Part-time degree seeking student c. Full-time other credit student d. Part-time other credit student 7. I would consider my English skills to be a. Excellent b. Very good c. Good d. Fair e. In need of improvement 8. I would consider my writing skills to be a. Excellent b. Very good c. Good d. Fair e. In need of improvement 9. I would consider my reading comprehension skills to be a. Excellent b. Very good c. Good d. Fair e. In need of improvement 10. Have you ever taken a Critical Thinking course (or a course similarly labeled) that
was devoted to teaching critical thinking skills or abilities? a. Yes b. No 11. Did you take one or more courses in grades 9-12 that explicitly taught critical
thinking skills while incorporating them into regular course work? a. Yes b. No 12. Have you taken one or more courses at a two or four year college or university
that explicitly taught critical thinking skills while incorporating them into regular course work?
a. Yes b. No Part II: Pre- and Post- Critical Thinking Survey
138
Please rate each of the following statements for how it describes you. Use a 3-point scale in which: 1 = True. 2 - False. 3 = Don’t Know. Choose a number for each statement. 1 2 3 Critical Thinking: Knowledge 13) _____Synthesis is a higher-level thinking skill than summary. 14) _____Critical thinking is using disciplined, self-directed thinking to achieve a desired
outcome. 15) _____Comprehension is a lower-level thinking skill than knowledge 16) _____Evaluation focuses on making a judgment based on analysis of a statement or
proposition. 17) _____Synthesis can be equated with creative thinking. 18) _____Strong critical thinkers are those who use the intellectual skills of critical
thinking selectively to promote and serve their own personal interests. 19) _____Analysis is a higher level skill than synthesis. 20) _____An argument consists of reasons for or against something 21) _____To be biased toward a particular idea or issue is to be fair-minded toward
opposing points of view. 22) _____A conclusion is the last step in the reasoning process. 23) _____To evaluate is to judge or determine the worth or quality of something. 24) _____Evidence is the data on which a judgment or conclusion might be based or by
which proof or probability might be established. 25) _____Fallacies or fallacious reasoning conforms to the rules of good reasoning. 26) _____An inference is an intellectual act by which one concludes that something is
true in light of something else being true 27) _____A premise is the last step in the reasoning process. Critical Thinking: Skills 28) _____I can recognize the connotative and denotative meaning of words when I read. 29) _____I can recognize the author’s purpose for writing a specific piece of literature. 30) _____Considering alternative points of view is helpful in problem solving. 31) _____Evaluating information for its relevance is a valuable critical thinking ability. 32) _____When exploring an issue, I frequently evaluate the credibility of the sources of
information. 33) _____I can accurately comprehend one’s point of view or frame of reference. 34) _____I have difficulty recognizing the difference between assumptions, beliefs, and
facts. 35) _____I avoid generalizations and oversimplifications. 36) _____I analyze and evaluate arguments, interpretations, beliefs, or theories. 37) _____I have difficulty comparing perspectives, interpretations, or theories. 38) _____I have difficulty taking into account opposing points of view.
139
39) _____I find it easy to offer relevant, constructive evidence to support my points of view.
40) _____I frequently compare or transfer insights learned in the classroom to real life situations.
41) _____When exploring an issue, I sometimes have difficulty recognizing contradictions..
42) _____I consider myself to be fair-minded. Critical Thinking: Dispositions 43) _____I consider myself a thoughtful person. 44) _____I frequently think about ways to improve my thinking. 45) _____I would prefer complex to simple problems. 46) _____I like to have the responsibility of handling a situation that requires a lot of
thinking. 47) _____I would rather do something which challenges my thinking abilities. 48) _____I try to anticipate and avoid situations where there is a likely chance I will have
to think in-depth about something. 49) _____I find satisfaction in deliberating hard and for long hours. 50) _____I prefer to think about small, daily projects than long-term ones. 51) _____I prefer projects which require very little thought. 52) _____The idea of relying on thought to make my way to the top appeals to me. 53) _____I really enjoy a task that involves coming up with new solutions to problems. 54) _____Learning new ways to think doesn’t excite me very much. 55) _____The notion of thinking abstractly is frightening to me. 56) _____I would prefer a task that is intellectual, difficult, and important to one that is
somewhat important but does not require much thought. 57) _____I usually end up deliberating about issues even when they do not affect me
personally. Part III: Student Perceptions of Critical Thinking Please circle the appropriate number for each item. Low High Score Score Argument Mapping To what extent does Argument Mapping help you identify 58) a simple argument 1 2 3 4 5 59) a complex argument 1 2 3 4 5 60) the author’s central thesis 1 2 3 4 5 61) the rhetorical/argumentative function of
each paragraph 1 2 3 4 5 62) the claims made by the author 1 2 3 4 5 63) the objections made by the author 1 2 3 4 5 64) the counter arguments 1 2 3 4 5 65) the inferences made by the author 1 2 3 4 5 66) the structure of the reasoning process 1 2 3 4 5
140
67) the missing layers in an argument 1 2 3 4 5 Richard Paul’s Thinker’s Guides To what extent do Thinker’s Guides help you identify 68) the author’s purpose 1 2 3 4 5 69) the questions or problem the author has 1 2 3 4 5 70) the point of view or frame of reference 1 2 3 4 5 71) unstated assumptions 1 2 3 4 5 72) inferences 1 2 3 4 5 73) bias or narrowness 1 2 3 4 5 74) contradictions in point of view 1 2 3 4 5 75) the author’s evidence 1 2 3 4 5 76) the author’s conclusions 1 2 3 4 5 77) the elements of thought and how they can assist in your critical thinking abilities 1 2 3 4 5 Socratic Questioning To what extent does Socratic Questioning help you identify 78) goals and purposes of writer 1 2 3 4 5 79) the nature of the question 1 2 3 4 5 80) the problem being addressed 1 2 3 4 5 81) relevant data and information to support 1 2 3 4 5 82) alternative interpretations of the data and information 1 2 3 4 5 83) key concepts and ideas 1 2 3 4 5 84) assumptions being made 1 2 3 4 5 85) implications and consequences of what is being said 1 2 3 4 5 86) alternative points of view 1 2 3 4 5 87) How effective was Socratic Questioning in preparing
you for discussion, writing, and for developing your arguments 1 2 3 4 5
Please rate each of the following statements for how it describes you. Use a 3-point scale in which: 1 = True. 2 - False. 3 = Don’t Know. Choose a number for each statement. 1 2 3 88) _____The instruction in Socratic Questioning has extended my knowledge of critical
thinking. 89) _____The instruction in argument mapping has extended my knowledge of critical
thinking.
141
90) _____The instruction in Richard Paul’s Thinker’s Guides has extended my knowledge of critical thinking.
91) _____I believe that my knowledge, skills, and dispositions have improved through direct instruction in critical thinking.
92) _____The critical thinking process that I have learned in English 104 has helped me to make better and more informed decisions in my everyday life.
93) _____I will be able to apply what I have learned in this course about thinking critically to a practical situation in my life.
Please answer the following questions: 94) What has been the most frustrating or difficult aspect of learning to think more critically? 95) Of the three strategies that you learned in Critical Thinking 104, which has been the most effective in helping you become a better thinker?
Note. The questions in the “Dispositions” portion of the Post-Critical Thinking Survey are from Assessing a Feasible Effective Critical Tthinking Protocol for Community Colleges (p. 227), by K.C. Metcalfe, 2007, Ph.D. Dissertation, Capella University, United States – Minnesota. Adapted for use permission.
142
APPENDIX C. REQUEST FOR PERMISSION TO DR. METCALF
March 31, 2010 Dr. Metcalfe, I am a doctoral student at Capella University. For my dissertation I am currently conducting a study concerning entitled Improving Student Critical Thinking and Perceptions of Critical Thinking Through Direct Instruction in Rhetorical Analysis at Victor Valley College in Apple Valley, California in a Critical Thinking 104 course. The quantitative measure of the study is the California Critical Thinking Skills Test-2000, the qualitative measure is a Pre- and Post- Critical Thinking Skills Survey, and both assessment will be administered at the beginning and at the end of the semester. The Pre- and Post- Critical Thinking Survey consists of three sections: “Knowledge,” “Skills, “and “Dispositions.” For the “Dispositions” portion of the survey, I would like your permission to use some of the questions from your Capella Dissertation, Assessing a Feasible Effective Critical Thinking Protocol for Community Colletes, Appendix E. Need for Cognition Scale, page 227, published in 2007. Specifically, I would like to use the following questions: I consider myself a thoughtful person. I frequently think about ways to improve my thinking. I would prefer complex to simple problems. I like to have the responsibility of handling a situation that requires a lot of
thinking. I would rather do something which challenges my thinking abilities. I try to anticipate and avoid situations where there is a likely chance I will
have to think in-depth about something. I find satisfaction in deliberating hard and for long hours. I prefer to think about small, daily projects than long-term ones. I prefer projects which require very little thought. The idea of relying on thought to make my way to the top appeals to me. I really enjoy a task that involves coming up with new solutions to problems. Learning new ways to think doesn’t excite me very much. The notion of thinking abstractly is frightening to me. I would prefer a task that is intellectual, difficult, and important to one that is
somewhat important but does not require much thought. I usually end up deliberating about issues even when they do not affect me
personally. If you agree, I would appreciate an email indicating your permission. Thank-you, Lauren McGuire
143
APPENDIX D. LETTER OF PERMISSION FROM DR. METCALF
March 31, 2010 Dear Lauren McGuire, Yes, you are free to use whatever you need from my work. Be sure to contact Dr. Diane Halpern, McKenna College Claremont University, (here in CA.), as much of my work was built on hers. She will be delighted, as am I, that you are engaged in this type of research and that you have found our work together useful. Best of luck to you! Kim C. Metcalfe Assistant Professor of Early Childhood Studies & Psychology Educational Psychology Ph.D. Assistant Chair Health Human & Public Services