Labadie Dissertation C7FACULTY: A QUALITATIVE STUDY
Tracy L. Labadie
This dissertation is submitted in partial fulfillment of the requirements for the degree of
Doctor of Education
Ferris State University
ASSESSMENT OF STUDENT LEARNING AS AN EMOTIONAL PRACTICE FOR
FACULTY: A QUALITATIVE STUDY
Dissertation Committee
ABSTRACT
Assessment of student learning is a necessary component of the work life for educators.
The real promise of student learning assessment depends on faculty embracing the process.
Although the need for formalized assessment has been in practice for several decades, faculty
engagement in the assessment process continues to be the number one barrier to effectively
cultivating a culture of assessment across the campus. Understanding faculty’s emotional journey
while doing assessment of student learning will enhance institutional ability to support faculty
through the process, garner faculty buy-in, and cultivate a culture of assessment on campus.
Continuing to ignore the emotional factors that may impact the assessment process may cause
some faculty to remain on the margins of the institution, a position that provides no opportunity
or incentive to participate in college-wide quality improvement efforts.
This qualitative study contributes to the limited research on the non-transactional
elements of student learning assessment by capturing stories that will increase awareness and
understanding of the faculty emotional landscape that accompanies the assessment experience.
The study’s design provides the higher education community, particularly faculty and
administrators, a better understanding of the emotional barriers that may inhibit faculty
motivation and participation. It will also promote a better understanding of the types of emotions
that may come into play and identify if any point in the process is a catalyst for faculty’s
willingness to engage. This study has the potential to contribute knowledge to support cultural
shifts in higher educational institutions.
i
KEY WORDS: assessment of student learning, faculty buy-in, culture of assessment, emotions,
emotions in the workplace, emotional landscape
ii
ACKNOWLEDGMENTS
Completion of this dissertation would not have been possible without the support and
guidance from many people. I wish to thank Dr. Gary Wheeler, Dr. Jennifer (Jenny) Schanker,
and Dr. Thomas (Tom) Quinn who comprised my dissertation committee. I would especially like
to thank Dr. Wheeler, whom I consider a confidant and mentor, for his unwavering guidance. I
owe much of what I had accomplished with this research to his continued patience, coaching,
and support. I would be remiss if I did not thank the faculty included in this study for their
participation and support. Their stories were enlightening and powerful.
My heartfelt gratitude goes out to Dr. Sandra (Sandy) Balkema of Ferris State University,
Dr. Tammy Russell of Glen Oaks Community College, Dr. Matthew (Matt) Fall of Lansing
Community College, Dr. Karen Hicks of Lansing Community College, and Rafeeq McGiveron
of Lansing Community. Their guidance and encouragement were instrumental in my ability and
desire to successfully complete this dissertation. I would also like to thank my colleagues from
Cohort 7 who were always there for me when I needed them most.
I must thank my family. I owe my success to their love, support, and sacrifice. I thank my
parents, Bill and Linda Hasbrouck, for always believing in me. My three biggest joys in life,
Julie, Justin, and Isabelle inspire me every day to be the best person I can be. Finally, I thank my
husband Nick for always being on Team Tracy.
iii
Summary of the Study .......................................................................................................74 Limitations and Delimitations ...........................................................................................77 Recommendations for Future Research.............................................................................78 Reflection ..........................................................................................................................80
APPENDIX C: INTERVIEW PARTICIPATION RECRUITMENT TRANSCRIPT .................94 APPENDIX D: INFORMED CONSENT FORM.........................................................................96
APPENDIX E: INTERVIEW GUIDE ........................................................................................101 APPENDIX F: CONFIDENTIALITY AGREEMENT FORM FOR IRB PROJECTS .............103
Page
Table 1: Emotion Categorization as adopted by Folkman and Lazarus (1985). ...........................42
Table 2: Interview Participants......................................................................................................47
Table 3: Assessment Technique ....................................................................................................48
Table 4: Emotion Categorization as adopted by Folkman and Lazarus (1985). ...........................48
Table 5: List of Emotions Cited by Each Interviewee ..................................................................59
Table 6: Compilation of emotions noted by each participant categorized by Folkman and Lazarus’ emotion categorization matrix .........................................................................63
vi
Page
Figure 1: Emotional categories cited by FTF1 during the PDSA cycle ........................................50
Figure 2: Emotional categories cited by FTF2 during the PDSA cycle ........................................51
Figure 3: Emotional categories cited by FTF3 during the PDSA cycle ........................................53
Figure 4: Emotional categories cited by FTF4 during the PDSA cycle ........................................54
Figure 5: Emotional categories cited by FTF5 during the PDSA cycle ........................................56
Figure 6: Emotions noted by each interviewee for each stage of the PDSA cycle. ......................66
vii
INTRODUCTION
The importance of classroom assessment of student learning outcomes continues to be a
featured element in higher learning. Accreditation bodies, both specialized, such as those for
engineering, nursing, or education, and regional accreditors, such as the Higher Learning
Commission (HLC), have made assessment of student learning a priority to meet their Criteria
for Accreditation (HLC, nd). There are numerous references in literature outlining best practices
for accomplishing assessment of student learning in the classroom, such as assessment design,
techniques, examples, data collection and analysis, and accountability (Banta, 2004a, 2004b;
Lane, Lane, Rich, & Wheeling, 2014; MSCHE, 2007; Nunley, Bers, & Manning, 2011; Serban
& Friedlander, 2004; Standahl, 2008; Walvoord, 2010). Additionally, there is frequent discussion
regarding the importance of faculty participation in the assessment process, along with tips on
how to best foster faculty engagement (Driscoli & Wood, 2007; Guetterman & Mitchell, 2015;
Lane et al., 2014; MacDonald, Williams, Lazowski, Horst, & Barron, 2014; Ndoye & Parker,
2010; Wang & Hurley, 2012). This discussion centers around the notion that faculty must be
properly motivated to participate in assessment of student learning and that this motivation stems
from gaining an understanding of the importance of undertaking such practices. What is missing
from the literature is a discussion regarding the non-transactional elements involved, specifically
the role that faculty emotions have on the assessment process and its efficacy.
This study proposed that assessing student learning is an emotional experience for
instructors. The goal was to gain a better understanding of the faculty experience to enhance
1
institutional efforts to support quality teaching and learning in the classroom. Emotional factors
and faculty response to them may have an influence on the instructor’s willingness to continue
using a particular assessment technique, impact their motivation to design and implement
experiential learning activities in the classroom, and potentially impact the efficacy of
assessment process itself.
Background for the Study
Assessment of student learning has evolved over the past century from standardized
testing to outcomes-based evaluation of student learning. The current era of assessment was a
reaction to increased political pressure for accountability of higher educational institutions,
resulting in a need for higher-level assessment of student learning to evaluate institutional
performance and to meet accreditation standards (Shavelson, 2007). Traditional, standardized
evaluations were no longer sufficient. With the understanding that standardized evaluations do
not automatically lead to educational improvement, pressure increased to institute outcomes-
based educational models.
expectations prior to delivering instruction, that students direct their learning efforts to meet
those learning expectations, and that student progress and completion be measured by the level
of achievement toward them (Driscoll & Wood, 2007). While instructors inherently wish for
their students to succeed, formalized assessment of student learning continues to be a challenge.
Faculty remain unconvinced that quantifying assessment of student performance brings value to
the learning experience. Driscoll and Wood (2007, p. 8) report faculty complain that formalizing
assessment creates inflexible, mechanistic practices limited innovation and creation in the
classroom. Kelly (July 2017) explains that common concerns include student changeover each
2
term leading to irrelevant data collection and analysis, a perception that assessment is just
busywork, the notion that assessment is not part of teaching responsibilities, and the standard
teaching workload does not leave room for any additional work such as assessment (para. 2).
Further, gauging how students are doing is an inherent part of teaching, and faculty often believe
that assigning final grades is a reflection of their assessment of student performance (Banta,
2004b). Cook-Sather (April 2009) notes that some faculty remain suspicious of administrator
motives for requiring formalized assessment practices, citing concerns that assessment will be
used as an evaluation tool for teacher performance, resulting in punitive action by administrators.
Confusion regarding assessment expectations continues to permeate higher education
campuses to this day. Although there is a plethora of literature and professional development
opportunities available, these resources do not reach all who need them. Instructors who teach in
higher education are typically content experts and have not received training on assessment, thus
limiting the scope of their skills, understanding, and confidence to effectively develop and
employ assessment tools in the classroom (Banta, 2004b; Driscoll & Wood, 2007).
Assessment of student learning in a public community college setting is especially
problematic. The open access model of community colleges results in multiple missions, leading
to multiple outcomes and learning expectations to be assessed. The demand for time on faculty
teaching in a community college is greater than that of their peers teaching at four-year
institutions. This is significant because faculty at two-year institutions typically carry a higher
teaching load then their colleagues at four-year institutions, leaving them with limited
availability to focus on student learning assessment (Caudle & Hammons, 2017).
To meet accreditation requirements, institutions of higher education need to document
student learning performance. This is accomplished by establishing learning outcomes and
3
expectations for students at all levels of the institution: course, program, and institutional. It is
not enough to claim that students are meeting learning expectations. Accreditors want to see
evidence (HLC, nd) that institutions are using assessment results to inform curriculum decisions.
Therefore, when the term “assessment” is spoken on campus, faculty immediately focus on the
data collection and reporting aspect of the process. However, the true nature of assessment is to
gain an understanding of the student’s learning experience. This can only be answered by faculty
(Hutchings, 2010). Student learning revolves around instructional effectiveness, and faculty are
more effective when using techniques that focus on authentic assessment of student learning
(Driscoll & Wood, 2007). Assessment of student learning goes beyond data collection and
reporting. The value-added component of the process is the analysis of the results collected to
understand and improve student learning. Regardless of the institution’s approach to assessment,
the success of this step in the process hinges on faculty motivation and engagement (MacDonald
et al., 2014).
The notion of faculty buy-in is a common concern cited in the literature and continues to
be reported as the primary barrier to developing a comprehensive culture of assessment. Many
institutional strategies to address this challenge are implemented with a top-down approach are a
result of external pressures and tend to fail because leadership does not foster faculty ownership
over the process. Researchers have reported that faculty need to own the process in order to fully
embrace it (Banta, 2004a; Caudle & Hammons, 2017; Guetterman & Mitchell, 2015; Serban &
Friedlander, 2004; Wang & Hurley, 2012). Cultivating faculty buy-in tends to follow two
primary strategies: (1) Holding faculty accountable; and (2) highlighting the value of assessment.
These strategies may be deployed individually or in conjunction with one another.
4
In an effort to hold faculty accountable for conducting and reporting on assessment of
student learning, some institutions include assessment as a component of faculty performance
evaluation, promotion, and tenure decisions (Suskie, 2015). Suskie (2015) argues that
accountability measures such as these are necessary to reinforce a culture of quality on campus.
Gardner, Kline, and Bresciani (2014) encourage administrators to allocate resources based on
evidence and results of student learning assessment.
The second strategy focuses on providing training and professional development for
faculty to help them better understand the process, with a goal of increasing their appreciation for
the value that assessment can bring to their classroom. Driscoll and Wood (2007) claim that
participation in the practice of developing student learning outcomes may be enlightening for
faculty and result in them having a greater appreciation for the assessment process. Banta
(2004a) further argues that student success in the classroom hinges on the effectiveness of the
instructor. She reports that the instructors who were formally trained in effective use of
assessment had changed their behavior in the classroom and were more likely to embrace the
value of the assessment process.
Cultivating a culture of assessment on campus is noted to be the most effective way to
garner faculty engagement in the assessment process. Researchers report this can be
accomplished by creating an atmosphere that is receptive, supportive, and enabling. However,
there is a lack of consensus of what this looks like. Some point to a culture of assessment as a
final result that fosters acceptance and engagement in assessment practices, while others deem
the establishment of a culture of assessment as the catalyst for institutional change (Banta, 1997;
Driscoll & Wood, 2007; Guetterman & Mitchell, 2015; Lakos & Phipps, 2004; Lane et al., 2014;
Ndoye & Parker, 2010).
5
Assessment, by design, goes beyond attempting to capture what students know and may
have learned from particular experiences. It is a practice of instructor self-reflection that may
lead to a sense of vulnerability. This feeling of vulnerability may be unpleasant enough to keep
the instructor from feeling motivated to engage in the assessment process or to accept its
instructional value. Driscoll and Wood (2007) note that a culture of assessment is centralized on
the concept of trust between faculty and administration and trust between faculty and students.
To obtain this level of trust there must be openness, mutual respect, responsiveness,
collaboration, enjoyment, and empowerment. Banta (2004b) further explains that barriers to
faculty engagement are oftentimes within the control of institutional leadership, such as the lack
of time needed to devote to the practice, the belief that assessment is already occurring, and the
feeling that the process is being imposed on them. She goes on to explain that institutional
support and commitment from all levels is essential. Recognizing assessment as an emotional
practice can better serve the pedagogical and assessment structure needs of instructors, as well as
provide adequate motivation to be successful teachers and assessment practitioners.
Problem Statement
An abundance of literature focused on the need for faculty commitment to the assessment
process to foster quality improvement of student learning experiences. The literature discussed
strategies and best practices that administrators may adopt to encourage a culture of assessment
on campus. Lack of faculty buy-in continues to be the number one factor inhibiting
comprehensive assessment of student learning for most institutions. Understanding how to
motivate faculty to actively engage and embrace the process creates a problem for institutional
administration.
6
In William H. Whyte’s book, The Organization Man, published in 1956, he explained
that effective organizational leaders are logical and reasoned decision makers. Emotions were
regarded as undesirable influencers and the showing of emotions was to be considered a sign of
weakness (As cited by Muchinsky, 2000). This concept of professionalism as one that is void of
emotion has permeated throughout business and industry for decades (Sharp-Page, August 2018,
p. 1). However, bringing emotions into the workplace allows employees to be their authentic
selves, improves workplace relationships, and increases innovation and productivity (Muchinsky,
2000; Sharp-Page, August 2018; TEDxTalks, March 2015). This authenticity encourages
appreciation, compassion, courage, and vulnerability. Vulnerability is the key driver in human
trust and connection, innovation, change, growth, risk taking, and development (TEDxTalks,
March 2015). Vulnerability is oftentimes perceived as bad. Driscoll and Wood (2007) note that
the level of vulnerability that faculty experience during the assessment process is sometimes
remarkable. Faculty have reported the feeling of exposure and the fear of risk when evaluating
assessment of student learning. Robbins (TEDxTalks, March 2015) argues that vulnerability is
good, and that it should be recognized and celebrated for the positive attribute that it is.
The problem driving this qualitative study is the lack of research regarding the faculty
emotional experience while conducting assessment of student learning. While some literature
made mention of emotional factors such as vulnerability and motivation, the authors danced
around the topic. For example, Driscoll and Wood (2007) discussed the importance of enjoyment
as a key factor in building a strong culture of assessment on campus and Serban and Friedlander
(2004) explained that faculty are likely to embrace the assessment process when they feel
motivated over a period of time. However, the authors did not expand further beyond a brief
mention of emotions. This study begins to establish the framework for understanding the true
7
experience that is assessment. The phenomenological design of this study captured the faculty
experience to identify if they are experiencing emotions during the assessment process, what
types of emotions they experience, and if there are certain points in the process that are triggers
for specific emotions. Institutional leadership will develop a comprehensive understanding of
faculty needs to foster a culture of assessment across the campus.
Significance of the Study
Assessment of student learning is a necessary component of the work life for educators.
Gone are the days when faculty can step into the classroom, teach, grade, and enjoy a summer
off. The need for formalized assessment of student learning emerged in the late 1970s when
external political pressure increased the need for accountability (Shavelson, 2007). This need for
accountability in higher education has seeped into accreditation requirements that could have a
significant and detrimental impact on the college should they fall out of compliance.
The real promise of student learning assessment depends on faculty embracing the
process. Although the need for formalized assessment has been in practice for several decades,
faculty engagement in the assessment process continues to be the number one barrier to
effectively cultivating a culture of assessment across the campus (Banta, 2004b). Assessment of
student learning goes beyond collecting data. Data must be evaluated to understand the student
experience and to identify opportunities for pedagogical improvements. Regardless of the
institution’s approach to assessment, the success of this step hinges on faculty motivation and
participation (MacDonald et al., 2014).
Faculty and administrators need to recognize and acknowledge that humans are
emotional creatures and encourage instructors to be authentic in the classroom. Continuing to
ignore the emotional factors that may impact the assessment process may cause some faculty to
8
remain on the margins of the institution, a position that provides no opportunity or incentive to
participate in college-wide quality improvement efforts. Both faculty and administrators will
benefit from having an understanding of the types of emotions faculty are experiencing and the
points of the process that may be the most challenging. Having an appreciation that assessment is
an emotional practice will allow faculty to anticipate the emotional journey they may embark
upon and develop strategies for dealing with it. Having an appreciation for the emotional journey
that faculty face will allow administrators to focus resource allocations and target support
systems where they are most needed.
This study contributes to the limited research on the non-transactional elements of student
learning assessment by capturing stories that will increase awareness and understanding of the
faculty emotional landscape that accompanies the assessment experience. The study’s design will
help increase faculty and administrator awareness of the emotional barriers that may inhibit
faculty motivation and participation. It will also promote a better understanding of the types of
emotions that may come into play and identify if any point in the process is a catalyst for
faculty’s willingness to engage. This study has the potential to contribute knowledge to support
cultural shifts in higher educational institutions.
Nature of the Study
Upon review of research methods, the researcher determined that a qualitative approach
is the most appropriate for this study. A phenomenological study is designed to gain a better
understanding of a phenomena and capture the subjective understanding of the interviewee’s
experience (Seidman, 2013). This study used a phenomenology approach with participants
employed as full-time instructional faculty by one of the mid-sized or large public community
colleges in the State of Michigan. Faculty were selected based on two criteria: (1) being full-time
9
tenured or non-tenured instructional faculty; and (2) having implemented within the past three
academic years (2015–2016, 2016–2017, 2017–2018), or currently implementing an assessment
technique that addresses one of the top three tiers of Bloom’s Taxonomy (analysis, evaluation,
and creating) (Krathwohl, 2002).
The theoretical framework was based on the literature related to the topic as discussed in
Chapter Two. The framework drew upon concepts, terms, and definitions cited in the literature,
particularly in regard to faculty motivation and willingness to embrace the assessment process.
For years, the use of course examinations as the assessment tool of choice resulted in a push for
authentic assessment, with directed efforts to capture the student’s true learning experience
(Driscoll & Wood, 2007). The literature discusses strategies and best practices that
administrators may adopt to encourage a culture of authentic assessment on campus. Lack of
faculty buy-in continues to be the number one factor inhibiting comprehensive assessment of
student learning for most institutions. This concern has permeated for decades despite the
countless number of resources, literature, conferences, and professional development
opportunities offering strategies and best practices to overcome the barrier. The strategies
emphasize the importance of encouraging faculty engagement by fostering an environment that
encourages authentic faculty ownership of the process (Banta, 2004b; Caudle & Hammons,
2017; Guetterman & Mitchell, 2015; Serban & Friedlander, 2004; Wang & Hurley, 2012). The
qualitative data collected through semi-structured interviews with study participants offers
insight into the emotional landscape that may accompany the assessment of student learning
process, shedding light on a potential root cause for faculty resistance that has not been explored.
10
Research Questions
The researcher was interested in confirming that emotions are indeed experienced during
the assessment of student learning process and discovering both what types of emotions may be
elicited and whether certain points in the process are especially challenging for faculty. Three
fundamental questions will guide this research:
1. Research Question 1. Are faculty experiencing emotions while doing assessment of student learning?
2. Research Question 2. What emotions are experienced by faculty throughout the implementation process?
3. Research Question 3. Are different emotions experienced at different points of the implementation process?
Definition of Terms
• Assessment – shorthand for assessment of student learning or educational assessment. It is the systematic process of collecting and documenting data on the knowledge and skill levels of students, to inform decisions that affect student learning (Walvoord, 2010).
• Bloom’s Taxonomy – developed in the 1950s and named after Benjamin Bloom, a set of hierarchical models used to classify educational learning objectives into levels of complexity and specificity. In 2001, the taxonomy was revised to identify cognition levels as remember, understand, apply, analyze, evaluate, and create (Krathwohl, 2002). A diagram of the taxonomy used for this study may be found in Appendix A.
• W. Edwards Deming’s Plan, Do, Study, Act (PDSA) Model – made popular by W. Edwards Deming, a four-step method used by organizations for continuous quality improvement of processes, services, and products (Moen, 2010). A diagram of the PDSA model used for this study may be found in Appendix B.
Limitations of the Study
Participants in this study were all full-time faculty teaching at a community college in the
State of Michigan. Their perspectives may not reflect those who are employed as adjunct faculty
or who teach at a different type of postsecondary institution. Thus, the ability to generalize this
study’s findings may be limited. There are six limitations of this study:
11
1. The study was limited to the number of full-time faculty who were able to be recruited from the identified community college sample in the State of Michigan. Participation was voluntary, and faculty willingness to participate was outside of the researcher’s control.
2. Interviews were conducted in the southeastern region of the State of Michigan. The researcher chose this approach based on ease of access to the study participant sample.
3. The study included full-time faculty who teach in a mid- to large-size urban community college setting. Adjunct faculty were intentionally omitted from the study. While adjunct faculty have become the mainstream teaching workforce in community colleges, “they have less frequent interactions with students and are less integrated into the campus cultures in which they work” (Jolley, Cross, & Bryant, 2013, p. 219). For this reason, it was the belief of the researcher that including adjunct faculty would change the scope and dynamic of the study. Further, faculty from other types of institutions were not included, such as those who teach in four-year institutions, rural community colleges, tribal colleges, and private institutions.
4. The study required faculty to reflect on experiences from the past. It is possible their responses are influenced by time and other experiences.
5. The study was designed to capture the emotional experience of the study participants during the assessment process. The study did not include contextual analysis, such as institutional culture, expectations of instructor workload, union landscape, and the politics of compliance and accountability on campus.
6. The researcher is not a trained psychologist and does not possess the expertise to analyze emotions or the root cause of emotions. The researcher relied on published work of trained psychologists to provide the context needed for study analysis.
Delimitations
This study was confined to interviewing and observing full-time tenured and non-tenured
faculty at a mid-size to large community college in the State of Michigan. Faculty from six
community colleges were targeted for interviews based on the number of full-time faculty
employed at each institution and the location of their institution. Participants were recruited
based on recommendations from assessment directors from each institution. Recommendations
included full-time faculty who within the past three years have implemented an assessment of
student learning technique that measures one of the top three tiers of Bloom’s Taxonomy.
12
The researcher utilized qualitative methodology. There was no intent to utilize a
quantitative approach. The phenomenological study was designed to gain a better understanding
of the phenomena and capture the subjective understanding of the interviewee’s experience.
SUMMARY
Community colleges experience increased pressure to demonstrate the value of a
postsecondary education by politicians and accreditors (Fain, 2017, March 2, 2018; HLC, June
2019). Being able to demonstrate that students are achieving learning objectives is a major
component of institutional accountability measures. Institutions are being asked to provide
evidence of student performance at the course, program, and institutional level (HLC, nd). Only
faculty have the capability to effectively complete these evaluations. The efficacy of student
learning assessment rides on faculty engagement in the process. Recognizing, understanding, and
embracing the emotional journey that assessment is for faculty will enhance administrator and
faculty ability to foster innovation in the classroom, leading to increased quality learning
experiences for students. However, the many literature citations available to improve assessment
practices do not explore this phenomenon.
In light of this gap in the literature, the primary goals of this study was to capture faculty
stories to identify if faculty experience emotions during the assessment of the student learning
process, discovered what types of emotions are experienced, and identified if certain points in the
implementation process trigger specific types of emotions. The researcher utilized a qualitative
methodology to establish a framework for increased awareness of the emotional barriers that
may inhibit faculty motivation and participation.
Chapter One provided an introduction and overview of the purpose of the study. Chapter
Two presents a review of the related literature exploring the current landscape for assessment of
13
student learning. Chapter Three outlines the research design and methodology of the study, the
instrument used to gather the data, the procedures followed, and parameters for sample selection.
Chapter Four will present a discussion of the findings and analysis of data collected. Chapter
Five contains a summary, conclusion, reflection, and recommendations for future research.
14
INTRODUCTION
It is widely understood that the real promise of student learning assessment depends
significantly on faculty embracing the process (Hutchings, 2010). The heart of assessment is
answering the question of whether or not students are learning what is being taught. This is a
question that can only be answered by faculty (Hutchings, 2010). Student learning revolves
around faculty effectiveness, and faculty are more effective when using techniques that foster
authentic assessment of student learning (Driscoll & Wood, 2007). It has been reported that
faculty behavior changes when properly trained to use effective assessment techniques.
However, lack of faculty support of assessment efforts continues to be a barrier for assessment of
student learning (Banta, 2004a). Guetterman and Mitchell (2015, p. 45) speculate “that the roles
of both senior administrators and faculty leaders are integral to successful assessment practices”
(p. 45).
Assessment of student learning goes beyond collecting data. The most critical aspect of
assessment is evaluating the results of assessment to understand and improve student learning.
Regardless of the institution’s approach to assessment, the success of this step in the process
hinges on faculty motivation and participation (MacDonald et al., 2014).
Teaching is recognized as an emotional practice, and the literature touches on faculty
emotional response to assessment; however, it falls short of making a connection between faculty
emotion and assessment of student learning (Hargreaves, 2000; Steinberg, 2008). Steinberg’s
15
(2008) review of the literature as it relates to assessment as an emotional practice for secondary
teachers reported that further exploration of the correlation between assessment and teacher
emotional response is necessary to gain a true understanding of the impact that teacher emotion
may have on assessment.
This literature review provides a brief history of assessment of student learning and
demonstrates connections between authors who draw theoretically upon the same ideas in regard
to faculty ownership of the assessment process, efforts to create a culture of assessment on
campus, teacher emotions and vulnerability, and how emotions factor into workplace motivation.
The review sets the foundation for why the study is needed when considering the limited
availability of data and research regarding the emotional experience of faculty during the
assessment process.
Brief History of Assessment of Student Learning
The Carnegie Foundation for the Advancement of Teaching is cited as being the founder
of student learning assessment. The first president of the foundation, Henry Pritchett, was
motivated by his concern for the caliber of higher education and recognized the potential impact
that objective testing may bring to monitoring quality (Shavelson, 2007). The assessment of
student learning evolved through four eras:
• 1900–1933: the origin of standardized tests of learning. Led by the Carnegie Foundation, this era spawned the use of multiple-choice tests to measure learning.
• 1933–1947: the assessment of learning for general education and graduate education. This era was witness to the first attempt to measure personal, social and moral outcomes of general education, in addition to the traditional cognitive measurements. The Graduate Record Examination (GRE) emerged in this era as well.
• 1948–1978: the rise of test providers. Fueled by the increased demand resulting from the G.I. Bill of Rights, this era led to the creation of the Educational Testing Service (ETS) and the American College Testing (ACT) program.
16
• 1979–Present: the era of external accountability with political pressure taking hold by the end of the 1970s, resulting in a need for higher level assessment of student learning. (Shavelson, 2007, p. 5)
It became apparent in the late 1970s that objective and standardized testing was
incompatible with the way faculty members wanted student learning to be assessed. Instructors
preferred open-ended, holistic, and problem-solving assessments to the highly structured
multiple choice test. According to Shavelson (2007), “student performance varies considerably
depending upon whether a task is presented as a multiple-choice question, an open-ended
question, or a concrete performance task” (Shavelson, 2007, p. 12).
It is this understanding that a culture of evidence does not automatically lead to
educational improvement that has spawned the current era of outcomes-based assessment for
student learning. A true outcomes-based educational model includes three primary
characteristics:
2. Students directing their learning efforts to meet outlined expectations.
3. Student progress and completion measured by the level of achievement of learning expectations (Driscoll & Wood, 2007).
While instructors inherently wish for their students to achieve success in the classroom,
assessing student learning continues to be a challenge (Serban & Friedlander, 2004). First,
faculty remain unconvinced that assessment of student learning has a beneficial impact on
student success. Second, instructors who teach in higher education are typically content experts
and have not received formal training on assessment. There is a lack of knowledge regarding
assessment processes, tools, and methodology. Third, effective assessment of student learning
requires faculty across the discipline to come to a consensus about what successful student
achievement looks like, and this consensus is difficult to acquire. Finally, implementing and
17
(Serban & Friedlander, 2004).
According to Caudle and Hammons (2017), assessment of student learning in a two-year
open-door institution is especially problematic. The open access model of community colleges
results in multiple missions, leading to multiple outcomes and expectations to be assessed for
student learning. This means that faculty at community colleges require more time to complete
assessment than their peers teaching at four-year institutions. This is significant because faculty
at two-year institutions typically carry a higher teaching load then their colleagues at four-year
institutions, leaving them with limited availability to focus on student learning assessment
(Caudle & Hammons, 2017).
The assessment of student learning process cycles through three steps:
1. Establishing learning goals that clearly define what students should be able to do when they complete their education, at the course level, program level, and institutional level.
2. Collecting data to measure how well students are achieving the goals and what factors are influencing their learning.
3. Using the data collected to inform decisions to improve student learning (Walvoord, 2010).
The assessment process is intended to be circular in nature with step three informing updates and
revisions to learning goals, along with changes in teaching methods, assessment tools, and
curriculum (Walvoord, 2010).
The assessment process mirrors W. Edwards Deming’s Plan, Do, Study, Act (PDSA)
model for quality improvement that has been adopted by business and industry since its original
development in 1950. The original model was an adaptation of the scientific method to fit the
18
cyclical nature of industry practices. Since then, the model has gone through some evolutions,
landing on the most widely accepted model that was developed in 1986 by Deming after
modifications to the initial model were adapted to fit Japanese manufacturing practices (Moen,
2010). The PDSA model serves as a guide for determining what goals should be accomplished,
what actions should be taken to achieve the goals, and how to determine if the action resulted in
improvements (Moen, 2010).
Faculty Engagement in the Assessment Process
The notion of faculty buy-in is a common theme in the literature and is reported to be the
number one barrier to developing a comprehensive culture of assessment. Many researchers
report that the key to faculty engagement is faculty ownership of the process (Banta, 2004a;
Caudle & Hammons, 2017; Guetterman & Mitchell, 2015; Serban & Friedlander, 2004; Wang &
Hurley, 2012). Many institutional strategies are implemented with a top-down approach, are a
result of external influence, and tend to fail because leadership does not allow time for faculty to
gain ownership of the process (Welsh & Metcalf, 2003). Increased accountability has led to a
new paradigm of assessment that calls for dual responsibility for learning in which both the
student and the instructor collaborate to forge a learning-centered relationship (DeBoy,
Monsilovich, & DeBoy, 2013).
In an attempt to determine how faculty achieve optimal motivation to participate in
effective assessment practices, Guetterman and Mitchell (2015) conducted a study of 26 faculty
leaders teaching general education courses at eight different undergraduate colleges and
universities. Their study included an analysis of assessment attitudes of the participants before
and after a three-step training process. Their training included emphasis on the importance of
assessment, in addition to offering best practices for how to implement and use assessment
19
results to improve student learning. While their study resulted in increased personal disposition
regarding assessment, it failed to dig deeper into the emotional factors that may have been an
influence (Guetterman & Mitchell, 2015). Further, the Guetterman and Mitchel study was a
survey of motivation and attitude as a result of a professional development workshop, rather than
a study of the assessment experience itself.
Fostering faculty buy-in tends to fall into two primary strategies: holding faculty
accountable by requiring assessment of student learning to serve as a measure of faculty
performance (Gardner et al., 2014; Suskie, 2015) and, secondly, pleading to faculty’s passion for
student success, educating them on the value of assessment, and supporting faculty with
professional development (Banta, 2004b; Driscoll & Wood, 2007; Fuller, 2013; Hutchings,
2010; Lane et al., 2014; Serban & Friedlander, 2004; Wang & Hurley, 2012; Welsh & Metcalf,
2003).
Suskie (2015) discusses “five dimensions of quality” for supporting accountability and
accreditation efforts on campus. Dimension five, “a culture of better,” focuses on efforts to
support changes, improvement, and innovations in the classroom, including assessment of
student learning techniques. Her guidance outlined methodology to link assessment of student
learning, supported by evidence, as an accountability measure included in faculty performance
evaluation, promotion, and tenure decisions. She argued that accountability is necessary to
reinforce a culture of quality on campus (Suskie, 2015). To further support a culture of
accountability, it is recommended that leadership collect and analyze evidence that assessment of
student learning is happening on campus; ensure that the institution’s strategic plan supports an
assessment culture; continuously promote the culture (using artifacts such as mottos, taglines,
20
etc.); make expectations clear during the hiring process; promote professional development for
assessment of student learning; and incorporate accountability by allocating resources based on
evidence and results of student learning assessment (Gardner et al., 2014).
Understanding the Value of Assessment
Institutional efforts for assessment of student learning depend on the assessment and
improvement of student learning at the classroom level. Faculty frequently question the value of
this exercise, citing infringement on academic freedom, limiting autonomy to choose lesson
plans, and mandating methods of evaluation of their students (MacDonald et al., 2014). Faculty
may resist the process when they perceive that it is being imposed on them by administration or
accrediting bodies and at times, faculty may question the motivation behind mandated
assessment, believing that the true purpose is to serve as a tool for scrutinizing their classroom
practices. Finally, faculty may view assessment as just another fad that adds additional workload
onto their already busy schedules (MacDonald et al., 2014). Faculty ownership of assessment
improves the viability of the assessment process and results in a meaningful and lasting process
(MacDonald et al., 2014; Serban & Friedlander, 2004). As faculty ownership takes hold, they
become motivated to be engaged in the process and are more likely to sustain the practice into
the future (Serban & Friedlander, 2004). “A greater voice in the assessment process results in the
faculty taking more of a personal interest in the performance of the students and greater
receptivity towards making changes to improvements in student learning” (Lane et al., 2014, p.
4).
Serban and Friedlander (2004) reported that there continues to be a lack of knowledge
about assessment of student learning techniques, tools, and evaluation. Few faculty have
received formalized training in developing measurable and valid learning outcomes, aligning
21
their curriculum with outcomes, selecting appropriate tools for measuring outcomes, and
evaluating the results of the assessment in a meaningful way (Driscoll & Wood, 2007;
Hutchings, 2010; Serban & Friedlander, 2004). Further, faculty struggle to determine the level of
ability or knowledge that students should attain to meet outlined learning expectations and to
build a common understanding of what learning outcomes mean (Driscoll & Wood, 2007;
MacDonald et al., 2014; Serban & Friedlander, 2004). Participation in the practice of developing
student learning outcomes is enlightening for faculty and results in having a greater appreciation
for the assessment process (Driscoll & Wood, 2007). Banta (2004a) further explains that student
success in the classroom hinges on the effectiveness of the instructor. Her study found that most
instructors who were formally trained in effective use of assessment had changed their behavior
in the classroom and were more likely to embrace the assessment process. Banta (2004b) also
points out that engaging in professional development appeals to the faculty’s natural curiosity to
learn about best practices in the classroom in support of helping their students learn. This is a
concept that is echoed by Driscoll and Wood (2007). They argue that when faculty apply their
intellectual curiosity to answer questions regarding student learning, they are unknowingly
aligning their teaching with assessment expectations. Institutional investment in resources, such
as investing in professional development opportunities for faculty, sends a message that
assessment of student learning is a valued initiative for the college (MacDonald et al., 2014).
A slightly different approach in faculty education is a focus on teaching faculty about the
importance and value of assessment, rather than focusing on teaching faculty how to incorporate
assessment of student learning in the classroom. MacDonald et. al. (2014) reported that many
faculty do not see the relevance or usefulness of performing assessment of student learning.
Fuller (2013) conducted a survey of faculty questioning what they believe the purpose of
22
assessment is. He identified five primary responses: to improve student learning, accreditation,
accountability expectations, compliance with government requirements, and tradition. Less than
half of his survey respondents indicated that a focus on student learning was the primary purpose
for assessment (Fuller, 2013). Fuller argued that the message to faculty should focus on the
interconnectedness between assessment of student learning and teaching and learning in the
classroom (Banta, 2004b; Hutchings, 2010). The Middle States Commission on Higher
Education (MSCHE) cautions that an intense focus on assessment should not turn into a focus on
data collection, but rather on teaching and learning (MSCHE, 2007). Embedded assessment
should naturally grow out of the teaching and learning that are happening in the classroom.
When faculty value assessment as a scholarly practice, they are more likely to embrace the
practice and consider it a valuable use of their time (Wang & Hurley, 2012). Additionally,
faculty who see and understand the results of their assessments are more likely to support the
process (Banta, 1997).
Creating a Culture of Assessment
Many researchers note that the most effective way to garner faculty engagement in
assessment is to create an atmosphere that is receptive, supportive, and enabling (Banta, 1997;
Driscoll & Wood, 2007; Guetterman & Mitchell, 2015; Lakos & Phipps, 2004; Lane et al., 2014;
Ndoye & Parker, 2010). There is a lack of consensus of what a culture of assessment looks like,
however. Some researchers point to a culture of assessment as a final result that fosters
acceptance of assessment practices, while others deem the establishment of a culture of
assessment as a catalyst for institutional change (Guetterman & Mitchell, 2015). According to
Lane et al. (2014), a culture of assessment is one in which the concept of assessment is widely
understood by faculty and staff with an intentional focus on continuous quality improvement. It
23
has been defined as “an organizational environment in which decisions are based on facts,
research, and analysis, and where services are planned and delivered in ways that maximize
positive outcomes and impacts for customers and stakeholders” (Lakos & Phipps, 2004, p. 352).
Common strategies for building a culture of assessment include strong institutional
leadership support, embedded assessment in daily practice, use of assessment data to drive
improvement decisions, and cross communication (Ndoye & Parker, 2010). A culture of
assessment is centralized on the concept of trust: trust between faculty and administration, and
trust between faculty and student (Driscoll & Wood, 2007). Other qualities necessary for a strong
campus culture are openness, mutual respect, responsiveness, collaboration, enjoyment, and
empowerment (Driscoll & Wood, 2007).
In a survey on building and sustaining a culture of assessment that was administered in
2007, Ndoye and Parker (2010, p. 32) noted that 76% of institutions that have reported
successfully achieving an assessment culture indicate that faculty engagement was paramount.
Fostering faculty engagement includes understanding, anticipating, and addressing faculty
concerns early in the process. Oftentimes, the concerns faculty raise are those that administration
have the capacity to address, such as the lack of time to devote to the practice, the feeling that the
process is being imposed on them, the belief that assessment is an administrative responsibility,
and the belief that they are already assessing student learning by assigning grades (Banta,
2004b). Faculty must feel as though assessment is not just another fad the institution is
following. Institutional support and commitment from all levels is essential (Banta, 2004b), with
visible and collaborative partnership between faculty and administration that incorporates strong
faculty leadership. Faculty need to be engaged in establishing institutional structures for
facilitating assessment across the campus (Morse & SantiagoJr, 2000), as faculty tend to trust
24
their peers and are more likely to engage in the process if they see their colleagues embracing the
practice (Banta, 2004b).
A Sense of Vulnerability
Driscoll and Wood (2007) noted that the level of vulnerability that faculty experience
during the assessment process is stunning. Faculty have reported the feeling of exposure and the
fear of risk when evaluating assessment of student learning and experiencing a level of anxiety
for not knowing how their students are performing in comparison to others (Driscoll & Wood,
2007). Additionally, faculty have voiced concerns that assessment is really a tool used implicitly
to evaluate their performance, to justify existence of their course, program, or department, or to
determine allocation of resources. Faculty who feel that they are being scrutinized, that
assessment is infringing on their academic freedom, or that they need to use assessment to prove
their value as an instructor, will become resistant to practicing assessment in their classrooms
(Banta, 2004b; MacDonald et al., 2014). Further, Garfolo and L’Huillier (2015) note that
assessment is a practice in self-evaluation that is personal and private for faculty, and Matthews
(July 6, 2017) reported that teachers resist change in the classroom for fear of looking stupid in
front of their students.
MacDonald et al. (2014) recognize that instructors need to find value in the practice of
assessment in order to feel motivated to participate in the process. Through their exploration, the
authors discussed how the failure to understand the intrinsic value of assessment may be a
contributing factor to faculty’s lack of motivation. They failed to dig deeper to determine if
assessment may have an emotional impact on the instructor. This study, however, revealed that
insecurity regarding ability to assess student learning effectively was a leading contributor to
faculty’s lack of motivation (MacDonald et al., 2014). This particular study gives us a glimpse at
25
a sense of vulnerability that may come into play when assessing student learning in the
classroom.
Understanding Teacher Emotions and Motivation
While research is starting to take a closer look at teacher emotion, the studies are
structured around the interpersonal aspects of the teacher role. Zembylas (2003) stated that “an
important feature of teacher emotional experience is the role of discourse in the classroom/school
setting in which a teacher works, and how this discourse influences the construction of
emotions” (p. 105). Zembylas reported in 2003 that there was little research that studied teacher
emotions, and this assessment remains true 15 years later. He attributes this to three factors:
Western culture has an embedded bias against emotion in the workplace, it is challenging to
study teacher emotion with objective measures, and emotions have primarily been linked with
female employees and subsequently ignored in the overall picture. Zembylas (2003) goes on to
explain that emotions in teaching have come to influence school policy issues and that
administrators and policymakers are beginning to accept that teaching is an emotional practice.
Hargreaves (2000) noted that education reform neglects the emotional dimension of
teaching: “Teaching, learning and leading may not be solely emotional practices, but they are
always irretrievably emotional in character” (Hargreaves, 2000, p. 812). Hargreaves (2000)
concluded that teaching is an emotional practice and that teacher emotion may raise classroom
standards or lower them. Oftentimes, emotions are acknowledged only as something that should
be managed when leading institutional change or to assist administrators in dealing with teacher
resistance to strategic plan implementation (Hargreaves, 1998).
Teachers tend to formulate pedagogy to accommodate their own emotional needs.
Emotions such as excitement and joy tend to lead pedagogy decisions, along with emotions that
26
are linked to a sense of creativity and achievement in teaching with students. Oftentimes,
teachers struggle when letting go of familiar practices and comfortable routines (Hargreaves,
1998).
Restructuring of the teacher role leads to increased occupational stress that may lead to a
feeling of “burnout,” which is defined as the feeling of physical, emotional, and mental
exhaustion (Hargreaves, 1998; Jepson & Forrest, 2006). Teacher burnout is becoming a
prevalent issue in the profession. Some contributing factors for teacher stress include heavy
teaching loads, overload of roles outside of teaching, classroom management, and anxiety over
evaluations (Jepson & Forrest, 2006). The Jepson and Forrest (2006) study indicated that those
who strive for a higher level of achievement tend to experience a greater level of stress.
According to Steinberg (2008), “assessment decisions are not ‘neutral’ but involve
teacher’s emotions” (p. 42). Steinberg conducted a review of the available research and literature
at the time and concluded that teachers “grapple with emotional complexity” (p. 58), but the field
remains under-researched. While the literature addressing teachers’ emotions is primarily
focused on secondary teachers, it is the belief of this researcher that these findings may be
applied to faculty teaching in higher education as well. Faculty emotions are a critical component
to curriculum development and teaching (Zembylas, 2003), and education policies, reforms, and
faculty professional development should take into account the role of emotions.
Sujitparapitaya (2014) completed a study to determine why some faculty are motivated to
participate in assessment of student learning and some are not. He concluded that faculty are
more likely to engage in assessment if they possess the confidence to be successful and if they
feel that it will be a valuable practice. His study, however, failed to dive deeper into the
emotional factors that may influence faculty motivation. Hargreaves (1998, 2000), Jepson and
27
Forrest (2006), and Zembylas (2003, 2007) all reported that teaching is an emotional practice.
Their studies focused primarily on the interpersonal interactions with students in the classroom at
the secondary level. Pedagogy is personal in nature, as teaching is a personal craft, and it can be
isolating. As a result, teachers are often hesitant to accept that they may need to change their
approach or revamp the curriculum (Hernandez, 2016).
Driscoll and Wood (2007) discussed the importance of empowerment and enjoyment as
key factors in building a strong culture of assessment across campus. Pink (2009) explained that
traditional reward models for motivating employees are not effective. Rather, employees need to
feel a sense of purpose for the work that they do. The most motivated people couple their efforts
with a cause that is larger than themselves. Having a sense of purpose is an emotional link to the
overarching goal. Sinek (November 2012) explained that humans are social animals and that our
survival depends on our ability to form cultures founded in trust. Trust is an emotion that leads to
a willingness to experiment, fail, and innovate (99U, November 2012), and it emerges when we
surround ourselves by those who believe what we believe. The assessment literature speaks to
this in some respect when the discussion turns to the importance of faculty buy-in to effectively
create a culture of assessment on campus. Serban and Friedlander (2004) summarize this well:
As faculty ownership becomes apparent in the assessment process, faculty are motivated to remain engaged and their interest is more likely to be sustained over the years that follow, even after the accreditation self-study has come and gone. (p. 33)
Many researchers reported that the key to faculty engagement is faculty ownership of the
process (Banta, 2004a; Caudle & Hammons, 2017; Guetterman & Mitchell, 2015; Serban &
Friedlander, 2004; Wang & Hurley, 2012). The literature fell short of demonstrating the notion
that emotions have an integral role in faculty motivation; however, Sinek maintains that people
want to be a part of something bigger than themselves and that experiencing a sense of
28
fulfillment leads to wanting to do more (99U, November 2012). The notion of understanding
human emotions to understand human behavior was supported by Ford (1992, p. 201) as he
explained that the true essence of motivating behavior is having a sound understanding about
“the role of motivational processes in effective functioning”. He goes on to argue,
The strategy of trying to motivate people through direct control of their actions – as opposed to indirect facilitation of their goals, emotions, and personal agency beliefs – should be reserved for situations in which swift attainment of a goal is urgent and no other means are available. (Ford, 1992, p. 203)
SUMMARY
This chapter reviewed current literature related to faculty buy-in of assessment of student
learning, establishing a culture of assessment, and teaching as an emotional practice. As
assessment of student learning has evolved over the years, faculty support of the assessment
process has remained a barrier for effective practice. Researchers offer two different pathways to
gaining faculty buy-in: incorporating accountability measures in faculty performance
evaluations, or helping faculty understand the intrinsic value of the assessment process.
The literature has shown that teaching is recognized as an emotional practice. The
process of teaching tends to be done in isolation and is considered to be a personal craft, making
it challenging for teachers to feel comfortable with external influences that may impact the
curriculum. Researchers reported that the key to faculty engagement is faculty ownership of the
process.
Correlations were found in the literature describing the necessity to understand human
emotions in the workplace and establishing a culture of assessment on campus. People are social
by nature and tend to surround themselves with others who believe what they believe, forming
cultures founded in trust. Trust is an emotion that increases one’s willingness to take risks and to
innovate. The most motivated employees have a sense of purpose for the work they do and
29
believe that their efforts are supporting a cause that is bigger than themselves. The literature
hinted at the emotional factors that may influence these practices, however, it failed to dive
deeper into the emotional factors that may influence faculty motivation. Further exploration of
the correlation between assessment and teacher emotional response is necessary to gain a true
understanding of the impact that teacher emotion may have on assessment.
30
INTRODUCTION
This study was designed to gain a better understanding of the faculty experience of
assessing student learning in the classroom. It investigated what emotions faculty experienced
throughout the design, implementation, and continuation of assessment practices that measure
the top three tiers of Bloom’s taxonomy (analysis, evaluation and creation).
For years, the use of multiple choice and true/false exams as the assessment tools of
choice have led to a push for what educators call authentic assessment, with directed efforts to
capture alternative evidence of student learning. Further, authentic assessment provides an
opportunity for students to integrate personal experiences with their academic learning (Driscoll
& Wood, 2007). Based on the principle of active learning, assessments should be designed to
appropriately align with Bloom’s taxonomy (Morton & Colbert-Getz, July 2016).
Bloom’s taxonomy was developed in the 1950s to categorize the level of cognition for
student learning goals. The taxonomy moves from basic level of cognition (knowledge) to
highest level of cognition (evaluation), with four levels in between (comprehension, application,
analysis, synthesis). In 2001, the taxonomy was revised to remember, understand, apply,
analyze, evaluate, and create (Krathwohl, 2002). “Teaching methods that provide opportunities
for active learning should increase the student’s ability to solve problems at a higher cognitive
level based on Bloom’s taxonomy” (Morton & Colbert-Getz, July 2016, p. 173). For this reason,
31
this study was designed to focus on the experience of faculty who have implemented assessment
techniques that measure a higher level of cognition.
Sinek states that “if you don’t understand people, you don’t understand business” (99U,
November 2012). He goes on to explain that humans are social animals and our survival depends
on our ability to form cultures. Trust emerges when we surround ourselves by those who believe
what we believe. Trust is an emotion that leads to a willingness to experiment, fail, and innovate
(99U, November 2012). The assessment literature speaks to this in some respect by emphasizing
the importance of faculty buy-in to effectively create a culture of assessment on campus. Serban
and Friedlander (2004) summarize this well:
As faculty ownership becomes apparent in the assessment process, faculty are motivated to remain engaged and their interest is more likely to be sustained over the years that follow, even after the accreditation self-study has come and gone. (p. 33)
Many researchers reported that the key to faculty engagement is faculty ownership of the
process (Banta, 2004a; Caudle & Hammons, 2017; Guetterman & Mitchell, 2015; Serban &
Friedlander, 2004; Wang & Hurley, 2012). The literature falls short of the notion that emotions
play a role in achieving faculty ownership. Sinek states that people want to be a part of
something bigger than themselves and experiencing a sense of fulfillment leading to the desire to
do more (99U, November 2012). This concept of understanding human emotions to understand
human behavior is supported by Ford (1992) as he explained the true essence of motivating
behavior by having a sound understanding regarding “the role of motivational processes in
effective functioning” (p. 201). He goes on to argue,
…the strategy of trying to motivate people through direct control of their actions – as opposed to indirect facilitation of their goals, emotions, and personal agency beliefs – should be reserved for situations in which swift attainment of a goal is urgent and no other means are available. (p. 203)
32
How emotions may factor into the faculty experience of assessing student learning gives
interest to this research. It is the hope of the researcher that this study may provide insight into
and an increased awareness of the faculty experience while employing techniques for assessing
student learning in the classroom.
Population Sample
The research population includes full-time tenured and non-tenured faculty who teach at
a mid-sized or large community college in the State of Michigan. While adjunct faculty have
become the mainstream teaching workforce in community colleges, “they have less frequent
interactions with students and are less integrated into the campus cultures in which they work”
(Jolley et al., 2013, p. 219). For this reason, it is believed that inclusion of adjunct faculty would
change the dynamic of the study.
The researcher limited the study to the State of Michigan where the researcher lives,
allowing for greater accessibility to the sample population. Seidman (2013) explains that the
purpose of an interview study is to understand the experience of those who are being
interviewed; it is not to test a hypotheses. Therefore the findings of this interview study are not
meant to be generalized to a broader population, but rather to present the experience of the
interviews in compelling detail and sufficient depth to allow the readers to connect with that
experience (Seidman, 2013).
Probability and nonprobability sampling are the two most common approaches for
selecting a research sample. Probability sampling is a statistical approach that allows for
generalization of the study results. Since this is not the goal of a qualitative study, nonprobability
sampling is recommended (Merriam & Tisdell, 2016). The researcher used purposive sampling,
the more common form of nonprobability sampling. Purposive sampling is appropriate when the
33
researcher wishes to gain insight and understanding, and therefore would benefit most from
selecting a sample from which the most may be learned (Merriam & Tisdell, 2016).
Participants
Michigan is home to 28 public community colleges with a collective student enrollment
of 411,764 (WorkforceDevelopmentAgency, February 2015) and employing 2,672 full-time
instructional faculty (IPEDS, nd). The researcher selected participants from seven community
colleges within the State of Michigan that for academic semester Fall 2016 employ more than
150 full-time faculty as reported in the Integrated Postsecondary Education Data System
(IPEDS). The researcher collaborated with assessment directors at these institutions to identify
full-time faculty members who met the study participant criteria:
• Full-time tenured or non-tenured faculty teaching at a mid-size or large public community college in Michigan.
• Have implemented within the past three academic years (2015–2016, 2016–2017, 2017–2018), or are currently implementing, an assessment technique that addresses one of the top three tiers of Bloom’s Taxonomy (analysis, evaluation, and creating).
Each institutional assessment director recommended three to five full-time faculty who
met the criteria. From those recommendations, the researcher invited two individuals from each
institution to participate. A letter of informed consent was sent to each individual who agreed to
participate in a qualitative research study. The researcher contacted potential participants in
person, by phone, or email to briefly explain the study, including explaining that each interview
was to be audio recorded and giving an outline of the research schedule, and to determine their
interest in participating.
Once an individual agreed to participate, a mutually agreed-upon time and location for
the interview was scheduled and the informed consent process initiated. The researcher provided
34
the participant with a copy of the written informed consent form in advance of the interview.
During the informed consent process, the researcher described to each participant the potential
risks and benefits from participation, as well as the measures that were taken to minimize risk
and maximize benefits. The participant was given an opportunity to ask any questions prior to
signing the form. The participant signed, scanned, and emailed the form back to the researcher,
or provided verbal consent. The researcher kept the signed form, indicating agreement to
participate in the study, and participants were provided a copy of the signed form for their
records. Prior to the interview, the researcher emailed participants a copy of the signed informed
consent. Prior to the start of the interview, the researcher reviewed the important aspects and
procedures.
Interview Process
Researchers often choose qualitative study design because they are interested in gaining
insight, discovery, and interpretation, rather than hypothesis testing. Qualitative in-depth
interviewing is the primary method of data collection when the goal is to capture the essence and
basic structure of an experience (Merriam & Tisdell, 2016). The researcher’s objectives for this
study were to follow a line of inquiry as reflected by the qualitative research design and to ask
questions in an unbiased manner. The design of a phenomenological qualitative study in
particular is well suited for the study of emotions and guides the researcher in capturing a
person’s subjective understanding of his or her experience (Seidman, 2013). Seeking to
reconstruct the participant’s experience from a subjective point of view requires the researcher to
reflect on their own assumptions and biases. It is common practice for researchers to set aside
these influential factors prior to embarking on a study in order to capture the true essence of the
participant’s experience (Merriam & Tisdell, 2016; Seidman, 2013).
35
The interviews were designed to explore what emotions may have been experienced by
faculty during the assessment process and to inductively build an understanding of the faculty
experience rather than to test concepts and theories of the researcher (Merriam & Tisdell, 2016).
Therefore, the interviews were semi-structured, with many of the questions asked in the
interview developing as the interviewees shared their experiences. Seidman (2013) explains that
interviewing is a relationship and, as such, each interview is individually crafted. The interviews
were designed to be guided conversations to elicit the interviewee’s story. The unstructured
discussions gave the participant an opportunity to expound on experiences he or she believed
would enhance the researcher’s understanding of their experiences.
One visual aid (Appendix A) portraying Bloom’s Revised Taxonomy was employed to
assist participants with defining the level of assessment utilized in the experience discussed. The
visual aid provided a visual hierarchy for each level of Bloom’s Taxonomy, with supporting
verbs to clarify the learning activities students are expected to exhibit during each level
(MacMeekin, June 5, 2014).
The standard three-step assessment process (establishing goals, collecting data, and
taking action) (Walvoord, 2010) correlates closely with W. Edward Deming’s Plan, Do, Study,
Act (PDSA) model for continuous quality improvement. The PDSA model has been widely
accepted by business and industry for three decades and serves as a comprehensive guide for
implementing and evaluating quality improvement processes (Moen, 2010). For this reason, the
researcher chose the PDSA model as a guide for the study. A visual aid was used (Appendix B)
to guide interviewees through the PDSA cycle while discussing the implementation of their
assessment tools.
Published literature, consultation with assessment and research experts, and the
researcher’s own experiences informed the architecture of the interview questions. The semi-
structured interviews were conducted by video or in-person and digitally recorded. The
researcher asked participants to share their experiences and strategies by responding to a series of
open-ended interview questions related to creating and implementing an assessment of student
learning technique. The researcher asked follow-up questions during the interview to encourage
elaboration and clarification. Interviews were digitally recorded for accuracy and then
professionally transcribed after the interviewee read and signed an informed consent form
(Appendix D). Recordings began when both the researcher and the interviewee were seated, and
the consent form was collected. An overview statement outlining the purpose of the study and
discussion of procedures for the interview were given prior to beginning each interview session
(Appendix C). Interviewees were encouraged to add commentary or ask questions for
clarification at any point. Interviewees were also given the option to stop the interview at any
time.
To analyze the interviews, files were professionally transcribed by Peterson Transcription
& Editing Services, LLC. The transcriptionist signed a confidentiality agreement (Appendix F)
and did not know the location of the study sites to reinforce the confidential nature of the study.
After each transcript was completed, the participant’s name in the text files was replaced with a
pseudonym to ensure confidentiality. These pseudonyms were used for the duration of the study,
and only the researcher knows the identity of each participant.
Each interview transcript was carefully reviewed by the researcher. Inductive analysis
was used to identify patterns, categories, and themes that led to a definitive interpretation, and
37
the constant comparative method was used throughout the research process. The interviewee was
allowed an opportunity to review the transcript and offer follow-up thoughts for clarification.
Thorough review and analysis of data using the words of the participants, along with a review of
the researcher’s field notes, allowed additional consideration for the interviewee’s meaning
before filtering the data into emerging themes. The researcher connected and identified the
various themes and common descriptions. Broad themes were refined to eliminate redundancy,
and the ultimate use of a limited number of themes provided a clear description of the
experiences discovered in this study.
Data Analysis
The methodology chosen was a qualitative research design. Qualitative research relies on
words as data. This differs from a quantitative research design that uses numbers as data that are
analyzed with statistical techniques (Merriam & Tisdell, 2016). The collection of data and
analysis of data happen simultaneously in a qualitative study: “The final product is shaped by the
data that are collected and the analysis that accompanies the entire process. Without ongoing
analysis, the data can be unfocused, repetitious, and overwhelming in the sheer volume of
material that needs to be processed” (Merriam & Tisdell, 2016, p. 197).
The theoretical framework is based on the literature related to the topic as discussed in
Chapter Two. The framework draws upon concepts, terms, and definitions cited in the literature,
particularly in regard to faculty motivation and willingness to embrace the assessment process.
For years, the use of course examinations as the assessment tool of choice resulted in a push for
authentic assessment, with directed efforts to capture the students’ true learning experience
(Driscoll & Wood, 2007). The literature discussed strategies and best practices that
administrators may adopt to encourage a culture of authentic assessment on campus. Lack of
38
faculty buy-in continues to be the number one factor inhibiting comprehensive assessment of
student learning for most institutions. This concern has permeated for decades, despite the
countless number of resources, literature, conferences, and professional development
opportunities offering strategies and best practices to overcome the barrier. The strategies
emphasize the importance of encouraging faculty engagement by fostering an environment that
encourages authentic faculty ownership of the process (Banta, 2004b; Caudle & Hammons,
2017; Guetterman & Mitchell, 2015; Serban & Friedlander, 2004; Wang & Hurley, 2012). The
qualitative data collected through semi-structured interviews with study participants offers
insight into the emotional landscape that may accompany the assessment of student learning
process, shedding light on a potential root cause for faculty resistance that has not been explored.
A phenomenological study is designed to gain a better understanding of a phenomena and
capture the subjective understanding of the interviewee’s experience (Seidman, 2013). This
study used a phenomenology approach with participants employed as full-time instructional
faculty by one of the public community colleges in the State of Michigan. Faculty were selected
based on two criteria: (1) full-time tenured or non-tenured instructional faculty teaching at a mid-
sized or large public community college in the State of Michigan; and (2) have implemented
within the past three academic years (2015–2016, 2016–2017, 2017–2018) or are currently
implementing, an assessment technique that addresses one of the top three tiers of Bloom’s
Taxonomy (analysis, evaluation, and creating).
Qualitative data analysis is both inductive and comparative. To achieve this, the
researcher is tasked with comparing one unit of information with the next to identify recurring
regularities in the data (Merriam & Tisdell, 2016). This analysis is done by first isolating any
biases or assumptions that the researcher may have in order to reflect on the experience of the
39
participants and to suspend judgment. This type of analysis focuses on capturing the essence of
the participant’s story. The goal is to arrive at structural descriptions of the experience and the
underlying, precipitating factors that may contribute to what is being experienced (Merriam &
Tisdell, 2016).
Interviewing is a basic mode of inquiry. The primary purpose is not to evaluate but to
understand the lived experience of other people and the meaning they make of that experience
(Seidman, 2013). The researcher used open-ended interview questions to address the following
research questions:
• Research Question 1. Are faculty experiencing emotions while doing assessment of student learning?
• Research Question 2. What emotions are experienced by faculty throughout the implementation process?
• Research Question 3. Are different emotions experienced at different points of the implementation process?
The research samples are similar in some respects but different in others. Comparison
research may be used as a method for explaining or extracting understood knowledge. The focus
of this study was to identify if assessment of student learning generates an emotional experience
for faculty. The interview portion of this study focused on reconstructing the faculty’s perception
of the emotional journey that assessment of student learning generated for them. The study
examined the types of emotions experienced at different stages within the assessment
implementation process.
The interview guide (Appendix E) and visual aids (Appendices A and B) were used as
the primary instruments for collecting data. The researcher’s field notes were supplemental data
collected during the study. The interview guide consisted of five questions to elicit the
participants’ stories. The visual aids were used as a guide to define emotions experienced at each
40
point of the assessment process and to measure the intensity of those emotions. Visual aids
served as a point of reference for faculty to identify the level of cognitive learning the assessment
technique measured, in accordance with Bloom’s Revised Taxonomy. The interviews were
digitally recorded, and the researcher maintained field notes that included (but were not limited
to) observations, emerging trends, and thoughts to follow up on. The researcher’s field notes
were also assigned a unique number that aligned with the participants’ pseudonyms and coding
system used.
Following each interview session, the researcher listened to the audio to ensure there
were no problems with the recordings and to determine if any clarification from the interviewee
was necessary. Once the interview files were transcribed, reviewed by the interviewees, and
pseudonyms assigned, the researcher uploaded the transcripts into the software, which allowed
the researcher to apply open coding for the first level of analysis. Open coding is the process of
categorizing the data into themes that are responsive to the purpose of the study (Merriam &
Tisdell, 2016).
The coding process used the interviewees’ own language to define coding classifications,
a practice referred to as in vivo coding (Merriam & Tisdell, 2016). Using the participants’
language and terms to define the codes is a strategy to prevent bias during the coding process
that may result otherwise if the researcher were to substitute perceptions of participant meaning
during the analysis process (Merriam & Tisdell, 2016). The coded transcripts were then reviewed
and narrowed further into fewer themes and refined to eliminate any redundancy or overlap in
the themes. The final themes were then correlated to the research questions. The limited number
of themes allowed the true essence of the faculty experience to emerge.
41
Emotions articulated by the participants were categorized using a scale adopted from a
1985 Folkman and Lazarus study. In this study, the researchers evaluated emotions experienced
by students during a college examination.
From the perspective of a cognitive theory of emotion, the quality and intensity of any emotion — anxiety, jealousy, sorrow, joy, relief — is generated by its own particular appraisal. For ex