Q METHODOLOGY AS A NEEDS ASSESSMENT TOOL FOR BIOLOGY GRADUATE TEACHING ASSISTANTS PARTICIPATING IN...

© 2013

AMY B. HOLLINGSWORTH

ALL RIGHTS RESERVED

Q METHODOLOGY AS A NEEDS ASSESSMENT TOOL FOR BIOLOGY

GRADUATE TEACHING ASSISTANTS PARTICIPATING IN AN

INSTRUCTIONAL TRAINING PROGRAM

A Dissertation

Presented to

The Graduate Faculty of The University of Akron

In Partial Fulfillment

of the Requirements for the Degree

Doctor of Philosophy

Amy B. Hollingsworth

November 1, 2013

i

Q METHODOLOGY AS A NEEDS ASSESSMENT TOOL FOR BIOLOGY

GRADUATE TEACHING ASSISTANTS PARTICIPATING IN AN

INSTRUCTIONAL TRAINING PROGRAM

Amy B. Hollingsworth

Dissertation

ii

Approved: Accepted:

______________________________ ______________________________Co-Chair Department ChairJennifer L. Milam, Ph.D. Susan J. Olson, Ph.D.

______________________________ ______________________________Co-Chair/ Methodologist Dean of the CollegeSusan E. Ramlo, Ph.D. Susan G. Clark, Ph.D., J.D.

______________________________ ______________________________Committee Member Dean of the Graduate SchoolRobert Joel Duff, Ph.D. Dr. George R. Newkome

______________________________ ______________________________Committee Member DateGary M. Holliday, Ph.D.

______________________________Committee MemberJohn B. Nicholas, Ph.D.

ABSTRACT

The purpose of this study is to demonstrate how Q Methodology can be used as a

needs assessment tool for a Biology graduate teaching assistant (GTA) instructional

training program. GTAs are used as the instructors of an increasingly diverse population

of undergraduate students. GTAs are a diverse population of students with varying

amounts of pedagogical preparation, research abilities, and motivation to complete their

graduate study. They are often expected to prepare and grade exams, write their own

syllabi, design course curriculum, prepare and present lectures, monitor student progress,

hold office hours, and assign final grades, all with minimal faculty supervision. Although

not all GTAs will become professors, many will, and the teaching assistantship remains

the major preparation for their roles as faculty members. Since the majority of science

professors have been GTAs, this instructional training program is of critical importance.

Approaches to developing instructional training programs for GTAs vary from

departmental workshops to campus-wide instructional seminars. Program evaluation is an

intrinsic part of assuring that such programs best serve GTA needs, and that GTAs can

best fulfill their roles in their respective departments. Q Methodology offers a number of

potential advantages over traditional survey techniques for assessing needs of GTAs

throughout their graduate school career, allowing program supervisors to evaluate and

modify the program relative to GTA needs. Q Methodology allows the researcher to

identify and interpret various viewpoints the GTAs hold in regard to graduate school.

This is not only important to the supervisors of GTA instructional programs, but to the

GTAs.

iii

This Q Methodology study led to three GTA viewpoints (“The Emerging

Teacher,” “The Preferred Researcher,” and “The Anxious GTA”) that provide insight

about GTA and programmatic needs. Q Methodology can provide predictor profiles, or

“typologies” that are more useful than simple variables and demographic information for

the classification of people, especially within program evaluation (Newman & Ramlo,

2011). “The Anxious GTA” viewpoint, which suggests a group of GTAs who may be at

risk for failure in their degree program, may be further investigated for retention and

program completion. The results of this study will be used to consider potential changes

or updates to the existing training program that may include scaffolding, differentiation,

peer or faculty mentoring, or self-directed learning strategies.

iv

ACKNOWLEDGEMENTS

v

TABLE OF CONTENTS

ABSTRACT

ACKNOWLEDGEMENTS

TABLE OF CONTENTS

List of Tables

List of Figures

List of Definitions

Prologue

Researcher Positionality

CHAPTER I

INTRODUCTION TO THE STUDY

Introduction

Purpose of the Study

Statement of the Problem

Significance of the Study

General Research Questions

Delimitations

Summary

Chapter II

REVIEW OF THE LITERATURE

Why go to graduate school?

The Usage of Graduate Teaching Assistants in Higher Education

Teaching “Assistant” or Course Instructor?

Instructional Training Programs for GTAs

Graduate School and the Socialization of Academics

Conflicting Priorities in a Graduate School Program

National Training Programs vs. Locally Developed Training Programs

The Modern Academic Workplace

Evaluating Graduate Teaching Assistant Training Programs

Q Methodology

Summary

vi

CHAPTER III

METHODOLOGY

Introduction and Overview


Rationale for the Research Design

Basic Procedures of Q Methodology

Setting

The P-Set

The Concourse

SRQ – Self Reflection Questionnaire

The Perceptions of Graduate School Survey

Statements from the Literature

Q Sample

Q Sort

The Pilot Study

Data Collection Procedures

Role of the Researcher

Limitations

Summary

CHAPTER IV

RESULTS

Descriptive Demographics

Data Collection

Data Analysis

Analysis and Interpretation

Factor 1

Factor 2

Factor 3

Consensus Statements

Results of Testing the Research Hypotheses

General Research Hypothesis 1



vii


Summary

CHAPTER V

SUMMARY, CONCLUSIONS, AND IMPLICATIONS

Summary of the Study


Statement of the Procedures

The Research Hypotheses





Conclusions


Implications

Differentiating the Instructional Training Program

Q Methodology as a Self-Diagnostic Tool

Collective Mentoring

Promises and Challenges of Q Methodology

Suggested Further Research

Summary

References

Appendices

Appendix 1: Concourse Development

Appendix 2: Q Sample

Appendix 3: Conditions of Instruction

Appendix 4: IRB Informed Consent Letter

Appendix 5: IRB Exemption Request

Appendix 6: IRB Exemption

viii

LIST OF TABLES

TABLE PAGE

Table 1 – P-Set Demographics

Table 2 - Development of the Concourse and Q Sample

Table 3 - Demographic Characteristics of GTAs completing the SRQ

Table 4 - Demographic Characteristics of TAs Completing the “Perceptions of Graduate

School Survey”

Table 5 – Demographics of New and Experienced Biology GTA

Table 6 - Coding System for Study Participants

Table 7 - Factor Matrix with X Indicating a Defining Sort

Table 8 - Factor Values for Each Statement

Table 9 - Eight Most-Like My View Statements for Factor 1 "The Emerging Teacher"

with a † indicating a Distinguishing Statement.

Table 10 - Eight Least-Like My View Statements for Factor 1 "The Emerging Teacher"


Table 11 - Distinguishing Statements for Factor 1 " The Emerging Teacher ".

Table 12 - Post-Sort Interview Responses for Factor 1

Table 13 - Eight Most-Like My View Statements for Factor 2 "The Preferred Researcher”


Table 14 - Eight Least-Like My View Statements for Factor 2 "The Preferred

Researcher” with a † indicating a Distinguishing Statement.

Table 15 - Distinguishing Statements for Factor 2 "The Preferred Researcher".

Table 16 - Post-Sort Interview Responses for Factor 2 “The Preferred Researchers”

Table 17 - Eight Most-Like My View Statements for Factor 3 “The Anxious GTA” with

a † indicating a Distinguishing Statement.

ix

Table 18 - Eight Least-Like My View Statements for Factor 3 “The Anxious GTA” with

a † indicating a Distinguishing Statement.

Table 19 - Distinguishing Statements for Factor 3 “The Anxious GTA.”

Table 20 - Post-Sort Interview Responses for Factor 3 “The Anxious GTA.”

Table 21 - Consensus Statements – Statements in Common Amongst Factors

Table 22 – Number of Q-Sorts Included in Each Factor

Table 23 – Breakdown of Number of Q-Sorts Included in Each Factor

x

LIST OF FIGURES

FIGURE PAGE

Figure 1 - "GTA Preparedness" based upon Cho et. al.

Figure 2 - The Five Stages of GEM (based upon McNeil et al., 2005)

Figure 3 - Sample Grid

Figure 4 - Sample Grid Showing “Normalized” or Gaussian Distribution

Figure 5 – Conditions of Instruction for “GTA Perceptions of Graduate School Q Sort

Figure 6 - Distribution Grid for “GTA Perceptions of Graduate School Q Sort”

Figure 7 – Representative Sort for Factor 1

xi

LIST OF DEFINITIONS

Age Measured chronologically, in years; self-reported by participants.

Biology Lab Coordinator

A staff member in The Department of Biology in a large, research-

focus, degree granting university, whose primary duty is to supervise

Biology GTAs while teaching undergraduate Biology laboratories.

Biology Lead Faculty Member

A faculty member in The Department of Biology in a large, research-

focused, degree granting university, who directs the teaching education

of new Biology GTAs.

Career Track Following a professionally developed path towards a desired career.

Concourse The flow of communicability surrounding any topic (Brown, 1993).

The collection of all the possible statements the respondents can make

about the subject at hand (Van Exel & De Graaf, 2005).

Condition of Instruction

Provided by the researcher, this is a set of instructions, used by a

participant, for sorting the Q Sort cards from his or her own point of

view (Brown, 1993; McKeown & Thomas, 1988; Van Exel & de

Graaf, 2005).

Country of origin - United States GTAs

A graduate level student born in and primarily educated in The United

States. Self-reported.

Country of origin - International GTAs

A graduate level student born in and primarily educated in a country

other than The United States. Self-reported.

Experience, in A division constituting half of the regular academic year, lasting

1

semesters typically from 15 to 18weeks (“the definition of semester,” n.d.). Self-

reported.

Experienced Biology GTA

A graduate level student who is seeking a master’s or doctoral degree

through The Department of Biology in a large, research-focused,

degree-granting university, with more than one year of formal teaching

experience, and who teaches an undergraduate-level laboratory for

approximately 20-hours a week in exchange for a fee-remission. This

GTA has completed an "Effective Teaching" GTA training program.

Gender Self-identification with roles and expectations attributed to men and

women in a given society (Phillips, 2005).

New Biology GTA A graduate level student who is seeking a master’s or doctoral degree

through The Department of Biology in a large, research-focused,

degree-granting university, with less than one year of formal teaching

experience, and who teaches an undergraduate-level laboratory for

approximately 20-hours a week in exchange for a fee-remission. This

GTA is currently enrolled in an "Effective Teaching" GTA training

program.

Professional Development

The development of a person in his or her professional roles. More

specifically, “Teacher professional development is the professional

growth a teacher achieves as a result of gaining increased experience

and examining his or her teaching systematically” (Glatthorn, 1995, p.

41).

2

P - Set The purposefully chosen set of participants, also called the sorters, or

the respondents (Brown, 1993).

Q Methodology A methodological tool that provides an objective way to measure

subjectivity. (Newman & Ramlo, 2011; Brown, 1980; Stephenson,

1953)

Q Sample The set of statements, selected from the concourse, which represent the

communicability of the topic; the respondents will sort these

statements into a grid, based on the condition of instruction (Newman

& Ramlo, 2011; Brown, 1980; Stephenson, 1953).

Q Sort The process of distributing the Q Sample into a researcher provided

grid. The statements are administered in the form of a pack of

randomly numbered cards (one statement to a card) with which the

person is instructed to sort according to "condition of instruction

(Brown, 1993).

Teaching experience, Formal

Teaching in an educational setting such as a university or training

institution, with a set curriculum, which is leading towards a

certification or degree (Dib, 1988).

Teaching experience, Informal

Teaching that occurs alongside formal teaching, such as tutoring,

afterschool, or informal learning situations, with a flexible curriculum,

that does not lead towards a degree or certification (Dib, 1988).

Theoretical Sorting A process where a study participant sorts their statements, according to

the conditions of instruction, based upon their own beliefs of how

3

another participant would sort.

4

PROLOGUE

Researcher Positionality

In September of 2000, having just graduated from my undergraduate university with a

degree in Biology, I moved from my small hometown in North Eastern Ohio, to Eagle Pass,

Texas, a Mexican border town. Even though I had not had a single education class, I was hired at

the local high school. At the age of 22, with no formal teacher training, I began teaching an 11th

grade Chemistry class. I was expected to teach 100 primarily Spanish-speaking students,

classified “at-risk” due to low socioeconomic status. I was only two to four years older than most

of them. My degree in Biology couldn’t have begun to prepare me for teaching. I taught

Chemistry the same way I had been taught Chemistry - “chalk and talk.”

Every morning during my first period “teacher prep time,” my colleague and I would sit

down in his classroom, eat breakfast tacos made by his lovely wife, and write lectures, find

worksheets, or figure out problems. He handed me what I was going to teach for the day, every

morning. Some days, my teaching was terrible. My students were difficult to understand,

because they were so unlike me. I wondered if they were learning, and I questioned whether I

should be teaching at all. Other days, I felt breakthroughs where they “got it,” we had fun

actively engaging in the laboratories, and I counseled them concerning problems in their lives. I

would advise them on getting into college, classes with other teachers, frustrations with their

parents, or achieving their dreams. Outwardly, it appeared I was “successful at teaching.” But

were my students successful at learning?

I continued teaching high school for ten years. After completing a teaching certification

program and a Master’s Degree in Education while teaching full time, I was offered a position in

my hometown writing Biology curriculum, working with Biology graduate teaching assistants as

5

the laboratory coordinator of the Natural Science Biology lab, and teaching at the college level.

While working at the university, I could also pursue a Ph.D. I became a graduate student in

Curriculum and Instruction, working alongside graduate student TAs in Biology.

I recognized in these GTAs many of the same feelings, insecurities, frustrations, and

fears that I had as an untrained high school teacher. Just as I was expected by the school district

to become a trained secondary teacher, GTAs are expected to utilize their teaching opportunities

to transform into a college instructor – whether that is their planned career path or not. Just as I

faced my students with no instructional training, so do these GTAs. However when I taught high

school, I was expected to take pedagogy courses to train as a teacher. Those courses were

invaluable in developing my skills in instruction, engaging with students, and classroom

management. These GTAs face their own students with no formal training, little feedback on

their teaching, and a feeling of “What am I doing here?” They just hope to survive the semester.

I recognize GTAs’ struggles, and make note of the challenges they face as they work with

undergraduate students, teach the lab, work with their advisors, take their own classes, do

original research, write theses and dissertations, and attempt to juggle it all with a personal life.

Each GTA comes to me with a unique story, a different path, and an individualized perspective

on graduate school. I have observed GTAs who were paralyzed with fear each time they faced

the class as well as those who were so brazenly cocky they saw their students as “stupid

undergrads.” GTAs with a “know-it-all” attitude often ended up with their classes revolting

against them. I wish I could hand them some equation, some formula for teaching that works for

all GTAs, which would answer all their questions before they ever faced with a student of their

own. Their faculty mentors often express that “all professors felt this way when they were

6

GTAs” and that the GTAs must face this awkward, frustrating experience of teaching just as they

did, and will either “sink or swim.”

Graduate school is hugely uncomfortable, for so many reasons, and I recognize this as I

struggle through graduate school myself. You just don’t know what you don’t know. It’s as

challenging for me as I know it is for my GTAs. In striving to make at least some parts of

graduate school less painful for them, I have come to understand the transformative graduate

school process for myself. Though I am a “participant observer” in my research, I also feel I

have been given a huge gift in my own doctoral program. While I have been researching the

challenges of masters and doctoral Biology students and looking at ways to increase their

teaching effectiveness and program completion, I have become a better teacher myself, and have

completed my own program.

My positionality, perspectives, and biography undoubtedly affect my work with Biology

GTAs on an everyday basis, and have affected my fieldwork. I am incapable of extracting myself

from my research, and I arguably should not try. I embrace my position as participant, my

shifting subjectivity, and my situated knowledge. My enthusiasm for teaching, research, and

science co-mingle inextricably. Q Methodology, which I have been drawn to for my research, is

inherently linked to who I am. Biology research is empirical, looking at how systems interact,

observing how organisms communicate with others, and within their environment. The scientist

in me wants to make observations, collect data, and do statistical analyses. The social scientist in

me wants thick, rich descriptions that persist in qualitative research. The perspectives of GTAs

and faculty who work with them have driven my research, and drive my daily life. Q

Methodology, a mixed method, allows me to study people’s subjectivities, or viewpoints, in a

way that pays homage to both my social sciences and hard sciences backgrounds. My research is

7

my attempt to provide instructional training for GTAs that is meaningful, relevant, and positively

impacts all the stakeholders involved.

8

CHAPTER I

INTRODUCTION TO THE STUDY

The purpose of this chapter is to present the problem, purpose of the study, and research

questions. In addition, the researcher discusses the significance of the study. A brief review of

the literature provides introductory information related to the six major topics of this study: The

history of graduate teaching assistants (GTAs) in higher education, the use of GTAs as course

instructors, the varying aspects of GTA instructional training programs, GTA socialization as

future faculty, needs assessments in program evaluation, and Q Methodology. Finally, the

delimitations of the study are stated.

Introduction

Graduate Teaching Assistants (GTAs) are frequently utilized as instructors in

undergraduate classrooms and science laboratories (Kendall & Schussler, 2012; Luft, Kurdziel,

Roehrig, & Turner, 2004; Nyquist & et al., 1991). GTAs provide universities a cost-effective

form of instructor while the GTAs are being simultaneously socialized into the roles of teacher,

researcher, and scholar (Carroll, 1980; Garland, 1983). GTAs represent a diverse population of

masters and doctoral-level students, with varying amounts of pedagogical preparation, research

abilities, and motivation to complete their graduate study (Boyle & Boice, 1998). GTAs who are

not adequately prepared to engage in teaching activities may display a wide range of behaviors,

from an overblown confidence in their abilities (Golde & Dore, 2001), to frustration and

insecurity (Eison & Vanderford, 1993). The main preparation for new faculty has been teaching

assistantships, so they are limited in their teaching repertoire by the nature of their particular

assignment—usually in a discussion section or laboratory for a large lecture class, often without

supervision or adequate mentoring (Luft et al., 2004; Nyquist & Woodford, 2000).

9

Instructional training programs for professionally developing graduate teaching assistants

vary extensively from institution to institution, and even between departments at the same

institution (Nyquist & Woodford, 2000; Parrett, 1987; Stockdale & Wochok, 1974). Calls for

instructional training programs for teaching assistants in the sciences (Carroll, 1980; Luft et al.,

2004), and more specifically in biology (Rushin et al., 1997; Tanner & Allen, 2006) have created

a continual demand for pedagogical training, in addition to content area mastery.

Responses to the calls for instructional training programs have included national projects

such as “Re-Envisioning the Ph.D.” (Nyquist & Woodford, 2000), the “Preparing Future

Faculty” project (Pruitt-Logan, Gaff, & Jentoft, 2002), and the “Responsive Ph.D.” project

(Woodrow Wilson National Fellowship Foundation, 2000). These projects focus broadly on

improving the outcomes of Ph.D. degree programs (Gilbert, Balatti, Turner, & Whitehouse,

2004). These large-scale projects are dependent on external grant funding, and though

institutions may retain certain aspects of these programs after the grant ends, their sustained

existence after the termination of funding has proved difficult (Ferren, Gaff, & Clayton-

Pedersen, 2002).

Locally developed GTA instructional training programs are much more common in

graduate schools or disciplinary departments, and are described at length in Chapter II. These

programs are led by graduate school or disciplinary faculty or GTA supervisors, and vary widely

in programmatic elements and effectiveness (Carroll, 1980; Parrett, 1987; Thornburg, Wood, &

Davis, 2000). Programs range from half day university-wide orientation sessions that introduce

new GTAs to university policies but provide no departmental training, to multiday university-

wide training, department-specific training, or even university-wide training coupled with full-

semester courses and seminars on teaching methods offered by specific departments (Rushin et

10

al., 1997). Thus the amount and type of professional development made available to GTAs

remains highly variable in higher education institutions.

Whether the GTA instructional training program emerges nationally, from the graduate

school, or the individual disciplinary department, the evaluation of that program is a complex and

necessary part of any type of professional development (Garet, Porter, Desimone, Birman, &

Yoon, 2001; Guskey, 1994). Program evaluation is an intrinsic part of any program or project

because it is used to both measure the effectiveness of that program or project as well as

investigate ways to increase that effectiveness (Newman & Ramlo, 2011). The literature

surrounding GTA training programs describes GTAs as having varying programmatic needs

based on numerous factors – prior formal or informal teaching experience, familiarity with

content, exposure to prior instructional training, demographic variables, career aspirations,

international status, etc. GTA programs often group cohorts of GTAs together for training

(Muzaka, 2009) – all masters students or all doctoral students in one department, all the GTAs in

a department or graduate school at the beginning of their program, all the GTAs teaching a

common laboratory course, etc. – the combinations are numerous. One of the first steps in

effective program evaluation is assessing the needs of the particular set of participants in that

program (Chen, 2005; McNeil, Newman, & Steinhauser, 2005).

A needs assessment is a “systematic set of procedures for the purpose of setting priorities

and making decisions about a program or organizational improvement and allocation of

resources. The priorities are based on identified needs (Witkin, 1995).” A need is a discrepancy

or gap between “what is,” or the present state of affairs in regards to the group and situation of

interest, and “what should be,” or a desired state of affairs. A needs assessment seeks to

determine such discrepancies, examine their nature, and set priorities for future action (Kaufman,

11

Rojas, & Mayer, 1993; Kaufman & Valentine, 1989; Leigh, Watkins, Platt, & Kaufman, 2000).

In order to do a needs assessment, there must be a needs assessment tool.

There are challenges to designing a needs assessment tool for instructional training

programs. GTA needs assessment tools for instructional training programs have usually been

modified teaching inventories (Angelo & Cross, 1993; Gibson & Dembo, 1984; Kohn,

Lafreniere, & Gurevich, 1990; Prieto & Altmaier, 1994; Renzulli & Smith, 1978), Likert-style

questionnaires (Cho, Sohoni, & French, 2010; Sohoni, Cho, & French, 2013), or basic

demographic surveys. These instruments may not provide useful or adequate understandings of

the various viewpoints that exist among GTAs about their needs in an instructional training

program. Classification of GTAs based on typologies, or predictor profiles, may be more useful

for program evaluation, because typically a program does not have the same level of

effectiveness for the entire population it serves (McNeil et al., 2005). Typologies may also be

helpful in determining the combination of criteria that would accurately predict the success of at-

risk students in graduate education (Nelson, Nelson, & Malone, 2000). Q Methodology offers a

number of potential advantages for assessing needs of GTAs throughout their graduate school

career – Q Methodology can be used with small numbers of individuals, within a group, and

completed anonymously (Peritore, 1989; Prasad, 2001). Q Methodology does not demand the

large number of participants that a Likert-style survey requires (Cummins & Gullone, 2000).

Because the literature about GTAs frequently refers to GTAs in different disciplines or different

types of schools, the needs of GTAs in other disciplines are not necessarily the needs of this

specific group of Biology GTAs. Q Methodology allows the researcher to determine the various

perspectives and consensus within the group (Ramlo, 2008).

12

Q Methodology was first described by William Stephenson in 1935 in “Correlating

Persons Instead of Tests (Stephenson, 1935).” He described how Q Methodology allows

researchers to identify, both quantitatively and qualitatively, the various viewpoints within a

group and the number of people within the group who hold these viewpoints (Ramlo, 2008). Q

Methodology provides a foundation for the systematic study of subjectivity, a person’s

“viewpoint, opinion, beliefs, attitude, and the like (Brown, 1993).”

Typically, in a Q Methodological study, sorters are presented with a sample of statements

about some topic, called the Q Sample. Respondents, called the P-set, are asked to rank-order the

statements from their individual point of view, according to some preference, judgment or

feeling about them, mostly using a quasi-normal distribution (Van Exel & de Graaf, 2005). By Q

Sorting, people give their subjective meaning to the statements, and by doing so reveal their

subjective viewpoint (Smith, 2001) or personal profile (Brouwer, 1999). Q Methodology allows

the researcher to identify and interpret various viewpoints, such as viewpoints held by GTAs in

regard to graduate school. These viewpoints may be important to both the supervisors of GTA

instructional programs and to the GTAs.

Purpose of the Study

The purpose of this study was to demonstrate that Q Methodology can be used as an

effective needs assessment tool for a Biology graduate teaching assistant (GTA) instructional

training program. Q Methodology offers a number of potential advantages in program evaluation

over traditional survey techniques for assessing needs of GTAs throughout their graduate school

career. Ramlo (2008) described how Q Methodology “is an appropriate choice whenever a

researcher wishes to determine the various perspectives and consensus within a group regarding

any topic.” GTAs often express frustration with balancing the challenges of teaching, working

13

with undergraduate students, rigorous graduate classes, learning to do research, and having a

personal life (Boyle & Boice, 1998; Drake, 2011; Gaff, 2002; Tice, Gaff, & Pruitt-Logan, 1998).

They are often expected to prepare and grade exams, write their own syllabi, design the course

curriculum, order textbooks, prepare and present lectures, monitor student progress, and assign

final grades, all with minimal faculty supervision (Mueller, Perlman, McCann, & McFadden,

1997; Nyquist, Abbott, & Wulff, 1989). In addition to the academic responsibilities that GTAs

assume, they are also called on to hold office hours (Mueller et al., 1997), which typically

involves assuming an advising role - guiding students on topics such as mastery of course

material, academic concerns, applying to graduate school, and even counseling students through

personal problems (Moore, 1991). As instructors of undergraduates , GTAs must make

instructional, curricular, and assessment decisions in their courses (Luft et al., 2004). GTAs are

not serving as merely “teaching assistants,” GTAs are often responsible for the much of the

instruction at the undergraduate level at major universities in the United States (Allen & Rueter,

1990).

The challenges that GTAs experience in graduate school evolve from the beginning of

their program to the culmination of a thesis or dissertation (Muzaka, 2009). GTAs may begin

their programs with serious doubts about their levels of content knowledge or abilities to teach,

which may evolve into frustrations about demands on their time, pressures to publish, and

difficulties with research. While many faculty and administrators posit the purpose of doctoral

education to be the preparation to conduct original research (e.g., Council of Graduate Schools,

1990), others contend that the purposes of doctoral education should be further reaching,

including the training to teach (Adams, 2002; Gaff, 2002a) as well as the development of generic

or transferable skills such as public speaking, writing for different types of audiences, teaching,

14

how to think about problems and dig into the literature unaided, time-management, and people-

management skills (Crebert, Bates, Bell, Patrick, & Cragnolini, 2004; Cryer, 1998; Gilbert,

Balatti, Turner, & Whitehouse, 2004). These skills are necessary for both teaching, and the labor

market outside of academia (Atwell, 1996; Golde & Walker, 2006; Jones, 2003). While their

institutions may articulate messages about the importance of the teaching mission, their advisors,

particularly in STEM fields, may urge them to avoid spending too much time on anything

besides research-related activities (Austin et al., 2009).

Virtually all graduate students receive their Ph.D.'s from a research university (Cassuto,

2011). They get their first classroom experience there, and their dissertations are mainly guided

by professors whose research occupies a prominent place in their work lives. The graduate

student works his or her way from outsider to the profession, to full member, under the

mentorship of their advisors (Filstad, 2004). But because most academic jobs aren't at research

universities (e.g. liberal arts college, for-profit schools, 2-year colleges, community colleges),

those other jobs look jarringly different to graduate students than the positions held by their role

models (Cassuto, 2011). Graduate students express concern about their lack of explicit feedback

about their development (Austin et al., 2009).

Whereas at one time, biology GTAs would have transitioned from graduate school to

biology researcher, the labor market in higher education is changing from tenure-track positions

to teaching-intensive positions (Anwar, 2013; Carpenter, 2010; Jones, 2003). GTAs often

struggle to gain the skills that help them to be successful in either an academic career or in

industry (Austin & Wulff, 2004; Cassuto, 2012; Hayes, 2007). As GTAs confront the challenges

of graduate school, it is important for their supervisors to evaluate the specific cohort’s needs and

modify the GTA program in relation to them.

15

Socialization in graduate school refers to the process through which individuals gain the

knowledge, skills, and values necessary for successful entry into a professional career requiring

an advanced level of specialized knowledge and skills (Gardner, 2005; Weidman, Twale, &

Stein, 2001). Socialization is also described as the process through which an individual learns to

adopt the values, skills, attitudes, norms, and knowledge needed for membership in a given

society, group, or organization (Merton, 1968; Tierney, 1997; Van Maanen, 1976). The

socialization of graduate students is an unusual double socialization. New students are

simultaneously directly socialized into the role of graduate student, while being given

preparatory socialization into the role of future faculty in a research institution (Golde, 2002).

There has been a concerted effort by faculty in disciplinary fields and in graduate schools

to continually address whether graduates are prepared adequately to perform the roles for which

they have been socialized, so that the graduate program can make appropriate adjustments. It is

desirable, but not always present, that there be regular opportunities for the voices of graduate

students to be heard, so that their perspective informs program development (Weidman et al.,

2001).


Despite the wealth of literature concerning elements of instructional training programs

for GTAs at the national, institutional, or departmental level, typically a program does not have

the same level of effectiveness for the entire population it serves (McNeil et al., 2005). The first

step in program evaluation – using a needs assessment tool to identify participant needs – is often

missing or incomplete. This study demonstrated how Q Methodology can be used as a needs

assessment tool in a Biology GTA instructional training program. Q Methodology can provide

16

predictor typologies that are more useful than simple variables and demographic information for

the classification of people, especially within program evaluation (Newman & Ramlo, 2011).

The researcher used Q Methodology to investigate new and experienced biology GTA

views of graduate school, including their views about teaching, learning, students, research, and

challenges to persisting in their program. Multiple survey instruments were used to gather initial

information about the participants and their views about their biology graduate program. The

concourse, discussed in Chapter III, for this study included a collection of statements made by

GTAs in a Self-Reflection Questionnaire, a “Perceptions of Graduate School Survey,” a graduate

student discussion forum (“Grad School Life,” 2012), and everyday conversations and emails

made between Biology GTAs and their supervisors. A Q Sample was selected from this

concourse. A pilot study with new Biology GTAs demonstrated the viability of the research

design and instrument and led to three viewpoints (“The Confident Teachers,” “The Preferred

Researchers,” and “GTA to Professor”). The research study was expanded to include both new

and experienced GTAs. The results of this study may be used to consider potential changes or

updates to the existing training program.

Significance of the Study

While the number of pre-service orientation programs, in-service workshops, seminars,

apprenticeship programs, intern programs, and extern programs for GTAs have increased in the

last 50 years (Carroll, 1980), the crucial step of conducting a needs assessment to assess GTA

need in their instructional training programs is often missing or incomplete. A review of the

literature revealed that GTA needs in a program are often collected using modified teacher

inventories (Angelo & Cross, 1993; Gibson & Dembo, 1984; Kohn et al., 1990; Prieto &

Altmaier, 1994; Renzulli & Smith, 1978), Likert-style surveys (Cho et al., 2010; Gorsuch, 2003),

17

using simple demographic variables – or are not assessed at all (Shannon, Twale, & Moore,

1998; Worthen, 1992).

The most commonly used formal needs assessment tools used for GTA “teaching needs”

are modified secondary teaching inventories. These have included The Learning Styles Inventory

(LSI) (Renzulli & Smith, 1978), The Teaching Goals Inventory (TGI) (Angelo & Cross, 1993),

The Teacher Efficacy Scale (TES) (Gibson & Dembo, 1984), The Self-Efficacy Toward

Teaching Inventory (SETI) (Prieto & Altmaier, 1994), and The Inventory of College Students'

Recent Life Experiences (ICSRLE) (Kohn et al., 1990). This is problematic, however, because

higher education instructors are vastly different than high school teachers (Marston, 2010).

GTAs will have different needs in an instructional training program than secondary school

teachers.

Likert-style surveys have been criticized for issues related to construct validity, scale

construction, the large number of respondents needed, and reliability (Cummins & Gullone,

2000). The Likert scale is used to measure attitudes and opinions through statements as each

subject expresses his/her agreement with the contents of the statements by choosing one

alternative: strongly agree, agree, uncertain, disagree, strongly disagree (Lalla, Facchinetti, &

Mastroleo, 2005). The closed question format obliges respondents to choose only from among

the available options that may not match their actual opinions or attitudes. What distinguishes

between strongly agree, and agree? Will the respondent always choose agree, or can the choice

vary based on certain factors? These inconsistencies leads to an increase in missing data and a

possible drift toward the social acceptability of the answers varying between individuals, over

space, and time (Orvik, 1972).

18

The only specific GTA needs assessment tool was a survey developed by Cho et al.

(2010) “to capture to what extent GTAs, faculty, and undergraduate engineering students rate the

importance of typical GTA roles and responsibilities. “ The Likert-style survey included 24

items, which were later grouped into four categories. The four categories were 1) GTA

preparation, 2) Instructional Practices, 3) Engagement with Students, and 4) Classroom

Management. The survey takers were asked to “rate the importance of typical GTA roles and

responsibilities” from “not at all important” to “critically important.”

In the first category, “GTA Preparedness,” GTAs indicated that all the items were

between “critically important” and “important” (See Figure 1). GTAs continued to mark all the

statements as close to “critically important” for the entire survey. The faculty rated all the items

as “important,” but not “critically important.” This survey provides questionable value when

participants have no frame of reference for prioritizing the statements, or can mark all the

statements in one fashion.

Q Methodology allows researchers to identify, both quantitatively and qualitatively, the

various opinions within a group, and the number of individuals who hold those opinions

(McKeown & Thomas, 1988; Stephenson, 1953). Thus, Q Methodology is an appropriate choice

whenever a researcher wishes to determine the various perspectives and consensus within a

group (Brown, 1980). Q Methodology is similar to the Likert -style survey in that the distribution

on the grid typically ranges from least like my view to most like my view (Ramlo, 2008).

However, it differs from Likert-style surveys in that Q Methodology involves participants

physically sorting items relative to each other into a normalized or Gaussian distribution, based

upon that participant’s opinion within a particular setting, known as the condition of instruction

(Brown, 1993; 1980; McKeown & Thomas, 1988; Ramlo & Nicholas, 2009).

19

Likert (1967) assumed that every statement is equally important to the overall attitude.

McKeown (2001) criticized this type of survey, in that the individuality of the respondents may

be lost, due to the averaging of scores. Q Methodology is self-referential, meaning that the

sorting refers to one’s own world view, or subjectivity (McKeown & Thomas, 1988). Rather

than simply indicating agreement or disagreement with statements, GTAs, when doing a Q Sort,

are asked to sort the statements in relation to the other statements in the Q Sample. After the

GTAs have completed their Q Sorts, factor analysis is performed. The resulting analyses and

tables will provide insight about the various viewpoints held by GTAs in their training program.

Identifying and incorporating perspectives of GTAs into their development program by

performing a needs assessment is an important first step in enhancing the effectiveness of the

training programs for GTAs. Fuller(1969) suggested that to ensure effective teacher development

20

GTA Competence Rating by GTAs and Faculty

Item Category/Statement Rating by GTA Rating by Faculty

Being familiar with the syllabus 4.30 3.28

Being familiar with the course objective 4.22 3.33

Being familiar with the course materials 4.32 3.67

Knowing answers to student questions 4.19 3.50

Knowing what is expected of the GTA 4.17 3.50

Dressing appropriately 4.08 3.65

Holding regular office hours 4.54 3.94

Figure 1 - "GTA Preparedness" based upon Cho et. al.

programs, it is critical to accurately assess teacher concerns. In addition, teacher training or

professional development programs that do not reflect the needs and interests of participants are

unlikely to motivate them (Clarke & Hollingsworth, 2002), which in turn can result in the failure

to attain the program’s educational goals and objectives (Cho et al., 2010). This speaks directly

to the importance of need assessment tools designed to identify what motivates and concerns

teachers, or in this case GTAs, in advance of developing training programs.

If a program is to be useful to its stakeholders—in this case, the Biology GTAs—it is

important to keep their expectations in mind. For graduate students to become proficient in the

skills desired from academia, they must be given opportunities to develop their teaching skills,

abilities, and knowledge with the same guidance and practice that is afforded to the development

of a quality researcher (Golde & Dore, 2001).Because stakeholder needs vary at different stages

in the program (Chen, 2005), identifying GTA needs as they progress from new to experienced

GTA allows for program supervisors to identify and modify program elements relative to GTA

needs.


1. What are the various viewpoints that exist among Biology GTAs about their graduate

school experiences?

2. What are the various viewpoints of the supervisors of graduate GTAs in The Department

of Biology relative to those of the GTAs?

3. What consensus exists among the GTAs in The Department of Biology about their

graduate school experiences?

4. How do the views differ between new GTAs versus experienced GTAs?

21

5. Do the varying views and consensus of GTAs about their graduate school experiences

provide sufficient information for a needs assessment that informs the existing training

program?

Delimitations

The researcher did not consider the content knowledge held by the GTAs. A degree in

Biology was considered to demonstrate Biology content knowledge. Demographic information

such as race was not considered important to this study, however age, gender, graduate status,

teaching experience, and nationality may be considered in the final analysis. The demographic

information and success rate from undergraduate students taught by GTAs was not included in

the study. The researcher did not sort with GTAs from other disciplines.

The various viewpoints obtained in this study are not considered to be generalizable to

different groups of GTAs or Biology supervisor populations, as Q Methodology results are not

considered to be generalizable to the larger population. Because this study used Q Methodology

as a needs assessment, the study was exploratory in nature, the viewpoints or typologies

uncovered by this study are not generalizable to larger GTA populations. Small numbers of

participants Q Sorting is not a problem because the primary purpose is to identify typologies, not

to test the typology's proportional distribution within the larger population (Valenta & Wigger,

1997). Within this study, the researcher is solely interested in the GTA population within this

department at this time.

Summary

Because GTAs are frequently used in college classrooms as the instructors for the course

or laboratory, their preparation for that role is immensely important. Instructional training

programs for GTAs vary across institutions. GTA programs must meet the needs of a diverse

22

population of graduate students. Not only do GTAs teach, but they are also being socialized into

their potential roles as future faculty and/or researchers. This study demonstrates how Q

Methodology can be used as a needs assessment tool in a Biology GTA instructional training

program. This study aims to answer the following questions:


school experiences?


of Biology relative to those of the GTAs?





provide sufficient information for a needs assessment that informs the existing training program?

23

CHAPTER II

REVIEW OF THE LITERATURE

The purpose of this chapter is to present a comprehensive review of the literature related

to this study. The literature review explores the motives and distinguishing characteristics of

graduate students and provides a historical overview of the use of Graduate Teaching Assistants

(GTAs) as instructors of undergraduates in the university system across The United States. This

chapter also contains a discussion of the shifting nature of the academic workplace and considers

the role of graduate school as socialization into academia. The details of various types of GTA

training programs are described. Finally, the chapter offers a deeper understanding of the role of

program evaluation in graduate education, and explains the use of Q Methodology as a

framework for the study.

Why go to graduate school?

Graduate school often gives students a chance to pursue theories they may hold, gather

recognition for their talents, or upgrade an outdated education (Evans, Forney, Guido, Patton, &

Renn, 2009). Graduate degrees also offer the chance for changing careers, whether out of desire

or necessity (Mason, Goulden, & Frasch, 2009). A graduate degree typically offers students

greater earning power and advancement in their careers (Astin, 1997). Some students enjoy

traveling opportunities, teaching opportunities, and the chance to do original research. Others

attend because they desire to be a part of a research team and to work on advanced and

multifaceted projects (Malaney, 1987). There are also students who do not know what to do with

their undergraduate degree, and decide to pursue graduate school because they lack employment

opportunities. Interest in postgraduate study is influenced by psychological and sociological

factors such as parental education, socioeconomic status (SES), and role models (Betz &

24

Fitzgerald, 1987). Graduate students may receive free tuition and/or a stipend for being a GTA.

Quite often, graduate students have multiple reasons for attending graduate school.

Admission criteria vary, but graduate schools and graduate programs in the sciences

generally look for a minimum B average in upper division work, acceptable performance on the

GRE, favorable letters of recommendation, and evidence of motivation and commitment to

graduate study (Smith, 2012). Noteworthy graduate programs require outstanding faculty with

national or international reputations in research and scholarship. “Critical masses” of faculty are

also necessary for excellence in graduate education. The best graduate (especially doctoral)

programs include course requirements in other areas. Cross-disciplinary and interdisciplinary

programs offer unique opportunities, and allow graduate students to advance with combined

majors, giving them a competitive career edge. Graduate students may spend two to three years

in graduate school for a master’s degree, or five to seven years for a doctorate (Kuther, 2013).

Once students enter graduate school, they are often met with unique challenges (Golde,

2005). Graduate school is often highly competitive, and emotionally exhausting (Jacobs & Dodd,

2003). It may be difficult to prioritize responsibilities when it comes to teaching, research,

studies, and balancing academics with a personal life (Ward & Wolf-Wendel, 2004; Ward,

1998). There may be stress in relationships, or due to finances (Mallinckrodt & Leong, 1992).

Writing a thesis or dissertation is extremely challenging, and may take longer than the student

expects (Bowman, Bowman, & DeLucia, 1990; Ohashi, Ohashi, & Paltridge, 2008). Working

with advisors or research teams is challenging, and may make students feel frustrated,

overwhelmed, isolated, or out-of-touch. The student may not be prepared for the specialized

writing demanded for research and publication (Bloom, 1981).

25

In the sciences, the organizational unit of the “lab” is critical to understanding life in the

departments (Golde & Dore, 2001). Each faculty member sits at the center of a small solar

system—graduate students at various stages and postdoctoral research fellows orbit around the

faculty advisor (often referred to as the P.I., or Principal Investigator, highlighting the primacy of

research). The faculty member both establishes the research direction and sustains the group by

garnering external funding for research expenses, stipends, and tuition. This organizational

structure in turn defines a number of key features of graduate student life. The lab is the site in

which research is carried out. There is an emphasis on knowledge acquisition in the lab (e.g.,

through lab meetings, subfield specific journal clubs, and informal interactions with lab mates)

rather than solely in classes. There is also an expectation that the dissertation research topic

relates to, stems from, and feeds back into the advisor’s research, highlighting the interconnected

nature of the research projects of lab mates. The faculty member provides the fledgling

researcher a topic for research and the stability of funding for the duration of graduate study

(Golde, 2005).

Whether a student persists through a graduate degree program is a well-studied

phenomenon. Girves and Wemmerus (1988) describe how department characteristics, student

characteristics, financial support, and student perceptions of their relationships with faculty

influence graduate student persistence. After the initial year, graduate grades, involvement in

one's program, satisfaction with the department, and alienation could contribute directly to

graduate student degree progress (Quist, 2011). There are distinct and unique challenges to

developing graduate programs that maximize completion rates while still allowing students to

recognize and acquire the skills they will need for future careers. Not only should GTAs be

afforded the chance to acquire the skills necessary to be successful in academia, but there also

26

exists the argument that GTAs need certain generic or transferable skills such as public speaking,

writing for different types of audiences, teaching, how to think about problems and dig into the

literature unaided, time-management, and people-management (Crebert, Bates, Bell, Patrick, &

Cragnolini, 2004; Cryer, 1998; Gilbert, Balatti, Turner, & Whitehouse, 2004).

The Usage of Graduate Teaching Assistants in Higher Education

Graduate students have not always served as instructors for courses, leaders of

recitations, and laboratory instructors. During Colonial times in The United States, the

student/professor relationship was often one of the faculty standing “in loco parentis,” where the

faculty not only supervised the student’s room and board, but his worship, recreation, and his

studies (Bush, 1969).The traditional university model was a religiously-affiliated clergy

preparatory school, modeled after Cambridge and Oxford in England (Brickman, 1972). As time

progressed, educational models evolved from being centered around a church, to centered around

a library. Thomas Jefferson (1743-1846), believed educating people was a good way to establish

an organized society. He believed schools should be paid for by the general public, so less

wealthy people could be educated as students (Grizzard, 2009).

The use of GTAs in higher education began in the late 1800s, as some universities began

offering fellowships (stipends offered to graduate students in exchange for advanced research) in

order to attract graduate students to the institution (Allen & Rueter, 1990). These “research

assistantships” were paid positions that both lessened the financial burdens of graduate school,

and allowed students to do advanced research. Gradually, GTA duties within the university were

expanded. In the 1890’s, GTAs progressed from research assistants to teaching assistants, and

services (such as grading, role-taking, and recitations) to the university beyond research were

increased to justify the payments to the graduate students (Drake, 2011). Higher education was

27

expanding rapidly (Schofer & Meyer, 2005), and filling the role of “university instructor” with

people qualified to lead was vital to the success of all higher education stakeholders (Davies,

Hides, & Casey, 2001).

With the end of World War II in 1945, a rapid influx of students began attending school

on the newly formed GI bill (Coomes, 2000). A flood of veterans enrolled in America’s colleges

and universities, accounting for approximately 70% of all male enrollment (Bound & Turner,

2002). The GI bill provided financial support to veterans wanting to reeducate themselves for

post-war employment (Gelber, 2005). This increase in undergraduate enrollment demanded

professors use graduate students as assistants to help with more administrative tasks (Hendrix,

1995). Eventually, graduate assistants shifted from being simple “assistants,” to teaching basic

undergraduate courses independently. This allowed professors to teach higher level classes and

focus on their research (McKeachie, 1990). Expanding enrollment demanded an increasing

number of instructors, and rather than trying to find faculty that did not yet exist, universities

hired flexible graduate students (Burmila, 2010).

This pivotal time in American history was monumental for higher education. Sidney

Burrell (1967) concludes that the G.I. Bill led to “what may have been the most important

educational and social transformation in American history” (p. 3). The G.I. Bill allowed a more

diverse population of students to attend college due to financial assistance, making college a

viable option for men from a range of socio-demographic backgrounds, including minorities,

first-generation Americans, and those from low-income households (Bound & Turner, 2002).

Colleges and universities needed instructors for this flood of new undergraduate students, and

they needed them immediately. While the GTA was an innovative approach to meeting the

demands of an ever-expanding undergraduate population, many GTAs were un(der)prepared to

28

teach – knowing little (if any) of good instructional practice, how to deal with students unlike

themselves, and curriculum development.

Teaching “Assistant” or Course Instructor?

There is often a disconnect between GTA knowledge and preparation, and their

prioritization of teaching and researching (Hendrix, 1995). Many students’ primary focus is

research, rather than instruction (Butler, Laumer, & Moore, 1993; Serow, 2000). Between the

1930s and 1960s, the idea of training GTAs in pedagogy gained support, as more institutions

began focusing on the need for their graduate instructors to be able to function successfully in the

college classroom (Drake, 2011). GTAs could serve as the sole instructor for one or more classes

a semester (Butler et al., 1993) or as the instructor of laboratory or discussion sections (Luft et

al., 2004; Travers, 1989). Administrators of university programs felt that GTAs should not only

show content mastery, but be able to teach that content effectively. At some universities, equal

preference was given to graduate students who could demonstrate instructional capabilities as

well as research competence (Butler, Laumer, & Moore, 1993).

GTAs today are being utilized by colleges and universities to teach a variety of courses,

in a variety of fields (Buerkel-Rothfuss & Fink, 1993; DeBoer, 1979; Marting, 1987). They now

commonly assume the teaching roles that once only faculty performed (Branstetter &

Handelsman, 2000). GTAs are often expected to prepare and grade exams, write their own

syllabi, design the course curriculum, order textbooks, prepare and present lectures, monitor

student progress, and assign final grades, all with minimal faculty supervision (Mueller et al.,

1997; Nyquist et al., 1989). In addition to the academic responsibilities that GTAs assume, they

are also called upon to hold office hours (Mueller et al., 1997), which typically involves

assuming an advising role - guiding undergraduate students on topics such as mastery of course

29

material, academic concerns, applying to graduate school, and even counseling students through

personal problems (Moore, 1991). As instructors of undergraduates, GTAs are not merely

teaching “assistants.” They must make instructional, curricular, and assessment decisions in

their courses (Luft et al., 2004). They assume the role of professor, not apprentice (Burmila,

2010) – and they face unique challenges in this role.

Amidst the ever-present fiscal restraints, limited or no-growth policies, and unpredictable

enrollment in universities nationwide, funding setbacks have further expanded the reliance on

GTAs for undergraduate education (Koocher & Keith-Spiegel, 2008). They play a prominent

role in undergraduate science education in most large research-oriented universities and colleges

in the United States by instructing the majority of the introductory laboratories and discussion

sections (Travers, 1989). Perkinson (1996) asserted that GTAs spend more time in the

undergraduate classroom than do full-time faculty. Because of age and status similarities,

undergraduate students frequently relate more strongly with GTAs than they do with professors

(Hendrix, 1995; Moore, 1991). In addition, research has suggested that educators who have the

most impact on students are those with whom students identify and have more out-of-classroom

interaction (e.g., (Gaff & Gaff, 1981). And, because of wavering undergraduate and graduate

enrollments, the need for new instructors cannot always be met with new faculty hires. GTAs

allow for flexibility that is crucial in meeting oscillating demand (Burmila, 2010). As GTAs play

an increasingly significant role in not just teaching, but in advising and mentoring

undergraduates, it is important to consider how this multifaceted socialization impacts GTA

development as graduate students and future academics.

30

Instructional Training Programs for GTAs

Training Biology GTAs for the multiplicity of roles expected of them in the academic

community - graduate student, instructor, advisor, fledgling researcher – is complex (Bhavsar et

al., 2007). Biology faculty are not simply preparing future research Biologists, they are prepping

GTAs to meet the challenges of multiple roles – researcher, teacher, and academic. These

challenges are felt by all disciplines. Departments that compartmentalize GTAs with only

specialized disciplinary knowledge are not adequately preparing them for the possible careers

they could hold outside of academia (Loughran, Mulhall, & Berry, 2004). Supervisors of GTA

professional development programs have to prepare GTAs to teach undergraduate students who

may be nothing like themselves (Howard, Buskist, & Stowell, 1993; Meitl, 2008), or who may

be taking a general education course and display no interest in the GTAs’ field. With so many

stakeholders in GTA success, the question of “who bears the responsibility of preparing GTAs to

teach” is a complex problem.

The first organized effort to provide this much-needed instructional training for GTAs

began in the 1930s with English instructors at the University of Chicago's Institute for

Administrative Offices (Marting, 1987). This program was developed because of complaints

about the inept instructors emerging from the graduate school, who needed further pedagogical

training in their content areas (Marting, 1987). It was then that the Institute's members decided

that content mastery alone was not enough to produce effective teaching assistants - pedagogical

training was needed. Likewise, calls for training programs for teaching assistants in the sciences

(Carroll, 1980; Luft et al., 2004), and more specifically in biology (Rushin et al., 1997; Tanner &

Allen, 2006) have created a continual demand for pedagogical training, in addition to content

area mastery.

31

Science graduate students have reported the most interest in teaching amongst all GTAs.

They display the most confidence in their ability to teach and advise students, in comparison

with their peers from other disciplines (Luft et al., 2004). A survey by Golde and Dore (2001) of

over 4000 doctoral students at 27 universities clearly documented that graduate students in the

sciences reported holding more teaching assistantships than did their peers in other disciplines.

However, these assistantships often consisted of limited placements, usually in laboratory

settings for a defined amount of time. Despite teaching more courses, only a third of the graduate

students in the sciences at most universities indicated they had participated in a teaching assistant

(GTA) training session to prepare them for their teaching duties.

Graduate students who are not adequately prepared to engage in teaching activities may

have an inflated confidence in their abilities (Golde & Dore, 2001; Rhodes, 1997). To assist

graduate students in becoming proficient instructors, they must be given quality opportunities to

develop their teaching skills, abilities, and knowledge with the same guidance and practice that is

afforded to the development of a quality researcher (Golde & Dore, 2001). However, because

teaching is often regarded as a second-tier profession in academic settings, graduate students in

the sciences may experience limited educational environments (Luft et al., 2004). It is well

documented that an emphasis on teaching is viewed as a secondary career in many academic

settings, such as in community colleges, at for-profit institutions, or as an adjunct instructor

(Shannon et al., 1998). GTAs in the sciences commonly regard teaching as a “fallback career,”

only to be embarked upon after a student fails to obtain a research position (Richardson & Watt,

2006).

GTAs may perceive teaching as a highly demanding career having a heavy workload,

high emotional demand (Hendrix, 1995), anxiety-provoking, and generally requiring hard work

32

(Deiro, 1996; Rhodes, 1997). At the same time, they may also perceive teaching as relatively low

in social status, paying a low salary, and reported experiences of quite strong social dissuasion

from a teaching career (Rhodes, 1997; Watt & Richardson, 2008). In addition, teaching

assistantships are awarded on the basis of academic potential, not teaching potential (DeBoer,

1979). Being thrust into an instructional role that they feel unprepared for, uncertain about, or

even resentful of, is not ideal for either graduate students or their students (Hendrix, 1995). No

matter what the perceptions of teaching GTAs hold, faculty who mentor and supervise GTAs

have a duty to prepare future science instructors (Gardner, 2010b; Rosen & Bates, 1967).

Instructional training necessitates an ongoing series of professional development courses

that span GTAs’ graduate school careers, rather than a one time, simple orientation. As Prieto

(1995) notes, less than half of all GTAs receive any type of supervision on an ongoing basis. As

Palmer (1993) notes, "we would be better teachers if we had one simple thing: a rich on-going

discourse about teaching and learning, not the perfunctory annual teaching-development

workshop, but a community of discourse that triangulates...from the many different angles

available from within the life of the faculty itself" (p. 9). Rather than learning to become

proficient researchers with pedagogy as an additive, GTAs need to learn how to become

exceptional teachers and use research to enhance their teaching and teaching to enhance their

research (Rhodes, 1997). Training can provide a safe environment to discuss alternative ways of

handling problems that may arise in and outside of the classroom (Andrews, 1983). Directors or

supervisors of these programs may act as "emotional mentor" by offering emotional support and

providing models of emotional display when GTAs are in the process of shaping their own

personal feeling rules. Supervisors, peers, and training in general can provide a supportive

community (Rhodes, 1997).

33

Graduate School and the Socialization of Academics

Socialization in graduate school refers to the process through which individuals gain the

knowledge, skills, and values necessary for successful entry into a professional career requiring

an advanced level of specialized knowledge and skills (Gardner, 2005; Weidman et al., 2001).

Socialization is also described as the process through which an individual learns to adopt the

values, skills, attitudes, norms, and knowledge needed for membership in a given society, group,

or organization (Braxton, Lambert, & Clark, 1995; Merton, 1968; Tierney, 1997; Van Maanen,

1976). Graduate schools aim to provide graduate students with knowledge of research

concerning the subject matter in their fields, and to make certain that these students can

independently demonstrate the research skills of their chosen field (Bess, 1978). Preparing GTAs

to assume the types of instructional roles and responsibilities of faculty members is an equally

integral part of graduate school (Nicklow, Marikunte, & Chevalier, 2007). Bess (1978) argues

that “since the source of college faculty is the graduate school, one way to generate faculty with

these orientations [skills] might be through changes in graduate education.” Faculty members

play a myriad of roles in the socialization of doctoral students, including instructors in the

classroom, supervisors for students with assistantships, committee members for the thesis or

dissertation, advisor or chair of the research process, and even mentor (Isaac, Quinlan, &

Walker, 1992; Pease, 1967; Weidman & Stein, 2003). In this way, faculty members serve as

gatekeepers into and out of doctoral programs (Weidman et al., 2001).

Golde (2002) described the process of graduate school socialization as one “in which a

newcomer is made a member of a community—in the case of graduate students, the community

of an academic department in a particular discipline” (p. 56). She continued, “The socialization

of graduate students is an unusual double socialization. New students are simultaneously directly

34

socialized into the role of graduate student and are given preparatory socialization into a future

career in academia” (p. 56).

Graduate students are also being immersed in the culture of the discipline. Borrowing

from Merton (1968), Tierney (1997) stated, “Culture is the sum of activities in the organization,

and socialization is the process through which individuals acquire and incorporate an

understanding of those activities” (p. 4). He continued, “An organization’s culture, then, teaches

people how to behave, what to hope for, and what it means to succeed or fail. Some individuals

become competent, and others do not. The new recruit’s task is to learn the cultural processes in

the organization and figure out how to use them” (p. 4). The values, attitudes, and beliefs of the

culture, in this case, the academic culture, are often dictated by the discipline itself. Disciplines

have their own particular qualities, cultures, codes of conduct, values, and distinctive intellectual

tasks (Becher, 1981), which ultimately influence the experiences of the faculty, staff, and

students involved. Becher and Trowler (1989, p. 44) underscored this point: “We may

appropriately conceive of disciplines as having recognizable identities and particular cultural

attributes.” In order to navigate a Biology department, GTAs must acquire an understanding of

what the department members value, what faculty attitudes are towards the various activities the

GTAs will participate in, and the beliefs shared by the department (Rushin et al., 1997). The

GTA must quickly learn which undertakings will help them persist in the field, and which

activities deserve less attention. In the “publish or perish” world of academia, research and grant-

obtaining are highly prized, while teaching does not carry as many easily identifiable rewards

(Breen, Brew, Jenkins, & Lindsay, 2004; Sonnert, 1995; Vannini, 2006). Research expectations

for university faculty are so valued that research productivity has become the dominant and

35

sometimes the sole criterion for hiring, tenure, and promotion at research universities (Prince,

Felder, & Brent, 2007; Rushin et al., 1997).

Tierney and Bensimon (1996) suggest that the graduate school experience acts as an

agent of anticipatory socialization as the graduate student begins to understand the role of

faculty. Doctoral students observe faculty and the activity of the academic department and

subsequently form attitudes and opinions about life as an academic. As students assume their

roles as teaching assistants, they have some insight into the work roles of faculty members and

how to perform in those roles (Weimer et al., 1989). They are also attempting to “fit in” to their

new environment based on the disciplinary norms of their chosen field of study (Weidman et al.,

2001). What anticipatory socialization does not account for is the changing career trajectories of

GTAs. Though Biology GTAs may be able to see themselves stepping into the role of research

university faculty, they may not be able to see themselves stepping into the role of community

college instructor, adjunct, non-tenure track faculty, or liberal arts instructor.

New graduate students must investigate their place in the organization in order to glean

the necessary attributes that are important to the existing members (Tierney, 1997; Weidman &

Stein, 2003; Weidman et al., 2001). Newcomers or novices within the academic setting must

make sense of their new roles and begin to conform to the “normal behavior” as exhibited by

those around them (Tierney & Bensimon, 1996). In attempting to conform to academic

surroundings, the graduate student is forced to make decisions as to which aspects of the

graduate school process assist the individual in socialization. Failure to understand the priorities

in academia may result in a negative experience while in graduate school, which may contribute

to a negative experience when pursuing a faculty career (Tierney & Bensimon, 1996; Tierney,

1997). After what may be a long, difficult process of attempting to find a tenure-track position,

36

multiple rejections, and ultimately accepting a position outside academia, the socialization

process must include generic or transferable skills that help graduate students to be successful in

multiple types of careers, not just academia (Crebert et al., 2004; Gilbert et al., 2004; Stoner &

Milner, 2010).

Over 1 .5 million graduate students were enrolled in graduate programs, including

students pursuing both master’s and doctoral degrees, in 2005 (Brown, 2005), as compared to

1.73 million graduate students today (Rampell, 2012). As the number of graduate students

pursuing Ph.D.'s increases, academic job prospects are diminishing. Indeed, the number of

students receiving doctorates in biology increased from 3,803 in 1981 to 8,135 in 2011, while the

number of biological-science Ph.D. recipients in tenure-track positions dropped precipitously

from 55 percent in 1973 to 15 percent in 2006. Thus, a large majority of students are being

trained for faculty positions they will never obtain (Shea, 2013). American Society for Cell

Biology President Ron Vale (2013) wrote a column suggesting that an acceptable, if not good,

alternative career for science Ph.D.'s is to become elementary- or secondary-school science

teachers. Ph.D. programs have not prepared GTAs to be elementary or secondary school

teachers, however. Going this route often involves working in private or charter schools that do

not require certification, obtaining an emergency certification for an area of need, or a program

like “Teach For America (Berliner, 2002; Darling-Hammond, 2005; Decker, Mayer, &

Glazerman, 2004).”

Other suggested career options besides academia or teaching have included science

policy, start-up businesses, science communication/writing, nonprofit work, science publishing,

patent law, technology transfer, and consulting (Columbia University, 2013). Institutions and

departments are slow to change. Even though it is widely recognized that GTAs need additional

37

training and that their chances of becoming a Biology faculty member are slim, they are not

being prepared for alternative careers. Virtually all graduate students receive their Ph.D.'s from a

research university (Cassuto, 2011). They get their first classroom experience there, and their

dissertations are mainly guided by professors whose research occupies a prominent place in their

work lives. The graduate student works his or her way from outsider to the profession, to full

member, under the mentorship of an advisor (Filstad, 2004). But because most academic jobs

aren't at research universities, those other jobs look jarringly different to graduate students than

the positions held by their mentors (Cassuto, 2011). Developing training programs that recognize

the importance of communication skills, transferrable skills, the scholarship of teaching, and

student success as pivotal and investment-worthy, while not sacrificing the research component

of a GTA program, are acknowledged as integral to GTA professional development (Boyer,

1991; Kreber, 2001, 2005; Tulane & Beckert, 2011).

While many posit the purpose of doctoral education to be the preparation to conduct

original research (e.g., (Council of Graduate Schools, 1990), others contend that Ph.D. programs

should be further reaching, including training to teach (Adams, 2002; Gaff, 2002a) and skills

necessary for the labor market outside of academia (Atwell, 1996; Golde & Walker, 2006; Jones,

2003). The Council of Graduate Schools (2004, p. 4) clearly delineated the independent nature of

doctoral education: “Beyond some beginning course work, the experience of each Ph.D. student

is individualized and varied. Ph.D. students bear a greater responsibility for defining the scope of

their educational experience than do other students. Further, the degree requires initiative and

creativity, and the award of the degree depends upon the individual performance of a student in

completing original research in the area of study.” The purpose of graduate school, therefore, is a

combination of what the graduate school offers, and what graduate students view as their needs.

38

Supervisors of GTAs could alleviate some of their anxiety by providing a clear picture of what

previous GTAs in their department, university, or discipline have struggled with, and a tool to

help them recognize how their own preconceptions will shape their education. A successful GTA

program should empower graduate students to maximize their strengths and correct their

weaknesses.

Some might suggest that it’s up to the discipline to decide what an advanced degree

means. Institutional context and culture uniquely influence the student experience (Kuh & Whitt,

1988). Perhaps only a Biology Department can attest to the characteristics of its master’s or

doctorate holders (de Valero, 2001; Ehrenberg, Jakubson, Groen, So, & Price, 2007; Weidman &

Stein, 2003). While a Master’s Degree in Biology usually involves two years of coursework and

a thesis, a Ph.D. in Biology usually involves a similar amount of coursework and an independent

research project demonstrating expertise in the field. A Ph.D. may take four to eight years to

complete (Kuther, 2013). By the culmination of their graduate school career, GTAs should

“know what to do” when it comes to teaching, students, and research in their given discipline

(Luft et al., 2004).

In the four to eight years graduate students spend in graduate school, under the guidance

of their faculty advisor, GTAs should be given the opportunity to improve on their teaching, but

a “sink or swim” philosophy is often employed (Friedrich & Powell, 1979; Myers, 1998; Russell,

2011; Trowler & Kreber, 2009). While academic advisors may provide guidance to graduate

students, they may also serve as a negative example of faculty lifestyles (Austin, 2002). Over

half of all doctoral students in the sciences drop out in their first year, due to poor career

outlooks, being a bad fit with a disciplinary department, or conflicts with advisors (Golde, 2002).

Theories of socialization have been connected to the issue of attrition in doctoral education, with

39

researchers often attributing poor or inappropriate socialization to a student’s decision to depart

the graduate program (Clark & Corcoran, 1986; Ellis, 2001; Gardner, 2007; Golde, 1998;

Lovitts, 2001). As newcomers to graduate school, the institution, the department, and the

laboratory, the process is inherently anxiety-producing, and the support offered to the GTA

varies greatly (Gardner, 2007, 2008).

Conflicting Priorities in a Graduate School Program

Holding a teaching assistant position may help graduate students pay for graduate school

(Austin, 2002); however, graduate students may be told by their advisors that research should be

their focus, and that teaching assistantships should not be held for multiple years because this

will jeopardize their careers (Jones, 1993). In the sciences, graduate students recognize the

prestige of a research assistant position, and note that a teaching assistant position holds less

value (Fox, 1983). In Serow’s (2000) study of faculty at research institutions, one natural

scientist said, “anyone not doing the right type and amount of research would “never be accepted

as a legitimate, card-carrying member of the faculty.” This culture in which GTAs exist places

them in a situation that is wrought with tension and difficult to change (Luft et al., 2004). GTAs

may enjoy teaching and perceive this work as important but may feel that their interest in

teaching does not contribute to their overall professional development as scientists (Ethington &

Pisani, 1993). A report published by the Association of American Colleges maintains that,

"Unless the reward system in higher education measures teaching performance as well as

research, all efforts to improve college teaching will be to no avail" (1985, p. 37).

At this juncture, GTAs are surrounded by a myriad of conflicting viewpoints, which may

affect their desire and ability to persist in their graduate programs (Tinto, 1991). As a student,

GTAs come to graduate school seeking to increase their content and disciplinary knowledge. As

40

teaching assistants, they may feel unprepared to teach (Boice, 1991), uncertain about the role of

teacher (Svinicki, 1994), and stressed about their future careers (Sorcinelli, 2006). As a

researcher, they are looking to their faculty advisor for guidance on navigating the university,

working with grant-funding agencies, or departmental politics. With all of these (sometimes)

conflicting interests, determining which priorities gets the time and attention by the GTA is a

difficult decision. Tinto (1991, p. 110) suggests that graduate persistence is "shaped by the

personal and intellectual interactions that occur within and between the students, faculty, and

student-faculty communities that make up the academic and social systems of the institution.”

Graduate programs may be described by GTAs with feelings of “family’ or ““camaraderie,” or

conversely, feelings of isolation, ambiguity, and feeling lost (Gardner, 2010a).

Despite the conflicting priorities GTAs express, the institutional graduate program has

multiple stakeholders invested in the success of GTAs – the undergraduate students who are

being taught by them, the advisors who have included them in their research and may serve as

mentors, the graduate schools who want a successful graduate program, and the universities who

are looking to GTAs as current students and future faculty (Coll, Zegwaard, & Hodges, 2002;

Duchelle et al., 2009; Enz, Renaghan, & Geller, 1993). GTA training programs are being

influenced by a number of interested parties, and depending on who the programs are being run

by, may include a variety of components (Aubel, 1995). Academic departments have a stake in

GTAs, both as researchers and as potential future faculty, students have a stake in the

effectiveness of their instructors, and the institution itself has a stake in completion rates. While

programs that provide training to GTAs have proliferated and the literature surrounding GTA

development has increased, models and designs for best practice of these training programs

remains varied (e.g., Barrus, Armstrong, Renfrew, & Garrard, 1974; Clark & McLean, 1979;

41

Druger, 1997; Lawrence, Heller, Keith, & Heller, 1992; McComas & Cox, 1999; Nyquist &

Wulff, 1996).

Descriptions of programs range from half day university-wide orientation sessions that

introduce new GTAs to university policies but provide no departmental training, to multiday

university-wide training, department-specific training, or even university-wide training coupled

with full-semester courses and seminars on teaching methods offered by specific departments

(Rushin et al., 1997). As departments or graduate schools weigh the evidence for creating their

own organic GTA training programs or choosing one of the national GTA training programs,

they must know that stakeholder needs are being met by the program. The supervisors of these

programs must modify or replace programs that do not meet GTA needs. Supervisors first must

know what the needs of the GTAs are.

National Training Programs vs. Locally Developed Training Programs

There are a series of large-scale projects, funded by charitable foundations, which have

reviewed the Ph.D. degree and stimulated considerable activity for reform of the doctoral

curriculum. These projects include “Re-Envisioning the Ph.D.,” developed at the University of

Washington (Nyquist & Woodford, 2000), the “Preparing Future Faculty” project from the

Association of American Colleges and Universities and the Council of Graduate Schools, 2002

(Pruitt-Logan et al., 2002), the “Responsive Ph.D.” project, developed in the Woodrow Wilson

National Fellowship Foundation (Weisbuch, 2004), and the “Carnegie Initiative on the

Doctorate” developed by the Carnegie Foundation for the Advancement of Teaching (Golde &

Walker, 2002). These projects focus broadly on improving the outcomes of Ph.D. degree

programs (Gilbert, Balatti, Turner, & Whitehouse, 2004). There are challenges inherent in large,

national, grant-funded programs such as PFF. The program may not meet the needs of GTAs

42

locally. It may spend time reinforcing skills that GTAs already possess, or that doesn’t fit the

content area. A first year graduate student, and a fourth year graduate student certainly have

different skill sets. An English GTA certainly has different challenges than a Biology GTA.

While departments may feel ownership over their own, organically grown GTA programs, they

may be resentful of the time unwieldy national programs demand. In order to maintain a training

program, the needs of all the stakeholders in the program must be heard and addressed.

One national program that focuses specifically on instructional training in multiple

institution types is the Preparing Future Faculty (PFF) program (DeNeef, 2002). This program

involved 43 doctoral-granting institutions and 295 partner institutions that worked in clusters.

The lead campus established relationships with institutions in different higher education sectors -

community colleges, liberal arts colleges, master’s degree granting institutions, public

institutions, and private institutions. The clusters of institutions offered an opportunity for

graduate students to learn about the various roles and responsibilities of a faculty member.

Offerings for graduate students include meeting with teaching mentors, attending seminars about

teaching, participating in extensive programs designed to enhance instruction, and observing

outstanding instruction by senior faculty. Ultimately, PFF designers make a conscious effort to

prepare GTAs formally as teachers.

The Council of Graduate Schools (CGS) and The Association of American Colleges and

Universities (AAUC) both promoted the PFF program, and the using of best practices in the

graduate school education of GTAs. However, once the funding for the PFF programs ended in

2010, few institutions continued the program in its entirety (Newton, Soleil, Utschig, &

Llewellyn, 2010). Reports about PFF suggest that graduate students in the nation’s Research I

universities see their faculty mentors as not only generally unsupportive of their desire for more

43

pedagogical training, but even antagonistic to such training, since the faculty assumption has

been that they are really preparing people for research positions just like their own. Results from

1998 and 2001 surveys of graduate students who had participated in the Preparing Future Faculty

program highlight this issue; one student illustrates faculty’s negative attitudes towards non-

research intensive jobs by stating, “if you get a job at a liberal arts school, that’s your failure

rather than your success (DeNeef, 2002).” Even as the chances of a GTA getting a research

position as a faculty member at a university decline, these coveted positions are also

transforming (Edgerton, Rice, & Chait, 1997; Finkelstein, Seal, & Schuster, 1998; Finkelstein,

2006; Schuster & Finkelstein, 2006).

The Modern Academic Workplace

In addition to stepping into accepting professorial responsibilities, “the modern academic

workplace” is characterized by student diversity, new technologies, changing societal

expectations, expanding faculty workloads, a shift in emphasis toward the learner, and a new

labor market for faculty (Austin, 2002). The traditional full time, tenure-track, faculty position

that graduate students once strove for, as the culminating point of their course of study, is no

longer the norm. The AAUP (American Association of University Professors) reported (“Tenure

and Teaching-Intensive Appointments,” 2007), that almost 70 percent of faculty members were

employed off the tenure track, in part-time or non-tenure-track, full-time teaching positions.

Graduate programs continue accepting more graduate students than can possibly obtain tenure-

track faculty positions in academia (Berrett, 2012). Based upon these statistics, graduate students

are faced with the facts that they may be doing everything right – conducting research,

publishing in prestigious journals, writing grant proposals, serving as a GTA, teaching, serving

on departmental committees – and still may not obtain a tenure track job.

44

Whereas Biology GTAs of the past may have aspired only to a Biology research-focused,

tenure-track position at a R1university, their new job prospects may include part-time teaching,

an instructor position, laboratory coordinator, community college instructor, for-profit school, or

a job for which their Biology research credentials are less important than their Biology

instructional skills (Fleet et al., 2006). Biology GTAs, in their position in graduate school, have a

Bachelor’s degree in Biology, and are being prepared to articulate clearly their knowledge, and

communicate it to students (Boyer, 1991). Having a degree in Biology is not a guarantee that a

GTA has effective communication skills. Oral communication, a skill GTAs will use extensively

in their future career, is laden with contextual motivations, purposes, audiences, and strategies

specific to each field of inquiry (Dannels, 2002). Training programs for GTAs must assess

whether or not they can communicate effectively within their discipline. Assessment practices

that evaluate the extent to which students achieve the communication outcomes determined by

certain disciplines to be valued, salient, and relevant, must be developed (Dannels, 2001).

Evaluating Graduate Teaching Assistant Training Programs

The experiences of science doctoral students are unique and complex and are influenced

by multiple communities including the discipline, institution, department, lab and advisor. Each

community may offer different types of support to the student at different junctures of the

doctoral journey (White, Nonnamaker, & Smith, 2008). What works for one department, one

college, one university, or nationally may not be appropriate for another particular cohort of

GTAs. Couple these unique circumstances of GTAs with the rapidly changing career

opportunities, training that occurred ten years ago may not meet the needs of GTAs today. This

juncture is where program evaluation is critical. Program evaluation ought to be an intrinsic part

of any program or project because it is used to both measure the effectiveness of that program or

45

project, as well as investigate ways to increase that effectiveness (Newman & Ramlo, 2011). In

order to effectively evaluate the components of a GTA instructional training program, there must

be a baseline for comparison (McNeil et al., 2005).

The real test of the value of a program in a location is the implementation and evaluation

of a program in that location (McNeil et al., 2005). Gredler (1996, p. 15) defines program

evaluation as a “systematic inquiry designed to provide information to decision makers and/or

groups interested in a particular program, policy, or other intervention.” Program evaluation may

also be described as “an ongoing, collaboratively designed, and stakeholder-led evaluation

process that has the primary purpose of serving organizational learning by evaluating the whole

logic model” (York, 2005, p. 8). Carroll (1980, p. 179) notes in his review of the research

surrounding GTA training programs, that “programs should be structured to encourage the

participation of experienced, senior GTAs who can share their insights and experiences with the

novice GTAs.” Also, stakeholders should insist on continuing evaluation of the training

programs they administer or support (Rossi, Lipsey, & Freeman, 2004). Since the benefits to the

department or institution could vary from one cohort of GTAs to another, it is important that

program evaluation be conducted regularly (Carroll, 1980). While there is literature on best

practices for GTA instructional training (Meitl, 2008), the literature on programs for training

GTAs that blend best practices with the needs of the particular cohort of GTAs is absent.

Just as one size does not fit all in undergraduate teaching, curriculum, instruction, and

textbooks, neither does one size fit all in training graduate GTAs. Not only will there be nuances

in subject matter by departments, there is no “one right way” of teaching. No single view of

learning or teaching dominated what might be called “good teaching.” There have been five

documented perspectives on teaching, each with the potential to be good teaching: transmission,

46

developmental, apprenticeship, nurturing, and social reform (Pratt, Boll, & Collins, 2007). There

are qualitative, quantitative, and mixed method approaches in understanding instructional

training (Creswell, 2008; Johnson & Christensen, 2007). There are large scale, national GTA

training programs, and there are small, department created programs, along with “no program at

all” being a possibility (Christensen, Alexander, Nelson-Laird, & Robinson, 2011; Gaff, 2002a;

Nyquist et al., 1989). Just as no two undergraduate students will be exactly alike, neither will two

graduate students. GTAs enter school with varying degrees of experience, prior teaching,

experiences with students, approaches to diversity, and motivation to persist in their programs.

Understanding the various viewpoints of GTAs serves as an important needs assessment, which

establishes a baseline starting point for their instructional training.

Needs assessment is the first stage in the General Evaluation Model (GEM) of program

evaluation (McNeil et al., 2005). Needs assessment is the process of collecting, from all the

stakeholders, information that indicates the nature of the program. The information is the

discrepancy between what should be, and what is. The program is then designed to eliminate the

discrepancy between what is, and what should be (Altschuld & Witkin, 1999). A needs

assessment typically includes eight tasks:

1. Identify stakeholders

2. Identify program areas

3. Identify sources of information

4. Develop a needs assessment instrument

5. Conduct the needs assessment

6. Write the needs assessment report

7. Disseminate to the stakeholders

47

8. Make sure the stakeholders buy into the program (McNeil et al., 2005, p. 30)

The General Evaluation Model (GEM) (McNeil et al., 2005), allows for the evaluation of

strengths and weaknesses in a program. The GEM is composed of five stages that are sequenced

and form a feedback loop (Figure 2). The five stages are needs assessment, baseline, procedures

to achieve objectives, program implementation, and post assessment. This research uses Q

Methodology as a needs assessment tool to identify the variety of viewpoints of GTAs as they

either begin their professional development program, or have completed their professional

development program. Including the various GTA viewpoints as a starting point for program

evaluation provides the unique opportunity to tailor GTA professional development to best meet

the various needs of stakeholders. This crucial step of conducting a needs assessment in GTA

instructional training programs is what is often missing from the literature.

Numerous instruments were examined to identify the needs of GTAs in their professional

development/training programs. There was only one instrument specific to GTA development,

48

Figure 2 - The Five Stages of GEM (based upon McNeil et al., 2005)

which was developed by Cho et al. (2010) from an earlier survey called “The Teacher Concern

Checklist (Borich & Fuller, 1974). This survey was intended “to capture to what extent GTAs,

faculty, and undergraduate engineering students rate the importance of typical GTA roles and

responsibilities. “ The Likert-style survey included 46 items, which were later grouped into four

categories. The four categories were 1) GTA preparation, 2) Instructional Practices, 3)

Engagement with Students, and 4) Classroom Management. To complete the survey, participants

were asked to read each statement and ask themselves, “When I think about teaching, am I

concerned about this? “As in the original Teacher Concerns Checklist, a 5-point Likert-style

response scale (1-Not Concerned through 5- Highly Concerned) was used. Statements included

items such as “Having too many students in a class,” or “Whether the students respect me.”

McKeown (2001) has suggested that Likert-style surveys may lead to a loss of meaning, as in

this case, where choosing “Highly concerned” or “Not Concerned” has no real meaning to

GTAs. McKeown also suggests that Likert-style scales “fail to account for respondent intent and

interpretation of scale items and imposing a priori meanings external and prior to the

respondents’ actions on the scale.” GTAs could answer every question with “Highly concerned.”

Answering “Highly concerned” on one statement had no relation to answers on other statements.

This study may not have captured the subjective views of the GTAs under study.

Other instruments evaluated for use as a needs assessment for K12 teachers, but not

specifically for GTAs, included The Learning Styles Inventory (LSI) (Renzulli & Smith, 1978),

The Teaching Goals Inventory (TGI) (Angelo & Cross, 1993), The Teacher Efficacy Scale

(TES) (Gibson & Dembo, 1984), The Self-Efficacy Toward Teaching Inventory (SETI) (Prieto

& Altmaier, 1994), and The Inventory of College Students' Recent Life Experiences (ICSRLE)

(Kohn et al., 1990). Inventories for secondary school teachers would not provide appropriate data

49

as a needs assessment for GTAs, however, because the many factors are different between

secondary teachers and GTAs, such as motivation to work with young people, serving society,

fulfilling a professional commitment, satisfaction with the subject area, intellectual challenges,

and the opportunity to be creative (Brunetti, 2001; Marston & Brunetti, 2009; Marston, 2010).

None of these inventories provided the data needed to evaluate GTA viewpoints, or were able to

be modified to provide the data needed. In order to provide an accurate portrayal of the

perspectives of GTAs enrolled in the “Effective Teaching” course, a Q Methodology instrument

was developed.

Q Methodology

Developed by psychologist William Stephenson in the 1930’s, Q Methodology, also

called the Q Sorting technique, or simply Q, allows researchers to identify, both quantitatively

and qualitatively, the various opinions within a group and the number of people within the group

who hold these opinions (Brown, 1993; Ramlo, 2008). Stephenson, an English physicist and

psychologist who criticized psychometrics, revealed Q Methodology in a letter to Nature in 1935

and later described it specifically as a way to scientifically measure subjectivity (Stephenson,

1953). Q Methodology is an appropriate choice whenever a researcher wishes to determine the

various perspectives and consensus within a group regarding any topic (Ramlo, 2008).

Stephenson was critical of the available tests that measured behavior. A crucial premise

of Q is that subjectivity is communicable, because only when subjectivity is communicated,

when it is expressed operantly, it can be systematically analyzed, just as any other behavior

(Stephenson, 1953, 1968). If each individual would have her/his own specific likes and dislikes,

Stephenson (1935) argued, their profiles will not correlate; if, however, significant clusters of

correlations exist, they could be factorized, described as common viewpoints (or tastes,

50

preferences, dominant accounts, typologies, et cetera), and individuals could be measured with

respect to them (Van Exel & de Graaf, 2005).

Although Q Methodology uses numerical classification, it is a mixed methods approach

because it also uses qualitative research techniques (Newman & Ramlo, 2010; Stainton Rogers &

Rogers, 2004). Q Methodology provides the researcher a systematic and rigorously quantitative

means for examining human subjectivity by encompassing a distinctive set of psychometric and

operational principles that are coupled with specialized statistical applications of correlational

and factor-analysis techniques (McKeown & Thomas, 1988, p. 7). Q Methodology has been

discussed qualitatively, with a focus on subjectivity and self-referential meaning (Brown, 2008;

Watts & Stenner, 2005) but also has been designated specifically as a quantitative method,

focusing on factor analysis and interpretation (Block, 2008; Brown, 2008; McKeown & Thomas,

1988; Nunnally, 19780. One of the reasons Q Methodology is attractive to educational

researchers is because of its position in the mixed-methods continuum (Newman & Ramlo, 2010,

2011; Ramlo & Newman, 2011).

Brown (1993) describes the Q Sorting process as, “most typically, a person is presented

with a set of statements about some topic, and is asked to rank-order them (usually from ‘agree’

to ‘disagree’), an operation referred to as ‘Q Sorting.’ The statements are matters of opinion only

(not fact), and the fact that the Q Sorter is ranking the statements from his or her own point of

view is what brings subjectivity into the picture.” Q Sorting may be used in a single case study,

where a single respondent is asked to sort the sample of statements under multiple conditions of

instruction, or Q Sorting may be used to with groups and then statistically analyzed. Statistical

analysis leads to correlation and factor analysis that exposes patterns of findings within the group

(Brown, 1980) .

51

One differentiating quality of Q Methodology is that the statements being sorted are the

sample, while the participants are described as the P Set, or set of persons who are theoretically

relevant to the problem under consideration (Brown, 1980). Stephenson (1935)wrote of Q

Methodology , “[w]hereas previously a large number of people were given a small number of

tests, now we give a small number of people a large number of test-items” Correlation between

personal profiles then indicates similar viewpoints, or segments of subjectivity which exist

(Brown 1993). Q is intended to get at patterning within individuals (case-wise) rather than

simply across individuals (factor-wise sorting) (Brown, 1997).

Q Methodology allows participants, in effect, to “create their own categories. (Brown &

Narayan, 2005)” If all participants were to hold the same beliefs, Q factor analysis would register

as a single factor. If there were two belief systems, there would be two factors, and so on. The

number and character of the factors is a function of the participants themselves, not of how the

investigator categorizes the statements used. Q factor analysis allows the researcher to group

sorters with similar viewpoints for tailoring training in order to improve program effectiveness

(Newman & Ramlo, 2011). Through Q Methodology, artificial categories are replaced by

operant categories that represent functional, not just logical distinctions (Brown, 1991). An

additional advantage of Q Methodology studies is that wholly unexpected Q factors may emerge

(Brown & Narayan, 2005).

In Q Methodology a list of statements is developed that is sufficiently representative of

the “universe of viewpoints” about a topic, which is called the concourse (Brown, 1993).The

concourse, which is used to develop the set of statements to be sorted, can be developed using a

variety of techniques including interviews, focus groups, free writing, etc. (Newman & Ramlo,

2010). The concourse is followed by a selection, called the Q Sample that the participants will be

52

asked to sort (Brown, 1980; McKeown & Thomas, 1988; Ramlo, 2008). The participants pre-sort

the items, typically statements on numbered strips of paper, into three categories, most like my

view, neutral and least like my view, according to a set of “conditions of instruction.” Once this

pre-sorting has been completed, participants physically sort items, relative to each other into a

normalized or Gaussian distribution onto a grid (Brown, 1980; Stephenson, 1953). An example

of a Q Sort grid is shown in Figure 3.

The “conditions of instruction” are the set of instructions given to the sorter, describing

the conditions under which the sorter should place the statements. This is an important guide to

the actual sorting process, and must be clearly defined before the sorting process begins (Watts

& Stenner, 2005). The respondent is instructed to rank the statements according to some rule –

the condition of instruction, typically the person’s point of view regarding the issue (Van Exel &

de Graaf, 2005). For example, in one Q Methodology study by Ramlo (2005), faculty

participants were asked to sort statements based upon their views about the creation of a School

of Technology. This study was, in effect, a needs assessment such as that which often takes place

53

within program evaluation (McNeil et al., 2005). In another Q Study, engineering and

engineering technology educators were asked to sort statement based on their view on the use of

educational technology in the classroom, and their views of student learning. Respondents were

asked to sort statements from “least like my view to most like my view” (Nicholas, 2011).

The sorters interpret statements based upon their own views of the statement’s meaning.

Q Methodology is self-referential, meaning that the sorting refers to one’s own experience, or

subjectivity. As such, the sorting process represents a communicative process (Brown, 1980;

Stephenson, 1953). Because of the self-reference of the sorters, post-sort interviews or written

comments are typically used to assist the researcher in interpreting the meaning of the sorts

54

Most unlike

my view

neutral

Most like my

view-5 -4 -3 -2 -1 0 1 2 3 4 5

Figure 3 - Sample Grid

(Ramlo & Newman, 2011). These post sort interviews aid in gathering of supporting information

from the participant (Watts & Stenner, 2005). This can be done via a brief post-sorting interview

(which can then be transcribed and subjected to analysis), or simply via some form of ‘response

booklet’ or post-sorting questionnaire with open ended questions (Wong, Eiser, Mrtek, &

Heckerling, 2004). Such post hoc analyses ordinarily investigate: (a) how the participant has

interpreted the items given especially high or low rankings in their Q Sort, and what implications

those items have in the context of their overall viewpoint; (b) if there are any additional items

they might have included in their own Q set (what they are, why they are important, and so on);

and (c) if there are any further items about which the participant would like to pass comment,

which they have not understood, or which they simply found confusing (Watts & Stenner, 2005).

After the participants have completed and the researcher has compiled the sorts, the

analysis of the data must be conducted. The analyses of the Q Sorts involve correlation, factor

loadings, factor analysis, and the calculation of factor scores (Brown, 1980; McKeown &

Thomas, 1988). Conceptually, factor loadings are correlation coefficients. The higher the factor

loading, the more highly the sorter is correlated with that factor or view (Cuppen, Breukers,

Hisschemöller, & Bergsma, 2010). Those sorters with similar views are more highly correlated

with the same factor. Several programs exist to specifically handle the type of data collection and

analyses in Q Methodology. PQ Method is one of the most common, and is available for free

(Schmolck & Atkinson, 2002).

The factor loadings express the extent to which each Q Sort is associated with each

factor. With little exception, only the first two or three factors contain significant loadings,

although it is possible that more than three factors may emerge (Brown, 1980; 1993). However,

the original set of factors is the raw materials from which the probing of these subjective

55

relationships can take place from the vantage points of interest (Brown, 1980, 1993, 2009;

McKeown & Thomas, 1988; Newman & Ramlo, 2010).

At this stage, the factors themselves have no meaning beyond identifying groups and

remain numerical abstractions until final interpretation (Eden, Donaldson, & Walker, 2005). The

aim of the post-Q Sort interview is to discover the rationale behind participants' placing of the

cards in the Q Sort response grid (Gallagher & Porock, 2010). Typically, when interpreting their

results, researchers coin a name for each factor and describe its viewpoint in a paragraph or two

of prose that rephrases key statements or lists key statements from the ‘ideal’ sort (Eden et al.,

2005). Although previous research may be used to explore the factor, it may not capture the

rationale behind these particular participants’ placement of the statements. Interviewing

participants after they have Q Sorted allows for the interpretation of the factors to be based on

the participants’ perceptions and attitudes to the phenomenon under study. This can be discussed

in line with previous research yet allows for new theory to be generated (Gallagher & Porock,

2010).

The major concern of Q Methodology is not with how many people subscribe to a

particular belief, but with why they believe what they do (McKeown & Thomas, 1988; Sexton,

Snyder, Wadsworth, Jardine, & Ernest, 1998). With Q Methodology small sample sizes are

psychometrically acceptable because the observational perspective is the respondents own

(McKeown & Thomas, 1988). This means that any observations or interpretive accounts that are

advanced by the researchers are subservient to the respondent’s frame of reference as made

operant by Q Sorting. Because of this, the validity and reliability tests that are so important in

conventional research are unessential within the psychometric framework of Q Methodology

(Brown, 1980, 1993; McKeown & Thomas, 1988).

56

Although Q Methodology is similar to the Likert-style survey in that the distribution on

the grid typically ranges from least like my view to most like my view (Ramlo, 2008), Q differs

from Likert-style surveys in that Q involves participants physically sorting items relative to each

other into a normalized or Gaussian distribution (Brown, 1993; Brown, 1980; McKeown &

Thomas, 1988; Ramlo, 2008; Ramlo & Nicholas, 2009). Likert (1967) assumed that every

statement is equally important to the overall attitude. Likert scales do not consider the weight

that sorters attach to individual items (ten Klooster, Visser, & de Jong, 2008) which can therefore

result in the loss of meaning (McKeown, 2001; Ramlo & McConnell, 2008).

Summary

There has been a concerted effort by faculty in disciplinary fields and in graduate schools

to continually address whether graduates are prepared adequately to perform the roles for which

they have been socialized, so that the graduate program can make appropriate adjustments. It is

desirable, but not always present, that there be regular opportunities for the voices of graduate

students to be heard, so that their perspective informs program development (Weidman et al.,

2001). The use of graduate students as instructors for undergraduate students has increased

significantly as the number of undergraduate students has risen. GTA instructional training

programs vary significantly in structure and effectiveness. Understanding the needs of GTAs

may assist supervisors of programs in evaluating their professional development programs, and

increasing the effectiveness of their program. Q Methodology is an appropriate methodology for

uncovering the various viewpoints of GTAs in their instructional training program. Improving

their training program will assist GTAs in meeting the challenges of an evolving job market, a

diverse set of undergraduate students, the complexities of doing original research, and balancing

life in graduate school.

57

CHAPTER III

METHODOLOGY

The purpose of this chapter is to present a comprehensive review of the methodology

related to this study. This chapter provides an overview of the research design, the derivations of

the general and specific research hypotheses, and the research questions. Other sections

designate the participants and sampling procedures. The basic procedures for a Q Methodology

study are described in detail. The instrument section describes the compilation of the concourse,

the Q Sample, the Q Sort, the conditions of instruction, and the pilot study conducted during the

Fall semester of 2012. The statistical treatment section explains how the results of the Q Sorts

will be factor analyzed and interpreted. The role of the researcher and limitations of the study

conclude the chapter.

Introduction and Overview

Since the purpose of this study is to explore Biology Graduate Teaching Assistants’

(GTA) experiences of graduate school, it is important to have a baseline assessment of GTA

needs. Using Q Methodology, GTA’s viewpoints can be made operational. Developed by

William Stephenson in 1935, Q was created to the scientific study of subjectivity. Q

Methodology studies patterns of subjective perspectives across participants rather than patterns

across variables (McKeown & Thomas, 1988; Watts & Stenner, 2005). Through Q Methodology,

artificial categories (from objective tests) are replaced by operant categories that represent

functional, not just logical distinctions (Brown, 1991).

Stephenson (1935)wrote of Q Methodology , “[w]hereas previously a large number of

people were given a small number of tests, now we give a small number of people a large

number of test-items” Correlation between personal profiles then indicates similar viewpoints, or

58

segments of subjectivity which exist (Brown 1993). Q is intended to get at patterning within

individuals (case-wise) rather than simply across individuals (factor-wise sorting) (Brown,

1997). The statements being sorted are the Q Sample, while the participants are described as the

P-set, or set of persons who are theoretically relevant to the problem under consideration

(Brown, 1980).

Q Methodology allows researchers to identify, both quantitatively and qualitatively, the

various opinions within a group and the number of people within the group who hold these

opinions (Brown, 1993; McKeown & Thomas, 1988; Ramlo, 2008; Stephenson, 1953). Q

Methodology is an appropriate choice whenever a researcher recognizes that there are differing

and consensus viewpoints in a group. Common perspectives held by group members can be used

to scaffold the instructional training of the participants (van der Valk & de Jong, 2009), based

upon consensus statements. This consensus within a group regarding any topic is uncovered and

made operational (Ramlo, 2008). Thus, Q Methodology is an appropriate choice whenever a

researcher wishes to determine and describe the various perspectives and consensus within a

group regarding any topic (Brown, 1980; Ramlo, 2008).


The research questions were developed to study the range of viewpoints that exist among

Biology GTAs about their graduate school experience, particular their instructional training

program. The researcher was interested in how the viewpoints of the GTAs and their viewpoints

of their supervisors would be similar or different. The Biology Lab Coordinator and the Lead

Biology Faculty Member have different backgrounds, positions within the university, and

priorities for GTAs. The GTAs themselves have different needs within their programs, which

may change as they complete their “Effective Teaching” course, experience teaching, take

59

classes of their own, and learn to do research. This chapter outlines the research design and data

analysis that were used to investigate the following research questions:


school experiences?


of Biology, relative to those of the GTAs?





provide sufficient information for a needs assessment that informs the existing training

program?

Next, a rationale for the use of Q Methodology as the research design for this study is

presented, along with a description of the P-Set, compilation of the concourse, the Q Sample, the

Q Sorting process, and the data analysis procedures that were used.

Rationale for the Research Design

This study utilized Q Methodology as a research approach. Ramlo (2008) described how

Q Methodology “is an appropriate choice whenever a researcher wishes to determine the various

perspectives and consensus within a group regarding any topic.” As Robbins and Krueger (2000)

stated, “Q Method’s approach renders empirical the question of who is similar, under what

conditions difference is expressed, and why (p. 644).” Q Methodology focuses on grouping

individuals with similar viewpoints, perspectives, ideas, or beliefs. Q Methodology is used to

study participants’ subjectivity, that is, their viewpoints, in a systematic way (Brown, 1991;

60

McKeown & Thomas, 1988). It allows a researcher to “understand a human experience rather

than identify cause-and-effect relationships” (Broady-Ortmann, 2002) while finding out different

opinions of group members and how many people in the group share specific opinions

(McKeown & Thomas, 1988; Ramlo, 2008; Stephenson, 1935).

GTAs may possess different views of graduate school. Professional development

programs for training GTAs vary extensively from institution to institution, and even between

departments at the same institution (Rushin et al., 1997). New GTAs enter graduate school with

vastly differing amounts of content knowledge, pedagogical knowledge, and research skills

(Gess-Newsome & Lederman, 1999; Shulman, 1986). They have had varied experiences with

students, and as students. “Teaching as they were taught” in their own science courses may lead

to further lack of understanding (Brown, Abell, Demir, & Schmidt, 2006; Longbottom & Butler,

1999).

GTA skills and knowledge change as they encounter learning experiences in graduate

school (Luft et al., 2004; Muzaka, 2009; Park, 2002; Prieto & Altmaier, 1994). In order to

improve the instructional training program for GTAs, the supervisors of the program must first

understand the various views held by the targeted population about their situation and needs

(Sohoni et al., 2013). Literature about GTA training has classically focused on faculty’s

perceived needs of GTAs (Boyle & Boice, 1998; DeChenne, 2010; Sohoni et al., 2013; Young &

Bippus, 2008). National training programs, such as The Preparing Future Faculty program, are

designed entirely by faculty, using faculty perceptions of what GTAs should know (Anderson,

Gaff, & Pruitt-Logan, 1997). While training programs must address material gleaned from

faculty experience, educational research, learning theory, etc., programs should also address

critical concerns from the GTA perspective as well (Williams & Roach, 1992).

61

Q Methodology has advantages over survey research for this study even though both

methods are used to obtain participants’ perceptions. Likert-style surveys of GTAs may have

limited effectiveness in understanding GTA needs (Cho et al., 2010). Surveys are common

methods for collecting feedback; however, they allow responders to give similar or identical

ratings to many or all items (Dennis, 1986). They can also result in missing data (Li-Fen Lilly Lu

& Jeng, 2006; McKeown, 2001; Sexton, Snyder, Wadsworth, Jardine, & Ernest, 1998). Missing

data or non-response bias resulting from non-respondents can be alleviated through use of Q

since data is collected one-on-one (Dennis, 1986). Surveys, polls, and scales can highlight

common or shared opinions that exist in the total group, but do not provide empirical evidence of

the differing views/factors (Collins, 2009).

Surveys rely upon large numbers of participants in order to generalize results from the

study to a larger population (Previte, Pini, & Haslam-McKenzie, 2007), however, the small

number of factors that emerge in most Q studies require relatively small numbers of participants

(Dennis, 1986; Previte et al., 2007). In fact, “a larger number of participants can be problematic,

because they can negate the complexities and fine distinctions which are essential features” in

carrying out a Q study (Previte et al., 2007, p. 139). Q Methodology employs a by-person factor

analysis in order to identify groups of participants who make sense of (and who hence Q ‘sort’)

a pool of items in comparable ways (Watts & Stenner, 2005). Q can be seen as a tool to make

more explicit the expectations and beliefs held by a group with respect to the dialogue (Steelman

& Maguire, 1999). Q reveals correlations and factors among persons, while R methodology, or

survey research, reveals correlations and factors among traits. In Q Methodology, the

correlations are based on the assumption that “persons significantly associated with a given

factor … share a common perspective” (McKeown & Thomas, 1988, p. 17). Thus, Q is useful in

62

understanding participant perspectives within groups (Cross, 2005; Previte et al., 2007; Ramlo,

2008; Steelman & Maguire, 1999).

Further advantages of the Q Methodology are identified by Peritore (1989), who

described how Q “respects the integrity of the respondent, results can be recorded anonymously

and factorial results cannot be predicted.” It is argued that Q Methodology combines the

strengths of both qualitative and quantitative research (Dennis & Goldberg, 1996) and provides a

bridge between the two paradigms of inquiry (Sell & Brown, 1984). Zraick and Boone (1991)

emphasize that Q Methodology is more focused than a general attitude questionnaire, and that Q

Sorts are normally distributed and therefore can also be used parametrically in intergroup

comparisons. Another factor underlying the Q approach to participants is that Q Methodology

has no interest in estimating population statistics; rather, the aim is to sample the range and

diversity of views expressed, not to make claims about the percentage of people expressing them

(Kitzinger, 1987).

By determining the diversity of perspectives held by the GTAs, the supervisors can then

tailor meaningful, relevant professional development opportunities that prepare GTAs for future

challenges such as those suggested by Marincovich, Prostko, and Stout (1998); Ross and Dunphy

(2007), and Tice et al. (1998). GTAs often express frustration in graduate school, for reasons that

include teaching, learning, research, working with students, and persisting in their program

(Muzaka, 2009). Completing a needs assessment with incoming GTAs can help to determine the

variety of viewpoints that exist in an instructional training program. Conducting this needs

assessment with GTAs who have completed the instructional training program and have

continued teaching can identify if there are aspects of the training program that could be better

addressed.

63

Despite the literature related to training teaching assistants, faculty who work with

graduate students may be unprepared themselves to mentor GTAs for any career other than a

faculty research career (DeNeef, 2002), GTAs express frustration with teaching, working with

undergraduate students, the challenges of graduate classes, learning to do research, and balancing

demands of their time with having a personal life (Drake, 2011; Eison & Vanderford, 1993).

Boyle and Boice (1998) found that new GTAs, who are just beginning graduate school, voice

different frustrations than experienced GTAs who may be writing a thesis, working with

advanced classes, writing articles for publication, or doing research. Q Methodology allowed the

researcher to uncover common viewpoints, or “predictor profiles” that may be different than

grouping GTAs based on “new or experienced” or “master’s or doctoral” or “teaching-focused or

research-focused.” Uncovering these typologies may lead to more effective means of scaffolding

the training for GTAs who express different viewpoints. With most of the current-day job

opportunities being outside of academia, GTAs often struggle to gain the transferrable skills that

help them to be successful outside of academia in a teaching career, industry, or an alternative

profession (Jenkins, 1996; Park, 2002). Understanding the needs of GTAs, which can lead to

improving the effectiveness of GTA training programs, may eventually lead to benefits for the

stakeholders involved in this program. Q Methodology has been shown to be an effective needs

assessment tool in program evaluation (Newman & Ramlo, 2011; Ramlo & Berit, 2013).

Basic Procedures of Q Methodology

The first general step of conducting a Q Methodology study is the purposeful selection of

participants, or the P-Set. The usual number ranges from 30 - 50, but this could vary according to

need. It is preferable to try to find people with varying views in order to experience a variety of

responses in the sorting process (Reid, 1999). Watts and Stenner (2005) stress the importance of

64

finding participants with a defined viewpoint, whose viewpoint matters in relation to the subject

at hand. They also reiterate that” more is better” does not apply to Q Methodology, when it

comes to participants. Brown (1980) suggests that Q Methodology only requires “enough

participants to establish the existence of a factor for purposes of comparing one factor with

another (p. 192).” Q Methodology has little interest in taking head counts, or generalizing to a

population of people. Q is more concerned with the exploration of meaning and quality (Willig

& Stainton-Rogers, 2007).

Van Exel and De Graaf (2005) emphasized that Q Methodological studies do not require

large sample sizes. They contended that the P-set is selected intentionally by compiling a sample

of “respondents who are theoretically relevant to the problem under consideration” (p. 6).

Participants are not randomly chosen (Brown, 1980; McKeown & Thomas, 1988; Quiles, 2009;

Webler, Danielson, & Tuler, 2009). Instead, individuals are recruited who are representative of

the issues and could provide the best insights on the topic under study. Baker et al. (2006)

describes how, in Q Methodology, “individuals are purposefully selected according to their

personal attributes, views they might express, or on the basis of their social position and

background. The sample will therefore depend on the research topic in question rather than on

the basis of statistical power.”

The next procedure in a Q Methodology study is development of the concourse.

Concourse development involves the creation of a large set of statements that illustrate a range of

attitudes and perceptions that have been expressed by people related to a particular subjective

topic of exploration. Van Exel and De Graaf (2005) noted, “The gathered material represents

existing opinions and arguments, things lay people, politicians, representative organizations,

professionals, scientists have to say about the topic; this is the raw material for a Q” (p. 4). The

65

theme of the research, and the inclusion of statements remain controlled by the researcher (Eden,

Donaldson, & Walker, 2005). The ideal concourse contains all the relevant aspects of themes

identified in all discourses about a given topic (De Graaf & Van Exel, 2008). The level of

discourse dictates the sophistication of the concourse (Brown, 1980).

Stephen Brown stated,

The concourse is the flow of communicability surrounding any topic. Concourse is the

very stuff of life, from the playful banter of lovers or chums to the heady discussions of

philosophers and scientists to the private thoughts found in dreams and diaries. From concourse,

new meanings arise, bright ideas are hatched, and discoveries are made: it is the wellspring of

creativity and identity formation in individuals…and it is Q Methodology’s task to reveal the

inherent structure of a concourse. (1993, pp. 94-95).

A concourse can be collected in a number of ways. The two most typical methods include

reviewing literature (theoretical) and/or interviewing people (naturalistic) and recording what is

said (McKeown & Thomas, 1988). A concourse may also be comprised of statements from both

naturalistic and theoretical sources, and would be considered a hybrid approach (Delnero &

Montgomery, 2001).

After the compilation of the concourse comes the development of the Q Sample. The Q

Sample items are a subset of the full concourse (Valenta & Wigger, 1997). Selections for the Q

Sample are made by the researcher. A wide variety of statements from the concourse must be

selected in order to create a Q Sample that is manageable, but is also representative of the same

perceptions and attitudes that are expressed in the full range of statements in the concourse (Van

Exel & De Graaf, 2005). The Q Sample is often made up of 40 – 80 items, but this number might

66

vary according to need. As the items are selected, any two statements should be positively

associated, negatively associated, or unassociated (Reid, 1999).

As with sampling persons in survey research, the main goal in selecting a Q Sample is to

provide a miniature, representative sample of the concourse, which, in major respects, contains

the comprehensiveness of the larger process being modeled (Brown, 1980; De Graaf & Van

Exel, 2008). The problem, of course, is how to select from the concourse so as to provide

representativeness in the Q Sample, and the main device relied upon to achieve this is Fisher's

experimental design principles (Brown & Ungs, 1970).

The next step Reid (1999) described as the administration of the Q Sort, following the

directions, known as the conditions of instruction, or the procedures participants follow as they

sort. Van Exel and De Graaf (2005) explained that the cards comprising the Q Sample are given

to participants in a pack of randomly numbered cards with one statement written on each one (p.

6). A participant must be able to effectively respond to the question by sorting the set of provided

items along a single, face-valid dimension, such as most agree to most disagree, most important

to most unimportant and so on. The condition of instruction must be written down and kept in

front of each participant as they sort, because the researcher must be certain the P-Set are all

answering the same question (Watts & Stenner, 2005).

The researcher tells the participant to make 3 general piles containing the same number of

statements, reflecting least like their view, neutral, and most like their view. Then, the

participants go on to discriminate further to into a forced, symmetrical/quasi normal distribution.

Participants are instructed to rank the cards onto a sorting grid, for example on a scale of -5 to

+5, with -5 being the most unlike their personal point of view, 0 being neutral, and +5 being the

most like the point of view that they most identify with (See Figure 4). It is recommended that Q

67

Sorts be followed with interviews to provide participants the opportunity to elaborate on their

points of view, especially in regard to the extreme ends of the spectrum, those most unlike and

those most closely aligned the participants’ points of view (p. 7).

Using an intentionally forced distribution in the sorting process limits the number of

items that participants can place in each category or ranking level. Unlike surveys and Likert

scales, sorting into a grid ensures that the participants make explicit choices about the ranking of

the sort items relative to the other items (Corr, 2001; McKeown & Thomas, 1988; Ramlo, 2008).

By sorting the items into a forced distribution, participants are required to discriminate among

them in a way they would not do otherwise (Dennis, 1986).

68

Most unlike

my view

neutral

Most like my

view-5 -4 -3 -2 -1 0 1 2 3 4 5

Figure 4 - Sample Grid Showing “Normalized” or Gaussian Distribution

The forced distribution ensures fine discrimination by the participant who cannot simply

sort all items into two categories such as strongly agree strongly disagree. The participant must

follow a symmetrical distribution and sort items into categories that reflect degrees of opinion of

preference (Dennis, 1986; Reid, 1999; Sexton, et al., 1998). The ranking may stretch, for

example, over a span of -5 to +5 for the sorter to show a range of opinions. The Q Sorting

process forces the sorter to make choices about what is more or less like their views. A clear and

'gestalt' configuration of items will duly emerge (Watts & Stenner, 2005). If the participant is

happy with this configuration, the various item numbers (and hence the 'form' of the overall

configuration) should be recorded (Brown, 1993). Each Q Sort is simply the perspective of the

person whose Q Sort it is (Brown, 1997). Participants inject statements with their own

understandings. Objective measures (e.g., IQ tests) have right answers, but this is not the case

within the realm of subjectivity, and it is for this reason that Stephenson always utilized factor

analysis rather than variance analysis in analyzing data obtained from Q technique (Brown,

1997).

The results of the sorting activity lead to the interview. Some researchers feel the

interview is optional, but for the full and complete administration of Q Methodology, the

interview is a necessary component and should not be left out (Reid, 1999). After completion of

the Q Sorting activity, the researcher discusses with the sorter the way decisions were made,

focusing on apparent contradictions, outlying selections, extreme responses, or unclear points.

The interview is often tape recorded. Along with the results of the sort, the interview information

serves as essential data for the study (Collins, 2009).

The final step in the study is the interpretation of the data. Following the sort, statistical

analysis commences in Q Methodology. The sorts are entered into PQMethod, the software

69

package design specifically for analyzing Q Methodology data (Schmolck & Atkinson, 2002).

This software package provides a variety of outputs, such as a correlation matrix, factor loadings,

distinguishing statements, and consensus statements. Data analysis occurs with factor analysis

highlighting intercorrelations of the Q Sorts as variables persons, not traits or Q Sample items are

correlated). The combined respondents’ factor loading indicates the extent to which each Q Sort

is similar or dissimilar to others (McKeown & Thomas, 1988).

The researcher looks for areas of agreement among sorters. The level of agreement or

disagreement among sorters ultimately gives rise to the factors. A factor analysis is applied to the

results of the sorts, looking for patterns that arise from among sorts (Collins, 2009). The factor

analysis reduces the many viewpoints down to a few salient factors, which reflect common or

shared ways of thinking (Reid, 1999; Sexton, et al., 1998). Generally, if four or more sorters load

on a common factor, that is an indication of increased reliability (Brown, 1980; Sexton et al.,

1998).

Additional insights into what is different about the two factors’ perspectives can be

achieved by examining the distinguishing statements, which are statement rankings which

distinguish the factors from each other, and consensus statements, which represent agreement

among all the factors (Brown, 1980; McKeown & Thomas, 1988). Brown (1991) pointed out that

when comparing the rankings assigned to the sort items, differences of 2 between ranking scores

could be considered significant. However, this was a guide and could serve as an alert for the

researcher to look more closely at these items and see what it was about distinguishing

statements that caused individuals to rank them so differently (Brown, 1980). These differences

could be used to help define distinctions between groups (Donner, 2001). Consensus statements

70

have allowed researchers to focus on agreement among different views, which can be used to

start a dialogue related to commonality (Ramlo & Newman, 2011; Ramlo, 2005).

From the review of Q Methodology design and unique characteristics, it was obvious that

Q Methodology was a suitable fit for the goals of this study. Q Methodology could assist in

answering the research questions, and was robust enough to satisfy both the quantitative and

qualitative aspects of this study. And, Q Methodology allowed the researcher to answer more

sophisticated questions about this group of people than either qualitative or quantitative analysis

could do on its own (Ramlo & Newman, 2011).

Setting

The research study was conducted at a large, public, urban university in the Midwest.

Enrollment was 28,771 in the Fall 2012 semester. The university offers over 300 Baccalaureate

programs, 200 Master’s programs, and 37 Ph.D. programs. The Department of Biology offers

eight Bachelors of Science degrees, two Master’s degrees, and one International Baccalaureate

(IB) doctoral program. In the Fall 2012 semester, 828 undergraduate students, 24 master’s degree

students, and 37 IB doctoral students were enrolled. There were 22 full time faculty and eight

part time faculty, 12 full time staff, 12 graduate research assistants, and 40 graduate teaching

assistants employed by the department (“The University of Akron : IR Home,” 2013). The

Department of Biology emphasizes collaborative and integrative research. Facilities include a

live animal research center, 400 acre field station, and greenhouse (The University of Akron,

2013a).

The Department of Biology has many areas of strength, including Ecology and

Evolutionary Biology, Physiology, Molecular Biology, and Organismal Biology. Areas of

interest for graduate research include: pollination biology, conservation biology, physiological

71

ecology, life history evolution, mating systems, aquatic ecology, evolution in developmental

processes, behavioral evolution, spider biology, biomedical research, hypertension and stress

research, bio-materials and biomechanics, developmental molecular biology and physiology,

comparative biochemistry, and evolutionary biomechanics (The University of Akron, 2013a).

Admission Requirements for the Biology graduate programs include:

A baccalaureate degree in biology or equivalent training.

A minimum cumulative grade point average of 3.0 (4.0=A) and a 3.0 average in biology (minimum 32 semester credit hours or equivalent).

Competence in chemistry and mathematics.

Scores from any one or more of the following standardized tests: GRE (General Test), GRE (Biology-specific Test), or the MCAT. Scores are expected to be above the 25th percentile to be competitive for admission.

A letter of interest indicating the proposed area of specialization and possible advisers in the Department of Biology.

Strong letters of recommendation (3 preferred).

A letter from the potential Biology Adviser indicating willingness to sponsor the applicant (The University of Akron, 2013a).

The M.S. degree in Biology is obtained upon completion of required coursework and a

research thesis. Each student, in conjunction with a graduate committee, plans coursework,

seminars and research based upon the student's background and interests. A total of 40 graduate

credit hours are required for the degree. Of these 40, a minimum of 12 must be in thesis research

credits, 24 in formal coursework, and 4 in colloquium. A non-thesis option is available for

individuals with a current teaching certificate or co-registration with the College of Education

toward obtaining teaching certification (The University of Akron, 2013a).

72

Full-time master’s degree graduate students pursuing thesis research may be supported

with graduate assistantships, either teaching or research assistant, by the appropriate Department,

generally for a period of two years. Full-time teaching assistants (GTAs) are expected to work 20

hours per week and must enroll as full-time students (currently 9 or more credit hours per

semester, including research). master’s degree GTAs are expected to enroll in a one credit-hour

“Effective Teaching” course at the beginning of their program, which serves as an orientation to

the department, to the graduate program, and to teaching a laboratory course (The University of

Akron, 2013a).

The Integrated Bioscience Ph.D. is obtained upon completion of required coursework and

a research dissertation. Each student, in conjunction with a graduate committee, plans

coursework, seminars and research based upon the student's background and interests. A

minimum of 80 credit hours is divided between formal courses, elective courses, colloquia, and

research. The mission of the Integrated Bioscience program is to address the need for Ph.D. level

graduates who have both deep and specific expertise in a bioscience, bioengineering or

biotechnology discipline and broad adaptability across related disciplines (The University of

Akron, 2013b).

The program is composed of six areas of excellence:

1. Molecular cell biology and genetics

2. Biochemistry and biopolymers

3. Bioinformatics and computational biology

4. Bio-engineering

5. Physiology and organismal biology, and

6. Ecology and evolutionary biology (The University of Akron, 2013b).

73

Full-time Integrated Bioscience Ph.D. graduate students pursuing dissertation research may be

supported with graduate assistantships, either teaching or research assistant, by the appropriate

Department, generally for a period of five years. Full-time teaching assistants are expected to

work 20 hours per week and must enroll as full-time students (currently 9 or more credit hours

per semester, including research). Integrated Bioscience Ph.D. GTAs are expected to enroll in a

one credit-hour “Effective Teaching” course at the beginning of their program, which serves as

an orientation to the department, to the graduate program, and to teaching a laboratory course

(The University of Akron, 2013b).

The P-Set

The P-Set for this study was purposefully selected, and included both new and

experienced Biology GTAs. A new Biology GTA is defined as a graduate level student who is

seeking a master’s or doctoral degree, has less than one year of formal teaching experience, and

teaches an undergraduate-level laboratory for approximately 20-hours a week in exchange for a

fee-remission. This GTA is currently enrolled in an "Effective Teaching" instructional training

course. An experienced Biology GTA is defined as a graduate level student, who is seeking a

master’s or doctoral Degree, has more than one year of formal teaching experience, and teaches

an undergraduate-level laboratory for approximately 20-hours a week in exchange for a fee-

remission. This GTA has completed an "Effective Teaching" instructional training course. This

P-Set was selected intentionally to include respondents that were stakeholders in a Biology GTA

instructional training program.

The study included participants sorting during two different phases. Q Sorting was

completed first by new GTAs enrolled in an “Effective Teaching” course in The Department of

74

Biology in the Fall 2012 semester, and the two supervisors of the course. These sorts occurred

during the second week of the “Effective Teaching” course. In this phase of the study, there were

21 total Q Sorts collected from 17 new Biology GTAs, the Biology Lab Coordinator sorting

twice, and the Biology Lead Faculty Member sorting twice.

The Biology Lab Coordinator and Biology Lead Faculty Member sorts were included in

the study because of their large degree of involvement with the instructional training of Biology

GTAs, and these supervisors of GTAs were most familiar with the viewpoints of both new

Biology GTAs and experienced Biology GTAs. The researcher was interested in whether the

supervisors of Biology GTAs would have similar or differing viewpoints than the actual GTAs.

The Biology Lab Coordinator and Biology Lead Faculty Member sorted during the first phase of

the study.

The second phase of this study included experienced GTAs who had completed the

“Effective Teaching” course and who had successfully taught in the Biology department for

more than one year. This sorting included all experienced GTAs who attended a weekly

mandatory Biology departmental colloquium meeting. There were an additional 14 Q Sorts

collected from the experienced GTAs, and one Q Sort collected from an additional Biology Lab

Coordinator, theoretically sorting as an experienced GTA. The demographics for the P-Set are

described in Table 1.

Table 1 – P-Set Demographics

Number Percent

Participation Rate 36 Sorts 100%

Session of Sort Completion

Effective Teaching 21 58%

75

Number Percent

Course

Colloquium 15 42%

Type of Participant

New GTAs 17 48%

Experienced GTAs 14 42%

Biology Lab Coordinator 3 7%

Biology Faculty Member 2 5%

Degree Track

Doctoral 16 52%

Masters 15 48%

Gender

Male 19 52%

Female 17 48%

Origin

International 2 6%

United States 34 94%

Teaching Experience

None 2 6%

Informal 6 17%

Formal 28 77%

76

The Concourse

The “GTA Perceptions of Graduate School Q Sort” was developed during the summer of

2012 for a Q Methodology seminar. The first stage of designing the Q Sort involved the

compilation of the concourse. The concourse for this Q Methodology study was created through

the examination of statements made by GTAs in a Self-Reflection Questionnaire (SRQ), a

“Perceptions of Graduate School Survey,” a graduate student discussion forum (“Grad School

Life,” 2012), everyday conversations and emails made between Biology GTAs and their

supervisors, and a thorough literature review. Being that both theoretical and naturalistic sources

were used for the compilation of the concourse, this would be considered a hybrid approach.

There were 93 statements collected from these sources (see Appendix 1). The themes for the

concourse in this study are identified in Table 2.

Table 2 - Development of the Concourse and Q Sample

ThemeNumber of Statements

in the ConcourseAdded to Q Sample

Advisor 3 1

Anxiety 10 4

Balance 4 2

Career 2 2

Collaboration 1 1

Confidence 10 7

Diversity 4 2

Effort 4 4

Emotional 1 1

Ethical 2 1

Fairness 2 1

Intelligence 2 1

Learning Styles 11 5

77

Practical 2 1

Preparation 8 4

Research 8 6

Respect 3 3

Teaching 16 9

Total number of statements 93 54

SRQ – Self Reflection Questionnaire

The first place that statements were compiled from was the Self Reflection

Questionnaire. Eight biology GTAs who taught Natural Science, a general education course for

non-majors, were asked to complete a Self-Reflection Questionnaire (SRQ) after successfully

teaching in the laboratory during the Spring semester of 2010. The questionnaire was developed

as part of a program improvement initiative under the direction of a new faculty coordinator for

Natural Science Biology. The SRQ consisted of eight open-ended questions which prompted

GTAs to reflect on their teaching philosophy, knowledge of biology concepts, and teaching

skills. The SRQ was designed using Shulman’s (1986; 1987) theory of Pedagogical Content

Knowledge (PCK), connecting teachers’ content knowledge and pedagogical principles and

practices, and was modeled after Hammrich’s (1996) open ended questionnaire that explored

how biology graduate students defined the “teaching of science.” The SRQ was critiqued by

Biology faculty and professors from the Department of Education. The eight prompts were

analyzed using content analysis techniques. The questions on the SRQ were:

1. When you took your Effective GTA Training course, you were asked to describe your

teaching philosophy. Do you still have a copy of this? If so, please copy and paste

below.

78

2. How knowledgeable do you think you are about the Biological concepts covered in the

course? (Extremely knowledgeable, very knowledgeable, somewhat knowledgeable)

What factors made you come to this conclusion?

3. What do you feel are your strengths? (Please list 3 to 5 of each)

4. What do you feel you could improve upon? (Please list 3 to 5 of each)

5. By the end of the semester, what do you think students should know once they have

finished the lab course (i.e. procedures, subject matter, skills) Please list the 3 to 5 most

important.

6. If you had the opportunity to teach these same students again, what would you do

differently?

7. What do you feel is the most challenging thing about teaching the lab?

8. What do you feel is the most challenging thing for students taking the lab?

The eight GTAs who completed the SRQ did so after teaching Natural Science Biology

for at least one semester. Their demographics are detailed in Table 3. Answers to the SRQ were

grouped into categories of statements displaying content knowledge, pedagogical knowledge,

and pedagogical content knowledge. Answers were used in the redesign of the instructional

training for GTAs who teach the non-majors Natural Science Biology laboratory in the Fall of

2010. GTAs who completed the SRQ did not complete the Q Sort.

Table 3 - Demographic Characteristics of GTAs completing the SRQ

8 graduate GTAs

Sex 3 females 5 males

Program 6 Master’s 2 Doctoral

79

Nationality 2 International 6 United States

Experience 4 first time teachers 4 experienced teachers

The Perceptions of Graduate School Survey

The second place that statements for the concourse were compiled from was the

“Perceptions of Graduate School” survey. Nine GTAs in The Department of Biology, who taught

Natural Science Biology, were asked to complete the “Perceptions of Graduate School Survey”

after teaching one semester of non-majors Natural Science Biology laboratory in the Spring of

2012. The “Perceptions of Graduate School Survey” was created through a literature review of

GTAs and their beliefs, attitudes, and perceptions of teaching, learning, students, and research.

The “Perceptions of Graduate School Survey” was designed to uncover teacher beliefs.

Puchta (1999) describes teaching beliefs as “the guiding principles” of teacher practice. He

describes how teaching beliefs guide teaching practice. Based upon their beliefs, teachers make

generalizations about teaching, learning, and students that shape the way they approach their

work. Through this iterative practice, they come to realizations that help them to make sense of

the world. Teachers form inner representations of cause and effect. Their beliefs influence the

way teachers think and act. Their presuppositions about students, teaching, and learning may be

displayed consistently or inconsistently in their practices. Pajares (1992) describes the study of

beliefs as a “messy construct.” He expresses frustration in the lack of precision that studying

beliefs is afforded. Other authors have connected teacher beliefs with practice both in university

academics (Kane, Sandretto, & Heath, 2002) and in physics teaching assistants (Spike &

Finkelstein, 2010).

80

The open-ended questions included in the “Perceptions of Graduate School Survey”

included:

1. Before you started teaching, what did you believe your students that you were going to

teach would be like?

2. Since you have been teaching, what do you believe about your students, or students in

general?

3. What do you wish all teaching assistants knew about students, before they started

teaching?

4. Before you started teaching, what did you think that “being a teaching assistant” was

going to be like?

5. After you had taught, how did your perceptions of being a teaching assistant change?

6. What do you wish all GTAs knew about teaching, before they started teaching?

7. Before you started teaching, what did you believe “doing research” would be like?

8. Since you have been doing research, how have your beliefs about “doing research”

changed?

9. What do you wish all GTAs knew before they started doing research?

10. What did you believe that graduate school was going to be like, before you started?

11. Did you have any surprises or challenges once you were in graduate school?

There were nine GTAs who completed the Perceptions of Graduate School Survey (See

Table 4). Responses to the Perceptions of Graduate School Survey were grouped into categories

of statements concerning students, teaching, research, and graduate school. Responses were used

in shaping the instructional training of new GTAs taking the “Effective Teaching” course in the

81

Fall 2012. GTAs who completed the “Perceptions of Graduate School Survey” did not complete

the Q Sort.

Table 4 - Demographic Characteristics of TAs Completing the “Perceptions of Graduate School Survey”

9 graduate GTAs

Sex 4 females 5 males

Program 8 Master’s 1 Doctoral

Nationality 3 International 6 United States

Experience 4 first time teachers 5 experienced teachers

Statements from the Literature

The remaining statements populating the concourse were compiled from an extensive

review of the academic literature concerning GTA and their experiences in graduate school

instructional training programs. Themes that had been identified through a review of the

literature while developing the SRQ and the Perceptions of Graduate School Survey were further

expanded to include statements made by GTAs in other studies. The theories depicted in the

literature reviews for the SRQ and the Perceptions of Graduate School Survey were a

combination of Shulman’s (1986, 1987) theory of Pedagogical Content Knowledge and Pajares’

(1992) theory of Teacher’s Beliefs. These studies contained qualitative data, and included

statements made during case studies, interviews, focus groups, in emails, and in communications

with supervisors. These statements allowed the researcher to provide a representative and

balanced coverage of the themes in the discourse surrounding GTAs and their perceptions of

graduate school.

82

Q Sample

Each of the 93 statements from the concourse were placed on a strip of paper, and were

physically sorted into piles that contained statements which expressed a similar theme. For

Strauss and Corbin (1990), the links between expressions and themes are “conceptual labels

placed on discrete happenings, events, and other instances of phenomena.” Themes, or

categories, are the classification of more discrete concepts. “This classification is discovered

when concepts are compared one against another and appear to pertain to a similar phenomenon.

Thus, the concepts are grouped together under a higher order, more abstract concept called a

category” (p. 61). As Stephenson said, a Q Set “may be designed purely on theoretical grounds,

or from naturally occurring (ecological) conditions, or as required for experimental purposes, to

suit the particular requirements of an investigation” (1952, p. 223).

The statements made by GTAs that populated the concourse represented their

perspectives about graduate school. The researcher collected statements that broadly represented

all the identified themes of discourse around a topic – in this case, a sample of statements that

was representative of the various statements made by GTAs about their graduate school

instructional training program in graduate school. The key themes and issues that defined the

subject matter for the study were identified by the researcher, who was looking for ways to

improve instructional training programs, common views of GTAs concerning their instructional

training programs, and ways to alleviate GTA frustration in their graduate school program. The

statements were systematically identified according to theme by the researcher and two Biology

GTAs who did not take part in the Q Sort.

The Q Sample (see Appendix 2) was derived by selecting nine representative statements

from each of six categories of interest to the researcher – teaching, learning, students, research,

83

challenges in graduate school, and GTA persistence in their program – for a total of 54

statements in the Q Sample. This represented a Fisherian design (Brown & Ungs, 1970). It is this

set of statements that was eventually presented to participants in the form of the “GTA

Perceptions of Graduate School Q Sort” (See Appendix 3), or the research instrument (Van Exel

& de Graaf, 2005). The Q Sample represented subjectivity on a given topic, in this case GTA

viewpoints on graduate school.

Q Sort

For the “GTA Perceptions of Graduate School Q Sort” the 54 chosen statements were

randomly numbered. Each statement was typed on an individual strip of paper, about the size of

an address label. The statements were printed on white paper, cut into strips, and placed in their

own envelope. The statements were administered in the form of an envelope of randomly

numbered strips of paper (one statement to a strip) with which the respondent was instructed to

operate according to the condition of instruction. The researcher was interested in the GTA’s

own point of view, and the GTA was instructed to sort the statements into three piles, based upon

their views of graduate school. The conditions of instruction for this study are shown in Figure 5.

84

Figure 5 – Conditions of Instruction for “GTA Perceptions of Graduate School Q Sort

The grid onto which the GTAs were asked to sort the statements was a quasi-normal distribution.

Although there has been some debate about the merits of “forced” versus “free” distributions

(Brown, 1968), quasi-normal distributions are most commonly employed in Q Methodological

research because of the statistical advantages they yield (Kitzinger, 1987). This grid was chosen

because it accommodated all 54 statements, and yielded a “top eight” most unlike my view, and

most like my view categories. The range of columns and the frequencies for statements was

4 – 4 – 5 – 5 – 6 – 6 – 6 – 5 – 5 – 4 – 4. That is four statements were placed in column one, four

in column two, five in column three, and so on. The array positions for columns one through

eleven were given values of -5, -4, -3, -2, -1, 0, +1, +2, +3, +4 and +5 for statistical analysis, as

demonstrated in Figure 6.

85

The Pilot Study

A pilot study was conducted to test logistics and gather information prior to the research

study. The pilot demonstrated the feasibility of the study, and tested the Q Sample instrument,

the instructions to the participants, the conditions of instruction, and the factor analysis of the

sorts. The pilot demonstrated that the materials did not need to be modified, and were suitable for

incorporation into the main study. When the sorting for the pilot study was completed, the sorts

were entered into PQMethod, the software package design specifically for analyzing Q

Methodology data (Schmolck & Atkinson, 2002).

The analysis of the pilot study revealed three factors concerning the views of New

Biology GTAs and graduate school:

86

4 4 5 5 6 6 6 5 5 4 4Most unlike

my view

neutral

Most like my

view-5 -4 -3 -2 -1 0 1 2 3 4 5

Figure 5 - Distribution Grid for “GTA Perceptions of Graduate School Q Sort”

Factor 1 or “The Confident Teachers,” are confident in both their teaching abilities, and

their place in graduate school as a GTA.

Factor 2 or “The Preferred Researchers” are characterized by their preference for research

over teaching, and their skepticism about working with students.

Factor 3 or “The GTA to Professor” are characterized by wanting to teach as they were

taught by their favorite professor and by wanting to use their time as a GTA to lead them towards

becoming a better professor.

The pilot study allowed the researcher to test Q Methodology as a needs assessment tool.

Because the pilot study demonstrated that there were various viewpoints among the population

of new Biology GTAs and their supervisors, Q Methodology could be used to modify the

existing training program. It was determined that Q Methodology was an appropriate design for

the study.

Data Collection Procedures

The researcher personally collected Q Sorts in two phases. During the first phase, the

Biology Lab Coordinator, the Lead Biology Faculty Member, and the new GTAs sorted in their

“Effective Teaching” course. The researcher distributed the Q Sorts, the instructions for the sort,

and a copy of the IRB “Informed Consent” letter (See Appendix 4) to each member of the class.

The basic concept of Q Sorting and instructions on how to perform a Q Sort were explained to

each participant in order to ensure that the content was fully understood. They were instructed to

sort these statements from a range of -5 to +5 indicating how the statement was most unlike to

most like their view of being a Biology GTA. Statements that participants felt neutral about were

placed in the zero column, while those statements they most strongly identified with were placed

in the positive number columns, and those they did not identify with were placed into the

87

negative number columns. The participants were then asked to complete short-answer exit

interview questions. When the participants completed their sort, the researcher collected the

statements and the completed grids. There was no discussion during the sorting process by any of

the participants.

During the second phase, a second Biology Lab Coordinator and the experienced GTAs

were asked to participate after their weekly Biology colloquium meeting. The researcher

distributed the Q Sorts, the instructions for the sort, and a copy of the IRB “Informed Consent”

letter to each participant, following the same procedures as used with the new GTAs. The

experienced GTAs openly discussed their sorts as they were completing them, and their

discussion was tape recorded and transcribed.

Role of the Researcher

The researcher is a staff member in The Department of Biology who supervises a non-

major, undergraduate Biology lab, and co-taught the “Effective Teaching” course for new

Biology GTAs, along with the Lead Biology Faculty Member. The researcher obtained IRB

approval prior to collecting data by requesting an IRB Exemption (Appendix 5). The researcher

submitted the “GTA Perceptions of Graduate School Q Sort” and used the instrument in a

commonly accepted educational setting, involving normal educational practices (The “Effective

Teaching” course and a departmental gathering of GTAs) (Appendix 6). The researcher

explained the study to potential participants and distributed all materials relating to the Q Sort.

The researcher collected the letters of consent (See Appendix 4) from willing participants before

conducting the Q Sorting activity. The researcher is unable to make decisions about the hiring or

firing, or assignment of Biology GTAs.

88

Limitations

The limitations of this pilot study follow. The first limitation is that Q Methodology is not

generalizable to the general population (Brown, 1980). With Q Methodology, statistical

reliability or the ability to generalize sample results to the general population is of less concern

because the results are the distinct subjectivities about a topic that are operant or measurable

within the group of participants. In this research, the viewpoints that arise are characteristic of

only this group of GTAs, at this university, in this department, at this point in time. This is the

group of interest for the study thus the larger population of GTAs in other content areas, at

different universities, or at different times is not the focus of the research. The focus of the views

within a specific group of people is an important distinction within Q research. Q

Methodological results are not the percentage of the sample or the general population that

adheres to any of the operant subjectivities (Thomas & Baas, 1992).

The second limitation is the Q Sample. The researcher conducted post-sort focus group

interviews during the “Effective Teaching” course with the participants of this pilot study. These

interviews revealed that new GTAs believed some of the statements used were irrelevant to their

position, and other statements seemed overly repetitive. Those statements have been noted, and

may be removed for future studies. Q Methodology “is useful in that it allows the researcher to

identify groups of participants whose viewpoints are similar, and to examine their differences

from participants having alternative viewpoints. In other words, Q Methodology employs a form

of multivariate analysis that is designed to identify the systematically different ways in which

people respond to propositional statements about a particular topic or issue (LeCouteur &

Delfabbro, 2001, p. 209).” Though the participants in the pilot study found some of the

statements questionable, the second set of sorters did not express the same sentiments about the

89

statements. Using the same Q Sort with the proposed sorters as used with the pilot study to

compare groups is necessary.

The final limitation is the experienced GTAs. Collecting sorts from the new GTAs as

they begin their “Effective Teaching” course provides the viewpoints of all the initial

participants. Some GTAs may leave the program after the “Effective Teaching” course, or may

leave prior to graduation, and thus have their viewpoint removed from the study. GTAs who

leave the program have an important viewpoint that is important to include, but may be hard to

execute. Using Q Methodology as a needs assessment tool may eventually be used to uncover

viewpoints that exist among GTAs who exit the program, but the enrollment statistics were not

included in this study.

Summary

This chapter provided an overview of the research design, the derivations of the general

and specific research hypotheses, and the research questions. Other sections designated the

participants and sampling procedures. The basic procedures for a Q Methodology study were

described in detail. The instrument section described the compilation of the concourse, the Q

Sample, the Q Sort, the conditions of instruction, and the pilot study conducted during the Fall

semester of 2012. There was a detailed descriptions of the methods used in this study. The

statistical treatment section explained how the results of the Q Sorts will be factor analyzed and

interpreted. The role of the researcher and limitations of the study conclude the chapter.

90

CHAPTER IV

RESULTS

The purpose of this chapter is to provide the demographic information on the participants

who sorted for this study, who are new and experienced Biology Graduate Teaching Assistants

(GTAs), two Biology Lab Coordinators, and a Lead Biology Faculty Member. This chapter

further provides the results of the analysis of the Q Sorts, and describes the results of the testing

of the specific research hypotheses. The various viewpoints that emerge from the data, using

factor analysis, are described.

Descriptive Demographics

The American Psychological Association Publication Manual (2010) states that, when the

participants in a research study are human, certain information about them such as demographic

variables, the number of participants, the assignment to groups, and other descriptors should be

adequately described. This would aid in assessing the results, generalizing the findings, and

making comparisons or replications. Q studies, however, are better suited to the exploration of

the specifics; the viewpoints of specific people, specific groups, specific demographics, or the

viewpoints at specific institutions (Watts & Stenner, 2005). Q Methodology is not a test of

differences among people, it looks for the similarities and differences between viewpoints (Van

Exel & de Graaf, 2005). Q allows individuals to self-categorize on the basis of the Q Sort they

produce. At the end of the analyses, we may come to understand an individual in terms of their

association with a particular group or factor (Watts & Stenner, 2005). The results of a Q

Methodological study can be used to describe a population of viewpoints and not a population of

people (Risdon, Eccleston, Crombez, & McCracken, 2003). The reporting of descriptive

91

demographics allows the researcher to look for patterns among sorters who share the same

viewpoints.

There were 34 participants who completed the Q Sort, who are considered the P-Set, or

Person-Set (See Table 5) (McKeown & Thomas, 1988; Van Exel & de Graaf, 2005). Participants

in the initial phase were nine master’s and eight doctoral-level new GTAs in The Department of

Biology in the Fall 2012 semester, who were enrolled in an “Effective Teaching” course for new

GTAs, and the two supervisors of this course. A “new GTA” is defined as “A graduate level

student, who is seeking a master’s or doctoral degree through The Department of Biology in a

large, research-focused, degree granting university, who has less than one year of formal

teaching experience, and who teaches an undergraduate-level laboratory for approximately 20-

hours a week, in exchange for a fee-remission. This GTA is currently enrolled in an "Effective

Teaching" GTA training program.”

The Biology Lab Coordinator and Biology Lead Faculty Member sorted twice each

during the initial phase of the study. They first sorted under the conditions of instruction, “Sort

based upon your view of how a new Biology GTA would perceive graduate school,” and then

completed a second sort under the condition of instruction, “Sort based upon your view of how

an experienced Biology GTA would perceive graduate school.” This type of sort would be

considered a Theoretical Q Sort, which can be constructed under a theoretical condition of

instruction, to represent the point of view of a participant (Brown, 1980). The Theoretical Q

Sorts of the supervisors were included in this study because of the supervisors’ large degree of

involvement with the instructional training of Biology GTAs. Both the Biology Lab Coordinator

and the Lead Biology Faculty Member were supervisors of this cohort of GTAs, and were most

familiar with the viewpoints of both new Biology GTAs and experienced Biology GTAs. The

92

Biology Lab Coordinator had never been a GTA, despite working closely with them, but the

Lead Biology Faculty Member had held a GTA position when completing his graduate degree.

The researcher was interested in whether the supervisors of Biology GTAs would have similar or

differing viewpoints from the actual GTAs. In this phase, there were 21 total Q Sorts collected

from 17 new Biology GTAs, the Biology Lab Coordinator theoretically sorting twice, and the

Biology Lead Faculty Member theoretically sorting twice, for a total of 19 participants in the P-

Set (See Table 5).

There were an additional 10 Q Sorts collected from the experienced GTAs and one sort

from a second Biology Lab Coordinator, in the second phase of the study. These experienced

GTAs had completed an "Effective Teaching" GTA training program. Additionally, these

participants attended the weekly mandatory Biology departmental colloquium for master’s

degree students, meeting on a Thursday evening, and were asked to participate in the study. The

additional Biology Lab Coordinator was asked to sort under the condition of instruction “Sort

based upon your view of how an experienced Biology GTA would perceive graduate school.” Of

the GTAs who attended this colloquium, 100% participated in the Q Sort.

The final set of Q Sorts were collected from experienced GTAs after their doctoral

colloquium, and consisted of four experienced GTAs. The experienced GTAs had completed an

"Effective Teaching" GTA training program. Of the doctoral students who attended this

colloquium, 100% participated in the Q Sort.

There were a final total of 34 participants in the P-Set, and 36 Q Sorts. There were a final

total of 16 doctoral GTAs, and 15 master’s degree GTAs who completed sorts. Because the Q

Sorts, conditions of instruction, and analysis remained the same, sorts from all the phases of the

study could be analyzed together to address the purpose of this study. The demographics of the

93

P-Set are further described in Table 5. There were roughly the same number of males and

females completing Q Sorts. There were only a small number of International GTAs (two)

completing the Q Sort. There was a wide variety of formal and informal teaching experience

among the GTAs, with only two GTAs having no teaching experience at all.

Table 5 – Demographics of New and Experienced Biology GTA

Number Percent

Participation Rate 36 Sorts 100%

Session of Sort Completion

Effective Teaching Course 21 58%

Colloquium 15 42%

Type of Participant

New GTAs 17 48%

Experienced GTAs 14 42%

Biology Lab Coordinator 3 7%

Biology Faculty Member 2 5%

Degree Track

Doctoral 16 52%

Masters 15 48%

Gender

Male 19 52%

Female 17 48%

Origin

International 2 6%

United States 34 94%

Teaching Experience

None 2 6%

Informal 6 17%

94

Formal 28 77%

Data Collection

Data collection for this study occurred during the Fall 2012 semester. Data used in this

research study were Q Sorts taken from two groups of participants. The first group of

participants included the new Biology GTAs, a Biology Lab Coordinator, and the Lead Biology

Faculty Member, sorting during their “Effective Teaching” course. The second group of

participants included the experienced Biology GTAs and an additional lab coordinator, sorting

after a master’s or doctoral weekly mandatory Biology departmental colloquium meeting.

The P-Set were coded based on certain characteristics. The participant identifiers

represented more detail about each participant, to aid in factor interpretation and participant

identification. Table 6 explains the coding system used to identify each participant. As an

example, Sorter #1 (coded den32mf) was a doctoral student (d), an experienced GTA (e), born

and educated in the United States (n), who was 32 years old (32), male (m), and had formal

teaching experience (f). Another example, Sorter #11 (coded men23mu) was a master’s degree

student (m), who was an experienced GTA (e), born and educated in the United States (n), who

was 23 years old (23), male (m), and did not answer the question about teaching experience on

the post-sort interview questions (u). Finally, Sorter #36 (coded blc1exp) was the first (1) The

Biology Lab Coordinator (BLC), theoretically sorting as an experienced GTA (exp), under the

conditions of instruction, “Sort based upon your view of how an experienced Biology GTA

would perceive graduate school.”

95

Table 6 - Coding System for Study Participants

Category Identifier Meaning

Program M Master’s degree

D Doctoral degree

BLC Biology Lab Coordinator

BFM Biology Lead Faculty Member

Experience N New

E Experienced

International Status Y International Student

NBorn and Educated in the United States

Age (numerical) Age in Years

Sex M Male

F Female

Teaching Experience F Formal Teaching Experiences

IInformal Teaching Experiences

U Unanswered

Q Sorting is the process in which participants are asked to sort a Q Sample, developed

from a concourse. The participants sorted based on the same set of conditions of instruction used

in the pilot study, “Based upon your views of graduate school, place each statement into one of

three piles.” The sorting was completed based on their perception of how strongly the statements

were like their views, unlike their views, or if they had a neutral feeling about the statement.

Sorting occurred via a predetermined number of groups (4 – 4 – 5 – 5 – 6 – 6 – 6 – 5 – 5 – 4 – 4)

between the two ends of the continuum (-5, -4, -3, -2, -1, 0, +1, +2, +3, +4, +5). The ends of the

continuum ranged from most unlike my view (-5) to most like my view (+5).

Unlike traditional surveys that require participants to answer each question independently

of their other responses, Q Sorting enables participants to consider the Q Statements in relation

96

to each other, creating a focalization on the participant’s individual subjectivity on the subject

(Bowe, 2010). Prasad (2001) argues that use of the forced choice method (forced matrix) means

that the respondents have to consider their attitudes more carefully, which can bring out true

feelings in response.

Data Analysis

All of the Q Sorts were analyzed using PQMethod (Schmolck & Atkinson, 2002).

Because there is no dedicated function in SPSS for flagging, or creating the descriptive table

required for interpretation of the factors used in Q Methodology, and PQMethod has been

purposefully built to do Q Methodology analyses, it is appropriate to use in a Q Study (Brown,

1980; McKeown & Thomas, 1988; Newman & Ramlo, 2010; Schmolck & Atkinson, 2002).

According to McKeown and Thomas (1988), “data analysis in Q Methodology typically involves

the sequential application of three sets of statistical procedures: correlation, factor analysis, and

the computation of factor scores” (p. 46). After each respondent has provided his own ranking of

the statements, the various ranks are correlated, and the correlation matrix is factor analyzed. A

factor in this case represents a group of persons who have ranked the statements in essentially the

same order - persons who have displayed a common perspective (Brown & Ungs, 1970).

The higher the factor loading, the more highly that sorter is correlated with that factor

(Newman & Ramlo, 2010; Ramlo, 2008). Consequently, those sorters whose views are similar

are highly correlated with the same factor. Thus, the factor loadings determine who loaded on

which factor (Brown, 1980). The factors represent viewpoints or perspectives which exist with

respect to the issue under consideration. The factors which result from a Q Study, therefore, in a

very real sense are results of behavior - that is, they exist as the consequence of a group of

respondents having responded in the same fashion. “Viewpoints,” in this usage, are operant, or

97

measurable. Factors in Q Methodology studies arise from the actual concrete operations of

persons as they model their viewpoints; a factor is the result of behavior (Brown & Ungs, 1970).

The factor-categories are genuine, as opposed to ad hoc categorical, and reflect true viewpoint

segmentation. They are more genuinely "operational definitions" of this-or-that attitude, since

whatever it is they are definitions of - for example, a pro-labor viewpoint has been made

manifest by virtue of behavioral operations expressed through the medium of Q Methodology

(Brown & Ungs, 1970).

Q analyses recognize the sorted-items (statements) as the sample, and the participants as

the variable (McKeown & Thomas, 1988). The fact that these opinions were “amenable to

numerical treatment opens the door to the possibility of clarity in understanding through the

detection of connections which unaided perception might pass over (Brown, 1991, Section 5).”

Factor analysis in Q Methodology reveals how many different perspectives there are (Brown,

1980, 1993). For example, in the pilot study, the factors indicated three different perspectives

new GTAs possess about graduate school. The GTAs who share a common view define the same

factor. In the pilot study, there were three different factors, or “viewpoints,” among new Biology

GTAs.

Analysis and Interpretation

The three factor matrix shown in Table 7 is the result of centroid factor analysis and

Varimax rotation. The combination of centroid and Varimax was chosen because it produced a

clear and detailed description of the data. Other combinations of principle component analysis

(PCA), centroid, Varimax, and hand rotations were completed, but the particular 3-factor

solution revealed the most connections, and was in line with post-sort interview questions. The

three distinct factors that emerged from this combination of data analysis techniques are shown

98

in Table 7, with automatic pre-flagging. An ‘X’ next to the factor loading, in bold, for that

participant, indicating a loading on that factor. The X’s indicate a sort that represents that

particular factor/viewpoint. Participants who did not load significantly on any of the factors, and

therefore were not flagged by the software, are not indicated by any X’s. Their sorts are not

included in a factor view and were not included in the development of the tables related to these

views.

The descriptions and analysis of the factor descriptions are determined by only those Q-

Sorters who are flagged on that factor (Brown, 2009; McKeown & Thomas, 1988; Newman &

Ramlo, 2010). It is necessary to flag sorters before the analyses produce a report that involves a

variety of tables (Newman & Ramlo, 2010). These tables are developed statistically using

PQMethod software (Schmolck & Atkinson, 2002), and they help the researcher’s description of

the views developed from the factor scores. Q Methodology maintains the relationship among

themes within the data as it minimizes the impact of the researcher’s frame of reference (Stainton

Rogers, 1995). It minimizes this impact through complex statistical analysis including

correlation and factor analysis (Brown, 1980; Newman & Ramlo, 2010; Stephenson, 1953).

Table 7 - Factor Matrix with X Indicating a Defining Sort

QSORT 1 2 3

1 den32mf 0.4890X 0.4702 -0.0221

2 den30ff 0.4274 0.6692X 0.0974

3 den29mf 0.5002X 0.323 0.0588

4 den32mf 0.1609 0.7388X 0.296

5 men23mf 0.1905 0.7200X 0.0882

6 men24ff 0.5849X 0.2318 0.1398

7 men24ff 0.4313X -0.0302 0.3947

99

QSORT 1 2 3

8 den27fu 0.7022X 0.4709 -0.0711

9 men24ff 0.7724X 0.1678 0.0866

10 blc2exp 0.7854X 0.1438 -0.0472

11 men23mu -0.3709 0.4872 0.3699

12 men47mf 0.8464X 0.2891 0.0674

13 men24mf 0.1147 0.084 0.7859X

14 den40mf 0.7279X 0.1966 0.0328

15 den30mf 0.5182X 0.1834 0.2462

16 blc1new 0.136 -0.1778 0.5292X

17 dnn30mi 0.9171X 0.0002 0.0496

18 dnn27mf 0.2338 0.5255X -0.1705

19 dnn22fi 0.7844X 0.1507 -0.0267

20 dnn25mf 0.0652 0.7488X -0.0595

21 dny30ff 0.4365X 0.2257 0.136

22 bfmexp 0.0395 0.5303X 0.1101

23 bfmnew -0.4248 0.2074 0.4266

24 mnn29fi 0.2372 0.0944 0.6970X

25 mnn25mu 0.6360X 0.2844 0.2788

26 mnn22fi 0.263 0.2558 0.3808X

27 mnn25fi 0.2914 0.4605X 0.2828

28 dnn22mu 0.3577 0.6426X 0.1123

29 mnn23fi 0.7135X 0.393 -0.0543

30 mnn23fi -0.0384 -0.0769 0.4103X

31 mnn24ff 0.8596X 0.2199 -0.0953

32 mnn22mf 0.8530X 0.1194 0.1228

33 dnn23mf 0.3345 0.6050X -0.0258

34 dnn23mi -0.0841 0.3697 0.6504X

35 mny24fi 0.5793X 0.0828 0.3166

36 blc1exp 0.2042 0.4141X -0.218

% expl.Var. 27% 15% 9%# per factor 18 10 6% per factor 50.0% 27.7% 17.6%

100

QSORT 1 2 3

Note: The coding for the Q Sort ID includes the program (M = Masters, D = Doctoral, BLC = Biology Lab Coordinator, LFM = Lead Faculty Member), experience (n = new, e = experience, international status (yes = international, no = American), age in years, sex (M = male, F = female), and teaching experience (f = formal, I = informal, u = unanswered).

As indicated in Table 7 above, there were three factors that emerged. There were 18 sorts

that were represented by Factor 1, and their loadings ranged from 0.43 to 0.91. Marked with an

X, these are sorts 1, 3, 6, 7, 8, 8, 9, 10, 12, 14, 15, 17, 19, 21, 25, 29, 31, 32, and 35. This factor

was named "The Emerging Teacher" by the researcher. The participants in the group included

ten females and eight males. There was an even split with nine experienced and nine new GTAs.

Seventeen out of the 18 sorts included participants with teaching experience, 12 with formal

experience, four with informal experience, and two who provided no answer.

Ten sorts were represented by Factor 2. Their loadings ranged from 0.41 to 0.75. Marked

with an X, these are sorts 2, 4, 5, 18, 20, 22, 27, 28, 33, and 36. This factor was named “The

Preferred Researcher” by the researcher. The participants in the group included three females,

and seven males. There was an even split with five experienced and five new GTAs. Every

participant had taught before, but one only had informal teaching experience.

Six sorts were represented by Factor 3. Their loadings ranged from 0.38 to 0.79. Marked

with an X, these are sorts13, 16, 24, 26, 30, and 34. This factor was named “The Anxious GTA”

by the researcher. The participants in this group included four females and two males. All of

these sorters were new GTAs except for one doctoral student. Every GTA had taught before, but

mostly (four sorts) in an informal setting, with two teaching in a formal setting.

101

Finally, there were two sorters who did not load on any factor. These two sorters were

sort 11 and 23. One no-loading sort was from a male Master’s degree, experienced GTA who

was 23, and born and educated in The United States. The other was from The Lead Biology

Faculty Member, theoretically sorting as a new GTA. The factors will be more fully described

later in Chapter IV.

After calculating factor scores, two of the tables that are developed for analysis in Q

Methodology are consensus and distinguishing factor statements, which allow the researcher to

explore what is common among and different between the factors (Brown, 1980, 1993, 2009;

McKeown & Thomas, 1988, Newman & Ramlo, 2010). In order to determine the distinguishing

statements, a difference score is calculated within PQMethod. A statement’s factor score is the

normalized weighted average statement score (Z-score) of respondents that define that factor.

Van Exel (2005, p.9) describes difference scores as follows: “The difference score is the

magnitude of difference between a statement’s z-score on any two factors that is required for it to

be statistically significant. When a statement’s score on two factors exceeds this difference score,

it is called a distinguishing statement.” When a statement is not distinguishing between any of

the factors it becomes a consensus statement (van Exel, 2005). Consensus statements are those

statements that are common among pairs of factors.

Table 8 below shows the factor scores, or where those statements would appear in the

ideal sort each factor, with the distinguishing statements for each factor marked by an asterisk

(*), and the consensus statements marked by a cross (†).

Table 8 - Factor Values for Each Statement

102

Factor Arrays

No. Statement 1 2 3

1 All my students are capable of understanding Biology † 2 1 3

2 Being a good teacher is as important as being a good researcher 3 -1* 4

3 Being a TA helps me ask better questions in my research 1 0 0

4 Being a TA will help me to be a good professor someday 5* 2 2

5 I am good at creating a respectful classroom environment 4* 1* -1*

6 I believe I know what it takes to be a good researcher 0 3* 0

7 I came to grad school mainly so I could do research -2* 5* 3*

8 I can balance being a good teacher with being a good student 4 4 0*

9 I dislike teaching, and wish I could spend more time on my research

-5 -1* -3*

10 I don't think teaching requires a lot of emotion † -2 -2 -3

11 I feel like an outsider, and that people at grad school won't accept me

-4 -4 -1*

12 I feel like I could go into teaching as a profession 5* 0* 3*

13 I feel like I need to constantly monitor my students for cheating -1 -2 -4*

14 I feel like I'm a good teacher because I am closer in age to my students

-1 -3 -2

15 I feel like it will be easy to manage my class 1 1 -3*

16 I feel like my fellow TAs will help me to teach better 1 1 5*

17 I feel like students look at me weird when I tell them I'm a TA -4 -5 -5

18 I feel overwhelmed with work my advisor gives me -2 -1* -3

19 I feel pretty comfortable using technology in my class † 2 5 1

20 I feel pretty confident that I'm a good teacher 5* 4* -5*

103

Factor Arrays

No. Statement 1 2 3

21 I feel self-confident when I teach 5* 4* -5*

22 I have a lot of anxiety about teaching, because I don't know what to expect

-5 -5 2*

23 I have had dreams about my teaching or research -1 0 0

24 I have lost sleep because I'm worried about teaching † -4 -4 -2*

25 I have no idea what students think about me, and that makes me uncomfortable

-4 -4 4*

26 I have no idea what the level of understanding is with these students

-2 -3 3*

27 I have to repeat myself over and over to get these students to understand me †

-1 0 -1

28 I know the university policies that relate to my research 0 -1 -1

29 I know what attributes make a good teacher 3 3 0*

30 I know what the Biology department expects from my teaching 0 1 -5*

31 I know what the department expects from my research 1* 3* -4*

32 I learned best by actively doing labs 1 4 2

33 I learned best by listening to professors teach 0* -2* 1*

34 I like doing research over teaching -3* 5* 1*

35 I like doing teaching over research 3* -5* -4*

36 I think all this teaching gets in the way of my research -5* 2* -2*

37 I think most of my students learn in a way that's similar to the way I learn †

-1 -2 -1

38 I think one of the most important things about being a TA is being 3 1* 3

104

Factor Arrays

No. Statement 1 2 3

ethical †

39 I think research is very challenging 1* 3 4

40 I think some people are natural teachers † 3 5 5

41 I think teaching is very challenging -2* -3* 5*

42 I think that I give good teaching presentations 4* 2* -2*

43 I think you can be "taught to teach" -1* 3 1

44 I want all students to actively participate in my class † 4 2* 4

45 I want to teach the same way my favorite professor taught 2* 0* 5*

46 I worry that certain students in my class might know more about Biology than I do

-3 -5* -2

47 If I teach well, I will get good student evaluations † 0 -1 0

48 I'm worried that the students won't be able to understand me -3 -3 2*

49 I've had family problems because of the pressures of graduate school†

-5 -4 -4

50 Most of my students will do just enough to get by 0* 2 1

51 My students will like Biology because I can make it interesting 2 -1* 1

52 My students will respect me because I'm fair 2 0* 2

53 Sometimes I worry that I might have chosen the wrong career † -3 -2 -1

54 Using social media (like Twitter or Facebook) helps me to feel like I’m not alone†

-3 -3 -3

Note: Distinguishing Statements marked by *, and consensus statements marked by †

105

A table of the distinguishing statements specifically for each factor will be presented in

the analyses for that factor, within the next section, to aid the reader within the interpretation of

each factor. The consensus statements are also discussed in detail within the next section, while

their interpretation will be revisited in the recommendations section of Chapter V. A

representative sort is created from the sorts of those who are represented by a particular factor.

This representative sort is one sort that represents that factor/viewpoint (Brown, 1980; Brown,

2008; McKeown & Thomas, 1988, Newman & Ramlo, 2010).

Brown & Good (2010, p.7) describe how the factor scores are calculated, “The factor scores are

then calculated by multiplying each statement’s Q Sort score by the weight and then summing

each statement across the weighted Q Sorts comprising the factor, with weighted statement sums

then being converted into a factor array presented in the form of the original metric.” An

example of a representative sort from Factor 1 is show in Figure 7 below and contains the same

information about Factor 1 contained in Table 8.

106

Generally, the statements ranked at the extreme ends of a representative sort are called

the characterizing statements. In other words, the statements that ranked as most like my view

and least like my view are used to provide a starting point to describe the view represented by

that factor. These statements demonstrate to the researcher the statements that sorters who load

on this factor feel the most strongly about. The consensus and distinguishing statements are used

to illuminate the similarities and differences between the factors respectively. To further

understand an individual participant’s sort, it is generally advisable to conduct a post-sort

interview to have the participant explain to the researcher why the sort was arranged the way it

was. This can allow the researcher to gain confirmation of the analysis and/or further insight into

the factor’s meaning (McKeown & Thomas, 1988; van Exel, 2005; Brown, 2009). For this study,

107

Least like my

viewneutra

l

Most like my

view-5 -4 -3 -2 -1 0 1 2 3 4 5

22 17 53 7 27 30 39 52 40 8 12

49 25 54 41 23 50 32 51 35 44 20

36 24 34 26 43 6 15 19 2 5 4

9 11 46 10 14 33 3 45 38 42 21

48 18 13 47 16 1 29

37 28 31

Figure 6 – Representative Sort for Factor 1

the researcher asked post-sort questions on the sorting grid sheet and allowed the participant to

self-report about the decision making process used during the sorting process. The questions

were:

1. Tell us why you selected the four statements you placed under +5 (most like my

view)?

2. Tell us why you selected the four statements you placed under -5 (least like my view)?

3. Please describe your decision-making process during the sort. Did you gain insight

about your views as you sorted the statements? If so, please describe.

4. What are you planning to do after graduate school?

5. Briefly (a few sentences) describe what you would like to get out of a TA

training program:

The responses to these questions allowed the researcher to further identify the

commonalities among those sorters who were represented by the same factor as well as to help

define and articulate the factor itself. Comments that help clarify the interpretation of each of the

factors will be included in the next section, along with the description of each factor.

Factor 1

In this study, the researcher asked the GTAs to sort 54 statements based upon their views

about graduate school and being a GTA. The analysis resulted in three views or factors. Of the

36 Q Sorts, 18 were represented by Factor 1, or what the researcher has named "The Emerging

Teacher". These GTAs are those who feel confident that they are good teachers, and that they

could go into teaching as a profession. Table 9 and Table 10 contain the top eight most-like and

108

least-like statements for this factor, respectively. Table 11 indicates the distinguishing statements

for Factor 1.

Table 9 - Eight Most-Like My View Statements for Factor 1 "The Emerging Teacher" with a † indicating a Distinguishing Statement.

No. Statement Grid Pos.

12 I feel like I could go into teaching as a profession † 5

20 I feel pretty confident that I'm a good teacher † 5

4 Being a TA will help me to be a good professor someday † 5

21 I feel self-confident when I teach † 5

8 I can balance being a good teacher with being a good student 4

44 I want all students to actively participate in my class 4

5 I am good at creating a respectful classroom environment † 4

42 I think that I give good teaching presentations 4

The top four “most like my view” statements (Table 9) (12, I feel like I could go into

teaching as a profession; 20, I feel pretty confident that I'm a good teacher; 4, Being a TA will

help me to be a good professor someday; 21, I feel self-confident when I teach) indicate

confidence in teaching abilities. This is further elucidated in that 12 of the 18 GTAs who were

represented by this factor specifically listed professor or teaching as their desired career path

(See Table 12). They also provided preferences for skills they would like to acquire from The

“Effective Teaching” course. Most of the GTAs requested advice about obtaining teaching skills,

ranging from teaching strategies and classroom management, to inspiring students. Over half of

these GTAs were experienced, and 13 of the 18 participants had formal teaching experience.

109

Multiple GTAs expressed that they were confident people in general, and participant MNN23FI

indicated that teaching was a “natural extension” for her, and that she enjoyed it.

Table 10 - Eight Least-Like My View Statements for Factor 1 "The Emerging Teacher" with a † indicating a Distinguishing Statement.


17 I feel like students look at me weird when I tell them I'm a TA -4


-4

24 I have lost sleep because I'm worried about teaching -4

11 I feel like an outsider, and that people at grad school won't accept me -4


-5

49 I've had family problems because of the pressures of graduate school -5

36 I think all this teaching gets in the way of my research † -5

9 I dislike teaching, and wish I could spend more time on my research † -5

GTAs loading on Factor 1 also indicated a preference for teaching over research, as

demonstrated by the distinguishing statements for this factor (Table 11). Two statements (35, I

like doing teaching over research; 39, I think research is very challenging), indicate that this

group of GTAs expresses less confident about research than they do teaching. Statement 31, I

know what the department expects from my research, and statement 7, I came to grad school

mainly so I could do research, were neutral to negative in a representative sort by these GTAs.

Participant MNN24FF said, “I love TAing, so I really don’t get anxiety about it. I can always

seek help from [the biology lab coordinator]…. my research, however….” Participant

MNN25MU said, “Teaching labs keeps me immersed in conducting and setting up experiments.”

110

He went on, “Research for me has been a bit strenuous for my first semester, but I believe

teaching will be beneficial for research.”

Table 11 - Distinguishing Statements for Factor 1 " The Emerging Teacher ".

Factors

1 2 3

No. Statement

Rank Score

Rank Score

Rank Score

12 I feel like I could go into teaching as a profession

5 0 3

20 I feel pretty confident that I'm a good teacher 5 4 -5

4 Being a TA will help me to be a good professor someday

5 2 2

21 I feel self-confident when I teach 5 4 -5

5 I am good at creating a respectful classroom environment

4 1 -1

42 I think that I give good teaching presentations 4 2 -2

35 I like doing teaching over research 3 -5 -4

45 I want to teach the same way my favorite professor taught

2 0 5

39 I think research is very challenging 1 3 4

31 I know what the department expects from my research

1 3 -4

50 Most of my students will do just enough to get by

0 2 1

33 I learned best by listening to professors teach 0 -2 1

43 I think you can be "taught to teach" -1 3 1

7 I came to grad school mainly so I could do research

-2 5 3

41 I think teaching is very challenging -2 -3 534 I like doing research over teaching -3 5 1

36 I think all this teaching gets in the way of my research

-5 2 -2

111

Factors

1 2 3

No. Statement

Rank Score

Rank Score

Rank Score


-5 -1 -3

Table 12 - Post-Sort Interview Responses for Factor 1

Code Specific Teaching Experience

Career plan GTA program Preferences

BLC2EXP 4 years Teach N/A (Biology Lab Coordinator)

DEN27FU 7 years Post Doc to Professor Teaching strategies, delivery, classroom management

DEN29MF 8 years TA, Field instructor, lecture 1 semester

NGO (Non-Governmental Organization)/Conservation Advocacy

Add to Teaching Portfolio

DEN30MF undergrad TA, ESL Post Doc to Professor clear expectations, classroom management, teaching portfolio prep, how to design courses

DEN32MF 5 years TA, high school, camp, Jr. high

Professor Help students excel

DEN40MF 10 years, youth group, church

Unsure clear expectations from courses, better matching of TAs to abilities (less favorite playing)

DNN22FI tutoring, research lab Professor looking to find her teaching style

DNN30MI school in China Professor mentorship program, individual lab prep

DNY30FF 4 semesters Academia or research teaching skills

MEN24FF 5 semesters as a grad student Intro, Micro, Genetics

Lab Tech, or get a teaching degree

How to break down procedures, work through processes, up to date with principles, diverse learners

MEN24FF 4 semesters A+P Med school Gain confidence, be more comfortable in front of students

112



MEN24FF 4 semesters Vet school or PA school public speaking training

MEN47MF 20 years high school, TA 2 years

PhD (could teach a TA training program)

MNN22MF 5 semesters Ph.D. to professor inspire students

MNN23FI coaching, horseback riding

Vet school or PA (Physicians’ Assistant) school

situational knowledge, diverse students, handling unique circumstances

MNN24FF 3 semesters PA (Physicians’ Assistant) program

how to motivate students, inspire people to want to learn

MNN25MU Unanswered PhD, consulting how to prepare, communication, refreshing protocols, procedures

MNY24FI 3 semesters informal Pharmaceutical or teach classroom management


Five GTAs who were represented by Factor 1 described, in response to post-sort

questions, how it was easy to pick statements that were least-like their views, but it was difficult

to prioritize what was like their views. Participant MNN22MF described how he could “instantly

discredit anything that suggested teaching is so unimportant.” He also described how he felt he

was “inherently a teacher” because he “possessed factors like patience and good communication

skills.” This sentiment is expressed by the eight least-like-my-view statements, which are noted

in Table 10.

Not only do the most-like statements in Factor 1 indicate teaching confidence, the least-

like statements indicate that this GTA group has not encountered many of the negative

sentiments that are often expressed in the literature, such as anxiety about teaching, family

problems from pressures of graduate school, or wishing they could spend less time teaching and

more time researching. These GTAs expressed in their post-sort interview questions that “I knew

113

what I was getting into (DNN22FI),” and “I know I chose the right career path (DNY30FF).”

Another experienced Factor 1 GTA noted, “I do not rely on teaching or classes for my social life

(DEN40MF).”

As Factor 1 GTAs described their decision-making processes during the sort, some

expressed that the activity gave them insight into who they were as teachers, what other GTAs

may be feeling, or that it reinforced things they already knew about themselves. Participant

DEN30MF described his insight, after the sort, “It made me think more clearly about how I teach

and how I feel, rather than how I WANT to feel or think I should, about teaching.” Participant

MNN24FF said, “I definitely learned things about myself, and this forced me to weigh situations

out in my head.” Participant DEN40MF described how he “was not comfortable in being

constrained to equalized piles, and limited to ranges. I would have much more in the -5, -4, -3

piles.”

Participant MNN23FI described what she would like to get out of a TA program, “I

would like to get answers to certain situations and possibly insight into different types of

students/circumstances I may be unaware of, because I was a different type of student than what

I teach.” Multiple GTAs suggested they would like classroom management skills, teaching

strategies, and how to better work with students. Participant MNN24FF suggested that learning

about how different types of students learn would help her “to become a better student myself.”

She indicated that learning how to help students want to learn, this “might stimulate and inspire

me to become a better student myself.”

“The Emerging Teachers” are GTAs in this group who feel confident in their teaching

abilities, and express interest in teaching as a profession. This factor is characterized by

expressions of confidence in their teaching abilities, self-confidence as teachers, and that their

114

position as GTAs will help them to become better teachers, as statements that are most-like their

views. GTAs who were represented by this factor ranked statements about anxiety about

teaching, family problems, lack of confidence in their abilities, and feeling like an outsider as

least-like their views. The majority of GTAs who were represented by this factor specifically

listed professor or teaching as their desired career path (See Table 12). This factor also included

the second Biology Lab Coordinator.

Factor 2

Of the 36 Sorts, 10 were represented by Factor 2, or what the researcher has named "The

Preferred Researcher.” These GTAs are those who came to graduate school mainly so they could

do research, and prefer research over teaching. This is further detailed in the post-sort interview

questions that these GTAs completed. Table 16 demonstrates that almost all of the GTAs who

were represented by Factor 2 described their career aspirations as “academia or research.” These

career aspirations further supported the researcher’s naming of Factor 2. The Lead Biology

Faculty Member and the Biology Lab Coordinator, theoretically sorting as experienced GTAs

were both included in this factor. Table 16 helped substantiate this view, which was further

supported by the Q Sorts. Table 13 and Table 14 contain the top eight most-like and least-like

statements for this factor, respectively.

Table 13 - Eight Most-Like My View Statements for Factor 2 "The Preferred Researcher” with a † indicating a Distinguishing Statement.


7 I came to grad school mainly so I could do research † 5

34 I like doing research over teaching † 5

40 I think some people are natural teachers 5

115


19 I feel pretty comfortable using technology in my class 5

32 I learned best by actively doing labs 4

8 I can balance being a good teacher with being a good student 4

21 I feel self-confident when I teach † 4

20 I feel pretty confident that I'm a good teacher † 4

Table 14 - Eight Least-Like My View Statements for Factor 2 "The Preferred Researcher” with a † indicating a Distinguishing Statement.



-4

24 I have lost sleep because I'm worried about teaching -411 I feel like an outsider, and that people at grad school won't accept me -449 I've had family problems because of the pressures of graduate school -4


-5

17 I feel like students look at me weird when I tell them I'm a TA -535 I like doing teaching over research † -546 I worry that certain students in my class might know more about

Biology than I do †-5

Table 15 - Distinguishing Statements for Factor 2 "The Preferred Researcher".

Factors

1 2 3

No

. Statement

Rank

Score

Rank

Score

Rank

Score


-2 5 3

116

Factors

1 2 3

No

. Statement

Rank

Score

Rank

Score

Rank

Score

34 I like doing research over teaching -3 5 1



6 I believe I know what it takes to be a good researcher

0 3 0


1 3 -4

44 I want all students to actively participate in my class

4 2 4

36

I think all this teaching gets in the way of my research

-5 2 -2


38 I think one of the most important things about being a TA is being ethical

3 1 3


4 1 -1


2 0 5

52 My students will respect me because I'm fair 2 0 2


5 0 3

18 I feel overwhelmed with work my advisor gives me

-2 -1 -3

2 Being a good teacher is as important as being a good research

3 -1 4


-5 -1 -3

51 My students will like Biology because I can make it interesting

2 -1 1

33 I learned best by listening to professors teach 0 -2 1

41 I think teaching is very challenging -2 -3 535 I like doing teaching over research 3 -5 -4


-3 -5 -2

117

The top four “most like my view” statements (7, I came to grad school mainly so I could

do research; 34, I like doing research over teaching; 40, I think some people are natural teachers;

19, I feel pretty comfortable using technology in my class) indicate preference for research

activities. This factor is dominated by six doctoral GTAs, with only two master’s GTAs loading

on this factor. Also, both the Biology Lead Faculty Member and the Biology Lab Coordinator

theoretically sorted on this factor (See Table 16). Participant MEN23MF explained that his most-

like items were regarding “research, and how it consumes my life.” The Biology Lead Faculty

Member expressed that GTAs “care more about research than when they started, and may have

more fears about research – finding it to be harder than they thought.”

GTAs represented by Factor 2 also indicated a preference for research over teaching, as

demonstrated by the distinguishing statements for this factor (See Table 15). Two statements (35,

I like doing teaching over research; 46, I worry that certain students in my class might know

more about Biology than I do), were ranked the lowest by GTAs who were represented by this

factor. These statements indicate that this group of GTAs perceives that there is no way their

students know more about Biology than they do. This prioritization of the statements suggests

the GTAs are highly confident in their Biology content knowledge. Participant DEN32MF

suggested that “I like research because it is challenging. Because of this, I decided to go to grad

school. Although I don’t mind teaching, research is my passion. I came to grad school for

research, not to teach.”

GTAs who were represented by Factor 2 were highly analytical, based on their written

responses to the post-sort questions, in making their choices about statements that were most-like

or most-unlike their views. Participant DNN25MF detailed how he used a combination of facts,

118

logic, experiential knowledge, inference, and speculation in his sorting process. Participant

DEN32MF said, “Most of the decisions were provided by the sorting mechanism itself,

otherwise I tried to use my gut reactions or emotional responses. It was difficult to sort some of

the statements because I had mixed feelings, or there was a negative tone. I don’t agree or

disagree with the

Table 16 - Post-Sort Interview Responses for Factor 2 “The Preferred Researchers”



BFMEXP 20+ years Faculty N/A Biology Faculty Member

BLC1EXP 15 years teach N/A Biology Lab Coordinator

DEN30FF 8 semesters TA, ballet and dance

Post Doc to Professor Effective Teaching Techniques, Conduct, How to prepare a course of her own, How to teach a lecture

DEN32MF 4 years Principles, guest taught evolutionary biology

Prefer research or industry

Prepare for teaching, lecture, work with students, diverse but not intensive, strike a balance, would be nice to have a more intense program for those TAs planning to teach

DNN22MU Unanswered Academia or research student motivation, confidence, foundations

DNN23MF 3 semesters Academia or research educational foundations, troubleshooting

DNN25MF 2 semester TA Post Doc to Professor new ways of teaching, understanding diverse students, puzzle-based

119



learning

DNN27MF 1 semester Business mentorship

MEN23MF 2 years Biotech, Pharmaceutical, or Industry

Preparation, group work

MNN25FI English abroad, nature center

NGO/Nonprofit motivation, peer mentoring


statement based on the tone, but rather the content. It was an interesting process.” The “Preferred

Researchers” were highly analytical in their processes, and detailed more about the sorting

process than GTAs who were represented by the other factors. They took a very scientific

approach to the process, which is interesting because they are very scientifically-minded

students, and approached sorting like doing research.

Factor 2 GTAs did not dislike teaching; they preferred research. Participant MNN25FI

indicated that “some people were born to teach, and have a natural gift. I don’t have much

anxiety about teaching, because I have experience.” The Lead Biology Faculty Member

indicated that as GTAs gained experience, they wouldn’t be scared of teaching anymore.

Participant DEN30FF further explained that “I am very clear about my goals and had thought out

my career path prior to grad school.” She indicated that she wanted to do a post-doc after

graduating, and then go on to become a professor. “I don’t hate teaching; I just prefer to do

research. My advisor expects a lot of me, and I tend to overwhelm myself with my research.”

120

She also detailed that she felt like she had changed dramatically as she progressed through her

grad school/TA career.

This group of GTAs acknowledges that they have some natural teaching abilities

(statement 40, I think some people are natural teachers), and that they can be good teachers (8, I

can balance being a good teacher with being a good student; 21, I feel self-confident when I

teach; and 20, I feel pretty confident that I'm a good teacher), despite their preference for

research. “The Preferred Researchers” further emphasized this point in their most-unlike

statements, as well. They were confident that the students looked at them positively (25, I have

no idea what students think about me, and that makes uncomfortable; and 17, I feel like students

look at me weird when I tell them I'm a TA), that they fit in, in grad school (11, I feel like an

outsider, and that people at grad school won't accept me), and that they were comfortable in their

GTA position in grad school (22, I have a lot of anxiety about teaching, because I don't know

what to expect).

In the distinguishing statements for “The Preferred Researchers,” statement 6 (I believe I

know what it takes to be a good researcher) and 31 (I know what the department expects from

my research) were indicators that these GTAs, more so than those who were represented by the

other factors, were research oriented. Many of the GTAs who were represented by this factor

indicated that teaching was easy, or at least not challenging compared to their research

(negatively placed statements 41, I think teaching is very challenging and 35, I like doing

teaching over research).

The post-sort interview questions (See Table 16) were revealing, in that most of the

Factor 2 GTAs planned to pursue a career in academia. Even though the sorters represented by

Factor 2 preferred research, they asked for professional development in the “Effective Teaching”

121

course that would enhance their teaching skills. They wanted to be assisted in developing their

teaching skills, working with diverse students, and motivating their students. Professional

development for these GTAs, who are planning to continue into academia and will be future

professors, may have the biggest impact on their success as there are potential long term effects –

beyond these GTAs’ time as GTAs at this university – that could influence the learning of future

students at other institutions as well as tenure and promotion of these potential future academics.

"The Preferred Researcher” are GTAs who came to graduate school mainly so they could

do research, and prefer research over teaching. This factor also included the Lead Biology

Faculty Member and the Biology Lab Coordinator, theoretically sorting as experienced GTAs.

Almost all of the GTAs who were represented by Factor 2 described their career aspirations as

academia or research. They felt like they were good teachers, and that they had some natural

teaching abilities, but that they just preferred research. They did not display anxiety about

teaching, or lack confidence in their abilities, and they were highly confident that they knew their

content matter. The majority of GTAs who were represented by Factor 2 were doctoral students.

Factor 3

Of the 36 Q Sorts, six were represented by Factor 3, or what the researcher has named

"The Anxious GTA". This included The Biology Lab Coordinator, sorting as a new GTA. These

GTAs do not show a preference for teaching or research, but instead look to juggle the two,

along with being a student. The top eight most-like my view statements are in Table 17. These

GTAs rated statement 41 (I think teaching is very challenging) and statement 39 (I think research

is very challenging) as most-like their view. They asserted a need to be good at both teaching and

research activities (statement 2, Being a good teacher is as important as being a good researcher),

while expressing that they may not be good at either (statement 25, I have no idea what students

122

think about me, and that makes me uncomfortable; statement 41, I think teaching is very

challenging; statement 39, I think research is very challenging). They want to teach like their

favorite professors (statement 45) yet also believe they will learn a lot from their peers

(statement 16, I feel like my fellow TAs will help me to teach better). This was the only factor

that indicated they believed they would learn from their peers.

Table 17 - Eight Most-Like My View Statements for Factor 3 “The Anxious GTA” with a † indicating a Distinguishing Statement.


40 I think some people are natural teachers 5

41 I think teaching is very challenging † 5

45 I want to teach the same way my favorite professor taught † 5

16 I feel like my fellow TAs will help me to teach better † 5

25 I have no idea what students think about me, and that makes me uncomfortable †

4

44 I want all students to actively participate in my class 4

39 I think research is very challenging 4

2 Being a good teacher is as important as being a good researcher 4

Table 18 - Eight Least-Like My View Statements for Factor 3 “The Anxious GTA” with a † indicating a Distinguishing Statement.


31 I know what the department expects from my research † -413 I feel like I need to constantly monitor my students for cheating † -435 I like doing teaching over research † -449 I've had family problems because of the pressures of graduate school -430 I know what the Biology department expects from my teaching † -517 I feel like students look at me weird when I tell them I'm a TA -521 I feel self-confident when I teach † -520 I feel pretty confident that I'm a good teacher † -5

123

Table 19 - Distinguishing Statements for Factor 3 “The Anxious GTA.”

Factors

1 2 3

No

. Statement

Rank

Score

Rank

Score

Rank

Score

41 I think teaching is very challenging -2 -3 545 I want to teach the same way my favorite

professor taught2 0 5

16 I feel like my fellow TAs will help me to teach better

1 1 5


-4 -4 4


-2 5 3


-2 -3 3


5 0 3

48 I'm worried that the students won't be able to understand me

-3 -3 2


-5 -5 2

33 I learned best by listening to professors teach 0 -2 134 I like doing research over teaching -3 5 1

8 I can balance being a good teacher with being a good student

4 4 0

29 I know what attributes make a good teacher 3 3 0


-4 -4 -1


4 1 -1

24 I have lost sleep because I'm worried about teaching

-4 -4 -2


-5 2 -2



-5 -1 -3

125

Factors

1 2 3

No

. Statement

Rank

Score

Rank

Score

Rank

Score

15 I feel like it will be easy to manage my class 1 1 -3


1 3 -4

13 I feel like I need to constantly monitor my students for cheating

-1 -2 -4

35 I like doing teaching over research 3 -5 -4

30 I know what the Biology department expects from my teaching

0 1 -5



GTAs who were represented by Factor 3 demonstrated a lack of confidence (25, I have

no idea what students think about me, and that makes me uncomfortable; and 48, I'm worried

that the students won't be able to understand me), which is not mentioned by GTAs loading on

the other two factors. In the distinguishing statements for this factor (See), statement 25 (I have

no idea what students think about me, and that makes me uncomfortable), statement 26 (I have

no idea what the level of understanding is with these students), statement 48 (I'm worried that the

students won't be able to understand me), helped distinguish this factor from the other two

factors. The post-sort interview questions which detailed the “GTA Program Preferences” are

found in Table 20, and further provided differences between this factor and the others.

Two statements made by sorters who were represented by Factor 3 appeared to be at odds

with one another. Statement 12 (I feel like I could go into teaching as a profession) was ranked

as a +3, and statement 20 was ranked as a -5 (I feel pretty confident that I'm a good teacher).

126

These two statements are at ranked on opposite ends of the grid. The GTAs feel that they are not

good teachers, but they feel they could go into teaching as a profession needs further

clarification. Do these GTAs feel they could go into teaching as a career, because it is easier than

research, or that they will eventually “get the hang of” teaching? These GTAs are obviously

uncomfortable and anxious, and it would be good to clarify what they mean by these statements.

The bottom four “least like my view” statements (20, I feel pretty confident that I'm a

good teacher; 21, I feel self-confident when I teach; 17, I feel like students look at me weird

when I tell them I'm a TA; 30, I know what the Biology department expects from my teaching)

for GTAs who were represented by this factor indicated not that they didn’t like teaching, but

that they were both unsure of their abilities to teach, and didn’t know how to teach yet (See

Table 18). They also seemed to lack a sense of expectation from the department in regards to

their teaching (statement 30, I know what the Biology department expects from my teaching).

They repeated that they were not confident in their teaching (statement 20, I feel pretty confident

that I'm a good teacher; statement 21, I feel self-confident when I teach).

Five out of the six GTAs who were represented by this factor were new GTAs. Four of

the six were Master’s degree students. Participant DNN23MI indicated, “I have witnessed

natural teachers, and I am sure I am not one. I always feel more comfortable after seeing

someone else do a similar talk. I know I suck at teaching, I have a lack of self-esteem,

whatever….” He continued to describe his process, “I did however realize I want to be a lot like

my past professors. Perhaps that’s part of the pressure. I am very aware of my lack of confidence

and nervousness. I just want to survive.” Another participant, MNN23FI described how “I feel

like I can only excel at one, if I put a lot of effort into both, each becomes mediocre.”

127

Factor 3 was characterized by displays of anxiety and lack of confidence, further

evidenced from the post-sort interview questions (See Table 20). Participant DNN23MI

expressed, “I know I suck at teaching…I lack self-esteem.” Participant MNN23FI stated, “I have

no experience in public speaking, I have social anxiety.” Participant MNN29FI said, “Public

speaking can terrify me, even if I’m good at hiding it, so I never feel confident when I’m

lecturing.” Participant MEN24MF explained, “I don’t feel I’ve been briefed on what is

Table 20 - Post-Sort Interview Responses for Factor 3 “The Anxious GTA.”



BLC1NEW 15 years Teach N/A Biology Lab Coordinator

DNN23MI tutoring elementary Academia or research

surviving teaching, feels very uncomfortable, insecure

MEN24MF TA 5 semesters, Nature program

PhD, then research/teach

basic managing and leadership skills, individual styles should be nourished, how to self-motivate, promote scholarship

MNN22FI tutoring Phd, research motivating students

MNN23FI tutoring, helping friends

industry reduce anxiety, confidence, teaching portfolio, how to teach clearly and speak in public

MNN29FI tutoring, life skills to developmentally disabled

Phd, research speaking to groups, explain using multiple methods


expected from my TA and I’m not sure what students expect and feel about my classes. This is

OK. Trial by fire is very effective. HA! I’m not confident that my managerial practices are

128

received well AT ALL.” The Biology Lab Coordinator also was represented by this factor, when

theoretically sorting as a new GTA. She wrote, “I thought about what it means to be a new GTA,

why they came here, and what they want to do – and how their expectations and frustrations

develop.”

“The Anxious GTA” are those GTAs who do not show a preference for teaching or

research, but instead look to juggle the two, along with being a student. They asserted a need to

good at both teaching and research activities, while expressing that they may not be good at

either. These GTAs expressed insecurity in their abilities, being uncomfortable in their position,

and feeling anxious. They also stated that they were frustrated because they didn’t know the

department’s expectations of them. GTAs who were represented by this factor were mostly

master’s degree students, and the Biology Lab Coordinator.

Consensus Statements

Q Methodology is a powerful tool for determining consensus and perspectives of a group

(Ramlo, 2011). The consensus statements can highlight the similarities between factors (See

Table 21). These would be views shared by all the GTAs. This study uncovered 13 consensus

statements, which allow the commonality of the GTAs’ sorting to be expressed.

Perceptions of GTA status were indicated by statements 14 (I feel like I'm a good teacher

because I am closer in age to my students) and 17 (I feel like students look at me weird when I

tell them I'm a GTA). This is note-worthy because all GTAs sorted statement 14 between the -2

and 0 columns. They also sorted statement 17 between the -3 and -4 columns. The literature

often suggests that GTAs find the opposite is true – being closer in age to their students is a

problem, rather than a positive (Austin, 2002). Because many new GTAs lack teaching

experience, have not had adequate training to deal with power issues in the classroom, and are

129

often close in age to (or younger than) the students they teach, they may feel that their credibility

is called into question by their students (Golish, 1999). GTA training research reveals that one of

GTAs' fears or uncertainties is establishing credibility with their students (Hendrix, 1995;

Simonds, Jones, & Bedore, 1994; Worthen, 1992). Graduate teaching assistants may feel like

they need to work harder to establish their credibility in the classroom because they lack the

initial credibility or status of full-time faculty (Hendrix, 1995), along with their young age.

Table 21 - Consensus Statements – Statements in Common Amongst Factors

Factor Arrays

1 2 3

No. Statement

14* I feel like I'm a good teacher because I am closer in age to my students

-2 1 0

17* I feel like students look at me weird when I tell them I'm a GTA

-4 -3 -3

19* I feel pretty comfortable using technology in my class

3 4 4

23* I have had dreams about my teaching or research

-1 0 -2

27 I have to repeat myself over and over to get these students to understand me

-1 0 -2

28 I know the university policies that relate to my research

0 0 -1

33* I learned best by listening to professors teach

0 1 0

37 I think most of my students learn in a way that's similar to the way I learn

-1 -3 0

38 I think one of the most important things about being a GTA is being ethical

2 2 3

40* I think some people are natural teachers 4 5 4

47 If I teach well, I will get good student evaluations

0 -2 0

130

49* I've had family problems because of the pressures of graduate school

-5 -5 -5

54* Using social media (like Twitter or Facebook) helps me to feel like I'm not alone

-3 -1 -2

All of the GTAs ranked statement 19 (I feel pretty comfortable using technology in my

class) as most-like their view. Because Biology is a technology-rich STEM discipline, GTAs

have developed their conception of their content knowledge using technology (Shulman, 1986).

GTAs across all the factors also agreed that statement 38 (I think one of the most important

things about being a GTA is being ethical) was most-like their view.

GTAs were in agreement across all factors that statements 49 (I've had family problems

because of the pressures of graduate school) and 54 (Using social media (like Twitter or

Facebook) helps me to feel like I'm not alone) were unlike their view. Graduate school is often

highly competitive, and emotionally exhausting (Jacobs & Dodd, 2003); however, this cohort of

GTAs did not seem to share this sentiment. The balance of school/personal life does not appear

to have affected this group of GTAs the way it is described in the literature (Ward & Wolf-

Wendel, 2004; Ward, 1998).

There were many statements that the three factors agreed were neutral to their viewpoint

(statement 23, I have had dreams about my teaching or research; statement 27, I have to repeat

myself over and over to get these students to understand me; statement 28, I know the university

policies that relate to my research; statement 33, I learned best by listening to professors teach;

and statement 47, If I teach well, I will get good student evaluations). These statements did not

hold particular significance for the sorters, but were represented in the literature. The importance

of the needs assessment stage of program evaluation becomes apparent, as this particular cohort

of GTAs share differing viewpoints from the GTA literature.

131

Consensus statements in a Q Methodology study may reveal similarly shared

perspectives that are important to working with groups. These consensus views may facilitate

dialogue and collaboration among the groups’ membership (Ramlo, 2005). This study uncovered

13 consensus statements, which allow the commonality of the GTAs’ sorting to be expressed.

GTAs across all the factors ranked items about being similar in age to their students, using

technology, and the pressures of graduate school, similarly. Many of the consensus items from

this study indicated neutral viewpoints.

Results of Testing the Research Hypotheses


The first research hypothesis stated that, “A variety of viewpoints about graduate school

will exist among biology GTAs.” Three GTA factors, or viewpoints about graduate school,

emerged as a result of analyzing the Q Sorts. These viewpoints were, “The Emerging Teacher,”

“The Preferred Researcher,” and “The Anxious GTA.” Each represents a distinctive view about

being a GTA. “The Emerging Teachers” are GTAs who feel confident that they are good

teachers, have a preference for teaching, and feel that they could go into teaching as a profession.

“The Preferred Researchers” are GTAs are those who came to graduate school mainly so they

could do research, and prefer research over teaching.” The Anxious GTAs” are GTAs who do

not show a preference for teaching or research, but instead expressed anxiety about both; they

lack confidence in teaching and research, unlike the other two views. The variety of viewpoints

was further elucidated by post-sort interview questions. Therefore, we reject the null research

hypothesis.

132


The second research hypothesis stated that “Experienced biology graduate GTAs will

have different views of graduate school than new biology graduate GTAs.” Of the three factors

revealed through analysis of the Q Sorts, Factor 1 had nine experienced GTAs and nine new

GTAs, Factor 2 had five experienced GTAs and five new GTAs, and Factor 3 had one

Table 22 – Number of Q-Sorts Included in Each Factor

Factor

1 2 3

Q-Sorts

New GTA 9 5 5

Experienced GTA 9 5 1

Biology Lab Coordinator 1 0 1 1


Biology Lead Faculty Member 0 1 0

experienced GTAs and five new GTAs (See Table 22). Each of the three viewpoints contained

both new and experienced GTAs. Therefore, being an experienced GTA or a new GTA was not

necessarily a predictor of holding a certain viewpoint. The null research hypothesis (Experienced

biology graduate TAs will have the same views of graduate school as new biology graduate

GTAs) cannot be rejected, because experienced and new GTAs do populate similar factors.

Some new GTAs share similar views with experienced GTAs, but some do not. Thus, status

(new or experienced) does not determine GTA views. Therefore, the researcher fails to reject the

null hypothesis.

133


Specific research hypothesis 3A.

The first specific research hypothesis states that “The Q Sorts will reveal differences

between the viewpoints of the GTAs and the Biology Lab Coordinator.” There were two Biology

Lab Coordinators who sorted, the first Biology Lab Coordinator theoretically sorting once as a

new GTA, and the second time as an experienced GTA. These sorts revealed that first The

Biology Lab Coordinator theoretically sorted similar to a new GTA loading on Factor 3 (The

Anxious GTA), and similar to an experienced GTA loading on Factor 2 (The Preferred

Researcher). The second Biology Lab Coordinator theoretically sorted as an experienced GTA,

and were represented by Factor 1. The null hypothesis, “The Q Sorts will reveal no differences

between the viewpoints of the GTAs and the Biology Lab Coordinator,” is rejected. The Biology

Lab Coordinator was represented by Factor 3 when she theoretically sorted as a new GTA, and

was represented by Factor 2 when she theoretically sorted as an experienced GTA. The second

Biology Lab Coordinator was represented by Factor 1 when she sorted as an experienced GTA.

The Biology Lab Coordinators’ views aligned with Factors that were populated with both new

and experienced GTAs, but did not end up on views that were not populated by any GTAs.

Therefore, we fail to reject the null hypothesis.

Specific research hypothesis 3B.

The second part of the third research hypothesis, “The Q Sorts will reveal differences

between the viewpoints of the GTAs and the Lead Biology Faculty Member” revealed that the

Lead Biology Faculty Member theoretically sorted similar to a new GTA did not load

significantly on any factor (his loading was mixed, and not significant on any one factor), and the

Lead Biology Faculty Member theoretically sorted as an experienced GTA, loading on Factor 2

134

(The Preferred Researcher). The null hypothesis, “The Q Sorts will not reveal differences

between the viewpoints of the GTAs and the Lead Biology Faculty Member” is failed to be

rejected. The Lead Biology Faculty Member demonstrated he believed the new and experienced

GTAs would have different viewpoints, as evidenced by his sorts. His sorts were also different

than both the new and experienced GTAs who were represented by Factors 1 and 3.


The fourth hypothesis is that “Consensus statements concerning GTA viewpoints will

emerge during factor analysis.” The analyses revealed 13 consensus statements. The null

hypothesis states that “Consensus statements concerning GTA viewpoints will not emerge during

factor analysis” is rejected. When a statement is not distinguishing between any of the factors it

becomes a consensus statement (van Exel, 2005). Consensus statements are those statements that

are common between any pair of factors. Therefore, the researcher rejected the null hypothesis.

Summary

The participants in this study included 17 new GTAs participating in an “Effective

Teaching” course in the Fall of 2012, the two supervisors of the “Effective Teaching” course, 14

experienced GTAs, and an additional Biology Lab Coordinator. There were a total of 36 sorts

collected. The Q Sort process revealed three factors or views on graduate school. The factors

were named “The Emerging Teacher,” “The Preferred Researcher,” and “The Anxious GTA.”

The names for the factors were determined using the eight most-like (+5 and +4) and least-like

my view (-5 and -4) statements, distinguishing statements, consensus statements, and answers to

post-sort questionnaire.

Factor 1 GTAs, or “The Emerging Teacher,” are those who feel confident that they are

good teachers, and that they could go into teaching as a profession. Factor 2 GTAs, or “The

135

Preferred Researcher,” are those who came to graduate school mainly so they could do research,

and prefer research over teaching. Factor 3 GTAs, or “The Anxious GTA,” do not show a

preference for teaching or research, but instead look to juggle the two, along with being a

student. This study uncovered 13 consensus statements, which allow the commonality of the

GTAs’ sorting to be expressed.

136

CHAPTER V

SUMMARY, CONCLUSIONS, AND IMPLICATIONS

The purpose of this chapter is threefold; first, a summary of the study, second,

conclusions, and third, implications and further research. The first major section, the summary of

the study, includes a brief restatement of the problem, a brief review of the procedures employed

in conducting the research, and the specific research hypotheses tested. The second section,

conclusions, is drawn from Chapter IV analyses. These include highlights of the major findings.

The final section discusses the implications of the findings. The emphasis is on interpretation of

the significance of the research findings and what they imply. The suggested further research

section includes possible ways of extending the current study, expanding the study to include

different participants, or additional variables that could be added to the current study to glean

additional insight.

Summary of the Study

Graduate Teaching Assistants (GTAs) are frequently utilized as instructors in

undergraduate classrooms and science laboratories (Kendall & Schussler, 2012; Luft et al., 2004;

Nyquist & et al., 1991). GTAs provide universities a cost-effective form of instructor, while the

GTAs themselves are being simultaneously socialized into the roles of teacher, researcher, and

scholar (Carroll, 1980; Garland, 1983). GTAs represent a diverse population of masters and

doctoral-level students with varying amounts of pedagogical preparation, research abilities, and

motivation to complete their graduate study (Boyle & Boice, 1998). GTAs who are not

adequately prepared to engage in teaching activities may display a wide range of behaviors, from

an overblown confidence in their abilities (Golde & Dore, 2001), to frustration and insecurity

(Eison & Vanderford, 1993).

137

Instructional training programs for professionally developing GTAs vary from institution

to institution, and even between departments at the same institution (Nyquist & Woodford, 2000;

Parrett, 1987; Stockdale & Wochok, 1974). Calls for instructional training programs for teaching

assistants in the sciences (Carroll, 1980; Luft et al., 2004), and more specifically in biology

(Rushin et al., 1997; Tanner & Allen, 2006) have created a continual demand for pedagogical

training, in addition to content area mastery. Locally developed GTA instructional training

programs are much more common in graduate schools or disciplinary departments than large-

scale, externally-funded, national programs. These local programs are led by graduate school or

disciplinary faculty, or GTA supervisors, and vary in programmatic elements and effectiveness

(Carroll, 1980; Parrett, 1987; Thornburg et al., 2000).

The lack of uniformity among instructional training programs may be another reason for

varying preparedness among GTAs (Mountain & Pleck, 2000). Programs might range from half

a day before the semester begins, to a week-long campus-wide orientation, to a department-

specific semester long course in teaching methods, to a university-wide graduate school

certification (Golde & Dore, 2001). There has been little agreement on “the best way to train

GTAs,” although the Council of Graduate Schools with the “Preparing Future Faculty” initiative

and the Association of American Colleges and Universities have provided some guidelines that

can be referenced (DeNeef, 2002; Gaff, 2002a). There has also been no consensus on who should

be doing the training, and for what purpose (Shannon et al., 1998). The amount and type of

professional development made available to GTAs remains highly variable among higher

education institutions.

Whether the GTA instructional training program is implemented by the state, the

university, or the individual disciplinary department, program evaluation is complex and

138

necessary (Garet et al., 2001; Guskey, 1994). Such evaluation should be an intrinsic part of any

program or project because it is used both to measure the effectiveness of that program or project

as well as to investigate ways to increase that effectiveness (Newman & Ramlo, 2011). The

literature surrounding GTA training programs describes GTAs as having varying programmatic

needs based on numerous factors – prior formal or informal teaching experience, familiarity with

content, exposure to prior instructional training, demographic variables, career aspirations,

international status, etc. GTA programs often group cohorts of GTAs together for training

(Muzaka, 2009) regardless of experience, assignment, career focus, or degree plan. However

GTAs are grouped, the first steps in effectively evaluating any professional development

program is assessing participants’ needs. This study used Q Methodology to assess the

professional development needs of GTAs in one Department of Biology at a large, public, urban

university in the Midwest.

Q Methodology provides a foundation for the systematic study of subjectivity, a person’s

viewpoint, opinion, beliefs, attitude, etc. (Brown, 1993; Van Exel & de Graaf, 2005). By

Q Sorting, people assign their subjective meaning to the statements and reveal their subjective

viewpoints (Smith, 2001) or personal profiles (Brouwer, 1999). Within this study,

Q Methodology was used to provide a needs assessment for an instructional program such that

the study allowed the researcher to identify and interpret the various viewpoints that GTAs hold

in regard to graduate school.

The factors that emerged as a result of analyzing the Q Sorts were named using

information provided by GTAs. In post-sort interview questions, GTAs answered questions

about their graduate school program and degree track. They described both formal and informal

teaching experiences. Their Q Sorts provided information that distinguished the most-like and

139

most-unlike my view statements. GTAs also described their decision making process about the

sorting procedure, and made clarifications about why they placed the statements into the grid.

Three distinct GTA viewpoints about graduate school emerged from the analysis of the data:

“The Emerging Teacher,” “The Preferred Researcher,” and “The Anxious GTA.” These three

factors provided the researcher with GTA typologies that may be more useful in designing

meaningful GTA professional development for this group of GTAs and differentiating the

training program than simple demographics or answers to survey questions. Distinguishing and

consensus statements provide the supervisors of the “Effective Teaching” course areas of

agreement and disagreement between GTAs, which allows for reinforcement, enhancement, or

supplementation of skills possessed by these GTAs.


The purpose of this study is to demonstrate that Q Methodology can be used as a needs

assessment tool for a Biology graduate teaching assistant (GTA) instructional training program.

Despite the wealth of literature concerning elements of instructional training programs for GTAs

at the national, institutional, or departmental level, there is little consistency among training

programs. Literature about faculty perceptions of GTAs suggests that faculty perceive GTAs as

preferring research over teaching (Austin, 2002; Boyle & Boice, 1998; Kurdziel, Turner, Luft, &

Roehrig, 2003; Levinson-Rose & Menges, 1981; Trice, 2003), which was supported by GTAs

loading on Factor 2 (“The Preferred Researcher”), but not the other factors. Literature about

GTA professional development also suggests that GTAs are timid and self-conscious when

teaching (Gibbs & Coffey, 2004; Park, 2002, 2004; Prieto & Altmaier, 1994; Salinas, Kozuh, &

Seraphine, 1999), which was supported by GTAs loading on Factor 3 (“The Anxious GTA”), but

not the other factors.

140

Because typically a program does not have the same level of effectiveness for the entire

population it serves (McNeil et al., 2005), a needs assessment to identify important and

significant issues that can be addressed during professional development with the unique cohort

of GTAs should be, but often is not, completed. Q Methodology can be useful as a needs

assessment tool, providing predictor typologies that are more useful than simple variables and

demographic information for the classification of people, especially within program evaluation

(Newman & Ramlo, 2011).

If a program is to be useful to its stakeholders, in this case the Biology GTAs and The

Department of Biology, it is important to keep GTA views and program preferences in mind. To

assist graduate students to become as proficient in both their teaching and their research, they

must be given opportunities to develop their teaching skills, abilities, and knowledge with the

same guidance and practice that is afforded to the development of a quality researcher (Golde &

Dore, 2001).Because stakeholder needs vary at different stages in the program (Chen, 2005),

identifying GTA needs at the various stages in their graduate school program allows for program

supervisors to identify and modify program elements relative to GTA needs. The resulting

factors/views that emerged in this study have implications for improving this training program

by improving GTA instruction to undergraduates and improving GTA success in graduate school

and their future careers.

Statement of the Procedures

The researcher used Q Methodology to investigate new and experienced biology GTAs’

views of their GTA experiences. Multiple survey instruments were used to gather initial

information about the participants and their views on their biology graduate school program,

which were used to populate the concourse. The concourse, discussed extensively in Chapter III,

141

for this study included a collection of statements made by GTAs in a Self-Reflection

Questionnaire, a “Perceptions of Graduate School Survey,” a graduate student discussion forum

(“Grad School Life,” 2012), and everyday conversations and emails made between Biology

GTAs and their supervisors.

The Q Sample (see Appendix 2) was derived from the concourse by selecting nine

representative statements from each of six categories of interest to the researcher – teaching,

learning, students, research, challenges in graduate school, and GTA persistence in their program

– for a total of 54 statements in the Q Sample. This represented a Fisherian design (Brown &

Ungs, 1970).The statements were sorted by the participants into a three piles, based upon the

conditions of instruction, “Read each statement, and then based on your views of graduate

school, place the statements into three equal piles; most unlike your view, neutral, and most like

your view.” Then, the sorter placed each statement onto the sorting grid. Rather than simply

indicating agreement or disagreement with statements, as in Likert-style surveys, participants in

this study completed the “GTA Perceptions of Graduate School Q Sort,” where they were asked

to sort statements in relation to other statements in the Q Sample. After the participants

completed their Q Sorts, the researcher factor analyzed the sorts. The descriptive tables that

result from the factor analysis, along with post-sort interview questions, led to an understanding

of the various viewpoints held by GTAs and their supervisors, in a GTA instructional training

program.

The Q Methodology instrument was pilot tested during the “Effective Teaching” course

with new GTAs and their supervisors, in The Department of Biology during the Fall semester of

2012. The pilot study demonstrated the viability of the research design and instrument and led to

three GTA viewpoints (“The Confident Teachers,” “The Preferred Researchers,” and “GTA to

142

Professor”) that provided greater insight about new GTAs, the “Effective Teaching” course, and

programmatic needs. The research study expanded the pilot with experienced GTAs and another

Biology Lab Coordinator, sorting under the same conditions of instruction.

After the 36 Q Sorts were collected, the sorts were entered into PQMethod, where they

could be factor analyzed and used to create detailed tables describing the different views and the

consensus. The factor analysis uncovered three factors (“The Emerging Teacher,” “The Preferred

Researcher,” and “The Anxious GTA”), a list of distinguishing statements, and a list of

consensus statements. These factors represent three distinct viewpoints that exist amongst GTAs

and their supervisors about being a GTA in The Department of Biology, at this institution, at the

time of the study. The results of this can be used to consider potential changes or updates to the

existing training program in The Department of Biology.

The Research Hypotheses


The first research hypothesis stated that, “A variety of viewpoints about graduate school

will exist among biology GTAs.” Three GTA factors, or viewpoints about graduate school,

emerged as a result of analyzing the Q Sorts. These viewpoints were, “The Emerging Teacher,”

“The Preferred Researcher,” and “The Anxious GTA.” Therefore, we reject the null research

hypothesis.


The second research hypothesis stated that “Experienced biology graduate GTAs will

have different views of graduate school than new biology graduate GTAs.” Of the three factors

revealed through analysis of the Q Sorts, Factor 1 had nine experienced GTAs and nine new

GTAs, Factor 2 had five experienced GTAs and five new GTAs, and Factor 3 had one

143

Table 23 – Breakdown of Number of Q-Sorts Included in Each Factor

Factor

1 2 3

Q-Sorts

New GTA 9 5 5

Experienced GTA 9 5 1



Biology Lead Faculty Member 0 1 0

experienced GTAs and five new GTAs (See Table 23). Each of the three viewpoints contained

both new and experienced GTAs. Therefore, being an experienced GTA or a new GTA was not

necessarily a predictor of holding a certain viewpoint. The null research hypothesis (Experienced

biology graduate TAs will have the same views of graduate school as new biology graduate

GTAs) cannot be rejected, because experienced and new GTAs do populate similar factors.

Some new GTAs share similar views with experienced GTAs, but some do not. Thus, status

(new or experienced) does not determine GTA views. Therefore, the researcher fails to reject the

null hypothesis.


Specific research hypothesis 3A.

The first part of the third research hypothesis states that “The Q Sorts will reveal

differences between the viewpoints of the GTAs and the Biology Lab Coordinator.” The null

hypothesis, “The Q Sorts will reveal no differences between the viewpoints of the GTAs and the

Biology Lab Coordinator,” is rejected. The Biology Lab Coordinators’ views aligned with

144

Factors that were populated with both new and experienced GTAs, but did not end up on views

that were not populated by any GTAs. Therefore, the researcher rejected the null hypothesis.

Specific research hypothesis 3B.

The second part of the third research hypothesis states that “The Q Sorts will reveal

differences between the viewpoints of the GTAs and the Lead Biology Faculty Member.” The

null hypothesis, “The Q Sorts will not reveal differences between the viewpoints of the GTAs

and the Lead Biology Faculty Member” is rejected. The Lead Biology Faculty Member

demonstrated he believed the new and experienced GTAs would have different viewpoints, as

evidenced by his sorts. His sorts were also different than both the new and experienced GTAs

who were represented by Factors 1 and 3. Therefore, the researcher rejected the null hypothesis.


The fourth hypothesis is that “Consensus statements concerning GTA viewpoints will

emerge during factor analysis.” The analyses revealed 13 consensus statements. The null

hypothesis states “Consensus statements concerning GTA viewpoints will not emerge during

factor analysis.” Therefore, the researcher rejected the null hypothesis.

Conclusions

This part of the chapter discusses the conclusions drawn from the results of this study.

First, statements related to the general research questions, followed by the specific research

questions, and concluded with a general discussion of the research questions.


What are the various viewpoints that exist among Biology GTAs about their graduate school experiences?

145

In this research study, three viewpoints, or factors, emerged among Biology GTAs about

their graduate school experience. The viewpoints were named by the researcher; Factor 1, or

“The Emerging Teacher,” Factor 2, or “The Preferred Researcher,” and Factor 3, or “The

Anxious GTA.”

Factor 1, or “The Emerging Teachers” view, represents the GTAs who feel confident that

they are good teachers, that their time as a GTA was preparing them to be a good professor, and

that they could go into teaching as a profession. The majority of GTAs who were represented by

this factor specifically listed professor or teaching as their desired career path. They indicated a

preference for teaching over research. They did not indicate anxiety about teaching, that they

were experiencing family problems because of grad school, or that they felt that they were an

outsider in grad school. Of the 36 Q Sorts, 18 were represented by Factor 1 including the second

Biology Lab Coordinator.

Factor 2, or “The Preferred Researchers” view, represents the GTAs who came to

graduate school mainly so they could do research, and prefer research over teaching. Almost all

of the GTAs who were represented by Factor 2 described their career aspirations as academia or

research. They felt like they were good teachers, and that they had some natural teaching

abilities, but that they just preferred research. They did not display anxiety about teaching, or

lack confidence in their abilities, and they were highly confident that they knew their content

matter. The majority of GTAs who were represented by Factor 2 were doctoral students, and this

factor also included the Lead Biology Faculty Member and the first Biology Lab Coordinator

theoretically sorting as a new GTA. Of the 36 Q Sorts, 10 were represented by Factor 2.

Factor 3, or “The Anxious GTA” view, represents the GTAs who do not show a

preference for teaching or research, but instead look to juggle the two, along with being a

146

student. They asserted a need to be good at both teaching and research activities, while

expressing that they may not be good at either. These GTAs expressed insecurity in their

abilities, being uncomfortable in their position, and feeling anxious. They also stated that they

were frustrated because they didn’t know the department’s expectations of them. GTAs who

were represented by this factor were mostly master’s degree students, and the first Biology Lab

Coordinator theoretically sorting as a new GTA. Of the 36 Q Sorts, six were represented by

Factor 3.

What are the various viewpoints of the supervisors of graduate GTAs in The Department of

Biology relative to those of the GTAs?

One supervisor of GTAs, the first Biology Lab Coordinator, theoretically sorted once as a

new GTA, and once as an experienced GTA. For the new GTAs, she was represented by Factor 3

(“The Anxious GTA”), and for the experienced GTAs, she was represented by Factor 2 (“The

Preferred Researcher”). The two different factor loadings support the Biology Lab Coordinator’s

perception that new and experienced GTAs would have different viewpoints. In theoretically

sorting as a new GTA, one of her most-like my view statements was statement 22 (I have a lot of

anxiety about teaching, because I don't know what to expect) in the +5 position. This was one of

the distinguishing statements for Factor 3, and correlates closely to GTAs who were represented

by Factor 3 (“The Anxious GTAs”). The literature describes how GTAs may perceive teaching

as a highly demanding career having a heavy workload, high emotional demand (Hendrix, 1995),

anxiety-provoking, and generally requiring hard work (Deiro, 1996; Rhodes, 1997). At the same

time, GTAs may also perceive teaching as relatively low in social status, paying a low salary,

and reported experiences of quite strong social dissuasion from a teaching career (Rhodes, 1997;

147

Watt & Richardson, 2008). The Biology Lab Coordinator’s theoretical sort as a new GTA

supported these views.

When theoretically sorting as an experienced GTAs, The Biology Lab Coordinator was

represented by Factor 2 (“The Preferred Researcher”). Her sort suggests that she believes that

experienced GTAs will have more confidence in themselves, and their teaching, after gaining

experience, which was supported by the GTAs sorting statements 20 (I feel pretty confident that

I'm a good teacher) and 21 (I feel self-confident when I teach) in the +4 column as “most like”

their view. These GTAs also sorted statements 7 (I came to grad school mainly so I could do

research) and 34 (I like doing research over teaching) in the +5 column, as “most like” their

views. The first Biology Lab Coordinator sorted into the least-like my view columns statement

25 (I have no idea what students think about me, and that makes me uncomfortable), statement

22 (I have a lot of anxiety about teaching, because I don't know what to expect), and statement

46 (I worry that certain students in my class might know more about Biology than I do), which

corresponds to the rankings by Factor 2 GTAs. The Biology Lab Coordinator’s view may have

been represented by Factor 2, or “The Preferred Researcher,” but she has not been trained as a

Biology researcher. She may or may not possess the skills to prepare the GTAs who are

represented by this factor, which were 10 of the 36 sorts. This is one of the reasons that having

both the Lead Biology Faculty Member and The Biology Lab Coordinator, with their differing

backgrounds, lead the “Effective Teaching” course is of benefit to the GTAs.

The second Biology Lab Coordinator theoretically sorted as an experienced GTA, and

was represented by Factor 1 (“The Emerging Teachers”). Three of her “most like” my view

statements, statement 12 (I feel like I could go into teaching as a profession), statement 20 (I feel

pretty confident that I'm a good teacher), and statement 21 (I feel self-confident when I teach),

148

were also in the top eight statements of the GTAs loading on Factor 1. She noted in her post-sort

interview questions, “I feel that teaching is as important as doing research and that professionals

at the university should invest in the two equally.” She also stated that “the statements that were

most-like her view were the easiest to identify.” This Biology Lab Coordinator, who teaches a

majors Biology lab, and wants to continue teaching, loaded highly on Factor 1.

The Biology Lead Faculty Member theoretically sorted once as a new GTA, and once as

an experienced GTA. For the new GTAs, the Biology Lead Faculty Member displayed a mixture

of all three factors, not loading significantly on any one factor. He loaded -0.4248 for Factor 1,

0.2074 for Factor 2, and 0.4266 for Factor 3. The negative loading for Factor 1 indicates that he

represented an opposing viewpoint to Factor 1, or “The Emerging Teachers.” He ranked

statement 9 (I dislike teaching, and wish I could spend more time on my research) as a +4, or

most-like his view, whereas GTAs loading on Factor 1 ranked this statement as their -5, or least-

like their view. He also ranked statement 7 (I came to grad school mainly so I could do research)

in the +5 column, where the Factor 1 GTAs ranked it in their -2, least like their view column. He

rated statement 34 (I like doing research over teaching) in his +4 column, while the GTAs ranked

it in their -3 column. He ranked statement 36 (I think all this teaching gets in the way of my

research) in the +5 column, while the Factor 1 GTAs ranked it in the -5 column. He ranked

statement 20 (I feel pretty confident that I'm a good teacher) in the -5 column, while the Factor 1

GTAs ranked it in their +5 column. He ranked statement 12 (I feel like I could go into teaching

as a profession) in the -4 column, while the Factor 1 GTAs ranked it in the +5 column. The

Biology Lead Faculty Member demonstrates that his viewpoint is the opposite of the GTAs who

were represented by Factor 1, which were 18 out of the 36 Q Sorts. This reinforces the need for a

needs assessment in the instructional training program, because the supervisor of the program

149

has an opposite view of the program than half the participants. Many of the national training

programs, such as The Preparing Future Faculty program, are designed entirely by faculty, using

faculty perceptions of what GTAs should know (Anderson et al., 1997). While their institutions

may articulate messages about the importance of the teaching mission, their advisors, particularly

in STEM fields, may urge them to avoid spending too much time on anything besides research-

related activities (Austin et al., 2009).

For the experienced GTAs, the Biology Lead Faculty Member was represented by Factor

2 (“The Preferred Researcher”) He ranked statement 7 (I came to grad school mainly so I could

do research) in the +5 column, as did the GTAs who were represented by Factor 2. He also

ranked statement 19 (I feel pretty comfortable using technology in my class) in the +5 column, as

well as statement 32 (I learned best by actively doing labs) which correlated with the Factor 2

GTAs. He ranked statement 25 (I have no idea what students think about me, and that makes me

uncomfortable), statement 11 (I feel like an outsider, and that people at grad school won't accept

me), statement 22 (I have a lot of anxiety about teaching, because I don't know what to expect),

and statement 17 (I feel like students look at me weird when I tell them I'm a TA) as least-like

his view, which correlated with Factor 2 GTAs. The Biology Lead Faculty Member theoretically

sorted as a “Preferred Researcher,” which correlates to his chosen profession. He may be best

suited to understanding the viewpoints of GTAs who were represented by Factor 2.

What consensus exists among the GTAs in The Department of Biology about their graduate

school experiences?

Q Methodology is a powerful tool for determining consensus and perspectives of a group

(Ramlo, 2011). A statement that is not distinguishing between any of the identified factors is

150

called a consensus statement (Van Exel & de Graaf, 2005). The consensus statements can

highlight the similarities between factors (See Table 21). It may be just as enlightening to

discover the statements that people have agreement on as where their views diverge. Participants

may agree positively, negatively or be neutral about the issue (Coogan & Herrington, 2011).

These would be statements that ranked similarly between all the typologies. This study

uncovered 13 consensus statements, which allow the commonality of the GTAs’ sorting to be

expressed.

There were three statements that the GTAs felt strongly were most-like their viewpoints.

These include statement 19 (I feel pretty comfortable using technology in my class), statement

38 (I think one of the most important things about being a GTA is being ethical), and statement

40 (I think some people are natural teachers). These statements indicate areas where additional

training may not be necessary, like using technology in the classroom, or where GTAs already

feel strongly, like conducting oneself in an ethical manner. Thinking that some people are natural

teachers may require further exploration, as this statement could be interpreted several ways.

GTAs may feel like some people are natural teachers, and they are a natural teacher, or they may

feel that some people are natural teachers, so they will never be a natural at teaching.

There were four statements that the GTAs felt strongly were most-unlike their

viewpoints. These include statement 17 (I feel like students look at me weird when I tell them

I'm a GTA), statement 49 (I've had family problems because of the pressures of graduate school),

statement 27 (I have to repeat myself over and over to get these students to understand me), and

statement 54 (Using social media (like Twitter or Facebook) helps me to feel like I'm not alone).

These indicate that GTAs are comfortable with their “label” as teaching assistants, and that they

are not feeling some of the pressures related to their families or feelings of loneliness that are

151

often indicated in the literature. These indicate areas where GTAs may not need added

professional development.

Finally, there were six statements that GTAs placed in the neutral categories. These

include statement 14, (I feel like I'm a good teacher because I am closer in age to my students),

statement 23 (I have had dreams about my teaching or research), statement 28 (I know the

university policies that relate to my research), statement 33 (I learned best by listening to

professors teach), statement 37 (I think most of my students learn in a way that's similar to the

way I learn), and statement 47 (If I teach well, I will get good student evaluations). Neutral

columns should be evaluated as items the GTAs do not have strong view are like or unlike their

views, in relation to other statements in the Q Sample. All the study factors have ranked the

items in pretty much the same way, Consensus statements might be used to highlight a possible

need for improvement in a program, or further training in a particular area (Watts & Stenner,

2005). In this study, the consensus statements did not provide as much support for scaffolding

the instructional training program as the top-eight and bottom eight statements, as well as the

distinguishing statements.

How do the views differ between new GTAs versus experienced GTAs?

There were GTAs from both the new and experienced groups who were represented by

each factor. There were nine new GTAs and nine experienced GTAs who were represented by

Factor 1 (“The Emerging Teachers”). There were five new GTAS and five experienced GTAs

who were represented by Factor 2 (“The Preferred Researcher”). There were five new GTAs and

one experienced GTA who was represented by Factor 3 (“The Anxious GTA”). For the first two

factors, there was an even amount of new and experienced GTAs populating each factor. For the

152

last factor, Factor 3, there was a five-to-one ratio of new to experienced GTAs who were

represented by the factor. This distribution of new and experienced GTAs across all three factors

indicates that none of these distinct views is the result of the “status” of a GTA as new or

experienced. Instead, these views appear to have other origins and are not directly associated

with being a new or experienced GTA.

Just as students in a classroom interact in different ways with the curriculum, bringing

prior experiences, ways of thinking, and motivation to the class, GTAs have different

experiences with graduate school. Rather than focusing on the narrative of specific individuals

(i.e. Determining whether an individual is a new or experienced GTA, or identifying whether an

individual has formal or informal teaching experience), Q methodology typically focuses on the

range of viewpoints that are favored (or which are otherwise ‘shared’) by specific groups of

participants (Watts & Stenner, 2005). In other words, the typical Q methodological study very

deliberately pursues constructions and representations of a social kind (Moscovici, 1988). “New”

or “Experienced” are labels that are placed on individuals. Typologies demonstrated shared

views between groups. These results are discussed further within the response to the next

research question.

Do the varying views and consensus of GTAs about their graduate school experiences provide

sufficient information for a needs assessment that informs the existing training program?

One of the first steps in effective program evaluation is assessing the needs of the

particular set of participants in that program (Chen, 2005; McNeil et al., 2005). A needs

assessment is a “systematic set of procedures for the purpose of setting priorities and making

decisions about a program or organizational improvement and allocation of resources. The

153

priorities are based on identified needs (Witkin, 1995).” The literature revealed that GTA needs

in a program are often collected using modified teacher inventories (Angelo & Cross, 1993;

Gibson & Dembo, 1984; Kohn et al., 1990; Prieto & Altmaier, 1994; Renzulli & Smith, 1978),

Likert-style surveys (Cho et al., 2010; Gorsuch, 2003), using simple demographic variables – or

are not assessed at all (Shannon et al., 1998; Worthen, 1992).The data analysis of the

“Perceptions of Graduate School Q Sort” provided three distinct viewpoints about graduate

school. Because of the rich qualitative data provided by Q Methodology, there is sufficient

information about the cohort of GTAs who participated in the study to use the “Perceptions of

Graduate School Q Sort” as a needs assessment to inform the existing training program.

Q Methodology offers a number of potential advantages for assessing needs of GTAs

throughout their graduate school careers (Peritore, 1989; Prasad, 2001), which were

demonstrated by this study. Q Methodology does not demand the large number of participants

that a Likert-style survey requires (Cummins & Gullone, 2000). This study involved 36 Q Sorts,

in one department, at one university. Because the literature about GTAs frequently refers to

GTAs in different disciplines or different types of schools, the needs of GTAs in other

disciplines are not necessarily the needs of this specific group of Biology GTAs. Q Methodology

allows the researcher to determine the various perspectives and consensus within the group

(Ramlo, 2008). This study uncovered three viewpoints within the group of study.

The only specific needs assessment for GTAs in the literature was provided by Sohoni et.

al. (2013) and was for engineering GTAs. GTAs, faculty, and students rated the importance of

each of 24 GTA roles and responsibilities on a 5-point Likert scale, and the perceived

competence of GTAs on these 24 items. This 5-point Likert scale was used with 1 representing

“Not at all important‟ and 5 representing “Critically important‟ on the roles and responsibilities

154

questionnaire. A similar scale was used for competence, with 1 representing “Lack of

competence‟ and 5 representing “Very competent.‟ The problem with this type of survey is that

participants could mark every item “Critically Important.” This type of instrument may not

provide useful or adequate understandings of the various viewpoints that exist among GTAs

about their needs in an instructional training program. Prasad (2001) argues that use of the forced

choice method (forced matrix) in Q Methodology means that the respondents have to consider

their attitudes more carefully, which can bring out true feelings in response. This study led to

three factors that provided the researcher with GTA typologies that may be more useful in

designing meaningful GTA professional development and training programs than simple

demographics or answers to survey questions. Classification of GTAs based on typologies, or

predictor profiles, may be more useful for program evaluation, because typically a program does

not have the same level of effectiveness for the entire population it serves (McNeil et al., 2005).

Program evaluation ought to be an intrinsic part of any program or project because it is

used to both measure the effectiveness of that program or project, as well as investigate ways to

increase that effectiveness (Newman & Ramlo, 2011). This study uncovered GTA programmatic

needs that were similar to, and differed from, those described in the literature. Carroll (1980, p.

179) notes in his review of the research surrounding GTA training programs, that “programs

should be structured to encourage the participation of experienced, senior GTAs who can share

their insights and experiences with the novice GTAs.” This study found that experienced GTAs

populated all three different factors/viewpoints. The viewpoint of one experienced GTA may not

necessarily be the same as another experienced GTA who was represented by a different factor.

Therefore, it may be a limitation of the “Effective Teaching” course to be limited to new GTAs,

or to limit peer mentoring to pairing one new GTA and one experienced GTA. This study has

155

demonstrated that differing viewpoints exist between the participants in the study, not just

because of simple demographic traits or experience, but because they have different perceptions

of graduate school. This is one of the advantages of using Q Methodology as a needs assessment

tool. The perspectives and consensus of the GTAs and their supervisors can be uncovered.

Since the benefits to the department or institution could vary from one cohort of GTAs to

another, it is important that program evaluation be conducted regularly (Carroll, 1980). GTAs

enter school with varying degrees of experience, prior teaching, experiences with students,

approaches to diversity, and motivation to persist in their programs. There were 18 sorts that

were represented by Factor 1 ("The Emerging Teacher"). The participants in the group included

ten females and eight males. There was an even split with nine experienced and nine new GTAs.

Seventeen out of the 18 sorts included participants with teaching experience, 12 with formal

experience, four with informal experience, and two who provided no answer. Ten sorts were

represented by Factor 2 (“The Preferred Researcher”). The participants in the group included

three females, and seven males. There was an even split with five experienced and five new

GTAs. Every participant had taught before, but one only had informal teaching experience. Six

sorts were represented by Factor 3 (“The Anxious GTA”). The participants in this group

included four females and two males. All of these sorters were new GTAs except for one

doctoral student. Every GTA had taught before, but mostly (four sorts) in an informal setting,

with two teaching in a formal setting. The varying experiences of the cohort of GTAs completing

this Q Study may or may not be similar to the cohort of GTAs in The Department of Biology the

next year. Only by repeating the needs assessment can the supervisors of the program determine

the viewpoints of the next cohort.

156

In addition to uncovering three distinct GTA viewpoints, this study also unearthed 13

consensus statements which allow the common voices of GTAs to be expressed. Q Methodology

is a powerful tool for determining consensus and perspectives of a group (Ramlo, 2011). The

consensus statements can highlight the similarities between factors. These would be views

shared by all the GTAs. In this study, the consensus statements were mostly items that the GTAs

felt neutral about, such as dreaming about teaching or research, knowing university policies,

repeating themselves while teaching, their own learning styles, or student evaluations. The

consensus statement that GTAs felt very strongly was unlike their view (statement 49, I've had

family problems because of the pressures of graduate school) indicated that this cohort of GTAs

was not experiencing the emotional exhaustion that often occurs in the academic community

(Repak, 2012). Consensus statements can be used to identify common ground within the

population of study, but in this case, were not as useful as the most-like, most-unlike, and

distinguishing statements.

Finally, it is also important to do a needs assessment that includes the supervisors of the

GTAs, so that they can be aware of how their viewpoints differ from the viewpoints of the GTAs

in their course. Just as GTAs must be ready to work with diverse students who may be

completely unlike them as students, so must the supervisors of GTAs. This study demonstrated

that the supervisors of the GTA instructional training program shared some consensus with the

GTAs, but also displayed different viewpoints. The Biology Lead Faculty Member demonstrated

a viewpoint that was the opposite of GTAs loading on Factor 1. Q Methodology allows the

supervisors of the course to uncover their own viewpoints, or perceptions of both new and

experienced GTAs, which may be helpful in designing effective professional development for

the GTAs.

157

Implications

Differentiating the Instructional Training Program

Once the typologies of GTAs are known through conducting a needs assessment,

professional development should be designed around their needs through differentiation.

Differentiation means tailoring instruction to meet individual needs (Tomlinson, 2012).

Supervisors of GTA instructional training programs may differentiate content, process, products,

or the learning environment, and use ongoing assessment and flexible grouping to make this a

successful approach to instruction. Differentiating GTA instructional training does not have to

mean teaching three different instructional training courses for the three different types of GTAs.

Q Methodology can provide the supervisors of the instructional training program the chance to

adjust their curriculum and instruction to maximize learning for all GTAs, depending on their

needs and ability levels (The IRIS Center, n.d.).

The department should encourage GTAs to explore workshops, courses, or seminars

offered outside the department by the graduate school or the faculty professional development

department on the campus. There may even be specialized certifications that the GTAs can

acquire such a certification for online teaching or working with students with disabilities. Factor

1 GTAs specifically asked for further development of classroom management techniques and

working with diverse students. Factor 2 GTAs asked for advice on how to motivate students and

deliver better classroom lectures. Factor 3 GTAs asked for public speaking advice and how to

work with groups. Each of these GTA preferences are often addressed by the Institute for

Teaching and Learning on campus, but only faculty are sent notices of these workshops. Giving

GTAs options to expand on their instructional skills outside their instructional training course, in

158

meaningful and specific ways that the GTA chooses, is vital to differentiating their professional

development. This is why it is also important for GTAs to understand their own typologies, and

what they mean as far as a needs assessment.

Q Methodology as a Self-Diagnostic Tool

Sharing the results of the study (i.e. which factor the GTA was represented by, what the

factors uncover about differing viewpoints, what differentiating or consensus statements were

uncovered by the study) with the GTAs could provide a “self-diagnostic tool.” GTAs could adapt

their own professional development by adaptively scaffolding their own self-directed learning

(Ley, Kump, & Gerdenitsch, 2010). Self-directed learning is a self-initiated action that involves

goal setting and regulating one’s efforts to reach the goal, and can be seen as a continuous

engagement in acquiring, applying and creating knowledge and skills in the context of an

individual learner’s unique problems (Fischer & Scharff, 1998).

Upon completion of the Q Sort and the subsequent analysis, when a GTA learns their

typology, or predicator profile, they might be given a preparatory list of common needs for their

typology. GTAs could then be paired with a peer mentor, faculty mentor, or supervisor who

subscribes to a different typology, and can provide additive scaffolding for the GTA. In self-

directed learning, the starting point is the perception of a knowledge need of the learners arising

in their actions. Based on this, GTAs determine the goals of learning, initiate purposive

information seeking behavior by identifying and choosing possible sources, and interact with the

sources to obtain the desired information (Choo, 1996). When approaching graduate school as a

“problem-based learning” environment, GTAs taking control of many facets of their own

acquisition of knowledge, facilitated by their supervisors and under the parameters of their

typology, would be more relevant and purposeful to their personal development.

159

Typically, all the new GTAs enrolled in the “Effective Teaching” course would be

provided a syllabus containing topics to be discussed each week, such as “Preparing Your

Teaching Portfolio,” “Conducting a Mid-Term Evaluation,” or “What to do About Plagiarism.”

These are topics that the Lead Biology Faculty Member and the Biology Lab Coordinator have

determined to be essential to the instructional training program for the GTAs. Besides the

traditional fixed scaffolding where a fixed list of learning goals for the task (provided by the

disciplinary experts) is given to GTAs, GTAs could be introduced to an adaptive scaffolding

condition where they seek out advice on several self-development strategies (i.e. Ask Factor 1

GTAs to collaborate with their faculty advisor about policies that concern their research. Ask

Factor 2 GTAs to explore teaching strategies that could improve their teaching presentation. Ask

Factor 3 GTAs to research strategies that facilitate a respectful teaching environment.), and then

journal their findings. They could share these findings in class or with their “Effective Teaching”

instructors. Recommendations would then be provided to the GTA on both their personal

approach to meeting their learning goals, and adaptations the GTAs could make in their

behaviors that would help them meet their goals by addressing their own needs.

By personalizing the GTAs’ learning, layering both a common set of topics with an

adaptive, self-directed set of tasks or topics based on their needs assessment, this would make

their professional development more meaningful and personally relevant. Rather than a “one-

size-fits-all” instructional training course that is the same for all the new GTAs, their

professional development becomes highly personalized, built on their personal preferences,

based on what the typologies in the course are, and not what the instructors perceive them to be.

The course would be adapted to account for the GTAs’ prior experiences, and their preferences

for a GTA program. Along with meeting the needs of the new GTAs who were represented by

160

each factor, identified through the Q Sort factor analysis, the supervisors can also account for the

future needs that have been identified by experienced GTAs.

Collective Mentoring

By adding a formal “collective mentoring” aspect to the “Effective Teaching” course, the

experienced GTAs, faculty, and staff could provide the kind of collaboration and expertise on the

way GTAs develop over the span of teaching multiple semesters at the university, that new

GTAs asked for in their post-sort interview questions. Collective mentoring is an evolution of the

multiple mentor/single mentee model whereby senior colleagues and the department take

responsibility for constructing and maintaining a mentoring team (Chesler & Chesler, 2002).

Tierney and Bensimon (1996) have argued that “The notion of a single experienced faculty

member being willing and able to play the all-inclusive role of mentor to a protégé is wishful

thinking.” Asking experienced GTAs, faculty, and staff to return to the course, as a “panel of

experts,” provides the new GTAs with various points of view from people who have experienced

what these new GTAs are preparing to do. In truth, a variety of individuals are required to help

meet a mentee’s diverse needs (Chesler & Chesler, 2002). This is currently done in a very

informal fashion, mostly occurring outside the realm on the “Effective Teaching” course.

Increasing the presence of other members of The Department of Biology in the “Effective

Teaching” course, whose differing perspectives may provide collegiality to the new graduate

students, may help the GTAs to see that there are other perspectives than their immediate faculty

advisor, supervisor, or lab mates.

Tierney and Bensimon (1996) point out that collective mentoring is a formal and

collective organizational task, part of the organization’s responsibility to orient and socialize its

new members. As such, “mentoring need not take place only in a senior faculty member’s office

161

or an orientation session at the beginning of the school year. The mail room, the faculty lounge,

and any number of other institutional locations have potential for socializing individuals to the

culture of the department and organization.” Included in that list, for GTAs in The Department of

Biology, are places such as laboratories, lab meetings, course meetings for the classes they teach,

colloquium, and the “Effective Teaching” course. Ginorio (1995) argues that students need to

find a meaningful community in science and engineering, one that “would not include…outdated

ideas of what a successful culture of science is: competitive, all engrossing, demanding to the

exclusion of any other interest, and open only to the handful of individuals who can pass all the

tests.” Organizational change that creates more egalitarian and caring communities will benefit

all students. In addition, promoting collegiality and civility between not only faculty members,

but GTAs, where they can passionately share ideas, and then work together as a department, sets

a positive example for socializing future faculty.

Promises and Challenges of Q Methodology

In using Q Methodology as a needs assessment tool for GTAs in an instructional training

class, there are a number of promises that Q Methodology holds, as well as some challenges. As

the typologies of the GTAs emerged, the profiles, or viewpoints, lacked characteristics of “the

needs of GTAs” that were described in the literature. Some had a few of the characteristics. The

three typologies of GTAs were based upon the factor analysis of the Q Sorts, not through asking

GTAs simply “What type of GTA are you?” The operant categories are functional, not just

logical distinctions (Brown, 1991). Because the typologies were uncovered using factor analysis,

they may be more meaningful than a Likert-style survey, which leads to the loss of meaning, or a

case study, which only examines a few views and is time-consuming, for the professional

development of GTAs.

162

Biology GTAs are situated in laboratory settings, with a PI or faculty member heading

the lab (Golde & Dore, 2001). The GTAs join established labs, and then being a GTA provides a

fee remission for their tuition as well as a stipend for living expenses. The GTAs in this

Department of Biology may have different viewpoints about their graduate school program than

GTAs in other disciplines, such as other STEM disciplines, the social sciences, or the

humanities. The reasons these GTAs described for entering graduate school were complex, from

one Factor 1 GTAs describing that “I took the job of being a GTA because I knew I was already

a good teacher,” and one Factor 2 GTA saying “Although I don’t mind teaching, research is my

passion. I came to grad school for research, not to teach.” While the literature does briefly

mention going to graduate school for “teaching opportunities (Malaney, 1987),” the Biology

GTAs in this study who were represented by Factor 1, “The Emerging Teacher,’ described being

a GTA as more than just a “teaching opportunity.” Participant MNN24FF stated “I love TAing.”

MNY24FI stated in the post-sort discussion, “I knew I could do this. I knew I’d love teaching,

and I’m happy to have the opportunity to do it while learning and conducting my research, which

is more challenging to me. Teaching is like the bright spot in my week.” This group of GTAs did

not need coaxing to embrace teaching as part of their job in graduate school, and didn’t express

trepidation about this part of the job. This group did not require a lecture about “why teaching is

important.” They already came to the department with this perspective. Through using Q

Methodology as a needs assessment tool, the post-sort interviews uncovered views of which the

supervisors of the “Effective Teaching” course were unaware. Rather than spending time in the

course persuading GTAs to embrace teaching, time could be spent on the professional

development of skills that GTAs indicated in their “program preferences” post-sort question.

163

Q Methodology provides a more accurate needs assessment for an instructional training

program than a Likert-style survey or questionnaire. Although Q Methodology is similar to the

Likert-style survey scale in that the distribution on the grid typically ranges from least like my

view to most like my view (Ramlo, 2008), Q differs from Likert-style surveys in that Q involves

participants physically sorting items relative to each other into a normalized or Gaussian

distribution (Brown, 1993; Brown, 1980; McKeown & Thomas, 1988; Ramlo, 2008; Ramlo &

Nicholas, 2009). Likert (1967) assumed that every statement is equally important to the overall

attitude. Likert scales do not consider the weight that sorters attach to individual items (ten

Klooster, Visser, & de Jong, 2008) which can therefore result in the loss of meaning (McKeown,

2001; Ramlo & McConnell, 2008). As was demonstrated in Cho et al. (2010), GTAs could mark

every statement as “critically important,” which does not help supervisors of the course

determine what is actually ranked the highest, in relation to the other statements. Cho, Sohoni,

and French’s (2010) needs assessment for GTAs missed data that is critical to understanding

GTA needs. Surveys are common methods for collecting feedback; however, they allow

responders to give similar or identical ratings to many or all items (Dennis, 1986). Though Q

Methodology is gaining recognition in education research, it is not as wide-spread or commonly

used as surveys or questionnaires.

Qualitative methods may generate transcripts of discussions which reveal much about

attitudes as they are expressed in the normal social context of a discussion. However, these

methods are often criticized on the grounds that they lack statistical rigor (Addams, 2000). There

have been numerous case-studies of GTAs (Darling & Staton, 1989; Muzaka, 2009; Park, 2002,

2004), but the time it would take to do an in depth interview with each GTA could prove

onerous. Interviews and other purely qualitative techniques are time consuming (Ramlo, 2006).

164

Q Methodology provided better results for guiding this instructional training program because of

Q’s efficiency. The qualitative data that is collected, along with the statistical analysis, provided

a rich understanding of the types of viewpoints that exist among the GTAs, without the time it

takes to do in-depth interviews.

One of several benefits of using Q methodology in this study, opposed to Likert-style

surveys or case studies, is that it allowed the researcher to examine how GTA viewpoints

compared to how their supervisor perceived their views (both as new GTAs and experienced

GTAs). It is interesting here to note that the first Biology Lab Coordinator’s theoretical sort as a

new GTA was represented by “The Anxious GTA,” and her theoretical sort as an experienced

GTA was represented by “The Preferred Researcher.” The second Biology Lab Coordinator’s

theoretical sort as an experienced GTA was represented by “The Emerging Teacher.” The Lead

Biology Faculty Member’s theoretical sort as a new GTA displayed a mixture of all three factors,

not loading significantly on any one factor, and his theoretical sort as an experienced GTA was

represented by “The Preferred Researcher.” The supervisors did not have one single perception

of new or experienced GTAs, and in the case of The Biology Lead Faculty Member’s theoretical

sort of new GTAs, did not load on a viewpoint at all. This may indicate supervisor’s

misconceptions about GTA needs that may have been missed if the supervisors’ perspectives had

not been included in the needs assessment. This study might be hand rotated in the future to force

loading on a factor by The Lead Biology Faculty Member, relative to the viewpoints of the

GTAs. But in this study, the three supervisors were represented by three different factors. This

reinforces the need for collaboration by faculty and staff in the Department of Biology, The

Department of Education, and The Graduate School about professional development.

Collaboration, especially in the form of team teaching, is not easy, however. It takes time and

165

energy to work together in planning, teaching, and evaluating. Questions of teaching loads may

come into play. And, of course, the egos of many academics often make collaborative teaching

difficult, or even impossible (George & Davis-Wiley, 2000).

One of the reasons Q Methodology may help in the collaboration of faculty and staff in

an instructional training program is that Q allows the supervisors of such programs to scaffold

the professional development of the GTAs according to both supervisor and GTA typology.

Results of the study could be used to start a dialogue between faculty members, supervisors of

GTAs, and mentors about what types of supports each person can provide. Scaffolds are

temporary supports that help a learner bridge the gap between what he or she can do and what he

or she needs to do to succeed at a learning task (Graves & Braaten, 1996). To guarantee that each

GTA can internalize complex concepts, supervisors of GTAs should consistently provide

scaffolding, often inventing supports on the spot as a GTA asks for advice about a specific

situation. Supervisors of GTAs should be able to draw on a rich mental database of examples,

metaphors, and enrichment ideas. The typology of the supervisor may reveal clues as to the

strengths or weaknesses of their viewpoints in relation to the GTAs, which allows for more

meaningful professional development than one supervisor being in charge of pedagogy, and the

other being in charge of research, or however the duties are divided. Because the supervisors of

such programs have vast experiences in higher education, and have a repertoire of experiences,

they should be able to offer GTAs insights into how to successfully deal with students, teaching,

or research situations.

One of the challenges of using Q Methodology as a needs assessment tool is that it

requires a researcher that is able to administer, analyze, and interpret the study; this requires a

researcher knowledgeable about Q Methodology. Because the factors that emerge during a Q

166

Study are not generalizable to larger populations of GTAs, and instead produce an in-depth

portrait of the typologies of perspectives that prevail in a given situation (Steelman & Maguire,

1999), the Q Sort may need to be repeated with each cohort of GTAs. The researcher may find

that, after repeating the Q Sort over several semesters, the ratios of typologies are relatively

stable, meaning they occur in approximately the same proportions each semester. If the

typologies are stable, the researcher could then repeat the Q Sort every two to three years to

demonstrate the stability, and ascertain any changes. Because Q Studies are dependent on local

cultural conditions and context specific factors, as in this case, the specific university climate and

the specific discipline (Biology), the Q Sort may need to be repeated (Baker et al., 2006). The

same three typologies may, or may not, emerge in each repetition of the Q Sort, as the climate at

the university or within the department changes. Interpreting new or additional viewpoints that

emerge from the data takes a researcher who is skilled at this type of data analysis. A Q

Methodology expert may not be available each semester the “Effective Teaching” course is

offered.

Suggested Further Research

The current research study revealed three factors that GTAs had about their graduate

school experience (“The Emerging Teacher,” “The Preferred Researcher,” and “The Anxious

GTA”). Expanding the study to include GTAs teaching in a variety of departments at the same

institution, or in Biology Departments at different institutions, may yield different factors. Other

departments at the same institution may have different pressures, attitudes towards teaching,

instructional training programs, or research programs that would affect the viewpoints of their

GTAs. Other Biology Departments at similar institutions, or at low versus high-research

institutions may yield different viewpoints among their GTAs. GTAs in the same Biology

167

Department sorting at different times (beginning of the semester, end of the semester, mid-

semester, summer, end of program) may affect their perceptions of the statements.

This Q Sort could also be expanded to include faculty members and staff in The

Department of Biology. It would be particularly useful if the faculty could recall their

experiences as a GTA, or to theoretically sort as they believe a new or experienced GTA would

sort. Faculty members have almost always had experience as a GTA themselves, in that they

have completed graduate school, and are now teaching in The Department of Biology. It would

be useful to know whether faculty aligned with a single factor, or whether faculty viewpoints are

distributed in a similar fashion to the GTAs’ viewpoints. This may allow for a faculty modeling

or peer mentoring program that better addresses GTA needs. This type of modeling or mentoring

could be formally developed by faculty in the department. Because faculty serve as mentors or

role models for GTAs, they help to shape the GTAs personal and professional development over

time. Knowing GTA viewpoints could help faculty convey both academic knowledge and the

“hidden curriculum” of academia. The increased awareness about GTA views may benefit

mentors as well, through greater productivity, career satisfaction, and personal gratification

(Rose, Rukstalis, & Schuckit, 2005). It is possible that supervisor views are out of date or out of

sync with today’s GTAs. Making faculty aware of how the needs of GTAs may have changes

since their time as a GTA would have implications for faculty/GTA relationships.

Repeating the Q Sort with GTAs who have sorted at the beginning of their program, and

then completed a year of graduate school, would yield potentially useful pretest/posttest results.

One statement made by supervisors about GTAs is “they don’t know what they don’t know.”

Sorting may take on different meaning after undertaking the graduate school experience. The

emergence of Factor 3, “The Anxious GTA,” would be interesting to revisit, to see if their

168

viewpoints change after formally teaching, or if they persist in their graduate school program.

New or improved interventions for GTAs who load of Factor 3 could improve retention rates

among this segment of GTAs.

Undergraduate learning outcomes are not part of the current study, but could be

incorporated into future studies. The end result of an improved professional development

program should be increased student outcomes – increased scores on Biology Concept

Inventories, grades, motivation, content knowledge, retention rates, etc. Biology GTAs also

teach a variety of courses, from introductory, non-majors laboratories, to upper-level content

specific courses. The improvement of the different courses that GTAs teach in would be of

interest.

Researchers could also pick individual GTAs who were represented by a certain factors,

and do a more in-depth, case-study approach to their viewpoints. This would provide a more

qualitative data about how the GTA came to their point of view than through limited, post-sort

interview questions. Alternatively, researchers could enter the details of the study into a

statistical analysis software package and look for correlations or ANOVAs.

Only one study has attempted to identify graduate students who are at risk for failure

(defined as non-degree completion), using such factors as Graduate Record Examination (GRE)

scores, graduate grade point average (GGPA) in the first nine hours of graduate study,

undergraduate grade point average (UGPA), age, gender, academic area of study, and type of

institution from which the baccalaureate degree was earned. When all records were analyzed, the

GRE verbal score combined with either UGPA or GGPA were significant predictors of degree

completion. The highest graduation rate occurred among students who earned their

undergraduate degrees from master's level institutions; students from bachelor's institutions had

169

the lowest graduation rate. The results varied, however, when individual academic areas were

assessed (Nelson et al., 2000).

Lindle and Rinehart (1998) state “the GRE was designed for ‘traditional’ graduate

students, those who pursue advanced studies full time immediately or shortly after attaining their

baccalaureates” (p.1). Other studies have found that older students score significantly lower

particularly on quantitative measures associated with the GRE (M. J. Clark, 1984; Hartle, 1983).

If the GRE is designed for “traditional” graduate students, and an estimated 48.6% of the

2,637,000 students entering graduate school in 2003 were over the age of 30 (Digest of

Educational Statistics, 2004), predicting success in graduate school needs a new tool for

measurement. Q Methodology could provide that needs assessment tool that indicates which

students are in need of additional support in their graduate school program. Because one group of

GTAs who were represented by Factor 3 demonstrated excessive frustration and anxiety, they

may need counseling, peer mentoring, or advising that GTAs displaying the other viewpoints do

not need. In the distinguishing statements for Factor 3, statement 25 (I have no idea what

students think about me, and that makes me uncomfortable), statement 26 (I have no idea what

the level of understanding is with these students), and statement 48 (I'm worried that the students

won't be able to understand me), helped define this factor. These GTAs may be more

comfortable in their passive position as a student than in their active position as a teacher or

researcher. They also demonstrate that they are worried about many aspects of graduate school

and may require additional support to persist in their programs. An area of future research may

be focusing on how many of these Factor 3 students complete their degrees, or how to

additionally support them.

170

Summary

Chapter V began with a summary of the study, a statement of the problem, a statement of

the procedures, and the general and specific research hypotheses. The conclusions of the study

are drawn from Chapter IV analyses. Three factors, or viewpoints, emerged from the sorting

process. The researcher names these three viewpoints, “The Emerging Teacher,” “The Preferred

Researcher,” and “The Anxious GTA.” Q Methodology can provide predictor typologies that are

more useful than simple variables and demographic information for the classification of people,

especially within program evaluation (Newman & Ramlo, 2011). The implications of the study

are discussed, along with possible future research.

Differentiating the GTAs’ professional development by using their Q Sort data may lead

to more meaningful and relevant professional development. Scaffolding instruction, using self-

directed learning, and peer or faculty mentoring may strengthen the skills of GTAs. GTA

training programs would be much more significant if it used the viewpoints of the different

typologies of GTAs to reinforce positive behaviors, enhance GTAs’ strengths, and supplemented

their skills where needed. Further studies that use these factors/viewpoints to modify GTA

professional development, or modify their graduate school program to encourage program

completion are needed. Future studies may use larger sets of GTAs, GTAs from different

departments, or GTAs from different institutions to further explore GTA viewpoints. Other data,

such as student learning outcomes, field observations, and case studies, may provide greater

detail for developing meaningful GTA professional development.

171

REFERENCES

Adams, K. A. (2002). What colleges and universities want in new faculty (Vol. 7). Association of

American Colleges and Universities. Retrieved from

http://www.aacu.org/pff/pdfs/PFF_Adams.PDF

Addams, H. (2000). Q methodology. Social discourse and environmental policy: An application

of Q methodology, 14–40.

Allen, R. R., & Rueter, T. (1990). Teaching Assistant Strategies: An Introduction to College

Teaching. Retrieved from http://www.eric.ed.gov/ERICWebPortal/recordDetail?

accno=ED313949

Altschuld, J. W., & Witkin, B. R. (1999). From Needs Assessment to Action: Transforming

Needs into Solution Strategies. SAGE.

American Psychological Association. (2010). Publication Manual of the American

Psychological Association (6th Edition.). Washington D.C.: American Psychological

Association. Retrieved from

http://informahealthcare.com/doi/pdf/10.3109/07380577.2011.629024

Anderson, H., Gaff, J. G., & Pruitt-Logan, A. S. (1997). Frequently asked questions about

preparing future faculty. Association of American Colleges and Universities.

Andrews, P. H. (1983). Creating a supportive climate for teacher growth: Developing graduate

students as teachers. Communication Quarterly, 31(4), 259–265.

Angelo, T. A., & Cross, K. P. (1993). Teaching goals inventory. Classroom Assessment

Techniques: A Handbook for College Teachers, 13–23.

172

Anwar, Y. (2013). Ph.D. students rethink the tenure track, scope out non-academic jobs.

Retrieved April 3, 2013, from http://newscenter.berkeley.edu/2013/03/20/beyond-

academia/

Association of American Colleges. (1985). Integrity in the College Curriculum: A Report to the

Academic Commuity. Washington D.C.: Association of American Colleges.

Astin, A. W. (1997). The changing American college student: Thirty-year trends, 1966-1996.

The Review of Higher Education, 21(2), 115–135.

Atwell, R. H. (1996). Doctoral Education Must Match the Nation’s Needs and the Realities of

the Marketplace. Chronicle of Higher Education, 43(14). Retrieved from

http://www.eric.ed.gov/ERICWebPortal/recordDetail?accno=EJ537853

Aubel, J. (1995). Participatory Program Evaluation. A Manual for Involving Program

Stakeholders in the Evaluation Process. Retrieved from

http://www.eric.ed.gov/ERICWebPortal/recordDetail?accno=ED421509

Austin, A. E. (2002). Preparing the Next Generation of Faculty: Graduate School as

Socialization to the Academic Career. The Journal of Higher Education, 73(1), 94–122.

Austin, A. E., Campa, H., Pfund, C., Gillian-Daniel, D., Mathieu, R., & Stoddart, J. (2009).

Preparing STEM doctoral students for future faculty careers. New Directions for

Teaching and Learning, 2009(117), 83–95. doi:10.1002/tl.346

Austin, A. E., & Wulff, D. H. (2004). The challenge to prepare the next generation of faculty.

Paths to the professoriate: Strategies for enriching the preparation of future faculty, 3–

16.

Baker, R. M., Thompson, C., & Mannion, R. (2006). Q methodology in health economics.

Journal of Health Services Research & Policy, 11(1), 38–45.

173

Becher, T. (1981). Towards a definition of disciplinary cultures. Studies in Higher Education,

6(2), 109–122.

Becher, T., & Trowler, P. R. (1989). Academic tribes and territories. Open University Press

Milton Keynes. Retrieved from http://ceps.pef.uni-lj.si/knjiznica/izbirni/D.c.

%202001%20Becher%20and%20Towler_Academic%20Tribes%20and

%20Teritories.pdf

Berliner, D. C. (2002). The Effectiveness of“ Teach for America” and Other Under-certified

Teachers on Student Academic Achievement: A Case of Harmful Public Policy Ildiko

Laczko-Kerr Arizona Department of Education. education policy analysis archives,

10(37). Retrieved from

http://www.mail.seriousgiving.org/files/unitedstates/TFA/Laczko-Kerr%20and

%20Berliner.%202002.%20The%20effectiveness%20of%20Teach%20for

%20America.pdf

Berrett, D. (2012, October 24). Today’s Faculty: Stressed, Focused on Teaching, and Undeterred

by Long Odds. The Chronicle of Higher Education. Retrieved from

http://chronicle.com/article/Todays-Faculty-Stressed-and/135276/

Bess, P. J. L. (1978). Anticipatory socialization of graduate students. Research in Higher

Education, 8(4), 289–317. doi:10.1007/BF00976801

Betz, N. E., & Fitzgerald, L. F. (1987). The career psychology of women. Academic Press.

Retrieved from http://psycnet.apa.org/psycinfo/1987-97913-000

Bhavsar, V., Burke, B., Carter, C., Jensen, J., Kraemer, P., Stevens, N., … Rutledge, K. W.

(2007). Handbook of Undergraduate Education. Retrieved from

http://www.uky.edu/~rljone00/Mega/ABOUT/ehandbook.pdf

174

Bloom, L. (1981). Why graduate students can’t write: Implications of research on writing anxiety

for graduate education. Journal of advanced composition, 2(1-2), 103–117.

Boice, R. (1991). New faculty as teachers. The Journal of Higher Education, 150–173.

Borich, G. D., & Fuller, F. F. (1974). Teacher concerns checklist: An instrument for measuring

concerns for self, task, and impact. Austin, Texas: University of Texas Research and

Development Center for Teacher Education.

Bound, J., & Turner, S. (2002). Going to war and going to college: Did World War II and the GI

Bill increase educational attainment for returning veterans? Journal of Labor Economics,

20(4), 784–815.

Bowe, R. S. (2010). Instructional technology adoption amongst college of education faculty: A

5-year phenomenological case study. National-Louis University.

Bowman, R. L., Bowman, V. E., & DeLucia, J. L. (1990). Mentoring in a graduate counseling

program: Students helping students. Counselor education and supervision, 30(1), 58–65.

Boyer, E. L. (1991). The Scholarship of Teaching from: Scholarship Reconsidered: Priorities of

the Professoriate. College Teaching, 39(1), 11–13.

Boyle, P., & Boice, B. (1998). Systematic Mentoring for New Faculty Teachers and Graduate

Teaching Assistants. Innovative Higher Education, 22(3), 157–179.

doi:10.1023/A:1025183225886

Branstetter, S. A., & Handelsman, M. M. (2000). Graduate Teaching Assistants: Ethical

Training, Beliefs, and Practices. Ethics & Behavior, 10(1), 27–50.

Braxton, J. M., Lambert, L. M., & Clark, S. C. (1995). Anticipatory socialization of

undergraduate college teaching norms by entering graduate teaching assistants. Research

in Higher Education, 36(6), 671–686.

175

Breen, R., Brew, A., Jenkins, A., & Lindsay, R. (2004). Reshaping Teaching in Higher

Education: A Guide to Linking Teaching with Research. Routledge.

Brickman, W. W. (1972). American higher education in historical perspective. The ANNALS of

the American Academy of Political and Social Science, 404(1), 31–43.

Broady-Ortmann, C. (2002). Teachers’ Perceptions of a Professional Development Distance

Learning Course: A Qualitative Case Study. Journal of Research on Technology in

Education, 35(1), 107.

Brouwer, M. (1999). Q is accounting for tastes. Journal of Advertising Research, 39(2), 35–39.

Brown, H. (2005). Graduate Enrollment and Degrees: 1986-2004. Council of Council of

Graduate Schools, Office of Research and Information Services. Retrieved from

http://www.immagic.com/eLibrary/ARCHIVES/GENERAL/COGS_US/C051021B.pdf

Brown, P. L., Abell, S. K., Demir, A., & Schmidt, F. J. (2006). College science teachers’ views

of classroom inquiry. Science education, 90(5), 784–802.

Brown, S. R. (1980). Political subjectivity: applications of Q methodology inpolitical science.

New Haven: Yale University Press.

Brown, S. R. (1991). The History and Principles of Q Methodology in Psychology and the Social

Sciences. Retrieved from http://facstaff.uww.edu/cottlec/QArchive/Bps.htm

Brown, S. R. (1993). A primer on Q methodology. Operant subjectivity, 16(3/4), 91–138.

Brown, S. R., & Narayan, D. (2005). Applying Q methodology to empowerment. Measuring

empowerment: Cross-disciplinary perspectives, 197–215.

Brown, S. R., & Ungs. (1970). Representativeness and the study of political behavior: An

application of Q technique to reactions to the Kent State incident. Social Science

Quarterly, 51(3), 514–526.

176

Brunetti, G. J. (2001). Why do they teach? A study of job satisfaction among Long-Term High

School Teachers. Teacher Education Quarterly. Retrieved from

http://teqjournal.org/backvols/2001/28_3/v28n306.pdf

Buerkel-Rothfuss, N. L., & Fink, D. S. (1993). Student perceptions of teaching assistants (TAs).

Basic Communication Course Annual, 5, 71–100.

Burmila, E. M. (2010). Graduate Students as Independent Instructors: Seven Things to Know

about Teaching Your Own Course while in Graduate School. PS: Political Science and

Politics, 43(3), 557–560.

Burrell, S. (1967). The G.I. Bill and the Great American Transformation. Boston University

Graduate Journal, 15, 3.

Bush, N. B. (1969). The Student and His Professor: Colonial Times to Twentieth Century. The

Journal of Higher Education, 40(8), 593. doi:10.2307/1978030

Butler, D. D., Laumer, J. F., & Moore, M. (1993). A Content Analysis of Pedagogical and Policy

Information Used in Training Graduate Teaching Assistants. Journal for Higher

Education Management, 9(1), 27–37.

Carpenter, S. (2010). Tenure-Track Jobs Remain Scarce | Science Careers. Retrieved April 3,

2013, from http://sciencecareers.sciencemag.org/career_magazine/previous_issues/

articles/2010_01_15/caredit.a1000006

Carroll, J. G. (1980). Effects of Training Programs for University Teaching Assistants: A

Review of Empirical Research. The Journal of Higher Education, 51(2), 167–183.

doi:10.2307/1981372

Cassuto, L. (2011). Changing the Way We Socialize Doctoral Students. Chronicle of Higher

Education, 57(19), A36–a37.

177

Cassuto, L. (2012, December 3). What If We Made Fewer Ph.D.’s? The Chronicle of Higher

Education. Retrieved from http://chronicle.com/article/What-If-We-Made-Fewer-

PhDs-/136083/

Chen, H.-T. (2005). Practical Program Evaluation: Assessing and Improving Planning,

Implementation, and Effectiveness. SAGE.

Chesler, N. C., & Chesler, M. A. (2002). Gender-Informed Mentoring Strategies for Women

Engineering Scholars: On Establishing a Caring Community. Journal of Engineering

Education, 91(1), 49–55.

Cho, Y. J., Sohoni, S., & French, D. P. (2010). Need Assessment for Graduate Teaching

Assistant Training: Identifying Important But Under-Prepared Roles. In ASEE Midwest

Section Annual Conference. Retrieved from

http://www.asee.org/documents/sections/midwest/2010/18_Cho.pdf

Choo, C. W. (1996). The knowing organization: how organizations use information to construct

meaning, create knowledge and make decisions. International Journal of Information

Management, 16(5), 329–340.

Christensen, T. B., Alexander, J., Nelson-Laird, T., & Robinson, J. (2011). TEACHING ON

PURPOSE: COLLABORATIVE REFLECTION IN A GTA TEACHER

PREPARATION COURSE. Retrieved from

http://www.tylerchristensen.com/research/dissertation_proposal_2011.pdf

Clark, M. J. (1984). Older and Younger Graduate Students: A Comparison of Goals, Grades, and

GRE Scores. Retrieved from http://www.eric.ed.gov/ERICWebPortal/recordDetail?

accno=ED245645

178

Clark, S. M., & Corcoran, M. (1986). Perspectives on the professional socialization of women

faculty: A case of accumulative disadvantage? The Journal of Higher Education, 20–43.

Clarke, D., & Hollingsworth, H. (2002). Elaborating a model of teacher professional growth.

Teaching and teacher education, 18(8), 947–967.

Coll, R. K., Zegwaard, K., & Hodges, D. (2002). Science and Technology stakeholders’ ranking

of graduate competencies Part 1: Employer perspective. Asia-Pacific Journal of

Cooperative Education, 3(2), 19–28.

Collins, L. J. (2009, January 1). Participants’ Perceptions of the Instructional Design of an

Online Professional Development Module for Teaching English Language Learners: A Q

Methodology Study. ProQuest LLC. Retrieved from eric.

Columbia University. (2013). Career Exploration for Ph.D.s in Science, Technology,

Engineering, and Math Disciplines | Center for Career Education. Retrieved August 13,

2013, from http://www.careereducation.columbia.edu/resources/tipsheets/non-academic-

career-options-phds-sciences-engineering-and-mathematics

Coogan, J., & Herrington, N. (2011). Q methodology: an overview. Research in Secondary

Teacher Education, 1(2), 24–28.

Coomes, M. D. (2000). The Historical Roots of Enrollment Management. New Directions for

Student Services, 2000(89), 5–18. doi:10.1002/ss.8901

Corr, S. (2001). An introduction to Q methodology, a research technique. The British Journal of

Occupational Therapy, 64(6), 293–297.

Council of Graduate Schools. (1990). The doctor of philosophy degree: A policy statement.

Washington D.C.: Council of Graduate Schools.

179

Council of Graduate Schools. (2004). Ph.D. completion and attrition: Policy, numbers,

leadership, and next steps. Washington D.C.: : Council of Graduate Schools.

Crebert, G., Bates, M., Bell, B., Patrick, C.-J., & Cragnolini, V. (2004). Developing generic

skills at university, during work placement and in employment: graduates’ perceptions.

Higher Education Research & Development, 23(2), 147–165.

Creswell, J. W. (2008). Research design: Qualitative, quantitative, and mixed methods

approaches. Sage Publications, Incorporated. Retrieved from

http://books.google.com/books?

hl=en&lr=&id=bttwENORfhgC&oi=fnd&pg=PR1&dq=There+are+qualitative,

+quantitative,

+and+mixed+method+researchers&ots=CaHtgP3616&sig=Uyg40vV9i11IE5v1BCTBgv

VBddI

Cross, R. M. (2005). Exploring attitudes: the case for Q methodology. Health Education

Research, 20(2), 206–213.

Cryer, P. (1998). Transferable skills, marketability and lifelong learning: the particular case of

postgraduate research students. Studies in Higher Education, 23(2), 207–216.

Cummins, R. A., & Gullone, E. (2000). Why we should not use 5-point Likert scales: The case

for subjective quality of life measurement. In Proceedings, second international

conference on quality of life in cities (pp. 74–93). Retrieved from

http://www.deakin.edu.au/research/acqol/iwbg/correspondence/2002/Cummins_Gullone_

2000_Likert_Scales.doc

Dannels, D. (2002). Communication across the curriculum and in the disciplines: Speaking in

engineering. Communication Education, 51(3), 254–268.

180

Darling, A. L., & Staton, A. Q. (1989). Socialization of graduate teaching assistants: A case

study in an American university. Internation Journal of Qualitative Studies in Education,

2(3), 221–235.

Darling-Hammond, L. (2005). Does Teacher Preparation Matter? Evidence about Teacher

Certification, Teach for America, and Teacher Effectiveness Linda Darling-Hammond,

Deborah J. Holtzman, Su Jin Gatlin, and Julian Vasquez Heilig. education policy analysis

archives, 13(42), 2.

Davies, J., Hides, M. T., & Casey, S. (2001). Leadership in higher education. Total Quality

Management, 12(7-8), 1025–1030.

De Graaf, G., & Van Exel, J. (2008). Using Q methodology in administrative ethics. Public

Integrity, 11(1), 63–78.

De Valero, Y. F. (2001). Departmental factors affecting time-to-degree and completion rates of

doctoral students at one land-grant research institution. Journal of Higher Education,

341–367.

DeBoer, K. B. (1979). Teacher preparation for graduate assistants. Communication Education,

28(4), 328–331. doi:10.1080/03634527909378374

DeChenne, S. E. (2010). Learning to teach effectively: science, technology, engineering, and

mathematics graduate teaching assistants’ teaching self-efficacy. Retrieved from

http://scholarsarchive.library.oregonstate.edu/jspui/handle/1957/15737

Decker, P. T., Mayer, D. P., & Glazerman, S. (2004). The effects of Teach for America on

students: Findings from a national evaluation. University of Wisconsin–Madison,

Institute for Research on Poverty. Retrieved from

http://www.voicesforservice.org/events/images/mathematica_glazerman.pdf

181

Deiro, J. A. (1996). Teaching with Heart. Making Healthy Connections with Students. ERIC.

Retrieved from http://www.eric.ed.gov/ERICWebPortal/recordDetail?accno=ED398230

Delnero, J., & Montgomery, D. (2001). Perceptions of work among California agriculture

teachers. Journal of Agricultural Education, 42(2), 56–67.

DeNeef, A. L. (2002). The Preparing Future Faculty Program: What Difference Does it Make?

Association of American Colleges and Universities. Retrieved from http://aacu-

secure.nisgroup.com/pff/pdfs/PFF_DeNeef.PDF

Dennis, K. E. (1986). Q methodology: Relevance and application to nursing research. Advances

in Nursing Science, 8(3), 6–17.

Drake, D. A. M. (2011). Effects of amount and type of graduate teaching assistant (GTA)

training on perceived teacher credibility and student motivation. Retrieved from

http://repositories.tdl.org/ttu-ir/handle/2346/16195

Duchelle, A. E., Biedenweg, K., Lucas, C., Virapongse, A., Radachowsky, J., Wojcik, D. J., …

Kainer, K. A. (2009). Graduate students and knowledge exchange with local

stakeholders: Possibilities and preparation. Biotropica, 41(5), 578–585.

Eden, S., Donaldson, A., & Walker, G. (2005). Structuring subjectivities? Using Q methodology

in human geography. Area, 37(4), 413–422. doi:10.1111/j.1475-4762.2005.00641.x

Edgerton, R., Rice, R. G., & Chait, J. (1997). New pathways: Faculty careers and employment in

the 21st century: A preliminary statement for consideration by colleagues. Presented at

the American Association for Higher Education, Washington D.C.: Unpublished Paper.

Ehrenberg, R. G., Jakubson, G. H., Groen, J. A., So, E., & Price, J. (2007). Inside the black box

of doctoral education: What program characteristics influence doctoral students’ attrition

182

and graduation probabilities? Educational Evaluation and policy analysis, 29(2), 134–

150.

Eison, J., & Vanderford, M. (1993). Enhancing GTA training in academic departments: Some

self-assessment guidelines. Retrieved from

http://digitalcommons.unl.edu/podimproveacad/277/

Ellis, E. M. (2001). The Impact of Race and Gender on Graduate School Socialization,

Satisfaction with Doctoral Study, and COmmitment to Degree Completion. Western

Journal of Black Studies, 25(1), 30.

Enz, C. A., Renaghan, L. M., & Geller, A. N. (1993). Graduate-level education: A survey of

stakeholders. The Cornell Hotel and Restaurant Administration Quarterly, 34(4), 90–95.

Evans, N. J., Forney, D. S., Guido, F. M., Patton, L. D., & Renn, K. A. (2009). Student

development in college: Theory, research, and practice. Jossey-Bass. Retrieved from


hl=en&lr=&id=glf01scdrsUC&oi=fnd&pg=PR13&dq=Graduate+school+may+give+the

+student+a+chance+to+pursue+theories+they+may+hold&ots=fXNrVubtZs&sig=j1kx4

UX1Qt_9PIxCa8xa7Bl4C38

Ferren, A., Gaff, J. G., & Clayton-Pedersen, A. (2002, Summer). Will Reforms Survive?

Strategies for Sustaining Preparing Future Faculty Programs. Retrieved March 12, 2013,

from https://www.aacu.org/liberaleducation/le-su02/le-su02feature2.cfm

Filstad, C. (2004). How newcomers use role models in organizational socialization. Journal of

Workplace Learning, 16(7), 396–409.

Finkelstein, M. J. (2006). The study of academic careers: Looking back, looking forward.

HIGHER EDUCATION:, 159–212.

183

Finkelstein, M. J., Seal, R. K., & Schuster, J. H. (1998). The new academic generation: A

profession in transformation. Johns Hopkins University Press. Retrieved from


hl=en&lr=&id=J4bezYICvZ8C&oi=fnd&pg=PR9&dq=Finkelstein,+M.+J.,+Seal,+R.

+K.,+%26+Schuster,+J.+H.+(1998).

+The+new+academic+generation&ots=JK_NnUnW8b&sig=LV7suvhDyVTIF_CBS0-

kqdBiGrc

Fischer, G., & Scharff, E. (1998). Learning technologies in support of self-directed learning.

Journal of Interactive Media in Education, 1998(2). Retrieved from

http://jime.open.ac.uk/jime/article/viewArticle/1998-4/26

Fleet, C. M., ROSSER, M. F. N., ZUFALL, R. A., PRATT, M. C., FELDMAN, T. S., &

LEMONS, P. P. (2006). Hiring Criteria in Biology Departments of Academic

Institutions. BioScience, 56(5), 430–436. doi:10.1641/0006-

3568(2006)056[0430:HCIBDO]2.0.CO;2

Fox, M. F. (1983). Publication productivity among scientists: A critical review. Social Studies of

Science, 13(2), 285–305.

Friedrich, G., & Powell, R. (1979). Workshop for Graduate Teaching Assistants. Improving

College and University Teaching, 27(4), 171–173.

Fuller, F. F. (1969). Concerns of teachers: A developmental characterization. American

Educational Research Journal, 6, 207–236.

Gaff, J. G. (2002a). Preparing Future Faculty and Doctoral Education. Change: The Magazine of

Higher Learning, 34(6), 63–66. doi:10.1080/00091380209605571

184

Gaff, J. G. (2002b). The disconnect between graduate education and the realities of faculty work:

A review of recent research. Liberal Education, 88(3), 6–13.

Gaff, J. G., & Gaff, S. S. (1981). Student-faculty relationships. The modern American college,

642–656.

Gallagher, K., & Porock, D. (2010). The Use of Interviews in Q Methodology: Card Content

Analysis. [Miscellaneous Article]. Nursing Research July/August 2010, 59(4), 295–300.

doi:10.1097/NNR.0b013e3181e4ffff

Gardner, S. K. (2005). “IF IT WERE EASY, EVERYONE WOULD HAVE A PH. D.”

DOCTORAL STUDENT SUCCESS: SOCIALIZATION AND DISCIPLINARY

PERSPECTIVES. Washington State University. Retrieved from

http://www.dissertations.wsu.edu/Dissertations/Spring2005/s_gardner_040605.pdf

Gardner, S. K. (2007). “I heard it through the grapevine”: Doctoral student socialization in

chemistry and history. Higher Education, 54(5), 723–740.

Gardner, S. K. (2008). Fitting the mold of graduate school: A qualitative study of socialization in

doctoral education. Innovative Higher Education, 33(2), 125–138.

Gardner, S. K. (2010a). Contrasting the socialization experiences of doctoral students in high-

and low-completing departments: a qualitative analysis of disciplinary contexts at one

institution. The Journal of Higher Education, 81(1), 61–81.

Gardner, S. K. (2010b). Faculty perspective on doctoral student socialization in five disciplines.

International Journal of Doctoral Studies, 5, 39–53.

Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes

professional development effective? Results from a national sample of teachers.

American educational research journal, 38(4), 915–945.

185

Garland, T. N. (1983). A Training Program for Graduate Teaching Assistants: The Experiences

of One Department. Teaching Sociology, 10(4), 487–503. doi:10.2307/1317297

Gelber, S. (2005). A“ Hard-Boiled Order”: The Reeducation of Disabled WWI Veterans in New

York City. journal of social history, 39(1), 161–180.

George, M. A., & Davis-Wiley, P. (2000). Team Teaching a Graduate Course. Case Study: A

Clinical Research Course. College Teaching, 48(2), 75–80.

Gibson, S., & Dembo, M. H. (1984). Teacher efficacy: A construct validation. Journal of

educational psychology, 76(4), 569.

Gilbert, R., Balatti, J., Turner, P., & Whitehouse, H. (2004). The generic skills debate in research

higher degrees. Higher Education Research & Development, 23(3), 375–388.

Ginorio, A. (1995). A Culture of Meaningful Community. Bridging the Gender Gap in

Engineering and Science.

Girves, J. E., & Wemmerus, V. (1988). Developing Models of Graduate Student Degree

Progress. The Journal of Higher Education, 59(2), 163–189. doi:10.2307/1981691

Golde, C. M. (1998). Beginning graduate school: Explaining first-year doctoral attrition. New

directions for higher education, 1998(101), 55–64.

Golde, C. M. (2002). Beginning graduate school: Explaining first-year doctoral attrition. New

directions for higher education, 1998, 55–64.

Golde, C. M. (2005). The role of the department and discipline in doctoral student attrition:

Lessons from four departments. The Journal of Higher Education, 76(6), 669–700.

Golde, C. M., & Dore, T. M. (2001). At cross purposes: What the experiences of today’s doctoral

students reveal about doctoral education. Philadelphia: Pew Charitable Trusts. Retrieved

from http://www.ssc.wisc.edu/~oliver/sociology/PhDEducationreport.pdf

186

Golde, C. M., & Walker, G. E. (2002). Overview of the Carnegie initiative on the doctorate.

Carnegie Foundation for the Advancement of Teaching.

Golde, C. M., & Walker, G. E. (2006). Envisioning the future of doctoral education: Preparing

stewards of the discipline. Carnegie Essays on the Doctorate, Jossey-Bass–Carnegie

Foundation for the Advancement of Teaching, San Francisco, CA. Retrieved from

http://www.lavoisier.fr/livre/notice.asp?id=OLSWL6AS32AOWX

Golish, T. D. (1999). Students’ use of compliance gaining strategies with graduate teaching

assistants: Examining the other end of the power spectrum. Communication Quarterly,

47(1), 12–32. doi:10.1080/01463379909370121

Gorsuch, G. J. (2003). The educational cultures of international teaching assistants and US

universities. TESL-EJ, 7(3), 1–17.

Grad School Life. (2012, August 16). The Chronicle of Higher Education. Retrieved from

http://chronicle.com/forums/index.php/board,42.0.html

Graves, M. F., & Braaten, S. (1996). Scaffolded reading experiences: Bridges to success.

Preventing School Failure: Alternative Education for Children and Youth, 40(4), 169–

173.

Gredler, M. E. (1996). Program evaluation. Merrill. Retrieved from

http://www.getcited.org/pub/103278431

Grizzard, F. E. (2009). Documentary History of the Construction of the Buildings at the

University of Virginia, 1817-1828 :: :: University of Virginia Library. Retrieved March

15, 2013, from http://xtf.lib.virginia.edu/xtf/view?docId=grizzard/uvaGenText/tei/

grizzard.xml

187

Guskey, T. R. (1994). Professional development in education: in search of the optimal mix.


Hartle, T. (1983). Older Students and the GRE Aptitude Test. Retrieved from


Hayes, M. M. (2007). Design and analysis of the student strengths index (SSI) for nontraditional

graduate students. University of Nebraska,, Lincoln, NE. Retrieved from

http://www.sagepub.com/creswellstudy/Sample%20Student%20Proposals/Proposal-

QUAN-Hayes.pdf

Hendrix, K. G. (1995). Preparing Graduate Teaching Assistants (GTAs) to Effectively Teach the

Basic Course. Retrieved from http://www.eric.ed.gov/ERICWebPortal/detail?

accno=ED384933

Howard, C., Buskist, W., & Stowell, J. (1993). Graduate Teaching Assistants. Retrieved from

http://sites.roosevelt.edu/smeyers/files/2011/11/Chapter-2-Creating-Effective-Working-

Relationships.pdf

Isaac, P. D., Quinlan, S. V., & Walker, M. M. (1992). Faculty perceptions of the doctoral

dissertation. The Journal of Higher Education, 241–268.

Jacobs, S. R., & Dodd, D. (2003). Student burnout as a function of personality, social support,

and workload. Journal of College Student Development, 44(3), 291–303.

Johnson, B., & Christensen, L. (2007). Educational research: Quantitative, qualitative, and

mixed approaches. Sage Publications, Incorporated. Retrieved from


hl=en&lr=&id=8qoaXPh6E4UC&oi=fnd&pg=PA1&dq=There+are+qualitative,

188

+quantitative,+and+mixed+method+researchers&ots=gH-

ZqU8ajG&sig=pjIilchoQjrpSoOs08x6pCLMWOw

Jones, E. (2003). Beyond supply and demand: Assessing the Ph. D. job market. Occupational

Outlook Quarterly, 46(4), 22–33.

Jones, J. L. (1993). TA training: From the TA’s point of view. Innovative Higher Education,

18(2), 147–161.

Kaufman, R. A., Rojas, A. M., & Mayer, H. (1993). Needs assessment: A user’s guide.

Educational Technology. Retrieved from http://books.google.com/books?

hl=en&lr=&id=Ek_l0MOsZHoC&oi=fnd&pg=PA1&dq=kaufman+1988+needs+assessm

ent&ots=0VQc6Qgbpg&sig=MGX-8ilP1rhn7auaC6LtHG7yKXY

Kaufman, R. A., & Valentine, G. (1989). Relating needs assessment and needs analysis.

Performance+ Instruction, 28(10), 10–14.

Kendall, K. D., & Schussler, E. E. (2012). Does Instructor Type Matter? Undergraduate Student

Perception of Graduate Teaching Assistants and Professors. CBE-Life Sciences

Education, 11(2), 187–199. doi:10.1187/cbe.11-10-0091

Kohn, P. M., Lafreniere, K., & Gurevich, M. (1990). The inventory of college students’ recent

life experiences: A decontaminated hassles scale for a special population. Journal of

Behavioral Medicine, 13(6), 619–630.

Koocher, G. P., & Keith-Spiegel, P. (2008). Ethics in Psychology: Professional Standards and

Cases (Third Edition). Journal of Ethics in Mental Health, 3(1), 1–2.

Kreber, C. (2001). The Scholarship of Teaching and Its Implementation in Faculty Development

and Graduate Education. New Directions for Teaching and Learning, 2001(86), 79–88.

doi:10.1002/tl.18

189

Kreber, C. (2005). Charting a critical course on the scholarship of university teaching movement.

Studies in Higher Education, 30(4), 389–405.

Kuh, G. D., & Whitt, E. J. (1988). The Invisible Tapestry. Culture in American Colleges and

Universities. ASHE-ERIC Higher Education, Report No. 1, 1988. ERIC. Retrieved from


Kuther, T. (2013). What is the Difference Between a Master’s Degree and a Doctoral Degree?

About.com Graduate School. Retrieved February 7, 2013, from

http://gradschool.about.com/od/admissionsadvice/a/masterphd.htm

Lalla, M., Facchinetti, G., & Mastroleo, G. (2005). Ordinal scales and fuzzy set systems to

measure agreement: An application to the evaluation of teaching activity. Quality and

Quantity, 38(5), 577–601. doi:10.1007/s11135-005-8103-6

Leigh, D., Watkins, R., Platt, W. A., & Kaufman, R. A. (2000). Alternate models of needs

assessment: Selecting the right one for your organization. Human Resource Development

Quarterly, 11(1), 87–93.

Ley, T., Kump, B., & Gerdenitsch, C. (2010). Scaffolding self-directed learning with

personalized learning goal recommendations. In User modeling, adaptation, and

personalization (pp. 75–86). Springer. Retrieved from

http://link.springer.com/chapter/10.1007/978-3-642-13470-8_9

Li-Fen Lilly Lu, & Jeng, I. (2006). Knowledge Construction in Inservice Teacher Online

Discourse: Impacts of Instructor Roles and Facilitative Strategies. Journal of Research on

Technology in Education, 39(2), 183–202.

Likert, R. (1967). The method of constructing an attitude scale. In M. Fishbein (Ed.), Readings in

attitude theory and measurement (pp. 90 – 99). New York: John Wiley.

190

Lindle, J. C., & Rinehart, J. S. (1998). Emerging Issues with the Predictive Applications of the

GRE in Educational Administration Programs: One Doctoral Program’s Experience.


Longbottom, J. E., & Butler, P. H. (1999). Why teach science? Setting rational goals for science

education. Science Education, 83(4), 473–492.

Loughran, J., Mulhall, P., & Berry, A. (2004). In search of pedagogical content knowledge in

science: Developing ways of articulating and documenting professional practice. Journal

of Research in Science Teaching, 41(4), 370–391. doi:10.1002/tea.20007

Lovitts, B. E. (2001). Leaving the ivory tower: The causes and consequences of departure from

doctoral study. Rowman & Littlefield Pub Incorporated. Retrieved from


hl=en&lr=&id=gMNC3NhxryUC&oi=fnd&pg=PR6&dq=lovitts+2001&ots=6GODLSZ

V4L&sig=YSQllQT25igG8GCRsYfU3S7jggE

Luft, J. A., Kurdziel, J. P., Roehrig, G. H., & Turner, J. (2004). Growing a garden without water:

Graduate teaching assistants in introductory science laboratories at a doctoral/research

university. Journal of Research in Science Teaching, 41(3), 211–233.

doi:10.1002/tea.20004

Malaney, G. D. (1987). Why Students Pursue Graduate Education, How They Find Out about a

Program, and Why They Apply to a Specific School. College and University, 62(3), 247–

58.

Mallinckrodt, B., & Leong, F. T. (1992). Social support in academic programs and family

environments: Sex differences and role conflicts for graduate students. Journal of

Counseling & Development, 70(6), 716–723.

191

Marincovich, M., Prostko, J., & Stout, F. (1998). The Professional Development of Graduate

Teaching Assistants. ERIC. Retrieved from


Marston, S. H. (2010). Why Do They Teach? A Comparison of Elementary, High School, and

College Teachers. Education, 131(2), 437–454.

Marston, S. H., & Brunetti, G. J. (2009). Job satisfaction of experienced professors at a liberal

arts college. Education, 130(2), 323–347.

Marting, J. (1987). An Historical Overview of the Training of Teaching Assistants. Retrieved

from http://www.eric.ed.gov/ERICWebPortal/detail?accno=ED285511

Mason, M. A., Goulden, M., & Frasch, K. (2009). Why graduate students reject the fast track.

Academe Online, 95(1). Retrieved from

http://www.aaup.org/AAUP/CMS_Templates/AcademeTemplates/AcademeArticle.aspx?

NRMODE=Published&NRNODEGUID=%7B3DCE39ED-F87D-4069-8117-

4F2A1F8CFA3E%7D&NRORIGINALURL=%2FAAUP%2Fpubsres%2Facademe

%2F2009%2FJF%2FFeat%2Fmaso.ht&NRCACHEHINT=NoModifyGuest

McKeachie, W. J. (1990). Research on college teaching: The historical background. Journal of

Educational Psychology, 82(2), 189.

McKeown, B. (2001). Loss of meaning in likert scaling: A note of the Q methodological

alternative. Operant Subjectivity, 24, 201–206.

McKeown, B., & Thomas, D. (1988). Q Methodology. Newbury Park: Sage.

McNeil, K., Newman, I., & Steinhauser, J. (2005). How to be Involved in Program Evaluation:

What Every Adminstrator Needs to Know. R&L Education. Retrieved from


192

hl=en&lr=&id=VNYvFPXCgA8C&oi=fnd&pg=PP1&dq=mcneil+newman+and+steinha

user&ots=qYGyU_U8o8&sig=vHJByH-pPtSK-VZYR6RMOTIcr8w

Meitl, J. L. (2008). Graduate teaching assistant (GTA) development in the college classroom.

Retrieved from http://ir.library.oregonstate.edu/jspui/handle/1957/9147

Merton, R. K. (1968). Social theory and social structure. Simon and Schuster. Retrieved from


hl=en&lr=&id=dyqZOcux9o0C&oi=fnd&pg=PR7&ots=_OrL3-

sZnC&sig=pZCCUTETNMRucNcD8Vkz1ke_So0

Moore, R. (1991). Preparing graduate teaching assistants to teach biology. Journal of College

Science Teaching, 20, 358–361.

Moscovici, S. (1988). Notes towards a description of social representations. European journal of

social psychology, 18(3), 211–250.

Mountain, J. R., & Pleck, M. H. (2000). The role of expectations in the determination of

educational outcomes. In Frontiers in Education Conference, 2000. FIE 2000. 30th

Annual (Vol. 2, p. S2B–17). Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?

arnumber=896640

Mueller, A., Perlman, B., McCann, L. I., & McFadden, S. H. (1997). A faculty perspective on

teaching assistant training. Teaching of Psychology, 24(3), 167–171.

Muzaka, V. (2009). The niche of graduate teaching assistants (GTAs): perceptions and

reflections. Teaching in Higher Education, 14(1), 1–12.

Myers, S. A. (1998). GTAs as organizational newcomers: The association between supportive

communication relationships and information seeking. Western Journal of

Communication, 62(1), 54–73. doi:10.1080/10570319809374597

193

Nelson, J. S., Nelson, C. V., & Malone, B. G. (2000). A Longitudinal Investigation of the Success

Rate of At-Risk Graduate Students: A Follow-Up Study.

Newman, I., & Ramlo, S. (2010). Using Q Methodology and Q factor analysis in mixed methods

research. Sage handbook of mixed methods in social & behavioral research, 505–530.

Newman, I., & Ramlo, S. (2011). Classifying individuals using Q Methodology and Q Factor

Analysis: Applications of Two Mixed Methodologies for Program Evaluation. Journal of

Research in Education, 21(2), 20–31.

Newton, S., Soleil, L., Utschig, T., & Llewellyn, D. (2010). Design and assessment of

professional educational development programming for graduate students at a research

extensive university. In American Society for Engineering Education.

Nicholas, J. B. (2011). INVESTIGATING ENGINEERING EDUCATORS’VIEWS ON THE

USE OF EDUCATIONAL TECHNOLOGY: AQ METHODOLOGY STUDY. Retrieved

from http://etd.ohiolink.edu/send-pdf.cgi/Nicholas%20John%20B.pdf?akron1308681264

Nicklow, J. W., Marikunte, S. S., & Chevalier, L. R. (2007). Balancing pedagogical and

professional practice skills in the training of graduate teaching assistants. Journal of

Professional Issues in Engineering Education and Practice, 133(2), 89–93.

Nyquist, J. D., Abbott, R. D., & Wulff, D. H. (1989). Teaching assistant training in the 1990s.

Jossey-Bass Inc Pub.

Nyquist, J. D., & et al. (1991). Preparing the Professoriate of Tomorrow to Teach. Selected

Readings in TA Training. ERIC. Retrieved from


194

Nyquist, J. D., & Woodford, B. J. (2000). Re-envisioning the Ph. D.: What Concerns do We

Have? University of Washington Washington. Retrieved from

http://www.naufrp.org/pdf/Re-envisioning%20the%20PhD.pdf

Ohashi, J., Ohashi, H., & Paltridge, B. (2008). Finishing the dissertation while on tenure track:

Enlisting support from inside and outside the academy. Learning the literacy practices of

graduate school: Insiders’ reflections on academic enculturation, University of Michigan

Press, Ann Arbor, 218–229.

Orvik, J. M. (1972). Social desirability for the individual, his group, and society. Multivariate

Behavioral Research. Retrieved from http://psycnet.apa.org/?

fa=main.doiLanding&uid=1972-27265-001

Pajares, M. F. (1992). Teachers’ Beliefs and Educational Research: Cleaning Up a Messy

Construct. Review of Educational Research, 62(3), 307–32.

Palmer, P. J. (1993). Remembering the heart of higher education. In From AAHE’s 1993

National Conference on Higher Education. Washington, DC: American Association of

Higher Education.

Park, C. (2002). Neither fish nor fowl? The perceived benefits and problems of using Graduate

Teaching Assistants (GTAs) to teach undergraduate students. Higher Education Review,

35(1), 50–62.

Park, C. (2004). The Graduate Teaching Assistant (GTA): Lessons from North American

Experience. Teaching in Higher Education, 9(3), 349–361.

Parrett, J. L. (1987). A ten-year review of TA training programs: Trends, patterns, and common

practices. In Institutional responsibilities and responses in the employment and education

of teaching assistants: Readings from a national conference (pp. 67–80).

195

Pease, J. (1967). Faculty Influence and Professional Participation of Doctoral Students*.

Sociological Inquiry, 37(1), 63–70.

Peritore, N. P. (1989). Brazilian party left opinion: A Q-Methodology profile. Political

Psychology, 10(4), 675–702. doi:10.2307/3791333

Perkinson, R. (1996). Bad marks for Yale’s labor policies: Rights of graduate-student teachers at

Yale University. The Progressive, 60(2), 20–21.

Prasad, R. S. (2001). Development of the HIV/AIDS Q-sort instrument to measure physician

attitudes [Clinical Research Methods]. Family Medicine, Nov/Dec, 772–778.

Pratt, D. D., Boll, S. L., & Collins, J. B. (2007). Towards a plurality of perspectives for nurse

educators. Nursing Philosophy, 8(1), 49–59.

Previte, J., Pini, B., & Haslam-McKenzie, F. (2007). Q Methodology and Rural Research.

Sociologia Ruralis, 47(2), 135–147. doi:10.1111/j.1467-9523.2007.00433.x

Prieto, L. R. (1995). Supervising graduate teaching assistants: An adaptation of the Integrated

Developmental Model. Journal of Graduate Teaching Assistant Development, 2(3), 93–

105.

Prieto, L. R., & Altmaier, E. M. (1994). The relationship of prior training and previous teaching

experience to self-efficacy among graduate teaching assistants. Research in Higher

Education, 35(4), 481–497.

Prince, M. J., Felder, R. M., & Brent, R. (2007). Does faculty research improve undergraduate

teaching? An analysis of existing and potential synergies. Journal of Engineering

Education, 96(4), 283–294.

Pruitt-Logan, A. S., Gaff, J. G., & Jentoft, J. E. (2002). Preparing future faculty in the sciences

and mathematics. Council of Graduate Schools, Association of American Colleges and

196

Universities, Washington, DC. Retrieved from

http://www.preparing-faculty.org/PFFWeb.PFF3Manual_preface.pdf

Quiles, J. A. (2009). Introduction to Q-methodology research: Theory & design. Summer

Residency, Revised readings-Q-method. Retrieved from

http://laureate.ecollege.com/ec/crs/default.learn?CourseID=3455733&CPURL=la

Quist, D. M. (2011). Seeking Further Education: A Rubric for Evaluating Organizational

Behavior PhD Programs. Retrieved from

http://digitalscholarship.unlv.edu/thesesdissertations/1158/

Ramlo, S. (2005). An application of Q methodology: Determining college faculty perspectives

and consensus regarding the creation of a school of technology. Journal of Research, 15.

Retrieved from

http://www.eeraonline.org/journal/files/v15/JRE_v15n1_Article_5_Ramlo.pdf

Ramlo, S. (2006). Student views of learning in an introductory college physics course: A study

using q methodology. Operant Subjectivity, 30(1/2), 52–63.

Ramlo, S. (2008). Determining The Various Perspectives And Consensus Within A Classroom

Using Q Methodology. In AIP Conference Proceedings (Vol. 1064, pp. 179–182).

doi:doi:10.1063/1.3021248

Ramlo, S. (2011). Determining Faculty and Student Views: Applications of Q Methodology in

Higher Education. Journal of Research in Education, 22(1). Retrieved from

http://www.eeraonline.org/journal/files/v22/JRE_v22n1_Article_5_Ramlo.pdf

Ramlo, S., & Berit, K. (2013). Determining the Various Perspectives of Caregivers of Aging

Adults With Q Methodology. The Family Journal, 21(1), 46–56.

doi:10.1177/1066480712456819

197

Ramlo, S., & McConnell, D. (2008). Perspectives of university faculty regarding faculty reading

circles: a study using Q methodology. The Journal of Faculty Development, 22(1), 24–

31.

Ramlo, S., & Newman, I. (2011). Q methodology and its position in the mixed methods

continuum. Operant Subjectivity: The International Journal of Q Methodology, 34(3),

173–192.

Ramlo, S., & Nicholas, J. B. (2009). In-service science teachers" views about learning physics

after a one-week workshop. Journal of Human Subjectivity, 7(2), 89–101.

Rampell, C. (2012, September 28). New Enrollment Drops Again in U.S. Graduate Schools. The

New York Times. Retrieved from http://www.nytimes.com/2012/09/28/business/new-

enrollment-drops-again-in-us-graduate-schools.html

Renzulli, J. S., & Smith, L. H. (1978). Learning styles inventory: A measure of student

preference for instructional techniques. Creative Learning Press.

Repak, N. (2012). Emotional Fatigue: Coping With Academic Pressure. Retrieved September 21,

2013, from http://www.gradresources.org/articles/emotional_fatigue.shtml

Rhodes, T. D. (1997). Graduate Teaching Assistant Training: Examining Issues of Socialization

and Emotionally Painful Incidents in the Classroom. North Dakota Journal of Speech &

Theatre, 10(1), 52–63.

Richardson, P. W., & Watt, H. M. G. (2006). Who chooses teaching and why? Profiling

characteristics and motivations across three Australian universities. Asia-Pacific Journal

of Teacher Education, 34(1), 27–56.

198

Risdon, A., Eccleston, C., Crombez, G., & McCracken, L. (2003). How can we learn to live with

pain? A Q-methodological analysis of the diverse understandings of acceptance of

chronic pain. Social Science & Medicine, 56(2), 375–386.

Robbins, P., & Krueger, R. (2000). Beyond bias? The promise and limits of Q method in human

geography. The Professional Geographer, 52(4), 636–648.

Rose, G. L., Rukstalis, M. R., & Schuckit, M. A. (2005). Informal mentoring between faculty

and medical students. Academic Medicine, 80(4), 344–348.

Rosen, B. C., & Bates, A. P. (1967). The Structure of Socialization In Graduate School*.

Sociological Inquiry, 37(1), 71–84.

Ross, C., & Dunphy, J. (2007). Strategies for Teaching Assistant and International Teaching

Assistant Development: Beyond Micro Teaching. John Wiley & Sons.

Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach. Sage.

Retrieved from http://books.google.com/books?

hl=en&lr=&id=8K3Ax5s7x5oC&oi=fnd&pg=PR9&dq=stakeholders+should+insist+on+

continuing+evaluation+of+the+training+programs+they+administer+or+support&ots=L

maVsAMiyJ&sig=9UEcW20NkEDfK2GzyivT_oDBvSw

Rushin, J. W., De Saix, J., Lumsden, A., Streubel, D. P., Summers, G., & Bernson, C. (1997).

Graduate Teaching Assistant Training: A Basis for Improvement of College Biology

Teaching & Faculty Development? The American biology teacher, 86–90.

Russell, J. A. (2011). Graduate Teaching-Assistant Development in College and University

Instructional Physical Activity Programs. Journal of Physical Education, Recreation &

Dance, 82(4), 22–32.

199

Schmolck, P., & Atkinson, J. (2002). PQMethod (version 2.11). Computer Software]. Available

from: http://www. qmethod. org/Tutorials/pqmethod. htm.

Schofer, E., & Meyer, J. W. (2005). The worldwide expansion of higher education in the

twentieth century. American sociological review, 70(6), 898–920.

Schuster, J. H., & Finkelstein, M. J. (2006). The American faculty: The restructuring of

academic work and careers. Johns Hopkins University Press. Retrieved from


hl=en&lr=&id=duTczcgOQzUC&oi=fnd&pg=PR7&dq=Finkelstein,+Martin.

+1999.+Academic+Careers+in+2000+and+Beyond.&ots=ci7cexHftp&sig=j4l9Rr1SZeA

1HOLCQYXxmVdnu-k

Serow, R. C. (2000). Research and teaching at a research university. Higher Education, 40(4),

449–463.

Sexton, D., Snyder, P., Wadsworth, D., Jardine, A., & Ernest, J. (1998). Applying Q

methodology to investigations of subjective judgments of early intervention

effectiveness. Topics in Early Childhood Special Education, 18(2), 95–107.

Shannon, D. M., Twale, D. J., & Moore, M. S. (1998). TA Teaching Effectiveness: The Impact

of Training and Teaching Experience. The Journal of Higher Education, 69(4), 440–466.

Shea, A. A. (2013, February 25). For Graduate Science Programs, It’s Time to Get Real. The

Chronicle of Higher Education. Retrieved from http://chronicle.com/article/Its-Time-for-

Graduate-Science/137541/

Shulman, L. S. (1986). Those Who Understand: A Conception of Teacher Knowledge. American

Educator, 10(1), 9–15,43–44.

200

Shulman, L. S. (1987). Knowledge and Teaching: Foundations of the New Reform. Harvard

Educational Review, 57(1), 1–22.

Smith, N. W. (2001). Current Systems in Psychology: History. Theory, Research and

Applications. Wadsworth, Belmont.

Smith, R. V. (2012). Graduate Research: A Guide for Students in the Sciences. University of

Washington Press.

Sohoni, S., Cho, Y. J., & French, D. (2013). A Survey to Capture Needs Assessment for

Graduate Teaching Assistant Training. Retrieved from

http://advances.asee.org/vol03/issue03/papers/aee-vol03-issue03-11.pdf

Sonnert, G. (1995). What Makes a Good Scientist?: Determinants of Peer Evaluation among

Biologists. Social Studies of Science, 25(1), 35–55. doi:10.1177/030631295025001003

Sorcinelli, M. D. (2006). New and junior faculty stress: Research and responses. New directions

for teaching and learning, 1992(50), 27–37.

Stainton Rogers, R. (1995). Q methodology. In J. A. Harre, J. A. Smith, & I. Van Longenhove

(Eds.), Rethinking Methods in Psychology. (pp. 178–193). London: Sage.

Steelman, T. A., & Maguire, L. A. (1999). Understanding participant perspectives: Q-

methodology in national forest management. Journal of Policy Analysis and

Management, 18(3), 361–388.

Stephenson, W. (1935). Correlating persons instead of tests. Journal of Personality, 4(1), 17–24.

Stephenson, W. (1952). Q-methodology and the projective techniques. Journal of Clinical

Psychology, 8(3), 219–229.

Stephenson, W. (1953). The study of behavior; Q-technique and its methodology. Retrieved from

http://psycnet.apa.org/psycinfo/1954-06810-000

201

Stephenson, W. (1968). Perspectives in psychology: XXVI. Consciousness out subjectivity in.

The Psychological Record. Retrieved from http://psycnet.apa.org/psycinfo/1969-13471-

001

Stockdale, D. L., & Wochok, Z. S. (1974). Training TA’s to Teach. Journal of College Science

Teaching. Retrieved from http://search.ebscohost.com/login.aspx?

direct=true&db=eric&AN=EJ100093&site=ehost-live

Stoner, G., & Milner, M. (2010). Embedding generic employability skills in an accounting

degree: development and impediments. Accounting Education: an international journal,

19(1-2), 123–138.

Svinicki, M. (1994). What They Don’t Know Can’t Hurt Them: The Role of Prior Knowledge in

Learning. Teaching Excellence, 5(4), 1–2.

Tanner, K., & Allen, D. (2006). Approaches to Biology Teaching and Learning: On Integrating

Pedagogical Training into the Graduate Experiences of Future Science Faculty. CBE-Life

Sciences Education, 5(1), 1–6.

Tenure and Teaching-Intensive Appointments. (2007). Retrieved January 23, 2013, from

http://www.aaup.org/report/tenure-and-teaching-intensive-appointments

The IRIS Center. (n.d.). Differentiated Instruction in Today’s Classrooms — National

Dissemination Center for Children with Disabilities. Retrieved September 21, 2013, from

http://nichcy.org/differentiated-instruction

Thornburg, N. A., Wood, F. E., & Davis, W. E. (2000). Keeping Established Teaching Assistant

Training Programs Vital: What Does It Take? Journal of Graduate Teaching Assistant

Development, 7(1), 77–83.

202

Tice, S. L., Gaff, J. G., & Pruitt-Logan, A. S. (1998). ‘Preparing Future Faculty Programs:

Beyond TA Development. The professional development of graduate teaching assistants,

275–292.

Tierney, W. G. (1997). Organizational socialization in higher education. Journal of Higher

Education, 1–16.

Tierney, W. G., & Bensimon, E. M. (1996). Promotion and tenure: Community and socialization

in academe. SUNY Press. Retrieved from http://books.google.com/books?

hl=en&lr=&id=kBy108Ie6JsC&oi=fnd&pg=PR9&dq=Tierney,+W.+G.,+

%26+Bensimon,+E.+M.+(1996).+Promotion+and+Tenure&ots=TcD6b4t-

pY&sig=5VP1giRZ-wPUTNkam1Uxmn0CB_o

Tinto, V. (1991). Toward a theory of doctoral persistence. In Paper presented at the annual

meeting of the American Educational Research Association. Chicago, IL.

Tomlinson, C. A. (2012). What Is Differentiated Instruction? Fundamentals of Gifted Education:

Considering Multiple Perspectives, 287.

Travers, P. D. (1989). Better Training for Teaching Assistants. College Teaching, 37(4), 147–49.

Trowler, P., & Kreber, C. (2009). Beyond epistemological essentialism. The university and its

disciplines: Teaching and learning within and beyond disciplinary boundaries. Retrieved

from http://books.google.com/books?

hl=en&lr=&id=vA8oCif1BmsC&oi=fnd&pg=PA181&dq=GTAs+sink+or+swim&ots=2

w-q4sqh8F&sig=M_s20d79YtaEh2VKEE5X4Cqdors

Tulane, S., & Beckert, T. E. (2011). Perceived utility and knowledge of graduate teaching

assistants. Journal of the Scholarship of Teaching & Learning, 11(4), 44–55.

203

Vale, R. (2013). Great Things Are Ahead for the ASCB. Retrieved from

https://www.ascb.org/files/1301newsletter.pdf

Valenta, A. L., & Wigger, U. (1997). Q-methodology: Definition and application in health care

informatics. Journal of the American Medical Informatics Association, 4(6), 501–510.

Van der Valk, T., & de Jong, O. (2009). Scaffolding Science Teachers in Open‐inquiry

Teaching. International Journal of Science Education, 31(6), 829–850.

doi:10.1080/09500690802287155

Van Exel, J., & de Graaf, G. (2005). Q methodology: A sneak preview. Online document.

http://www. qmethodology. net/PDF/Q-methodology. Retrieved from

http://qmethod.org/articles/vanExel.pdf

Van Maanen, J. (1976). Breaking in: Socialization to work. En: Robert Dubin (Ed.), Handbook

of work, organization and society. Chicago: Rand McNally.[Links].

Vannini, P. (2006). Dead Poets’ Society: Teaching, Publish-or-Perish, and Professors’

Experiences of Authenticity. Symbolic Interaction, 29(2), 235–257.

doi:10.1525/si.2006.29.2.235

Ward, K. (1998). Addressing academic culture: Service learning, organizations, and faculty

work. New Directions for Teaching and Learning, 1998(73), 73–80.

Ward, K. K. A., & Wolf-Wendel, L. (2004). Academic motherhood: Managing complex roles in

research universities. The Review of Higher Education, 27(2), 233–257.

Watt, H. M. G., & Richardson, P. W. (2008). Motivations, perceptions, and aspirations

concerning teaching as a career for different types of beginning teachers. Learning and

Instruction, 18(5), 408–428. doi:10.1016/j.learninstruc.2008.06.002

204

Watts, D. S., & Stenner, D. P. (2005). Doing Q methodology: theory, method and interpretation.

Qualitative Research in Psychology, 2(1), 67–91.

Webler, T., Danielson, S., & Tuler, S. (2009). Using Q method to reveal social perspectives in

environmental research. Greenfield, MA: Social and Environmental Research Institute.

Retrieved from http://www.fairnessdiscourse.org/pdf/Webler%20-%20Using%20Q

%20Method%20to%20Reveal%20Social%20Perspectives.pdf

Weidman, J. C., & Stein, E. L. (2003). Socialization of doctoral students to academic norms.

Research in Higher Education, 44(6), 641–656.

Weidman, J. C., Twale, D. J., & Stein, E. L. (2001). Socialization of Graduate and Professional

Students in Higher Education: A Perilous Passage? ASHE-ERIC Higher Education

Report, Volume 28, Number 3. Jossey-Bass Higher and Adult Education Series. ERIC.


Weisbuch, R. (2004). Toward a responsive Ph. D.: New partnerships, paradigms, practices, and

people. Paths to the professoriate: strategies for enriching the preparation of future

faculty. San Francisco, CA: Jossey-Bass.

White, J., Nonnamaker, J., & Smith, C. A. (2008). Belonging and mattering: how science

doctoral students experience community. NASPA Journal, 45(3), 350–372.

Williams, D. E., & Roach, K. D. (1992). Graduate teaching assistant perceptions of training

programs. Communication Research Reports, 9(2), 183–192.

doi:10.1080/08824099209359910

Willig, C., & Stainton-Rogers, W. (2007). The SAGE handbook of qualitative research in

psychology. Sage. Retrieved from http://books.google.com/books?

205

hl=en&lr=&id=Tu3IqexENBEC&oi=fnd&pg=PR7&dq=willig+and+stainton&ots=3iqLj

2o_yT&sig=gpdrTDMXvO39b9VqQgnGoXAEam4

Witkin, B. R. (1995). Planning and Conducting Needs Assessments: A Practical Guide. SAGE.

Wong, W., Eiser, A. R., Mrtek, R. G., & Heckerling, P. S. (2004). By-person factor analysis in

clinical ethical decision making: Q methodology in end-of-life care decisions. The

American Journal of Bioethics, 4(3), W8–W22.

Worthen, T. K. (1992). The Frustrated GTA: A Qualitative Investigation Identifying the Needs

within the Graduate Teaching Assistant Experience.

York, P. (2005). A funder’s guide to evaluation: leveraging evaluation to improve nonprofit

effectiveness. Fieldstone Alliance.

Young, S., & Bippus, A. (2008). The Effectiveness of Teacher Preparation: An Assessment of a

Graduate Teaching Assistant (GTA) Training Program. Conference Papers --

International Communication Association, 1–26. doi:Article

206

APPENDICES

207

Appendix 1: Concourse Development

Concourse Statement

SourceConcourse

theme

Used in Q

Sample?Q Sample

Q Sample theme

I feel overwhelmed with work my

advisor gives me

SRQ Advisor 1/3 yes

I feel overwhelmed with work my advisor gives

me

Advisor 1/1

I feel like my advisor will help me learn to teach

better

SRQ Advisor 2/3 no

If I didn't know what to do in a lab, I feel like I

know who to ask for help

SRQ Advisor 3/3 no

I feel like an outsider, and that

people at grad school won't

accept me

PGSS Anxiety 1/10 yes

I feel like an outsider, and that people at grad school won't accept

me

Anxiety 1/4

I have lost sleep because I'm

worried about teaching

(Tierney & Bensimon,

1996)Anxiety 2/10 yes

I have lost sleep because I'm worried

about teaching

Anxiety 2/4

I've had family problems

because of the pressures of

graduate school

PGSS Anxiety 3/10 yes

I've had family problems

because of the pressures of

graduate school

Anxiety 3/4

Using social media (like Twitter or

Facebook) helps me to feel like I'm not alone

(Bates Holland, 2008)

Anxiety 4/10 yes

Using social media (like Twitter or Facebook)

helps me to feel like I'm not

alone

Anxiety 4/4

I am so worried about taking tests

in school that I feel sick

(Darling & Staton, 1989)

Anxiety 5/10 no

208

The amount of debt I have, because of

school, makes me upset

(Golde & Dore, 2001)

Anxiety 6/10 no

My students nitpick about every single

point I take off on assignments

SRQ Anxiety 7/10 no

I will never be able to learn

every student's name

SRQ Anxiety 8/10 no

Several times, I've felt like I'm going to have a

nervous breakdown

SRQ Anxiety 9/10 no

Everyone in grad school is really stressed out and

unfriendly

SRQAnxiety 10/10

no

I can balance being a good teacher with being a good

student

SRQ Balance 1/4 yes

I can balance being a good teacher with being a good

student

Balance 1/2

I dislike teaching, and wish I could

spend more time on my research.

SRQ Balance 2/4 yes

I dislike teaching, and wish I could spend more time on my research.

Balance 1/2

I want to go to social activities on campus, but I don't have time

(Boyle & Boice, 1998)

Balance 3/4 no

I believe I'm a very competent

researcherPGSS Balance 4/4 no

Being a TA will help me to be a good professor

someday

SRQ Career 1/2 yes Being a TA will help me to

be a good professor

Career 1/2

209

someday

Sometimes I worry that I might have

chosen the wrong career path


Career 2/2 yes

Sometimes I worry that I might have chosen the

wrong career path

Career 2/2

I feel like my fellow TAs will help me to teach

better

SRQCollaboration

1/1yes

I feel like my fellow TAs will

help me to teach better

Collaboration 1/1

Being a TA helps me ask better

questions

(Feldon et al., 2011)

Confidence 1/10

yes

Being a TA helps me ask

better questions in my research

Confidence 1/7

I feel like it will be easy to

manage my class

(Luo, Bellows, & Grady,

2000)

Confidence 2/10

yes

I feel like it will be easy to

manage my class

Confidence 2/7

I feel like students look at me weird when I tell them I'm a

TA, like I'm not good enough to

be teaching at the university.

SRQConfidence

3/10yes

I feel like students look at me weird when I tell them I'm a TA, like I'm not good enough to be teaching at the university.

Confidence 3/7

I feel pretty comfortable

using technology in my class

(Marincovich, Prostko, &

Stout, 1998)

Confidence 4/10

yes

I feel pretty comfortable

using technology in

my class

Confidence 4/7

I feel pretty confident that I'm

a good teacher


(Prieto & Altmaier,

1994)

Confidence 5/10

yes

I feel pretty confident that

I'm a good teacher

Confidence 5/7

I feel self-confident when I

teach

(Prieto & Altmaier,

1994)

Confidence 6/10

yesI feel self-

confident when I teach

Confidence 6/7

I have a lot of anxiety about

teaching, because I don't know

what to expect

SRQ Confidence 7/10

yes I have a lot of anxiety about

teaching, because I don't know what to

Confidence 7/7

210

expect

My students think I'm

interestingSRQ

Confidence 8/10

no

I feel confident I will be a good

teacherSRQ

Confidence 9/10

no

I am apprehensive

about teaching Biology

SRQConfidence

10/10no

I have no idea what the level of understanding is

with these students

SRQ Diversity 1/4 yes

I have no idea what the level

of understanding is with these

students

Diversity 1/2

I'm worried that the students

won't be able to understand me

SRQ Diversity 2/4 yes

I'm worried that the students

won't be able to understand me

Diversity 2/2

I think all of my students will be on pretty much the same level

(Nyquist & et al., 1991)

Diversity 3/4 no

My students come from

various backgrounds

(Nyquist & et al., 1991; Loreto R. Prieto &

Meyers, 2001)

Diversity 4/4 no

I feel like I need to constantly monitor my students for

cheating

SRQ Effort 1/4 yes

I feel like I need to

constantly monitor my students for

cheating

Effort 1/4

I have to repeat myself over and over to get these

students to understand me

SRQ Effort 2/4 yes

I have to repeat myself over

and over to get these students to understand

me

Effort 2/4

If I teach well, I will get good

(Marsh, 1984; Wachtel, 1998)

Effort 3/4 yesIf I teach well, I

will get good Effort 3/4

211

student evaluations

student evaluations

Most students will do just

enough to get bySRQ Effort 4/4 yes

Most of my students will do just enough to

get by

Effort 4/4

I think one of the most important

things about being a TA is being ethical

(Branstetter & Handelsman,

2000)Ethical 1/2 yes

I think one of the most important

things about being a TA is being ethical

Ethical 1/1

I would like to get to know my students outside

of class

(Cotten & Wilson, 2006)

Ethical 2/2 no

My students will respect me

because I'm fair

(Burrowes, 2003) personal correspondance

Fair 1/2 yesMy students

will respect me because I'm fair

Fair 1/1

I will try to treat all my students

the same, because that is

what's fair.

SRQ Fair 2/2 no

I worry that certain students

in my class might know more about Biology than I do

(Rushin et al., 1997)

Intelligence 1/2

yes

I worry that certain students

in my class might know more about

Biology than I do

Intelligence 1/1

I'm afraid that my students will

think I'm not smart

SRQIntelligence

2/2no

All my students are capable of understanding

Biology

SRQlearning

styles 1/11yes

All my students are capable of understanding

Biology

Learning styles 1/5

I learned best by actively doing

labs(Kugel, 1993)

learning styles 2/11

yesI learned best by actively doing labs

Learning styles 2/5

I learned best by listening to

(Kugel, 1993)learning

styles 3/11yes

I learned best by listening to

Learning styles 3/5

212

professors teachprofessors

teachI think most of

my students learn in a way that's similar to the way I learn

(Golish, 1999)learning

styles 4/11yes

I think most of my students

learn in a way that's similar to the way I learn

Learning styles 4/5

My students will like Biology because I can

make it interesting

SRQlearning

styles 5/11yes

My students will like Biology

because I can make it

interesting

Learning styles 5/5

I want my students to do higher order

thinking

(Burrowes, 2003)


no

I know how to get groups to work together

(Knight & Wood, 2005)


no

Most of my students in my

lab learn just like me

(Luft et al., 2004)


no

I will be friendly to my students

while I share my knowledge with

them

(Luft et al., 2004)


no

I want my students to be

able to memorize facts about a

subject

PClearning

styles 10/11no

My students will appreciate

science when they are done

with this course

SRQlearning

styles 11/11no

Being a TA has helped me to afford grad

school

(Girves & Wemmerus,

1988)Practical 1/2 yes

Being a TA has helped me to afford grad

school

Practical 1/1

I think I could create a syllabus

(Davis, 2009) Practical 2/2 no

213

for a course I might teach

Being a good teacher is as important as being a good

researcher

(Hattie & Marsh, 1996; Nyquist et al.,

1999)

Preparation 1/8

yes

Being a good teacher is as important as being a good

researcher

Preparation 1/4

I believe I know what it takes to

be a good researcher

(Nyquist & Woodford,

2000)

Preparation 2/8

yes

I believe I know what it takes to be a

good researcher

Preparation 2/4

I know what the Biology

department expects from my

teaching

SRQPreparation

3/8yes

I know what the Biology department

expects from my teaching

Preparation 3/4

I know what the department

expects from my research


2000)

Preparation 4/8

yes

I know what the department expects from my research

Preparation 4/4

I know what the department

expects from grad students


2000)

Preparation 5/8

no

I feel like I'm prepared to

handle challenging

students

(Young & Bippus, 2008)

Preparation 6/8

no

How am I supposed to help

my students when I don't even

know what to do?

PCPreparation

7/8no

My to-do lists are a mile long

PCPreparation

8/8no

I came to grad school mainly so

I could do research


2000)Research 1/8 yes

I came to grad school mainly so I could do

research

Research 1/6

I know the university

policies that

(Luft, Kurdziel,

Roehrig, & Research 2/8 yes


policies that Research 2/6

214

relate to my research

Turner, 2004)relate to my

research

I like doing research over

teaching

(Colbeck, 1998;

Levinson-Rose & Menges,

1981)

Research 3/8 yesI like doing

research over teaching

Research 3/6

I like doing teaching over

research

(Colbeck, 1998;

Levinson-Rose & Menges,

1981)

Research 4/8 yesI like doing

teaching over research

Research 4/6

I think all this teaching gets in the way of my

research


Research 5/8 yes

I think all this teaching gets in the way of my

research

Research 5/6

I think research is very

challengingSRQ Research 6/8 yes

I think research is very

challengingResearch 6/6

My research is making an important

contribution to science

(Marsh, 1980) Research 7/8 no

I think my students would

find my research to be completely

boring

SRQ Research 8/8 no

I am good at creating a respectful classroom

environment

(Golish, 1999) Respect 1/3 yes

I am good at creating a respectful classroom

environment

Respect 1/3

I feel like I'm a good teacher because I am

closer in age to my students

SRQ Respect 2/3 yes

I feel like I'm a good teacher because I am

closer in age to my students

Respect 2/3

I have no idea what students

think about me, and that makes

me uncomfortable

(Rubin & Smith, 1990)

Respect 3/3 yes

I have no idea what students

think about me, and that makes

me uncomfortable

Respect 3/3

215

I don't think teaching requires a lot of emotion

SRQTeaching

1/16yes

I don't think teaching

requires a lot of emotion

Teaching 1/9

I feel like I could go into teaching as a profession

SRQTeaching

2/16yes

I feel like I could go into teaching as a profession

Teaching 2/9

I know what attributes make a

good teacherSRQ

Teaching 3/16

yesI know what

attributes make a good teacher

Teaching 3/9

I think some people are

natural teachersSRQ

Teaching 4/16

yesI think some people are

natural teachersTeaching 4/9

I think teaching is very

challengingSRQ

Teaching 5/16

yesI think teaching

is very challenging

Teaching 5/9

I think that I give good teaching presentations

SRQTeaching

6/16yes

I think that I give good teaching

presentations

Teaching 6/9

I think you can be "taught to

teach"(Hardré, 2003)

Teaching 7/16

yesI think you can be "taught to

teach"Teaching 7/9

I want all students to

actively participate in my

class

(Smith et al., 2005)

Teaching 8/16

yes

I want all students to

actively participate in

my class

Teaching 8/9

I want to teach the same way my favorite professor

taught

SRQTeaching

9/16yes

I want to teach the same way my favorite professor

taught

Teaching 9/9

I will teach differently than I

was taught(Hardré, 2003)

Teaching 10/16

no

I know how to deal with

disruptive or inappropriate

(Luo et al., 2000)

Teaching 11/16

no

216

students

I know how to handle an in-

class discussion

(Prieto & Altmaier,

1994)

Teaching 12/17

no

I feel confident that I could

handle problems in my classroom

PGSSTeaching

13/17no


policies that relate to my

teaching

PGSSTeaching

14/16no

As I become a better teacher,

my students will receive quality

instruction

SRQTeaching

15/16no

I have developed a pretty good

general teaching philosophy

SRQTeaching

16/16no

I have had dreams about my

teaching or research

(Kerry, 2005; Nyquist et al.,

1999) (PC)Unknown 1/1 yes

I have had dreams about

my teaching or research

Unknown 1/1

Note: SRQ – Self-Reflection Questionnaire, PGS – Perceptions of Graduate School Survey, GLF – Grad Life Forum, PC – Personal Correspondence

217

Appendix 2: Q Sample

1 All my students are capable of understanding Biology

2 Being a good teacher is as important as being a good researcher

3 Being a TA helps me ask better questions in my research

4 Being a TA will help me to be a good professor someday


6 I believe I know what it takes to be a good researcher


8 I can balance being a good teacher with being a good student

9 I dislike teaching, and wish I could spend more time on my research.

10 I don't think teaching requires a lot of emotion



13 I feel like I need to constantly monitor my students for cheating

14 I feel like I'm a good teacher because I am closer in age to my students

15 I feel like it will be easy to manage my class

16 I feel like my fellow TAs will help me to teach better

17I feel like students look at me weird when I tell them I'm a TA, like I'm not good enough to be teaching at the university.

18 I feel overwhelmed with work my advisor gives me

19 I feel pretty comfortable using technology in my class

20 I feel pretty confident that I'm a good teacher

21 I feel self-confident when I teach

218


23 I have had dreams about my teaching or research

24 I have lost sleep because I'm worried about teaching



27 I have to repeat myself over and over to get these students to understand me

28 I know the university policies that relate to my research

29 I know what attributes make a good teacher

30 I know what the Biology department expects from my teaching


32 I learned best by actively doing labs

33 I learned best by listening to professors teach

34 I like doing research over teaching

35 I like doing teaching over research


37 I think most of my students learn in a way that's similar to the way I learn

38 I think one of the most important things about being a TA is being ethical

39 I think research is very challenging

40 I think some people are natural teachers

41 I think teaching is very challenging

42 I think that I give good teaching presentations

43 I think you can be "taught to teach"

219

44 I want all students to actively participate in my class



47 If I teach well, I will get good student evaluations

48 I'm worried that the students won't be able to understand me

49 I've had family problems because of the pressures of graduate school

50 Most of my students will do just enough to get by

51 My students will like Biology because I can make it interesting

52 My students will respect me because I'm fair

53 Sometimes I worry that I might have chosen the wrong career path

54 Using social media (like Twitter or Facebook) helps me to feel like I'm not alone

220

Appendix 3: Conditions of Instruction

TA Perceptions of Graduate School Q Sort

Thank you for taking time to help us better understand the views related to this topic. Please follow the instructions below. If you have a question, just ask!

Instructions 1. Please read and consider each statement in the envelope carefully as it relates to your view of graduate school and as a Biology Teaching Assistant.. 2. Remove the pieces of paper from the attached envelope. Each of the 54 pieces of paper contains one statement. 3. Read each statement and then, based upon your views of graduate school place each statement into one of three piles while attempting to make these piles of EQUAL size (about 18 statements in each pile) HERE:

MOST UNlike my view

(~18 statements here)

Neutral view about this statement

(~18 statements

here)

Most like my view(~18 statements here)

4. Now take the MOST LIKE pile and distribute it on the distribution sheet (last page - BLUE) putting your top FOUR (4) MOST LIKE statements in position +5, and then working toward the 0-column. Repeat with the UNLIKE pile and finally with the NEUTRAL pile. 5. You may move your statements around until you are satisfied with their placement. 6. Each square on the grid should have only one statement number; each number is only used once. 7. Write the statement-number in their appropriate location on the grid on the next page (PAGE 2). 8. Finally, answer the questions underneath the grid.

*********************************************************************************

221

Please answer the following questions:

Are you in a Master’s or Doctoral degree program (circle)? Master’s Doctoral

Are you an international student? Yes No

What is your age, in years? ________________

What is your gender? _____________

Do you have any experience teaching (either formal or informal)? ______ If yes, in what setting, and how many semesters or how many years?

What are you planning to do after graduate school?

Briefly (a few sentences) describe what you would like to get out of a TA training program:

222

TA Perceptions of Graduate School Q Sort

Each box must contain ONE (1) statement number; each number must be used only once.

Sort based on your view of being a Biology TA in graduate school

4 4 5 5 6 6 6 5 5 4 4

Most

unlike

my

views neutral

Most

like

my

views

-5 -4 -3 -2 -1 0 +1 +2 +3 +4 +5

Please answer the following questions regarding your sort:

Tell us why you selected the four statements you placed under +5 (most like my view)?

Tell us why you selected the four statements you placed under -5 (most unlike my view)?

Please describe your decision-making process during the sort. Did you gain insight about your views as you sorted the statements? If so, please describe. You may continue writing on the back of this sheet.

223

Appendix 4: IRB Informed Consent Letter

Amy HollingsworthNatural Science Biology Lab CoordinatorThe University of Akron302 Buchtel CommonsAkron, OH 44312

You are invited to participate in a study being conducted by Amy Hollingsworth, the Natural Science Biology Lab Coordinator and a doctoral student in the Department of Curricular and Instructional Studies at The University of Akron and Susan E. Ramlo, PhD, STEM Initiatives and The College of Education at The University of Akron. The project investigates the views of graduate biology teaching assistants on teaching, learning, students, and research. The project includes a Q Sort which you will be asked to conduct. Analysis of views will be done using the Q Methodology analysis technique.

Should you agree to participate you can expect the time to perform the Q Sort to be about 15-30 minutes. You may be contacted after the Q Sort has been completed if further clarification about your sort is needed.

The Q Sort will be conducted during the “Effective Teaching” class in the Fall semester of 2012. If you agree to participate, you may refuse to answer any questions and may withdraw from the study at any time.

Completion of the Q Sort will serve as your consent to participate in this study. You may keep this form for your records. Participants’ answers on the questionnaires will be recorded, without any identifying information, and used only by the researchers for organizational and analytical purposes. Your confidentiality will be protected throughout the study. Any data obtained from you (Q Sort) will be kept confidential and will not be viewed by anyone but the researchers. All identifying information will be retained in a locked cabinet or other locked storage area. The data will be kept for no more than two (2) years and will be destroyed upon completion of the project. There are no anticipated benefits or risks to you as a participant.

If you have any questions about the research project, you can call either Dr. Ramlo at 330-972-7057 ([email protected]) or Amy Hollingsworth at 330-972-5268 ([email protected]). This research project has been reviewed and approved by The University of Akron Institutional Review Board for the Protection of Human Subjects. Questions about your rights as a research participant can be directed to Ms. Sharon McWhorter, Associate Director, Research Services, at 1-330-972-7666.

Thank you for your participation!

224

Sincerely,

Amy HollingsworthNatural Science Biology Lab CoordinatorThe University of Akron

225

Appendix 5: IRB Exemption Request

226

Appendix 6: IRB Exemption

1a. Provide a brief description of the purpose of the proposed project and the procedures to be used. What will research subjects be asked to do? How long will it take?

The “Teaching Assistant Perceptions of Graduate School Q Survey” will be used to survey the viewpoints of current Teaching Assistants in the Biology Department as to their perceptions of teaching, students, research, and graduate school life. The participants will rank order 54 statements into a grid for least like their viewpoints, to most like their viewpoints. The survey will take between approximately 30 minutes to complete.

1b. Provide the process by which individuals will be recruited. Describe any qualifying characteristics of the subject population such as gender, age ranges, ethnic background and health status. Indicate any special classes of subjects that might be included in the study population (e.g., socially or economically disadvantaged, minors, mentally disabled.) Estimate the number of subjects to be recruited.

The Q Survey will take place during the Biology TAs’ “Effective Teaching” class. Most teaching assistants are between the ages of 22 and 25. There are no special classes included in the study population. There are 11 graduate biology students enrolled in the “Effective Teaching” course.

1c. Where will data collection take place (e.g. university, outside agency, school district, hospital, etc) and who will collect the data? Attach letter(s) of authorization to perform the research from all off-campus sites.

Data collection will take place in the classroom, which is located in the Auburrn Science and Engineering Building at The University of Akron. The data will be collected by the researcher, and results will be stored in the researcher’s secure and locked office.

1d. Describe any potential risks - physical, psychological, economic, social, legal or other. Indicate how you will eliminate or reduce any potential risks to subjects. Only minimum risk research is eligible for exemption.

There is no potential risk to the participants. The Q Survey asks Teaching Assistants to reflect on their viewpoints of the Biology Laboratory and Teaching Assistant experience, which will be done anonymously. The data will not be reported individually, but will be summarized for each of the sections. TAs may discontinue their participation at any time. There is little likelihood that a 30-minute survey done in class would trigger psychological stress.

1e. Describe any potential benefits of the research to subjects or to society.

230

There are no known benefits to completion of the survey. Survey results will guide the creation of future teaching assistant courses in the Biology Department.

1f. Explain how individual privacy will be protected. For example, if interviewing, where will that be conducted?

Q Surveys will be completed individually. They will be asked to reflect on their sorting experience anonymously.

1g. Explain how individual confidentiality or anonymity will be protected. What kind of information will be recorded and how will it be protected? Who will have access to the data and where will it be kept? Will any identifying information be included in publications or presentations of the research?

There is limited demographic information on the survey, so individual participants cannot be identified. The instrument will be administered anonymously. There will not be enough identifying information on the instrument to trace answers back to participants. The primary and co-investigator will be the only individuals who have access to the data. The data files will be shared electronically between Dr. Susan Ramlo and Ms. Hollingsworth. Both Dr. Ramlo and Ms. Hollingsworth have computers that are password protected. No identifying information will be included in any publication or presentation.

Questionnaires will be coded in order to match them with the Q Sorts. Only these codes will be used in the recording of data in an PQ Method file. Only the PI and CI will have access to the original data and it will remain secured in a locked filing cabinet. Once the data for a participant is complete and recorded, the original documents will be shredded.

No identifying information will be included in any publication or presentation of the data.

1h. Describe your consent procedures. Provide justification if you do not plan to collect a signed consent from each participant. (Provide a copy of the consent form or information sheet you will provide to participants.)

We seek a waiver of signed informed consent for the following reason:

“That the research presents no more than minimal risk of harm to subjects and involves no procedures for which written consent is normally required outside of the research context.”

Please see attached informed consent instructions.

231

Q METHODOLOGY AS A NEEDS ASSESSMENT TOOL FOR BIOLOGY GRADUATE TEACHING ASSISTANTS PARTICIPATING IN...

Education

Transcript of Q METHODOLOGY AS A NEEDS ASSESSMENT TOOL FOR BIOLOGY GRADUATE TEACHING ASSISTANTS PARTICIPATING IN...