Post on 13-Jul-2015
How to use Focus Groups to Win Friends and Influence PeopleCAIR, San Diego, CA.
November 19-21, 2014
Scheduled at 3:00-3:45 Thursday in Salon A
Nathan Garrett, PhD
Assistant Professor of IT
Assistant Dean, School of Business
Nathan.Garrett@Woodbury.edu
Bruce Feinstein, MOL
Institutional Researcher
Woodbury University
Bruce.Feinstein@Woodbury.edu
Abstract
More than just getting input, we used focus groups to build a policy, grow support, and fast-track implementation. For our small tuition-dependent university, admissions standards are a politically charged topic. A small team began by creating data-driven admission criteria. We then used focus groups to engage with stakeholders campus-wide. Each focus group began with a brief presentation, and then quickly transitioned into a guided dialogue. The meetings allowed the team to find and address points of opposition. By the time the proposal went to the full faculty body and administration, most of the potential critics had already been engaged. This session will present the focus group methodology used to build consensus.
Woodbury University 211/20/2014
Slides online at profgarrett.com
Woodbury University 311/20/2014
Why are we here?
Timeline
Summer ‘13: Began Process
Fall ‘13: Approved Faculty
Spring ‘14: Approved Cabinet/President
Spring ’15: Implementation
2.75 GPA Admission Policy Proposal
Domestic freshmen entrants need a minimum High
School GPA of 2.75.
SAT ignored, and Scholarships based on HS GPA.
Woodbury University 411/20/2014
Questions welcomed at any time
11/20/2014 Woodbury University 5
Outcomes
1. Understand our focus group methodology
2. Recreate useful data displays
3. Describe ways to overcome resistance with data
Agenda
1. Methodology/Process
2. Concerns & Questions
Methodology/Process
Woodbury University 611/20/2014
What is a focus group?
• Guided discussion
• Understand new or complex issues
• Originally used in marketing research
Woodbury University 711/20/2014
May-August ’13
August-September
October-February
Process
Woodbury University 8
Developed
Policy
Interviews
Focus
groups
Data
Gathering
11/20/2014
Use a traditional focus group to generate quick, rich, and slightly-biased data
Advantages
• Low cost & fast
• Messy or unstructured problems
• Authentic participant voices
Disadvantages
• Groupthink/dominant voices
• Non-quantifiable/statistical
• Biased selection
• Poor for sensitive topics
• Can be shallower than interviews
Woodbury University 911/20/2014
Use a traditional focus group to generate quick, rich, and slightly-biased data and agreement
Advantages
• Low cost & fast
• Messy or unstructured problems
• Authentic participant voices
Disadvantages
• Groupthink/dominant voices
• Non-quantifiable/statistical
• Biased selection
• Poor for sensitive topics
• Can be shallower than interviews
Woodbury University 1011/20/2014
Use a traditional focus group to generate quick, rich, and slightly-biased data and agreement
Advantages
• Low cost & fast
• Messy or unstructured problems
• Authentic participant voices
• Groupthink
• Non-quantifiable/statistical
Disadvantages
• Groupthink/dominant voices
• Non-quantifiable/statistical
• Biased selection
• Poor for sensitive topics
• Can be shallower than interviews
Woodbury University 1111/20/2014
Ideal focus group elements
• Location• Neutral
• Round table
• Participants• 6-10 people
• Sample• Avoid power differentials,
• Homogeneity for comfortable discussions.
• Diverse to avoid groupthink
• Confidential
• Sessions• Continue until saturation
• Moderator• Starts with open-ended
questions
• Probes into responses
• Neutral
• Data• Note-taker or a/v recording
• Coded for analysis
Woodbury University 1211/20/2014
Focus Group Participants• Build initial dataset
• Bruce/Nathan
• Admissions• VP Enrollment, Dir. Enrollment
• All admissions staff
• Student Support • VP, Dean, Director
• Academic Leadership• Chairs
• Deans (one-on-one)
• Faculty/Community• Colloquium presentation• Educational Planning Committee
• Senate/WUFA
• Cabinet/President
• Cross-disciplinary meetings
Woodbury University 13
Staff, 35%
Faculty, 50%
VPs, 15%
11/20/2014
What we did wrong
Ideal
Neutral Facilitator
Gather participant voices
Getting data
Reality
Data-driven facilitator
Build proposal & consensus
Building/displaying data
Woodbury University 1411/20/2014
The Result
1 Paragraph proposal
3 Page explanation
15 Pages of Q&A
Woodbury University 1511/20/2014
Why not use the SAT?
Woodbury University 16
High School GPA SAT Verbal SAT Math
Pearson Correlati
on
Sig. (2-tailed)
N Pearson Correlati
on
Sig. (2-tailed)
N Pearson Correlati
on
Sig. (2-tailed)
N
Is the student male? -.134 .000 680 -.035 .317 823 .112 .001 823 Is the student Hispanic?
.005 .890 680 -.144 .000 823 -.136 .000 823
Is the student in a studio course?
.052 .184 659 .079 .026 793 .145 .000 793
Has the student graduated?
.213 .000 680 .043 .220 823 .060 .086 823
High School GPA .278 .000 577 .330 .000 577
SAT Verbal .278 .000 577 .513 .000 823
SAT Math .330 .000 577 .513 .000 823 Fall Freshman Year GPA
.493 .000 662 .244 .000 800 .198 .000 800
Spring Freshman Year GPA
.489 .000 604 .267 .000 733 .161 .000 733
11/20/2014
High School GPA SAT Verbal SAT Math
Pearson Correlati
on
Sig. (2-tailed)
N Pearson Correlati
on
Sig. (2-tailed)
N Pearson Correlati
on
Sig. (2-tailed)
N
Is the student male? -.134 .000 680 -.035 .317 823 .112 .001 823 Is the student Hispanic?
.005 .890 680 -.144 .000 823 -.136 .000 823
Is the student in a studio course?
.052 .184 659 .079 .026 793 .145 .000 793
Has the student graduated?
.213 .000 680 .043 .220 823 .060 .086 823
High School GPA .278 .000 577 .330 .000 577
SAT Verbal .278 .000 577 .513 .000 823
SAT Math .330 .000 577 .513 .000 823 Fall Freshman Year GPA
.493 .000 662 .244 .000 800 .198 .000 800
Spring Freshman Year GPA
.489 .000 604 .267 .000 733 .161 .000 733
Concerns and Questions
Woodbury University 1711/20/2014
Did we make the right choice, is this a good fit?
11/20/2014 Woodbury University 18
How will this affect enrollment?
11/20/2014 Woodbury University 19
YearFreshman class who
would be excluded
2004
2005
2006
2007
2008
2009
2010
2011
2012
Average
Woodbury is approx. 2/3
transfer, 1/3 freshmen.
Is there a difference between studio and non-studio programs?
11/20/2014 Woodbury University 20
How will this change affect my program?
11/20/2014 Woodbury University 21
College ProgramWould have been
admitted
Would have
been
denied
Would have
been
denied %
Unknown due
to lack of HS
GPA
Total
Architecture Architecture
Interior Architecture
Overall
Business Accounting
Fashion Marketing
Management
Marketing
Overall
ITS
MCD Animation Arts
Communication
Fashion Design
Filmmaking
Game Art and Design
Graphic Design
Psychology
Overall
(blank)
How would this change affect our student population?
• % Pell recipients among Domestic Freshmen
• Average high school GPA
• average SAT composite
• Demographics:• Asian –
• Black or African American –
• Hispanic –
• Native Hawaiian –
• White –
• Two or more –
• International –
• Unknown –
11/20/2014 Woodbury University 22
How would this change affect our student population –HSI status for example?
11/20/2014 Woodbury University 23
Gender Ethnicity
Would
have
been
admitted
Would
have
been
denied
Would
have been
denied %
Unknown
due to
lack of HS
GPA
Total
Female
Overall
Asian or Pacific Islander
Black, non-Hispanic
Hispanic
White, non-Hispanic
Male
Overall
Asian or Pacific Islander
Black, non-Hispanic
Hispanic
White, non-Hispanic
Overall
How does Woodbury compare nationally?
11/20/2014 Woodbury University 24
Woodbury HERI
Range n Grad? n Grad?
No SAT
600-699
2,688 39.8%
700-799
800-899 6,309 45.2%
900-999 9,583 52.3%
1000-1099 9,150 63.2%
1100-1199 8,707 68.0%
1200-1299 6,772 73.1%
1300+ 5,685 76.5%
Avg
Graduation rates by SAT
Source: http://www.heri.ucla.edu/PDFs/pubs/TFS/Special/Monographs/Degree AttainmentRates
AtAmericanCollegesAndUniversities.pdf
Woodbury HERI
Range n Grad? Range n Grad?
No GPA
2.0-2.5 <C, C-, C 1,212 20%
2.5-2.9 C+ 2,582 33%
3.0-3.4 B- 4,527 40%
B 11,434 48%
3.5-3.9 B+ 12,090 59%
4.0+A- 12,261 68%
A, A+ 12,112 78%
Graduation rates by GPA
Why not offer more assistance to these students?
• Remedial math courses
• Remedial writing courses
• Writing Center
• Tutors
• Peer mentors
• Counselors
11/20/2014 Woodbury University 25
Why not offer more assistance to these students?
11/20/2014 Woodbury University 26
HS GPAPopulation
of students
Number
of
students
sampled
Average
number of
years each
student was
at WU
No
enrollment
after
Woodbury?
Reverse
Transfer to
Community
College?
Went to a
different 4
year school
after WU &
CC?
Came back
to WU after a
CC?
< 2.5
2.5 - 2.75
Total
Source: National Student Clearinghouse.
11/20/2014 Woodbury University 27
Did we make the right choice,
is this a good fit?
•For the Student
•For the University
Conclusions & Take-aways
1. Focus groups are a social/qualitative methodology
2. We used the methodology to:1. Create useful data displays
2. Discover and address peoples’ issues
3. Effective in our context
11/20/2014 Woodbury University 28