Increasing Student Interest and Response Rates

Will Miller, Ph.D.Director of Institutional

Research and Effectiveness

Flagler College• St. Augustine, FL• Approximately 2,500

students• Undergraduate only• Liberal arts tradition• Non-religious• Non-tenure system• Using IDEA since at

least 2004

Old Approach to Evaluations

• Prior to Fall 2014– Veteran faculty• Select two courses (one upper/one lower)• Paper/pencil administration• One attempt—controlled by faculty

–New and adjunct faculty• All courses evaluated• Paper/pencil administration• One attempt—controlled by faculty

Results with the Old Approach

• Received a high response rate given that students were a captured audience– However, there were significant administrative

costs– Concerns about faculty role in administration– Concerns about thoughtfulness of students

• Especially with written comments• Additionally, even with the high response

rate, we were only evaluating 50% of courses chosen by faculty

Approach to Evaluations• Beginning in Fall 2014– All faculty• All courses evaluated• Online administration through Campus Labs• Multiple approaches

Some Faculty Respond• “Students will

never take the survey online.”

• “Not every student has the capability to complete this online.”

• “They’ll all sit together and talk negatively about me.”

The Response Rate Issue• Biggest concern going online• Campus-wide conversation on how

to assure response rates are where we want them to be ( > 70%)

The Punitive Approach• Student can’t access grades

– Or at least not as quickly• Student can’t register for next term• Student can’t graduate

– The Bachelor of Arts degree is awarded to students who meet the following conditions:

• Faculty revolt

Why We Went Another Direction

• Student Autonomy– From the moment freshmen arrive, we

preach student autonomy– Thus, it is our job to demonstrate to

them WHY they should complete course evaluations

Results

Results• Faculty satisfaction (and, in some

cases, lack thereof)– Comments– Response rates– Less loss of class time– Customization options

Method• Faculty are told to set aside a day

and time to still complete the evaluations in class, if they wish, along with talking points– Loaner tablets available if they know

students will need them• Significant effort spent designing a

communication plan to reach ALL students

Communication Plan• Pre-administration social media and

campus media blasts

Communication Plan• Initial e-mail at beginning of

administration– Serious tone– Discusses importance of course

evaluations and student responsibility in completing them

– KEY: reminding them to follow faculty instruction. We leave administrations open for 2-3 weeks and allow faculty to provide guidance on timing when relevant.

Sample Language• These results are used by the institution to

improve teaching and assure students are receiving the best instruction possible. Each semester IDEA results are looked at by faculty, department chairs, and administrators. Your honest feedback helps us in our efforts to continually improve instruction. Please take these evaluations seriously as they are our only opportunity to measure your satisfaction or dissatisfaction with your courses this semester.

Communication Plan• Follow-up e-mails– Less serious in tone– Have faculty provide short case studies on

what they learn from IDEA results and how they’ve made positive changes (attempt to use faculty from their majors)

– Keep them updated on response rate

Communication Plan• Follow-up e-mails– Sample language

• I know you’re tired of hearing from my office about these, but they really are important. It will take less than 5 minutes per course to complete and will help us do a better job teaching you effectively. We’ve extended the deadline till December 7 for you to complete these and really hope you will take the time to complete them. However, the sooner you complete them, the sooner we can stop reminding you.

• Yes, it’s us again. Looks like you still haven't completed course evaluations for the following courses: ECO 331 (A): International Economics We won't notify you anymore about this. Just reminding you that your ability to evaluate these courses will close on Sunday night at midnight. Log in now (even from a mobile device) to do your part.

Communication Plan

Communication Plan

Communication Plan

Communication Plan

Tricks• Redirect to a different URL–www.flagler.edu/courseevaluations• Easier for faculty and students to remember

• Having students help in crafting messaging

• Appropriate seriousness in each message

• Subject lines of e-mails!– Pretty please! (responses and

comments)

Conclusion• It IS possible to have high response

rates without taking draconian control over elements of student lives– Faculty will be happier– Students will be happier

Questions and Discussion