Power up your Program Evaluations Marketing, Development, & Assessment.
-
Upload
trinity-stamp -
Category
Documents
-
view
222 -
download
1
Transcript of Power up your Program Evaluations Marketing, Development, & Assessment.
![Page 1: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/1.jpg)
Power up your Program EvaluationsMarketing, Development, & Assessment
![Page 2: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/2.jpg)
Evaluation Form (paper or online) Responses reviewed by SAO Responses shared with faculty If paper, some reviews placed in the
resource center for prospective students to review
If online (Zoomerang, SurveyMonkey, etc.) reviews may be downloaded & stored.
Other actions taken on a case by case basis.
Traditional Approach
![Page 3: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/3.jpg)
Definition: A systematic exploration to discover and judge something that we think has value.
Evaluation is “Ex Valore” in Latin, which means to “strengthen” or “empower” Ex – “out of, from” Valor – “strength, value, valor”
It was originally a “means” rather than an “end”
A means to “empowerment” synonymous to positive growth and development
A means that involves people, processes, and results associated with what we do
What is evaluation?
![Page 4: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/4.jpg)
Practices - Offices that manage the process, third-party subcontracts, host institution services, information sessions, student advising, pre-departure, policies, procedures, etc.
Programs - Direct enroll, faculty-led, third-party, consortium, student teaching, internships, etc. (their administration, academics, housing, social, expenses, etc.)
Projects - Freshmen orientation, study abroad fair, marketing workshop for faculty, study abroad ambassadors project, welcome back reception, and other special events.
to… Prove (show) something is working or needed Improve the practice, program, or project Discover something new for development
We can evaluate…
![Page 5: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/5.jpg)
Coordinate evaluation process for multidimensional goals Student Satisfaction Data (regarding Administration, Academics,
Housing, Social, Expenses) helps us in marketing and development Learning outcomes, intercultural development, progress towards
global citizenship guides our questions regarding assessment Shrewdly and efficiently share information with stakeholders.
Power up your evaluations…
Stakeholders Why? Purpose/Goal
Prospective Students-Parents, Alumni, potential Donors
Earn respect, convince of value-benefits-outcomes, and grow participation
Marketing
OSA, Offices, Partner Institutions, Organizations, Faculty, & Academic Departments
Monitor, grow, improve, foster goals & objectives in programs and operations.
Development
OSA, the field of study abroad, Faculty, Academic Departments, Accreditation Bodies, Upper Administration, Donors
To show that stakes are valid, gain support, lobby, grow the education abroad as a unique & important form of education.
Assessment
![Page 6: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/6.jpg)
AbroadScout 2.0Mission: to empower students, schools, and the education abroad industry with quality information and Web 2.0 tools for marketing, development, and assessment.
![Page 7: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/7.jpg)
AS 2.0 Marketing, Development, Assessment Tools
Add your programs and scholarships to the main AS directory (also edit/delete)
Create your own PROGRAM FINDER (a customized directory of the sending institution’s approved list of programs with scholarships and reviews)
Create your own COURSE FINDER (a course articulation database tied to your customized PROGRAM FINDER) - Co managed
Customize the AbroadScout Program Evaluation alongside your program partners. One survey per student works for multiple parties (home, provider, host).
Create and manage new surveys . Share surveys and results with select partner institutions, offices, etc. Great for development and assessment projects.
![Page 8: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/8.jpg)
AS2.0 Funding Model FREE – All Marketing-Development-Assessment tools. PRO ACCOUNT – Allows you to upload a logo. Cost $300/yr.
Marketing - It gives your programs more visibility, as they are randomly sorted among all pro-account programs in the first tier of any search (in the main directory and customized program finders). Also, more information is included in the initial search results.
Added Customization - In your Program Finder and Course Finder, it gives your pages a more personalized feel with the logo from your institution on top.
ADVERTISING POLICY –To keep information unbiased, and avoid conflicts of interest, the directories (programs, scholarships, and reviews) are completely free of advertising, aside from the pro-account option we offer to all program sponsors equally.
FUNDING MODEL – We fund the site through pro-accounts in the directories and banners-links in continent/country sections of the website. These sections contain articles and news, relevant to student interests, many of which are written by students.
![Page 9: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/9.jpg)
Study Abroad @Abilene Christian University
Private Master’s Level University 4000 Undergraduate and 700 Graduate 3 Permanent sites, 2 Temp sites, 8 consortium sites
Semester and Summer Programs Oxford, England Montevideo, Uruguay Leipzig, Germany
![Page 10: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/10.jpg)
Approx. 220 students abroad split evenly between semester-long and summer programs
General Education during semesters Discipline specific during summers
ACU On-site directors, ACU visiting faculty, local adjuncts and partner universities
Study Abroad @ Abilene Christian University
![Page 11: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/11.jpg)
University Mission and Vision Increase Student Learning Increase participation to reach 50% by 2020
(currently 28%) Expectation that we balance our budget
![Page 12: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/12.jpg)
Accountability to constituents Students, faculty, parents,
donors, board of trustees, home and host communities
Regional Accrediting Body – SACS
Institutional Drivers of Marketing, Development and Assessment
Abilene Christian University
![Page 13: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/13.jpg)
What, How and Why Global Perspective Inventory
Pre-departure and post completion Embedded in an orientation course
Course evaluations Paper and pencil Online
From Assessment to Marketing
![Page 14: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/14.jpg)
Online student program evaluation Yes/No and open ended-text box questions obtained
at the close of the program Reflection papers embedded in a course Faculty debriefing
From Assessment to Marketing
![Page 15: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/15.jpg)
Internal Global Perspectives Inventory (GPI) and
online program evaluation results to Program Directors, Senior University Administrators, PR/marketing and Board of Trustees (BOT)
Getting from A (Assessment) to M (Marketing)
![Page 16: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/16.jpg)
Course Evaluations – faculty, Dept. Chairs and College Deans, PR/marketing offices
Student Reflection papers –faculty, PR/marketing offices
Student numbers and demographics - Senior Administrators, BOT – sign leases, purchase property etc…
Getting from Assessment to Marketing – Internal (continued)
![Page 17: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/17.jpg)
External Quotes incorporated into
Web page articles, ads etc… Information relayed to
current and prospective students and their families via promotional and recruiting materials
Host community is provided feedback – i.e. volunteer organizations, Property/facility owners, neighbors, partner institutions
Getting from A to M (continued)
![Page 18: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/18.jpg)
Increased student learning and self-efficacy Value Added Holistic education
Increased student enrollment opportunities for growth Paying the bills
Increased funding to improveand expand services and sites
C *e*l*e*b*r*a*t*i*o*n*s
![Page 19: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/19.jpg)
Making every one happy vs. creating a culture of quality enhancement and improvement
Closing the Loop – Follow up assessment to maximize continual improvement
Biting off more than we can chew – trying to do too much
C-h-a-l-l-e-n-g-e-s
![Page 20: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/20.jpg)
4-1-4 calendar (5-week January term), abroad and on-campus courses offered
About 1,400 UD students abroad in 2010/11; 75% on January programs (1060 in 2011); 90% on short-term programs (summer and winter programs)
Over 40% of undergraduates (population 15,000) study abroad
About 70 UD programs annually, 50 in January, all faculty led, representing 75% of academic departments
About 20 semester/exchange programs (about 130 students total)
Study Abroad at Univ. Delaware
![Page 21: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/21.jpg)
Paper evaluation (satisfaction survey) unwieldy to administer data hard to retrieve/summarize/distribute
Office of Educational Assessment created 2005 institutional interest in assessing undergraduate
learning campus resource for expertise
Discovery Learning Experience required for all undergrads as of 2006
good reason for mandatory assessment study abroad counts as DLE!
Where We Were
![Page 22: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/22.jpg)
Given that: the number of students and programs is very large; students are enrolled in a wide variety of courses; most are not enrolled in a foreign language course
(though many are); programs are located across the globe in both rural and
urban settings; the range of program conditions is great (housing,
mobility, interaction with host culture); and faculty directors have different program goals:
What can we measure that applies to all students?and
How can we measure it?
Thoughts on How to Move Forward
![Page 23: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/23.jpg)
What We Do – Data CollectionLook for general impacts (changes from pre to post) in areas of knowledge, skills, and attitudes (applicable to all programs)Design a short instrument (31 Likert scale items, 2 short answer)Incorporate into required online pre-departure orientationInstitute a required post-program assessment (linked to University-wide Discovery Learning ExperienceInclude standard satisfaction survey (subjective feedback on program quality)Use existing online course evaluation system (via registration for UNIV 370, pass/fail, 0- credit,
study abroad marker)
![Page 24: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/24.jpg)
What We Do – Data Organization
Download satisfaction survey feedback; send as pdf to faculty after grades are submitted
Download course feedback; send as pdf to dept. chairs
Program coordinators prepare summaries of each program, addressing budget, faculty organization and performance, incidents abroad, student feedback (numerical ratings, selected quotations), and suggestions for next time
Summaries sent with course syllabi to dept. chair and dean
Assessment results stored and analyzed internally
![Page 25: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/25.jpg)
Largely responsibility of faculty for short-term programs
Responses to short-answer questions can be used in publications
What We Get and Use: Marketing
![Page 26: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/26.jpg)
What We Get: Development Masses of data: 98% return
rate Electronic data collection,
retrieval, storage using UD’s course evaluation system
Student satisfaction survey results helpful for faculty and program administrators
Used in program summary sent to faculty, chairs, deans
Documentation of problem cases
![Page 27: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/27.jpg)
What We Get: Assessment Quantitative and qualitative pre- and post-
sojourn data from about 1400 students on 70 programs annually
98% return rate Electronic data collection, retrieval, storage,
using UD’s existing online course evaluation system
Ongoing data collection Improve academic
programming and student learning
Friends in Assessment Office
Positive contribution to institutional re-accreditation
![Page 28: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/28.jpg)
assessment is self-reported
on-campus control group difficult to maintain
in-house instrument (untested beyond UD)
data backlog (lack of time for analysis)
Limitations/Disadvantages summaries very time-
consuming and labor-intensive to compose and send
not completely objective
impact of summaries unknown (Do chairs/ deans read them?)
![Page 29: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/29.jpg)
What kind of assessment or evaluation of education abroad programs is done at your institution?
Who sees the results? How are the results used? What prevents results from being used
for marketing purposes, for implementing programmatic or procedural changes, or for better understanding student learning?
Questions/Discussion
![Page 30: Power up your Program Evaluations Marketing, Development, & Assessment.](https://reader038.fdocuments.net/reader038/viewer/2022102808/56649c395503460f948e1c14/html5/thumbnails/30.jpg)
Lisa Chieffo, Ed.D.Associate Director
Institute for Global StudiesUniversity of Delaware
Newark, Delaware, USA302-831-2852
[email protected]/international
How to Find UsKevin Kehl, Ed.D.Executive Director
Center for International EducationAbilene Christian University
Abilene, Texas, USA325-674-2710
[email protected]/academics/studyabroad
Wendy WilliamsonDirector
Office of Study AbroadEastern Illinois UniversityCharleston, Illinois USA
[email protected]/~edabroad