Research Proposal
-
Upload
susan-wilson -
Category
Documents
-
view
17 -
download
0
description
Transcript of Research Proposal
Research Proposal
Online Student Satisfaction
Susan Wilson
University of British Columbia
Introduction
In his 2006 research article, Herbert stated that the rate of discontinuance in
online, post-secondary courses is ten to twenty percent higher than that of traditional
classroom-based courses. As online education moves into secondary education
institutions, the problem of retention will compound. Post-secondary institutions work
with students who have already successfully completed one tier of education; students
who are motivated to learn and have a financial interest in successful course completion.
Public secondary schools educate students of varied abilities, attitudes, and backgrounds.
I believe that the rate of discontinuance in online, secondary courses is even greater than
that of online, post-secondary courses.
According to the 2008 Saskatchewan Education Indicators Report,
Saskatchewan’s population of 1 015 985 on July 1, 2008 was made up of approximately
40% of people aged 45 and over and only 27% of people aged 20 or younger. In addition
to the relatively small population, the report states that the number of people living in
rural communities has declined at a higher rate (11% decrease from 1971 to 2001) than
the Canadian average (4% decrease from 1971 to 2001). Saskatchewan’s large
geographical area and dispersed population make it necessary to operate small schools,
some of which have fewer than 50 students.
Online learning is a practical solution to challenges presented by rural
depopulation and a large geographical area. It is also a viable learning environment for
adult students, students who are required to work to support themselves financially, and
students who have significant family responsibilities. Saskatchewan’s 2005 rate of teen
pregnancy was the highest among the provinces and more than double that of the
Canadian rate (Saskatchewan Indicators, 2008).
The 2008 Saskatchewan Indicators report suggests that approximately 20% of
youth do not graduate. Retention of secondary courses is an important consideration in
Saskatchewan and everywhere. Having a Grade 12 certificate increases the chances that
a person will be employed (Saskatchewan Indicators, 2008) and increases the probability
that a student will continue their education. Having a post-secondary certificate, diploma
or degree increases the average income level to an average of 33% (statistic from 2005)
higher than that earned by those without the post-secondary training (Saskatchewan
Indicators).
As an online educator in rural Saskatchewan, I am motivated to learn how to best
implement online education opportunities into our high-schools. Rural depopulation and
economic factors translate to teacher cuts in our school division. If we are to remain
viable, we must offer and accept online education into our school. What factors must we
be cognizant of when designing online courses? How can we best support our students
before and during their involvement in online education?
Online education from K-Post Secondary reaches millions of students every day
(Corry, 2008). If online courses are being offered to students in situations like ours, then
students are not making a voluntary choice to take a class online, they are required to take
online education to complete graduation requirements. At times, this is because small
rural schools do not have the staing to offer all required courses, they do not have the
staing to offer various electives, or they do not have the staff to offer multiple forms of a
class to accommodate students who have missed credits or changed schools. If courses
are not otherwise be available, we, as educators, must ensure that students are satisfied
with their online experience and that they are receiving quality education.
Education is for everyone and online education should not be any different. It is
imperative that reliable instruments are developed to measure student success and
provide quality assurance to all stakeholders of public education. Educators who have
access to an instrument that can predict the probability of student success will be able to
design pro-active interventions and supports to facilitate student success thus widening
the doors to online education.
With this in mind, this research proposal is designed to answer the following
questions:
1. What factors do students perceive to aid in their satisfaction and success in
online courses at the secondary level?
2. What factors do online educators perceive to aid in student satisfaction
and success in online courses at the secondary level?
In answering these questions, I hope to collect data that will aid in the development of a
predictive instrument for use with Saskatchewan high-school students entering into
online education. The data will also be used to aid in the development of a student
support system that will be used to target areas of student weakness in an effort to
increase their probability of success in secondary, online courses. The data will not be
used to limit or restrict enrolment in online, secondary courses.
Quantitative and qualitative data will be collected by using student and instructor
survey responses. I hypothesize that student demographics (age, gender, location of
residence) will not be identified as factors correlating to student success. I believe that
institutional factors such as instructional design, hardware and human infrastructure,
communication, time, and course organisation will be identified by both students and
instructors as being significantly correlated to student satisfaction and success. I expect
that personal student factors such as ability, experience and motivation will have an effect
on satisfaction and success, but that they will not be as significant as institutional factors.
While circumstantial variables may present themselves and influence both satisfaction
and success, they are not controllable by student, instructor or institution and will
therefore not be considered in this study.
Student satisfaction will refer to the degree to which students enjoy the
educational experience and are engaged and interested in the course. Student success
refers to a student’s passing the course and retention refers to finishing the course.
Literature Review
Studies relating to factors influencing student success and satisfaction in online education
are becoming more prevalent as online education undergoes rapid and continuing growth
(Dailey-Hebert et al, 2005; Herbert, 2006; Leong et al, 2006; Roblyer & Davis, 2008;
Yukselturk & Bulut, 2007). Expressing a concern with growing rates of discontinuance,
researchers (Dailey-Hebert et al, 2005; Herbert, 2006; Leong et al, 2006; Roblyer &
Davis, 2008) have attempted to identify factors relating to student success and student
satisfaction in online courses.
Articles collected for this review study factors affecting online student satisfaction (by
perception) and online student success (passing the course), with emphasis on retention
(completing the course). Research surrounding factors affecting student satisfaction in
online education is extremely important as students who are satisfied with their online
experience are more likely to retain the course (Herbert, 2006; Swan et al, 2006).
Assessing student satisfaction can be valuable in terms of program and course
improvement (Herbert).
The literature review is organized according to guidelines by Gay, Mills & Airasian
(2009) using the headings Post-Secondary Student Satisfaction, Post-Secondary Student
Success and High-School Student Success.
Post-Secondary Student Satisfaction
1. Staying the course: A study in online students’ satisfaction and retention.
Herbert’s (2006) cross-sectional study was designed to identify institutional variables
affecting retention. He used the Noel-Levitz Priorities Survey for Online Learners™,
which was e-mailed to every student enrolled in an online course at a small, Midwestern
university during the 2005/06 school year. Herbert cited evidence that student variables,
among other factors, can be measured to predict the degree to which a student will
complete an online course. He also referenced research findings that institutional and
demographic variables are predictive as well.
Analysis of survey results was done by the Noel Levitz company. Of the 4100 students,
122 completed surveys were returned representing 40.1% of the students who did not
successfully complete their online course. When asked for the reason for discontinuance,
61.3% cited time constraints and 16.1%, personal problems.
The most significant variable identified was faculty response to student needs followed
by the quality of online instruction, timely feedback, quick institutional response to
requests, and frequency of student-instructor interaction. Student-student interactions
scored lowest. There was no statistical difference based on institutional variables
between course completers and non-completers. However, results of an independent
samples t-test showed a statistically significant difference between satisfaction levels of
completers and non-completers. Students who were satisfied with their online experience
were more likely to retain the course.
2. An empirical investigation of student satisfaction with web-based courses.
This quantitative study surveyed 128 students enrolled in 29 University of Hawaii, online
courses to examine the relationship between demographic variables, online course
experience and satisfaction. The researchers’ rationale was to identify factors to increase
student satisfaction.
Based on previous research, Leong et al (2006) produced a 47-item survey to address
instruction, instructor characteristics, management, technology, interaction, experience,
workload and fairness of grading. Two additional questions based on overall course
satisfaction and comparisons to face-to-face instruction were asked. A response rate of
25.2% (128 surveys) was achieved and results were calculated using factor and
regression analysis. Also, t-test and univariate analysis of variance were used. Results
showed that overall student satisfaction was influenced by four dimensions: instructor,
system-wide technology, workload/difficulty and interaction but not by demographic
factors or students’ prior experience.
Post-Secondary Student Success
3. Learner attribute research juxtaposed with online instructor experience: Predictors
of success in the accelerated online classroom.
Dailey-Hebert et al (2005) compared characteristics of learner attributes with online
instructor experience. The rationale was to develop practice-orientated understanding of
factors predicting student success. Literature exists on online student retention, their
attributive influences and their reasons for choosing online delivery but Dailey-Hebert et
al posited a need for more information about predictors of success as research based on
internal learner attributes is of limited use unless paired with external data elicited from
online educators.
The study surveyed a self-selected sample of 96 online educators with an average
teaching experience level of 3.5 years. All educators offered accelerated, 8-week courses
and no demographics were collected on the sample however future studies may compare
educator characteristics with factors influencing success. An e-mail solicited their
opinion on the five factors most likely to predict successful completion of an online
course and a survey identified skills, strategies or factors they perceived to lead to student
success. A content analysis of the responses identified 23 relevant factors which were
then grouped into six themes, student competence, student initiative, student personal
issues, time, technology, and instructional factors.
The four most predictive issues were time [timely, active involvement in the course
(67.71%), effective time-management (67.71%) and timely access to instructional resources
(19.79%)], initiative [personal initiative (52.08%), asking questions and seeking help
(37.50%), self-motivation (22.92%) and a positive attitude (4.17%)], technology [access
to efficient computer and internet literacy (40.63)], and competence [reading comprehension
(23.96%), writing skills (22.96%), communication (17.71%), awareness of expectations,
environment and workload (16.67%), and organizational skills (13.54%)].
4. Predictors for Student Success in an Online Course
Yukselturk and Bulut (2007) designed a correlational study to analyse student
characteristic variables (gender, age, education level, locus of control, learning style),
motivational beliefs (intrinsic and extrinsic goal orientation, task value, self-efficacy, and
test anxiety) and self-regulated learning components (cognitive strategy use and self-
regulation) that predict online student success. They also examined instructor views on
predicative factors. They hypothesized that the variables do not explain variance in
student success.
The researchers posited a need for the inclusion of instructor perception. Their rationale
was to fill a void in the research and to identify procedures for the design of high-quality
online learning environments. In a well-written, extensive literature review, Yukselturk &
Bulut provided history on online education and stated a need for maintaining quality.
The authors defined terms used in the study.
The sample of two course instructors and 80 voluntary participants enrolled in the Data
Structure and Algorithms with C course at the Middle East Technical University in
Ankara, Turkey in the 2005/06 school year. All students were computer literate with
intermediate English. There were more males (N = 56) than females (N = 24) and the
majority were between 19 and 29 years of age. In addition, most had an assimilator
learning style and a university degree.
Quantitative and qualitative data was collected through four online questionnaires:
Demographic Survey, Internal-External Locus of Control Scale, Learning Style
Inventory, and Motivated Strategies for Learning Questionnaire. Semi-structured
interviews were conducted; the ten questions were developed around central themes then
examined by two experts in the field of instructional technology.
Intrinsic goal orientation, task value, self-efficacy, cognitive-strategy use, and self-
regulation were significantly positively correlated with success. Educational level and
external locus of control were negatively correlated. Varied student characteristics and
general personal characteristics, did not directly affect success.
5. Online student success: Making a difference
Beyrer’s (2005) study compared online students enrolled in Online Student Success
(OSS), a course designed to prepare students for online learning, and online students who
have never taken this class. He also compared online performance of students before and
after completion of OSS. Beyrer hypothesized that students who take OSS will be more
successful than those who do not and they will be more successful after completion of
OSS than they were previous.
Beyrer conducted a well-written, comprehensive literature review stating that most
research focused on instructional design factors affecting student success, tips to aid
student success, and technology and interface characteristics. Beyrer defined the terms
used in the study.
The convenience sample included all students enrolled in fully online classes from
Cosumnes River College, Sacramento CA from 2003 to 2005. They were divided into 4
subgroups: those who had never enrolled, those who enrolled and passed, those who
enrolled and did not pass and those who enrolled but dropped. Historical data (academic
performance, enrollment, and demographic) was collected and analyzed and an online
survey of students was conducted. Twenty-one respondents completed the 18-question
(open and closed answers) follow-up survey.
Compared to students who did not take OSS, the ones that successfully completed OSS
had a higher success rate in their online classes. Those who were unsuccessful or dropped
OSS had a lower success rate. Though only 11 students took online classes before and
after enrolling in OSS, the improvement in their success rate was dramatic. Follow-up
survey responses from all three OSS groups were positive.
High-School Student Success
6. Predicting success for virtual school students: Putting research-
based models into practice.
Roblyver and Davis (2008) posited that prediction models should be developed support
school-age student success in online courses. They explained that the model should be
based on the combined factors that contribute to predicting success as identified by
research and should provide efficient measurement and implementation in virtual school
settings. The article contains an example of a prediction model, the Educational Success
Prediction Instrument (ESPRI) and a description of data collection and statistical
processes used to derive it. They outlined procedures for implementation to increase the
accuracy and utility of predictions.
Model testing was conducted using 4,110 students in the Virtual High School Global
Consortium (VHS) who were enrolled in 196 VHS courses in 2006. An electronic version
of the instrument was placed in course spaces and students were offered 10 points extra
credit on their first week's assignments to complete the survey. A completed ESPRI
survey, demographic data, and course scores and status were obtained for 2,162 students
or about 53% of the total school population. Reliability with the 25-item instrument
was .92. The model correctly predicted 93% of those who were successful, but only
30.4% of those who failed.
Common Themes
In general, research methods have consisted of survey instruments distributed online to
students enrolled in online courses at the post-secondary level with few conducted at the
high-school level. All studies in this literature review are cross-sectional in nature
indicating a need for longitudinal research which can identify trends from which
appropriate revisions can be made (Herbert, 2006).
Herbert’s literature review found demographic factors to be predictive of satisfaction and
retention but the studies compiled by the other researchers showed conflicting
conclusions or no evidence that demographics can predict success. Dailey-Hebert et al
and Yukselturk and Bulut identified a prevalence of reseach based on the effects of
student characteristics. Both research studies widened their examination of predictive
factors to include the perception of experienced online instructors. Linking student
responses with faculty perceptions can be powerful in creating a practical understanding
of the influences on student satisfaction and success.
Faculty response to student needs and student-instructor interaction were significant
factors identified in the research studies. Time constraints and management issues were
also significant predictors of satisfaction and success. Yukselturk and Bulut were the
only researchers to focus on student characteristics such as motivation and self-
regulation.
Summary
All researchers indicated a desire to inform educators of desirable factors to influence
student satisfaction and student success in an online environment. Most stated a desire to
counter-act course discontinuance at the post-secondary level. Studies of student and
institutional factors produce varied and somewhat contradicting results. Longitudinal
research ventures may aid in identifying trends in student success that will better inform
improvements. Predictive instruments of student success will be helpful if they can be
refined to identify student supports needed and not used to limit access.
This idea that research findings provide actionable information is supported by
Beyrer (2005) and Roblyer (2008). Both projects involved the development, testing or
implementation of an instrument, one predicative and one instructive, used to determine
student suitability to online education and support students in need. Beyrer’s work
focused on the benefit of enrolling students in an Online Student Success (OSS) course
designed to improve probability of student success in online courses. Combining this
idea with Roblyer’s focus on predicative models for online student success would allow
institutions to offer pre-assessment and targeted support to potential students. It is
important to note that the purpose of these instruments is to increase enrolment and
success in online courses, not to rank, stream or prohibit students.
Methodology
Elaine Strachota (2008) developed a 27-item survey instrument, The Student
Satisfaction Survey, based on student satisfaction with respect to online interaction
(learner-content, learner-instructor, learner-learner, and learner-technology), and general
satisfaction. Strachota defined each construct. Adult and distance education experts
edited the survey questions to establish content-validity. Construct validity was
established by conducting a pilot study of 249 online students at a Midwest Technical
College in the US. Data from the pilot underwent factor analysis to verify that items
loaded on the intended constructs. Those that did not load were eliminated.
This research project will use survey question ideas from Strachota’s Student
satisfaction Survey as well as student readiness survey question ideas from Slick (2004)
to develop an instrument suited to secondary students. Content validity will be
established through review by the Instructional Technology Department Consultant and
Coordinators of the SouthEast Cornerstone School Division, SK. This will take place at
our next scheduled meeting on April 29, 2009. Construct validity will be established
through a trial with online students in our division which will take place in June, 2009.
Statistical analyses on data received will inform decisions about question retention or
removal. Each survey item will require a Likert-response consisting of a 4-point scale:
strongly agree, agree, disagree, strongly disagree.
Once finalized, the survey will be distributed to a convenience sample of students
enrolled in online education courses through the SouthEast Cornerstone School Division
through their course management system. Course instructors will notify students of the
survey by e-mail and will send up to three reminders over the two week implementation
period. Completion of the survey will be expected as part of the assigned course work
and will be given to students in the final two weeks of their course. Parental permission
will not be required as the research will be done through the students’ attending school
division for the purpose of improving instruction.
Data for this longitudinal study will be collected and analysed in January and June
of each year for two complete years. Once complete, a trends analysis will be performed
to determine factors affecting student satisfaction and success. Results will be quantified
by converting frequency of responses to percentages. Results from open-ended survey
questions will be compiled and categorised. Data from respondents indicating course
satisfaction and feelings of success will be evaluated for factor correlation. Data from
respondents who report dissatisfaction and lack of success will be evaluated for factor
correlation.
This data, analysed and interpreted over a two year period, will be used to aid in
the development of a predicative instrument for student success in online environments.
It will also be used to develop a student support system that targets factors influencing
student success.
Significance
Even though online education reaches millions of students every day (Corry,
2008), the field of online learning is still in its developmental stage with respect to having
valid and reliable measurement instruments (Strachots, 2008). Creating a reliable survey
instrument, developed for and piloted on secondary school students will add to the field
of distance education at the secondary level. Further work with survey results to develop
a predictive instrument and student support component will directly benefit online
students.
Student feedback is an important component in the provision of quality education
when data received is used to improve student instruction, satisfaction and chance of
success. Many secondary teachers are new to online delivery. Survey results will be an
important component of professional development for online teachers. Data obtained
will help inform department or even provincial policies and standards created to ensure
quality in online course delivery. Data obtained from online instructors will provide
good fodder for conversations around instructional design, pedagogy and best practices.
Continuing the research over the course of two years will provide more reliable
and normally distributed data allowing for trend-analysis of student satisfaction, success,
and retention at the secondary level. This information will provide a significant
contribution to the field of distance education at the secondary level.
Conclusion
Canadian school divisions are developing internal capacity for online course
design, delivery and implementation. Current research around student satisfaction and
success concentrates on post-secondary students. Research needs to be done on factors
affecting the satisfaction and success of online, secondary students.
As evidenced in the research, surveys are effective tools for gathering both
quantitative and qualitative data from students and instructors. Embedding the survey
into the course management system ensures that all involved are presented with the
opportunity to respond. Managing the survey distribution and data collection through
such technology reduces errors and strengthens the reliability of results. Field testing and
content validation will also increase reliability.
Research results will be used to develop a more refined survey instrument that
will predict student success in an online learning environment. The instrument will
identify individual student strengths and weaknesses for the purpose of providing student
support. Student satisfaction and success factors will also be considered in the
development of a student support instrument that will enable students to improve their
weaknesses thus improving their chances of retention and success in online courses.
References
Beyrer, Gregory M. D. (2005) Online student success: Making a difference. Cosumnes
River College. @ONE Carnegie Scholar. 2005-06. Retrieved from
http://www.cccone.org/scholars/05-06/GregBeyrerMonograph.pdf
Corry, M. (2008). Online student success: Six categories of importance to the future. In
K. McFerrin et al. (Eds.), Proceedings of Society for Information Technology and
Teacher Education International Conference 2008 (pp. 312-317). Chesapeake,
VA: AACE. Retrieved from www.editlib.org/index.cfm/files/paper_27178.pdf?
fuseaction=Reader.DownloadFullText&paper_id=27178
Dailey-Hebert, Amber, Donnelli, Emily, and Mandernach, B. Jean. (2006). Learner
attribute research juxtaposed with online instructor experience: Predictors of
success in the accelerated online classroom. The Journal of Educators Online,
Volume 3, Number 2, July 2006. Retrieved from
http://www.thejeo.com/Volume3Number2/MandernachFinal.pdf
Gay, L.R., Mills, G.E., & Airasian, P.W. (2009). Educational research: Competencies for
analysis and application (9th ed.). Upper Saddle River, NJ: Merrill Prentice Hall.
Herbert, Michael PhD. (2006). Staying the course: A study in online students
satisfaction and retention. Online Journal of Distance Learning Administration,
Volume IX, Number IV, Winter 2006 University of West Georgia, Distance
Education Center. Retrieved from
http://www.westga.edu/~distance/ojdla/winter94/herbert94.htm
Leong, Peter, Ho, Curtis P., Saromines-Ganne, Barbara. (Winter, 2006). An empirical
investigation of student satisfaction with web-based courses. Online Journal of
Distance Learning Administration, Volume IX, Number IV, Winter 2006.
University of West Georgia, Distance Education Center. Retrieved from
www.editlib.org/index.cfm/files/paper_9442.pdf?
fuseaction=Reader.DownloadFullText&paper_id=9442
Roblyer, M.D., Davis, Lloyd. (2008). Predicting success for virtual
school students: Putting research-based models into practice.
Online Journal of Distance Learning Administration. Winter 2008.
Volume 11. Issue 4. University of West Georgia. Retrieved from
http://www.westga.edu/~distance/ojdla/winter114/roblyer114.html
Saskatchewan Education Indicators: Pre-Kindergarten to Grade 12. 2008. Available
from http://www.education.gov.sk.ca/adx/aspx/adxGetMedia.aspx?
DocID=609,135,107,81,1,Documents&MediaID=6109&Filename=Web+Version
+2008.pdf
Slick, J. (2004). Using Student Readiness as a Predictor of Satisfaction in the Online
Environment. In G. Richards (Ed.), Proceedings of World Conference on E-
Learning in Corporate, Government, Healthcare, and Higher Education 2004
(pp. 2130-2136). Chesapeake, VA: AACE. Retrieved from
www.editlib.org/index.cfm/files/paper_11492.pdf?
fuseaction=Reader.DownloadFullText&paper_id=11492
Strachota, Elaine. (2008). The use of survey research to measure student satisfaction in
online courses. Online Journal of Distance Learning Administration, Volume XI,
Number IV, Winter, 2008. University of West Georgia, Distance Education
Center. Retrieved from
http://www.umsl.edu/divisions/conted/education/mwr2p06/pdfs/D/Strachota_Use
_of_Survey_Research.pdf
Swan, Karen; Schenker, Jason; Lin, YiMei; Shea, Peter;; Aviv, Reuven (2006).
Student satisfaction with online learning: A concept analysis. Proc. Annual
Meeting of the American, Educational Research Association, San Francisco, April
2006. Retrieved April 13 from http://tinyurl.com/d2twn2
Yukselturk, E. & Bulut, S. (2007). Predictors for Student Success in an Online Course.
Educational Technology & Society, 10 (2), 71-83. 71 ISSN 1436-4522 (online)
and 1176-3647 (print). © International Forum of Educational Technology &
Society (IFETS). Retrieved from http://www.ifets.info/journals/10_2/7.pdf