EDF 711: Survey Research...seminar evaluations have been lacking in providing specific feedback to...
Transcript of EDF 711: Survey Research...seminar evaluations have been lacking in providing specific feedback to...
1
Marshall University Doctoral Seminar Evaluation
Project Completed as Course Requirement in
EDF 711: Survey Research
Summer 2015
By: Laura McCullough, Pamela Meadows, and
Rebecca Metzger
Marshall University
College of Education and Professional Development
November 22, 2015
2
Abstract
The purpose of this study was to determine the level of satisfaction with the Marshall
University College of Education and Professional Development Doctoral Seminar. There is
currently no body of research regarding the development of the doctoral seminar and the
doctoral seminar planning committee identified a need for a formal evaluation of the seminar.
The study consisted of an individual session survey and overall seminar survey containing
quantitative and qualitative questions to measure student and faculty satisfaction. The results
from this study will be utilized by the doctoral seminar planning committee to identify areas for
improvement and aid in the scheduling and recruitment of speakers for future seminars.
3
Table of Contents
Introduction ..................................................................................................................................................... 4
Statement of the Problem .............................................................................................................................. 4
Purpose of Study ........................................................................................................................................... 5
Research Questions ....................................................................................................................................... 5
Survey Design/Methods .................................................................................................................................. 6
Findings ........................................................................................................................................................... 7
Participant Characteristics ............................................................................. Error! Bookmark not defined.
Seminar Logistics…………………………………………………………………………………….……9
Presentation Quality………………………………………………...…………………..…………..……10
Keynote Speaker……………………………………………………………………………………………….………………………………11
Conclusions…………………………………………………………………………………………………11
Discussion/Implications/Recommendations ............................................................................................... 12
Appendix A…………………………………………………………………………………………………16
Consent……………………………………………………………………………………………….….16
Appendix B…………………………………………………………………………………………………17
Individual Session Evaluation Survey Instrument……………………………………………………....17
Appendix C………………………………………………………………………………………………...18
Overall Doctoral Seminar Evaluation Survey Instrument………………………………………………18
Appendix D………………………………………………………………………………………………...19
IRB Approval Letter………………………………………………….………………………………...19
Appendix E………………………………………………………………………………..……………….20
Data: Participant Characteristics……………………………………………………………………….20
Appendix F………………………………………………………………………………………………...22
Data: Overall Doctoral Seminar Evaluations…………………………………………………………..22
Appendix G………………………………………………………………………………………………..28
Data: Individual Session Evaluations………………………………………………………………….28
4
Introduction
Doctoral programs have long histories of requiring students to complete residencies as a
component of their studies. Residencies provide students with the opportunity to build
meaningful relationships with their peers and staff to further develop research and professional
skills. When Marshall University was granted permission to offer a doctorate in education
degree, the program sought students who were full-time professionals who desired to earn an
advanced degree as a part-time student. Residency requirements changed to meet the needs of
the student population.
Doctoral students are expected to participate in an annual seminar, scheduled on a
Saturday each fall. Seminars are coordinated by a group of doctoral students and a faculty
member, and students who assist in coordinating the seminar or presenting at a session use their
participation as an artifact in their doctoral portfolio. Students who participate in a more inactive
role use the seminar as a way to increase their knowledge about the program and develop
relationships with faculty that may lead to the identification of a chairperson and committee.
For over ten years, the doctoral seminar at the Marshall University College of Education
and Professional Development has provided students with networking opportunities, while
guiding them through the various stages of the doctoral program. The doctoral seminar planning
committee has welcomed assistance with the development of an evaluation system for the
seminar. The purpose of this study was to evaluate student satisfaction with regard to the annual
doctoral seminar.
Statement of the Problem
Participation in the annual seminar is a requirement for doctoral students. The seminar
planning committee needs to evaluate student and faculty satisfaction with individual sessions
5
and the overall seminar to determine if the training is meeting residency goals. Previous
seminar evaluations have been lacking in providing specific feedback to the planning
committee, and inconsistencies with surveys were also present. Research is needed to
determine the satisfaction level of doctoral students and faculty with the seminar and to
provide information to the planning committee to improve future seminars. The development
of session and overall seminar surveys can also provide baseline data for future training.
Purpose of Study
The purpose of this study was to determine the satisfaction level of doctoral students and
faculty with their experiences in attending the Marshall University College of Education and
Professional Development 2015 Doctoral Seminar. The results of this study will identify areas
for improvement and will be utilized by the doctoral seminar planning committee for the
development of future doctoral seminars at Marshall University Graduate College. Data
collected regarding individual session evaluations will aid speakers in improvement of
presentation skills, development of their curriculum vitae or resume, and will serve as
documentation required for doctoral portfolio, promotion, or tenure.
Research Questions
The following specific questions were used to guide the study:
1. What are the demographics of seminar attendees?
2. How do seminar attendees rate the effectiveness of the registration process?
3. How do attendees rate the seminar sessions in meeting individual and professional
needs?
4. How do attendees rate the quality of seminar training sessions?
5. How do seminar attendees rate networking opportunities?
6
6. How do attendees rate their satisfaction with facilities and refreshments?
7. What do attendees identify as the major benefits of the seminar?
8. What recommendations do attendees have for improving speaker performance
and the overall seminar?
Survey Design/Methods
A total of sixteen sessions were offered during the Marshall University College of
Education and Professional Development 2015 Doctoral Seminar. Guidance from the seminar
planning committee regarding evaluation criteria and program needs was utilized to develop two
survey instruments. The survey instruments addressed attendee level of satisfaction with various
seminar logistics, topics, and speaker effectiveness.
Current doctoral students in a survey design course and the doctoral seminar planning
committee assisted in validating the surveys. Session evaluation surveys were color-coded for
organization following the seminar, and collection boxes were placed in session rooms to
receive anonymous surveys at the end of the sessions. Four moderators were responsible for
administering an anonymous paper survey to all session attendees within the last five minutes
of the individual sessions, and seminar overall evaluation surveys were provided five minutes
prior to the last session ending. No identifying information was obtained from participants.
The seminar session evaluation surveys asked respondents to identify the session
attended and the level of expertise in the subject area prior to attending the session using a
five-point Likert scale. Participants were asked seven quantitative questions to rate the
quality of the training session using a seven-point Likert scale. The questions focused on the
organization of the presentation, effective use of time, visual aids, and responsiveness of the
presenter. Participants were also asked to rate their agreement in recommending the presenter
7
for future seminars and in session content contributing to their overall knowledge. Two
qualitative questions asked respondents to identify areas of growth as a result of the session
and recommendations to aid the presenter(s) for future sessions.
The demographic section of the seminar overall evaluation consisted of one question
to identify the current status of the respondent in the doctoral program as a program graduate,
faculty, guest, or current student. Current students were asked to identify their program status
as currently completing regular coursework, admitted to candidacy/prospectus not approved,
and admitted to candidacy/prospective approved. Students who identified themselves as
currently completing regular coursework were asked to approximate the number of hours they
had completed in the doctoral program.
A total of 10 quantitative questions on the overall evaluation addressed satisfaction
with the registration process, session topics, facilities, and refreshment using a seven-point
Likert scale. Respondents were also asked to gage if their understanding of the doctoral
program and procedures significantly increased as a result of the attending the seminar. The
qualitative questions asked participants to identify the major benefits of attending the seminar
and suggestions to improve the seminar.
Findings
The primary purpose of this survey was to evaluate the perceptions of the doctoral
students, faculty, and guests who participated in the 2015 Marshall University Doctoral seminar.
Specifically, the questions asked were to measure the demographic characteristics of the
participants, logistics of the seminar, and quality of the presentations. There was also one item
polling if the participants believed having a keynote speaker for next year’s seminar would
improve their experience. Examples of the two instruments used are provided in Appendix B and
Appendix C. Detailed survey responses were organized by question and are provided in
8
Appendices E, F and G.
Participant Characteristics
Thirty people who attended the Marshall University Doctoral Seminar completed an
overall conference evaluation survey (n=30). Twenty-eight (93.3%) of evaluation completers
were current students, one was a guest, and one was a program graduate. The majority of the
current students (67.9%) who completed an overall seminar evaluation were completing course
work. Seven respondents (25%) reported they were either admitted to candidacy or that their
prospectus was not yet approved. Two respondents (7.1%) reported that they were admitted to
candidacy or that their prospectus was approved.
The frequency of the credit hours completed by the students completing course work was
not normally distributed. Eight current students, representing 47.2% of the respondents, had
completed fewer than 10 credit hours of course work. One current student, representing 5.9% of
the total respondents, reported having completed 18 credit hours. Two current students,
representing 11.8% of the respondents, had completed more than 20, but less than 29 credit
hours. Four current students, representing 23.6% of the respondents, had completed a range from
30 to 39 credit hours. One student, representing 5.9% of the respondents, had completed 43
credit hours. One student, representing 5.9% of the respondents, had completed 56 credit hours.
The percentages reflected above are “valid percentages,” because 13 current students neglected
to report how many credit hours they had already completed. A summary of participant
characteristics is located in Appendix. E.
One hundred, eighteen duplicated seminar session attendees self-reported their expertise
before attending each of the seminar sessions they chose to attend. The most frequent response
9
(33%) from attendees indicated an intermediate level of prior expertise. Twenty-five percent
perceived their prior expertise as basic. Sixteen percent judged their prior expertise to be
between intermediate and advanced. Fourteen percent believed their prior expertise was between
basic and intermediate. The percentage of students who had advanced levels of expertise prior to
attending one of the seminar’s sessions was the lowest at 11%.
Seminar Logistics
All of the attendees who completed the doctoral seminar evaluation either somewhat
agreed, agreed, or strongly agreed the registration process was organized and 93% agreed the
registration process was efficient (93%). Seminar participants either somewhat agreed, agreed, or
strongly agreed the seminar was relevant to the attendee’s programmatic needs (97%) and was
relevant to their individual needs (97%). At 97%, participants believed the sessions were of the
appropriate length of time, and that networking opportunities were available. One hundred
percent of the people who completed the doctoral seminar evaluation believed the facilities met
their needs. Eighty-seven percent of the people who completed the doctoral seminar evaluation
thought that the food and beverages were satisfactory and that their understanding increased as a
result of attending the seminar.
Eighteen people who completed the doctoral seminar evaluation made positive statements
for major benefits of attending the doctoral seminar. There were three themes: (1) useful for
gathering information about the future steps in their program; (2) enjoyed networking with
others; and, (3) general compliments on the seminar. There were only three suggestions for
improvement: (1) information from other sessions would be helpful; (2) please make sure
everyone is informed about the date; and, (3) shorten.
10
The findings from the overall doctoral seminar evaluation survey were overwhelmingly
positive (Appendix F). The only limitation is that the number of people completing this
instrument was small. According to registration records, a total of 58 people attended the
doctoral seminar.
Presentation Quality
Participants completed evaluations regarding each of the seminar’s presentations they
attended. There was a range of 130 to 131 responses to the items being measured. The item with
the highest mean score (6.84 with a standard deviation of .408) measured the presenter’s
responsiveness to the audience’s questions. Presenters ranked highly in items related to their
knowledge (6.78 mean with a standard deviation of .469), efficient use of time (6.70 mean with a
standard deviation of .591), organized presentation (6.60 mean with standard deviation of .731),
and style of presenting and visual aids (6.58 mean with a standard deviation of .794). The
participants were likely to recommend the speakers for future sessions (6.75 mean with a
standard deviation of .62). They also believed that the sessions contributed to their overall
knowledge of the subject (6.66 mean with a standard deviation of .73). The mean scores varied
slightly, ranging from 6.58 to 6.84. The two items with the most variance measured presentation
style and visual aids and contribution to overall knowledge.
Reflection on the Portfolio Process was the most popular session with 15 attendees
(11.5% of the population). Current Issues C&I with Dr. Edna Meisel was the least-attended
session with 4 (3.1%) participants. This session, however, along with IRB, Makey-Makey, and
Library Skills received a perfect rating of seven. The lowest rated seminar session was Current
11
Issues: Higher Education and K-12 (mean 6.0 with a standard deviation of 1.291). To view a
distribution table illustrating the attendance and a table showing the mean score for each seminar
session held, see Appendix G.
There were several reoccurring patterns in the responses to the open-ended items on the
seminar session evaluation, most of them were positive. In the areas of growth item, 25
participants noted that their knowledge increased as a result of the session. Nine people said the
sessions gave them some sort of guidance. Six people offered praise to the presenter. Three
people felt encouraged after attending a session. There were also miscellaneous comments in this
field. In the recommendations to the speaker item, 16 offered praise to the presenter. Four
participants wanted the session to be longer for either more content (2) or for more time for
discussion (2). Two people offered the presenter suggestions for improvement. For all
comments, please see Appendix G.
Keynote Speaker
Fifty-nine valid percent (one response was missing) of the people who completed the
doctoral seminar evaluation believed that having a keynote speaker would improve the
experience. Fourteen valid percent of respondents either disagreed or, strongly disagreed. Thirty-
seven valid percent of respondents neither agreed nor disagreed with the statement regarding a
keynote speaker.
Conclusions
Overall, the majority of attendees rated the doctoral seminar favorably. The registration
process was well-received by attendees. Those who attended the seminar expressed that both
12
their individual and programmatic needs were met. The information gathered from the surveys
also indicated that networking and the opportunity to learn more about their program of study
were beneficial. Attendee evaluation of presenters was also positive and the majority of
attendees are in favor of the addition of a keynote speaker in the future.
While the general viewpoint of attendees demonstrated satisfaction with the seminar,
there were a few areas where the need for improvement was indicated. Attendees noted slight
dissatisfaction with regard to food and beverage selection, however, the overall viewpoint was
positive for this element of the seminar. The open-ended questions revealed additional areas of
suggestion for improvement, including the sharing of content for all sessions, timely and
accurate dissemination of seminar date, and decreasing the length of the seminar. Examination of
individual session evaluations was also positive; however, answers demonstrated greater
variance for questions regarding presentation style and visual aids, as well as the contribution of
session information to the overall knowledge of attendees.
Discussion/Implications/Recommendations
Overall and individual session evaluations indicated that attendees were highly satisfied
with the doctoral seminar. There appeared to be a fair distribution of attendees, ranging from
recently admitted students to graduates. Evaluation data revealed that attendees found various
aspects of the seminar to be helpful and enlightening.
According to the administrative assistant for Leadership Studies, one-hundred thirty-four
invitations for the doctoral seminar were sent to current students, as well as recent graduates.
Registration records, however, indicated that only fifty-eight people attended the doctoral
seminar. Using the information given, it can be deducted that less than half of those individuals
13
invited actually attended the seminar. While current students are required to attend at least three
doctoral seminars, graduates are invited to receive awards in recognition of their academic
achievements.
Poor attendance among current students is a primary concern with regard to the planning
of future doctoral seminars. The number of attendees per session (Appendix G) indicates that
some sessions were poorly attended. It should also be noted that although fifty-eight people
attended the seminar, only thirty completed the overall seminar evaluation, which was
disseminated during the final session of the day. This finding suggests that although individuals
are coming to the seminar, many are not attending through the end of the day. It is possible that
some attendees are leaving after the lunch break, during which time they are required to meet
with their committee chairs. To ensure that attendees stay for the duration of the seminar, it is
recommended that the seminar planning committee develop a plan to ensure accountability of
students with regard to attendance. Further examination of individual session attendance would
reveal the approximate number of people who leave before the conclusion of the seminar.
Based on information from the evaluations, it is evident that attendees most appreciate
the opportunities that the doctoral seminar allows for networking with fellow students and
faculty. By providing students with an avenue for interacting with other students at various
stages within the program, students become more comfortable with the expectations of the
program. Networking also allows for the development of friendships and peer-relationships and
can be a key factor with regard to emotional support, which plays a pivotal role in the success of
graduate students. Sessions based on the critical aspects of the doctoral program, including
introduction to doctoral studies and mock portfolio and dissertation defenses, offer students
guidance with regard to critical information needed to ensure timely completion of the doctoral
14
program.
The inclusion of more sessions that allow for discussion among students, and between
students and faculty, is recommended for future seminars. The doctoral seminar is often the only
chance that students have to meet faculty members. By creating more opportunity for
networking, students will not only gain a better understanding of program requirements, but they
will also be equipped with the information needed to aid them in making a more informed
decision in regard to the selection of a committee chair.
While eighty-seven percent of respondents indicated that they were satisfied with the
food and beverage selection, slight room for improvement may be indicated. It may be
advantageous for the planning committee to survey attendees prior to future seminars to
determine meal preferences. While it is impossible to satisfy all attendees, a short meal
preference survey would help to identify the types and quantities of food needed, while
eliminating waste.
Other suggestions for improvement include the inclusion of a keynote speaker,
availability of session presentations, and re-examination of seminar length. A majority of
attendees were in favor of the addition of a keynote speaker. It is recommended that the
planning committee consider recruiting an enthusiastic former graduate or general motivational
speaker to offer a keynote address at future seminars. The inclusion of a general session keynote
would also eliminate the need for recruitment of speakers for one time block. As suggested by
attendee comments, it would be helpful for presenters to post their presentations in an online
repository where students could review or retrieve the learning materials from sessions that they
were unable to attend. For future doctoral seminar evaluations, it may also be beneficial to
15
include a question regarding the efficacy of the award ceremony. Although data was not
collected regarding the awards ceremony at this seminar, general attendance appeared low
among both current students and graduates. The shortening or elimination of the award
ceremony would decrease the total number of hours required for attendance. Another option
would be to combine a general keynote with the award session, while offering three additional
time blocks of sessions. This option would fulfill attendees’ desires for a keynote speaker, while
still including the award ceremony and decreasing the overall seminar time by one hour.
16
Appendix A: Consent Form
Doctoral Seminar Evaluation Anonymous Survey Consent
You are invited to participate in a research project entitled Marshall University Doctoral Seminar
Evaluation designed to analyze the experiences of students, faculty, and guests attending the doctoral
seminar at Marshall University Graduate College. The study will evaluate the quality of individual
sessions, as well as the overall seminar. Dr. Ronald Childress is the principal investigator, and this
research is being conducted as part of the Survey Research in Education Course requirements for Pamela
Meadows (co- principal investigator), Laura McCullough (co-principal investigator), and Rebecca
Metzger (co-principal investigator).
Each survey is comprised of twelve brief questions that may take between 1 to 3 minutes to complete.
Your replies will be anonymous, so do not put your name anywhere on the survey. There are no known
risks involved with this study. Participation is completely voluntary, and there will be no penalty or loss
of benefits if you choose to not participate in this research study or to withdraw. You may choose to not
answer any question by simply leaving it blank. Submitting the paper survey indicates your consent for
use of the answers you supply. If you have any questions about the study you may contact Dr. Ronald
Childress at (304) 746-1904, Pamela Meadows at (304) 550-4439, Laura McCullough at (304) 546-5372,
or Rebecca Metzger at (304) 419-4453.
If you have any questions concerning your rights as a research participant you may contact the Marshall
University Office of Research Integrity at (304) 696-4303.
By completing this survey and submitting it, you are also confirming that you are 18 years of age or older.
Please keep this page for your records.
Thank you.
Pamela Meadows
Laura McCullough
Rebecca Metzger
17
Appendix B: Individual Session Evaluation Survey Instrument
Marshall University Doctoral Seminar Session Evaluation-Fall 2015
Please circle the name of the session you attended.
session title 1 session title 2 session title 3 session title 4
On the scale below, indicate your level of expertise in this subject area
prior to this session:
(Basic) (Intermediate) (Advanced)
1 2 3 4 5
Using the scale of 1-7 below, please rate the quality of the training session by circling the number that
corresponds to your opinion.
Strongly Disagree
Disagree Somewhat Disagree
Neither Agree or Disagree
Somewhat Agree
Agree Strongly
Agree
The presentation was well organized. 1 2 3 4 5 6 7
The speaker effectively utilized the allotted time.
1 2 3 4 5 6 7
The speaker presentation style & visual aids were effective.
1 2 3 4 5 6 7
The speaker(s) was knowledgeable about the topic.
1 2 3 4 5 6 7
The speaker(s) was responsive to audience questions.
1 2 3 4 5 6 7
I would recommend this speaker for future seminars.
1 2 3 4 5 6 7
The session contributed to my overall knowledge of the topic.
1 2 3 4 5 6 7
COMMENTS:
My areas of growth resulting from the session are:
Recommendations to aid the speaker(s) for future sessions:
18
Appendix C: Overall Doctoral Seminar Evaluation Survey Instrument
Marshall University Doctoral Seminar Overall Evaluation - Fall 2015
Please indicate your current status in the doctoral program:
_____ Program Graduate
_____ Faculty
_____ Guest
_____ Current Student (please identify your program progress below):
_____ Currently completing regular coursework
Approximate Number of Hours Completed _____
_____ Admitted to Candidacy/Prospectus Not Approved
_____ Admitted to Candidacy/Prospectus Approved
Using the scale provided, please evaluate the following elements of the doctoral seminar.
Strongly Disagree
Disagree Somewhat Disagree
Neither Agree or Disagree
Somewhat Agree
Agree Strongly Agree
The seminar registration process was organized. 1 2 3 4 5 6 7
The seminar registration process was efficient. 1 2 3 4 5 6 7
Session topics were relevant to my programmatic needs.
1 2 3 4 5 6 7
Session topics were relevant to my individual needs.
1 2 3 4 5 6 7
Session lengths of 50 minutes were appropriate. 1 2 3 4 5 6 7
Opportunities to network were provided. 1 2 3 4 5 6 7
The facilities met my needs. (room size, seat availability, heating/cooling, etc.)
1 2 3 4 5 6 7
The food and beverage selection was satisfactory. 1 2 3 4 5 6 7
My understanding of the doctoral program & procedures significantly increased as a result of attending the seminar.
1 2 3 4 5 6 7
A keynote speaker would improve the seminar experience.
1 2 3 4 5 6 7
COMMENTS:
The major benefits of attending the seminar were:
What suggestions do you have for improving the doctoral seminar?
19
Appendix D: IRB Approval Letter
20
Appendix E: Participant Characteristics
Statistics
Attendee Status Student Status
Credit Hours
Completed
N Valid 30 28 17
Missing 0 2 13
Mean 3.87 1.39 20.59
Std. Deviation .571 .629 16.440
Variance .326 .396 270.257
Skewness -4.782 1.398 .622
Std. Error of Skewness .427 .441 .550
Sum 116 39 350
Percentiles 10 4.00 1.00 3.20
Attendee Status
Frequency Percent Valid Percent
Cumulative
Percent
Valid Program Graduate 1 3.3 3.3 3.3
Guest 1 3.3 3.3 6.7
Student 28 93.3 93.3 100.0
Total 30 100.0 100.0
21
Appendix E: Summary of Participant Characteristics (continued)
Student Status
Frequency Percent Valid Percent
Cumulative
Percent
Valid Currently Completing Course
Work
19 63.3 67.9 67.9
Admitted to Candidacy or
Prospectus Not Approved
7 23.3 25.0 92.9
Admitted to Candidacy or
Prospectus Approved
2 6.7 7.1 100.0
Total 28 93.3 100.0
Missing System 2 6.7
Total 30 100.0
Credit Hours Completed
Frequency Percent Valid Percent
Cumulative
Percent
Valid 0 1 3.3 5.9 5.9
4 1 3.3 5.9 11.8
6 2 6.7 11.8 23.5
7 2 6.7 11.8 35.3
9 2 6.7 11.8 47.1
18 1 3.3 5.9 52.9
21 1 3.3 5.9 58.8
28 1 3.3 5.9 64.7
30 1 3.3 5.9 70.6
34 1 3.3 5.9 76.5
36 2 6.7 11.8 88.2
43 1 3.3 5.9 94.1
56 1 3.3 5.9 100.0
Total 17 56.7 100.0
Missing System 13 43.3
Total 30 100.0
22
Appendix F: Overall Doctoral Seminar Evaluations
Registration Was Organized
Frequency Percent Valid Percent
Cumulative
Percent
Valid Neither Agree or Disagree 1 3.3 3.3 3.3
Somewhat Agree 1 3.3 3.3 6.7
Agree 10 33.3 33.3 40.0
Strongly Agree 18 60.0 60.0 100.0
Total 30 100.0 100.0
Registration Was Efficient
Frequency Percent Valid Percent
Cumulative
Percent
Valid Neither Agree or Disagree 2 6.7 6.7 6.7
Somewhat Agree 1 3.3 3.3 10.0
Agree 9 30.0 30.0 40.0
Strongly Agree 18 60.0 60.0 100.0
Total 30 100.0 100.0
Relevant to Programmatic Needs
Frequency Percent Valid Percent
Cumulative
Percent
Valid Neither Agree or Disagree 1 3.3 3.3 3.3
Somewhat Agree 5 16.7 16.7 20.0
Agree 10 33.3 33.3 53.3
Strongly Agree 14 46.7 46.7 100.0
Total 30 100.0 100.0
23
Appendix F: Overall Doctoral Seminar Evaluations (continued)
Relevant to Individual Needs
Frequency Percent Valid Percent
Cumulative
Percent
Valid Neither Agree or Disagree 1 3.3 3.3 3.3
Somewhat Agree 5 16.7 16.7 20.0
Agree 8 26.7 26.7 46.7
Strongly Agree 16 53.3 53.3 100.0
Total 30 100.0 100.0
Sessions were for the Appropriate Length of time
Frequency Percent Valid Percent
Cumulative
Percent
Valid Disagree 1 3.3 3.3 3.3
Neither Agree or Disagree 1 3.3 3.3 6.7
Somewhat Agree 2 6.7 6.7 13.3
Agree 7 23.3 23.3 36.7
Strongly Agree 19 63.3 63.3 100.0
Total 30 100.0 100.0
Networking Opportunities were Available
Frequency Percent Valid Percent
Cumulative
Percent
Valid Somewhat Disagree 1 3.3 3.3 3.3
Somewhat Agree 4 13.3 13.3 16.7
Agree 7 23.3 23.3 40.0
Strongly Agree 18 60.0 60.0 100.0
Total 30 100.0 100.0
24
Appendix F: Overall Doctoral Seminar Evaluations (continued)
Facilities met my needs
Frequency Percent Valid Percent
Cumulative
Percent
Valid Agree 10 33.3 33.3 33.3
Strongly Agree 20 66.7 66.7 100.0
Total 30 100.0 100.0
Food and Beverages were satisfactory
Frequency Percent Valid Percent
Cumulative
Percent
Valid Somewhat Disagree 1 3.3 3.3 3.3
Neither Agree or Disagree 3 10.0 10.0 13.3
Agree 7 23.3 23.3 36.7
Strongly Agree 19 63.3 63.3 100.0
Total 30 100.0 100.0
Understanding Increased
Frequency Percent Valid Percent
Cumulative
Percent
Valid Neither Agree or Disagree 4 13.3 13.3 13.3
Somewhat Agree 5 16.7 16.7 30.0
Agree 10 33.3 33.3 63.3
Strongly Agree 11 36.7 36.7 100.0
Total 30 100.0 100.0
25
Appendix F: Overall Doctoral Seminar Evaluations (continued)
Keynote Speaker would improve the experience
Frequency Percent Valid Percent
Cumulative
Percent
Valid Strongly disagree 1 3.3 3.4 3.4
Disagree 3 10.0 10.3 13.8
Neither Agree or Disagree 8 26.7 27.6 41.4
Somewhat Agree 3 10.0 10.3 51.7
Agree 5 16.7 17.2 69.0
Strongly Agree 9 30.0 31.0 100.0
Total 29 96.7 100.0
Missing System 1 3.3
Total 30 100.0
26
Appendix F: Overall Doctoral Seminar Evaluations (continued)
Major Benefits of Attending
Frequency Percent Valid Percent
Cumulative
Percent
Valid 18 60.0 60.0 60.0
Better than last year 1 3.3 3.3 63.3
Gather info about program.
See faculty and students that
I haven't seen in a while.
1 3.3 3.3 66.7
Had a good time 1 3.3 3.3 70.0
Hearing from other students
and learning about their
experiences; It helps to hear
others and how they dealt
with situations
1 3.3 3.3 73.3
Learning about the next
steps in the program.
1 3.3 3.3 76.7
Meeting students; Learning
about the process
1 3.3 3.3 80.0
Networking and seeing other
students/staff
1 3.3 3.3 83.3
Real helpful. Thank you! 1 3.3 3.3 86.7
Talking to other students and
faculty.
1 3.3 3.3 90.0
Thank you for all of your
hard work!!
1 3.3 3.3 93.3
Very Beneficial! 1 3.3 3.3 96.7
Very informative regarding
portfolio process
1 3.3 3.3 100.0
Total 30 100.0 100.0
27
Appendix F: Overall Doctoral Seminar Evaluations (continued)
Suggestions for Improvement
Frequency Percent Valid Percent
Cumulative
Percent
Valid 26 86.7 86.7 86.7
I enjoyed the seminar.
Excellent options for
seminars to at attend. Great
Job!
1 3.3 3.3 90.0
Info from other sessions
would be helpful
1 3.3 3.3 93.3
Please make sure everyone
is informed about the date
1 3.3 3.3 96.7
Shorten 1 3.3 3.3 100.0
Total 30 100.0 100.0
28
Appendix G: Individual Session Evaluations
Distribution Table of Session Attendance
Session Title Frequency Percent Valid Percent
Cumulative
Percent
Valid Higher Ed Policy Updates 8 6.1 6.1 6.1
Current Issues: C&I w/
Meisel
4 3.1 3.1 9.2
IRB 5 3.8 3.8 13.0
Crowdsourcing Research 12 9.2 9.2 22.1
Current Issues: Higher Ed &
K-12
7 5.3 5.3 27.5
3-D Printing Tour 5 3.8 3.8 31.3
K-12 Policy Updates &
Implications
10 7.6 7.6 38.9
Current Issues:C&I
w/Hammond
7 5.3 5.3 44.3
Intro to Doc Studies 9 6.9 6.9 51.1
Student Panel 8 6.1 6.1 57.3
Survey Monkey 8 6.1 6.1 63.4
Makey-Makey 6 4.6 4.6 67.9
Library Skills 9 6.9 6.9 74.8
Career Development after
Dissertations
5 3.8 3.8 78.6
Ed Tech 8 6.1 6.1 84.7
Reflectons on the
Dissertaton Process
5 3.8 3.8 88.5
Reflections on the Portfolio
Process
15 11.5 11.5 100.0
Total 131 100.0 100.0
29
Appendix G: Individual Session Evaluations (continued)
Prior Expertise
Frequency Percent Valid Percent
Cumulative
Percent
Valid Basic 30 22.9 25.4 25.4
Basic to Intermediate 17 13.0 14.4 39.8
Intermediate 39 29.8 33.1 72.9
Intermediate to Advanced 19 14.5 16.1 89.0
Advanced 13 9.9 11.0 100.0
Total 118 90.1 100.0
Missing System 13 9.9
Total 131 100.0
Presentation Organized
Frequency Percent Valid Percent
Cumulative
Percent
Valid Somewhat Disagree 2 1.5 1.5 1.5
Somewhat Agree 7 5.3 5.3 6.9
Agree 31 23.7 23.7 30.5
Strongly Agree 91 69.5 69.5 100.0
Total 131 100.0 100.0
Efficient Use of Time
Frequency Percent Valid Percent
Cumulative
Percent
Valid Neither Agree or Disagree 2 1.5 1.5 1.5
Somewhat Agree 3 2.3 2.3 3.8
Agree 27 20.6 20.6 24.4
Strongly Agree 99 75.6 75.6 100.0
Total 131 100.0 100.0
30
Appendix G: Individual Session Evaluations (continued)
Efficient Style & Visual Aids
Frequency Percent Valid Percent
Cumulative
Percent
Valid Neither Agree or Disagree 5 3.8 3.8 3.8
4.5 1 .8 .8 4.6
Somewhat Agree 8 6.1 6.2 10.8
Agree 21 16.0 16.2 26.9
Strongly Agree 95 72.5 73.1 100.0
Total 130 99.2 100.0
Missing System 1 .8
Total 131 100.0
Speaker Knowledgeable
Frequency Percent Valid Percent
Cumulative
Percent
Valid Somewhat Agree 3 2.3 2.3 2.3
Agree 23 17.6 17.6 19.8
Strongly Agree 105 80.2 80.2 100.0
Total 131 100.0 100.0
Speaker Responsive to Questions
Frequency Percent Valid Percent
Cumulative
Percent
Valid Somewhat Agree 2 1.5 1.5 1.5
Agree 17 13.0 13.0 14.5
Strongly Agree 112 85.5 85.5 100.0
Total 131 100.0 100.0
31
Appendix G: Individual Session Evaluations (continued)
Would Recommend Speaker
Frequency Percent Valid Percent
Cumulative
Percent
Valid Neither Agree or Disagree 3 2.3 2.3 2.3
Somewhat Agree 4 3.1 3.1 5.3
Agree 16 12.2 12.2 17.6
Strongly Agree 108 82.4 82.4 100.0
Total 131 100.0 100.0
Session Contributed to Overall My Knowledge
Frequency Percent Valid Percent
Cumulative
Percent
Valid Neither Agree or Disagree 5 3.8 3.8 3.8
Somewhat Agree 5 3.8 3.8 7.7
Agree 19 14.5 14.6 22.3
Strongly Agree 101 77.1 77.7 100.0
Total 130 99.2 100.0
Missing System 1 .8
Total 131 100.0
32
Appendix G: Individual Session Evaluations (continued)
Areas of Growth
Frequency Percent Valid Percent
Cumulative
Percent
Valid 67 51.1 51.1 51.1
#edtech 1 .8 .8 51.9
7 1 .8 .8 52.7
7Great hand-outs, Great
survey on Board Members
1 .8 .8 53.4
A fantastic model for me to
know how to organize and
structure my portfolio
1 .8 .8 54.2
A foundation of basic info.
As I had never heard of this
before.
1 .8 .8 55.0
Background about the
common core standards
history was increased.
1 .8 .8 55.7
benefits of social media 1 .8 .8 56.5
Beth is great! 1 .8 .8 57.3
better idea of how to
approach IRB approval
1 .8 .8 58.0
CV development 1 .8 .8 58.8
Educational policy reform 1 .8 .8 59.5
Endnote 1 .8 .8 60.3
Examples were very helpful.
Makes this process less
intimidating.
1 .8 .8 61.1
Excellent 1 .8 .8 61.8
exposure to different
technology,
1 .8 .8 62.6
Found the topics comforting
to my current feelings in the
process of the doctoral
program
1 .8 .8 63.4
33
Gave me more knowledge to
where Ohio might be
headed. We need to turn
the triangle upside down too!
1 .8 .8 64.1
Gave me more to think about
when I start my study.
1 .8 .8 64.9
Good information o the CV.
Good handouts.
1 .8 .8 65.6
Great (possible) research
technique
1 .8 .8 66.4
Great discussions and
thoughtful presenters on
pitfalls and shortcomings in
crowdsourcing.
1 .8 .8 67.2
Great info 1 .8 .8 67.9
Great workshop! 1 .8 .8 68.7
Helped me see possibilities
of 3D printing
1 .8 .8 69.5
I had never seen this before.
Very new technology to me.
1 .8 .8 70.2
I have limited knowledge. I
find this interesting and
wanted to learn more
1 .8 .8 71.0
I knew nothing beforehand
about the policies and how
they impact West Virginia
students. He needs to come
back for future seminars.
1 .8 .8 71.8
I learned a great deal about
crowdsourcing and issues
related to application.
1 .8 .8 72.5
I learned more about the
chair process.
1 .8 .8 73.3
I loved hearing what people
had to say. I am very
interested in this area.
1 .8 .8 74.0
I understand more about C&I 1 .8 .8 74.8
34
I understood the portfolio a
little better
1 .8 .8 75.6
I want one 1 .8 .8 76.3
IDS and EZ Borrow 1 .8 .8 77.1
Importance of differentiation
in teaching methods and
student work.
1 .8 .8 77.9
Improved knowledge in
standards forming and 2510
1 .8 .8 78.6
Indifferent, because the
program seems to be in the
infancy stages, therefore, I
am unsure about it's validity.
1 .8 .8 79.4
Inductive teaching - I learned
more about the topic of
inductive teaching.
1 .8 .8 80.2
Information regarding policy
in higher education in West
Virginia.
1 .8 .8 80.9
Interesting discussion
regarding integrating social
media into the classroom.
1 .8 .8 81.7
Knowledge of WV HE Policy 1 .8 .8 82.4
Learned about many helpful
resources
1 .8 .8 83.2
learned more about the topic 1 .8 .8 84.0
Learning about new tools.
Little bits.
1 .8 .8 84.7
Learning about policies and
where the WVDE is going in
the future to empower
teachers and increase
student learning. The
session had a powerful
message.
1 .8 .8 85.5
35
Learning little tricks how to
make makey makey more
user friendly: bracelet to
gmuadring, use of washers,
etc.
1 .8 .8 86.3
loved this! Great to get to
see how it works & play a
little with it.
1 .8 .8 87.0
Most fun I had all day :-) 1 .8 .8 87.8
New knowledge about
electronic portals to use and
the process. Helpful tips.
1 .8 .8 88.5
None 1 .8 .8 89.3
PortaPortal 1 .8 .8 90.1
Preparing ahead of time is
the key to having a smooth
process in writing the paper.
1 .8 .8 90.8
Provided very good
information for preparing me
for portfolio
1 .8 .8 91.6
Questions answered of
artifacts process
1 .8 .8 92.4
Really interesting. Re-
emphasized a lot of info we
learned in CI 707.
1 .8 .8 93.1
Received much needed
examples concerning
portfolio.
1 .8 .8 93.9
stronger knowledge base of
IRB process
1 .8 .8 94.7
Thank you for the personal
growth experiences you
shared w/ us!
1 .8 .8 95.4
This answered so many
questions!
1 .8 .8 96.2
Understanding of data
sources
1 .8 .8 96.9
36
Understanding of
requirements within program
1 .8 .8 97.7
Understanding
comprehensively HEPC
organization and the status
of state financially. Future
policy decision at institutions
1 .8 .8 98.5
Very good connection to
classroom use.
1 .8 .8 99.2
Very great material! 1 .8 .8 100.0
Total 131 100.0 100.0
37
Appendix G: Individual Session Evaluations (continued)
Recommendations to Speaker
Frequency Percent Valid Percent
Cumulative
Percent
Valid 98 74.8 74.8 74.8
Deb & Beth are great! 1 .8 .8 75.6
Excellent! 1 .8 .8 76.3
Great Job! 1 .8 .8 77.1
Great job!! Very informative 1 .8 .8 77.9
Great session - visual aids
always help me :-)
1 .8 .8 78.6
Great! 1 .8 .8 79.4
longer session 1 .8 .8 80.2
More info as a requestor. 1 .8 .8 80.9
more time per session 1 .8 .8 81.7
N/A 2 1.5 1.5 83.2
Nice job! Thanks! 1 .8 .8 84.0
None 3 2.3 2.3 86.3
None-Thank you for the
information.
2 1.5 1.5 87.8
None-Thank you so much. I
really needed this
information. It was so
helpful.
1 .8 .8 88.5
None-wonderful, engaging
presentation. I love your
passion for teaching!
1 .8 .8 89.3
None; perfect 1 .8 .8 90.1
None! 1 .8 .8 90.8
Outstanding and
inspirational!
1 .8 .8 91.6
38
Overall, very well run.
Would have liked more time
at the end for discussion.
1 .8 .8 92.4
Show us on the website
where to get forms, etc.
1 .8 .8 93.1
This was a great session &
should be included each
year - it should be suggested
as "idea topic starters" for a
study
1 .8 .8 93.9
Very broad topic for the
allotted time. I would only
discuss 1 issue; rather than
a few….
1 .8 .8 94.7
Very enjoyable and
informative
1 .8 .8 95.4
very good~ 1 .8 .8 96.2
Very interesting. Would like
more discussion although it
picked up at the end. Best
of luck with these initiatives.
1 .8 .8 96.9
Visuals not interesting (black
and white PowerPoint)
1 .8 .8 97.7
Well done - thank you! 1 .8 .8 98.5
Well done! Thank you! 1 .8 .8 99.2
Yes-Extremely Helpful! 1 .8 .8 100.0
Total 131 100.0 100.0
39
Appendix G: Individual Session Evaluations (continued)
Case Processing Summary
Cases
Included Excluded Total
N Percent N Percent N Percent
Prior Expertise * Session
Title
118 90.1% 13 9.9% 131 100.0%
Presentation Organized *
Session Title
131 100.0% 0 0.0% 131 100.0%
Efficient Use of Time *
Session Title
131 100.0% 0 0.0% 131 100.0%
Efficient Style & Visual Aids
* Session Title
130 99.2% 1 0.8% 131 100.0%
Speaker Knowledgeable *
Session Title
131 100.0% 0 0.0% 131 100.0%
Speaker Responsive to
Questions * Session Title
131 100.0% 0 0.0% 131 100.0%
Would Recommend Speaker
* Session Title
131 100.0% 0 0.0% 131 100.0%
Session Contributed to
Overall My Knowledge *
Session Title
130 99.2% 1 0.8% 131 100.0%
Areas of Growth * Session
Title
131 100.0% 0 0.0% 131 100.0%
Recommendations to
Speaker * Session Title
131 100.0% 0 0.0% 131 100.0%
40
Appendix G: Individual Session Evaluations (continued)
Session Title
Prior
Expertise
Presentation
Organized
Efficient
Use of
Time
Efficient Style
& Visual Aids
Speaker
Knowledgeable
Would
Recommend
Speaker
Session
Contributed to
Overall My
Knowledge
Higher Ed Policy
Updates
Mean 3.71 6.13 6.63 6.125 6.88 6.75 6.75
N 7 8 8 8 8 8 8
Std. Deviation 1.496 .835 .518 1.1260 .354 .463 .463
Variance 2.238 .696 .268 1.268 .125 .214 .214
Current Issues: C&I
w/ Meisel
Mean 3.33 7.00 7.00 7.000 7.00 7.00 7.00
N 3 4 4 4 4 4 4
Std. Deviation 2.082 .000 .000 .0000 .000 .000 .000
Variance 4.333 .000 .000 .000 .000 .000 .000
IRB Mean 2.80 6.60 6.80 6.750 7.00 7.00 7.00
N 5 5 5 4 5 5 5
Std. Deviation .447 .894 .447 .5000 .000 .000 .000
Variance .200 .800 .200 .250 .000 .000 .000
Crowdsourcing
Research
Mean 1.67 6.67 6.75 6.000 6.50 6.67 6.58
N 12 12 12 12 12 12 12
Std. Deviation .888 .492 .452 1.2060 .522 .651 .900
Variance .788 .242 .205 1.455 .273 .424 .811
Current Issues:
Higher Ed & K-12
Mean 4.33 6.29 6.43 6.571 6.57 6.57 6.00
N 6 7 7 7 7 7 7
Std. Deviation .816 .951 .787 .7868 .535 .787 1.291
Variance .667 .905 .619 .619 .286 .619 1.667
41
3-D Printing Tour Mean 1.80 6.60 6.60 6.800 6.40 6.60 6.80
N 5 5 5 5 5 5 5
Std. Deviation 1.304 .548 .548 .4472 .548 .548 .447
Variance 1.700 .300 .300 .200 .300 .300 .200
K-12 Policy Updates
& Implications
Mean 3.13 6.80 6.80 6.700 6.90 6.90 6.90
N 8 10 10 10 10 10 10
Std. Deviation .991 .422 .422 .4830 .316 .316 .316
Variance .982 .178 .178 .233 .100 .100 .100
Current Issues: C&I
w/Hammond
Mean 3.43 6.71 6.71 6.857 6.86 6.86 6.86
N 7 7 7 7 7 7 7
Std. Deviation .535 .488 .488 .3780 .378 .378 .378
Variance .286 .238 .238 .143 .143 .143 .143
Intro to Doc Studies Mean 2.11 6.78 6.89 6.667 6.89 6.89 6.89
N 9 9 9 9 9 9 9
Std. Deviation 1.054 .441 .333 .7071 .333 .333 .333
Variance 1.111 .194 .111 .500 .111 .111 .111
Student Panel Mean 2.57 6.25 6.25 6.313 6.75 6.63 6.25
N 7 8 8 8 8 8 8
Std. Deviation 1.618 1.389 1.165 1.2800 .707 1.061 1.165
Variance 2.619 1.929 1.357 1.638 .500 1.125 1.357
Survey Monkey Mean 2.88 6.63 6.75 6.625 6.88 6.75 6.50
N 8 8 8 8 8 8 8
Std. Deviation 1.458 .518 .463 .5175 .354 .463 1.069
Variance 2.125 .268 .214 .268 .125 .214 1.143
Makey-Makey Mean 2.00 6.83 7.00 7.000 7.00 7.00 7.00
N 4 6 6 6 6 6 6
Std. Deviation 1.155 .408 .000 .0000 .000 .000 .000
42
Variance 1.333 .167 .000 .000 .000 .000 .000
Library Skills Mean 2.75 6.78 6.89 6.889 7.00 7.00 7.00
N 8 9 9 9 9 9 9
Std. Deviation 1.165 .441 .333 .3333 .000 .000 .000
Variance 1.357 .194 .111 .111 .000 .000 .000
Career Development
after Dissertations
Mean 3.50 6.40 6.40 6.200 6.60 6.40 6.20
N 4 5 5 5 5 5 5
Std. Deviation 1.000 .548 .548 .8367 .548 .894 .447
Variance 1.000 .300 .300 .700 .300 .800 .200
Ed Tech Mean 2.14 6.38 6.25 6.375 6.38 6.25 6.25
N 7 8 8 8 8 8 8
Std. Deviation 1.464 .916 1.165 .9161 .916 1.165 1.165
Variance 2.143 .839 1.357 .839 .839 1.357 1.357
Reflectons on the
Dissertaton Process
Mean 3.25 5.80 6.60 6.200 6.60 6.20 6.40
N 4 5 5 5 5 5 5
Std. Deviation 1.708 1.643 .548 1.3038 .548 1.304 .894
Variance 2.917 2.700 .300 1.700 .300 1.700 .800
Reflections on the
Portfolio Process
Mean 2.57 6.93 6.93 6.933 6.93 6.93 6.79
N 14 15 15 15 15 15 14
Std. Deviation 1.016 .258 .258 .2582 .258 .258 .426
Variance 1.033 .067 .067 .067 .067 .067 .181
Total Mean 2.73 6.60 6.70 6.581 6.78 6.75 6.66
N 118 131 131 130 131 131 130
Std. Deviation 1.305 .731 .591 .7944 .469 .624 .732
Variance 1.704 .535 .349 .631 .220 .390 .536