FDP301x Mentoring Educators in Educational Technology€¦ · Evaluation Report for FDP Design Page...
Transcript of FDP301x Mentoring Educators in Educational Technology€¦ · Evaluation Report for FDP Design Page...
Evaluation Report for FDP Design
Page 1
FDP301x
Mentoring Educators in Educational
Technology
EVALUATION REPORT FOR
FDP DESIGN ACTIVITY on
“Active Learning: Think-Pair-Share and Peer
Instruction”
Sunita Milind Dol: [email protected]
Neha Sahu: [email protected]
Mercy Shalinie S: [email protected]
Vaibhav Singh: [email protected]
This work is licensed under the Creative Commons Attribution 4.0 International
License. To view a copy of this license, visit
http://creativecommons.org/licenses/by/4.0/.
Evaluation Report for FDP Design
Page 2
Table of Content
Content Page No.
ANALYSIS OF ENGAGEMENT 3
A) What was the gross engagement in the course? [Macro level] 3
B) What was the engagement pattern of individual LeDs, LbDs, LxIs, and LxTs?
[Micro Level]
5
C) What is the transition pattern of engagement of learners across LeD, LbD, LxI, and
LxT?
6
ANALYSIS OF LEARNING 7
A) What is the gross learning in the course?[Macro] 7
B) What is the learning pattern across LbDs and Quizzes? [Micro] 7
C) What is the transition pattern of learning across LbDs and Quizzes? [Meso Level] 8
ANALYSIS OF PERCEPTION 11
What was the perception of learners about the course? [Macro] 11
REFLECTIONS ON ANALYSIS 13
What do gross data about learning, perception and engagement tell about the course? 13
What do the analysis tell about the course experience? 13
What needs improvement in the current course offering? 13
WORKLOAD DISTRIBUTION 14
MENTORS INTERACTION DETAILS 16
Evaluation Report for FDP Design
Page 3
ANALYSIS OF ENGAGEMENT
A) What was the gross engagement in the course? [Macro level]
The MOODLE course on Active Learning: Think-Pair-Share and Peer Instruction Activity was
originally designed as part of FDP 301x Mentoring Educators in Educational Technology. There
were around 12 particiapnts assigned by IIT, Bombay who were the participants of this FDP 301x.
These 12 particiapants assessed the course during 20th
June to 27th
June 2018 . The course involved the
following activities that were to be completed by participants during 20th
June to 27th
June 2018:
Topic: Active Learning: Think-Pair-Share and Peer Instruction Activity
Welcome Note
Course Entry Survey
Introduction to Active Learning: Think-Pair-Share and Peer Instruction Activity
Learning Objectives of Active Learning: Think-Pair-Share and Peer Instruction Activity
Key Concepts
Active Learning: Think-Pair-Share and Peer Instruction Activity
o LeD 1- Active Learning
o LbD 1 -Active Learning
o LeD 2 – Think-Pair-Share (TPS) Activity – What
o LbD 2 - Think-Pair-Share (TPS) Activity – What
o LeD 3 – Think-Pair-Share (TPS) Activity – How
o LbD 3 - Think-Pair-Share (TPS) Activity – How
o LeD 4 – Peer Instruction (PI) Activity – What
o LbD 4 - Peer Instruction (PI) Activity – What
o LeD 5 – Peer Instruction (PI) Activity – How
o LbD 5 - Peer Instruction (PI) Activity – How
o Knowledge Quiz
Discussion and Reflection Quiz
o Discussion Forum on TPS Activity
o Reflection Quiz on TPS Activity
o Discussion Forum on PI Activity
o Reflection Quiz on PI Activity
Advancing Your Learning
o LxT Video 1: Think-Pair-Share for Active Learning
o LxT Video 2: Think-Pair-Share Activity Example
o LxT Video 3: Peer Instruction Activity
o LxT Video 4: Peer Instruction Activity Example
o Assimilation Quiz on on LxT Video 1 and LxT Video 2
o Assimilation Quiz on on LxT Video 3 and LxT Video 4
Designing your own TPS, PI Activity and Assessment
o Designing your own TPS Activity
o TPS Activity Constructor
o TPS Example
Evaluation Report for FDP Design
Page 4
o Resource Creation Assignment on TPS
o Designing your own PI Activity
o PI Example
o Resource Creation Assignment on PI
Evaluation and Feedback
Acknowledgement and References
About Team-120
The activity completion report generated from the course Reports section shows the engagement of
participants in the activities as shown in Table 1.
Table 1: Activity Completion Summary
Activity Completion Status
Course Entry
Survey
11
LeDs 1 11
2 11
3 11
4 11
5 11
LbDs 1 11
2 11
3 11
4 11
5 11
KQ 12
Discussion on TPS
activity
11
Discussion on PI
activity
11
RQ1 10
RQ2 10
LxT1 11
LxT2 10
LxT3 10
LxT4 11
AQ1 10
AQ2 10
Design Your Own
TPS
2
Design Your Own PI 3
Peer Review of TPS 2
Peer Review of PI 2
Evaluation Report for FDP Design
Page 5
B) What was the engagement pattern of individual LeDs, LbDs, LxIs, and LxTs? [Micro Level]
Looking into the performance and engagement of individuals in the course activities, following is the
engagement pattern of selected participants in the course. Twelve participants were selected for the
analysis so that we have a sample groups showing high engagement, medium engagement and low
engagement pattern.
Table 2 shows the level of engagement of the twelve sampled participants in the course.
Table 2: Participation of individual learners in the course
Activity P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12
Course
Entry
Survey
Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
LeDs 1 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
2 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
3 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
4 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
5 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
LbDs 1 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
2 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
3 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
4 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
5 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
KQ Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
Discussion
on TPS
activity
Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
Discussion
on TPS
activity
Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
RQ1 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes
RQ2 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes
LxT1 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
LxT2 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes
LxT3 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes
LxT4 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes
AQ1 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes
AQ2 Yes No Yes Yes Yes Yes Yes Yes No Yes Yes Yes
Design
Your Own
TPS
Yes Yes No No Yes Yes Yes No Yes Yes No Yes
Design
Your Own
PI
Yes Yes No No Yes Yes Yes No No Yes No Yes
Evaluation Report for FDP Design
Page 6
C) What is the transition pattern of engagement of learners across LeD, LbD, LxI, and LxT?
Evaluation Report for FDP Design
Page 7
ANALYSIS OF LEARNING
A) What is the gross learning in the course?[Macro]
Analysis of the learning of the students is done by analysing the marks obtained by the participants in
attempting various quizzes and activities. There were five types of quizzes and activities for the
participants.
The average marks for each of them is given below.
Learning by Doing (LbD) Activities- 5 Nos [10 marks]
Reflection Quiz- 2 Nos [20 marks]
Assimilation Quiz- 2 Nos [20 marks]
Knowledge Quiz [20 marks]
Think-Pair-Share and Peer Instruction Design Activity [30 marks]
Total 100 marks
Table 3: Number of participants who attempted each activity and the average marks and percentage
they scored.
LbDs KQ RQ AQ Activity
(TPS)
Activity
(PI)
Any All
No of
Persons
attempted
11 12 10 10 2 3 10
2
(including
RCA on
TPS and
PI)
Average
marks
6.67 12.33 9.79 7.17 20 10 48.33 70.5
Maximum
marks
10 20 20 20 20 10 100 100
Average
percentage
48.33 70.5
B) What is the learning pattern across LbDs and Quizzes? [Micro]
Table 4: Number of participants who have attempted each LbDs. The total is calculated in 10 as it was
the average given to the LbDs in the whole course scoring.
LbDs LbD1 LbD2 LbD3 LbD4 LbD5 Any of the
LbDs
All of the
LbDs
No of
Persons
attempted
11 11 11 11 11 0 11
Average
marks
1.25 1.42 1.08 1.58 1.33 0 72.7
Maximum
marks
2 2 2 2 2 0 2 (Each
LbD having
2 marks)
Average
percentage
62.5 71 54 79 66.5 0 66.6
Evaluation Report for FDP Design
Page 8
Table 5: Number of participants who submitted the Think-Pair-Share (TPS) and Peer Instruction
design (PI) activity, peer reviewed these activities and the marks they have been given. The total is
calculated in 30 as it was the average marks given to the Design activity in the whole course scoring.
PI Design
Activity
Submission
Peer Review
of PI Design
Activity
TPS
Design
Activity
Submission
Peer
Review of
TPS Design
Activity
Total Marks
No of persons
attempted
6 4 6 14 30
Average
Marks
6 4 6 14 30
Maximum
Marks
6 4 6 14 14
Average
Percentage
100 100 100 100 100
C) What is the transition pattern of learning across LbDs and Quizzes? [Meso Level - It would
be good to use iSAT here]
Evaluation Report for FDP Design
Page 9
Evaluation Report for FDP Design
Page 10
Evaluation Report for FDP Design
Page 11
ANALYSIS OF PERCEPTION
What was the perception of learners about the course? [Macro]
We can analyse the overall perception of learners through their feedback in the three surveys[5]
:
Perception of Learning
Perception of Engagement
Perception of Usability
Perception of Learning
For each of the parameters to analyse perception of learning, we can see that the participants seem to
have a good perception about the learning offered by the course.
Talking about the learning avenue offered by additional resources/reference materials, most of the
learners are strongly agree about their learning potential.
Fig 1. Perception of Learning
Evaluation Report for FDP Design
Page 12
Perception of Engagment
It is clearly evident that the perception of engagement has been high.
Observing each criteria, we can find that the participants are easily able to get actively involved each
time they access the course and find the additional resources/reference materials engaging enough.
Fig 2. Perception of Engagement
Perception of Usability
Here’s a visual summary of the survey responses of participants.
Fig 3. Perception of Usability
From the graph, we can analyse each of the 9 categories used to evaluate the usability and further drill
down into the response of each participant.
Going by the bar graphs, we can see that most of the participants have agreed in the range of Mildly
and Strongly on the usability of the course.
Evaluation Report for FDP Design
Page 13
REFLECTIONS ON ANALYSIS
What do gross data about learning, perception and engagement tell about the course?
The gross data about engagement shows that the number of participants completing the
activities/quizzes are good.
Also, from the data on scores of participants, we can see that the scores of the participants are
gradually fluctuating, although the number of participants attempting the final Knowledge Quiz (KQ)
is also good. This indicates that the perceived learning value of the course is good.
From the participants' perception of the course, it can be observed that the pace of the course and
consistency of course material design is good.
What do the analysis tell about the course experience?
The analysis shows that the course was engaging enough and the levels of usability was also good, and
the levels of learning were not bad.
What needs improvement in the current course offering?
The following sections need considerable improvement:
Engagement:
Engagement in Designing your own activity
Usability
Duration of Resource creation activity
Learning
Provide the additional example related to Designing your own activity
Evaluation Report for FDP Design
Page 14
WORKLOAD DISTRIBUTION
The team consisted of 4 participants. The workload is distributed among them as follows:
Member 1: (Team leader – Mrs. Sunita M Dol)
Preparing the workload distribution
Overall Structure of FDP Design course
Active Learning: Think-Pair-Share and Peer Instruction Activity
o LeD 2 – Think-Pair-Share (TPS) Activity – What
o LbD 2 - Think-Pair-Share (TPS) Activity – What
o LeD 3 – Think-Pair-Share (TPS) Activity – How
o LbD 3 - Think-Pair-Share (TPS) Activity – How
o Knowledge quiz
Designing your own TPS, PI Activity and Assessment
o Designing your own TPS Activity
o TPS Activity Constructor
o TPS Example
o Resource Creation Assignment on TPS
o Designing your own PI Activity
o PI Example
o Resource Creation Assignment on PI
Progress Bar
Grading Policies
FDP design analysis report: analysis of course engagement
Member 2 (Neha Sahu)
Active Learning: Think-Pair-Share and Peer Instruction Activity
o LeD 4 – Peer Instruction (PI) Activity – What
o LbD 4 - Peer Instruction (PI) Activity – What
o LeD 5 – Peer Instruction (PI) Activity – How
o LbD 5 - Peer Instruction (PI) Activity – How
o Knowledge Quiz
Discussion and Reflection Quiz
o Discussion Forum on TPS Activity
o Reflection Quiz on TPS Activity
About Team-120
Progress Bar
Grading Policies
FDP design analysis report: analysis of learning
Member 3 (Mercy Shalinie S.)
Introduction to Active Learning: Think-Pair-Share and Peer Instruction Activity
Evaluation Report for FDP Design
Page 15
Learning Objectives of Active Learning: Think-Pair-Share and Peer Instruction
Activity
Key Concepts
Discussion and Reflection Quiz
o Discussion Forum on PI Activity
o Reflection Quiz on PI Activity
Evaluation and Feedback
Acknowledgement and References
Progress Bar
Grading Policies
FDP design analysis report: analysis of learner feedback
Member 4 (Vaibhav Singh)
Active Learning: Think-Pair-Share and Peer Instruction Activity
o LeD 1- Active Learning
o LbD 1 -Active Learning
Advancing Your Learning
o LxT Video 1: Think-Pair-Share for Active Learning
o LxT Video 2: Think-Pair-Share Activity Example
o LxT Video 3: Peer Instruction Activity
o LxT Video 4: Peer Instruction Activity Example
o Assimilation Quiz on on LxT Video 1 and LxT Video 2
o Assimilation Quiz on on LxT Video 3 and LxT Video 4
Acknowledgement and References
Progress Bar
Grading Policies
FDP design analysis report: Self-Reflection based on analysis of the first three points
(what have the team members learnt from the data collected above and your experience)
‘
Evaluation Report for FDP Design
Page 16
MENTORS GROUP INTERACTION DETAILS
Date: 1th
June 2018
Evaluation Report for FDP Design
Page 17
Date: 2
nd June 2018
Evaluation Report for FDP Design
Page 18
Evaluation Report for FDP Design
Page 19
Date: 4th
June 2018
Evaluation Report for FDP Design
Page 20
Date: 5th
June 2018
Evaluation Report for FDP Design
Page 21
Date: 6th
June 2018
Evaluation Report for FDP Design
Page 22
Evaluation Report for FDP Design
Page 23
Date: 7th
June 2018
Evaluation Report for FDP Design
Page 24
Evaluation Report for FDP Design
Page 25
Date: 9th
June 2018
Evaluation Report for FDP Design
Page 26
Date: 10th
June 2018
Evaluation Report for FDP Design
Page 27
Date: 11th
June 2018
Evaluation Report for FDP Design
Page 28
Evaluation Report for FDP Design
Page 29
Evaluation Report for FDP Design
Page 30
Evaluation Report for FDP Design
Page 31
Evaluation Report for FDP Design
Page 32
Date: 12th
June 2018
Evaluation Report for FDP Design
Page 33
Evaluation Report for FDP Design
Page 34
Evaluation Report for FDP Design
Page 35
Evaluation Report for FDP Design
Page 36
Date: 13th
June 2018
Evaluation Report for FDP Design
Page 37
Evaluation Report for FDP Design
Page 38
Evaluation Report for FDP Design
Page 39
Date: 14th
June 2018
Evaluation Report for FDP Design
Page 40
Date: 15th
June 2018