carolinecooneyedtech.files.wordpress.com€¦ · Web viewSpring 2015. Submitted to: Dr ... Over...

Summative Evaluation Report: Teacher use of X2 Aspen as a class Website

Prepared and Submitted byCaroline C CooneyEd Tech 505-4202

Boise State UniversitySpring 2015

Submitted to: Dr John Thompson, Boise State University

Caroline C Cooney, Ed Tech 505-4202, Boise State University, Spring 2015

2

SummaryThe program under evaluation is a teacher’s course website utilizing the district approved

and provided Aspen X2 platform, a secure student information system (SIS) that allows schools to track attendance, enter grades, and create schedules. The SIS also supports classroom webpages, and this teacher piloted and evaluated the use of Aspen X2 as the classroom website.The main objective of the classroom website is to “increase the student’s responsibility for their learning” by giving students access to grades, documents, notes, and weekly/daily agendas. The evaluation seeks to determine if the objectives of the website are being met, and to make suggestions for improving delivery of the course using the website.

The evaluation included a student survey that used forced choice likert scales, multiple choice, multiple response, and open ended questions. All students (146) in the teacher’s classes from trimesters two and three were administered the survey. The results show that students liked using the website, and students responded that using the website helped to increase their feeling of responsibility for their learning. Students survey results showed that not all areas of the website were utilized such as accessing the site when absent, using the site to prepare or review for class.

Based on the evaluation results, the teacher should continue with the website. The teacher should also look at student suggestions for improving the website. Lastly, the teacher should evaluate the goals to determine if the goals for the future use of the website should be adjusted.

Program DescriptionProgram History

Over the past several years, the teacher used an online grading website with her students to communicate grades, and eventually added course content. The platforms used included Engrade, and My Haiku LMS. While the teacher reported that both platforms had pros and cons, the teacher decided to create a class website that utilized the district supported platform Aspen X2, a student information system that includes attendance, grading, and website capabilities. Currently, the high school uses the attendance and grading portions (for entering end of trimester grades), but only a few (less than five) teachers use the website capabilities to communicate assignments, grades, etc to students. The Aspen X2 platform is a secure website, and all students at the high school have a sign-on and password to access the program. Students must access Aspen X2 to select their courses each year. For most students, the yearly course selection process is their only interaction with the Aspen X2 system.

At the beginning of the 2014-2015 academic school year, the teacher switched her grading system to the school provided Aspen X2. During trimester one, the teacher published grades for the students, and set up a pilot class website. The pilot website included a daily agenda, grade information, class notes links, and website links. By the start of the second trimester, the teacher created and “rolled out” a website for each of the classes that she taught: Physics, Earth Science: Atmosphere, Weather, Climate, and Earth Science: Dynamic Earth and Space. The updated website included changes based on first trimester suggestions from students. The revised website gave students access to: weekly/daily agendas, links to websites used in class, links to documents used in class, links to class notes, electronic submission of selected assignments, and class discussion boards. As part of the daily class routine, the teacher pulled the website up on the SmartBoard in the classroom to show students the agenda for the day, what information was available on the website, and where to find the information on the website.


3

Based on feedback from trimester two students, the teacher tweaked the website layout and information, and continued to utilize with students in class during trimester three of the 2014-2015 academic school year. All students in trimester two Physics, Earth Science: Atmosphere, Weather, Climate, and Earth Science: Dynamic Earth and Space and all students in Trimester three Physics and Earth Science: Dynamic Earth and Space had access to the website, and all students were given the link to the online survey on website use.

Program ObjectivesThe objectives of the website were developed by the teacher based on her observations of

classes, and based on prior years’ use of other website platforms. Prior year students overwhelmingly reported that they liked having a class website, and that they primarily used website to view daily agenda/assignments, and to check grades. They reported that they did not use the website very much for content or for accessing class documents. The teacher’s objective in creating a class website was to help students be more independent, and to help the students take more ownership of their schoolwork.

The objectives of creating the class website are to: 1. increase student ownership for their work, 2. increase student interaction with the course content, 3. give students the ability to prepare for class, 4. give students the ability to review the material from prior lessons, 5. give students access to information when absent from class,6. give students access to current grades.

Program ComponentsThe program (website) was designed and piloted during the first trimester of the 2014-

2015 academic school year. Based on student feedback, the website was tweaked and rolled out for full use for trimesters two and three. The website current design includes four tabs (the maximum allowed by Aspen X2). Refer to Appendix A for screenshots of the website. The tabs include:

● “This week in class”: a page that includes the current weekly/daily agenda, as well as prior weeks. This page is the “hub” of the website, and students learn what we will do each day , as well as access web links and documents needed for class. Also, with each week is posted a visual and/or problem to help understand the topics being discussed that week in class.

● “Course Resource”: a page where students can access all web links and documents organized by content.

● “Grades”: a page where students can see their average, submit electronic assignments, and participate in online discussions.

● General course information including policy, procedure, etc. Students were taught how to log on to the website, and how to navigate the website the

first week of the trimester. Additionally, the teacher pulled the class website up to the SmartBoard each day in class to review daily/weekly agenda, to show students where and how to access information, and to model using the website. Students were encouraged throughout both trimesters to regularly check the class website both during class and outside of class. Any assignments that involve using the internet have links posted on the class website, and students


4

are encouraged to access the links from the class website rather than typing in the URL address. Students are encouraged to check website for homework, to prepare for upcoming classes, to review material, and to check for work when absent. Selected assignments may be electronically submitted.

Evaluation MethodThe evaluation model chosen for the evaluation is the goal-based method as the evaluation seeks to determine if the program objectives are being met, and to make suggestions for improvement.

ParticipantsAll students in trimester two Physics, Earth Science: Atmosphere, Weather, Climate, and

Earth Science: Dynamic Earth and Space and all students in Trimester three Physics, and Earth Science: Dynamic Earth and Space were involved in the evaluation of the class website. All students had access to the website, and all students were given the link to the online survey (on the class website). All students were asked to complete the survey. There were 64 students enrolled in trimester two, and 82 students enrolled in trimester three for a total of 146 student participants. The break out by class is as follows:

Class Trimester TwoNumber of students

Trimester Three Number of students

Total Tri 2 & 3 Number of Students

Physics 30 43 73

Earth Science: Atmosphere, Weather, Climate

14 0 14

Earth Science: Dynamic Earth and Space

20 39 59

Total 64 82 146

ProcedureDuring the two trimesters of the evaluation period, students were given explicit

instructions about how to access the website and related course materials. Trimester two students were administered the survey via a link on the class website when the class met after the final exam. Trimester two students accessed the website for the full twelve weeks. Trimester three students were administered the survey via a link on the class website four weeks into the trimester.

Data SourceThe main data source is a survey created in Google Forms, and administered through the

Aspen X2 class website. The same survey was administered to both trimester two and trimester three classes (refer to Appendix B). The evaluation questions developed as a result of the objectives are:


5

1. Have students taken more responsibility for their learning as a result of the website?A question on the survey used a forced choice likert scale (1 strongly disagree to 4 strongly agree) and asked the student if using the website helped the student take more responsibility for their learning. Another related question asked if using the class website made the students feel more in control of their learning.

2. Have students increased the amount of time they spend interacting with content by using the website?

Again, using a forced likert scale, the survey asked if using the class website increased the amount of time students spend interacting with course content compared to their “normal or average” amount of time.

3. Do students use the website to prepare for class?4. Do students use the website to review material from past lessons/classes?5. Do students use the website when absent to find out what was missed?6. Do students use the website to check their grades and monitor their progress in class?

Question three through six were addressed in the surveys using a frequency scale regarding how often they used the website for each of the stated purposes. Also, questions three through six were addressed using a multiple response question asking students which areas they accessed. Lastly, the survey asked students for open ended responses to describe what content and layout changes would make the website better.

ResultsThe response rate to the survey was good overall with 75% of trimester two students responding, and 60% of trimester three students responding for a total response rate of 66%. Survey Results by ObjectiveEach objective will be looked at in terms of the survey results.Objective One: Increase Student Ownership for their workStudents from both trimesters two and trimester three were in agreement with 73% responding agree or strongly agree that using the website increased their responsibility for their work.


6

Additionally 75% of students from both trimesters agreed or strongly agreed that using the website made them feel in control of their learning.

Objective Two: Increase student interaction with the course contentApproximately 57% of survey respondents either agreed or strongly agreed with the statement that using the website increased the amount of time spent interacting with the course content.


7

Objective Three: Give students ability to prepare for classApproximately 32% of students report using the website weekly or more to prepare for class. While 49% report using the website once per month or less to prepare for clasObjective Four: Give Students ability to review material from prior classesAlmost 25% of students never visit the website to review prior material, but approximately 25% do visit the website at least once per week to review material from prior classes.

Objective Six: Give Students access to gradesMore than half of the students (54%) check the website at least once per week to check grades.


8

The three activities that students reported using the website most frequently were: checking grades (25%), accessing class links for materials used in class (22%), and checking the weekly/daily agenda (18%)

. The next chart shows the frequency that students report using the website for different activities. Checking grades was the activity that students reported as the most frequent activity. Students tend to check grades weekly with 34% reporting they use the website to check grades weekly. The aggregate data for trimesters two and three is posted below.


9

Objective Five: Give Students access to material when absent from classStudents occasionally use the website to access materials from class.

The survey also asked some open ended questions, and students suggested tweaking the format, and moving some content so it is more obvious. The changes suggested by trimester two students were implemented for trimester three, including making “this week in class” the first tab, and changing the order of this week in class so that current week is at the top. Several students also suggested changing the website to make it easier to find grades both overall and for individual assignments. Several students mentioned that the instructions given made the website easier to navigate. A couple responses from trimester two include “I really liked how you could check your updated grades and what you were missing”, and “it is very helpful and it is nice to have when you’re at home and forget your notes or didn’t write something down.” A trimester three student reports, “I don’t have any criticism, if anything I commend you for integrating technology into the classroom setting that actually benefits the students’ needs,” and another student from trimester three reports, “I like it, only class that has it online. Extremely helpful.” A couple students do report that it is hard to navigate. Another trimester three student reported, “I would be more inclined to use the web page if it were easier to navigate.” Between the trimester two and trimester three classes, there were less than five responses that commented on difficulty navigating the website.

DiscussionThe purpose of the evaluation was to determine if the objectives of the website are being met, to determine if the website should continue, and lastly if it is to continue are there any suggestions for improvements. Objective One: Increase Student Ownership for their workThe data from the survey shows that students from both trimester two and trimester three agree that having access to Aspen X2 class website increases their feelings of ownership for their work. It is interesting to note that 73% reported that the website increase their own responsibility for learning, and 82% reported the website increased their feeling of “being in control” of their learning. It is interesting to note that 24% of students visited the website once per month or less. A future evaluation may want to determine the correlation between frequency of website access and feeling of responsibility. The objective of increasing student ownership for their work is being achieved by the website.


10

Objective Two: Increase student interaction with the course contentAnother objective was to increase the amount of time that students spent interacting with the course content. The results for this objective are ambiguous with just over half or 57% reporting that using the website increased their time interacting with course content. The teacher should consider how to increase interaction with material through training.Objective Three: Give students ability to prepare for classObjective Four: Give Students ability to review material from prior classesObjective Five: Give Students access to material when absent from classObjectives three, four, and five are ambiguous with almost 25% of students reporting they never visit the website to review, 30% reporting they never go to the website to preview upcoming material, and 20% reporting they never access the website to see what they missed when absent. Suggestions for the future include evaluating why students do not use the website for these preparing, reviewing, and when absent. Another suggestion is to review the objectives, and determine if these objectives should continue or perhaps be updated to reflect an updated vision for the website.Objective Six: Give Students access to grades

Overwhelmingly, this was the most popular activity on the website with 54% of students checking the website for grades once per week or more. Only four percent report never checking grades. Based on the survey responses, the teacher should continue to offer instruction on how to access grades on the website, as some students still find the navigation of grades in Aspen X2 confusing. Overall, this objective was met during the evaluation period. Suggestions for future

Based on the evaluation, students clearly enjoyed having access to the information on the website, and objectives one (increased responsibility for learning) and six (checking grade) are being met. Additionally, the teacher should continue to actively train students on how and where to access information, and use student feedback to continue to improve course delivery. The teacher should review the objectives that have ambiguous results (objectives three, four, five), as well as the objective that is not being met (objective six) to determine if these objectives are valid. The teacher should consider looking into evaluating why students are not using the website to preview, review, or when absent. The focus of this evaluation could be to determine if students are not utilizing these features because they do not know how, or if the information is available in other formats (papers from class). Lastly, some survey questions to consider are: Why students do or do not access materials to prepare for class, review from class, or when absent. Another question would be to ask students which parts of the website the found helpful or not, and how to improve. The class website using Aspen X2, overall was met with success, and the teacher should continue to use, while improving and updating content based on feedback and future evaluations.


11

Appendix A Screenshots of WebsiteScreenshot of Earth Science: Dynamic Earth and Space Website This week in class tab

Screenshot of Earth Science: Dynamic Earth and Space Website This week in class tab showing visual aids


12

Screenshot of Earth Science: Dynamic Earth and Space Website Class Information tab

Screenshot of Earth Science: Dynamic Earth and Space Website Grades tab

Screenshot of Earth Science: Dynamic Earth and Space Website Class Resources tab


13


14

Appendix B SurveyThe following survey was administered to all classesThe trimester two survey was administered last week of trimester. http://goo.gl/forms/oKj7CUOJ1jThe trimester three survey was administered to students during the fourth week of the trimester. http://goo.gl/forms/DtuUkwiYDc

Cooney Class Website SurveyThis survey is designed to determine if the class website is meeting its goals, as well as determine ways to improve content of website for students.

Are you enrolled in*Required

● Earth Science● Physics

In general, how often do you visit the class web page?*Required

● never● once or twice per trimester● monthly● 2-3 times per month● once per week● more than once per week● daily

How often do you visit the class web page to check your grades?*Required


How often do you visit the class web page to check to see if you are missing any assignments?*Required


How often do you visit the class web page to review material from prior classes?*Required

● never● once or twice per trimester


15

● monthly● 2-3 times per month● once per week● more than once per week● daily

How often do you visit the class web page to preview and prepare for class?*Required


How often do you visit the class webpage to check to see what you missed when absent from class?*Required

● never● occasionally when I am absent● every time I am absent from class

How closely do you agree with this statement. Using the class website increased the amount of time that I spent interacting and doing school work (compared to my "normal" amount).*Required

1 2 3 4

Strongly Disagree Strongly agree

How closely do you agree with this statement. Using the class website helped me to take more responsibility for my learning.*Required

1 2 3 4


How closely do you agree with this statement. Using the class website helped me feel like I have control over my learning*Required

1 2 3 4


In terms of content that is available on the website, please check off all that you used*Required

● Grades● The weekly and daily agenda● The links to websites that we use for class● The files for documents such as worksheets and labs


16

● The files for class notes● Pictures and problems to help with content● Other:

In terms of content, what other information would have been helpful to you to have on the class website? *Required

In terms of web page layout, did you find the page easy to navigate? If not, how can the webpage be improved?*Required

In terms of web page layout, how would you change the layout to make it easier to use?*Required

What other information would you like to share with me about the class website?*Required

Here is a screenshot of what the survey looked like for students taking it.


carolinecooneyedtech.files.wordpress.com€¦ · Web viewSpring 2015. Submitted to: Dr ... Over...

Documents

Transcript of carolinecooneyedtech.files.wordpress.com€¦ · Web viewSpring 2015. Submitted to: Dr ... Over...