Sociology Program Assessment -...

Sociology Program Assessment

(compiled by Susan Gaulden, SLOAT Facilitator) (Final Reports for SLOAT Fall 2010 Courses)

Final Reports for SLOAT Fall 2010 Courses

AFE 083Troy Hamilton, Center for Academic Foundations AFM 083Violeta De Pierola & Arturo Vera, Center for Academic FoundationsART 100Barbara Pogue, HumanitiesBIO 121Jill Stein, Biology & ChemistryBUS 101Nate Himelstein, BusinessENG 096Eileen De Freece, HumanitiesENG 101Richard Bogart, HumanitiesMTH 092Eman Aboelnaga & Barbara Satterwhite, Mathematics & PhysicsMTH 100Carlos Castillo & Soraida Romero, Mathematics & PhysicsMTH 127Susan Gaulden, Mathematics & PhysicsSOC 101Akil Khalfani, Social Sciences

Members of the Student Learning Outcomes Assessment Team (SLOAT) at ECC are assigned the task of determining the level of student mastery of various SLOs (i.e., Course Goals and all associated MPOs, General Education Goals if applicable, and Program Goals if applicable) specific to their SLOAT course. This book is a compilation of the Fall 2010 SLOAT Final Reports, which each contain course-specific findings. These final reports, along with other SLOAT course assessment documents, may be found on the ECC SLO Assessment website http://sloat.mathography.org, which is maintained by Professor Ron Bannon, Mathematics & Physics.

Abstract 1

AFE 083 Student Learning Outcomes Assessment Team (SLOAT)

Fall 2010 Final Report by Troy Hamilton

Introduction

Academic Foundations English 083 (AFE 083) is a non-General Education course that focuses on writing, reading, and study skills. This course is designed to emphasize fluency, the writing process, sentence structure, editing and revision, paragraph and essay development, the comprehension and analysis of texts, and effective study habits and skills. The reading skills will be applied to selections of fiction and non-fiction including essays reflecting the various rhetorical modes. Study skills instruction includes emphasis on listening, note taking, following directions, understanding texts, goal setting, time management, and test taking. Special emphasis will be placed on helping students to overcome the anxieties of testing, reading, writing and studying. The following are the listed course goals for AFE 083:

Upon successful completion of the course students should be able to:

1. Write a five-paragraph essay.

2. Utilize proper research techniques necessary to write a fully-cited research paper using Modern Language Association (MLA) style format.

Purpose

This assessment of student learning outcomes (SLOs) conducted by Troy Hamilton seeks to assist the English instructors working for the Center for Academic Foundation to better understand exactly what topics/skills students are mastering. More than just engaging in information gathering, the desired intention of this study is to analyze and understand students progress for the purpose of modifying and refining how they are being taught. The hope and expectation is that after the data is shared, each instructor will be encouraged to alter and/or modify their teaching approach to meet the needs of their individual students.

Methodology

The student learning outcomes (SLO) analysis implemented in Fall of 2010 for AFE 083 was used to assess course goal 1. Each of the 5 AFE 083 instructors that participated in this assessment analyzed students essays to see, among other things, how well students were able to develop and sustain arguments throughout their essays. The sample population of students selected to participate in the assessment were twenty five, in total, from AFE 083 sections 001, 002, 003, DE1 and DE2. Instructors were asked to identify the first five students listed on their class roster to be part of the data collection. This approach helped prevent instructors from picking only their best students to be part of the assessment. The essays reviewed in the assessment were the two essays students submitted right before the midterm and final exams were administered. The checklist rubric used for the essay assessment is given below.

Essex County College Student Learning Outcomes Assessment Survey

AFE 083/Fall 2010

English Assignment Survey

Assignment due Date_______________ Section Number____________________

Assignment Topic____________________________________________________________

For each of the following outcomes identified for inclusion in a paper assigned in AFE 083, please indicate their occurrence or lack of occurrence in your students writing:

1. The essay includes an appropriate introductory paragraph.

Yes Somewhat No

2. There is an appropriate topic sentence with the authors name and the title of the essay.

Yes Somewhat No

3. A clear thesis statement was incorporated within the introductory paragraph.

Yes Somewhat No

4. A relevant three-point sentence was established.

Yes Somewhat No

5. Body paragraphs were well-structured.

Yes Somewhat No

6. Body paragraphs were related to each other in a logical structure that supported the thesis.

Yes Somewhat No

7. Proper in- text citation was utilized.

Yes Somewhat No

8. The essay contained an appropriate concluding paragraph.

Yes Somewhat No

9. The paper demonstrates an acceptable level of mechanics and language usage.

Yes Somewhat No

The Measurable Course Performance Objectives (MPOs) associated with course goal 1 are as follows:

1.1 write an appropriate introductory paragraph;

1.2 compose an appropriate topic sentence;

1.3 construct a clear thesis statement;

1.4 establish a three-point sentence;

1.5 compose an appropriate topic sentence for each body paragraph;

1.6 utilize in-text citation;

1.7 write a concluding sentence for each body paragraph;

1.8 write an appropriate concluding paragraph; and

1.9 use correct grammar and syntax throughout the essay

Student achievement of MPOs 1.1 through 1.3 provides evidence of how well students are able to develop an appropriate introductory paragraph, whereas achievement of MPOs 1.4 through 1.7 and 1.9 and MPO 1.8 were used to assess the body paragraphs and the conclusion paragraph respectively.

Assessment Results for Midterm Exam:

60% of the students were able to develop an appropriate introductory paragraph (i.e., achieved MPOs 1.1 1.3) before the midterm exam (both the mid-term and the final exams were designed by the Humanities Division). The first three questions on the checklist rubric were formulated to assess the introductory paragraph.

63% of the students were able to develop an appropriate body paragraph utilizing MLA in-text citation (i.e., achieved MPOs 1.4 1.7 and 1.9) before the midterm exam. Questions 4 through 7 and 9 on the checklist rubric were formulated to assess the body paragraphs.

61% of the students were able to develop an appropriate concluding paragraph before the midterm exam. Question 8 on the checklist rubric was designed to assess student ability to develop an appropriate concluding paragraph.

( MPOs Considered Achieved )

Diagram 1

These findings as displayed in Diagram 1, indicate that by the Fall 2010 midterm exam only 60% (introductory paragraph), 63% (body paragraphs), and 61% (concluding paragraph) of the sampled students attained the corresponding Measurable Performance Objectives (MPOs). For this study, if 70% of the students are able to accomplish a given objective, it is considered achieved. In this particular study conducted prior to midterm, none of the MPOs were achieved. This implies that by the eighth week of the semester students were not performing/writing at the expected, desired level. It seems that, based on anecdotal information gathered in discussions with students after the midterm assessment, a number of factors contributed to their lack of achievement. Some of the factors reported by students were external, such as employment, financial difficulties, and child rearing. It is safe to conclude that these factors prohibited students from attending class regularly, which directly impacted their academic performance. In spite of this and based on the study findings, it is clear that AFE English instructors and Supplemental Instructors (SIs) need to come up with more effective pedagogical strategies in an effort to enhance their students performance preferably exceed 70% MPO achievability before the midterm exam.

Assessment Results for Final Exam:

By the Final Exam MPOs 1.1 thru 1.9 were achieved by the AFE students. However, this achievement might be accepted with hesitance the level of success was certainly by the narrowest of margin, especially when you factor in that AFE 083 is a developmental course.

71% of students were able to develop an appropriate introductory paragraph before the final exam, which is a small but considerable improvement from the midterm assessment (60%).

75% of students were able to develop an appropriate body paragraph utilizing MLA in-text citation before the final exam, which is a small but considerable improvement from the midterm assessment (63%).

71% of the students were able to develop an appropriate concluding paragraph before the final exam which is a small but considerable improvement from the midterm assessment (61%).

( MPOs considered Achieved)

Diagram 2

Summary

As indicated in Diagrams 1 and 2, which are based on the information gathered from the assessment, AFE 083 instructors in the CAF department need to spend more time developing their students writing skills before the midterm exam if 70% or more of the students are to be able to achieve the course MPOs by the eighth week of the semester. This will definitely improve the writing skills of the students within CAF. When students can develop a good introductory paragraph, it is more likely that the overall structuretone, diction and structural argumentsof the essay will be adequate as well. These are some of the strategies English instructors can implement in order to improve their students success:

spend more class time on the pre-writing process

have student engaging in timed writing drills

require that the supplemental instructor (SI) spend more time with struggling students

offer more collaborative learning opportunities

AFE 083 1

AFE 083 6

MTH 083 Student Learning Outcomes Assessment Team (SLOAT)

Fall 2010 Final Report by Violeta De Pierola and Arturo Vera

*Introduction

Academic Foundations Math AFM 083 is a beginning mathematics course designed to take students from concrete arithmetic ideas to the more abstract algebraic forms of these ideas. One of the instructional components of AFM 083 is two mandatory sessions (one hour each) of tutoring per week and the required completion of ALEKS (computer software) assignments. AFM 083 course outline lists the following goals:

1.Demonstrate knowledge of the fundamental concepts and theories from arithmetic, algebra and geometry.

2. Utilize various problem-solving and critical-thinking techniques to set up and solve real-world applications.

3. Communicate accurate mathematical terminology and notation in written and/or oral form in order to explain strategies to solve problems as well as to interpret found solutions.

*Purpose

The purpose of the SLOAT was to determine if the students enrolled in AFM 083 are learning the goals set in the course outline. It was also meant to help the Math instructors understand how the students enrolled in the Center for Academic Foundations learn and what different teaching techniques they can use. This assessment was conducted by Violeta De Pierola and Arturo Vera and it was done based on 32 students.

*Methodology

For the Fall 2010 SLOAT the two goals that were assessed were goals 1 and 2 from the course outline.

1. Demonstrate knowledge of the fundamental concepts and theories from arithmetic, algebra and geometry:

1.1 perform arithmetic operations on signed numbers;

1.2 perform arithmetic operations on fractions;

1.3 perform arithmetic operations on decimals;

1.4 perform arithmetic operations on percents;

1.5 determine the perimeter and area for simple geometric figures;

1.6 determine whether a ratio is a proportion;

1.7 convert from one unit of measure to another;

1.8 simplify basic algebraic operations; and

1.9 solve simple linear equations involving one operation

2. Utilize various problem-solving and critical-thinking techniques to set up and solve real world applications:

2.1apply arithmetic to solve application problems encountered in daily life

*Population

We decided to use a total of 32 students as a sample size selected from 4 sections of AFM 083; i.e., even though we gave all of the students in the 4 sections the pre-test, questionnaire, and post-test we sampled only 32 of those students for this SLOAT study.

Since one of the purposes for this assessment was to help determine whether students taking AFM are achieving the Measureable Performance Objectives (MPOs) for this course, we decided to start by blueprinting the pre-test given to all students. From there we randomly chose 8 students from each section and kept track of those students questionnaire responses, quiz and test scores, and Aleks (an online homework software) statistics.

*Instrumentation

For this study we used data from 3 sources:

Blueprinted multiple-choice questions*

Questionnaires*

Statistics from Aleks

*Copies of these assessment instruments used are included in Appendix A of this report.

*Results

Pre- and Post-test Results

Graphs comparing the scores the 32 sampled students received on the pre- and post-tests are below.

Pre- and Post-test scores

Pre- and Post-test scores (continued)

The pre- and post-tests were given by the instructors in class. The pre-test was given on the first day of class. Students were told to try their best and were not allowed to use calculators. The post-test was also given by the instructors and, by looking at the graphs above, we can see how remarkably the students scores changed.

In section CW1, the average score on the pre-test was 30% and 79.8% on the post-test indicating an increase of 49.8%. In section 003, the average score on the pre-test was 35.6% and 81.7 % on the post-test indicating an increase of 46.1%. In section 005, the average score on the pre-test was 20.6% and 79.1% on the post-test indicating an increase of 58.5%. In section 014, the average score for the pre-test was 28.1% and 77.5% on the post-test indicating an increase of 49.4%.

By looking at the graphs we can also see that the students scored low as they entered this course. Many different factors could have influenced this result; for example, not expecting the test, coming back to school after many years, and not being well prepared. These are just a few reasons given by the students explaining why they did not perform that well on the pre-test.

Questionnaire Results

The students were given two questionnaires, the first one was conducted before they took the midterm and the second one was administered before the final. Average student responses to both questionnaires are given in Appendix B. From looking at the average responses of students answers to the survey, we can conclude that the students still need extra help when working on the following MPOs:

1.4Perform arithmetic operations on fractions;

1.8Simplify basic algebraic operations;

2.1Apply arithmetic to solve application problems encountered in daily life.

We came to this conclusion after noticing that the average student responses for each question related to these three MPOs were less than 70%. Since students need at least 70% to pass the course, the instructors should ensure that students spend more time practicing these topics.

This information is already being shared with CAF instructors and students. One such way is by creating the Class Syllabi for the Spring 2011 semester which reflect the course outlines prepared by SLOAT members. These are the following categories that we included in the syllabus:

Course Number and Name

Credit Hours

Prerequisites

Co-requisites

Course Description

Course Goals

Course Requirements

Methods of Evaluation

Academic Integrity

Student Code of Conduct

Course Content Outline (by week)

We decided to include these elements because doing so will make the students aware of what is expected from them on day one, and they will know what they must learn by end of the semester.

*Suggestions

After reviewing all the data collected from the assessment, we realize that instructors need to spend more time reviewing fractions (adding, subtracting, multiplying and dividing), relating word problems to real life, and solving basic algebraic equations. Some suggestions to help students learn more effectively include the following:

Instructors can have SIs work with students struggling in these areas.

Have the students work on Aleks at least one day per week during tutoring time.

Have the students work together while working on these difficult topics.

AFM 083 6

APPENDIX A

Center Academic Foundations

Academic Foundation Mathematics

AFM 083 Pre-test

Name: ____________________________ Date: ____________Section: ________

1.Name the property that is illustrated

4 x (3+6) = (4 x 30 + (4 x 6)

2.What is the product of 1636 and 58?

3.Write in expanded form 353, 999.

4.Write in expanded form x5.

5.What is nine thousand seventy eight in standard form?

6.The average age of students in College X is 23.58 and the average age of students in College Y is 23.5798. Which College has the lower average?

7.Express seventy one thousand seventy-one and seventy-six hundredths in decimal form.

8.Evaluate: -22 + (-11) + 11 + (-10)

9.Evaluate: d3 b3 if b = 5; d = 6.

10.Write the following phrase, using symbols: Twice the difference of x and y



AFM 083 Midterm Questionnaire

Name: ____________________________ Date: ____________ Section:________

Please mark only one X on the one that applies to you.

1.How do you think you will do on the midterm?

__ Really good__ good__ average__ not so good

On a scale from 0% - 100%:

2.How comfortable do you feel rounding? (Round 8,416 to the nearest

thousand.)

__ 100%__ 75%__50%__25%__0%

3. How comfortable do you feel dividing fractions? ( )

__ 100%__ 75%__50%__25%__0%

4. How comfortable do you feel finding product? (Find the product of -90 and

60.)

__ 100%__ 75%__50%__25%__0%

5. How comfortable do you feel with prime factorization? (Find the prime factorization of 144.)

__ 100%__ 75%__50%__25%__0%

6. How comfortable do you feel finding the perimeter? (A rectangle has a length

of 26m and a width of 12m. Find the perimeter of the rectangle.)

__ 100%__ 75%__50%__25%__0%

7. How comfortable do you feel evaluating expressions? (Evaluate for x=12

and y = -3.)

__ 100%__ 75%__50%__25%__0%

8. How comfortable do you feel simplifying? (Simplify -8 2 + (-6))

__ 100%__ 75%__50%__25%__0%

9. How comfortable do you feel with absolute value? (Write in ascending order

-4 , -9 , - 17 , 8 )

__ 100%__ 75%__50%__25%__0%

10. How comfortable are you multiplying fractions? [( ) x ( ) x ( )]

__ 100%__ 75%__50%__25%__0%



AFM 083 Final Questionnaire

Name: ____________________________ Date: ____________Section: _________


1How do you think you will do on the final?


2.Do you think you will do better on the final then you did on the midterm?

___ yes___ no


3.How comfortable do you feel working with order of operations? (20-16 4)

__ 100%__ 75%__50%__25%__0%

4.How comfortable do you feel working with percents? (Write 0.818 as a

percent.)

__ 100%__ 75%__50%__25%__0%

5.How comfortable do you feel solving proportions? ( = is this proportion

true?)

__ 100%__ 75%__50%__25%__0%

6.How comfortable do you feel simplifying variable expressions? [4+5m+(-5m)]

__ 100%__ 75%__50%__25%__0%

7.How comfortable do you feel solving equations? (Solve 2d = -12.)

__ 100%__ 75%__50%__25%__0%

8.How comfortable do you feel dividing fractions? (-110 3 )

__ 100%__ 75%__50%__25%__0%

9.How comfortable do you feel solving word problems?

(A digital camera with a regular price of $265 is on sale for 19% off the regular price. Find the sale price.)

__ 100%__ 75%__50%__25%__0%

10.How comfortable do you feel multiplying exponential expressions? ( z * z * z)

__ 100%__ 75%__50%__25%__0%

AFM 083 Appendix A 3

APPENDIX B



AFM 083 Midterm Questionnaire

Name: ____________________________ Date: ____________ Section:________


1.How do you think you will do on the midterm?



2.How comfortable do you feel rounding? (Round 8,416 to the nearest

thousand.)

85%

3. How comfortable do you feel dividing fractions? ( )

67%

4. How comfortable do you feel finding product? (Find the product of -90 and 60)

73%

5. How comfortable do you feel with prime factorization? (Find the prime

factorization of 144.)

80%

6. How comfortable do you feel finding the perimeter? (A rectangle has a length

of 26m and a width of 12m. Find the perimeter of the rectangle.)

71%

7. How comfortable do you feel with evaluating expressions? (Evaluate for

x=12 and y = -3.)

78%

8. How comfortable do you feel simplifying? (Simplify -8 2 + (-6))

72%

9. How comfortable do you feel with absolute value? (Write in ascending order

-4 , -9 , - 17 , 8 .)

86%

10. How comfortable do you multiplying fractions? ( ) x ( ) x ( )

62%



AFM 083 Final Questionnaire

Name: ____________________________ Date: ____________Section: ________


1How do you think you will do on the final?

good

2.Do you think you will do better on the final then you did on the midterm?

yes


3.How comfortable do you feel working with order of operations? (20-16 4)

95%

4.How comfortable do you feel working with percents? (Write 0.818 as a

percent.)

81%

5.How comfortable do you feel solving proportions? ( = is this proportion

true?)

76%

6.How comfortable do you feel simplifying variable expressions? [4+ 5m+(-5m)]

68%

7.How comfortable do you feel solving equations? (Solve 2d = -12.)

78%

8.How comfortable do you feel dividing fractions? (-110 3 )

69%

9.How comfortable do you feel solving word problems?

(A digital camera with a regular price of $265 is on sale for 19% off the regular price. Find the sale price.)

69%

10.How comfortable do you feel multiplying exponential expressions? ( z * z * z)

71%

AFM 083 Appendix B 2

SLOAT Summary 1/24/11 Prof. Barbara Pogue

ART APPRECIATION ART 100 3 credit hours

ART 100 BO1 and ART 100 007 were the two classes chosen for analysis; at semesters end (Fall 2010) there were 60 students evaluated.

The above course is both an AA and AS degree program additional requirement under the General Education category of Humanistic Perspective. The purpose of Art Appreciation is to give students an introduction to great works of world art as well as to encourage them to question the nature and relevance of art to daily life. Students are also introduced to the elements and principles of design and asked to demonstrate comprehension of the principles of design by constructing a Principles of Design booklet.

For SLOAT, I decided to focus attention on the Principles of Design booklet, as it required a wide variety of educational behaviors in comprehension, knowledge, evaluation, analysis and application: distinguishing, comparing, interpreting, identifying, deciding, classifying and explaining the various principles of design: variety and unity, symmetrical and asymmetrical balance, rhythm, scale, proportion, emphasis and subordination. Students choose two examples of each principle from popular magazines, carefully place them in a booklet, and type an explanation of why each example was chosen.

The Principles of Design booklet relates to Course Goal number 4: prepare and present information using a computer by doing the Principles of Design booklet (ART 100 Course Outline, p.2) and to Measurable Performance Objective number 4: use magazines to gather examples of each of the principles of design; use computers to type up explanations of why each illustration fits the criteria for the principles; assemble images, titles and explanations, along with a cover sheet, into a booklet. (ART 100 Course Outline, p.2)

I prepare a handout of instructions on how to do the booklet (attached) for students, as well as speak extensively on the subject and show previous examples of booklets. Students read the relevant chapter in their textbooks to further enhance their knowledge of the principles. Finally, there is one day devoted to doing the booklets in class, and then the students are required to complete the project on their own time.

For SLOAT, I prepared a checklist of how I would evaluate the booklets (attached) and got ready to check off 60 booklets to see how well the students understood the principles of design. The booklets, by their nature, take a long time to do, so I give the instructions about two months before the due date, which is the last week of class.

The last week, I collected the booklets and used my check-off sheets to determine what grade the student would receive for his/her booklet. Thats when the problem became obvious. I discovered my so-called carefully-prepared list was totally inadequate to determine grades. Time and again, students (actually all of them) met all the check-off requirements, yet their booklets could not be given an A or a B+ grade. Many of the booklets were adequate, but lacked an indefinable something. For example, the illustration chosen might have been average or okay, but lacked subtle design qualities which would have made it superior. Or perhaps a students explanation, while technically correct, might have been repetitious, showed a lack of imagination or awkward rhetoric or lapses in grammar.

The result of doing the SLOAT assessment was that I discovered that all my students did understand the principles of design and all did pass the principles of design booklet assignment, but that my instrument was too crude to predict grade distinctions. Therefore, if I want to figure out what is an A, B+, B, C+, C or D grade, Ill have to design a different instrument. The question is how do you absolutely define or measure what is a superior illustration, a very good illustration, a good one, an above-average one, a fair one, or a poor one? How do you communicate these distinctions so students will understand them? How do you design an instrument that will take these distinctions into account without being unduly cumbersome to use?

As a check on the rationale for doing the booklet, one of my final exam questions was: On the back of this paper, evaluate the course. What worked? What didnt? What did you like the most and the least? What did you learn the most from doing? What could I do better in the future? This question is worth 10 pointsand , you dont have to be positive. Negative comments are also helpful.

Out of a total of 39 responses (students who had an A average werent required to take the final, so there were only 39 instead of 60 students reporting), 23 commented that they learned most from either the lectures or the in-class projects such as the color wheel, the collages and the masks. Fifteen reported that the single thing they learned the most from doing was the Principles of Design booklet, and one commented that the booklet was too difficult, but that doing it was a helpful learning experience. Many students (14) didnt like doing the oral reports; some (7) didnt like watching the videos. Not one student reported he/she didnt like or didnt learn from doing the booklet. Not one chose it as something he/she learned the least from doing.

Therefore, I must conclude that students feel doing the booklet is important, but perhaps not as important as I believe it to be. Nevertheless, it is a valuable learning tool and I will continue to utilize it. I must try to develop a more subtle instrument for measuring the booklet itself.

ART 100 2

Biology 121 Fall 2010 SLOAT Final Report Jill Stein

Introduction

The purpose of this assessment is to document the progress of a cohort of students taking Biology 121 during the Fall 2010 semester.

Biology 121 (Anatomy & Physiology I) is the first semester of a two semester Anatomy & Physiology sequence, and is a pre-requisite for application to the Nursing, Physical Therapist Assistant, Radiography, Dental Hygiene/Assisting, Respiratory Care and Dietary Manager Programs here at Essex County College (ECC). It is also required for admission to other area Nursing programs, such as those at Rutgers, UMDNJ, and Seton Hall. Along with the subsequent successful completion of Biology 122 (Anatomy & Physiology II), the student is able to fulfill the General Education requirement for a laboratory science sequence, and General Education goal 3 regarding Scientific Knowledge and Reasoning. The Biology 121/122 sequence can also be used to fulfill an elective requirement for the Biology major.

Approximately 21 sections, which each contain 24 students, are offered for this course in both fall and spring semesters. Four additional sections containing 24 students each are offered during the summer, bringing the total number of students enrolled in this course each year to approximately 1100.

Currently, the only pre-requisites for Biology 121 are completion of all remedial courses, if necessary, or demonstration of college level readiness in Mathematics and English. There is no college science pre-requisite. The course is content heavy, and taught at a rapid pace, a situation unfamiliar to many of our students. This can lead to a high attrition rate, a problem not just at ECC, but at colleges all over the country.1 Since mastery of this course is crucial for so many students in pursuit of their goal to become a health professional, it was chosen for this pilot assessment project.

Methodology

Fifty students, comprising two sections, were part of the initial cohort. This number was based on the official class lists after the no shows were removed. Eight exams were given in class at varying intervals throughout the semester starting with the third week of class. Each exam contained a variety of short answer questions which were blueprinted to specific MPOs. Assessment methods used were direct, summative, quantitative, and objective. Multiple questions were used to assess each MPO. Each question was scored for the number of students selecting the correct answer. This number was converted to a percentage of the total number of students taking the exam. Multiple percentages for each MPO were then averaged to determine the overall student success for each MPO.

Results

As shown in Figure 1 below, the number of students taking the exams declined gradually over the course of the semester, from 45 to 20. Although 50 students were registered, only 45 took the first exam, an immediate loss of 10%. Three of those five did not return to class. The other two returned to take the second exam; one dropped immediately after, and the other attended irregularly until the end of the semester. The steepest declines occurred after exams 3 (histology and integumentary system) and 4 (bone physiology). Exam 5 was given after midterm warnings were sent out, but just before the final drop date. By that time, most of the students had made their decision as to whether or not to remain in the class. Twenty students took both exams 7 and 8, although only 18 of the 20 were the same for both exams.

Student success for each MPO is shown in Table 1 below. Most of the success rates are between 60-70%. Notable exceptions are Exam 2, where percentages are below 60% for all MPOs assessed, and Exam 5, where the score for MPO 2.2 is 57.5%. The material covered for Exam 2 is essentially a crash course in biochemistry and cell biology. Exam 5 is a practical exam on bone and joint identification.

Table 1: Student Success for Each MPO on Exams 1 8

MPO

Exam 1

Exam 2

Exam 3

Exam 4

Exam 5

Exam 6

Exam 7

Exam 8

1.1

68.7

1.2

61.2

1.3

56.7

1.4

49.4

1.5

55.9

1.6

62.3

2.1

62.7

2.2

73.1

57.5

2.3

65.6

2.4

70.1

69.2

3.1

56.4

3.2

70.8

3.3

68.2

72.2

3.4

69.9

3.5

61.3

72.5

Students were asked to complete brief surveys after Exams 1 and 2. Thirty-nine students completed the survey after Exam 1 (see Appendix A for full results). Several students did not have a book or lab manual, but most felt that the exam was reasonable, and virtually no one felt time pressure. Unfortunately, 16 students felt that they did not study enough due to overcommitted schedules. This fact appeared repeatedly when students listed other factors which contributed to their exam scores. Students also wrote: stress not related to class; focused more on math class; lack of sleep, nervous; didnt make it as much of a priority as I should have; careless mistakes with reading questions; and misunderstood two questions.

Thirty one students completed the survey after Exam 2 (see Appendix B for full results). As noted, successful completion of MPOs was generally lower on this exam. Here again, 10 students cited an overcommitted schedule, with comments including family troubles/didnt study, and crammed this past weekend and froze at test time. Several students indicated that they did not feel that they studied the correct information.

Discussion

Biology 121 is the first part of a two semester course in Anatomy & Physiology. As such, the content is both quantitatively and qualitatively difficult to master. In addition, critical thinking and analysis are required to answer some of the questions. Anecdotally and demonstrably, many students are unaccustomed to this type of course, are insufficiently prepared, and lacking in the appropriate study skills. Thus, while successful completion rates for the MPOs generally hovered between 60-70%, this was not surprising.

Successful completion rates for MPOs 1.3, 1.4, and 1.5 (Exam 2) fell below the 60% rate. These MPOs encompass biochemistry and cell biology, basic information that is crucial for success. Currently, the only pre-requisites required to take Biology 121 are successful completion of developmental math and English/reading if required, or demonstration of college level competency in these areas. Students were surveyed at the beginning of the semester to make sure that these requirements were completed (data not shown). There is no biology and/or chemistry requirement. While the course obviously provides instruction in these areas, it is quite a rapid pace of delivery for such critical information. Biology is not required for high school graduation in New Jersey2, and many students begin BIO 121 with the most rudimentary of scientific backgrounds. An in depth, detailed examination of these fundamental topics of biochemistry and cell biology, however, is provided in Biology 100, a preparatory class which may be taken while taking developmental classes, prior to the start of BIO 121. Many students should take advantage of this class, but fail to do so since it is not a requirement.

Successful completion of MPO 2.2 (Exam 5) was 57.5%. The topics covered on this exam included bone and joint anatomy. In this case, the issues are slightly different. First, the exam format is a practical exam, rather than the standard short answer to which the students have become accustomed. Students are asked to identify bones, markings, and articulations from the sample presented on the bench. Most students have never had this type of exam, and many panic at the format change. Second, the quantity of information is very large, although not as complex as the biochemistry/cell biology. Again, many students do not have the time or the skills necessary to adequately learn the material, or sufficiently recognize the time needed to master these skills until too late.

A rigorous background in Anatomy & Physiology is critical for success in the health related professions. Many students are interested in these professions, as they are ones in which well-paying jobs are available without excessive years of education. Problems similar to those noted here have been observed at other institutions. At the University of Southern Indiana, a Supplement course for students identified as high risk, for failing or withdrawing due to many of the same issues presented here, was implemented.1 Success rates (as measured by a final grade of C or better) improved for those enrolled in the Supplement. A supplemental course, BIO 100, already exists at ECC. Therefore, a discussion about the pre-requisites for BIO 121 should be held by the Division Curriculum Committee. In addition, consideration should be given to the quantity of material on Exam 5, or possibly breaking it up into two smaller exams.

Endnotes

1. Hopper, M. 2011. Student Enrollment in a Supplement Course for Anatomy and Physiology Results in Improved Retention and Success. Journal of College Science Teaching 40 (3): 70-79.

2. Brody, Leslie. N.J. Puts Off Plans for High School Biology Test. Bergen Record Online 28 Dec. 2010. Web. 31 Jan 2011.

BIO 121 5

Appendix A Student Survey Administered After Exam 1 in BIO 121

N=39SCORE EARNED ON EXAM #1 ________

PLEASE ANSWER THE FOLLOWING QUESTIONS HONESTLY:

1. How prepared were you for this exam based on the work youve done so far? (Circle one.)

FULLY PREPARED 12 SOMEWHAT PREPARED 23

NOT AT ALL PREPARED 3

2. Do you have a copy of the required textbook (or a similar book by a different author)?

YES 35NO 4

3. Do you have a copy of the lab manual?

YES 33NO 6

4. How much homework have you done so far this term? (Circle one.)

ALL 20MOST(>50%) 14SOME(10/100 Questions Incorrectly

00178.3 16 2

00278.7 17 1

00380.1 15 1

00669.6 17 16

00777.0 22 13

00977.8 26 20

The questions which the students marked wrong will be given to the instructors so that next semester these areas can be further explained.

The three instructors in sections 006 , 007 and 009 do not have the teaching experience that those instructing sections 001, 002 and 003 have, however those areas must be noted as indicating students having deficiency in that subject area.

Interesting to also note that the same question which had > than 10 wrong was in Sec 001 and Sec 002 not in Sec 003.

Please also note that the mean of 5 of the 6 sections are in close range of each other.

The specific questions in which more than 10 students provided the wrong answers were as follows:

Exam Question #

5. Taxes and government regulations are part of the ? environment of business. (economic and legal).

11. Which of the following prohibits monopolies, attempts to monopolize, and any restraints of trade? (Sherman Act)

23. Which of the following statements about S corporations is most accurate? (The major attraction of S corporations is that they avoid the problem of double taxation.)

24. A ? is two firms combining to form one company. (merger)

35. ? is the management function of creating a vision for the organization and guiding, training, coaching, and motivating employees to help achieve the goals and objectives of the organization. (Leading)

42. Henri Fayol and Max Weber are best known for heir contributions to: (organization theory).

59. The Hawthorne studies were conducted by ? and his colleagues from Harvard University. (Elton Mayo)

61. In Maslows hierarchy of needs, the desire for love and acceptance would fall into the category of: (social needs)

62. Herzberg found that good pay: (was a hygiene factor rather than a motivator)

78. Historically, the ? strengthened he labor unions, while the ?

supported management efforts. (National Labor Relations Act, Taft-Hartley Act.)

82. The Taft-Hartley Act: (allowed individual states to pass right-to-work laws prohibiting compulsory union membership)

92. Which of the following product attributes is least emphasized on television ads? (price)

96. A firm that wants to distribute its products as widely in a market as possible would us a(n) ? distribution strategy. (intensive)

100. In evaluating the bet advertising medium to reach a specific target market the clear is (direct mail)

These areas will be shared with all faculty (full and adjunct) so that when the topics are covered these concepts should be given further emphasis in the lectures and assignments

Based on the means of the final exam, the MPOs are demonstrated in the departmental final exam.

BUS 101 3

ENG 096 SLOAT Fall 2010 REPORT Submitted by: Dr. Eileen De Freece

Introduction

The use of two instruments helped measure the achievement of outcomes in ENG 096 during the Fall 2010 semester. Both were based upon Course Goals developed in the submitted revised ENG 096 course outline. The purpose of these instruments provided two outcomes: First, a rubric was used to randomly assess the achievement of essay development skills by students across the course, based upon the first Course Goal listed in the revised outline, write a composition. This rubric was used twice during the semester to measure any change noted in the outcome.

The second instrument was a questionnaire, completed by students in the ENG 096 sections, regarding their readiness for the mid-term essay, in order to measure, to some extent, the second Course Goal, implement critical reading to analyze selected materials.

Support from faculty and students was strong in this effort, and the results were interesting and worthy of consideration.

Methodology

Of the two instruments used in this study, the first was meant as a limited, random sample. Therefore, the anonymous nature of the study was made clear to the instructors who received a rubric designed to evaluate each of the Measureable Performance Outcomes (MPOs) listed in the revised ENG 096 outline under the first Course Goal, write a composition. Instructors were given ample copies of the rubric and asked to evaluate their students most recent essay for the elements, focusing on the first five students on their roster. By selecting alphabetically, the random nature of the study was maintained. Instructors rated each on a three point scale labeled, Yes, Somewhat, or No.

The first distribution and collection of the surveys happened in late September and October while the second was completed in December to note any changes in the outcomes. Since these questionnaires were anonymous, the Humanities Division work study students helped total results by category.

In the September/October rubric, eight ENG 096 sections participated, while just two sections participated in the December rubric. Unfortunately, timing may have been an issue in December as many instructors may have been preparing for the Divisional final essay.

The second instrument, a student questionnaire regarding mid-term essay preparedness, was distributed to ENG 096 instructors the week following the Divisional mid-term essay. A total of 135 students responded to the survey, a representative sample of the student enrollment in the course.

As with the essay rubric, the questionnaire was administered and evaluated anonymously; therefore, the Humanities Divisions work study students again helped tally the results for each category. The results and original instruments were returned to me and are stored in my office.

Results

Results from the rubric distributed to ENG 096 instructors and the survey administered to ENG 096 students after the mid-term essay provide insight into students abilities and awareness.

The questions on the rubric, distributed twice, were:

1. The paper includes an appropriate subject based upon the given writing

assignment.

2. A specific topic was addressed within the assigned subject for the paper.

3. A relevant thesis statement was incorporated within the introductory paragraph.

4. Body paragraphs were well structured.

5. Body paragraphs were related to each other in a logical structure that supported

the thesis.

6. The paper contained an appropriate concluding paragraph.

7. The paper approaches an acceptable level of mechanics and language usage.


Instructors were asked to evaluate the first five papers alphabetically, to insure randomness, and for each question respond Yes, Somewhat, or No in each category. The categories were drawn from the Student Learning Outcomes (SLO) in the ENG 096 revised course outline.

Results of the first distribution of the rubric during September/October 2010 follow:

1. The paper includes an appropriate subject based upon the given writing assignment.

Yes 63 (61%) Somewhat 29 (28%) No 11 (11%)


Yes 98 (89%) Somewhat 7 (6%) No 5 (5%)


Yes 75 (23%) Somewhat 31 (5%) No 5 (5%)


Yes 88 (81%) Somewhat 19 (17%) No 2 (2%)


Yes 77 (86%) Somewhat 12 (13%) No 1 (1%)


Yes 28 (76%) Somewhat 8 (22%) No 1 (3%)


Yes 22 (65%) Somewhat 12 (35%) No 1 (1%)


Yes 22 (63%) Somewhat 12 (34%) No 1 (4%)

In terms of the first question, the Yes responses were mostly encouraging, especially for numbers 2, 4, 5 and 6 with the least positive response being for number 3 at 23%, which implies the need for thesis development in the introductory paragraph for the ENG 096 level. The other least positive responses, though exceeding 60% indicate the need for an acceptable level of mechanics and language usage. Question number 1, like 3, seems to point to a need for stronger critical thinking skills for this level.

The results can be charted as follows:

In December 2010, ENG 096 instructors were once again given the same rubric to complete for a subsequent assignment. It was anticipated that some patterns of improvement would be seen in the second study. Faculty participation was very low in the second study, which was disappointing, but timing was an issue considering faculty absorption in Divisional final essay preparation and other concerns.

December 2010 results follow:

1. The paper includes an appropriate subject based upon the given writing assignment.

Yes 9 (90%) Somewhat 0 (0%) No 1 (10%)


Yes 8 (80%) Somewhat 1 (10%) No 1 (10%)


Yes 7 (70%) Somewhat 2 (20%) No 1 (10%)


Yes 5 (50%) Somewhat 3 (30%) No 2 (20%)


Yes 6 (60%) Somewhat 3 (30%) No 1 (10%)


Yes 5 (50%) Somewhat 2 (20%) No 3 (30%)


Yes 5 (50%) Somewhat 2 (20%) No 3 (30%)


Yes 5 (50%) Somewhat 2 (20%) No 3 (30%)

Although the December 2010 survey noted fewer faculty members participated, it was interesting that question number 1 presented more positive results than in the first survey with 90% marked Yes. And question number 3 illustrates an increase from 23% to 70%. There remains more support needed in terms of paragraph development represented in questions 4 and 5 at the end of the semester that appeared more positive in the first survey. The support for mechanics and language usage is evident here with 50% approaching an acceptable level in question 7, and 50% demonstrating an acceptable level in question 8.


Another instrument that was used during the Fall 2010 semester was a questionnaire that was distributed to ENG 096 students a week after taking the Divisional mid-term essay. The anonymous responses allowed students to openly consider their own preparedness for the midterm essay. Students were asked five questions dealing with their performance and preparedness. Note the following results:

1. Are you pleased with your score on the exam?

Yes 20 (41%) Somewhat 12 (25%) No 17 (35%)

2. Did the professor accurately tell you what to expect on the exam before the exam date?

Yes 59 (95%) Somewhat 3 (5%) No 0 (0%)

3. Did you adequately prepare for the exam?

Yes 46 (63%) Somewhat 25 (34%) No 2 (3%)

If not, why not? (Circle all that apply)

A. Did not have time to study

B. Did not know the exam was scheduled

C. Have an overcommitted schedule

D. Did not study enough during the semester

4. Did the exam relate to what you have been learning in class?

Yes 57 (93%) Somewhat 3 (5%) No 1 (2%)

5. Do you feel confident that you will attain college level writing by semesters end?

Yes 31 (86%) Somewhat 5 (14%) No 0 (0%)


Of the 135 students who responded to the mid-term questionnaire, I noted 27 responses to question number 3, A through D. Nine students checked A, 1 chose B, 12 circled C, and 7 marked off D. What is most telling here is that most responses revealed that our students have overcommitted schedules. Nine students who checked A admitted that they did not have time to study, while 7 did not study enough during the semester. Only 1 student did not know the exam was scheduled.

As can be seen, question number 1 notes that only 25% were Somewhat satisfied with their mid-term scores, while 35% were not pleased at all, which indicates a healthy determination to improve.

A positive response to question number 2 revealed that 95% of professors accurately explained what to expect on the exam before the exam date. In question number four, 93% revealed that the exam related to what was being learned in class, while question number 5 noted that 86% felt confident of attaining college level writing by semesters end.

Summary

The evaluation of Student Learning Outcomes for ENG 096 during Fall 2010 semester proved to be a positive, enlightening experience. Both full and part-time faculty members seemed to understand and even appreciate the importance of conducting this study. Many were eager to participate. Results indicated several strengths in terms of the course structure and the effective approach taken by faculty to fulfill the course requirements.

The survey also points to the need for further support for ENG 096 students struggling with thesis development and issues with mechanics and sentence structure. Although the college offers tutorial resources in The Learning Center, at this level, students are in dire need of a separate grammar course. When one considers that the Humanities Division offers three remedial level courses (ENG 085, ENG 096, and RDG 096), it seems plausible to offer students a grammar course. Such a course would also offer support, if needed, for the college level composition courses.

ENG 096 7

ENG 101 SLOAT FALL 2010 REPORT

Submitted by: Prof. Richard Bogart

Introduction

Two instruments were used to measure the achievement of outcomes in ENG 101 during the Fall 2010 semester. These were based upon the Course Goals developed in the revised ENG 101 outline, and the approach developed in the plan which was submitted and approved for use. The purpose of these instruments was two-fold. The first, a rubric, was designed to randomly assess the achievement of essay development skills by students across the course. This was based upon the first of the Course Goals listed in the current outline, write a composition. This rubric was used twice during the semester to measure any change which took place in this outcome. The second was a questionnaire, filled out by the students in the 101 sections, regarding their readiness for the mid-term exam. This was in order to measure, to some degree, the second of the Course Goals, recognize and use various modes of writing in order to develop expository, argumentative, and creative compositions.

Faculty and student support were very strong for this undertaking and the results were logical and worthy of consideration.

Methodology

As stated above, two instruments were used. The first was intended for a limited sample. In order to randomize the sample, and avoid instructors weighing the results by selecting subject, the anonymous nature of the study was made clear. Instructors were given a rubric, approved by the SLOAT chair, that was designed to evaluate each of the eight Measurable Performance Outcomes (MPOs) listed under the first Course Goal, write a composition. Instructors were given sufficient copies of the forms and were asked to take the most recent essay which their students had submitted and to evaluate each of the first five essays, selected alphabetically to maintain the random nature of the study, for the eight elements. They rated each on a three point scale labeled, yes, somewhat, or no.

The first distribution and collection of questionnaires took place during October. The forms were returned to me and, since they were anonymous, work studies in the Division totaled the results by category.

This activity was repeated in December, in order to measure any changes in the outcomes.

Twenty-four sections participated in the October rubric. Fifteen sections participated in the second, which was conducted during December. While the decrease in participation seems disappointing, December may have been a late point at which to conduct the survey. Some instructors may well have returned the short essays they had assigned at that point and moved onto looking strictly at research papers and preparing for the Divisional final.

The second instrument, the questionnaire regarding preparedness for the mid-term exam was distributed at the end of the week following the Divisional exam. A total of 546 students responded to the survey, therefore, representing a very significant sampling of the student enrollment in the course.

As with the essay rubric, the survey was administered and evaluated in a manner that was completely anonymous. Since there was no identifying information on the forms, work studies, again were used to total the results for each category.

The results, and original instruments, were returned to me, and the original instruments are currently stored in my office.

RESULTS

The results from the rubric distributed to ENG 101 faculty and the questionnaire distributed to ENG 101 students following the mid-term exam drew a clear picture of abilities and perceptions.

The questions on the rubric, which was distributed twice, were:

1) The paper includes an appropriate subject based upon the given writing assignment.

2) A specific topic was developed within the assigned subject for the paper.

3) An introduction provides background regarding the papers thesis.

4) A relevant thesis statement is incorporated into the introduction.

5) Body paragraphs were well-structured.

6) Body paragraphs were related to each other in a logical structure that supported the thesis.

7) The paper contained an appropriate concluding paragraph.

8) The paper demonstrates an acceptable level of mechanics and language usage.

For each question, the faculty members were asked to use the first five papers alphabetically, to insure randomness, and to respond yes, somewhat, or no for each category. The categories were directly drawn from the Student Learning Outcomes on the ENG 101 course outline.

For the first administration of the rubric, distributed during October of 2010, the following results were received:


Yes 107 (91%) Somewhat 11 (9%)No 0


Yes 93 (74%) Somewhat 31 (25%)No 1 (1%)


Yes 65 (50%)Somewhat 60 (46%) No 5 (4%)


Yes 74 (56%)Somewhat 48 (36%) No 11 (8%)


Yes 54 (48%)Somewhat 50 (44%)No 9 (8%)


Yes 62 (53%)Somewhat 53 (45%)No 3 (2%)


Yes 68 (56%)Somewhat 42 (35%)No 11 (9%)


Yes 51 (41%)Somewhat 60 (49%)No 12 (10%)

For the first question in each of the two submissions of the rubric, the yes responses were overwhelming, exceeding 90% in both cases. For the remainder of the questions, responses were strong, but with the least positive responses being for questions 5 and 8, regarding paragraph structure and an acceptable level of mechanics and language usage, respectively.


As can be clearly seen from the chart, there were very few no answers given. This may indicate a relatively strong level of preparation for the course, either in earlier education, or, when applicable, in developmental programs. The weakest scores in paragraph structure and mechanics would, however, indicate a need to look to those areas and discuss ways in which support services could be provided, perhaps outside of the traditional classroom setting, for students still struggling in those areas.

In December 2010, ENG 101 instructors were once again given the same rubric to fill out for a subsequent assignment. It was anticipated that some patterns of improvement would be seen in the second study.

Participation was still healthy, if a little lower, for this attempt. December is busy month with classes preparing for the Divisional final exam, which may offer some explanation.

The results for the December 2010 rubric were as follows:


Yes 71 (95%) Somewhat 4 (5%)No 0


Yes 66 (85%)Somewhat 15 (20%)No 0


Yes 60 (80%)Somewhat 20 (15%)No 0


Yes 60 (80%)Somewhat 19 (14%)No 1 (1%)


Yes 52 (69%)Somewhat 19 (25%)No 4 (5%)


Yes 52 (69%)Somewhat 21 (28%)No 2 (3%)


Yes 50 (64%)Somewhat 20 (26%)No 8 (10%)


Yes 34 (45%)Somewhat 38 (51%)No 3 (4%)

The results for the December 2010 rubric can be charted as follows:

While the sample size for the December study is smaller, there is apparent improvement reflected both in the data and the charts above from the October to the December rubrics.

Regarding criteria 5, paragraph structure, the yes evaluations rose from 48% to 69%. In the area of mechanics and sentence structure, however (criteria 8), there was a more limited improvement with the yes score moving only from 41% to 45% and the somewhat score moving from 49% to 51%.

Again, the issue of supplemental support for ENG 101 students, relevant to mechanics and sentence structure, should be explored.

Another instrument that was used during the Fall 2010 semester was a questionnaire which was to be distributed to ENG 101 students following the mid-term exam. Both instruments were conducted in a completely anonymous fashion. Doing so with this instrument, in particular, is vital to encouraging open participation by both students and faculty members.

That open participation appears to be the case in that students were often very open to identifying their own failure to study sufficiently as a factor in their mid-term exam performance.

Students were asked four questions dealing with their performance and preparedness for the exam. The results of the questionnaire are as follows:

1) Are you pleased with your grade on the exam?

Yes 331 (62%)Somewhat 114 (22%)No 85 (16%)

2) Did you prepare adequately for the exam?

Yes 321 (59%)Somewhat 195 (36%)No 27 (5%)

If not, why not? (Circle all that apply)

a) Did not have time to study

b) Did not know the exam was scheduled

c) Have an overcommitted schedule

d) Did not study enough during the semester

3) Did the exam relate to what you have been learning in class?

Yes 426 (78%)Somewhat 85 (16%)No 35 (6%)

4) Did your instructor accurately tell you what to expect on the exam prior to the exam date?

Yes 449 (83%)Somewhat 72 (13%)No 19 (4%)

While students did respond to the choices from a to d under question 2, practicality did not permit me to fully analyze the results, although I have maintained the questionnaire themselves. As you can see, approximately 545 students responded individually to the questionnaire.

My review of the questionnaires, however, indicates that the clear majority of answers under the number 2 choices fell into the categories of the last two choices, either being over-committed or not having studied enough during the semester.

Interestingly, a number of students responded to the a to d choice in spite of the fact that they had responded to the initial question with a yes. This conditional approach to conducting a survey question may not have communicated well with a number of students.

As can be seen, 84% of the respondents indicated that they were at least somewhat satisfied with their mid-term exam score while only 62% were satisfied enough to choose yes. Assuming that those who fell into either category had probably passed the exam, the number of students in the somewhat category may very well indicate a healthy determination to do better in the upcoming final.

As can be seen, a very minimal 4% indicated that their instructor had not accurately told them what to expect on the exam prior to the exam date. This very low percentage would have to be viewed as outliers and an indication that students do feel informed of the nature of the exam. A strong 83% gave that category a solid yes.

Another positive sign is the fact that 78% strongly indicated that the exam was related to course work with a Yes, while only 6% indicated that it was not.

SUMMARY

The evaluation of Student Learning Outcomes for ENG 101 during the Fall 2010 semester was certainly a positive experience. Faculty members, both full and part time, appeared to understand the important nature of the study and were willing to cooperate.

The results were predictable and indicated several strengths regarding the structure of the course and the appropriate approach taken by the faculty to the fulfillment of the course requirements.

The survey also indicated the need for greater support for many students who are struggling with issues of mechanics and sentence structure. Since this is not the case with more than 40% of the students, the results would not indicate the need for greater emphasis on these factors in the course. At this level, such work would probably be best approached in a prescriptive manner rather than a class-wide manner.

The College, of course, already has several opportunities available for this type of directed work including the student access of the Learning Center.

The possibility of more being needed, however, should be considered. One possibility might be a separate grammar course for students on the Freshman Composition level, similar to the course that has been offered at Rutgers-Newark in the past.

ENG 101 8

ESSEX COUNTY COLLEGE

STUDENT LEARNING OUTCOMES (SLO) ASSESSMENT REPORTMTH 092 Elementary AlgebraFALL 2010

PREPARED BY:Dr. Eman Y. Aboelnaga Pr. Barbara SatterwhiteDr. Alvin Williams

INTRODUCTION

According to Linda Suskie in her book Assessing Student Learning: a common sense guide, 2nd edition, assessment is the ongoing process of: (1) establishing clear, measurable expected student learning outcomes; (2) ensuring that students have sufficient opportunities to achieve those outcomes; (3) systematically gathering, analyzing, and interpreting evidence to determine how well students learning matches our expectations; and (4) using the results to understand and improve student learning. Assessment can be achieved in a variety of diverse processes.

This report focuses mainly on student learning outcomes assessment of the course MTH 092, an Elementary Algebra course offered by the Mathematics and Physics Department at ECC. In addition, social factors governing students perceptions of their performance were also investigated. The types of assessment used, as well as the methodology and administration of the assessment tools used, are all described. The results and findings are presented and explained. Finally, in order to close the loop, feedback based on the findings is suggested in an effort to initiate positive change and improve student learning.

1.1 PURPOSE

MTH 092, Elementary Algebra, is one of the most important courses at ECC. It is the second level in a remedial math sequence that is a prerequisite for 100-level (i.e., college-level) mathematics courses. Furthermore, it is a prerequisite for many other majors and courses available at ECC. These include Nursing, Biology, Chemistry, Engineering, Computer Science, Radiography, in addition to many social science majors. Therefore, the importance of assessing student learning outcomes and correlating them to measurable course performance objectives cannot be underestimated. As co-course coordinators of MTH 092, we are aware that there are many factors that contribute to student success, i.e., student learning. As members of SLOAT in Fall 2010, we endeavored, by choosing three forms of assessment, to investigate some of these factors. These forms of assessment will be discussed in detail in the following sections. Our goal is to use our findings to provide insight into some of the factors that affect student success thereby suggesting possible improvements for future semesters. Furthermore, we hope that by sharing our results and suggestions, we will jumpstart a discourse that will include the exchange of ideas, initiation of other assessment tasks, and is overall focused on the improvement of our MTH 092 course.

METHODOLOGY2.1 POPULATION

Based on our initial assessment plan, we decided to assess ten sections of MTH 092 with an approximate average of 30 students in each section. This gave us a sample size of approximately 300 students. We hoped to ensure at least a 5% margin error (278 out of 1000 corresponds to a 5% margin error) (Suskie). We attempted to sample a variety of sections. However, based on the response we received from professors willing to participate, the breakdown was as follows:

Professor

Sections

Campus

Daytime/

Evening

Full-time/ Adjunct

Eman Aboelnaga

006 and 028

Main Campus

Daytime

Full-time

Barbara Satterwhite

009 and 012

Main Campus

Daytime

Full-time

Susan Gaulden

011 and 019

Main Campus

Daytime

Full-time

Nasser Moheb

003 and 014

Main Campus

Daytime

Full-time

Shohreh Andresky

CW2

West Essex

Daytime

Full-time

Gordon Nanton

3WC

West Essex

Evening

Adjunct

As is evident from the table, 80% of the sections were day time courses taught at the Main Campus by full-time professors.

2.2 ADMINISTRATION

As co-course coordinators of MTH 092, we chose to assess the following student learning outcomes (SLOs):

SLO #1: Course Goal 1: Demonstrate knowledge of the fundamental concepts and theories from algebra and geometry.

SLO #2: Course Goal 2: Utilize various problem-solving and critical-thinking techniques to set up and solve real-world applications.

These correspond to the following measurable course performance objectives (MPOs):

1. Demonstrate knowledge of the fundamental concepts and theories from algebra and geometry.

1.1 simplify and evaluate variable expressions;

1.2 translate verbal expressions into variable expressions;

1.3 perform basic operations on polynomial, rational, and exponential expressions;

1.4 factor polynomial expressions;

1.5 solve linear, literal, and factorable quadratic equations;

1.6 graph a line in the Rectangular Coordinate System;

1.7 identify and find the slope and intercepts of a line; and

1.8 find the equation of a line based on given geometric properties

2. Utilize problem-solving and critical-thinking techniques to set up and solve real-world applications.

2.1 apply algebraic methods to solve varied real-world applications (such as integer problems, uniform motion problems, and perimeter and area problems) that can be modeled by a linear equation or a quadratic equation

We chose three forms of assessment. The first was an anonymous survey designed to investigate students attitudes towards math, their classmates, and their professor, as well as their perception of their progress and standing in the class. This is an indirect, summative, and subjective type of assessment based on the input of the students. The second form of assessment was an indirect, process, and objective type of assessment designed to investigate the correlation, if any, between the number of absences and the final grade and between use/completion of online homework and the final grade. Our last form of assessment was direct, summative, and objective, and was designed to measure the student learning outcomes mentioned earlier.

Regarding the first form of assessment: the attitudinal survey, we developed several drafts before finally choosing one that best expressed the issues we were investigating. We initially intended to have it ready to administer by late October after Midterms or early November. We then sent it to Dr. Alvin Williams who then modified the form of questioning in order to make it easier to calibrate. He then provided us with ScanTron-like answer sheets that corresponded to our questionnaire. The next step was to deliver the questionnaires to the participating instructors along with detailed instructions regarding their administration. This process took more time than anticipated so that the questionnaire was actually administered in late November. Students were asked to keep them anonymous and were given about 15 minutes of class time to complete them. We then assembled all the questionnaire answer sheets and delivered them to Dr. Williams. He calibrated the results and provided us with a breakdown of the percentages of students answers for each question. We then analyzed the results and formed conclusions and suggestions for future change. These will be discussed further in the Results section.

In order to gather information for our second form of assessment, we prepared an Excel spreadsheet template requesting information from each participating instructor as follows:

Instructor: Eman Aboelnaga

SLOAT Coordinators:

Section: 006

Eman Aboelnaga

Total # of Students: 25

Barbara Satterwhite

Student

ID#

Absences

Online HW

Final Grade

Agunanna, Princess O.

9000-

0

yes

B

Bryant Jr., Aniko

9000-

31

no

F

Chung, Claudia R.

9000-

8

yes

D

Coley, Tyler C.

9000-

7

no

D

Desilhomme, Jean H

9000-

1

yes

B

This information was sent to Dr. Williams, where he proceeded to statistically analyze it. He sent the results back to us and we formed conclusions and suggestions for future change.

Finally, since our third form of assessment was designed to measure student learning outcomes, we decided to blueprint the MTH 092 cumulative final exam to all of the MPOs from the entire set of course goals stated earlier. However, for the purpose of this study, the original plan was to increase the number of multiple choice questions so that every MPO corresponding to SLOs 1 and 2 were addressed within the multiple choice section of the final exam. This means that every specific MPO under Course goals 1 and 2 (CG 1 and CG 2) was mapped to a multiple choice question. This allows us to use ScanTron-like sheets in order to collect the data. When preparing the final exam, we found that it was going to be confusing to make application problems (i.e., word problems) multiple choice when neither the students nor the instructors were used to such a situation. We decided to map the MPOs from the first goal to ten multiple choice questions (a reasonable amount of multiple choice questions for a MTH 092 final exam) and hold off on assessing the second course goal. On the day of the final exam, participating professors were provided with a ScanTron-like answer sheet for each of their students. They were requested to have their students answer the multiple choice questions on the actual exam as well as on the ScanTron-like answer sheets. Repeated instructions were given to the students to bubble in their answers carefully and completely and to include the version of the exam that they completed. Despite that, some submitted answer sheets were still found to be incorrectly filled out. After final exams were completed, participating professors were asked to submit their ScanTron-like answer sheets to us. We delivered them to Dr. Williams who calibrated the results, providing us with a spreadsheet with the percentages of students who had answered each question correctly, thereby mastering that corresponding MPO. Details and results are discussed in the results section of this report.

2.2.1 INSTRUMENTATION2.2.1.1 ATTITUDINAL SURVEY

An anonymous questionnaire was administered to ten sections of MTH 092 within the last week of November and the first week of December. The questions as well as the calibrated results performed by Dr. Alvin Williams are included in Appendix A.

2.2.1.2 RUBRICS

In order to gather information on attendance and use of online homework, we formed an Excel spreadsheet template that professors involved in the study could use to input the students information. A sample was shown in section 2.2 of this report. Regarding the grading rubric for the final exam, the ten questions assessed were all multiple choice. The following table illustrates the question along with the MPO it is being mapped to.

MPO # - Topic/Skill

Sample Final Exam question(s) used to determine MPO acquisition

(1.1) simplify and evaluate variable expressions

Evaluate the variable expression when b = -2 and c = -1:

(1.2) translate verbal expressions into variable expressions

Translate the following into a variable expression, then simplify: a number added to the difference between twice the number and twenty

(1.3) perform basic operations on polynomial, rational, and exponential expressions

Simplify:

Simplify:

Simplify:

(1.4) factor polynomial expressions

Factor bz + 3b 2z 6

(1.5) solve linear, literal, and factorable quadratic equations

Solve:

(1.6) graph a line in the Rectangular Coordinate System

Graph: 2x 3y = -6

(1.7) identify and find the slope and intercepts of a line

Find the slope and y-intercept of the line

(1.8) find the equation of a line based on given geometric properties

Find the equation of the line that contains the point (4, -8) and has slope -3.

Each question assessed an MPO of the course and a correct answer was taken as the student having mastered the objective. If the student answered incorrectly, then that MPO was considered not mastered. The results by exam version as well as mean results are referenced in the Results section. The multiple-choice portion of the MTH 092 final exam Fall 2010, version 1 used in this assessment can be found in Appendix B.

RESULTS3.1 FINDINGS3.1.1 ATTITUDINAL SURVEY

We developed the SLOAT Student Questionnaire for MTH 092 Elementary Algebra to determine how influential social factors are over MTH 092 students' academic performance or if social factors influence their performance at all. We considered two possible social factors which could influence students: the instructor of the class and the other students in the class. Thus we wanted to determine whether the professor or classmates had the more dominant influence on a student. Furthermore, we looked at whether social factors influence which section of MTH 092 a student registers for, and if so, which is the more powerful social factor. Additionally, we asked questions investigating students' attitudes towards the discipline of mathematics, their confidence level in math, and their knowledge of studying math. Finally, we wanted to see how a student's attitude towards their instructor and fellow classmates does (or does not) affect their performance in MTH 092.

Questions 1, 7, and 12 were designed to investigate the importance of social factors in MTH 092 enrollment, and the results indicated that social factors are not very influential in students' decision to register for a particular section of MTH 092. Only 5% of students registered for their section of MTH 092 because they wanted to take the course with a classmate enrolled in that section, while 29% of students registered because they wanted to take a course with the Professor who instructs their section. Yet question 1's results show that 60% of students chose to take MTH 092 to fulfill a math requirement, and another 30% were repeating the course. The questionnaire responses indicate that students are more likely motivated by the need to fulfill graduation requirements than by the desire to take a certain instructor's course or to take MTH 092 with a friend. However students are more likely to register for a section because they like the instructor instead of registering because they like another student in that section.

Questions 2 and 5 show that many students have negative feelings about the discipline of mathematics and low confidence in their math skills before coming to ECC. It was found that almost half (46%) of students disliked math or were anxious about math and roughly the same percentage of students were not very confident or not confident at all in their ability to study math. MTH 092 instructors should be mindful of the prevalence of negative feelings towards math among new students, so that they can better help them to build their confidence level in math. However, questions 3 and 6 reveal that 52% of students feel that they now know how to study math, and approximately 75% of students are either somewhat confident in math or very confident in math at present. The results convey that students' confidence level in math increases dramatically at ECC, as well as their ability to study mathematics.

In examining what might cause the change in students ability to study math, question 4 asked where students would go for help if they did not know how to study math. The student responses show that the majority (59%) of students would ask their professor for help. Thus, if we can attribute the rise in confidence level of the students to any social factor, it would be the instructor since they are the ones whom students ask for help from the most.

In analyzing students' attitudes towards their MTH 092 instructor and towards the other students, we found that the overwhelming majority (76%) of students surveyed consider it important that they like their Professor. Comparatively, approximately 16% of students consider it important that they like their classmates. Therefore, the students surveyed indicated that they care significantly more about who is instructing their class, than who is taking the class with them. The importance of one's professor over one's classmates was reflected in the responses to questions 10 and 15 as well. Almost half of the students surveyed indicated that their like or dislike of their professor affects their performance in the course. The vast majority (73%) of students did not agree that their like or dislike of their classmates affects their performance in the course. The responses to questions 10 and 15 emphasize the fact that many students feel that it is important to like their professor, and that their attitude towards their professor will directly affect their performance. Few students consider their like or dislike of their classmates important, and even fewer agree that their like or dislike of their classmates has an effect on their academic performance. Interestingly, an overwhelming majority of students (86%) like their professor for MTH 092, while a lesser but still strong majority (63%) like their classmates, and 78% of students were passing MTH 092 with a grade of C or better.

Thus, in analyzing these responses, one can see the importance of the professor, not just pedagogically, but as the most important social factor influencing a student's academic performance. Consequently, MTH 092 instructors may be able to use their prominent social position in the classroom to help improve their students' academic performance. Professors are in a position to exercise their social influence upon their students' in a positive way, so as to build a rapport with their students which not only makes the students more comfortable with the professor, but also makes the professor more comfortable with the students. This attitudinal survey should serve as another means of learning more about our students and their mindset. If we can become more familiar with our students and maintain our professionalism at the same time, we can then hope to reach them in ways we may have never considered or have underutilized up until now.

3.1.2 ATTENDANCE AND HOMEWORK

The belief that lack of attendance and homework completion adversely affects a students grade tends to be widely held by many faculty members. As course coordinators and faculty members, we also hold this belief; particularly in a mathematics course where topics, for the most part, are sequential. In other words, the following class is dependent on mastery of the previous classs content. Furthermore, a thorough understanding of mathematics includes a conceptual understanding of the topics, the ability to apply this conceptual understanding to a problem, and the necessary skills to solve it. Homework is a tool that is used to develop and harness these attributes.

Therefore, as part of our student learning outcomes assessment plan, we decided to assess the correlation, if any, between attendance, homework, and final grade. The MTH 092 sample contained the following data: Student Name, ID#, Number of Absences, Use of Online Homework (yes/no), and Final Grade. This sample included 210 students from nine sections that completed the course for the Fall 2010 semester. Two forms of analysis were conducted.

First, a regression analysis was performed that looked at the group of investigated variables specified above. In this case, the MTH 092 final grade was considered to be a dependent variable while the number of absences and use of online homework were each considered independent variables. Regression analysis reflected how the typical value of the dependent variable (the MTH 092 final grade) changed when any one of the independent variables (absences or online homework use) were varied, while the other independent variables were fixed. It was found that the only variable that affected the MTH 092 grade in a statistically significant way was the number of absences. The independent variable, online homework use, did not affect the variability in MTH 092 grades in a statistically significant way for this particular sample.

Second, the variables were investigated further using an Independent Samples T-Test to discover which variables were significantly different statistically between the group of

Sociology Program Assessment -...

Documents

Transcript of Sociology Program Assessment -...