ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response...

27
ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang

Transcript of ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response...

Page 1: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

ELECTRICAL AND COMPUTER ENGINEERING (ECE)

Qin Wang

Page 2: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

This report provides evidence that students are achieving end-of-program learning goals and that graduates are attaining achievement outcomes established by the program.

Name of the program: Electrical and Computer Engineering

Year (e.g., AY17-18) of assessment report: AY17-18

Date Submitted: June 1, 2018

Contact: Qin Wang

Email: [email protected]

The Statement of Program Learning Goals and Curricular Matrix are available and updated at: http://www.nyit.edu/planning/academic_assessment_plans_reports.

I. Annual Program Learning Assessment

1. CLOSING THE LOOP: Many programs proposed improvement actions based AY 16-17 assessment results. Please report where the program is at implementing the improvement plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years

Year of Assessment

Results

Brief Name of Program Learning Goal (e.g., Writing)

Improvements Implemented Based on

Assessment Results

Impact of Improvements

(report reassessment

results if available)

AY13-14 PO: An ability to apply programming language concepts such as data models, control structures, language translation and testing and debugging in the development of software systems.

There were no changes made There were no changes made

AY14-15 PO: An ability to apply programming language concepts such as data models, control structures, language translation, testing and debugging in the development of software systems.

There were no changes made There were no changes made

AY15-16 PO: An ability to apply knowledge of mathematics, science, and engineering.

There were no changes made There were no changes made

AY16-17 PO: An ability to apply knowledge of mathematics, science, and engineering.

There were no changes made There were no changes made

Page 3: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

2. GOALS

List program learning goals that have been assessed in AY17-18. Course evaluated: EENG 270 Introduction to Electronic Circuits - Circuit design is an

important component of a lot of courses in the Electrical and Computer Engineering (ECE) department. Thus, sound circuit design skills provide a key foundation for succeeding in the ECE program.

Program learning outcome for EENG 270: “An ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability.”

Upon graduation, students will be able to:

a) Identify and summarize the I-V characteristics, operations, and design of circuits

b) Design a component or process using existing knowledge

c) Analyze electronic circuits

d) Draw circuits diagram within realistic constraints and specific goals.

e) Draw I-V lines of different circuit components, such as a diode, transistor

3. METHOD

Describe the method of assessment and attach measurement instruments (e.g., rubric, exam items, scoring guide for a particular task, supervisor evaluation form, and standardized assessment tool).

Data Source: The data for analyzing the above program outcome will be collected in the form of two separate tests and one short survey given to ECE sophomore students taking the “EENG 270 -Introduction to Electronic Circuits” course. The list of participants included in this study is mentioned in Table 1.

Class #of students

Sophomore Cohort 1 30

Sophomore Cohort 2 29

Sophomore Cohort 3 30

Table 1: List of participants

Measuring Instrument (Evidence Source): Two tests and one survey were conducted to see how much the students’ ability will be improved by taking this course and whether it is true that “group cooperation can help students have better performance.” The answer sheets are graded on a 1 to 5 scale, where 1 corresponds to “worst” and 5 to “best.”

Page 4: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Two tests were used to see the performance gap between students to design circuits individually or by the group. The samples were from the sophomores who are taking the course “EENG 270: Introduction to Electronic Circuits”. The first test was designed to provide individual information regarding the knowledge of students in the topics of designing circuit. Each student completed the first test independently in 40 minutes. The second test was designed to show the students’ group ability to design a circuit to meet desired characteristics within realistic constraints. During this test, the students were divided into 9-12 groups for each cohort. Each group completed the second test in 40 minutes. The group performance was analyzed and compared with their individual performance.

A survey was designed to assess students’ knowledge about circuit design before and after taking this course. The samples were from the freshmen who haven’t taken this course, sophomores who are taking his course, and juniors who have taken this course.

4. ANALYSIS Report assessment results per learning criteria (e.g., per row of rubric, subset of test items, components of a learning task).

4.1. Analysis of the test result

To answer the question that whether it is better to assign circuit design projects to grouped students or individuals. Two tests were conducted for sophomore students who are taking the course EENG 270.

Fig. 1 shows the performance gap between the individual circuit design scores and the grouped circuit design scores. Each test has two questions, each of which was graded on a 1 to 5 scale. In most cases, the scores from the grouped students are much better than those from individual test. The performance of the 34th group has been enhanced from 1.5 to 9.

Figure 1

4 4.54.3 45

8 8.58.59.3

7.5

4 4 4.5

9.5

4.5

8.5 98 8

44.7

7.3

2

5.56.5

7.5

2.3

109

67.3

64.7

1.5

10

6.7

4

98

9 98 8

4

67

10

76

4

7

109

87 7

4

9 9

4

89

10

8

1010

8 8

6

109 9

10

8

0

2

4

6

8

10

12

G1 G3 G5 G7 G9 G11 G13 G15 G17 G19 G21 G23 G25 G27 G29 G31 G33 G35 G37

Average individual scores Grouped scores

Page 5: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Fig. 2 shows that 65% of sophomores have improved the test scores due to grouping when designing circuits. 24% of students have worse performance even when they are grouped together to design circuits. One of the reasons is that the circuit design problem in grouped test is more difficult than the circuit design problem of individual exercise. Even though, most students have higher scores, with 11% unchanged. Thus, doing circuit design project in the way of grouping is better to make use of each student’s strengths.

Figure 2

4.2. Analysis of the survey result

To evaluate the course EENG 270 and verify that the students can grasp circuit design ability by takin this course, the performance before and after taking the course EENG270 was compared by the way of conducting survey. The data were from freshmen, sophomores, and juniors in ECE department.

1) I know the simple concept of semiconductor.

Fig. 3 shows the result that 61% of juniors agree with this statement and the highest percentage 74 is of sophomores. The lowest percentage 34 (31% for AGREEABLE and 3% for VERY AGREEABLE) belongs to the freshmen. Sophomores’ response who were taking the course at the time of the survey is also quite high. The freshmen students having not taken this course did not know semiconductor more (14% and 24%). One surprising result that came out of this question was that 38% sophomores and 19% juniors did not know the concept of semiconductor.

No change11%

Improve65%

Decline24%

Page 6: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 3

2) I know LED is a kind of diode.

Fig. 4 shows the result for this statement. It shows clearly that the awareness about LED, which is a very common stuff being used in our life. The highest (49% and 26%) was from the sophomores who were taking this course as the time of the survey and the lowest was from freshmen.

Figure 4

3) I know the simple concept of transistor.

The result to this statement shown in Fig. 5 evaluate the students’ awareness about transistor. The sophomores know the statement more (48% and 24%), with the juniors following it (24% and 39%). The highest for VERY AGREEABLE was from juniors who took the source one year ago. The good thing is that some freshmen knew some knowledge of the basic concept of transistor, which is good for their future study of ECE courses.

14%

24%29% 31%

3%

12%

2%

12%

43%

31%

11%8%

19%24%

37%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - VeryAgreeable

Freshman Sophomore Junior

11% 13%

29%38%

10%11%2%

12%

49%

26%

5%11%

23%18%

44%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - VeryAgreeable

Freshman Sophomore Junior

Page 7: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 5

4) I know the difference between “DC analysis” and “AC analysis”.

Fig. 6 shows the data collected for this survey statement related to two kinds of analysis methods used in electronic circuits. The graph clearly shows that the awareness among the sophomores who were taking the course at the time of the survey is the highest (28% and 52%) and the freshmen almost have no awareness regarding this issue (15% and 0).

Figure 6

5) I know the relationship between resistance, current, and voltage.

Fig. 7 shows the answer to this question and we can clearly see that the awareness among the sophomores who were taking the course at the time of the survey is the highest (56% for VERY AGREEABLE and 26% for AGREEABLE) again and the freshmen has the lowest awareness regarding this current-voltage relationship issue. And because it was discussed in the class, the percentage of sophomores who are aware of this statement is very high, followed by juniors.

15%22%

39%

19%

4%9% 5%

15%

48%

24%

7% 5%

26% 24%

39%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - VeryAgreeable

Freshman Sophomore Junior

35%26% 24%

15%

0%9% 5% 6%

28%

52%

8% 7%

18%26%

42%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - VeryAgreeable

Freshman Sophomore Junior

Page 8: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 7

6) I know the Kirchhoff Current law (KCL): At any point in the circuit the total current enters, is exactly equal to the total current leaves the point.

Fig. 8 shows the result to this very important issue in analyzing and designing circuits. The result proves that the percentage of having no awareness to KCL belongs to freshmen, even they have learned this knowledge in high school. Most sophomores and juniors know KCL for sure. And because it was frequently used in the class, the percentage of sophomores who are aware of this statement is very high, followed by juniors.

Figure 8

7) I know a transistor circuit (MOSFEC or BJT) has three applications: inverter, NOR logical gate, and small-signal amplifier.

As every other result, the data to the above statement shown in Fig. 9 also shows that the freshmen are the least aware and sophomores are the most aware of all, with juniors following it.

13%19%

31%24%

14%9%

4% 6%

26%

56%

7% 7%13%

19%

55%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - VeryAgreeable

Freshman Sophomore Junior

31%

13%

33%

19%

4%10%

1%9%

29%

51%

8% 5%

18%

31%39%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - VeryAgreeable

Freshman Sophomore Junior

Page 9: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 9

8) I can distinguish between n-channel MOSFET and p-channel MOSFET via their circuit symbols.

The result to this statement in Fig. 10 shows that the percentage for VERY AGREEABLE becomes lower than the results in Figs. 6-8. It is a concept which is not as common as other concepts of this course. Thus, the freshmen have lowest awareness to the statement.

Figure 10

9) I can design a circuit completely according to the design objective.

As every other result, the result to the above question in Fig.11 also shows that the freshmen are the least aware and sophomores are the most aware of all.

35%

24% 25%

15%

1%7%

4%

24%

34%31%

8%

0%

31%36%

26%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - VeryAgreeable

Freshman Sophomore Junior

36%

22% 24%15%

3%4% 7%

17%

40%32%

8%0%

31% 34%27%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - VeryAgreeable

Freshman Sophomore Junior

Page 10: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 11

10) I can use many different types of circuit elements (pn junction, diodes, transistors, amplifiers) confidently as materials for circuit design.

It is no surprise again that 65% of sophomores agreed to the above question (Fig. 12) as they were the ones who were currently taking this course. 63% juniors also agreed to this.

Figure 12

11) It is easy for me to determine which state (turn on or turn off, saturation region or non-saturation region) a transistor is biased.

The result of this statement (Fig. 13) shows that 49% of sophomores agree as the need for learning about the topics covered in the source.

22% 19%

35%

21%

3%4%9%

23%

43%

22%

8% 7%

24%

36%

26%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - VeryAgreeable

Freshman Sophomore Junior

26% 26% 29%

18%

0%5%

9%

21%

44%

22%

7% 7%

27%

37%

23%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - Very Agreeable

Freshman Sophomore Junior

Page 11: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 13

12) I know how to design an amplifier circuit.

Because the sophomores were learning circuit design in the course, they have the highest percentage (48%) for AGREEABLE. The percentages for VERY AGREEABLE has become lower with freshmen 0%. Even through some freshmen know some knowledge about circuit design, resulting in AGREEABLE percentage 19%.

Figure 14

13) I can confidently get my hands on the fundamental theoretical knowledge to design circuits.

The data distribution in Fig. 15 is like those in Figs. 12-14. Sophomores and juniors know more about the fundamental theoretical knowledge to design electronic circuits.

19% 22%

35%

22%

1%0%9%

22%

49%

21%

5%10%

23%

42%

21%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - Very Agreeable

Freshman Sophomore Junior

26% 24%31%

19%

0%4% 7%

22%

48%

20%

5% 3%

23%

45%

24%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - Very Agreeable

Freshman Sophomore Junior

Page 12: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 15

14) I think the ability of circuit design is important for my additional study.

From Fig.16, sophomores and juniors both agree that circuit design ability is important for their additional study, and more than half of freshmen agree to this fact.

Figure 16

15) I think the ability of circuit design is important for my future work.

Comparing Fig. 17 to Fig. 16, more students agree that the circuit design ability is vital for their future work. It is the truth since they will use the design theory if they need to complete a component design in a company. The fundamental knowledge will always benefit their future work and study.

24% 26%

36%

14%

0%2%6%

26%

45%

21%

3% 5%

31%

40%

21%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - Very Agreeable

Freshman Sophomore Junior

14% 17%

28% 28%

14%

4% 6%

16%

43%

32%

7% 5%

18%

39%32%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - Very Agreeable

Freshman Sophomore Junior

Page 13: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 17

16) My cooperation ability can be enhanced by doing group project about circuit design.

More than 80 percent students agree or somewhat agree that grouped projects help them a lot, especially the cooperation ability. During designing a circuit, they communicate what they have learned and grasped in class.

Figure 18

17) My innovation ability can be enhanced by doing circuit design projects.

More than 80 percent students also agree or somewhat agree that grouped projects help improve their innovation ability. By solving the problems and exchanging the knowledge during circuit design, they enlarge what they have learned in textbook to deal with a practical issue.

15%7%

35%29%

14%4% 4%

20%

40%33%

5% 8%16%

36% 36%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - Very Agreeable

Freshman Sophomore Junior

14%8%

31%36%

11%5% 4%

20%

48%

24%

8%3%

18%

32%39%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - Very Agreeable

Freshman Sophomore Junior

Page 14: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 19

18) I observe and think about circuit applications in our life.

It is surprising to know that most students observe and think about circuit applications in life, as shown in Fig. 20. Based on this face, the relationship between the circuits in textbook with the practical issues in our life should be introduced more in future classes.

Figure 20

19) I want to learn how to design more complex circuits as an engineering student.

It is surprising to know that more than 80 percent students agree or somewhat agree to learn more complex circuits in class, from Fig. 21. It indicates that more circuits and complex circuits could be introduced in ECE courses, including EENG 270.

8%

17%

33%28%

14%

4% 4%

20%

46%

27%

8%5%

16%

31%

40%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - VeryAgreeable

Freshman Sophomore Junior

11%17%

43%

19%

10%6% 5%

20%

46%

23%

7% 3%

27% 31% 32%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - Very Agreeable

Freshman Sophomore Junior

Page 15: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 21

20) I think taking this course has enhanced my understanding of electronic circuits and circuit design.

Fig. 22 clearly shows that more than 70% sophomores and juniors agree or very agree to the statement that this course has enhanced their understanding of electronic circuits and circuit design. The percentage of freshmen who have not taken this course also somewhat agree to it for some reason.

Figure 22

5. INTERPRETATION

It provides an interpretation of student strengths and weaknesses for a given program learning outcome.

A. Main Findings from Test a) The degree of cooperation between students will influence the final grouped performance

10%14%

29%36%

11%5% 5%

27%

38%

26%

7%11%

29%32%

21%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - Very Agreeable

Freshman Sophomore Junior

11% 8%

36%29%

15%

4% 6%13%

46%

31%

7% 8%15%

50%

21%

1 - VeryDisagreeable

2 - SomewhatDisagreeable

3 - SomewhatAgreeable

4 - Agreeable 5 - Very Agreeable

Freshman Sophomore Junior

Page 16: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

b) Group cooperation can help students have better performance

B. Main Findings from Survey a) From Q1-Q6, freshmen have worse background knowledge than sophomores and juniors.

b) From Q7-Q13, students can design electronic circuits after taking this course; the students’ circuit design ability can be improved by taking this course

c) From Q14-Q20, taking this course helps students in their additional study or future work; their cooperation and innovation ability will be improved by doing grouped circuit design projects.

6. IMPROVEMENTS

If any weakness has been identified, provide a plan for improvement, including timeline, and personal responsibility for completion.

Positive Outcome:

a) All the five learning outcomes for EENG 270 are being met successfully without any problem.

b) This course is overall benefitting students in innovation and cooperation in designing circuits.

Scope for Improvement

Even though all the learning outcomes for this course are mostly met very well but there is always a room for improvement in every case. The overall efficiency and achievement of the learning outcomes will surely be improved with the suggested improvements below.

c) More time should be spent in circuit design training, especially grouped circuit design besides the lecture.

d) The circuit applications and the relationship with practical life should be introduce more.

e) In order to improve their carelessness or innovation about electronic circuit design, at least one electronic contest should be organized for the students in ECE apartment. Encourage the students to participate the contest hosted by other organizations or universities.

II. Brief Description of Faculty Engagement in the Current Annual Assessment Report:

After being asked by our campus dean, Prof. Keh Kwek the committee talked about the process we need to follow to conduct the assessment via emails. Prof. Sonali Chandel shared a work plan sample template as an option from last year’s assessment process with the team. The work plan for ECE PLO assessment was then shared with the ECE apartment dean Prof. Aaron Yan at Nanjing on 19 March 2017 requesting him to provide feedback. He thought that the overall work plan was satisfactory and did not make any specific comments/suggestions regarding it. On Mar. 21, 2018, the assessment committee members and the consultant had a meeting to talk about the global outcome plan. Each member confirms his or her selected assessed course and the

Page 17: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

responding learning outcomes. The work plan drafts and the methodology which were supposed to be used for data collection were also talked in the meeting. On April 4, 2018, the consultant holds a meeting to share her experience in using survey website (Survey Planet). The committee members also confirmed how to conduct the GLO survey. On April 11, 2018, the survey delivery plan was confirmed. A common report template was shared by and Prof. Dr. Abdul Razaque and Prof. Sonali Chandel.

III. Annual Program Achievement Goals:

Please provide examples of readily available data on program student achievement (e.g., first-year retention rates, six-year graduation rates, average time to degree completion, certification exam pass rate, student satisfaction survey results, employer satisfaction results, % pursuing an advanced degree, % of job placement, etc.)

Data for this section was not available at the time of writing this report.

Page 18: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Physics for ECE & CS

Alfonso Reina

Page 19: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

This report provides evidence that students are achieving end-of-program learning goals and that graduates are attaining achievement outcomes established by the program.

Name of the program: Physics for Electrical and Computer Engineering (ECE) and Computer Science (CS)

Year of assessment report: AY 2017-2018

Date Submitted: June 1, 2018

Contact: Alfonso Reina

The Statement of Program Learning Goals and Curricular Matrix are available at: http://www.nyit.edu/planning/academic_assessment_plans_reports.

I. Annual Program Learning Assessment:

1. GOALS CRITICAL/ANALYTICAL THINKING. Students make decisions and solve problems based on research, logic, and qualitative and quantitative analyses of appropriate and relevant data and information. Upon graduation students will be able to:

1. Identify and summarize the problem, issue, or question to be investigated. 2. Present existing knowledge, research, and/or views. 3. Design an inquiry process 4. Analyze research/evidence 5. Draw inferences and conclusions from analyses

2. METHOD Comments: the goal stated above was assessed in the context of the course of PHYS170: General Mechanics, which is taken by all ECE and CS students. The course level outcome relevant to the instrument chosen is: “Understanding and Applying Newton’s Laws of Motion”.

2.1. Instrument The Force Concept Inventory, a multiple-choice test of 30 questions, was selected for the study. The test is often used to assess learning outcomes of undergraduate physics courses in the USA. The test measures the understanding of the nature of forces and their role on the motion of objects. The distractors are common misconceptions identified by more than 1000 interviews of college and high school students [1]. Because our students are non-native speakers, three previously published versions of the tests were used: English [1], Simplified English [2] and Chinese [3]. Each student was assigned randomly one version. Sample questions of each versions are included in the appendix. A brief description is given below for each version:

• English: the test is written in college-level English, comparable to that used in common University Physics textbooks used in the USA. Diagrams are not included

Page 20: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

in all questions, and it is expected that students can draw inferences from the text to make a mental model of the situation/context of the prompt [1].

• Simplified English: more diagrams are included to help the student to imagine the situation (Figure A1). Less text is used in questions with redundant or irrelevant information (Figure A2). The context of some questions is simplified without changing the learning goal being tested (Figure A3) [2].

• Chinese: the test is a direct translation from English [3].

2.2 Participants The test was given to 92% of students enrolled in ECE and CS during the academic year 2017-1018. This allowed to compare results between majors and across all levels. Freshmen are currently taking the subject assessed, while sophomores and juniors took the class one and two years ago, respectively. The number and percentage (of the total population) of students assessed are listed in Table 1. TABLE 1: Number of participants in assessment process

Major Participant Students Enrolled Percentage (%) ECE 263 269 98% CS 300 345 87% Total (ECE & CS) 563 614 92%

Participants by major, level and version of test taken TABLE 2: showing Major, level and version of test taken

Major/Level Total Participants English Simplified English Chinese ECE Freshmen 88 29 29 30

ECE Sophomores 89 34 29 26 ECE Juniors 86 39 21 26

CS Freshmen 89 36 28 25 CS Sophomores 102 35 34 33

CS Juniors 109 37 34 38 2.3 Test statistics used In all comparisons a T-test and/or ANOVA were used to determine the significance of the means when comparing the groups above. Tests for normality were done by the graphical method with quantile-quantile plots (QQ-plots).

Page 21: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

3. ANALYSIS 3.1. Benchmark

The benchmark chosen to compare student performance was a multiyear study conducted at Georgia Tech with 5000 students enrolled in engineering, science and non-STEM programs who took physics courses with various teaching styles [4]. Based on this report, students perform within the 60 to 70% range after completing their physics course. We took this range to be the criteria for satisfactory performance. The results were compared three ways: among majors (ECE & CS), levels (freshmen, sophomores, and juniors) and among types of tests.

3.2 Results by majors and levels (Figure 1)

There was no significant difference between the performances of ECE and CS majors (p>0.78, t-tests among ECE and CS groups within each level), suggesting that the level of learning did not depend on which program students were enrolled in. There was a minor significant difference between juniors and the rest of the levels (p~0.05-0.10, t-test between Juniors and Sophomores or Freshmen for both ECE and CS majors) depicted in Figure 1.

Figure 1. Results by majors and levels. Benchmark for satisfactory performance is shown in red (60-70%). Black bar represents the 95% confidence interval of the mean for each group. ECE is shown in blue and CS in yellow. 3.3 Results by test version (Figure 2 and 3) Figure 2 shows the performance of students grouped by each version of the test. Each group (e.g. English, Simplified English, or Chinese) contains students from all majors and levels (see tables in the method section). Students perform better in the Chinese version, followed by the Simplified English version. The difference of performance between the Chinese and English versions is equivalent to 12 percent points, roughly four questions out of 30. The differences are significant, with a p-values below 0.05 as determined by t-test among each pair and ANOVA among the three groups. Figure 4 shows the comparison

Page 22: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

between each test version within each program and by levels. The trend identified in Figure 3 is consistent except CS Sophomores.

Figure 2. Results grouped by test version. The benchmark chosen for satisfactory performance is shown in red. Black bars represent the 95% confidence interval for each group.

Figure 3. Results grouped by test version, levels and majors. The difference between language performance is consistent within levels and programs.

3.4 Item analysis (Figure 4) The percentage of correct responses were calculated for each question in the test, regardless of the version. This was done to identify possible weaknesses and learning gaps. Questions were flagged if the percentage of correct responses was below 60% (based on the benchmark selected of 60-70% as the acceptable range). A total of four questions did not satisfy the criteria (Questions #3, 8, 11 and 23, Figure A4)

Page 23: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure 4. Percentage of correct answers for each question of the Force Concept Inventory. The percentage was calculated using the whole population of students without grouping by majors, levels or test version.

4. INTERPRETATION: 4.1 Majors and levels (Figure 1) It is believed that students understand the concept of force well enough independently of their major or level. Freshmen and sophomores perform overall above the benchmark, while juniors perform within the benchmark. It is possible that juniors fall short compared to the other levels because they took this physics course two years ago 4.2 Test version and language (Figures 2 and 3) The data collected shows that test language plays a role in student performance. In average, a student would miss four more questions than a student taking the same test in English. Furthermore, a simplified version of the test improves the odds of a better performance, but it is not equivalent to the results that can be obtained by using the native language version. The test language effect is observed regardless of majors and levels, as shown in Figure 3. The only exception is the case of CS Sophomores, where both the Simplified English and Chinese versions have comparable results. Even in this case the English tests remains with the lowest performance. These results underscore the influence of language during standardize testing of English Language Learners (ELLs.) and the necessary attention needed while designing test items in physics to assess learning during midterms and finals. 4.3 Identifying knowledge gaps (Figure 4)

Page 24: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Only four questions were identified to be problematic to the overall student population. These four questions represent 13% of the content covered, which confirms the notion that students have a good grasp of the concept of forces. The four questions are shown in the appendix. Common themes within these questions include: • The prediction of the trajectory taken by an object after (or during) the action of a force

(Qs #3, 11, 23, Figure A4) • Identifying the force(s) from knowing the trajectory taken by an object (Q #8, Figure

A4))

5. IMPROVEMENTS - PLANNED:

5.1 Test Language and design: It is important to be aware of possible language interferences when assessing and testing students. We currently provide translation in Chinese of challenging vocabulary in midterms and finals. Also, it is a requirement for every physics instructor to provide diagrams in all problems in each test to avoid false negatives during testing. As our results show, simplification of language and the incorporation of diagrams may help mitigate invalid assessment results. Although we believe that testing must be done in the language of instruction, it is important to raise awareness among faculty about the obstacles English Learners may face during science and physics classes. It is also important to be aware of test designs that may not be suitable for English Learners. More detailed analysis may be needed in the future to validate the differences seen among test takers in English, Simplified English and Chinese. For example, it is necessary to interview students to make a detailed description of their thought processes while answering the same question in English and Chinese. It would be good to identify the type of “language traps” that are common among our students to be able to provide more detailed guidelines for test designs. 5.2 Integration of knowledge and skills Although student performance is strong within the concept of force, we believe that student learning can be improved by emphasizing more the connection between the action of a force and the trajectories and properties of motion of the objects affected by such force. Furthermore, it may be necessary that students predict such connections in both ways (from force to trajectory and vice-versa). It is not surprising that such questions tend to be the most challenging to our students. These questions demand students to predict the behavior of common phenomena (rocket propulsion, ice-hockey, and the speed change of a falling orange) but at a detailed level that goes beyond the analysis done by an average person while performing these activities. Moreover, the prediction of the trajectory taken by a rocket in questions 8 and 11 is only possible by drawing on mathematical models that consider Newton’s laws of motion. Students do not only need to be familiar on how a force

Page 25: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

acts, but also need to be able to model such action mathematically to predict the shape of the path. This is a task that can only be accomplished by the integration of multiple skills and concepts. Therefore, we believe that future lessons could benefit from including more integration of concepts and skills.

II. Summary of Improvements Made in Response to Assessment Results in the past few years:

Not applicable. This is the first assessment report of physics in the Nanjing Campus.

III. Brief Description of Faculty Engagement in the Current Annual Assessment Report:

The results were processed during the beginning of May due to the narrow window of time given to perform the tasks of this year’s assessment. They will be shared with other physics instructors by the end of the academic year. The themes addressed in this assessment have been discussed multiple times during meetings with physics faculty while planning courses, midterms and finals. All instructors are aware of issues of language that we all must consider during the writing of tests.

IV. Annual Program Achievement Goals:

Not applicable to physics.

Please provide examples of readily available data on program student achievement (e.g., first-year retention rates, six-year graduation rates, average time to degree completion, certification exam pass rate, student satisfaction survey results, employer satisfaction results, % pursuing an advanced degree, % of job placement, etc.)

References

[1] Hestenes, David et al. Force Concept Inventory. The Physics Teacher. 30(3) 141. 1992

[2] S. Osborn Popp and J. Jackson, Can Assessment of Student Conceptions of Force be Enhanced Through Linguistic Simplification? American Educational Research Association 2009, San Diego, CA, 2009.

[3] Guo, Chenyue. Force Concept Inventory. Simplified Chinese Translation. Online: www.physport.org. American Association of Physics Teachers.

[4] M. Caballero, E. Greco, E. Murray, K. Bujak, M. Marr, R. Catrambone, M. Kohlmyer, and M. Schatz, Comparing large lecture mechanics curricula using the Force Concept Inventory: A five thousand student study. Am. J. Phys. 80 (7), 638 (2012).

Appendix

Page 26: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure A1. Sample question from the English, Simplified English and Chinese versions of the Force Concept Inventory. The Simplified English version utilizes diagrams and underlining of words

Figure A2. Sample question from the English, Simplified English and Chinese versions of the Force Concept Inventory showing reduction of text in the Simplified English version.

Page 27: ELECTRICAL AND COMPUTER ENGINEERING (ECE) Qin Wang · plan. TABLE 1: Improvements Made in Response to Assessment Results in the past years . Year of Assessment Results ... (ECE) department.

Figure A3. Sample question from the three versions of the Force Concept Inventory showing the simplification of the context implemented in the Simplified English version

Figure A4. Questions identified with performance below the benchmark chosen (60% of correct answers).